2024-04-24T23:12:03.641 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-24T23:12:03.641 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-workunit/suites/fsx}} --archive /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672189 --owner scheduled_teuthology@teuthology --name teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi --verbose -- /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672189/orig.config.yaml 2024-04-24T23:12:03.694 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-24T23:12:03.791 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672189 branch: squid description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-workunit/suites/fsx}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7672189' kernel: client: sha1: distro kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - slow metadata IO - SLOW_OPS - slow request sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 subvols: create: 2 subvol_options: --size 25000000000 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - mds.dir_split install: ceph: flavor: default sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=secure - wsync syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 8528 sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 sleep_before_teardown: 0 subset: 122/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi080.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMfkpGuLodaIMGxyO3ew6OGCQMhmGce6YeQgbc/L0GoJALnJd+ieQDvMBf1nP//vUIkWuhFkJTH9nim4SZwAsrA= smithi134.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJLfJ5mhzX16mpPe9YV6p4eHLv7USo0CBUIrFqOF1DlNHnlFM66BDtOrHNeJbtSJ7H06Zt4NupSbjNvrGsCCzRM= smithi148.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMZ+s72X4mo4KKu4DWptXk2E4sLii0Ouj8PIOMAf2YPxQssgQIYWOSPJx0vZoey7l7mg0a9DFv5El/rIb4fdkvg= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - workunit: clients: all: - suites/fsx.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-24_21:24:02 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-24T23:12:03.791 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa; will attempt to use it 2024-04-24T23:12:03.791 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa/tasks 2024-04-24T23:12:03.792 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-24T23:12:03.793 INFO:teuthology.task.internal:Checking packages... 2024-04-24T23:12:03.817 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'd2c45d977142e024a59b68a8f1dce8f0b9840551' 2024-04-24T23:12:03.817 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-24T23:12:03.817 INFO:teuthology.packaging:ref: None 2024-04-24T23:12:03.817 INFO:teuthology.packaging:tag: None 2024-04-24T23:12:03.817 INFO:teuthology.packaging:branch: squid 2024-04-24T23:12:03.817 INFO:teuthology.packaging:sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T23:12:03.818 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-04-24T23:12:04.051 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2445.gd2c45d97 2024-04-24T23:12:04.053 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-24T23:12:04.060 INFO:teuthology.task.internal:no buildpackages task found 2024-04-24T23:12:04.060 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-24T23:12:04.082 INFO:teuthology.task.internal:Saving configuration 2024-04-24T23:12:04.102 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-24T23:12:04.108 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-24T23:12:04.142 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi080.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672189', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-24 23:07:00.395184', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMfkpGuLodaIMGxyO3ew6OGCQMhmGce6YeQgbc/L0GoJALnJd+ieQDvMBf1nP//vUIkWuhFkJTH9nim4SZwAsrA='} 2024-04-24T23:12:04.171 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi134.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672189', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-24 23:07:00.391732', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJLfJ5mhzX16mpPe9YV6p4eHLv7USo0CBUIrFqOF1DlNHnlFM66BDtOrHNeJbtSJ7H06Zt4NupSbjNvrGsCCzRM='} 2024-04-24T23:12:04.201 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi148.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672189', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-24 23:07:00.393666', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMZ+s72X4mo4KKu4DWptXk2E4sLii0Ouj8PIOMAf2YPxQssgQIYWOSPJx0vZoey7l7mg0a9DFv5El/rIb4fdkvg='} 2024-04-24T23:12:04.201 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-24T23:12:04.207 INFO:teuthology.task.internal:roles: ubuntu@smithi080.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-04-24T23:12:04.207 INFO:teuthology.task.internal:roles: ubuntu@smithi134.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-04-24T23:12:04.208 INFO:teuthology.task.internal:roles: ubuntu@smithi148.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-04-24T23:12:04.208 INFO:teuthology.run_tasks:Running task console_log... 2024-04-24T23:12:04.337 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fb94e3b6160>, signals=[15]) 2024-04-24T23:12:04.337 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-24T23:12:04.344 INFO:teuthology.task.internal:Opening connections... 2024-04-24T23:12:04.344 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi080.front.sepia.ceph.com 2024-04-24T23:12:04.346 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T23:12:04.430 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi134.front.sepia.ceph.com 2024-04-24T23:12:04.431 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi134.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T23:12:04.514 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi148.front.sepia.ceph.com 2024-04-24T23:12:04.515 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T23:12:04.598 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-24T23:12:04.605 DEBUG:teuthology.orchestra.run.smithi080:> uname -m 2024-04-24T23:12:04.622 INFO:teuthology.orchestra.run.smithi080.stdout:x86_64 2024-04-24T23:12:04.622 DEBUG:teuthology.orchestra.run.smithi080:> cat /etc/os-release 2024-04-24T23:12:04.678 INFO:teuthology.orchestra.run.smithi080.stdout:NAME="CentOS Stream" 2024-04-24T23:12:04.678 INFO:teuthology.orchestra.run.smithi080.stdout:VERSION="9" 2024-04-24T23:12:04.678 INFO:teuthology.orchestra.run.smithi080.stdout:ID="centos" 2024-04-24T23:12:04.679 INFO:teuthology.orchestra.run.smithi080.stdout:ID_LIKE="rhel fedora" 2024-04-24T23:12:04.679 INFO:teuthology.orchestra.run.smithi080.stdout:VERSION_ID="9" 2024-04-24T23:12:04.679 INFO:teuthology.orchestra.run.smithi080.stdout:PLATFORM_ID="platform:el9" 2024-04-24T23:12:04.679 INFO:teuthology.orchestra.run.smithi080.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-24T23:12:04.679 INFO:teuthology.orchestra.run.smithi080.stdout:ANSI_COLOR="0;31" 2024-04-24T23:12:04.679 INFO:teuthology.orchestra.run.smithi080.stdout:LOGO="fedora-logo-icon" 2024-04-24T23:12:04.679 INFO:teuthology.orchestra.run.smithi080.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-24T23:12:04.679 INFO:teuthology.orchestra.run.smithi080.stdout:HOME_URL="https://centos.org/" 2024-04-24T23:12:04.679 INFO:teuthology.orchestra.run.smithi080.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-24T23:12:04.679 INFO:teuthology.orchestra.run.smithi080.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-24T23:12:04.679 INFO:teuthology.orchestra.run.smithi080.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-24T23:12:04.680 INFO:teuthology.lock.ops:Updating smithi080.front.sepia.ceph.com on lock server 2024-04-24T23:12:04.718 DEBUG:teuthology.orchestra.run.smithi134:> uname -m 2024-04-24T23:12:04.736 INFO:teuthology.orchestra.run.smithi134.stdout:x86_64 2024-04-24T23:12:04.736 DEBUG:teuthology.orchestra.run.smithi134:> cat /etc/os-release 2024-04-24T23:12:04.792 INFO:teuthology.orchestra.run.smithi134.stdout:NAME="CentOS Stream" 2024-04-24T23:12:04.793 INFO:teuthology.orchestra.run.smithi134.stdout:VERSION="9" 2024-04-24T23:12:04.793 INFO:teuthology.orchestra.run.smithi134.stdout:ID="centos" 2024-04-24T23:12:04.793 INFO:teuthology.orchestra.run.smithi134.stdout:ID_LIKE="rhel fedora" 2024-04-24T23:12:04.793 INFO:teuthology.orchestra.run.smithi134.stdout:VERSION_ID="9" 2024-04-24T23:12:04.793 INFO:teuthology.orchestra.run.smithi134.stdout:PLATFORM_ID="platform:el9" 2024-04-24T23:12:04.793 INFO:teuthology.orchestra.run.smithi134.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-24T23:12:04.793 INFO:teuthology.orchestra.run.smithi134.stdout:ANSI_COLOR="0;31" 2024-04-24T23:12:04.793 INFO:teuthology.orchestra.run.smithi134.stdout:LOGO="fedora-logo-icon" 2024-04-24T23:12:04.793 INFO:teuthology.orchestra.run.smithi134.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-24T23:12:04.793 INFO:teuthology.orchestra.run.smithi134.stdout:HOME_URL="https://centos.org/" 2024-04-24T23:12:04.793 INFO:teuthology.orchestra.run.smithi134.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-24T23:12:04.793 INFO:teuthology.orchestra.run.smithi134.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-24T23:12:04.793 INFO:teuthology.orchestra.run.smithi134.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-24T23:12:04.794 INFO:teuthology.lock.ops:Updating smithi134.front.sepia.ceph.com on lock server 2024-04-24T23:12:04.843 DEBUG:teuthology.orchestra.run.smithi148:> uname -m 2024-04-24T23:12:04.861 INFO:teuthology.orchestra.run.smithi148.stdout:x86_64 2024-04-24T23:12:04.861 DEBUG:teuthology.orchestra.run.smithi148:> cat /etc/os-release 2024-04-24T23:12:04.918 INFO:teuthology.orchestra.run.smithi148.stdout:NAME="CentOS Stream" 2024-04-24T23:12:04.918 INFO:teuthology.orchestra.run.smithi148.stdout:VERSION="9" 2024-04-24T23:12:04.918 INFO:teuthology.orchestra.run.smithi148.stdout:ID="centos" 2024-04-24T23:12:04.918 INFO:teuthology.orchestra.run.smithi148.stdout:ID_LIKE="rhel fedora" 2024-04-24T23:12:04.918 INFO:teuthology.orchestra.run.smithi148.stdout:VERSION_ID="9" 2024-04-24T23:12:04.918 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM_ID="platform:el9" 2024-04-24T23:12:04.918 INFO:teuthology.orchestra.run.smithi148.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-24T23:12:04.918 INFO:teuthology.orchestra.run.smithi148.stdout:ANSI_COLOR="0;31" 2024-04-24T23:12:04.919 INFO:teuthology.orchestra.run.smithi148.stdout:LOGO="fedora-logo-icon" 2024-04-24T23:12:04.919 INFO:teuthology.orchestra.run.smithi148.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-24T23:12:04.919 INFO:teuthology.orchestra.run.smithi148.stdout:HOME_URL="https://centos.org/" 2024-04-24T23:12:04.919 INFO:teuthology.orchestra.run.smithi148.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-24T23:12:04.919 INFO:teuthology.orchestra.run.smithi148.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-24T23:12:04.919 INFO:teuthology.orchestra.run.smithi148.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-24T23:12:04.919 INFO:teuthology.lock.ops:Updating smithi148.front.sepia.ceph.com on lock server 2024-04-24T23:12:04.959 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-24T23:12:04.970 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-24T23:12:04.976 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-24T23:12:04.976 DEBUG:teuthology.orchestra.run.smithi080:> test '!' -e /home/ubuntu/cephtest 2024-04-24T23:12:04.978 DEBUG:teuthology.orchestra.run.smithi134:> test '!' -e /home/ubuntu/cephtest 2024-04-24T23:12:04.980 DEBUG:teuthology.orchestra.run.smithi148:> test '!' -e /home/ubuntu/cephtest 2024-04-24T23:12:04.997 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-24T23:12:05.004 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-24T23:12:05.004 DEBUG:teuthology.orchestra.run.smithi080:> test -z $(ls -A /var/lib/ceph) 2024-04-24T23:12:05.035 DEBUG:teuthology.orchestra.run.smithi134:> test -z $(ls -A /var/lib/ceph) 2024-04-24T23:12:05.037 DEBUG:teuthology.orchestra.run.smithi148:> test -z $(ls -A /var/lib/ceph) 2024-04-24T23:12:05.142 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-24T23:12:05.247 INFO:teuthology.run_tasks:Running task kernel... 2024-04-24T23:12:05.262 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'} 2024-04-24T23:12:05.262 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-04-24T23:12:05.262 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-04-24T23:12:05.262 DEBUG:teuthology.orchestra.run.smithi080:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-24T23:12:05.278 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T23:12:05.279 DEBUG:teuthology.orchestra.run.smithi080:> uname -r 2024-04-24T23:12:05.336 INFO:teuthology.orchestra.run.smithi080.stdout:5.14.0-437.el9.x86_64 2024-04-24T23:12:05.336 INFO:teuthology.task.kernel:Running kernel on smithi080: 5.14.0-437.el9.x86_64 2024-04-24T23:12:05.336 DEBUG:teuthology.orchestra.run.smithi080:> sudo yum install -y kernel 2024-04-24T23:12:08.518 INFO:teuthology.orchestra.run.smithi080.stdout:CentOS Stream 9 - BaseOS 5.3 MB/s | 8.0 MB 00:01 2024-04-24T23:12:11.378 INFO:teuthology.orchestra.run.smithi080.stdout:CentOS Stream 9 - AppStream 11 MB/s | 19 MB 00:01 2024-04-24T23:12:16.003 INFO:teuthology.orchestra.run.smithi080.stdout:CentOS Stream 9 - CRB 6.2 MB/s | 6.2 MB 00:01 2024-04-24T23:12:17.587 INFO:teuthology.orchestra.run.smithi080.stdout:CentOS Stream 9 - Extras packages 72 kB/s | 16 kB 00:00 2024-04-24T23:12:19.330 INFO:teuthology.orchestra.run.smithi080.stdout:Extra Packages for Enterprise Linux 13 MB/s | 21 MB 00:01 2024-04-24T23:12:24.521 INFO:teuthology.orchestra.run.smithi080.stdout:lab-extras 13 kB/s | 1.7 kB 00:00 2024-04-24T23:12:26.369 INFO:teuthology.orchestra.run.smithi080.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-24T23:12:26.369 INFO:teuthology.orchestra.run.smithi080.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-24T23:12:26.459 INFO:teuthology.orchestra.run.smithi080.stdout:Dependencies resolved. 2024-04-24T23:12:26.460 INFO:teuthology.orchestra.run.smithi080.stdout:Nothing to do. 2024-04-24T23:12:26.460 INFO:teuthology.orchestra.run.smithi080.stdout:Complete! 2024-04-24T23:12:26.571 DEBUG:teuthology.orchestra.run.smithi080:> echo no | sudo yum reinstall kernel || true 2024-04-24T23:12:27.066 INFO:teuthology.orchestra.run.smithi080.stdout:Last metadata expiration check: 0:00:03 ago on Wed 24 Apr 2024 11:12:24 PM UTC. 2024-04-24T23:12:27.184 INFO:teuthology.orchestra.run.smithi080.stdout:Dependencies resolved. 2024-04-24T23:12:27.199 INFO:teuthology.orchestra.run.smithi080.stdout:================================================================================ 2024-04-24T23:12:27.199 INFO:teuthology.orchestra.run.smithi080.stdout: Package Architecture Version Repository Size 2024-04-24T23:12:27.199 INFO:teuthology.orchestra.run.smithi080.stdout:================================================================================ 2024-04-24T23:12:27.199 INFO:teuthology.orchestra.run.smithi080.stdout:Reinstalling: 2024-04-24T23:12:27.199 INFO:teuthology.orchestra.run.smithi080.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-24T23:12:27.199 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-24T23:12:27.199 INFO:teuthology.orchestra.run.smithi080.stdout:Transaction Summary 2024-04-24T23:12:27.199 INFO:teuthology.orchestra.run.smithi080.stdout:================================================================================ 2024-04-24T23:12:27.199 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-24T23:12:27.200 INFO:teuthology.orchestra.run.smithi080.stdout:Total download size: 255 k 2024-04-24T23:12:27.200 INFO:teuthology.orchestra.run.smithi080.stdout:Installed size: 0 2024-04-24T23:12:27.200 INFO:teuthology.orchestra.run.smithi080.stderr:Operation aborted. 2024-04-24T23:12:27.265 INFO:teuthology.orchestra.run.smithi080.stdout:Is this ok [y/N]: 2024-04-24T23:12:27.265 DEBUG:teuthology.orchestra.run.smithi080:> sudo yum reinstall -y kernel || true 2024-04-24T23:12:27.757 INFO:teuthology.orchestra.run.smithi080.stdout:Last metadata expiration check: 0:00:03 ago on Wed 24 Apr 2024 11:12:24 PM UTC. 2024-04-24T23:12:27.873 INFO:teuthology.orchestra.run.smithi080.stdout:Dependencies resolved. 2024-04-24T23:12:27.875 INFO:teuthology.orchestra.run.smithi080.stdout:================================================================================ 2024-04-24T23:12:27.875 INFO:teuthology.orchestra.run.smithi080.stdout: Package Architecture Version Repository Size 2024-04-24T23:12:27.875 INFO:teuthology.orchestra.run.smithi080.stdout:================================================================================ 2024-04-24T23:12:27.875 INFO:teuthology.orchestra.run.smithi080.stdout:Reinstalling: 2024-04-24T23:12:27.875 INFO:teuthology.orchestra.run.smithi080.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-24T23:12:27.875 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-24T23:12:27.876 INFO:teuthology.orchestra.run.smithi080.stdout:Transaction Summary 2024-04-24T23:12:27.876 INFO:teuthology.orchestra.run.smithi080.stdout:================================================================================ 2024-04-24T23:12:27.876 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-24T23:12:27.876 INFO:teuthology.orchestra.run.smithi080.stdout:Total download size: 255 k 2024-04-24T23:12:27.876 INFO:teuthology.orchestra.run.smithi080.stdout:Installed size: 0 2024-04-24T23:12:27.876 INFO:teuthology.orchestra.run.smithi080.stdout:Downloading Packages: 2024-04-24T23:12:28.565 INFO:teuthology.orchestra.run.smithi080.stdout:kernel-5.14.0-437.el9.x86_64.rpm 424 kB/s | 255 kB 00:00 2024-04-24T23:12:28.565 INFO:teuthology.orchestra.run.smithi080.stdout:-------------------------------------------------------------------------------- 2024-04-24T23:12:28.565 INFO:teuthology.orchestra.run.smithi080.stdout:Total 369 kB/s | 255 kB 00:00 2024-04-24T23:12:28.583 INFO:teuthology.orchestra.run.smithi080.stdout:Running transaction check 2024-04-24T23:12:28.766 INFO:teuthology.orchestra.run.smithi080.stdout:Transaction check succeeded. 2024-04-24T23:12:28.767 INFO:teuthology.orchestra.run.smithi080.stdout:Running transaction test 2024-04-24T23:12:28.771 INFO:teuthology.orchestra.run.smithi080.stdout:Transaction test succeeded. 2024-04-24T23:12:28.772 INFO:teuthology.orchestra.run.smithi080.stdout:Running transaction 2024-04-24T23:12:28.893 INFO:teuthology.orchestra.run.smithi080.stdout: Preparing : 1/1 2024-04-24T23:12:28.982 INFO:teuthology.orchestra.run.smithi080.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-24T23:12:28.982 INFO:teuthology.orchestra.run.smithi080.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-24T23:12:29.690 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-24T23:12:29.690 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-24T23:12:30.402 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-24T23:12:30.402 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-24T23:12:30.403 INFO:teuthology.orchestra.run.smithi080.stdout:Reinstalled: 2024-04-24T23:12:30.403 INFO:teuthology.orchestra.run.smithi080.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-24T23:12:30.403 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-24T23:12:30.403 INFO:teuthology.orchestra.run.smithi080.stdout:Complete! 2024-04-24T23:12:30.571 DEBUG:teuthology.orchestra.run.smithi080:> rpm -q kernel | sort -rV | head -n 1 2024-04-24T23:12:30.618 INFO:teuthology.orchestra.run.smithi080.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-24T23:12:30.618 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-24T23:12:30.618 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-24T23:12:30.618 DEBUG:teuthology.parallel:result is None 2024-04-24T23:12:30.619 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-24T23:12:30.626 INFO:teuthology.task.internal:Creating test directory... 2024-04-24T23:12:30.626 DEBUG:teuthology.orchestra.run.smithi080:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-24T23:12:30.628 DEBUG:teuthology.orchestra.run.smithi134:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-24T23:12:30.631 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-24T23:12:30.649 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-24T23:12:30.656 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-24T23:12:30.668 INFO:teuthology.task.internal:Creating archive directory... 2024-04-24T23:12:30.669 DEBUG:teuthology.orchestra.run.smithi080:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-24T23:12:30.683 DEBUG:teuthology.orchestra.run.smithi134:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-24T23:12:30.689 DEBUG:teuthology.orchestra.run.smithi148:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-24T23:12:30.794 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-24T23:12:30.801 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-24T23:12:30.801 DEBUG:teuthology.orchestra.run.smithi134:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-24T23:12:30.817 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T23:12:30.818 DEBUG:teuthology.orchestra.run.smithi148:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-24T23:12:30.834 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T23:12:30.835 DEBUG:teuthology.orchestra.run.smithi080:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-24T23:12:30.837 DEBUG:teuthology.orchestra.run.smithi134:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-24T23:12:30.860 DEBUG:teuthology.orchestra.run.smithi148:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-24T23:12:30.867 INFO:teuthology.orchestra.run.smithi080.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-24T23:12:30.882 INFO:teuthology.orchestra.run.smithi080.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-24T23:12:30.904 INFO:teuthology.orchestra.run.smithi148.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-24T23:12:30.910 INFO:teuthology.orchestra.run.smithi134.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-24T23:12:30.932 INFO:teuthology.orchestra.run.smithi134.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-24T23:12:30.933 INFO:teuthology.orchestra.run.smithi148.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-24T23:12:30.935 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-24T23:12:30.942 INFO:teuthology.task.internal:Configuring sudo... 2024-04-24T23:12:30.942 DEBUG:teuthology.orchestra.run.smithi080:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-24T23:12:30.945 DEBUG:teuthology.orchestra.run.smithi134:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-24T23:12:30.947 DEBUG:teuthology.orchestra.run.smithi148:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-24T23:12:31.001 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-24T23:12:31.012 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-24T23:12:31.012 DEBUG:teuthology.orchestra.run.smithi080:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-24T23:12:31.015 DEBUG:teuthology.orchestra.run.smithi134:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-24T23:12:31.018 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-24T23:12:31.058 DEBUG:teuthology.orchestra.run.smithi080:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-24T23:12:31.112 DEBUG:teuthology.orchestra.run.smithi080:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-24T23:12:31.157 DEBUG:teuthology.orchestra.run.smithi080:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-24T23:12:31.242 DEBUG:teuthology.orchestra.run.smithi080:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-24T23:12:31.277 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-24T23:12:31.277 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-24T23:12:31.340 DEBUG:teuthology.orchestra.run.smithi134:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-24T23:12:31.384 DEBUG:teuthology.orchestra.run.smithi134:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-24T23:12:31.437 DEBUG:teuthology.orchestra.run.smithi134:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-24T23:12:31.525 DEBUG:teuthology.orchestra.run.smithi134:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-24T23:12:31.562 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-24T23:12:31.562 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-24T23:12:31.627 DEBUG:teuthology.orchestra.run.smithi148:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-24T23:12:31.678 DEBUG:teuthology.orchestra.run.smithi148:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-24T23:12:31.729 DEBUG:teuthology.orchestra.run.smithi148:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-24T23:12:31.815 DEBUG:teuthology.orchestra.run.smithi148:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-24T23:12:31.852 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-24T23:12:31.853 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-24T23:12:31.918 DEBUG:teuthology.orchestra.run.smithi080:> sudo service rsyslog restart 2024-04-24T23:12:31.920 DEBUG:teuthology.orchestra.run.smithi134:> sudo service rsyslog restart 2024-04-24T23:12:31.923 DEBUG:teuthology.orchestra.run.smithi148:> sudo service rsyslog restart 2024-04-24T23:12:31.957 INFO:teuthology.orchestra.run.smithi134.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-24T23:12:31.959 INFO:teuthology.orchestra.run.smithi080.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-24T23:12:31.994 INFO:teuthology.orchestra.run.smithi148.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-24T23:12:32.321 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-24T23:12:32.330 INFO:teuthology.task.internal:Starting timer... 2024-04-24T23:12:32.330 INFO:teuthology.run_tasks:Running task pcp... 2024-04-24T23:12:32.339 INFO:teuthology.run_tasks:Running task selinux... 2024-04-24T23:12:32.347 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-04-24T23:12:32.347 DEBUG:teuthology.orchestra.run.smithi080:> sudo service auditd rotate 2024-04-24T23:12:32.420 INFO:teuthology.orchestra.run.smithi080.stdout:Rotating logs: 2024-04-24T23:12:32.422 DEBUG:teuthology.orchestra.run.smithi134:> sudo service auditd rotate 2024-04-24T23:12:32.484 INFO:teuthology.orchestra.run.smithi134.stdout:Rotating logs: 2024-04-24T23:12:32.486 DEBUG:teuthology.orchestra.run.smithi148:> sudo service auditd rotate 2024-04-24T23:12:32.555 INFO:teuthology.orchestra.run.smithi148.stdout:Rotating logs: 2024-04-24T23:12:32.557 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-24T23:12:32.557 DEBUG:teuthology.orchestra.run.smithi080:> /usr/sbin/getenforce 2024-04-24T23:12:32.579 INFO:teuthology.orchestra.run.smithi080.stdout:Permissive 2024-04-24T23:12:32.579 DEBUG:teuthology.orchestra.run.smithi134:> /usr/sbin/getenforce 2024-04-24T23:12:32.602 INFO:teuthology.orchestra.run.smithi134.stdout:Permissive 2024-04-24T23:12:32.602 DEBUG:teuthology.orchestra.run.smithi148:> /usr/sbin/getenforce 2024-04-24T23:12:32.626 INFO:teuthology.orchestra.run.smithi148.stdout:Permissive 2024-04-24T23:12:32.627 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi080.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi134.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi148.front.sepia.ceph.com': 'permissive'} 2024-04-24T23:12:32.627 DEBUG:teuthology.orchestra.run.smithi080:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-24T23:12:32.653 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T23:12:32.654 DEBUG:teuthology.orchestra.run.smithi134:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-24T23:12:32.679 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T23:12:32.679 DEBUG:teuthology.orchestra.run.smithi148:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-24T23:12:32.704 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T23:12:32.705 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-24T23:12:32.705 DEBUG:teuthology.orchestra.run.smithi080:> sudo /usr/sbin/setenforce permissive 2024-04-24T23:12:32.730 DEBUG:teuthology.orchestra.run.smithi134:> sudo /usr/sbin/setenforce permissive 2024-04-24T23:12:32.756 DEBUG:teuthology.orchestra.run.smithi148:> sudo /usr/sbin/setenforce permissive 2024-04-24T23:12:32.787 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-24T23:12:32.797 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-04-24T23:12:32.804 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-04-24T23:12:32.998 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-24T23:12:33.013 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-24T23:12:33.015 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi080.front.sepia.ceph.com,smithi134.front.sepia.ceph.com,smithi148.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-24T23:16:30.338 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi080.front.sepia.ceph.com'), Remote(name='ubuntu@smithi134.front.sepia.ceph.com'), Remote(name='ubuntu@smithi148.front.sepia.ceph.com')] 2024-04-24T23:16:30.339 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi080.front.sepia.ceph.com' 2024-04-24T23:16:30.340 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T23:16:30.435 DEBUG:teuthology.orchestra.run.smithi080:> true 2024-04-24T23:16:30.514 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi080.front.sepia.ceph.com' 2024-04-24T23:16:30.514 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi134.front.sepia.ceph.com' 2024-04-24T23:16:30.514 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi134.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T23:16:30.595 DEBUG:teuthology.orchestra.run.smithi134:> true 2024-04-24T23:16:30.632 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi134.front.sepia.ceph.com' 2024-04-24T23:16:30.632 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi148.front.sepia.ceph.com' 2024-04-24T23:16:30.633 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T23:16:30.714 DEBUG:teuthology.orchestra.run.smithi148:> true 2024-04-24T23:16:30.796 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi148.front.sepia.ceph.com' 2024-04-24T23:16:30.797 INFO:teuthology.run_tasks:Running task clock... 2024-04-24T23:16:30.808 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-24T23:16:30.808 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-24T23:16:30.809 DEBUG:teuthology.orchestra.run.smithi080:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-24T23:16:30.811 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-24T23:16:30.811 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-24T23:16:30.814 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-24T23:16:30.814 DEBUG:teuthology.orchestra.run.smithi148:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-24T23:16:30.840 INFO:teuthology.orchestra.run.smithi080.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-24T23:16:30.841 INFO:teuthology.orchestra.run.smithi134.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-24T23:16:30.855 INFO:teuthology.orchestra.run.smithi080.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-24T23:16:30.855 INFO:teuthology.orchestra.run.smithi134.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-24T23:16:30.864 INFO:teuthology.orchestra.run.smithi148.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-24T23:16:30.878 INFO:teuthology.orchestra.run.smithi148.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-24T23:16:30.878 INFO:teuthology.orchestra.run.smithi134.stderr:sudo: ntpd: command not found 2024-04-24T23:16:30.879 INFO:teuthology.orchestra.run.smithi080.stderr:sudo: ntpd: command not found 2024-04-24T23:16:30.889 INFO:teuthology.orchestra.run.smithi134.stdout:506 Cannot talk to daemon 2024-04-24T23:16:30.890 INFO:teuthology.orchestra.run.smithi080.stdout:506 Cannot talk to daemon 2024-04-24T23:16:30.900 INFO:teuthology.orchestra.run.smithi148.stderr:sudo: ntpd: command not found 2024-04-24T23:16:30.902 INFO:teuthology.orchestra.run.smithi134.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-24T23:16:30.903 INFO:teuthology.orchestra.run.smithi080.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-24T23:16:30.910 INFO:teuthology.orchestra.run.smithi148.stdout:506 Cannot talk to daemon 2024-04-24T23:16:30.915 INFO:teuthology.orchestra.run.smithi134.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-24T23:16:30.917 INFO:teuthology.orchestra.run.smithi080.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-24T23:16:30.923 INFO:teuthology.orchestra.run.smithi148.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-24T23:16:30.936 INFO:teuthology.orchestra.run.smithi148.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-24T23:16:30.968 INFO:teuthology.orchestra.run.smithi134.stderr:bash: line 1: ntpq: command not found 2024-04-24T23:16:30.971 INFO:teuthology.orchestra.run.smithi134.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-24T23:16:30.971 INFO:teuthology.orchestra.run.smithi134.stdout:=============================================================================== 2024-04-24T23:16:30.972 INFO:teuthology.orchestra.run.smithi134.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T23:16:30.972 INFO:teuthology.orchestra.run.smithi134.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T23:16:30.972 INFO:teuthology.orchestra.run.smithi134.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T23:16:30.972 INFO:teuthology.orchestra.run.smithi134.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T23:16:30.978 INFO:teuthology.orchestra.run.smithi080.stderr:bash: line 1: ntpq: command not found 2024-04-24T23:16:30.982 INFO:teuthology.orchestra.run.smithi080.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-24T23:16:30.982 INFO:teuthology.orchestra.run.smithi080.stdout:=============================================================================== 2024-04-24T23:16:30.982 INFO:teuthology.orchestra.run.smithi080.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T23:16:30.982 INFO:teuthology.orchestra.run.smithi080.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T23:16:30.982 INFO:teuthology.orchestra.run.smithi080.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T23:16:30.982 INFO:teuthology.orchestra.run.smithi080.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T23:16:30.992 INFO:teuthology.orchestra.run.smithi148.stderr:bash: line 1: ntpq: command not found 2024-04-24T23:16:30.995 INFO:teuthology.orchestra.run.smithi148.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-24T23:16:30.995 INFO:teuthology.orchestra.run.smithi148.stdout:=============================================================================== 2024-04-24T23:16:30.995 INFO:teuthology.orchestra.run.smithi148.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T23:16:30.995 INFO:teuthology.orchestra.run.smithi148.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T23:16:30.995 INFO:teuthology.orchestra.run.smithi148.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T23:16:30.995 INFO:teuthology.orchestra.run.smithi148.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T23:16:30.996 INFO:teuthology.run_tasks:Running task install... 2024-04-24T23:16:31.005 DEBUG:teuthology.task.install:project ceph 2024-04-24T23:16:31.005 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551'}} 2024-04-24T23:16:31.005 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551'} 2024-04-24T23:16:31.005 INFO:teuthology.task.install:Using flavor: default 2024-04-24T23:16:31.013 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-24T23:16:31.013 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-24T23:16:31.014 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'tag': None, 'wait_for_package': False} 2024-04-24T23:16:31.014 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T23:16:31.016 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'tag': None, 'wait_for_package': False} 2024-04-24T23:16:31.016 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T23:16:31.017 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'tag': None, 'wait_for_package': False} 2024-04-24T23:16:31.017 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T23:16:31.174 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/ 2024-04-24T23:16:31.174 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2445.gd2c45d97 2024-04-24T23:16:31.190 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/ 2024-04-24T23:16:31.191 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2445.gd2c45d97 2024-04-24T23:16:31.244 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/ 2024-04-24T23:16:31.244 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2445.gd2c45d97 2024-04-24T23:16:31.293 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-24T23:16:31.293 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-24T23:16:31.293 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-24T23:16:31.308 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-24T23:16:31.308 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-24T23:16:31.308 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-24T23:16:31.325 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-24T23:16:31.325 DEBUG:teuthology.orchestra.run.smithi134:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d2c45d977142e024a59b68a8f1dce8f0b9840551/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-24T23:16:31.333 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-24T23:16:31.333 DEBUG:teuthology.orchestra.run.smithi148:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d2c45d977142e024a59b68a8f1dce8f0b9840551/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-24T23:16:31.391 DEBUG:teuthology.orchestra.run.smithi134:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-24T23:16:31.398 DEBUG:teuthology.orchestra.run.smithi148:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-24T23:16:31.420 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-24T23:16:31.420 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-24T23:16:31.421 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-24T23:16:31.446 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-24T23:16:31.446 DEBUG:teuthology.orchestra.run.smithi080:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d2c45d977142e024a59b68a8f1dce8f0b9840551/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-24T23:16:31.465 DEBUG:teuthology.orchestra.run.smithi134:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-24T23:16:31.471 DEBUG:teuthology.orchestra.run.smithi148:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-24T23:16:31.512 DEBUG:teuthology.orchestra.run.smithi080:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-24T23:16:31.528 INFO:teuthology.orchestra.run.smithi134.stdout:check_obsoletes = 1 2024-04-24T23:16:31.530 DEBUG:teuthology.orchestra.run.smithi134:> sudo yum clean all 2024-04-24T23:16:31.535 INFO:teuthology.orchestra.run.smithi148.stdout:check_obsoletes = 1 2024-04-24T23:16:31.536 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum clean all 2024-04-24T23:16:31.587 DEBUG:teuthology.orchestra.run.smithi080:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-24T23:16:31.651 INFO:teuthology.orchestra.run.smithi080.stdout:check_obsoletes = 1 2024-04-24T23:16:31.652 DEBUG:teuthology.orchestra.run.smithi080:> sudo yum clean all 2024-04-24T23:16:31.755 INFO:teuthology.orchestra.run.smithi134.stdout:48 files removed 2024-04-24T23:16:31.761 INFO:teuthology.orchestra.run.smithi148.stdout:48 files removed 2024-04-24T23:16:31.782 DEBUG:teuthology.orchestra.run.smithi134:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-24T23:16:31.789 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-24T23:16:31.876 INFO:teuthology.orchestra.run.smithi080.stdout:48 files removed 2024-04-24T23:16:31.903 DEBUG:teuthology.orchestra.run.smithi080:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-24T23:16:32.429 INFO:teuthology.orchestra.run.smithi134.stdout:ceph packages for x86_64 216 kB/s | 83 kB 00:00 2024-04-24T23:16:32.465 INFO:teuthology.orchestra.run.smithi148.stdout:ceph packages for x86_64 199 kB/s | 83 kB 00:00 2024-04-24T23:16:32.505 INFO:teuthology.orchestra.run.smithi080.stdout:ceph packages for x86_64 240 kB/s | 83 kB 00:00 2024-04-24T23:16:32.841 INFO:teuthology.orchestra.run.smithi148.stdout:ceph noarch packages 35 kB/s | 11 kB 00:00 2024-04-24T23:16:32.871 INFO:teuthology.orchestra.run.smithi134.stdout:ceph noarch packages 29 kB/s | 11 kB 00:00 2024-04-24T23:16:32.931 INFO:teuthology.orchestra.run.smithi080.stdout:ceph noarch packages 30 kB/s | 11 kB 00:00 2024-04-24T23:16:33.221 INFO:teuthology.orchestra.run.smithi134.stdout:ceph source packages 6.0 kB/s | 1.9 kB 00:00 2024-04-24T23:16:33.258 INFO:teuthology.orchestra.run.smithi148.stdout:ceph source packages 4.9 kB/s | 1.9 kB 00:00 2024-04-24T23:16:33.448 INFO:teuthology.orchestra.run.smithi080.stdout:ceph source packages 3.9 kB/s | 1.9 kB 00:00 2024-04-24T23:16:33.867 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.0 MB 00:00 2024-04-24T23:16:34.823 INFO:teuthology.orchestra.run.smithi080.stdout:CentOS Stream 9 - BaseOS 6.0 MB/s | 8.0 MB 00:01 2024-04-24T23:16:34.871 INFO:teuthology.orchestra.run.smithi134.stdout:CentOS Stream 9 - BaseOS 5.0 MB/s | 8.0 MB 00:01 2024-04-24T23:16:35.660 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS Stream 9 - AppStream 25 MB/s | 19 MB 00:00 2024-04-24T23:16:37.007 INFO:teuthology.orchestra.run.smithi080.stdout:CentOS Stream 9 - AppStream 17 MB/s | 19 MB 00:01 2024-04-24T23:16:38.239 INFO:teuthology.orchestra.run.smithi134.stdout:CentOS Stream 9 - AppStream 8.2 MB/s | 19 MB 00:02 2024-04-24T23:16:41.025 INFO:teuthology.orchestra.run.smithi080.stdout:CentOS Stream 9 - CRB 13 MB/s | 6.2 MB 00:00 2024-04-24T23:16:42.219 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS Stream 9 - CRB 2.1 MB/s | 6.2 MB 00:02 2024-04-24T23:16:42.256 INFO:teuthology.orchestra.run.smithi134.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.2 MB 00:00 2024-04-24T23:16:42.561 INFO:teuthology.orchestra.run.smithi080.stdout:CentOS Stream 9 - Extras packages 40 kB/s | 16 kB 00:00 2024-04-24T23:16:43.724 INFO:teuthology.orchestra.run.smithi134.stdout:CentOS Stream 9 - Extras packages 49 kB/s | 16 kB 00:00 2024-04-24T23:16:43.979 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS Stream 9 - Extras packages 27 kB/s | 16 kB 00:00 2024-04-24T23:16:44.156 INFO:teuthology.orchestra.run.smithi080.stdout:Extra Packages for Enterprise Linux 14 MB/s | 21 MB 00:01 2024-04-24T23:16:45.442 INFO:teuthology.orchestra.run.smithi134.stdout:Extra Packages for Enterprise Linux 13 MB/s | 21 MB 00:01 2024-04-24T23:16:45.462 INFO:teuthology.orchestra.run.smithi148.stdout:Extra Packages for Enterprise Linux 15 MB/s | 21 MB 00:01 2024-04-24T23:16:49.220 INFO:teuthology.orchestra.run.smithi080.stdout:lab-extras 14 kB/s | 1.7 kB 00:00 2024-04-24T23:16:50.483 INFO:teuthology.orchestra.run.smithi134.stdout:lab-extras 16 kB/s | 1.7 kB 00:00 2024-04-24T23:16:50.579 INFO:teuthology.orchestra.run.smithi148.stdout:lab-extras 12 kB/s | 1.7 kB 00:00 2024-04-24T23:16:51.346 INFO:teuthology.orchestra.run.smithi080.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-24T23:16:51.346 INFO:teuthology.orchestra.run.smithi080.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-24T23:16:51.357 INFO:teuthology.orchestra.run.smithi080.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-24T23:16:51.358 INFO:teuthology.orchestra.run.smithi080.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-24T23:16:51.359 INFO:teuthology.orchestra.run.smithi080.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-24T23:16:51.359 INFO:teuthology.orchestra.run.smithi080.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-24T23:16:51.361 INFO:teuthology.orchestra.run.smithi080.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-24T23:16:51.363 INFO:teuthology.orchestra.run.smithi080.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-24T23:16:51.364 INFO:teuthology.orchestra.run.smithi080.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-24T23:16:51.425 INFO:teuthology.orchestra.run.smithi080.stdout:Dependencies resolved. 2024-04-24T23:16:51.434 INFO:teuthology.orchestra.run.smithi080.stdout:======================================================================================= 2024-04-24T23:16:51.434 INFO:teuthology.orchestra.run.smithi080.stdout: Package Arch Version Repository Size 2024-04-24T23:16:51.434 INFO:teuthology.orchestra.run.smithi080.stdout:======================================================================================= 2024-04-24T23:16:51.434 INFO:teuthology.orchestra.run.smithi080.stdout:Installing: 2024-04-24T23:16:51.434 INFO:teuthology.orchestra.run.smithi080.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-24T23:16:51.434 INFO:teuthology.orchestra.run.smithi080.stdout: ceph x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 6.6 k 2024-04-24T23:16:51.434 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-base x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 5.4 M 2024-04-24T23:16:51.434 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-fuse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 890 k 2024-04-24T23:16:51.434 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 145 k 2024-04-24T23:16:51.434 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-mgr x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 1.5 M 2024-04-24T23:16:51.434 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 141 k 2024-04-24T23:16:51.435 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 3.5 M 2024-04-24T23:16:51.435 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 7.4 M 2024-04-24T23:16:51.435 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-mgr-rook noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 49 k 2024-04-24T23:16:51.435 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-radosgw x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 10 M 2024-04-24T23:16:51.435 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-test x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 48 M 2024-04-24T23:16:51.435 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-volume noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 264 k 2024-04-24T23:16:51.435 INFO:teuthology.orchestra.run.smithi080.stdout: cephadm noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 762 k 2024-04-24T23:16:51.435 INFO:teuthology.orchestra.run.smithi080.stdout: cephfs-mirror x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 222 k 2024-04-24T23:16:51.435 INFO:teuthology.orchestra.run.smithi080.stdout: cephfs-shell x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 27 k 2024-04-24T23:16:51.435 INFO:teuthology.orchestra.run.smithi080.stdout: cephfs-top noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 18 k 2024-04-24T23:16:51.435 INFO:teuthology.orchestra.run.smithi080.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-24T23:16:51.435 INFO:teuthology.orchestra.run.smithi080.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-24T23:16:51.435 INFO:teuthology.orchestra.run.smithi080.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-24T23:16:51.435 INFO:teuthology.orchestra.run.smithi080.stdout: libcephfs-devel x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 32 k 2024-04-24T23:16:51.435 INFO:teuthology.orchestra.run.smithi080.stdout: libcephfs2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 745 k 2024-04-24T23:16:51.436 INFO:teuthology.orchestra.run.smithi080.stdout: librados-devel x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 127 k 2024-04-24T23:16:51.436 INFO:teuthology.orchestra.run.smithi080.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-24T23:16:51.436 INFO:teuthology.orchestra.run.smithi080.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-24T23:16:51.436 INFO:teuthology.orchestra.run.smithi080.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-24T23:16:51.436 INFO:teuthology.orchestra.run.smithi080.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-24T23:16:51.436 INFO:teuthology.orchestra.run.smithi080.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-24T23:16:51.436 INFO:teuthology.orchestra.run.smithi080.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-24T23:16:51.436 INFO:teuthology.orchestra.run.smithi080.stdout: python3-cephfs x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 162 k 2024-04-24T23:16:51.436 INFO:teuthology.orchestra.run.smithi080.stdout: python3-rados x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 321 k 2024-04-24T23:16:51.436 INFO:teuthology.orchestra.run.smithi080.stdout: python3-rbd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 299 k 2024-04-24T23:16:51.436 INFO:teuthology.orchestra.run.smithi080.stdout: python3-rgw x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 99 k 2024-04-24T23:16:51.436 INFO:teuthology.orchestra.run.smithi080.stdout: rbd-fuse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 85 k 2024-04-24T23:16:51.436 INFO:teuthology.orchestra.run.smithi080.stdout: rbd-mirror x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.1 M 2024-04-24T23:16:51.436 INFO:teuthology.orchestra.run.smithi080.stdout: rbd-nbd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 171 k 2024-04-24T23:16:51.436 INFO:teuthology.orchestra.run.smithi080.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-24T23:16:51.437 INFO:teuthology.orchestra.run.smithi080.stdout:Upgrading: 2024-04-24T23:16:51.437 INFO:teuthology.orchestra.run.smithi080.stdout: librados2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.4 M 2024-04-24T23:16:51.437 INFO:teuthology.orchestra.run.smithi080.stdout: librbd1 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.1 M 2024-04-24T23:16:51.437 INFO:teuthology.orchestra.run.smithi080.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-24T23:16:51.437 INFO:teuthology.orchestra.run.smithi080.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-24T23:16:51.437 INFO:teuthology.orchestra.run.smithi080.stdout:Installing dependencies: 2024-04-24T23:16:51.437 INFO:teuthology.orchestra.run.smithi080.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-24T23:16:51.437 INFO:teuthology.orchestra.run.smithi080.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-24T23:16:51.437 INFO:teuthology.orchestra.run.smithi080.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-24T23:16:51.437 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-common x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 21 M 2024-04-24T23:16:51.437 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 32 k 2024-04-24T23:16:51.437 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-mds x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 2.4 M 2024-04-24T23:16:51.437 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 249 k 2024-04-24T23:16:51.437 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-mon x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 4.7 M 2024-04-24T23:16:51.437 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-osd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 17 M 2024-04-24T23:16:51.437 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 15 k 2024-04-24T23:16:51.438 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-selinux x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 25 k 2024-04-24T23:16:51.438 INFO:teuthology.orchestra.run.smithi080.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-24T23:16:51.438 INFO:teuthology.orchestra.run.smithi080.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-24T23:16:51.438 INFO:teuthology.orchestra.run.smithi080.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-24T23:16:51.438 INFO:teuthology.orchestra.run.smithi080.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-24T23:16:51.438 INFO:teuthology.orchestra.run.smithi080.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-24T23:16:51.438 INFO:teuthology.orchestra.run.smithi080.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-24T23:16:51.438 INFO:teuthology.orchestra.run.smithi080.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-24T23:16:51.438 INFO:teuthology.orchestra.run.smithi080.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-24T23:16:51.438 INFO:teuthology.orchestra.run.smithi080.stdout: libcephsqlite x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 163 k 2024-04-24T23:16:51.438 INFO:teuthology.orchestra.run.smithi080.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-24T23:16:51.438 INFO:teuthology.orchestra.run.smithi080.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-24T23:16:51.438 INFO:teuthology.orchestra.run.smithi080.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-24T23:16:51.439 INFO:teuthology.orchestra.run.smithi080.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-24T23:16:51.439 INFO:teuthology.orchestra.run.smithi080.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-24T23:16:51.439 INFO:teuthology.orchestra.run.smithi080.stdout: libradosstriper1 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 501 k 2024-04-24T23:16:51.439 INFO:teuthology.orchestra.run.smithi080.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-24T23:16:51.439 INFO:teuthology.orchestra.run.smithi080.stdout: librgw2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 5.3 M 2024-04-24T23:16:51.439 INFO:teuthology.orchestra.run.smithi080.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-24T23:16:51.439 INFO:teuthology.orchestra.run.smithi080.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-24T23:16:51.439 INFO:teuthology.orchestra.run.smithi080.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-24T23:16:51.439 INFO:teuthology.orchestra.run.smithi080.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-24T23:16:51.439 INFO:teuthology.orchestra.run.smithi080.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-24T23:16:51.439 INFO:teuthology.orchestra.run.smithi080.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-24T23:16:51.439 INFO:teuthology.orchestra.run.smithi080.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-24T23:16:51.439 INFO:teuthology.orchestra.run.smithi080.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-24T23:16:51.439 INFO:teuthology.orchestra.run.smithi080.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-24T23:16:51.440 INFO:teuthology.orchestra.run.smithi080.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-24T23:16:51.440 INFO:teuthology.orchestra.run.smithi080.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-24T23:16:51.440 INFO:teuthology.orchestra.run.smithi080.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-24T23:16:51.440 INFO:teuthology.orchestra.run.smithi080.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-24T23:16:51.440 INFO:teuthology.orchestra.run.smithi080.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-24T23:16:51.440 INFO:teuthology.orchestra.run.smithi080.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-24T23:16:51.440 INFO:teuthology.orchestra.run.smithi080.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-24T23:16:51.440 INFO:teuthology.orchestra.run.smithi080.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-24T23:16:51.440 INFO:teuthology.orchestra.run.smithi080.stdout: python3-ceph-argparse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 45 k 2024-04-24T23:16:51.440 INFO:teuthology.orchestra.run.smithi080.stdout: python3-ceph-common x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 133 k 2024-04-24T23:16:51.440 INFO:teuthology.orchestra.run.smithi080.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-24T23:16:51.440 INFO:teuthology.orchestra.run.smithi080.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-24T23:16:51.440 INFO:teuthology.orchestra.run.smithi080.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-24T23:16:51.441 INFO:teuthology.orchestra.run.smithi080.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-24T23:16:51.441 INFO:teuthology.orchestra.run.smithi080.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-24T23:16:51.441 INFO:teuthology.orchestra.run.smithi080.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-24T23:16:51.441 INFO:teuthology.orchestra.run.smithi080.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-24T23:16:51.441 INFO:teuthology.orchestra.run.smithi080.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-24T23:16:51.441 INFO:teuthology.orchestra.run.smithi080.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-24T23:16:51.441 INFO:teuthology.orchestra.run.smithi080.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-24T23:16:51.441 INFO:teuthology.orchestra.run.smithi080.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-24T23:16:51.441 INFO:teuthology.orchestra.run.smithi080.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-24T23:16:51.441 INFO:teuthology.orchestra.run.smithi080.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-24T23:16:51.441 INFO:teuthology.orchestra.run.smithi080.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-24T23:16:51.442 INFO:teuthology.orchestra.run.smithi080.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-24T23:16:51.442 INFO:teuthology.orchestra.run.smithi080.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-24T23:16:51.442 INFO:teuthology.orchestra.run.smithi080.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-24T23:16:51.442 INFO:teuthology.orchestra.run.smithi080.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-24T23:16:51.442 INFO:teuthology.orchestra.run.smithi080.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-24T23:16:51.442 INFO:teuthology.orchestra.run.smithi080.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-24T23:16:51.442 INFO:teuthology.orchestra.run.smithi080.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-24T23:16:51.442 INFO:teuthology.orchestra.run.smithi080.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-24T23:16:51.442 INFO:teuthology.orchestra.run.smithi080.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-24T23:16:51.442 INFO:teuthology.orchestra.run.smithi080.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-24T23:16:51.442 INFO:teuthology.orchestra.run.smithi080.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-24T23:16:51.442 INFO:teuthology.orchestra.run.smithi080.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-24T23:16:51.442 INFO:teuthology.orchestra.run.smithi080.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-24T23:16:51.443 INFO:teuthology.orchestra.run.smithi080.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-24T23:16:51.443 INFO:teuthology.orchestra.run.smithi080.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-24T23:16:51.443 INFO:teuthology.orchestra.run.smithi080.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-24T23:16:51.443 INFO:teuthology.orchestra.run.smithi080.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-24T23:16:51.443 INFO:teuthology.orchestra.run.smithi080.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-24T23:16:51.443 INFO:teuthology.orchestra.run.smithi080.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-24T23:16:51.443 INFO:teuthology.orchestra.run.smithi080.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-24T23:16:51.443 INFO:teuthology.orchestra.run.smithi080.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-24T23:16:51.443 INFO:teuthology.orchestra.run.smithi080.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-24T23:16:51.443 INFO:teuthology.orchestra.run.smithi080.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-24T23:16:51.443 INFO:teuthology.orchestra.run.smithi080.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-24T23:16:51.443 INFO:teuthology.orchestra.run.smithi080.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-24T23:16:51.443 INFO:teuthology.orchestra.run.smithi080.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-24T23:16:51.444 INFO:teuthology.orchestra.run.smithi080.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-24T23:16:51.444 INFO:teuthology.orchestra.run.smithi080.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-24T23:16:51.444 INFO:teuthology.orchestra.run.smithi080.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-24T23:16:51.444 INFO:teuthology.orchestra.run.smithi080.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-24T23:16:51.444 INFO:teuthology.orchestra.run.smithi080.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-24T23:16:51.444 INFO:teuthology.orchestra.run.smithi080.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-24T23:16:51.444 INFO:teuthology.orchestra.run.smithi080.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-24T23:16:51.444 INFO:teuthology.orchestra.run.smithi080.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-24T23:16:51.444 INFO:teuthology.orchestra.run.smithi080.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-24T23:16:51.444 INFO:teuthology.orchestra.run.smithi080.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-24T23:16:51.444 INFO:teuthology.orchestra.run.smithi080.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-24T23:16:51.444 INFO:teuthology.orchestra.run.smithi080.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-24T23:16:51.444 INFO:teuthology.orchestra.run.smithi080.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-24T23:16:51.445 INFO:teuthology.orchestra.run.smithi080.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-24T23:16:51.445 INFO:teuthology.orchestra.run.smithi080.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-24T23:16:51.445 INFO:teuthology.orchestra.run.smithi080.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-24T23:16:51.445 INFO:teuthology.orchestra.run.smithi080.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-24T23:16:51.445 INFO:teuthology.orchestra.run.smithi080.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-24T23:16:51.445 INFO:teuthology.orchestra.run.smithi080.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-24T23:16:51.445 INFO:teuthology.orchestra.run.smithi080.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-24T23:16:51.445 INFO:teuthology.orchestra.run.smithi080.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-24T23:16:51.445 INFO:teuthology.orchestra.run.smithi080.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-24T23:16:51.445 INFO:teuthology.orchestra.run.smithi080.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-24T23:16:51.445 INFO:teuthology.orchestra.run.smithi080.stdout:Installing weak dependencies: 2024-04-24T23:16:51.445 INFO:teuthology.orchestra.run.smithi080.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-24T23:16:51.445 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-24T23:16:51.445 INFO:teuthology.orchestra.run.smithi080.stdout:Transaction Summary 2024-04-24T23:16:51.446 INFO:teuthology.orchestra.run.smithi080.stdout:======================================================================================= 2024-04-24T23:16:51.446 INFO:teuthology.orchestra.run.smithi080.stdout:Install 148 Packages 2024-04-24T23:16:51.446 INFO:teuthology.orchestra.run.smithi080.stdout:Upgrade 4 Packages 2024-04-24T23:16:51.446 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-24T23:16:51.446 INFO:teuthology.orchestra.run.smithi080.stdout:Total download size: 220 M 2024-04-24T23:16:51.447 INFO:teuthology.orchestra.run.smithi080.stdout:Downloading Packages: 2024-04-24T23:16:52.171 INFO:teuthology.orchestra.run.smithi080.stdout:(1/152): ceph-19.0.0-2445.gd2c45d97.el9.x86_64. 27 kB/s | 6.6 kB 00:00 2024-04-24T23:16:52.496 INFO:teuthology.orchestra.run.smithi080.stdout:(2/152): ceph-base-19.0.0-2445.gd2c45d97.el9.x8 9.4 MB/s | 5.4 MB 00:00 2024-04-24T23:16:52.555 INFO:teuthology.orchestra.run.smithi080.stdout:(3/152): ceph-immutable-object-cache-19.0.0-244 2.4 MB/s | 145 kB 00:00 2024-04-24T23:16:52.634 INFO:teuthology.orchestra.run.smithi134.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-24T23:16:52.635 INFO:teuthology.orchestra.run.smithi134.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-24T23:16:52.646 INFO:teuthology.orchestra.run.smithi134.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-24T23:16:52.646 INFO:teuthology.orchestra.run.smithi134.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-24T23:16:52.647 INFO:teuthology.orchestra.run.smithi134.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-24T23:16:52.648 INFO:teuthology.orchestra.run.smithi134.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-24T23:16:52.650 INFO:teuthology.orchestra.run.smithi134.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-24T23:16:52.652 INFO:teuthology.orchestra.run.smithi134.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-24T23:16:52.653 INFO:teuthology.orchestra.run.smithi134.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-24T23:16:52.713 INFO:teuthology.orchestra.run.smithi134.stdout:Dependencies resolved. 2024-04-24T23:16:52.713 INFO:teuthology.orchestra.run.smithi148.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-24T23:16:52.714 INFO:teuthology.orchestra.run.smithi148.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-24T23:16:52.725 INFO:teuthology.orchestra.run.smithi148.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-24T23:16:52.726 INFO:teuthology.orchestra.run.smithi148.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-24T23:16:52.727 INFO:teuthology.orchestra.run.smithi148.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-24T23:16:52.728 INFO:teuthology.orchestra.run.smithi148.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-24T23:16:52.729 INFO:teuthology.orchestra.run.smithi148.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-24T23:16:52.731 INFO:teuthology.orchestra.run.smithi148.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-24T23:16:52.732 INFO:teuthology.orchestra.run.smithi148.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-24T23:16:52.733 INFO:teuthology.orchestra.run.smithi134.stdout:======================================================================================= 2024-04-24T23:16:52.733 INFO:teuthology.orchestra.run.smithi134.stdout: Package Arch Version Repository Size 2024-04-24T23:16:52.733 INFO:teuthology.orchestra.run.smithi134.stdout:======================================================================================= 2024-04-24T23:16:52.733 INFO:teuthology.orchestra.run.smithi134.stdout:Installing: 2024-04-24T23:16:52.733 INFO:teuthology.orchestra.run.smithi134.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-24T23:16:52.733 INFO:teuthology.orchestra.run.smithi134.stdout: ceph x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 6.6 k 2024-04-24T23:16:52.733 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-base x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 5.4 M 2024-04-24T23:16:52.733 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-fuse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 890 k 2024-04-24T23:16:52.733 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 145 k 2024-04-24T23:16:52.733 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 1.5 M 2024-04-24T23:16:52.734 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 141 k 2024-04-24T23:16:52.734 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 3.5 M 2024-04-24T23:16:52.734 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 7.4 M 2024-04-24T23:16:52.734 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-rook noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 49 k 2024-04-24T23:16:52.734 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-radosgw x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 10 M 2024-04-24T23:16:52.734 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-test x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 48 M 2024-04-24T23:16:52.734 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-volume noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 264 k 2024-04-24T23:16:52.734 INFO:teuthology.orchestra.run.smithi134.stdout: cephadm noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 762 k 2024-04-24T23:16:52.734 INFO:teuthology.orchestra.run.smithi134.stdout: cephfs-mirror x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 222 k 2024-04-24T23:16:52.734 INFO:teuthology.orchestra.run.smithi134.stdout: cephfs-shell x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 27 k 2024-04-24T23:16:52.734 INFO:teuthology.orchestra.run.smithi134.stdout: cephfs-top noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 18 k 2024-04-24T23:16:52.734 INFO:teuthology.orchestra.run.smithi134.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-24T23:16:52.734 INFO:teuthology.orchestra.run.smithi134.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-24T23:16:52.735 INFO:teuthology.orchestra.run.smithi134.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-24T23:16:52.735 INFO:teuthology.orchestra.run.smithi134.stdout: libcephfs-devel x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 32 k 2024-04-24T23:16:52.735 INFO:teuthology.orchestra.run.smithi134.stdout: libcephfs2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 745 k 2024-04-24T23:16:52.735 INFO:teuthology.orchestra.run.smithi134.stdout: librados-devel x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 127 k 2024-04-24T23:16:52.735 INFO:teuthology.orchestra.run.smithi134.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-24T23:16:52.735 INFO:teuthology.orchestra.run.smithi134.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-24T23:16:52.735 INFO:teuthology.orchestra.run.smithi134.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-24T23:16:52.735 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-24T23:16:52.735 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-24T23:16:52.735 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-24T23:16:52.735 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cephfs x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 162 k 2024-04-24T23:16:52.735 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rados x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 321 k 2024-04-24T23:16:52.735 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rbd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 299 k 2024-04-24T23:16:52.735 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rgw x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 99 k 2024-04-24T23:16:52.736 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-fuse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 85 k 2024-04-24T23:16:52.736 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-mirror x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.1 M 2024-04-24T23:16:52.736 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-nbd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 171 k 2024-04-24T23:16:52.736 INFO:teuthology.orchestra.run.smithi134.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-24T23:16:52.736 INFO:teuthology.orchestra.run.smithi134.stdout:Upgrading: 2024-04-24T23:16:52.736 INFO:teuthology.orchestra.run.smithi134.stdout: librados2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.4 M 2024-04-24T23:16:52.736 INFO:teuthology.orchestra.run.smithi134.stdout: librbd1 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.1 M 2024-04-24T23:16:52.736 INFO:teuthology.orchestra.run.smithi134.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-24T23:16:52.736 INFO:teuthology.orchestra.run.smithi134.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-24T23:16:52.736 INFO:teuthology.orchestra.run.smithi134.stdout:Installing dependencies: 2024-04-24T23:16:52.736 INFO:teuthology.orchestra.run.smithi134.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-24T23:16:52.736 INFO:teuthology.orchestra.run.smithi134.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-24T23:16:52.737 INFO:teuthology.orchestra.run.smithi134.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-24T23:16:52.737 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-common x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 21 M 2024-04-24T23:16:52.737 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 32 k 2024-04-24T23:16:52.737 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mds x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 2.4 M 2024-04-24T23:16:52.737 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 249 k 2024-04-24T23:16:52.737 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mon x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 4.7 M 2024-04-24T23:16:52.737 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-osd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 17 M 2024-04-24T23:16:52.737 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 15 k 2024-04-24T23:16:52.737 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-selinux x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 25 k 2024-04-24T23:16:52.737 INFO:teuthology.orchestra.run.smithi134.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-24T23:16:52.737 INFO:teuthology.orchestra.run.smithi134.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-24T23:16:52.737 INFO:teuthology.orchestra.run.smithi134.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-24T23:16:52.737 INFO:teuthology.orchestra.run.smithi134.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-24T23:16:52.737 INFO:teuthology.orchestra.run.smithi134.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-24T23:16:52.738 INFO:teuthology.orchestra.run.smithi134.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-24T23:16:52.738 INFO:teuthology.orchestra.run.smithi134.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-24T23:16:52.738 INFO:teuthology.orchestra.run.smithi134.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-24T23:16:52.738 INFO:teuthology.orchestra.run.smithi134.stdout: libcephsqlite x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 163 k 2024-04-24T23:16:52.738 INFO:teuthology.orchestra.run.smithi134.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-24T23:16:52.738 INFO:teuthology.orchestra.run.smithi134.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-24T23:16:52.738 INFO:teuthology.orchestra.run.smithi134.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-24T23:16:52.738 INFO:teuthology.orchestra.run.smithi134.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-24T23:16:52.738 INFO:teuthology.orchestra.run.smithi134.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-24T23:16:52.738 INFO:teuthology.orchestra.run.smithi134.stdout: libradosstriper1 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 501 k 2024-04-24T23:16:52.738 INFO:teuthology.orchestra.run.smithi134.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-24T23:16:52.738 INFO:teuthology.orchestra.run.smithi134.stdout: librgw2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 5.3 M 2024-04-24T23:16:52.738 INFO:teuthology.orchestra.run.smithi134.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-24T23:16:52.739 INFO:teuthology.orchestra.run.smithi134.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-24T23:16:52.739 INFO:teuthology.orchestra.run.smithi134.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-24T23:16:52.739 INFO:teuthology.orchestra.run.smithi134.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-24T23:16:52.739 INFO:teuthology.orchestra.run.smithi134.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-24T23:16:52.739 INFO:teuthology.orchestra.run.smithi134.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-24T23:16:52.739 INFO:teuthology.orchestra.run.smithi134.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-24T23:16:52.739 INFO:teuthology.orchestra.run.smithi134.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-24T23:16:52.739 INFO:teuthology.orchestra.run.smithi134.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-24T23:16:52.739 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-24T23:16:52.739 INFO:teuthology.orchestra.run.smithi134.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-24T23:16:52.739 INFO:teuthology.orchestra.run.smithi134.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-24T23:16:52.739 INFO:teuthology.orchestra.run.smithi134.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-24T23:16:52.739 INFO:teuthology.orchestra.run.smithi134.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-24T23:16:52.739 INFO:teuthology.orchestra.run.smithi134.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-24T23:16:52.740 INFO:teuthology.orchestra.run.smithi134.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-24T23:16:52.740 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-24T23:16:52.740 INFO:teuthology.orchestra.run.smithi134.stdout: python3-ceph-argparse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 45 k 2024-04-24T23:16:52.740 INFO:teuthology.orchestra.run.smithi134.stdout: python3-ceph-common x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 133 k 2024-04-24T23:16:52.740 INFO:teuthology.orchestra.run.smithi134.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-24T23:16:52.740 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-24T23:16:52.740 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-24T23:16:52.740 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-24T23:16:52.740 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-24T23:16:52.740 INFO:teuthology.orchestra.run.smithi134.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-24T23:16:52.740 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-24T23:16:52.740 INFO:teuthology.orchestra.run.smithi134.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-24T23:16:52.740 INFO:teuthology.orchestra.run.smithi134.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-24T23:16:52.740 INFO:teuthology.orchestra.run.smithi134.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-24T23:16:52.741 INFO:teuthology.orchestra.run.smithi134.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-24T23:16:52.741 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-24T23:16:52.741 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-24T23:16:52.741 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-24T23:16:52.741 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-24T23:16:52.741 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-24T23:16:52.741 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-24T23:16:52.741 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-24T23:16:52.741 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-24T23:16:52.741 INFO:teuthology.orchestra.run.smithi134.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-24T23:16:52.741 INFO:teuthology.orchestra.run.smithi134.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-24T23:16:52.741 INFO:teuthology.orchestra.run.smithi134.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-24T23:16:52.741 INFO:teuthology.orchestra.run.smithi134.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-24T23:16:52.741 INFO:teuthology.orchestra.run.smithi134.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-24T23:16:52.742 INFO:teuthology.orchestra.run.smithi134.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-24T23:16:52.742 INFO:teuthology.orchestra.run.smithi134.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-24T23:16:52.742 INFO:teuthology.orchestra.run.smithi134.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-24T23:16:52.742 INFO:teuthology.orchestra.run.smithi134.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-24T23:16:52.742 INFO:teuthology.orchestra.run.smithi134.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-24T23:16:52.742 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-24T23:16:52.742 INFO:teuthology.orchestra.run.smithi134.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-24T23:16:52.742 INFO:teuthology.orchestra.run.smithi134.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-24T23:16:52.742 INFO:teuthology.orchestra.run.smithi134.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-24T23:16:52.742 INFO:teuthology.orchestra.run.smithi134.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-24T23:16:52.742 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-24T23:16:52.742 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-24T23:16:52.742 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-24T23:16:52.742 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-24T23:16:52.743 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-24T23:16:52.743 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-24T23:16:52.743 INFO:teuthology.orchestra.run.smithi134.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-24T23:16:52.743 INFO:teuthology.orchestra.run.smithi134.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-24T23:16:52.743 INFO:teuthology.orchestra.run.smithi134.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-24T23:16:52.743 INFO:teuthology.orchestra.run.smithi134.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-24T23:16:52.743 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-24T23:16:52.743 INFO:teuthology.orchestra.run.smithi134.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-24T23:16:52.743 INFO:teuthology.orchestra.run.smithi134.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-24T23:16:52.743 INFO:teuthology.orchestra.run.smithi134.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-24T23:16:52.743 INFO:teuthology.orchestra.run.smithi134.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-24T23:16:52.743 INFO:teuthology.orchestra.run.smithi134.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-24T23:16:52.743 INFO:teuthology.orchestra.run.smithi134.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-24T23:16:52.743 INFO:teuthology.orchestra.run.smithi134.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-24T23:16:52.744 INFO:teuthology.orchestra.run.smithi134.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-24T23:16:52.744 INFO:teuthology.orchestra.run.smithi134.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-24T23:16:52.744 INFO:teuthology.orchestra.run.smithi134.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-24T23:16:52.744 INFO:teuthology.orchestra.run.smithi134.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-24T23:16:52.744 INFO:teuthology.orchestra.run.smithi134.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-24T23:16:52.744 INFO:teuthology.orchestra.run.smithi134.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-24T23:16:52.744 INFO:teuthology.orchestra.run.smithi134.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-24T23:16:52.744 INFO:teuthology.orchestra.run.smithi134.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-24T23:16:52.744 INFO:teuthology.orchestra.run.smithi134.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-24T23:16:52.744 INFO:teuthology.orchestra.run.smithi134.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-24T23:16:52.744 INFO:teuthology.orchestra.run.smithi134.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-24T23:16:52.744 INFO:teuthology.orchestra.run.smithi134.stdout:Installing weak dependencies: 2024-04-24T23:16:52.744 INFO:teuthology.orchestra.run.smithi134.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-24T23:16:52.744 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-24T23:16:52.745 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction Summary 2024-04-24T23:16:52.745 INFO:teuthology.orchestra.run.smithi134.stdout:======================================================================================= 2024-04-24T23:16:52.745 INFO:teuthology.orchestra.run.smithi134.stdout:Install 148 Packages 2024-04-24T23:16:52.745 INFO:teuthology.orchestra.run.smithi134.stdout:Upgrade 4 Packages 2024-04-24T23:16:52.745 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-24T23:16:52.745 INFO:teuthology.orchestra.run.smithi134.stdout:Total download size: 220 M 2024-04-24T23:16:52.745 INFO:teuthology.orchestra.run.smithi134.stdout:Downloading Packages: 2024-04-24T23:16:52.772 INFO:teuthology.orchestra.run.smithi080.stdout:(4/152): ceph-mds-19.0.0-2445.gd2c45d97.el9.x86 11 MB/s | 2.4 MB 00:00 2024-04-24T23:16:52.793 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-04-24T23:16:52.807 INFO:teuthology.orchestra.run.smithi148.stdout:======================================================================================= 2024-04-24T23:16:52.808 INFO:teuthology.orchestra.run.smithi148.stdout: Package Arch Version Repository Size 2024-04-24T23:16:52.808 INFO:teuthology.orchestra.run.smithi148.stdout:======================================================================================= 2024-04-24T23:16:52.808 INFO:teuthology.orchestra.run.smithi148.stdout:Installing: 2024-04-24T23:16:52.808 INFO:teuthology.orchestra.run.smithi148.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-24T23:16:52.808 INFO:teuthology.orchestra.run.smithi148.stdout: ceph x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 6.6 k 2024-04-24T23:16:52.808 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-base x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 5.4 M 2024-04-24T23:16:52.808 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-fuse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 890 k 2024-04-24T23:16:52.808 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 145 k 2024-04-24T23:16:52.808 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 1.5 M 2024-04-24T23:16:52.808 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 141 k 2024-04-24T23:16:52.808 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 3.5 M 2024-04-24T23:16:52.808 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 7.4 M 2024-04-24T23:16:52.809 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-rook noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 49 k 2024-04-24T23:16:52.809 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-radosgw x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 10 M 2024-04-24T23:16:52.809 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-test x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 48 M 2024-04-24T23:16:52.809 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-volume noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 264 k 2024-04-24T23:16:52.809 INFO:teuthology.orchestra.run.smithi148.stdout: cephadm noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 762 k 2024-04-24T23:16:52.809 INFO:teuthology.orchestra.run.smithi148.stdout: cephfs-mirror x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 222 k 2024-04-24T23:16:52.809 INFO:teuthology.orchestra.run.smithi148.stdout: cephfs-shell x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 27 k 2024-04-24T23:16:52.809 INFO:teuthology.orchestra.run.smithi148.stdout: cephfs-top noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 18 k 2024-04-24T23:16:52.809 INFO:teuthology.orchestra.run.smithi148.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-24T23:16:52.809 INFO:teuthology.orchestra.run.smithi148.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-24T23:16:52.809 INFO:teuthology.orchestra.run.smithi148.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-24T23:16:52.809 INFO:teuthology.orchestra.run.smithi148.stdout: libcephfs-devel x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 32 k 2024-04-24T23:16:52.809 INFO:teuthology.orchestra.run.smithi148.stdout: libcephfs2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 745 k 2024-04-24T23:16:52.809 INFO:teuthology.orchestra.run.smithi148.stdout: librados-devel x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 127 k 2024-04-24T23:16:52.810 INFO:teuthology.orchestra.run.smithi148.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-24T23:16:52.810 INFO:teuthology.orchestra.run.smithi148.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-24T23:16:52.810 INFO:teuthology.orchestra.run.smithi148.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-24T23:16:52.810 INFO:teuthology.orchestra.run.smithi148.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-24T23:16:52.810 INFO:teuthology.orchestra.run.smithi148.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-24T23:16:52.810 INFO:teuthology.orchestra.run.smithi148.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-24T23:16:52.810 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cephfs x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 162 k 2024-04-24T23:16:52.810 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rados x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 321 k 2024-04-24T23:16:52.810 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rbd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 299 k 2024-04-24T23:16:52.810 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rgw x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 99 k 2024-04-24T23:16:52.810 INFO:teuthology.orchestra.run.smithi148.stdout: rbd-fuse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 85 k 2024-04-24T23:16:52.810 INFO:teuthology.orchestra.run.smithi148.stdout: rbd-mirror x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.1 M 2024-04-24T23:16:52.811 INFO:teuthology.orchestra.run.smithi148.stdout: rbd-nbd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 171 k 2024-04-24T23:16:52.811 INFO:teuthology.orchestra.run.smithi148.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-24T23:16:52.811 INFO:teuthology.orchestra.run.smithi148.stdout:Upgrading: 2024-04-24T23:16:52.811 INFO:teuthology.orchestra.run.smithi148.stdout: librados2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.4 M 2024-04-24T23:16:52.811 INFO:teuthology.orchestra.run.smithi148.stdout: librbd1 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.1 M 2024-04-24T23:16:52.811 INFO:teuthology.orchestra.run.smithi148.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-24T23:16:52.811 INFO:teuthology.orchestra.run.smithi148.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-24T23:16:52.811 INFO:teuthology.orchestra.run.smithi148.stdout:Installing dependencies: 2024-04-24T23:16:52.811 INFO:teuthology.orchestra.run.smithi148.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-24T23:16:52.811 INFO:teuthology.orchestra.run.smithi148.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-24T23:16:52.811 INFO:teuthology.orchestra.run.smithi148.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-24T23:16:52.811 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-common x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 21 M 2024-04-24T23:16:52.811 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 32 k 2024-04-24T23:16:52.811 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mds x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 2.4 M 2024-04-24T23:16:52.812 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 249 k 2024-04-24T23:16:52.812 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mon x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 4.7 M 2024-04-24T23:16:52.812 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-osd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 17 M 2024-04-24T23:16:52.812 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 15 k 2024-04-24T23:16:52.812 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-selinux x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 25 k 2024-04-24T23:16:52.812 INFO:teuthology.orchestra.run.smithi148.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-24T23:16:52.812 INFO:teuthology.orchestra.run.smithi148.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-24T23:16:52.812 INFO:teuthology.orchestra.run.smithi148.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-24T23:16:52.812 INFO:teuthology.orchestra.run.smithi148.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-24T23:16:52.812 INFO:teuthology.orchestra.run.smithi148.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-24T23:16:52.812 INFO:teuthology.orchestra.run.smithi148.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-24T23:16:52.812 INFO:teuthology.orchestra.run.smithi148.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-24T23:16:52.812 INFO:teuthology.orchestra.run.smithi148.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-24T23:16:52.813 INFO:teuthology.orchestra.run.smithi148.stdout: libcephsqlite x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 163 k 2024-04-24T23:16:52.813 INFO:teuthology.orchestra.run.smithi148.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-24T23:16:52.813 INFO:teuthology.orchestra.run.smithi148.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-24T23:16:52.813 INFO:teuthology.orchestra.run.smithi148.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-24T23:16:52.813 INFO:teuthology.orchestra.run.smithi148.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-24T23:16:52.813 INFO:teuthology.orchestra.run.smithi148.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-24T23:16:52.813 INFO:teuthology.orchestra.run.smithi148.stdout: libradosstriper1 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 501 k 2024-04-24T23:16:52.813 INFO:teuthology.orchestra.run.smithi148.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-24T23:16:52.813 INFO:teuthology.orchestra.run.smithi148.stdout: librgw2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 5.3 M 2024-04-24T23:16:52.813 INFO:teuthology.orchestra.run.smithi148.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-24T23:16:52.813 INFO:teuthology.orchestra.run.smithi148.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-24T23:16:52.813 INFO:teuthology.orchestra.run.smithi148.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-24T23:16:52.813 INFO:teuthology.orchestra.run.smithi148.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-24T23:16:52.813 INFO:teuthology.orchestra.run.smithi148.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-24T23:16:52.814 INFO:teuthology.orchestra.run.smithi148.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-24T23:16:52.814 INFO:teuthology.orchestra.run.smithi148.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-24T23:16:52.814 INFO:teuthology.orchestra.run.smithi148.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-24T23:16:52.814 INFO:teuthology.orchestra.run.smithi148.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-24T23:16:52.814 INFO:teuthology.orchestra.run.smithi148.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-24T23:16:52.814 INFO:teuthology.orchestra.run.smithi148.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-24T23:16:52.814 INFO:teuthology.orchestra.run.smithi148.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-24T23:16:52.814 INFO:teuthology.orchestra.run.smithi148.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-24T23:16:52.814 INFO:teuthology.orchestra.run.smithi148.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-24T23:16:52.814 INFO:teuthology.orchestra.run.smithi148.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-24T23:16:52.814 INFO:teuthology.orchestra.run.smithi148.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-24T23:16:52.814 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-24T23:16:52.814 INFO:teuthology.orchestra.run.smithi148.stdout: python3-ceph-argparse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 45 k 2024-04-24T23:16:52.815 INFO:teuthology.orchestra.run.smithi148.stdout: python3-ceph-common x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 133 k 2024-04-24T23:16:52.815 INFO:teuthology.orchestra.run.smithi148.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-24T23:16:52.815 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-24T23:16:52.815 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-24T23:16:52.815 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-24T23:16:52.815 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-24T23:16:52.815 INFO:teuthology.orchestra.run.smithi148.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-24T23:16:52.815 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-24T23:16:52.815 INFO:teuthology.orchestra.run.smithi148.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-24T23:16:52.815 INFO:teuthology.orchestra.run.smithi148.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-24T23:16:52.815 INFO:teuthology.orchestra.run.smithi148.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-24T23:16:52.815 INFO:teuthology.orchestra.run.smithi148.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-24T23:16:52.815 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-24T23:16:52.815 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-24T23:16:52.816 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-24T23:16:52.816 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-24T23:16:52.816 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-24T23:16:52.816 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-24T23:16:52.816 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-24T23:16:52.816 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-24T23:16:52.816 INFO:teuthology.orchestra.run.smithi148.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-24T23:16:52.816 INFO:teuthology.orchestra.run.smithi148.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-24T23:16:52.816 INFO:teuthology.orchestra.run.smithi148.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-24T23:16:52.816 INFO:teuthology.orchestra.run.smithi148.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-24T23:16:52.816 INFO:teuthology.orchestra.run.smithi148.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-24T23:16:52.816 INFO:teuthology.orchestra.run.smithi148.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-24T23:16:52.817 INFO:teuthology.orchestra.run.smithi148.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-24T23:16:52.817 INFO:teuthology.orchestra.run.smithi148.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-24T23:16:52.817 INFO:teuthology.orchestra.run.smithi148.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-24T23:16:52.817 INFO:teuthology.orchestra.run.smithi148.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-24T23:16:52.817 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-24T23:16:52.817 INFO:teuthology.orchestra.run.smithi148.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-24T23:16:52.817 INFO:teuthology.orchestra.run.smithi148.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-24T23:16:52.817 INFO:teuthology.orchestra.run.smithi148.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-24T23:16:52.817 INFO:teuthology.orchestra.run.smithi148.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-24T23:16:52.817 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-24T23:16:52.817 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-24T23:16:52.818 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-24T23:16:52.818 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-24T23:16:52.818 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-24T23:16:52.818 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-24T23:16:52.818 INFO:teuthology.orchestra.run.smithi148.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-24T23:16:52.818 INFO:teuthology.orchestra.run.smithi148.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-24T23:16:52.818 INFO:teuthology.orchestra.run.smithi148.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-24T23:16:52.818 INFO:teuthology.orchestra.run.smithi148.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-24T23:16:52.818 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-24T23:16:52.818 INFO:teuthology.orchestra.run.smithi148.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-24T23:16:52.818 INFO:teuthology.orchestra.run.smithi148.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-24T23:16:52.818 INFO:teuthology.orchestra.run.smithi148.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-24T23:16:52.819 INFO:teuthology.orchestra.run.smithi148.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-24T23:16:52.819 INFO:teuthology.orchestra.run.smithi148.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-24T23:16:52.819 INFO:teuthology.orchestra.run.smithi148.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-24T23:16:52.819 INFO:teuthology.orchestra.run.smithi148.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-24T23:16:52.819 INFO:teuthology.orchestra.run.smithi148.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-24T23:16:52.819 INFO:teuthology.orchestra.run.smithi148.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-24T23:16:52.819 INFO:teuthology.orchestra.run.smithi148.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-24T23:16:52.819 INFO:teuthology.orchestra.run.smithi148.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-24T23:16:52.819 INFO:teuthology.orchestra.run.smithi148.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-24T23:16:52.819 INFO:teuthology.orchestra.run.smithi148.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-24T23:16:52.819 INFO:teuthology.orchestra.run.smithi148.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-24T23:16:52.819 INFO:teuthology.orchestra.run.smithi148.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-24T23:16:52.819 INFO:teuthology.orchestra.run.smithi148.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-24T23:16:52.820 INFO:teuthology.orchestra.run.smithi148.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-24T23:16:52.820 INFO:teuthology.orchestra.run.smithi148.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-24T23:16:52.820 INFO:teuthology.orchestra.run.smithi148.stdout:Installing weak dependencies: 2024-04-24T23:16:52.820 INFO:teuthology.orchestra.run.smithi148.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-24T23:16:52.820 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-24T23:16:52.820 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-04-24T23:16:52.820 INFO:teuthology.orchestra.run.smithi148.stdout:======================================================================================= 2024-04-24T23:16:52.820 INFO:teuthology.orchestra.run.smithi148.stdout:Install 148 Packages 2024-04-24T23:16:52.820 INFO:teuthology.orchestra.run.smithi148.stdout:Upgrade 4 Packages 2024-04-24T23:16:52.820 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-24T23:16:52.821 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 220 M 2024-04-24T23:16:52.821 INFO:teuthology.orchestra.run.smithi148.stdout:Downloading Packages: 2024-04-24T23:16:52.830 INFO:teuthology.orchestra.run.smithi080.stdout:(5/152): ceph-fuse-19.0.0-2445.gd2c45d97.el9.x8 1.3 MB/s | 890 kB 00:00 2024-04-24T23:16:52.905 INFO:teuthology.orchestra.run.smithi080.stdout:(6/152): ceph-mgr-19.0.0-2445.gd2c45d97.el9.x86 11 MB/s | 1.5 MB 00:00 2024-04-24T23:16:53.810 INFO:teuthology.orchestra.run.smithi134.stdout:(1/152): ceph-19.0.0-2445.gd2c45d97.el9.x86_64. 22 kB/s | 6.6 kB 00:00 2024-04-24T23:16:54.085 INFO:teuthology.orchestra.run.smithi134.stdout:(2/152): ceph-fuse-19.0.0-2445.gd2c45d97.el9.x8 3.2 MB/s | 890 kB 00:00 2024-04-24T23:16:54.106 INFO:teuthology.orchestra.run.smithi080.stdout:(7/152): ceph-osd-19.0.0-2445.gd2c45d97.el9.x86 14 MB/s | 17 MB 00:01 2024-04-24T23:16:54.113 INFO:teuthology.orchestra.run.smithi148.stdout:(1/152): ceph-19.0.0-2445.gd2c45d97.el9.x86_64. 23 kB/s | 6.6 kB 00:00 2024-04-24T23:16:54.151 INFO:teuthology.orchestra.run.smithi134.stdout:(3/152): ceph-immutable-object-cache-19.0.0-244 2.1 MB/s | 145 kB 00:00 2024-04-24T23:16:54.314 INFO:teuthology.orchestra.run.smithi080.stdout:(8/152): ceph-common-19.0.0-2445.gd2c45d97.el9. 8.8 MB/s | 21 MB 00:02 2024-04-24T23:16:54.431 INFO:teuthology.orchestra.run.smithi080.stdout:(9/152): ceph-selinux-19.0.0-2445.gd2c45d97.el9 216 kB/s | 25 kB 00:00 2024-04-24T23:16:54.840 INFO:teuthology.orchestra.run.smithi134.stdout:(4/152): ceph-mds-19.0.0-2445.gd2c45d97.el9.x86 3.4 MB/s | 2.4 MB 00:00 2024-04-24T23:16:54.927 INFO:teuthology.orchestra.run.smithi134.stdout:(5/152): ceph-base-19.0.0-2445.gd2c45d97.el9.x8 3.8 MB/s | 5.4 MB 00:01 2024-04-24T23:16:55.210 INFO:teuthology.orchestra.run.smithi134.stdout:(6/152): ceph-mgr-19.0.0-2445.gd2c45d97.el9.x86 4.0 MB/s | 1.5 MB 00:00 2024-04-24T23:16:55.264 INFO:teuthology.orchestra.run.smithi148.stdout:(2/152): ceph-fuse-19.0.0-2445.gd2c45d97.el9.x8 774 kB/s | 890 kB 00:01 2024-04-24T23:16:55.464 INFO:teuthology.orchestra.run.smithi148.stdout:(3/152): ceph-immutable-object-cache-19.0.0-244 725 kB/s | 145 kB 00:00 2024-04-24T23:16:55.865 INFO:teuthology.orchestra.run.smithi080.stdout:(10/152): ceph-radosgw-19.0.0-2445.gd2c45d97.el 5.8 MB/s | 10 MB 00:01 2024-04-24T23:16:55.939 INFO:teuthology.orchestra.run.smithi148.stdout:(4/152): ceph-base-19.0.0-2445.gd2c45d97.el9.x8 2.5 MB/s | 5.4 MB 00:02 2024-04-24T23:16:56.031 INFO:teuthology.orchestra.run.smithi080.stdout:(11/152): ceph-mon-19.0.0-2445.gd2c45d97.el9.x8 1.5 MB/s | 4.7 MB 00:03 2024-04-24T23:16:56.056 INFO:teuthology.orchestra.run.smithi080.stdout:(12/152): cephfs-mirror-19.0.0-2445.gd2c45d97.e 1.1 MB/s | 222 kB 00:00 2024-04-24T23:16:56.124 INFO:teuthology.orchestra.run.smithi080.stdout:(13/152): cephfs-shell-19.0.0-2445.gd2c45d97.el 294 kB/s | 27 kB 00:00 2024-04-24T23:16:56.149 INFO:teuthology.orchestra.run.smithi080.stdout:(14/152): libcephfs-devel-19.0.0-2445.gd2c45d97 343 kB/s | 32 kB 00:00 2024-04-24T23:16:56.241 INFO:teuthology.orchestra.run.smithi080.stdout:(15/152): libcephsqlite-19.0.0-2445.gd2c45d97.e 1.7 MB/s | 163 kB 00:00 2024-04-24T23:16:56.341 INFO:teuthology.orchestra.run.smithi080.stdout:(16/152): librados-devel-19.0.0-2445.gd2c45d97. 1.2 MB/s | 127 kB 00:00 2024-04-24T23:16:56.473 INFO:teuthology.orchestra.run.smithi148.stdout:(5/152): ceph-mgr-19.0.0-2445.gd2c45d97.el9.x86 2.8 MB/s | 1.5 MB 00:00 2024-04-24T23:16:56.559 INFO:teuthology.orchestra.run.smithi080.stdout:(17/152): libradosstriper1-19.0.0-2445.gd2c45d9 2.3 MB/s | 501 kB 00:00 2024-04-24T23:16:56.651 INFO:teuthology.orchestra.run.smithi080.stdout:(18/152): libcephfs2-19.0.0-2445.gd2c45d97.el9. 1.4 MB/s | 745 kB 00:00 2024-04-24T23:16:56.711 INFO:teuthology.orchestra.run.smithi134.stdout:(7/152): ceph-mon-19.0.0-2445.gd2c45d97.el9.x86 2.6 MB/s | 4.7 MB 00:01 2024-04-24T23:16:56.751 INFO:teuthology.orchestra.run.smithi080.stdout:(19/152): python3-ceph-argparse-19.0.0-2445.gd2 451 kB/s | 45 kB 00:00 2024-04-24T23:16:56.868 INFO:teuthology.orchestra.run.smithi080.stdout:(20/152): python3-ceph-common-19.0.0-2445.gd2c4 1.1 MB/s | 133 kB 00:00 2024-04-24T23:16:57.068 INFO:teuthology.orchestra.run.smithi080.stdout:(21/152): python3-cephfs-19.0.0-2445.gd2c45d97. 810 kB/s | 162 kB 00:00 2024-04-24T23:16:57.352 INFO:teuthology.orchestra.run.smithi080.stdout:(22/152): python3-rados-19.0.0-2445.gd2c45d97.e 1.1 MB/s | 321 kB 00:00 2024-04-24T23:16:57.652 INFO:teuthology.orchestra.run.smithi080.stdout:(23/152): python3-rbd-19.0.0-2445.gd2c45d97.el9 997 kB/s | 299 kB 00:00 2024-04-24T23:16:57.727 INFO:teuthology.orchestra.run.smithi080.stdout:(24/152): librgw2-19.0.0-2445.gd2c45d97.el9.x86 4.5 MB/s | 5.3 MB 00:01 2024-04-24T23:16:57.753 INFO:teuthology.orchestra.run.smithi080.stdout:(25/152): python3-rgw-19.0.0-2445.gd2c45d97.el9 993 kB/s | 99 kB 00:00 2024-04-24T23:16:57.811 INFO:teuthology.orchestra.run.smithi080.stdout:(26/152): rbd-fuse-19.0.0-2445.gd2c45d97.el9.x8 1.0 MB/s | 85 kB 00:00 2024-04-24T23:16:57.898 INFO:teuthology.orchestra.run.smithi148.stdout:(6/152): ceph-mds-19.0.0-2445.gd2c45d97.el9.x86 994 kB/s | 2.4 MB 00:02 2024-04-24T23:16:57.903 INFO:teuthology.orchestra.run.smithi080.stdout:(27/152): rbd-nbd-19.0.0-2445.gd2c45d97.el9.x86 1.8 MB/s | 171 kB 00:00 2024-04-24T23:16:58.003 INFO:teuthology.orchestra.run.smithi080.stdout:(28/152): ceph-grafana-dashboards-19.0.0-2445.g 317 kB/s | 32 kB 00:00 2024-04-24T23:16:58.104 INFO:teuthology.orchestra.run.smithi080.stdout:(29/152): ceph-mgr-cephadm-19.0.0-2445.gd2c45d9 1.4 MB/s | 141 kB 00:00 2024-04-24T23:16:58.294 INFO:teuthology.orchestra.run.smithi134.stdout:(8/152): ceph-osd-19.0.0-2445.gd2c45d97.el9.x86 5.4 MB/s | 17 MB 00:03 2024-04-24T23:16:58.373 INFO:teuthology.orchestra.run.smithi148.stdout:(7/152): ceph-mon-19.0.0-2445.gd2c45d97.el9.x86 2.5 MB/s | 4.7 MB 00:01 2024-04-24T23:16:58.386 INFO:teuthology.orchestra.run.smithi134.stdout:(9/152): ceph-selinux-19.0.0-2445.gd2c45d97.el9 275 kB/s | 25 kB 00:00 2024-04-24T23:16:58.796 INFO:teuthology.orchestra.run.smithi080.stdout:(30/152): ceph-mgr-dashboard-19.0.0-2445.gd2c45 5.1 MB/s | 3.5 MB 00:00 2024-04-24T23:17:00.078 INFO:teuthology.orchestra.run.smithi134.stdout:(10/152): ceph-common-19.0.0-2445.gd2c45d97.el9 3.2 MB/s | 21 MB 00:06 2024-04-24T23:17:00.221 INFO:teuthology.orchestra.run.smithi080.stdout:(31/152): rbd-mirror-19.0.0-2445.gd2c45d97.el9. 1.3 MB/s | 3.1 MB 00:02 2024-04-24T23:17:00.236 INFO:teuthology.orchestra.run.smithi134.stdout:(11/152): cephfs-mirror-19.0.0-2445.gd2c45d97.e 1.4 MB/s | 222 kB 00:00 2024-04-24T23:17:00.322 INFO:teuthology.orchestra.run.smithi134.stdout:(12/152): cephfs-shell-19.0.0-2445.gd2c45d97.el 317 kB/s | 27 kB 00:00 2024-04-24T23:17:00.329 INFO:teuthology.orchestra.run.smithi080.stdout:(32/152): ceph-mgr-diskprediction-local-19.0.0- 4.8 MB/s | 7.4 MB 00:01 2024-04-24T23:17:00.396 INFO:teuthology.orchestra.run.smithi080.stdout:(33/152): ceph-mgr-rook-19.0.0-2445.gd2c45d97.e 742 kB/s | 49 kB 00:00 2024-04-24T23:17:00.397 INFO:teuthology.orchestra.run.smithi134.stdout:(13/152): libcephfs-devel-19.0.0-2445.gd2c45d97 423 kB/s | 32 kB 00:00 2024-04-24T23:17:00.438 INFO:teuthology.orchestra.run.smithi080.stdout:(34/152): ceph-mgr-modules-core-19.0.0-2445.gd2 1.1 MB/s | 249 kB 00:00 2024-04-24T23:17:00.463 INFO:teuthology.orchestra.run.smithi080.stdout:(35/152): ceph-prometheus-alerts-19.0.0-2445.gd 229 kB/s | 15 kB 00:00 2024-04-24T23:17:00.465 INFO:teuthology.orchestra.run.smithi148.stdout:(8/152): ceph-common-19.0.0-2445.gd2c45d97.el9. 3.2 MB/s | 21 MB 00:06 2024-04-24T23:17:00.615 INFO:teuthology.orchestra.run.smithi148.stdout:(9/152): ceph-selinux-19.0.0-2445.gd2c45d97.el9 168 kB/s | 25 kB 00:00 2024-04-24T23:17:00.630 INFO:teuthology.orchestra.run.smithi080.stdout:(36/152): cephadm-19.0.0-2445.gd2c45d97.el9.noa 4.5 MB/s | 762 kB 00:00 2024-04-24T23:17:00.672 INFO:teuthology.orchestra.run.smithi080.stdout:(37/152): ceph-volume-19.0.0-2445.gd2c45d97.el9 1.1 MB/s | 264 kB 00:00 2024-04-24T23:17:00.689 INFO:teuthology.orchestra.run.smithi134.stdout:(14/152): libcephfs2-19.0.0-2445.gd2c45d97.el9. 2.5 MB/s | 745 kB 00:00 2024-04-24T23:17:00.706 INFO:teuthology.orchestra.run.smithi080.stdout:(38/152): cephfs-top-19.0.0-2445.gd2c45d97.el9. 242 kB/s | 18 kB 00:00 2024-04-24T23:17:00.823 INFO:teuthology.orchestra.run.smithi134.stdout:(15/152): ceph-radosgw-19.0.0-2445.gd2c45d97.el 2.5 MB/s | 10 MB 00:04 2024-04-24T23:17:00.848 INFO:teuthology.orchestra.run.smithi134.stdout:(16/152): libcephsqlite-19.0.0-2445.gd2c45d97.e 1.0 MB/s | 163 kB 00:00 2024-04-24T23:17:00.913 INFO:teuthology.orchestra.run.smithi080.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 401 kB/s | 80 kB 00:00 2024-04-24T23:17:00.915 INFO:teuthology.orchestra.run.smithi134.stdout:(17/152): librados-devel-19.0.0-2445.gd2c45d97. 1.4 MB/s | 127 kB 00:00 2024-04-24T23:17:00.999 INFO:teuthology.orchestra.run.smithi080.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 1.5 MB/s | 195 kB 00:00 2024-04-24T23:17:01.024 INFO:teuthology.orchestra.run.smithi080.stdout:(41/152): libgfortran-11.4.1-3.el9.x86_64.rpm 2.5 MB/s | 803 kB 00:00 2024-04-24T23:17:01.082 INFO:teuthology.orchestra.run.smithi080.stdout:(42/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.0 MB/s | 253 kB 00:00 2024-04-24T23:17:01.107 INFO:teuthology.orchestra.run.smithi134.stdout:(18/152): libradosstriper1-19.0.0-2445.gd2c45d9 1.9 MB/s | 501 kB 00:00 2024-04-24T23:17:01.141 INFO:teuthology.orchestra.run.smithi080.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-04-24T23:17:01.199 INFO:teuthology.orchestra.run.smithi134.stdout:(19/152): python3-ceph-argparse-19.0.0-2445.gd2 493 kB/s | 45 kB 00:00 2024-04-24T23:17:01.200 INFO:teuthology.orchestra.run.smithi080.stdout:(44/152): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-04-24T23:17:01.241 INFO:teuthology.orchestra.run.smithi080.stdout:(45/152): python3-cryptography-36.0.1-4.el9.x86 5.6 MB/s | 1.2 MB 00:00 2024-04-24T23:17:01.267 INFO:teuthology.orchestra.run.smithi080.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 1.8 MB/s | 125 kB 00:00 2024-04-24T23:17:01.299 INFO:teuthology.orchestra.run.smithi134.stdout:(20/152): python3-ceph-common-19.0.0-2445.gd2c4 1.3 MB/s | 133 kB 00:00 2024-04-24T23:17:01.300 INFO:teuthology.orchestra.run.smithi080.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 3.6 MB/s | 215 kB 00:00 2024-04-24T23:17:01.391 INFO:teuthology.orchestra.run.smithi134.stdout:(21/152): python3-cephfs-19.0.0-2445.gd2c45d97. 1.7 MB/s | 162 kB 00:00 2024-04-24T23:17:01.459 INFO:teuthology.orchestra.run.smithi080.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 3.5 MB/s | 689 kB 00:00 2024-04-24T23:17:01.518 INFO:teuthology.orchestra.run.smithi080.stdout:(49/152): bison-3.7.4-5.el9.x86_64.rpm 4.2 MB/s | 943 kB 00:00 2024-04-24T23:17:01.543 INFO:teuthology.orchestra.run.smithi080.stdout:(50/152): boost-program-options-1.75.0-8.el9.x8 1.3 MB/s | 107 kB 00:00 2024-04-24T23:17:01.560 INFO:teuthology.orchestra.run.smithi080.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 515 kB/s | 21 kB 00:00 2024-04-24T23:17:01.566 INFO:teuthology.orchestra.run.smithi134.stdout:(22/152): python3-rados-19.0.0-2445.gd2c45d97.e 1.8 MB/s | 321 kB 00:00 2024-04-24T23:17:01.594 INFO:teuthology.orchestra.run.smithi080.stdout:(52/152): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-04-24T23:17:01.619 INFO:teuthology.orchestra.run.smithi080.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 560 kB/s | 33 kB 00:00 2024-04-24T23:17:01.669 INFO:teuthology.orchestra.run.smithi080.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 364 kB/s | 18 kB 00:00 2024-04-24T23:17:01.728 INFO:teuthology.orchestra.run.smithi080.stdout:(55/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 22 MB/s | 3.0 MB 00:00 2024-04-24T23:17:01.733 INFO:teuthology.orchestra.run.smithi134.stdout:(23/152): python3-rbd-19.0.0-2445.gd2c45d97.el9 1.8 MB/s | 299 kB 00:00 2024-04-24T23:17:01.753 INFO:teuthology.orchestra.run.smithi080.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 133 kB/s | 11 kB 00:00 2024-04-24T23:17:01.779 INFO:teuthology.orchestra.run.smithi080.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-04-24T23:17:01.804 INFO:teuthology.orchestra.run.smithi080.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 902 kB/s | 45 kB 00:00 2024-04-24T23:17:01.815 INFO:teuthology.orchestra.run.smithi148.stdout:(10/152): ceph-radosgw-19.0.0-2445.gd2c45d97.el 3.0 MB/s | 10 MB 00:03 2024-04-24T23:17:01.833 INFO:teuthology.orchestra.run.smithi134.stdout:(24/152): python3-rgw-19.0.0-2445.gd2c45d97.el9 996 kB/s | 99 kB 00:00 2024-04-24T23:17:01.837 INFO:teuthology.orchestra.run.smithi080.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 662 kB 00:00 2024-04-24T23:17:01.871 INFO:teuthology.orchestra.run.smithi080.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 8.5 MB/s | 582 kB 00:00 2024-04-24T23:17:01.896 INFO:teuthology.orchestra.run.smithi080.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 434 kB/s | 25 kB 00:00 2024-04-24T23:17:01.921 INFO:teuthology.orchestra.run.smithi080.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 946 kB/s | 47 kB 00:00 2024-04-24T23:17:01.923 INFO:teuthology.orchestra.run.smithi148.stdout:(11/152): cephfs-mirror-19.0.0-2445.gd2c45d97.e 2.0 MB/s | 222 kB 00:00 2024-04-24T23:17:01.933 INFO:teuthology.orchestra.run.smithi134.stdout:(25/152): rbd-fuse-19.0.0-2445.gd2c45d97.el9.x8 854 kB/s | 85 kB 00:00 2024-04-24T23:17:01.947 INFO:teuthology.orchestra.run.smithi080.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-04-24T23:17:01.972 INFO:teuthology.orchestra.run.smithi080.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-04-24T23:17:01.974 INFO:teuthology.orchestra.run.smithi148.stdout:(12/152): cephfs-shell-19.0.0-2445.gd2c45d97.el 544 kB/s | 27 kB 00:00 2024-04-24T23:17:01.997 INFO:teuthology.orchestra.run.smithi080.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 700 kB/s | 35 kB 00:00 2024-04-24T23:17:02.049 INFO:teuthology.orchestra.run.smithi148.stdout:(13/152): libcephfs-devel-19.0.0-2445.gd2c45d97 423 kB/s | 32 kB 00:00 2024-04-24T23:17:02.165 INFO:teuthology.orchestra.run.smithi080.stdout:(66/152): openblas-openmp-0.3.21-2.el9.x86_64.r 25 MB/s | 4.7 MB 00:00 2024-04-24T23:17:02.232 INFO:teuthology.orchestra.run.smithi080.stdout:(67/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 19 MB/s | 4.4 MB 00:00 2024-04-24T23:17:02.273 INFO:teuthology.orchestra.run.smithi080.stdout:(68/152): postgresql-13.14-1.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-04-24T23:17:02.307 INFO:teuthology.orchestra.run.smithi080.stdout:(69/152): postgresql-contrib-13.14-1.el9.x86_64 12 MB/s | 911 kB 00:00 2024-04-24T23:17:02.316 INFO:teuthology.orchestra.run.smithi148.stdout:(14/152): libcephfs2-19.0.0-2445.gd2c45d97.el9. 2.7 MB/s | 745 kB 00:00 2024-04-24T23:17:02.332 INFO:teuthology.orchestra.run.smithi080.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 2.2 MB/s | 133 kB 00:00 2024-04-24T23:17:02.416 INFO:teuthology.orchestra.run.smithi080.stdout:(71/152): protobuf-3.14.0-13.el9.x86_64.rpm 12 MB/s | 1.0 MB 00:00 2024-04-24T23:17:02.417 INFO:teuthology.orchestra.run.smithi148.stdout:(15/152): libcephsqlite-19.0.0-2445.gd2c45d97.e 1.6 MB/s | 163 kB 00:00 2024-04-24T23:17:02.500 INFO:teuthology.orchestra.run.smithi080.stdout:(72/152): postgresql-server-13.14-1.el9.x86_64. 30 MB/s | 5.8 MB 00:00 2024-04-24T23:17:02.500 INFO:teuthology.orchestra.run.smithi148.stdout:(16/152): librados-devel-19.0.0-2445.gd2c45d97. 1.5 MB/s | 127 kB 00:00 2024-04-24T23:17:02.525 INFO:teuthology.orchestra.run.smithi080.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 806 kB/s | 87 kB 00:00 2024-04-24T23:17:02.583 INFO:teuthology.orchestra.run.smithi080.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 4.1 MB/s | 244 kB 00:00 2024-04-24T23:17:02.676 INFO:teuthology.orchestra.run.smithi080.stdout:(75/152): python3-babel-2.9.1-2.el9.noarch.rpm 34 MB/s | 6.0 MB 00:00 2024-04-24T23:17:02.700 INFO:teuthology.orchestra.run.smithi148.stdout:(17/152): libradosstriper1-19.0.0-2445.gd2c45d9 2.4 MB/s | 501 kB 00:00 2024-04-24T23:17:02.701 INFO:teuthology.orchestra.run.smithi080.stdout:(76/152): python3-jinja2-2.11.3-5.el9.noarch.rp 2.1 MB/s | 248 kB 00:00 2024-04-24T23:17:02.726 INFO:teuthology.orchestra.run.smithi080.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 513 kB/s | 26 kB 00:00 2024-04-24T23:17:02.742 INFO:teuthology.orchestra.run.smithi134.stdout:(26/152): librgw2-19.0.0-2445.gd2c45d97.el9.x86 2.9 MB/s | 5.3 MB 00:01 2024-04-24T23:17:02.743 INFO:teuthology.orchestra.run.smithi080.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 454 kB/s | 19 kB 00:00 2024-04-24T23:17:02.777 INFO:teuthology.orchestra.run.smithi080.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 3.3 MB/s | 172 kB 00:00 2024-04-24T23:17:02.794 INFO:teuthology.orchestra.run.smithi080.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 691 kB/s | 35 kB 00:00 2024-04-24T23:17:02.861 INFO:teuthology.orchestra.run.smithi080.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 3.7 MB/s | 250 kB 00:00 2024-04-24T23:17:02.867 INFO:teuthology.orchestra.run.smithi134.stdout:(27/152): rbd-nbd-19.0.0-2445.gd2c45d97.el9.x86 1.3 MB/s | 171 kB 00:00 2024-04-24T23:17:02.928 INFO:teuthology.orchestra.run.smithi080.stdout:(82/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 34 MB/s | 5.1 MB 00:00 2024-04-24T23:17:02.953 INFO:teuthology.orchestra.run.smithi080.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 2.4 MB/s | 222 kB 00:00 2024-04-24T23:17:02.968 INFO:teuthology.orchestra.run.smithi134.stdout:(28/152): ceph-grafana-dashboards-19.0.0-2445.g 317 kB/s | 32 kB 00:00 2024-04-24T23:17:02.970 INFO:teuthology.orchestra.run.smithi080.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 1.8 MB/s | 77 kB 00:00 2024-04-24T23:17:02.995 INFO:teuthology.orchestra.run.smithi080.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 1.0 MB/s | 42 kB 00:00 2024-04-24T23:17:03.029 INFO:teuthology.orchestra.run.smithi080.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 4.5 MB/s | 269 kB 00:00 2024-04-24T23:17:03.054 INFO:teuthology.orchestra.run.smithi080.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 159 kB 00:00 2024-04-24T23:17:03.068 INFO:teuthology.orchestra.run.smithi134.stdout:(29/152): rbd-mirror-19.0.0-2445.gd2c45d97.el9. 2.7 MB/s | 3.1 MB 00:01 2024-04-24T23:17:03.079 INFO:teuthology.orchestra.run.smithi080.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-04-24T23:17:03.093 INFO:teuthology.orchestra.run.smithi134.stdout:(30/152): ceph-mgr-cephadm-19.0.0-2445.gd2c45d9 1.1 MB/s | 141 kB 00:00 2024-04-24T23:17:03.105 INFO:teuthology.orchestra.run.smithi080.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-04-24T23:17:03.130 INFO:teuthology.orchestra.run.smithi080.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-04-24T23:17:03.564 INFO:teuthology.orchestra.run.smithi080.stdout:(91/152): ceph-test-19.0.0-2445.gd2c45d97.el9.x 5.3 MB/s | 48 MB 00:09 2024-04-24T23:17:03.589 INFO:teuthology.orchestra.run.smithi080.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 91 kB/s | 42 kB 00:00 2024-04-24T23:17:03.665 INFO:teuthology.orchestra.run.smithi080.stdout:(93/152): qatlib-24.02.0-1.el9.x86_64.rpm 2.9 MB/s | 221 kB 00:00 2024-04-24T23:17:03.690 INFO:teuthology.orchestra.run.smithi080.stdout:(94/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 352 kB/s | 44 kB 00:00 2024-04-24T23:17:03.723 INFO:teuthology.orchestra.run.smithi080.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 615 kB/s | 36 kB 00:00 2024-04-24T23:17:03.765 INFO:teuthology.orchestra.run.smithi080.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 631 kB/s | 47 kB 00:00 2024-04-24T23:17:03.790 INFO:teuthology.orchestra.run.smithi080.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 4.4 MB/s | 305 kB 00:00 2024-04-24T23:17:03.816 INFO:teuthology.orchestra.run.smithi080.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-04-24T23:17:03.841 INFO:teuthology.orchestra.run.smithi080.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.8 MB/s | 94 kB 00:00 2024-04-24T23:17:03.867 INFO:teuthology.orchestra.run.smithi080.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-04-24T23:17:03.892 INFO:teuthology.orchestra.run.smithi080.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 893 kB/s | 45 kB 00:00 2024-04-24T23:17:04.092 INFO:teuthology.orchestra.run.smithi080.stdout:(102/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 16 MB/s | 16 MB 00:00 2024-04-24T23:17:04.109 INFO:teuthology.orchestra.run.smithi148.stdout:(18/152): librgw2-19.0.0-2445.gd2c45d97.el9.x86 3.8 MB/s | 5.3 MB 00:01 2024-04-24T23:17:04.110 INFO:teuthology.orchestra.run.smithi134.stdout:(31/152): ceph-mgr-dashboard-19.0.0-2445.gd2c45 3.4 MB/s | 3.5 MB 00:01 2024-04-24T23:17:04.117 INFO:teuthology.orchestra.run.smithi080.stdout:(103/152): lua-devel-5.4.4-4.el9.x86_64.rpm 89 kB/s | 22 kB 00:00 2024-04-24T23:17:04.159 INFO:teuthology.orchestra.run.smithi148.stdout:(19/152): python3-ceph-argparse-19.0.0-2445.gd2 911 kB/s | 45 kB 00:00 2024-04-24T23:17:04.226 INFO:teuthology.orchestra.run.smithi080.stdout:(104/152): protobuf-compiler-3.14.0-13.el9.x86_ 2.5 MB/s | 863 kB 00:00 2024-04-24T23:17:04.227 INFO:teuthology.orchestra.run.smithi134.stdout:(32/152): ceph-mgr-modules-core-19.0.0-2445.gd2 2.1 MB/s | 249 kB 00:00 2024-04-24T23:17:04.259 INFO:teuthology.orchestra.run.smithi148.stdout:(20/152): python3-ceph-common-19.0.0-2445.gd2c4 1.3 MB/s | 133 kB 00:00 2024-04-24T23:17:04.294 INFO:teuthology.orchestra.run.smithi134.stdout:(33/152): ceph-mgr-rook-19.0.0-2445.gd2c45d97.e 741 kB/s | 49 kB 00:00 2024-04-24T23:17:04.326 INFO:teuthology.orchestra.run.smithi148.stdout:(21/152): python3-cephfs-19.0.0-2445.gd2c45d97. 2.4 MB/s | 162 kB 00:00 2024-04-24T23:17:04.361 INFO:teuthology.orchestra.run.smithi134.stdout:(34/152): ceph-prometheus-alerts-19.0.0-2445.gd 231 kB/s | 15 kB 00:00 2024-04-24T23:17:04.451 INFO:teuthology.orchestra.run.smithi148.stdout:(22/152): python3-rados-19.0.0-2445.gd2c45d97.e 2.5 MB/s | 321 kB 00:00 2024-04-24T23:17:04.493 INFO:teuthology.orchestra.run.smithi080.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 73 kB/s | 19 kB 00:00 2024-04-24T23:17:04.503 INFO:teuthology.orchestra.run.smithi134.stdout:(35/152): ceph-volume-19.0.0-2445.gd2c45d97.el9 1.8 MB/s | 264 kB 00:00 2024-04-24T23:17:04.560 INFO:teuthology.orchestra.run.smithi148.stdout:(23/152): python3-rbd-19.0.0-2445.gd2c45d97.el9 2.7 MB/s | 299 kB 00:00 2024-04-24T23:17:04.626 INFO:teuthology.orchestra.run.smithi080.stdout:(106/152): gperftools-libs-2.9.1-2.el9.x86_64.r 607 kB/s | 309 kB 00:00 2024-04-24T23:17:04.627 INFO:teuthology.orchestra.run.smithi148.stdout:(24/152): python3-rgw-19.0.0-2445.gd2c45d97.el9 1.4 MB/s | 99 kB 00:00 2024-04-24T23:17:04.678 INFO:teuthology.orchestra.run.smithi148.stdout:(25/152): rbd-fuse-19.0.0-2445.gd2c45d97.el9.x8 1.7 MB/s | 85 kB 00:00 2024-04-24T23:17:04.702 INFO:teuthology.orchestra.run.smithi080.stdout:(107/152): abseil-cpp-20211102.0-3.el9.x86_64.r 900 kB/s | 548 kB 00:00 2024-04-24T23:17:04.894 INFO:teuthology.orchestra.run.smithi080.stdout:(108/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 97 kB/s | 26 kB 00:00 2024-04-24T23:17:05.029 INFO:teuthology.orchestra.run.smithi134.stdout:(36/152): ceph-test-19.0.0-2445.gd2c45d97.el9.x 7.3 MB/s | 48 MB 00:06 2024-04-24T23:17:05.052 INFO:teuthology.orchestra.run.smithi080.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 139 kB/s | 49 kB 00:00 2024-04-24T23:17:05.062 INFO:teuthology.orchestra.run.smithi134.stdout:(37/152): cephadm-19.0.0-2445.gd2c45d97.el9.noa 1.3 MB/s | 762 kB 00:00 2024-04-24T23:17:05.086 INFO:teuthology.orchestra.run.smithi148.stdout:(26/152): rbd-mirror-19.0.0-2445.gd2c45d97.el9. 7.6 MB/s | 3.1 MB 00:00 2024-04-24T23:17:05.087 INFO:teuthology.orchestra.run.smithi134.stdout:(38/152): cephfs-top-19.0.0-2445.gd2c45d97.el9. 314 kB/s | 18 kB 00:00 2024-04-24T23:17:05.137 INFO:teuthology.orchestra.run.smithi148.stdout:(27/152): rbd-nbd-19.0.0-2445.gd2c45d97.el9.x86 3.4 MB/s | 171 kB 00:00 2024-04-24T23:17:05.187 INFO:teuthology.orchestra.run.smithi148.stdout:(28/152): ceph-grafana-dashboards-19.0.0-2445.g 637 kB/s | 32 kB 00:00 2024-04-24T23:17:05.187 INFO:teuthology.orchestra.run.smithi134.stdout:(39/152): ceph-mgr-diskprediction-local-19.0.0- 3.5 MB/s | 7.4 MB 00:02 2024-04-24T23:17:05.236 INFO:teuthology.orchestra.run.smithi080.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 197 kB/s | 67 kB 00:00 2024-04-24T23:17:05.245 INFO:teuthology.orchestra.run.smithi148.stdout:(29/152): ceph-mgr-cephadm-19.0.0-2445.gd2c45d9 2.4 MB/s | 141 kB 00:00 2024-04-24T23:17:05.394 INFO:teuthology.orchestra.run.smithi080.stdout:(111/152): libarrow-9.0.0-10.el9.x86_64.rpm 4.9 MB/s | 4.4 MB 00:00 2024-04-24T23:17:05.478 INFO:teuthology.orchestra.run.smithi080.stdout:(112/152): luarocks-3.9.2-1.el9.noarch.rpm 355 kB/s | 151 kB 00:00 2024-04-24T23:17:05.530 INFO:teuthology.orchestra.run.smithi134.stdout:(40/152): fuse-2.9.9-15.el9.x86_64.rpm 172 kB/s | 80 kB 00:00 2024-04-24T23:17:05.662 INFO:teuthology.orchestra.run.smithi148.stdout:(30/152): ceph-mgr-dashboard-19.0.0-2445.gd2c45 8.4 MB/s | 3.5 MB 00:00 2024-04-24T23:17:05.697 INFO:teuthology.orchestra.run.smithi134.stdout:(41/152): libquadmath-11.4.1-3.el9.x86_64.rpm 384 kB/s | 195 kB 00:00 2024-04-24T23:17:05.797 INFO:teuthology.orchestra.run.smithi134.stdout:(42/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 949 kB/s | 253 kB 00:00 2024-04-24T23:17:05.828 INFO:teuthology.orchestra.run.smithi080.stdout:(113/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 124 kB/s | 43 kB 00:00 2024-04-24T23:17:05.906 INFO:teuthology.orchestra.run.smithi134.stdout:(43/152): libgfortran-11.4.1-3.el9.x86_64.rpm 982 kB/s | 803 kB 00:00 2024-04-24T23:17:05.912 INFO:teuthology.orchestra.run.smithi080.stdout:(114/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 1.2 MB/s | 838 kB 00:00 2024-04-24T23:17:05.931 INFO:teuthology.orchestra.run.smithi134.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 798 kB/s | 106 kB 00:00 2024-04-24T23:17:05.996 INFO:teuthology.orchestra.run.smithi148.stdout:(31/152): ceph-osd-19.0.0-2445.gd2c45d97.el9.x8 2.1 MB/s | 17 MB 00:08 2024-04-24T23:17:06.021 INFO:teuthology.orchestra.run.smithi080.stdout:(115/152): python3-asyncssh-2.13.2-5.el9.noarch 876 kB/s | 548 kB 00:00 2024-04-24T23:17:06.039 INFO:teuthology.orchestra.run.smithi134.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 1.0 MB/s | 135 kB 00:00 2024-04-24T23:17:06.064 INFO:teuthology.orchestra.run.smithi134.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 935 kB/s | 125 kB 00:00 2024-04-24T23:17:06.098 INFO:teuthology.orchestra.run.smithi134.stdout:(47/152): python3-cryptography-36.0.1-4.el9.x86 3.1 MB/s | 1.2 MB 00:00 2024-04-24T23:17:06.104 INFO:teuthology.orchestra.run.smithi148.stdout:(32/152): ceph-mgr-modules-core-19.0.0-2445.gd2 2.2 MB/s | 249 kB 00:00 2024-04-24T23:17:06.129 INFO:teuthology.orchestra.run.smithi080.stdout:(116/152): python3-cachetools-4.2.4-1.el9.noarc 107 kB/s | 32 kB 00:00 2024-04-24T23:17:06.148 INFO:teuthology.orchestra.run.smithi134.stdout:(48/152): python3-urllib3-1.26.5-5.el9.noarch.r 1.9 MB/s | 215 kB 00:00 2024-04-24T23:17:06.180 INFO:teuthology.orchestra.run.smithi148.stdout:(33/152): ceph-mgr-rook-19.0.0-2445.gd2c45d97.e 660 kB/s | 49 kB 00:00 2024-04-24T23:17:06.204 INFO:teuthology.orchestra.run.smithi080.stdout:(117/152): python3-certifi-2021.10.8-2.el9.noar 51 kB/s | 15 kB 00:00 2024-04-24T23:17:06.240 INFO:teuthology.orchestra.run.smithi134.stdout:(49/152): automake-1.16.2-8.el9.noarch.rpm 3.8 MB/s | 689 kB 00:00 2024-04-24T23:17:06.255 INFO:teuthology.orchestra.run.smithi148.stdout:(34/152): ceph-prometheus-alerts-19.0.0-2445.gd 204 kB/s | 15 kB 00:00 2024-04-24T23:17:06.265 INFO:teuthology.orchestra.run.smithi134.stdout:(50/152): boost-program-options-1.75.0-8.el9.x8 916 kB/s | 107 kB 00:00 2024-04-24T23:17:06.291 INFO:teuthology.orchestra.run.smithi134.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 429 kB/s | 21 kB 00:00 2024-04-24T23:17:06.333 INFO:teuthology.orchestra.run.smithi134.stdout:(52/152): bison-3.7.4-5.el9.x86_64.rpm 3.9 MB/s | 943 kB 00:00 2024-04-24T23:17:06.358 INFO:teuthology.orchestra.run.smithi134.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 490 kB/s | 33 kB 00:00 2024-04-24T23:17:06.372 INFO:teuthology.orchestra.run.smithi148.stdout:(35/152): ceph-volume-19.0.0-2445.gd2c45d97.el9 2.2 MB/s | 264 kB 00:00 2024-04-24T23:17:06.391 INFO:teuthology.orchestra.run.smithi134.stdout:(54/152): flex-2.6.4-9.el9.x86_64.rpm 2.4 MB/s | 313 kB 00:00 2024-04-24T23:17:06.417 INFO:teuthology.orchestra.run.smithi134.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 310 kB/s | 18 kB 00:00 2024-04-24T23:17:06.442 INFO:teuthology.orchestra.run.smithi134.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 221 kB/s | 11 kB 00:00 2024-04-24T23:17:06.467 INFO:teuthology.orchestra.run.smithi134.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-04-24T23:17:06.480 INFO:teuthology.orchestra.run.smithi080.stdout:(118/152): python3-cheroot-8.6.0-4.el9.noarch.r 374 kB/s | 172 kB 00:00 2024-04-24T23:17:06.492 INFO:teuthology.orchestra.run.smithi134.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 903 kB/s | 45 kB 00:00 2024-04-24T23:17:06.526 INFO:teuthology.orchestra.run.smithi134.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 662 kB 00:00 2024-04-24T23:17:06.581 INFO:teuthology.orchestra.run.smithi148.stdout:(36/152): cephadm-19.0.0-2445.gd2c45d97.el9.noa 3.6 MB/s | 762 kB 00:00 2024-04-24T23:17:06.585 INFO:teuthology.orchestra.run.smithi134.stdout:(60/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 12 MB/s | 3.0 MB 00:00 2024-04-24T23:17:06.618 INFO:teuthology.orchestra.run.smithi134.stdout:(61/152): libtool-2.4.6-45.el9.x86_64.rpm 4.5 MB/s | 582 kB 00:00 2024-04-24T23:17:06.644 INFO:teuthology.orchestra.run.smithi134.stdout:(62/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 217 kB/s | 25 kB 00:00 2024-04-24T23:17:06.669 INFO:teuthology.orchestra.run.smithi134.stdout:(63/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 567 kB/s | 47 kB 00:00 2024-04-24T23:17:06.673 INFO:teuthology.orchestra.run.smithi148.stdout:(37/152): ceph-mgr-diskprediction-local-19.0.0- 7.3 MB/s | 7.4 MB 00:01 2024-04-24T23:17:06.680 INFO:teuthology.orchestra.run.smithi080.stdout:(119/152): python3-cherrypy-18.6.1-2.el9.noarch 651 kB/s | 358 kB 00:00 2024-04-24T23:17:06.694 INFO:teuthology.orchestra.run.smithi134.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-04-24T23:17:06.698 INFO:teuthology.orchestra.run.smithi148.stdout:(38/152): cephfs-top-19.0.0-2445.gd2c45d97.el9. 156 kB/s | 18 kB 00:00 2024-04-24T23:17:06.713 INFO:teuthology.orchestra.run.smithi080.stdout:(120/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 596 kB/s | 303 kB 00:00 2024-04-24T23:17:06.719 INFO:teuthology.orchestra.run.smithi134.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 700 kB/s | 35 kB 00:00 2024-04-24T23:17:06.745 INFO:teuthology.orchestra.run.smithi134.stdout:(66/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.3 MB/s | 292 kB 00:00 2024-04-24T23:17:06.756 INFO:teuthology.orchestra.run.smithi080.stdout:(121/152): python3-colorama-0.4.4-12.el9.noarch 128 kB/s | 35 kB 00:00 2024-04-24T23:17:06.845 INFO:teuthology.orchestra.run.smithi134.stdout:(67/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 35 MB/s | 4.4 MB 00:00 2024-04-24T23:17:06.912 INFO:teuthology.orchestra.run.smithi134.stdout:(68/152): openblas-openmp-0.3.21-2.el9.x86_64.r 22 MB/s | 4.7 MB 00:00 2024-04-24T23:17:06.945 INFO:teuthology.orchestra.run.smithi134.stdout:(69/152): postgresql-13.14-1.el9.x86_64.rpm 7.9 MB/s | 1.6 MB 00:00 2024-04-24T23:17:06.971 INFO:teuthology.orchestra.run.smithi134.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 2.2 MB/s | 133 kB 00:00 2024-04-24T23:17:07.004 INFO:teuthology.orchestra.run.smithi134.stdout:(71/152): postgresql-contrib-13.14-1.el9.x86_64 5.6 MB/s | 911 kB 00:00 2024-04-24T23:17:07.046 INFO:teuthology.orchestra.run.smithi134.stdout:(72/152): protobuf-3.14.0-13.el9.x86_64.rpm 13 MB/s | 1.0 MB 00:00 2024-04-24T23:17:07.072 INFO:teuthology.orchestra.run.smithi134.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-04-24T23:17:07.114 INFO:teuthology.orchestra.run.smithi134.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 5.8 MB/s | 244 kB 00:00 2024-04-24T23:17:07.115 INFO:teuthology.orchestra.run.smithi148.stdout:(39/152): ceph-test-19.0.0-2445.gd2c45d97.el9.x 7.4 MB/s | 48 MB 00:06 2024-04-24T23:17:07.189 INFO:teuthology.orchestra.run.smithi134.stdout:(75/152): postgresql-server-13.14-1.el9.x86_64. 24 MB/s | 5.8 MB 00:00 2024-04-24T23:17:07.189 INFO:teuthology.orchestra.run.smithi080.stdout:(122/152): python3-grpcio-tools-1.46.7-10.el9.x 333 kB/s | 144 kB 00:00 2024-04-24T23:17:07.214 INFO:teuthology.orchestra.run.smithi134.stdout:(76/152): python3-jinja2-2.11.3-5.el9.noarch.rp 2.4 MB/s | 248 kB 00:00 2024-04-24T23:17:07.214 INFO:teuthology.orchestra.run.smithi080.stdout:(123/152): python3-google-auth-2.29.0-1.el9.noa 385 kB/s | 206 kB 00:00 2024-04-24T23:17:07.239 INFO:teuthology.orchestra.run.smithi134.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 513 kB/s | 26 kB 00:00 2024-04-24T23:17:07.264 INFO:teuthology.orchestra.run.smithi134.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 382 kB/s | 19 kB 00:00 2024-04-24T23:17:07.342 INFO:teuthology.orchestra.run.smithi134.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 1.6 MB/s | 172 kB 00:00 2024-04-24T23:17:07.365 INFO:teuthology.orchestra.run.smithi134.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 346 kB/s | 35 kB 00:00 2024-04-24T23:17:07.373 INFO:teuthology.orchestra.run.smithi080.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 58 kB/s | 11 kB 00:00 2024-04-24T23:17:07.373 INFO:teuthology.orchestra.run.smithi148.stdout:(40/152): fuse-2.9.9-15.el9.x86_64.rpm 115 kB/s | 80 kB 00:00 2024-04-24T23:17:07.449 INFO:teuthology.orchestra.run.smithi134.stdout:(81/152): python3-babel-2.9.1-2.el9.noarch.rpm 15 MB/s | 6.0 MB 00:00 2024-04-24T23:17:07.473 INFO:teuthology.orchestra.run.smithi080.stdout:(125/152): python3-grpcio-1.46.7-10.el9.x86_64. 2.7 MB/s | 2.0 MB 00:00 2024-04-24T23:17:07.474 INFO:teuthology.orchestra.run.smithi134.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.3 MB/s | 250 kB 00:00 2024-04-24T23:17:07.499 INFO:teuthology.orchestra.run.smithi080.stdout:(126/152): python3-jaraco-classes-3.2.1-5.el9.n 62 kB/s | 18 kB 00:00 2024-04-24T23:17:07.500 INFO:teuthology.orchestra.run.smithi134.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 4.3 MB/s | 222 kB 00:00 2024-04-24T23:17:07.525 INFO:teuthology.orchestra.run.smithi134.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-04-24T23:17:07.550 INFO:teuthology.orchestra.run.smithi134.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 843 kB/s | 42 kB 00:00 2024-04-24T23:17:07.575 INFO:teuthology.orchestra.run.smithi134.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 5.2 MB/s | 269 kB 00:00 2024-04-24T23:17:07.600 INFO:teuthology.orchestra.run.smithi134.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-04-24T23:17:07.624 INFO:teuthology.orchestra.run.smithi148.stdout:(41/152): libquadmath-11.4.1-3.el9.x86_64.rpm 384 kB/s | 195 kB 00:00 2024-04-24T23:17:07.625 INFO:teuthology.orchestra.run.smithi134.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 5.5 MB/s | 279 kB 00:00 2024-04-24T23:17:07.649 INFO:teuthology.orchestra.run.smithi148.stdout:(42/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 921 kB/s | 253 kB 00:00 2024-04-24T23:17:07.651 INFO:teuthology.orchestra.run.smithi134.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-04-24T23:17:07.666 INFO:teuthology.orchestra.run.smithi080.stdout:(127/152): python3-jaraco-collections-3.0.0-8.e 79 kB/s | 23 kB 00:00 2024-04-24T23:17:07.676 INFO:teuthology.orchestra.run.smithi134.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-04-24T23:17:07.741 INFO:teuthology.orchestra.run.smithi080.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 73 kB/s | 19 kB 00:00 2024-04-24T23:17:07.749 INFO:teuthology.orchestra.run.smithi148.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 1.0 MB/s | 106 kB 00:00 2024-04-24T23:17:07.760 INFO:teuthology.orchestra.run.smithi134.stdout:(91/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 12 MB/s | 5.1 MB 00:00 2024-04-24T23:17:07.766 INFO:teuthology.orchestra.run.smithi080.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 74 kB/s | 20 kB 00:00 2024-04-24T23:17:07.783 INFO:teuthology.orchestra.run.smithi148.stdout:(44/152): libgfortran-11.4.1-3.el9.x86_64.rpm 740 kB/s | 803 kB 00:01 2024-04-24T23:17:07.785 INFO:teuthology.orchestra.run.smithi134.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 384 kB/s | 42 kB 00:00 2024-04-24T23:17:07.810 INFO:teuthology.orchestra.run.smithi134.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 887 kB/s | 44 kB 00:00 2024-04-24T23:17:07.836 INFO:teuthology.orchestra.run.smithi134.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-04-24T23:17:07.850 INFO:teuthology.orchestra.run.smithi148.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 1.3 MB/s | 135 kB 00:00 2024-04-24T23:17:07.861 INFO:teuthology.orchestra.run.smithi134.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 715 kB/s | 36 kB 00:00 2024-04-24T23:17:07.886 INFO:teuthology.orchestra.run.smithi134.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 944 kB/s | 47 kB 00:00 2024-04-24T23:17:07.892 INFO:teuthology.orchestra.run.smithi148.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 1.1 MB/s | 125 kB 00:00 2024-04-24T23:17:07.911 INFO:teuthology.orchestra.run.smithi134.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 5.9 MB/s | 305 kB 00:00 2024-04-24T23:17:07.937 INFO:teuthology.orchestra.run.smithi134.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-04-24T23:17:07.950 INFO:teuthology.orchestra.run.smithi148.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 2.1 MB/s | 215 kB 00:00 2024-04-24T23:17:07.962 INFO:teuthology.orchestra.run.smithi134.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.8 MB/s | 94 kB 00:00 2024-04-24T23:17:07.987 INFO:teuthology.orchestra.run.smithi134.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-04-24T23:17:08.012 INFO:teuthology.orchestra.run.smithi134.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 895 kB/s | 45 kB 00:00 2024-04-24T23:17:08.083 INFO:teuthology.orchestra.run.smithi080.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 135 kB/s | 46 kB 00:00 2024-04-24T23:17:08.108 INFO:teuthology.orchestra.run.smithi080.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 231 kB/s | 79 kB 00:00 2024-04-24T23:17:08.125 INFO:teuthology.orchestra.run.smithi148.stdout:(48/152): python3-cryptography-36.0.1-4.el9.x86 2.4 MB/s | 1.2 MB 00:00 2024-04-24T23:17:08.230 INFO:teuthology.orchestra.run.smithi134.stdout:(102/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 28 MB/s | 16 MB 00:00 2024-04-24T23:17:08.242 INFO:teuthology.orchestra.run.smithi148.stdout:(49/152): automake-1.16.2-8.el9.noarch.rpm 1.9 MB/s | 689 kB 00:00 2024-04-24T23:17:08.255 INFO:teuthology.orchestra.run.smithi134.stdout:(103/152): lua-devel-5.4.4-4.el9.x86_64.rpm 83 kB/s | 22 kB 00:00 2024-04-24T23:17:08.268 INFO:teuthology.orchestra.run.smithi148.stdout:(50/152): bison-3.7.4-5.el9.x86_64.rpm 2.9 MB/s | 943 kB 00:00 2024-04-24T23:17:08.301 INFO:teuthology.orchestra.run.smithi148.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 367 kB/s | 21 kB 00:00 2024-04-24T23:17:08.326 INFO:teuthology.orchestra.run.smithi148.stdout:(52/152): boost-program-options-1.75.0-8.el9.x8 532 kB/s | 107 kB 00:00 2024-04-24T23:17:08.347 INFO:teuthology.orchestra.run.smithi134.stdout:(104/152): gperftools-libs-2.9.1-2.el9.x86_64.r 3.3 MB/s | 309 kB 00:00 2024-04-24T23:17:08.352 INFO:teuthology.orchestra.run.smithi148.stdout:(53/152): flex-2.6.4-9.el9.x86_64.rpm 3.7 MB/s | 313 kB 00:00 2024-04-24T23:17:08.359 INFO:teuthology.orchestra.run.smithi080.stdout:(132/152): python3-kubernetes-26.1.0-2.el9.noar 1.5 MB/s | 1.0 MB 00:00 2024-04-24T23:17:08.372 INFO:teuthology.orchestra.run.smithi134.stdout:(105/152): abseil-cpp-20211102.0-3.el9.x86_64.r 3.8 MB/s | 548 kB 00:00 2024-04-24T23:17:08.377 INFO:teuthology.orchestra.run.smithi148.stdout:(54/152): flexiblas-3.0.4-8.el9.x86_64.rpm 433 kB/s | 33 kB 00:00 2024-04-24T23:17:08.397 INFO:teuthology.orchestra.run.smithi134.stdout:(106/152): grpc-data-1.46.7-10.el9.noarch.rpm 388 kB/s | 19 kB 00:00 2024-04-24T23:17:08.410 INFO:teuthology.orchestra.run.smithi148.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 311 kB/s | 18 kB 00:00 2024-04-24T23:17:08.431 INFO:teuthology.orchestra.run.smithi134.stdout:(107/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 786 kB/s | 26 kB 00:00 2024-04-24T23:17:08.434 INFO:teuthology.orchestra.run.smithi080.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 165 kB/s | 58 kB 00:00 2024-04-24T23:17:08.436 INFO:teuthology.orchestra.run.smithi148.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 190 kB/s | 11 kB 00:00 2024-04-24T23:17:08.456 INFO:teuthology.orchestra.run.smithi134.stdout:(108/152): protobuf-compiler-3.14.0-13.el9.x86_ 1.9 MB/s | 863 kB 00:00 2024-04-24T23:17:08.478 INFO:teuthology.orchestra.run.smithi148.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.3 MB/s | 160 kB 00:00 2024-04-24T23:17:08.481 INFO:teuthology.orchestra.run.smithi134.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 966 kB/s | 49 kB 00:00 2024-04-24T23:17:08.503 INFO:teuthology.orchestra.run.smithi148.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 678 kB/s | 45 kB 00:00 2024-04-24T23:17:08.557 INFO:teuthology.orchestra.run.smithi134.stdout:(110/152): libarrow-9.0.0-10.el9.x86_64.rpm 24 MB/s | 4.4 MB 00:00 2024-04-24T23:17:08.582 INFO:teuthology.orchestra.run.smithi134.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 1.5 MB/s | 151 kB 00:00 2024-04-24T23:17:08.607 INFO:teuthology.orchestra.run.smithi134.stdout:(112/152): libunwind-1.6.2-1.el9.x86_64.rpm 447 kB/s | 67 kB 00:00 2024-04-24T23:17:08.612 INFO:teuthology.orchestra.run.smithi148.stdout:(59/152): libtool-2.4.6-45.el9.x86_64.rpm 5.3 MB/s | 582 kB 00:00 2024-04-24T23:17:08.617 INFO:teuthology.orchestra.run.smithi080.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 63 kB/s | 16 kB 00:00 2024-04-24T23:17:08.637 INFO:teuthology.orchestra.run.smithi148.stdout:(60/152): librdkafka-1.6.1-102.el9.x86_64.rpm 4.1 MB/s | 662 kB 00:00 2024-04-24T23:17:08.641 INFO:teuthology.orchestra.run.smithi134.stdout:(113/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 9.8 MB/s | 838 kB 00:00 2024-04-24T23:17:08.658 INFO:teuthology.orchestra.run.smithi134.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 868 kB/s | 43 kB 00:00 2024-04-24T23:17:08.671 INFO:teuthology.orchestra.run.smithi148.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 435 kB/s | 25 kB 00:00 2024-04-24T23:17:08.683 INFO:teuthology.orchestra.run.smithi134.stdout:(115/152): python3-asyncssh-2.13.2-5.el9.noarch 5.3 MB/s | 548 kB 00:00 2024-04-24T23:17:08.696 INFO:teuthology.orchestra.run.smithi148.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 812 kB/s | 47 kB 00:00 2024-04-24T23:17:08.709 INFO:teuthology.orchestra.run.smithi134.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 299 kB/s | 15 kB 00:00 2024-04-24T23:17:08.717 INFO:teuthology.orchestra.run.smithi080.stdout:(135/152): python3-pecan-1.4.2-3.el9.noarch.rpm 447 kB/s | 272 kB 00:00 2024-04-24T23:17:08.729 INFO:teuthology.orchestra.run.smithi148.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.9 MB/s | 292 kB 00:00 2024-04-24T23:17:08.734 INFO:teuthology.orchestra.run.smithi134.stdout:(117/152): python3-cachetools-4.2.4-1.el9.noarc 349 kB/s | 32 kB 00:00 2024-04-24T23:17:08.755 INFO:teuthology.orchestra.run.smithi148.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 3.1 MB/s | 188 kB 00:00 2024-04-24T23:17:08.759 INFO:teuthology.orchestra.run.smithi134.stdout:(118/152): python3-cheroot-8.6.0-4.el9.noarch.r 2.2 MB/s | 172 kB 00:00 2024-04-24T23:17:08.784 INFO:teuthology.orchestra.run.smithi134.stdout:(119/152): python3-cherrypy-18.6.1-2.el9.noarch 4.6 MB/s | 358 kB 00:00 2024-04-24T23:17:08.788 INFO:teuthology.orchestra.run.smithi148.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 601 kB/s | 35 kB 00:00 2024-04-24T23:17:08.809 INFO:teuthology.orchestra.run.smithi134.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 702 kB/s | 35 kB 00:00 2024-04-24T23:17:08.843 INFO:teuthology.orchestra.run.smithi134.stdout:(121/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.7 MB/s | 303 kB 00:00 2024-04-24T23:17:08.847 INFO:teuthology.orchestra.run.smithi148.stdout:(66/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 5.7 MB/s | 3.0 MB 00:00 2024-04-24T23:17:08.859 INFO:teuthology.orchestra.run.smithi080.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 211 kB/s | 90 kB 00:00 2024-04-24T23:17:08.868 INFO:teuthology.orchestra.run.smithi134.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 2.4 MB/s | 206 kB 00:00 2024-04-24T23:17:08.893 INFO:teuthology.orchestra.run.smithi080.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 102 kB/s | 28 kB 00:00 2024-04-24T23:17:08.893 INFO:teuthology.orchestra.run.smithi134.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 2.8 MB/s | 144 kB 00:00 2024-04-24T23:17:08.919 INFO:teuthology.orchestra.run.smithi134.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 213 kB/s | 11 kB 00:00 2024-04-24T23:17:08.944 INFO:teuthology.orchestra.run.smithi134.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 352 kB/s | 18 kB 00:00 2024-04-24T23:17:08.969 INFO:teuthology.orchestra.run.smithi134.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 462 kB/s | 23 kB 00:00 2024-04-24T23:17:08.985 INFO:teuthology.orchestra.run.smithi080.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 115 kB/s | 31 kB 00:00 2024-04-24T23:17:08.994 INFO:teuthology.orchestra.run.smithi134.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 388 kB/s | 19 kB 00:00 2024-04-24T23:17:09.020 INFO:teuthology.orchestra.run.smithi134.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 392 kB/s | 20 kB 00:00 2024-04-24T23:17:09.047 INFO:teuthology.orchestra.run.smithi148.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 7.9 MB/s | 1.6 MB 00:00 2024-04-24T23:17:09.062 INFO:teuthology.orchestra.run.smithi134.stdout:(129/152): python3-kubernetes-26.1.0-2.el9.noar 15 MB/s | 1.0 MB 00:00 2024-04-24T23:17:09.087 INFO:teuthology.orchestra.run.smithi134.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 694 kB/s | 46 kB 00:00 2024-04-24T23:17:09.104 INFO:teuthology.orchestra.run.smithi134.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 1.8 MB/s | 79 kB 00:00 2024-04-24T23:17:09.129 INFO:teuthology.orchestra.run.smithi134.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 1.3 MB/s | 58 kB 00:00 2024-04-24T23:17:09.155 INFO:teuthology.orchestra.run.smithi134.stdout:(133/152): python3-pecan-1.4.2-3.el9.noarch.rpm 5.3 MB/s | 272 kB 00:00 2024-04-24T23:17:09.181 INFO:teuthology.orchestra.run.smithi148.stdout:(68/152): postgresql-contrib-13.14-1.el9.x86_64 6.7 MB/s | 911 kB 00:00 2024-04-24T23:17:09.189 INFO:teuthology.orchestra.run.smithi134.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 280 kB/s | 16 kB 00:00 2024-04-24T23:17:09.230 INFO:teuthology.orchestra.run.smithi134.stdout:(135/152): python3-grpcio-1.46.7-10.el9.x86_64. 4.8 MB/s | 2.0 MB 00:00 2024-04-24T23:17:09.245 INFO:teuthology.orchestra.run.smithi080.stdout:(139/152): python3-rsa-4.9-2.el9.noarch.rpm 169 kB/s | 59 kB 00:00 2024-04-24T23:17:09.255 INFO:teuthology.orchestra.run.smithi134.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 894 kB/s | 90 kB 00:00 2024-04-24T23:17:09.260 INFO:teuthology.orchestra.run.smithi080.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 130 kB/s | 36 kB 00:00 2024-04-24T23:17:09.273 INFO:teuthology.orchestra.run.smithi148.stdout:(69/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 9.1 MB/s | 4.4 MB 00:00 2024-04-24T23:17:09.281 INFO:teuthology.orchestra.run.smithi134.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 307 kB/s | 28 kB 00:00 2024-04-24T23:17:09.294 INFO:teuthology.orchestra.run.smithi080.stdout:(141/152): python3-routes-2.5.1-5.el9.noarch.rp 434 kB/s | 188 kB 00:00 2024-04-24T23:17:09.298 INFO:teuthology.orchestra.run.smithi148.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 1.1 MB/s | 133 kB 00:00 2024-04-24T23:17:09.306 INFO:teuthology.orchestra.run.smithi134.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 615 kB/s | 31 kB 00:00 2024-04-24T23:17:09.331 INFO:teuthology.orchestra.run.smithi134.stdout:(139/152): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-04-24T23:17:09.356 INFO:teuthology.orchestra.run.smithi134.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 714 kB/s | 36 kB 00:00 2024-04-24T23:17:09.365 INFO:teuthology.orchestra.run.smithi148.stdout:(71/152): openblas-openmp-0.3.21-2.el9.x86_64.r 7.8 MB/s | 4.7 MB 00:00 2024-04-24T23:17:09.382 INFO:teuthology.orchestra.run.smithi134.stdout:(141/152): python3-routes-2.5.1-5.el9.noarch.rp 1.5 MB/s | 188 kB 00:00 2024-04-24T23:17:09.407 INFO:teuthology.orchestra.run.smithi134.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 676 kB/s | 51 kB 00:00 2024-04-24T23:17:09.423 INFO:teuthology.orchestra.run.smithi148.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-04-24T23:17:09.432 INFO:teuthology.orchestra.run.smithi134.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-04-24T23:17:09.457 INFO:teuthology.orchestra.run.smithi148.stdout:(73/152): protobuf-3.14.0-13.el9.x86_64.rpm 6.4 MB/s | 1.0 MB 00:00 2024-04-24T23:17:09.458 INFO:teuthology.orchestra.run.smithi134.stdout:(144/152): python3-webob-1.8.7-6.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-04-24T23:17:09.483 INFO:teuthology.orchestra.run.smithi134.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 398 kB/s | 20 kB 00:00 2024-04-24T23:17:09.508 INFO:teuthology.orchestra.run.smithi134.stdout:(146/152): re2-20211101-3.el9.x86_64.rpm 3.7 MB/s | 192 kB 00:00 2024-04-24T23:17:09.524 INFO:teuthology.orchestra.run.smithi148.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 3.6 MB/s | 244 kB 00:00 2024-04-24T23:17:09.542 INFO:teuthology.orchestra.run.smithi134.stdout:(147/152): python3-werkzeug-2.0.3-3.el9.1.noarc 3.1 MB/s | 427 kB 00:00 2024-04-24T23:17:09.584 INFO:teuthology.orchestra.run.smithi134.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-04-24T23:17:09.586 INFO:teuthology.orchestra.run.smithi080.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 149 kB/s | 51 kB 00:00 2024-04-24T23:17:09.591 INFO:teuthology.orchestra.run.smithi148.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 3.6 MB/s | 248 kB 00:00 2024-04-24T23:17:09.636 INFO:teuthology.orchestra.run.smithi080.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 262 kB/s | 90 kB 00:00 2024-04-24T23:17:09.650 INFO:teuthology.orchestra.run.smithi148.stdout:(76/152): python3-jsonpatch-1.21-16.el9.noarch. 440 kB/s | 26 kB 00:00 2024-04-24T23:17:09.708 INFO:teuthology.orchestra.run.smithi148.stdout:(77/152): python3-jsonpointer-2.0-4.el9.noarch. 330 kB/s | 19 kB 00:00 2024-04-24T23:17:09.778 INFO:teuthology.orchestra.run.smithi080.stdout:(144/152): python3-webob-1.8.7-6.el9.noarch.rpm 444 kB/s | 230 kB 00:00 2024-04-24T23:17:09.784 INFO:teuthology.orchestra.run.smithi148.stdout:(78/152): postgresql-server-13.14-1.el9.x86_64. 11 MB/s | 5.8 MB 00:00 2024-04-24T23:17:09.809 INFO:teuthology.orchestra.run.smithi148.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 1.7 MB/s | 172 kB 00:00 2024-04-24T23:17:09.834 INFO:teuthology.orchestra.run.smithi134.stdout:(149/152): librados2-19.0.0-2445.gd2c45d97.el9. 10 MB/s | 3.4 MB 00:00 2024-04-24T23:17:09.842 INFO:teuthology.orchestra.run.smithi148.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 596 kB/s | 35 kB 00:00 2024-04-24T23:17:09.904 INFO:teuthology.orchestra.run.smithi080.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 75 kB/s | 20 kB 00:00 2024-04-24T23:17:09.926 INFO:teuthology.orchestra.run.smithi148.stdout:(81/152): python3-babel-2.9.1-2.el9.noarch.rpm 12 MB/s | 6.0 MB 00:00 2024-04-24T23:17:09.951 INFO:teuthology.orchestra.run.smithi148.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.3 MB/s | 250 kB 00:00 2024-04-24T23:17:09.985 INFO:teuthology.orchestra.run.smithi148.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 3.7 MB/s | 222 kB 00:00 2024-04-24T23:17:10.010 INFO:teuthology.orchestra.run.smithi148.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-04-24T23:17:10.044 INFO:teuthology.orchestra.run.smithi148.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 723 kB/s | 42 kB 00:00 2024-04-24T23:17:10.069 INFO:teuthology.orchestra.run.smithi148.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 4.5 MB/s | 269 kB 00:00 2024-04-24T23:17:10.103 INFO:teuthology.orchestra.run.smithi148.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.7 MB/s | 159 kB 00:00 2024-04-24T23:17:10.144 INFO:teuthology.orchestra.run.smithi148.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 3.6 MB/s | 279 kB 00:00 2024-04-24T23:17:10.170 INFO:teuthology.orchestra.run.smithi148.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 761 kB/s | 51 kB 00:00 2024-04-24T23:17:10.170 INFO:teuthology.orchestra.run.smithi080.stdout:(146/152): python3-werkzeug-2.0.3-3.el9.1.noarc 731 kB/s | 427 kB 00:00 2024-04-24T23:17:10.204 INFO:teuthology.orchestra.run.smithi080.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 450 kB/s | 192 kB 00:00 2024-04-24T23:17:10.237 INFO:teuthology.orchestra.run.smithi148.stdout:(90/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 12 MB/s | 5.1 MB 00:00 2024-04-24T23:17:10.242 INFO:teuthology.orchestra.run.smithi134.stdout:(150/152): librbd1-19.0.0-2445.gd2c45d97.el9.x8 4.5 MB/s | 3.1 MB 00:00 2024-04-24T23:17:10.262 INFO:teuthology.orchestra.run.smithi148.stdout:(91/152): python3-requests-oauthlib-1.3.0-12.el 458 kB/s | 54 kB 00:00 2024-04-24T23:17:10.295 INFO:teuthology.orchestra.run.smithi148.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 714 kB/s | 42 kB 00:00 2024-04-24T23:17:10.321 INFO:teuthology.orchestra.run.smithi148.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 753 kB/s | 44 kB 00:00 2024-04-24T23:17:10.354 INFO:teuthology.orchestra.run.smithi148.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-04-24T23:17:10.368 INFO:teuthology.orchestra.run.smithi134.stdout:(151/152): openssl-3.2.1-1.el9.x86_64.rpm 1.7 MB/s | 1.3 MB 00:00 2024-04-24T23:17:10.371 INFO:teuthology.orchestra.run.smithi148.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 715 kB/s | 36 kB 00:00 2024-04-24T23:17:10.405 INFO:teuthology.orchestra.run.smithi148.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 943 kB/s | 47 kB 00:00 2024-04-24T23:17:10.441 INFO:teuthology.orchestra.run.smithi148.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 4.3 MB/s | 305 kB 00:00 2024-04-24T23:17:10.491 INFO:teuthology.orchestra.run.smithi148.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 675 kB/s | 58 kB 00:00 2024-04-24T23:17:10.504 INFO:teuthology.orchestra.run.smithi080.stdout:(148/152): librbd1-19.0.0-2445.gd2c45d97.el9.x8 10 MB/s | 3.1 MB 00:00 2024-04-24T23:17:10.516 INFO:teuthology.orchestra.run.smithi148.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.2 MB/s | 94 kB 00:00 2024-04-24T23:17:10.550 INFO:teuthology.orchestra.run.smithi148.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-04-24T23:17:10.575 INFO:teuthology.orchestra.run.smithi148.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 766 kB/s | 45 kB 00:00 2024-04-24T23:17:10.650 INFO:teuthology.orchestra.run.smithi148.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 223 kB/s | 22 kB 00:00 2024-04-24T23:17:10.713 INFO:teuthology.orchestra.run.smithi080.stdout:(149/152): thrift-0.15.0-2.el9.x86_64.rpm 2.0 MB/s | 1.6 MB 00:00 2024-04-24T23:17:10.759 INFO:teuthology.orchestra.run.smithi148.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 4.6 MB/s | 863 kB 00:00 2024-04-24T23:17:10.776 INFO:teuthology.orchestra.run.smithi134.stdout:(152/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 2.5 MB/s | 2.4 MB 00:00 2024-04-24T23:17:10.779 INFO:teuthology.orchestra.run.smithi134.stdout:-------------------------------------------------------------------------------- 2024-04-24T23:17:10.779 INFO:teuthology.orchestra.run.smithi134.stdout:Total 12 MB/s | 220 MB 00:18 2024-04-24T23:17:10.855 INFO:teuthology.orchestra.run.smithi080.stdout:(150/152): openssl-3.2.1-1.el9.x86_64.rpm 3.9 MB/s | 1.3 MB 00:00 2024-04-24T23:17:10.976 INFO:teuthology.orchestra.run.smithi148.stdout:(104/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 20 MB/s | 16 MB 00:00 2024-04-24T23:17:11.280 INFO:teuthology.orchestra.run.smithi080.stdout:(151/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 4.2 MB/s | 2.4 MB 00:00 2024-04-24T23:17:11.351 INFO:teuthology.orchestra.run.smithi148.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 56 kB/s | 19 kB 00:00 2024-04-24T23:17:11.431 INFO:teuthology.orchestra.run.smithi080.stdout:(152/152): librados2-19.0.0-2445.gd2c45d97.el9. 2.7 MB/s | 3.4 MB 00:01 2024-04-24T23:17:11.433 INFO:teuthology.orchestra.run.smithi080.stdout:-------------------------------------------------------------------------------- 2024-04-24T23:17:11.434 INFO:teuthology.orchestra.run.smithi080.stdout:Total 11 MB/s | 220 MB 00:19 2024-04-24T23:17:11.476 INFO:teuthology.orchestra.run.smithi148.stdout:(106/152): abseil-cpp-20211102.0-3.el9.x86_64.r 664 kB/s | 548 kB 00:00 2024-04-24T23:17:11.535 INFO:teuthology.orchestra.run.smithi148.stdout:(107/152): gperftools-libs-2.9.1-2.el9.x86_64.r 412 kB/s | 309 kB 00:00 2024-04-24T23:17:11.577 INFO:teuthology.orchestra.run.smithi148.stdout:(108/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 259 kB/s | 26 kB 00:00 2024-04-24T23:17:11.677 INFO:teuthology.orchestra.run.smithi148.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 342 kB/s | 49 kB 00:00 2024-04-24T23:17:11.702 INFO:teuthology.orchestra.run.smithi148.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 538 kB/s | 67 kB 00:00 2024-04-24T23:17:11.819 INFO:teuthology.orchestra.run.smithi148.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 1.0 MB/s | 151 kB 00:00 2024-04-24T23:17:11.833 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction check 2024-04-24T23:17:11.936 INFO:teuthology.orchestra.run.smithi148.stdout:(112/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 3.5 MB/s | 838 kB 00:00 2024-04-24T23:17:11.977 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction check succeeded. 2024-04-24T23:17:11.977 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction test 2024-04-24T23:17:12.053 INFO:teuthology.orchestra.run.smithi148.stdout:(113/152): python3-asyncssh-2.13.2-5.el9.noarch 2.3 MB/s | 548 kB 00:00 2024-04-24T23:17:12.078 INFO:teuthology.orchestra.run.smithi148.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 306 kB/s | 43 kB 00:00 2024-04-24T23:17:12.146 INFO:teuthology.orchestra.run.smithi148.stdout:(115/152): libarrow-9.0.0-10.el9.x86_64.rpm 5.6 MB/s | 4.4 MB 00:00 2024-04-24T23:17:12.179 INFO:teuthology.orchestra.run.smithi148.stdout:(116/152): python3-cachetools-4.2.4-1.el9.noarc 257 kB/s | 32 kB 00:00 2024-04-24T23:17:12.204 INFO:teuthology.orchestra.run.smithi148.stdout:(117/152): python3-certifi-2021.10.8-2.el9.noar 119 kB/s | 15 kB 00:00 2024-04-24T23:17:12.355 INFO:teuthology.orchestra.run.smithi148.stdout:(118/152): python3-cheroot-8.6.0-4.el9.noarch.r 823 kB/s | 172 kB 00:00 2024-04-24T23:17:12.380 INFO:teuthology.orchestra.run.smithi148.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 1.7 MB/s | 303 kB 00:00 2024-04-24T23:17:12.405 INFO:teuthology.orchestra.run.smithi148.stdout:(120/152): python3-cherrypy-18.6.1-2.el9.noarch 1.6 MB/s | 358 kB 00:00 2024-04-24T23:17:12.480 INFO:teuthology.orchestra.run.smithi148.stdout:(121/152): python3-colorama-0.4.4-12.el9.noarch 281 kB/s | 35 kB 00:00 2024-04-24T23:17:12.514 INFO:teuthology.orchestra.run.smithi148.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 1.5 MB/s | 206 kB 00:00 2024-04-24T23:17:12.525 INFO:teuthology.orchestra.run.smithi080.stdout:Running transaction check 2024-04-24T23:17:12.614 INFO:teuthology.orchestra.run.smithi148.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 1.1 MB/s | 144 kB 00:00 2024-04-24T23:17:12.648 INFO:teuthology.orchestra.run.smithi148.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 80 kB/s | 11 kB 00:00 2024-04-24T23:17:12.664 INFO:teuthology.orchestra.run.smithi080.stdout:Transaction check succeeded. 2024-04-24T23:17:12.664 INFO:teuthology.orchestra.run.smithi080.stdout:Running transaction test 2024-04-24T23:17:12.690 INFO:teuthology.orchestra.run.smithi148.stdout:(125/152): python3-grpcio-1.46.7-10.el9.x86_64. 7.2 MB/s | 2.0 MB 00:00 2024-04-24T23:17:12.740 INFO:teuthology.orchestra.run.smithi148.stdout:(126/152): python3-jaraco-classes-3.2.1-5.el9.n 141 kB/s | 18 kB 00:00 2024-04-24T23:17:12.766 INFO:teuthology.orchestra.run.smithi148.stdout:(127/152): python3-jaraco-collections-3.0.0-8.e 197 kB/s | 23 kB 00:00 2024-04-24T23:17:12.791 INFO:teuthology.orchestra.run.smithi148.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 193 kB/s | 19 kB 00:00 2024-04-24T23:17:12.858 INFO:teuthology.orchestra.run.smithi148.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 168 kB/s | 20 kB 00:00 2024-04-24T23:17:12.908 INFO:teuthology.orchestra.run.smithi148.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 396 kB/s | 46 kB 00:00 2024-04-24T23:17:12.941 INFO:teuthology.orchestra.run.smithi148.stdout:(131/152): python3-kubernetes-26.1.0-2.el9.noar 5.8 MB/s | 1.0 MB 00:00 2024-04-24T23:17:12.975 INFO:teuthology.orchestra.run.smithi148.stdout:(132/152): python3-more-itertools-8.12.0-2.el9. 672 kB/s | 79 kB 00:00 2024-04-24T23:17:13.017 INFO:teuthology.orchestra.run.smithi148.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 531 kB/s | 58 kB 00:00 2024-04-24T23:17:13.067 INFO:teuthology.orchestra.run.smithi148.stdout:(134/152): python3-pecan-1.4.2-3.el9.noarch.rpm 2.1 MB/s | 272 kB 00:00 2024-04-24T23:17:13.093 INFO:teuthology.orchestra.run.smithi148.stdout:(135/152): python3-portend-3.1.0-2.el9.noarch.r 140 kB/s | 16 kB 00:00 2024-04-24T23:17:13.118 INFO:teuthology.orchestra.run.smithi148.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 895 kB/s | 90 kB 00:00 2024-04-24T23:17:13.176 INFO:teuthology.orchestra.run.smithi148.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 259 kB/s | 28 kB 00:00 2024-04-24T23:17:13.193 INFO:teuthology.orchestra.run.smithi148.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 306 kB/s | 31 kB 00:00 2024-04-24T23:17:13.235 INFO:teuthology.orchestra.run.smithi148.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 1.6 MB/s | 188 kB 00:00 2024-04-24T23:17:13.302 INFO:teuthology.orchestra.run.smithi148.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 470 kB/s | 59 kB 00:00 2024-04-24T23:17:13.327 INFO:teuthology.orchestra.run.smithi148.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 267 kB/s | 36 kB 00:00 2024-04-24T23:17:13.353 INFO:teuthology.orchestra.run.smithi148.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 435 kB/s | 51 kB 00:00 2024-04-24T23:17:13.428 INFO:teuthology.orchestra.run.smithi148.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 1.8 MB/s | 230 kB 00:00 2024-04-24T23:17:13.487 INFO:teuthology.orchestra.run.smithi148.stdout:(144/152): python3-werkzeug-2.0.3-3.el9.1.noarc 3.1 MB/s | 427 kB 00:00 2024-04-24T23:17:13.512 INFO:teuthology.orchestra.run.smithi148.stdout:(145/152): python3-websocket-client-1.2.3-2.el9 487 kB/s | 90 kB 00:00 2024-04-24T23:17:13.545 INFO:teuthology.orchestra.run.smithi148.stdout:(146/152): python3-zc-lockfile-2.0-10.el9.noarc 170 kB/s | 20 kB 00:00 2024-04-24T23:17:13.587 INFO:teuthology.orchestra.run.smithi148.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 1.9 MB/s | 192 kB 00:00 2024-04-24T23:17:13.734 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction test succeeded. 2024-04-24T23:17:13.735 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction 2024-04-24T23:17:13.779 INFO:teuthology.orchestra.run.smithi148.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 5.9 MB/s | 1.6 MB 00:00 2024-04-24T23:17:13.913 INFO:teuthology.orchestra.run.smithi148.stdout:(149/152): librados2-19.0.0-2445.gd2c45d97.el9. 9.2 MB/s | 3.4 MB 00:00 2024-04-24T23:17:14.435 INFO:teuthology.orchestra.run.smithi080.stdout:Transaction test succeeded. 2024-04-24T23:17:14.435 INFO:teuthology.orchestra.run.smithi080.stdout:Running transaction 2024-04-24T23:17:14.630 INFO:teuthology.orchestra.run.smithi148.stdout:(150/152): librbd1-19.0.0-2445.gd2c45d97.el9.x8 3.0 MB/s | 3.1 MB 00:01 2024-04-24T23:17:14.696 INFO:teuthology.orchestra.run.smithi148.stdout:(151/152): openssl-3.2.1-1.el9.x86_64.rpm 1.5 MB/s | 1.3 MB 00:00 2024-04-24T23:17:14.930 INFO:teuthology.orchestra.run.smithi148.stdout:(152/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 2.4 MB/s | 2.4 MB 00:01 2024-04-24T23:17:14.936 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2024-04-24T23:17:14.936 INFO:teuthology.orchestra.run.smithi148.stdout:Total 10 MB/s | 220 MB 00:22 2024-04-24T23:17:16.040 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2024-04-24T23:17:16.186 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2024-04-24T23:17:16.187 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2024-04-24T23:17:17.938 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2024-04-24T23:17:17.938 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2024-04-24T23:17:19.602 INFO:teuthology.orchestra.run.smithi134.stdout: Preparing : 1/1 2024-04-24T23:17:19.688 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-24T23:17:19.734 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-24T23:17:19.866 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-24T23:17:20.006 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-24T23:17:20.031 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-24T23:17:20.074 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-24T23:17:20.354 INFO:teuthology.orchestra.run.smithi080.stdout: Preparing : 1/1 2024-04-24T23:17:20.403 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-24T23:17:20.458 INFO:teuthology.orchestra.run.smithi080.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-24T23:17:20.497 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-24T23:17:20.515 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-24T23:17:20.550 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-24T23:17:20.584 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-24T23:17:20.610 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-24T23:17:20.635 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-24T23:17:20.644 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-24T23:17:20.658 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-24T23:17:20.748 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-24T23:17:20.778 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 10/156 2024-04-24T23:17:20.926 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-24T23:17:20.975 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-24T23:17:20.976 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-24T23:17:21.003 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-24T23:17:21.086 INFO:teuthology.orchestra.run.smithi080.stdout: Upgrading : librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-24T23:17:21.089 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-24T23:17:21.113 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 14/156 2024-04-24T23:17:21.175 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 14/156 2024-04-24T23:17:21.197 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-24T23:17:21.207 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-24T23:17:21.218 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-24T23:17:21.259 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-24T23:17:21.354 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-24T23:17:21.373 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-24T23:17:21.393 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 10/156 2024-04-24T23:17:21.405 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-24T23:17:21.513 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-24T23:17:21.540 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-24T23:17:21.589 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-24T23:17:21.597 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-24T23:17:21.621 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-24T23:17:21.637 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-24T23:17:21.671 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-24T23:17:21.721 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-24T23:17:21.735 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-24T23:17:21.743 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 14/156 2024-04-24T23:17:21.779 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-24T23:17:21.798 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 14/156 2024-04-24T23:17:21.819 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-24T23:17:21.823 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-24T23:17:21.843 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-24T23:17:21.855 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-24T23:17:21.883 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-24T23:17:21.951 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-24T23:17:22.013 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-24T23:17:22.044 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-24T23:17:22.045 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-24T23:17:22.077 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-24T23:17:22.079 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-24T23:17:22.155 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-24T23:17:22.185 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-24T23:17:22.190 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9. 31/156 2024-04-24T23:17:22.234 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el 32/156 2024-04-24T23:17:22.262 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_6 33/156 2024-04-24T23:17:22.291 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-24T23:17:22.327 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-24T23:17:22.332 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-24T23:17:22.362 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-24T23:17:22.402 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-24T23:17:22.450 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-24T23:17:22.475 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-24T23:17:22.494 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-24T23:17:22.537 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-24T23:17:22.570 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-24T23:17:22.597 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-24T23:17:22.615 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-24T23:17:22.667 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-24T23:17:22.679 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-24T23:17:22.728 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-24T23:17:22.759 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-24T23:17:22.802 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-24T23:17:22.879 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-24T23:17:22.913 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9. 31/156 2024-04-24T23:17:22.957 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el 32/156 2024-04-24T23:17:22.985 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_6 33/156 2024-04-24T23:17:23.054 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-24T23:17:23.100 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-24T23:17:23.174 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-24T23:17:23.191 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-24T23:17:23.275 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-24T23:17:23.304 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-24T23:17:23.306 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-24T23:17:23.347 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-24T23:17:23.359 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-24T23:17:23.393 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_6 43/156 2024-04-24T23:17:23.434 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-24T23:17:23.581 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-24T23:17:23.629 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-24T23:17:23.684 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-24T23:17:23.757 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-24T23:17:23.797 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-24T23:17:23.833 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-24T23:17:23.880 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-24T23:17:23.883 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2024-04-24T23:17:23.883 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-24T23:17:23.916 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-24T23:17:23.958 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-24T23:17:23.964 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-24T23:17:23.967 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-24T23:17:23.993 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-24T23:17:24.001 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-24T23:17:24.027 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-24T23:17:24.066 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_6 43/156 2024-04-24T23:17:24.072 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-24T23:17:24.108 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-24T23:17:24.136 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-24T23:17:24.190 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-24T23:17:24.220 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-24T23:17:24.252 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-24T23:17:24.262 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-24T23:17:24.273 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-24T23:17:24.293 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-24T23:17:24.319 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-24T23:17:24.327 INFO:teuthology.orchestra.run.smithi080.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-24T23:17:24.355 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-24T23:17:24.362 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-24T23:17:24.382 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-24T23:17:24.395 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-24T23:17:24.435 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-24T23:17:24.461 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-24T23:17:24.463 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-24T23:17:24.485 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-24T23:17:24.494 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-24T23:17:24.531 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-24T23:17:24.559 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-24T23:17:24.577 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-24T23:17:24.614 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-24T23:17:24.657 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-24T23:17:24.686 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-24T23:17:24.699 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-24T23:17:24.700 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-24T23:17:24.718 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-24T23:17:24.737 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-24T23:17:24.772 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-24T23:17:24.810 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-24T23:17:24.811 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-24T23:17:24.835 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-24T23:17:24.842 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-24T23:17:24.872 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-24T23:17:24.888 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-24T23:17:24.906 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-24T23:17:24.918 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-24T23:17:24.932 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-24T23:17:24.940 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-24T23:17:24.959 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-24T23:17:24.962 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-24T23:17:24.981 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-24T23:17:25.016 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-24T23:17:25.053 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-24T23:17:25.063 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-24T23:17:25.091 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 10/156 2024-04-24T23:17:25.102 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-24T23:17:25.102 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-24T23:17:25.142 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-24T23:17:25.154 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-24T23:17:25.167 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-24T23:17:25.186 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-24T23:17:25.192 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-24T23:17:25.238 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-24T23:17:25.266 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-24T23:17:25.288 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-24T23:17:25.316 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-24T23:17:25.317 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-24T23:17:25.393 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-24T23:17:25.402 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-24T23:17:25.423 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-24T23:17:25.425 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 14/156 2024-04-24T23:17:25.450 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-24T23:17:25.477 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-24T23:17:25.479 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-24T23:17:25.488 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 14/156 2024-04-24T23:17:25.509 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-24T23:17:25.520 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-24T23:17:25.529 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-24T23:17:25.559 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-24T23:17:25.591 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-24T23:17:25.621 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-24T23:17:25.655 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-24T23:17:25.688 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-24T23:17:25.726 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-24T23:17:25.729 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-24T23:17:25.759 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-24T23:17:25.764 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-24T23:17:25.850 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-24T23:17:25.866 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-24T23:17:25.894 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-24T23:17:25.934 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-24T23:17:25.968 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-24T23:17:25.981 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-24T23:17:26.008 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-24T23:17:26.043 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-24T23:17:26.078 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-24T23:17:26.107 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-24T23:17:26.127 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-24T23:17:26.195 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-24T23:17:26.243 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-24T23:17:26.276 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-24T23:17:26.308 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-24T23:17:26.352 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-24T23:17:26.375 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-24T23:17:26.384 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-24T23:17:26.413 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-24T23:17:26.426 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-24T23:17:26.480 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-24T23:17:26.511 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-24T23:17:26.534 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-24T23:17:26.562 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-24T23:17:26.565 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-24T23:17:26.600 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-24T23:17:26.675 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-24T23:17:26.711 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9. 31/156 2024-04-24T23:17:26.731 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-24T23:17:26.754 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el 32/156 2024-04-24T23:17:26.760 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-24T23:17:26.783 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_6 33/156 2024-04-24T23:17:26.838 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-24T23:17:26.846 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-24T23:17:26.852 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-24T23:17:26.891 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-24T23:17:26.906 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-24T23:17:26.925 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-24T23:17:26.980 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-24T23:17:27.022 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-24T23:17:27.091 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-24T23:17:27.111 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-24T23:17:27.167 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-24T23:17:27.177 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-24T23:17:27.211 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-24T23:17:27.388 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-24T23:17:27.417 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-24T23:17:27.469 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-24T23:17:27.495 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-24T23:17:27.623 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-24T23:17:27.678 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-24T23:17:27.762 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-24T23:17:27.792 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-24T23:17:27.834 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-24T23:17:27.872 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_6 43/156 2024-04-24T23:17:27.914 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-24T23:17:28.072 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-24T23:17:28.130 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-24T23:17:28.142 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-24T23:17:28.196 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-24T23:17:28.248 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-24T23:17:28.261 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-24T23:17:28.292 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-24T23:17:28.329 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-24T23:17:28.375 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-24T23:17:28.411 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-24T23:17:28.454 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-24T23:17:28.497 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-24T23:17:28.640 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-24T23:17:28.694 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-24T23:17:28.724 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-24T23:17:28.765 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-24T23:17:28.829 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-24T23:17:28.858 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-24T23:17:28.905 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-24T23:17:28.915 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-24T23:17:28.924 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-24T23:17:28.949 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-24T23:17:28.955 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-24T23:17:28.981 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-24T23:17:29.006 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-24T23:17:29.079 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-24T23:17:29.207 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-24T23:17:29.246 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-24T23:17:29.258 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-24T23:17:29.274 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-24T23:17:29.309 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-24T23:17:29.312 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 99/156 2024-04-24T23:17:29.347 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-24T23:17:29.387 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-24T23:17:29.402 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 99/156 2024-04-24T23:17:29.417 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-24T23:17:29.433 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 100/156 2024-04-24T23:17:29.452 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-24T23:17:29.469 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-24T23:17:29.484 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-24T23:17:29.485 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-24T23:17:29.527 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-24T23:17:29.561 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-24T23:17:29.563 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-24T23:17:29.591 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-24T23:17:29.624 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-24T23:17:29.647 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-24T23:17:29.700 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-24T23:17:29.731 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-24T23:17:29.778 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-24T23:17:29.877 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-24T23:17:29.924 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-24T23:17:29.937 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-24T23:17:29.939 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-24T23:17:29.965 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 103/156 2024-04-24T23:17:29.979 INFO:teuthology.orchestra.run.smithi080.stdout: Upgrading : librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 99/156 2024-04-24T23:17:30.035 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 103/156 2024-04-24T23:17:30.058 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 104/156 2024-04-24T23:17:30.068 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 99/156 2024-04-24T23:17:30.101 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 100/156 2024-04-24T23:17:30.171 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-24T23:17:30.641 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-24T23:17:30.691 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-24T23:17:30.722 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-24T23:17:30.818 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-24T23:17:30.820 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-24T23:17:30.840 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 103/156 2024-04-24T23:17:30.879 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-24T23:17:30.911 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 103/156 2024-04-24T23:17:30.942 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 104/156 2024-04-24T23:17:30.967 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-24T23:17:31.049 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-24T23:17:31.227 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-24T23:17:31.255 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-24T23:17:31.332 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-24T23:17:31.972 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-24T23:17:32.093 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-24T23:17:32.337 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-24T23:17:32.381 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-24T23:17:32.918 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-24T23:17:32.951 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 106/156 2024-04-24T23:17:33.043 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 106/156 2024-04-24T23:17:33.043 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-24T23:17:33.043 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-24T23:17:33.043 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-24T23:17:33.102 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 107/156 2024-04-24T23:17:33.198 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-24T23:17:33.231 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-24T23:17:33.327 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-24T23:17:33.417 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-24T23:17:33.444 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-24T23:17:33.478 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-24T23:17:33.766 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-24T23:17:33.788 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-24T23:17:33.817 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 106/156 2024-04-24T23:17:33.832 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 99/156 2024-04-24T23:17:33.899 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 106/156 2024-04-24T23:17:33.899 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-24T23:17:33.900 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-24T23:17:33.900 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-24T23:17:33.922 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 99/156 2024-04-24T23:17:33.926 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 107/156 2024-04-24T23:17:33.954 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 100/156 2024-04-24T23:17:33.990 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-24T23:17:34.458 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-24T23:17:34.485 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 103/156 2024-04-24T23:17:34.539 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 103/156 2024-04-24T23:17:34.562 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 104/156 2024-04-24T23:17:36.832 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-24T23:17:36.875 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-24T23:17:37.411 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-24T23:17:37.470 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 106/156 2024-04-24T23:17:37.560 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 106/156 2024-04-24T23:17:37.560 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-24T23:17:37.560 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-24T23:17:37.560 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-24T23:17:37.587 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 107/156 2024-04-24T23:17:44.503 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 107/156 2024-04-24T23:17:44.503 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /sys 2024-04-24T23:17:44.503 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /proc 2024-04-24T23:17:44.503 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /mnt 2024-04-24T23:17:44.504 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /var/tmp 2024-04-24T23:17:44.504 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /home 2024-04-24T23:17:44.504 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /root 2024-04-24T23:17:44.504 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /tmp 2024-04-24T23:17:44.504 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-24T23:17:44.663 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 108/156 2024-04-24T23:17:44.677 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 108/156 2024-04-24T23:17:44.677 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:17:44.677 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-24T23:17:44.678 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-24T23:17:45.543 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 107/156 2024-04-24T23:17:45.543 INFO:teuthology.orchestra.run.smithi080.stdout:skipping the directory /sys 2024-04-24T23:17:45.543 INFO:teuthology.orchestra.run.smithi080.stdout:skipping the directory /proc 2024-04-24T23:17:45.543 INFO:teuthology.orchestra.run.smithi080.stdout:skipping the directory /mnt 2024-04-24T23:17:45.543 INFO:teuthology.orchestra.run.smithi080.stdout:skipping the directory /var/tmp 2024-04-24T23:17:45.543 INFO:teuthology.orchestra.run.smithi080.stdout:skipping the directory /home 2024-04-24T23:17:45.543 INFO:teuthology.orchestra.run.smithi080.stdout:skipping the directory /root 2024-04-24T23:17:45.544 INFO:teuthology.orchestra.run.smithi080.stdout:skipping the directory /tmp 2024-04-24T23:17:45.544 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-24T23:17:45.716 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 108/156 2024-04-24T23:17:45.730 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 108/156 2024-04-24T23:17:45.730 INFO:teuthology.orchestra.run.smithi080.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:17:45.731 INFO:teuthology.orchestra.run.smithi080.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-24T23:17:45.731 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-24T23:17:46.130 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 109/156 2024-04-24T23:17:46.155 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 109/156 2024-04-24T23:17:46.155 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:17:46.155 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-24T23:17:46.155 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-24T23:17:46.155 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-24T23:17:46.155 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-24T23:17:46.382 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 110/156 2024-04-24T23:17:46.404 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 110/156 2024-04-24T23:17:46.404 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:17:46.404 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-24T23:17:46.404 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-24T23:17:46.405 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-24T23:17:46.405 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-24T23:17:46.850 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 111/156 2024-04-24T23:17:46.873 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 111/156 2024-04-24T23:17:46.873 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:17:46.873 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-24T23:17:46.873 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-24T23:17:46.873 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-24T23:17:46.873 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-24T23:17:46.988 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-24T23:17:47.045 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-24T23:17:47.101 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-24T23:17:47.211 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 109/156 2024-04-24T23:17:47.236 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 109/156 2024-04-24T23:17:47.237 INFO:teuthology.orchestra.run.smithi080.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:17:47.237 INFO:teuthology.orchestra.run.smithi080.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-24T23:17:47.237 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-24T23:17:47.237 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-24T23:17:47.237 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-24T23:17:47.285 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-24T23:17:47.342 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-24T23:17:47.412 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-24T23:17:47.461 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 110/156 2024-04-24T23:17:47.483 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 110/156 2024-04-24T23:17:47.483 INFO:teuthology.orchestra.run.smithi080.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:17:47.483 INFO:teuthology.orchestra.run.smithi080.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-24T23:17:47.484 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-24T23:17:47.484 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-24T23:17:47.484 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-24T23:17:47.499 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-24T23:17:47.630 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-24T23:17:47.667 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-24T23:17:47.700 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-24T23:17:47.728 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-24T23:17:47.921 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 111/156 2024-04-24T23:17:47.943 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 111/156 2024-04-24T23:17:47.943 INFO:teuthology.orchestra.run.smithi080.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:17:47.943 INFO:teuthology.orchestra.run.smithi080.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-24T23:17:47.943 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-24T23:17:47.943 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-24T23:17:47.943 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-24T23:17:48.042 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-24T23:17:48.090 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-24T23:17:48.145 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-24T23:17:48.167 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 123/156 2024-04-24T23:17:48.192 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 123/156 2024-04-24T23:17:48.215 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.e 124/156 2024-04-24T23:17:48.269 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97. 125/156 2024-04-24T23:17:48.299 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 126/156 2024-04-24T23:17:48.331 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-24T23:17:48.371 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-24T23:17:48.440 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-24T23:17:48.529 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-24T23:17:48.668 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-24T23:17:48.704 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-24T23:17:48.737 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-24T23:17:48.765 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-24T23:17:49.027 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 107/156 2024-04-24T23:17:49.027 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /sys 2024-04-24T23:17:49.027 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /proc 2024-04-24T23:17:49.027 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /mnt 2024-04-24T23:17:49.027 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /var/tmp 2024-04-24T23:17:49.027 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /home 2024-04-24T23:17:49.027 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /root 2024-04-24T23:17:49.027 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /tmp 2024-04-24T23:17:49.028 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-24T23:17:49.164 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 123/156 2024-04-24T23:17:49.183 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 108/156 2024-04-24T23:17:49.197 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 123/156 2024-04-24T23:17:49.198 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 108/156 2024-04-24T23:17:49.198 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:17:49.198 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-24T23:17:49.198 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-24T23:17:49.228 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.e 124/156 2024-04-24T23:17:49.290 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97. 125/156 2024-04-24T23:17:49.328 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 126/156 2024-04-24T23:17:49.329 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 126/156 2024-04-24T23:17:49.359 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 127/156 2024-04-24T23:17:50.319 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 127/156 2024-04-24T23:17:50.342 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 128/156 2024-04-24T23:17:50.363 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 126/156 2024-04-24T23:17:50.405 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 127/156 2024-04-24T23:17:50.442 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 128/156 2024-04-24T23:17:50.596 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el 129/156 2024-04-24T23:17:50.627 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 130/156 2024-04-24T23:17:50.650 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 130/156 2024-04-24T23:17:50.650 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:17:50.650 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-24T23:17:50.650 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-24T23:17:50.650 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-24T23:17:50.651 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-24T23:17:50.694 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 109/156 2024-04-24T23:17:50.695 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 131/156 2024-04-24T23:17:50.705 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 131/156 2024-04-24T23:17:50.720 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 109/156 2024-04-24T23:17:50.720 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:17:50.720 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-24T23:17:50.720 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-24T23:17:50.720 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-24T23:17:50.720 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-24T23:17:50.812 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 132/156 2024-04-24T23:17:50.860 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 133/156 2024-04-24T23:17:50.944 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 110/156 2024-04-24T23:17:50.967 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 110/156 2024-04-24T23:17:50.967 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:17:50.967 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-24T23:17:50.967 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-24T23:17:50.967 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-24T23:17:50.967 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-24T23:17:51.368 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 127/156 2024-04-24T23:17:51.406 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 128/156 2024-04-24T23:17:51.412 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 111/156 2024-04-24T23:17:51.434 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 111/156 2024-04-24T23:17:51.434 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:17:51.435 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-24T23:17:51.435 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-24T23:17:51.435 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-24T23:17:51.435 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-24T23:17:51.508 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 128/156 2024-04-24T23:17:51.558 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-24T23:17:51.606 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-24T23:17:51.662 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-24T23:17:51.669 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el 129/156 2024-04-24T23:17:51.699 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 130/156 2024-04-24T23:17:51.722 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 130/156 2024-04-24T23:17:51.722 INFO:teuthology.orchestra.run.smithi080.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:17:51.722 INFO:teuthology.orchestra.run.smithi080.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-24T23:17:51.723 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-24T23:17:51.723 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-24T23:17:51.723 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-24T23:17:51.754 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-24T23:17:51.767 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 131/156 2024-04-24T23:17:51.778 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 131/156 2024-04-24T23:17:51.789 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 135/156 2024-04-24T23:17:51.811 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 135/156 2024-04-24T23:17:51.811 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:17:51.811 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-24T23:17:51.811 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-24T23:17:51.812 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-24T23:17:51.812 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-24T23:17:51.855 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-24T23:17:51.856 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 136/156 2024-04-24T23:17:51.875 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 132/156 2024-04-24T23:17:51.878 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 136/156 2024-04-24T23:17:51.878 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:17:51.878 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-24T23:17:51.878 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-24T23:17:51.904 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-24T23:17:51.922 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 133/156 2024-04-24T23:17:51.924 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 137/156 2024-04-24T23:17:51.946 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 137/156 2024-04-24T23:17:51.946 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:17:51.946 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-24T23:17:51.946 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-24T23:17:51.947 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-24T23:17:51.947 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-24T23:17:51.980 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-24T23:17:52.070 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-24T23:17:52.201 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-24T23:17:52.237 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-24T23:17:52.249 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 138/156 2024-04-24T23:17:52.269 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-24T23:17:52.271 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 138/156 2024-04-24T23:17:52.272 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:17:52.272 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-24T23:17:52.272 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-24T23:17:52.272 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-24T23:17:52.272 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-24T23:17:52.297 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-24T23:17:52.747 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 123/156 2024-04-24T23:17:52.771 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 123/156 2024-04-24T23:17:52.794 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.e 124/156 2024-04-24T23:17:52.821 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-24T23:17:52.844 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 135/156 2024-04-24T23:17:52.847 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97. 125/156 2024-04-24T23:17:52.867 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 135/156 2024-04-24T23:17:52.867 INFO:teuthology.orchestra.run.smithi080.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:17:52.867 INFO:teuthology.orchestra.run.smithi080.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-24T23:17:52.867 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-24T23:17:52.867 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-24T23:17:52.867 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-24T23:17:52.877 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 126/156 2024-04-24T23:17:52.921 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 136/156 2024-04-24T23:17:52.943 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 136/156 2024-04-24T23:17:52.943 INFO:teuthology.orchestra.run.smithi080.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:17:52.943 INFO:teuthology.orchestra.run.smithi080.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-24T23:17:52.943 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-24T23:17:52.996 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 137/156 2024-04-24T23:17:53.017 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 137/156 2024-04-24T23:17:53.018 INFO:teuthology.orchestra.run.smithi080.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:17:53.018 INFO:teuthology.orchestra.run.smithi080.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-24T23:17:53.018 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-24T23:17:53.018 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-24T23:17:53.018 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-24T23:17:53.322 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 138/156 2024-04-24T23:17:53.344 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 138/156 2024-04-24T23:17:53.344 INFO:teuthology.orchestra.run.smithi080.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:17:53.344 INFO:teuthology.orchestra.run.smithi080.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-24T23:17:53.344 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-24T23:17:53.344 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-24T23:17:53.344 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-24T23:17:53.915 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 126/156 2024-04-24T23:17:53.945 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 127/156 2024-04-24T23:17:54.916 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 127/156 2024-04-24T23:17:54.946 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 128/156 2024-04-24T23:17:55.048 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 128/156 2024-04-24T23:17:55.217 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el 129/156 2024-04-24T23:17:55.248 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 130/156 2024-04-24T23:17:55.271 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 130/156 2024-04-24T23:17:55.272 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:17:55.272 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-24T23:17:55.272 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-24T23:17:55.272 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-24T23:17:55.272 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-24T23:17:55.324 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 131/156 2024-04-24T23:17:55.334 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 131/156 2024-04-24T23:17:55.432 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 132/156 2024-04-24T23:17:55.488 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 133/156 2024-04-24T23:17:56.379 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-24T23:17:56.418 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 135/156 2024-04-24T23:17:56.440 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 135/156 2024-04-24T23:17:56.440 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:17:56.440 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-24T23:17:56.440 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-24T23:17:56.440 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-24T23:17:56.440 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-24T23:17:56.485 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 136/156 2024-04-24T23:17:56.507 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 136/156 2024-04-24T23:17:56.507 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:17:56.507 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-24T23:17:56.507 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-24T23:17:56.569 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 137/156 2024-04-24T23:17:56.591 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 137/156 2024-04-24T23:17:56.591 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:17:56.591 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-24T23:17:56.591 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-24T23:17:56.591 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-24T23:17:56.592 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-24T23:17:56.753 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 139/156 2024-04-24T23:17:56.794 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 140/156 2024-04-24T23:17:56.835 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 141/156 2024-04-24T23:17:56.867 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-24T23:17:56.894 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 138/156 2024-04-24T23:17:56.895 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-24T23:17:56.917 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 138/156 2024-04-24T23:17:56.917 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:17:56.917 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-24T23:17:56.917 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-24T23:17:56.917 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-24T23:17:56.917 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-24T23:17:57.076 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 144/156 2024-04-24T23:17:57.115 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-24T23:17:57.145 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_ 146/156 2024-04-24T23:17:57.730 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T23:17:57.768 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T23:17:57.787 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T23:17:57.866 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 139/156 2024-04-24T23:17:57.908 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 140/156 2024-04-24T23:17:57.949 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 141/156 2024-04-24T23:17:57.981 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-24T23:17:58.008 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-24T23:17:58.190 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 144/156 2024-04-24T23:17:58.230 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-24T23:17:58.259 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_ 146/156 2024-04-24T23:17:58.619 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 148/156 2024-04-24T23:17:58.733 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-24T23:17:58.779 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-24T23:17:58.806 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T23:17:58.849 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T23:17:58.862 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-24T23:17:58.868 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T23:17:58.893 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-24T23:17:58.893 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-24T23:17:58.977 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-24T23:17:59.010 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-24T23:17:59.010 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-24T23:17:59.100 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-24T23:17:59.166 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T23:17:59.695 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 148/156 2024-04-24T23:17:59.788 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-24T23:17:59.829 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-24T23:17:59.910 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-24T23:17:59.967 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-24T23:17:59.967 INFO:teuthology.orchestra.run.smithi080.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-24T23:18:00.043 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-24T23:18:00.068 INFO:teuthology.orchestra.run.smithi080.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-24T23:18:00.068 INFO:teuthology.orchestra.run.smithi080.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-24T23:18:00.141 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-24T23:18:00.208 INFO:teuthology.orchestra.run.smithi080.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T23:18:01.432 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 139/156 2024-04-24T23:18:01.482 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 140/156 2024-04-24T23:18:01.522 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 141/156 2024-04-24T23:18:01.554 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-24T23:18:01.581 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-24T23:18:01.763 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 144/156 2024-04-24T23:18:01.819 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-24T23:18:01.849 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_ 146/156 2024-04-24T23:18:02.471 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T23:18:02.513 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T23:18:02.533 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T23:18:03.368 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 148/156 2024-04-24T23:18:03.452 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-24T23:18:03.492 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-24T23:18:03.574 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-24T23:18:03.606 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-24T23:18:03.606 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-24T23:18:03.689 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-24T23:18:03.731 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-24T23:18:03.731 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-24T23:18:03.819 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-24T23:18:03.888 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T23:18:04.921 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T23:18:04.922 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 1/156 2024-04-24T23:18:04.922 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 2/156 2024-04-24T23:18:04.922 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 3/156 2024-04-24T23:18:04.922 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 4/156 2024-04-24T23:18:04.922 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 5/156 2024-04-24T23:18:04.922 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 6/156 2024-04-24T23:18:04.922 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 7/156 2024-04-24T23:18:04.922 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-24T23:18:04.922 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-24T23:18:04.923 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 10/156 2024-04-24T23:18:04.923 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 11/156 2024-04-24T23:18:04.923 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 12/156 2024-04-24T23:18:04.923 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-24T23:18:04.923 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 14/156 2024-04-24T23:18:04.923 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_ 15/156 2024-04-24T23:18:04.923 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 16/156 2024-04-24T23:18:04.923 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 17/156 2024-04-24T23:18:04.923 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_6 18/156 2024-04-24T23:18:04.923 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 19/156 2024-04-24T23:18:04.924 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 20/156 2024-04-24T23:18:04.924 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el 21/156 2024-04-24T23:18:04.924 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9. 22/156 2024-04-24T23:18:04.924 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_6 23/156 2024-04-24T23:18:04.924 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 24/156 2024-04-24T23:18:04.924 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 25/156 2024-04-24T23:18:04.924 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 26/156 2024-04-24T23:18:04.924 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 27/156 2024-04-24T23:18:04.924 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 28/156 2024-04-24T23:18:04.924 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 29/156 2024-04-24T23:18:04.925 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97. 30/156 2024-04-24T23:18:04.937 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 31/156 2024-04-24T23:18:04.938 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 32/156 2024-04-24T23:18:04.938 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 33/156 2024-04-24T23:18:04.938 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el 34/156 2024-04-24T23:18:04.938 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 35/156 2024-04-24T23:18:04.938 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.e 36/156 2024-04-24T23:18:04.938 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 37/156 2024-04-24T23:18:04.938 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 38/156 2024-04-24T23:18:04.938 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 39/156 2024-04-24T23:18:04.938 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-24T23:18:04.938 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-24T23:18:04.938 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-24T23:18:04.939 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-24T23:18:04.939 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-24T23:18:04.939 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-24T23:18:04.939 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-24T23:18:04.939 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-24T23:18:04.939 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-24T23:18:04.939 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-24T23:18:04.939 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-24T23:18:04.939 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-24T23:18:04.939 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-24T23:18:04.939 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-24T23:18:04.939 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-24T23:18:04.940 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-24T23:18:04.940 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-24T23:18:04.940 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-24T23:18:04.940 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-24T23:18:04.940 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-24T23:18:04.940 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-24T23:18:04.940 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-24T23:18:04.940 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-24T23:18:04.940 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-24T23:18:04.940 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-24T23:18:04.940 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-24T23:18:04.940 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-24T23:18:04.941 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-24T23:18:04.941 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-24T23:18:04.941 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-24T23:18:04.941 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-24T23:18:04.941 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-24T23:18:04.941 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-24T23:18:04.941 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-24T23:18:04.941 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-24T23:18:04.941 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-24T23:18:04.941 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-24T23:18:04.941 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-24T23:18:04.941 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-24T23:18:04.941 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-24T23:18:04.941 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-24T23:18:04.942 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-24T23:18:04.942 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-24T23:18:04.942 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-24T23:18:04.942 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-24T23:18:04.942 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-24T23:18:04.942 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-24T23:18:04.942 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-24T23:18:04.942 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-24T23:18:04.942 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-24T23:18:04.942 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-24T23:18:04.943 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-24T23:18:04.943 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-24T23:18:04.943 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-24T23:18:04.943 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-24T23:18:04.943 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-24T23:18:04.943 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-24T23:18:04.943 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-24T23:18:04.943 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-24T23:18:04.943 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-24T23:18:04.943 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-24T23:18:04.943 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-24T23:18:04.943 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-24T23:18:04.943 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-24T23:18:04.943 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-24T23:18:04.944 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-24T23:18:04.944 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-24T23:18:04.944 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-24T23:18:04.944 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-24T23:18:04.944 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-24T23:18:04.944 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-24T23:18:04.944 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-24T23:18:04.944 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-24T23:18:04.944 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-24T23:18:04.944 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-24T23:18:04.944 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-24T23:18:04.944 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-24T23:18:04.944 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-24T23:18:04.944 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-24T23:18:04.945 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-24T23:18:04.945 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-24T23:18:04.945 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-24T23:18:04.945 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-24T23:18:04.945 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-24T23:18:04.945 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-24T23:18:04.945 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-24T23:18:04.945 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-24T23:18:04.945 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-24T23:18:04.945 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-24T23:18:04.945 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-24T23:18:04.945 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-24T23:18:04.945 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-24T23:18:04.945 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-24T23:18:04.946 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-24T23:18:04.946 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-24T23:18:04.946 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-24T23:18:04.946 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-24T23:18:04.946 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-24T23:18:04.946 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-24T23:18:04.946 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-24T23:18:04.946 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-24T23:18:04.946 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-24T23:18:04.946 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-24T23:18:04.946 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-24T23:18:04.946 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-24T23:18:04.946 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-24T23:18:04.946 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-24T23:18:04.947 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-24T23:18:04.947 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-24T23:18:04.947 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 149/156 2024-04-24T23:18:04.947 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-24T23:18:04.947 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 151/156 2024-04-24T23:18:04.947 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-24T23:18:04.947 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-24T23:18:04.947 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-24T23:18:04.947 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-24T23:18:05.913 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T23:18:05.913 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 1/156 2024-04-24T23:18:05.913 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 2/156 2024-04-24T23:18:05.913 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 3/156 2024-04-24T23:18:05.913 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 4/156 2024-04-24T23:18:05.913 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 5/156 2024-04-24T23:18:05.913 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 6/156 2024-04-24T23:18:05.913 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 7/156 2024-04-24T23:18:05.913 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-24T23:18:05.913 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-24T23:18:05.913 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 10/156 2024-04-24T23:18:05.913 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 11/156 2024-04-24T23:18:05.914 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 12/156 2024-04-24T23:18:05.914 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-24T23:18:05.914 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 14/156 2024-04-24T23:18:05.914 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_ 15/156 2024-04-24T23:18:05.914 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 16/156 2024-04-24T23:18:05.914 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 17/156 2024-04-24T23:18:05.914 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_6 18/156 2024-04-24T23:18:05.914 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 19/156 2024-04-24T23:18:05.914 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 20/156 2024-04-24T23:18:05.914 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el 21/156 2024-04-24T23:18:05.914 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9. 22/156 2024-04-24T23:18:05.914 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_6 23/156 2024-04-24T23:18:05.914 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 24/156 2024-04-24T23:18:05.914 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 25/156 2024-04-24T23:18:05.915 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 26/156 2024-04-24T23:18:05.915 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 27/156 2024-04-24T23:18:05.915 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 28/156 2024-04-24T23:18:05.915 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 29/156 2024-04-24T23:18:05.915 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97. 30/156 2024-04-24T23:18:05.915 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 31/156 2024-04-24T23:18:05.915 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 32/156 2024-04-24T23:18:05.915 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 33/156 2024-04-24T23:18:05.915 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el 34/156 2024-04-24T23:18:05.915 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 35/156 2024-04-24T23:18:05.915 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.e 36/156 2024-04-24T23:18:05.915 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 37/156 2024-04-24T23:18:05.915 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 38/156 2024-04-24T23:18:05.915 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 39/156 2024-04-24T23:18:05.916 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-24T23:18:05.916 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-24T23:18:05.916 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-24T23:18:05.916 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-24T23:18:05.916 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-24T23:18:05.916 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-24T23:18:05.916 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-24T23:18:05.916 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-24T23:18:05.916 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-24T23:18:05.916 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-24T23:18:05.916 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-24T23:18:05.916 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-24T23:18:05.916 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-24T23:18:05.916 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-24T23:18:05.917 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-24T23:18:05.917 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-24T23:18:05.917 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-24T23:18:05.917 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-24T23:18:05.917 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-24T23:18:05.917 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-24T23:18:05.917 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-24T23:18:05.917 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-24T23:18:05.917 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-24T23:18:05.917 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-24T23:18:05.917 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-24T23:18:05.917 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-24T23:18:05.917 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-24T23:18:05.917 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-24T23:18:05.917 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-24T23:18:05.917 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-24T23:18:05.917 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-24T23:18:05.918 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-24T23:18:05.918 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-24T23:18:05.918 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-24T23:18:05.918 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-24T23:18:05.918 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-24T23:18:05.918 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-24T23:18:05.918 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-24T23:18:05.918 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-24T23:18:05.918 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-24T23:18:05.918 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-24T23:18:05.918 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-24T23:18:05.918 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-24T23:18:05.918 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-24T23:18:05.919 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-24T23:18:05.919 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-24T23:18:05.919 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-24T23:18:05.919 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-24T23:18:05.919 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-24T23:18:05.919 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-24T23:18:05.919 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-24T23:18:05.919 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-24T23:18:05.919 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-24T23:18:05.919 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-24T23:18:05.919 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-24T23:18:05.919 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-24T23:18:05.919 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-24T23:18:05.919 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-24T23:18:05.919 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-24T23:18:05.919 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-24T23:18:05.920 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-24T23:18:05.920 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-24T23:18:05.920 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-24T23:18:05.920 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-24T23:18:05.920 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-24T23:18:05.920 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-24T23:18:05.920 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-24T23:18:05.920 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-24T23:18:05.920 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-24T23:18:05.920 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-24T23:18:05.920 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-24T23:18:05.920 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-24T23:18:05.920 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-24T23:18:05.921 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-24T23:18:05.921 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-24T23:18:05.921 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-24T23:18:05.921 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-24T23:18:05.921 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-24T23:18:05.921 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-24T23:18:05.921 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-24T23:18:05.921 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-24T23:18:05.921 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-24T23:18:05.921 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-24T23:18:05.921 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-24T23:18:05.921 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-24T23:18:05.921 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-24T23:18:05.921 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-24T23:18:05.921 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-24T23:18:05.921 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-24T23:18:05.922 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-24T23:18:05.922 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-24T23:18:05.922 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-24T23:18:05.922 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-24T23:18:05.922 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-24T23:18:05.922 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-24T23:18:05.922 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-24T23:18:05.922 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-24T23:18:05.922 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-24T23:18:05.922 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-24T23:18:05.922 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-24T23:18:05.922 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-24T23:18:05.922 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-24T23:18:05.922 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-24T23:18:05.923 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-24T23:18:05.923 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-24T23:18:05.923 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-24T23:18:05.923 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-24T23:18:05.923 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-24T23:18:05.923 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-24T23:18:05.923 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 149/156 2024-04-24T23:18:05.923 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-24T23:18:05.923 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 151/156 2024-04-24T23:18:05.923 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-24T23:18:05.923 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-24T23:18:05.923 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-24T23:18:05.923 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-24T23:18:07.082 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T23:18:07.082 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-24T23:18:07.082 INFO:teuthology.orchestra.run.smithi134.stdout:Upgraded: 2024-04-24T23:18:07.083 INFO:teuthology.orchestra.run.smithi134.stdout: librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:18:07.083 INFO:teuthology.orchestra.run.smithi134.stdout: librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:18:07.083 INFO:teuthology.orchestra.run.smithi134.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-24T23:18:07.083 INFO:teuthology.orchestra.run.smithi134.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-24T23:18:07.083 INFO:teuthology.orchestra.run.smithi134.stdout:Installed: 2024-04-24T23:18:07.083 INFO:teuthology.orchestra.run.smithi134.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-24T23:18:07.083 INFO:teuthology.orchestra.run.smithi134.stdout: automake-1.16.2-8.el9.noarch 2024-04-24T23:18:07.083 INFO:teuthology.orchestra.run.smithi134.stdout: bison-3.7.4-5.el9.x86_64 2024-04-24T23:18:07.083 INFO:teuthology.orchestra.run.smithi134.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-24T23:18:07.083 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:18:07.083 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:18:07.083 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:18:07.083 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:18:07.084 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:18:07.084 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-immutable-object-cache-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:18:07.084 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:18:07.084 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:18:07.084 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:18:07.084 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:18:07.084 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:18:07.084 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:18:07.084 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:18:07.084 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:18:07.084 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:18:07.084 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:18:07.085 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:18:07.085 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:18:07.085 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:18:07.085 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:18:07.085 INFO:teuthology.orchestra.run.smithi134.stdout: cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:18:07.085 INFO:teuthology.orchestra.run.smithi134.stdout: cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:18:07.085 INFO:teuthology.orchestra.run.smithi134.stdout: cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:18:07.085 INFO:teuthology.orchestra.run.smithi134.stdout: cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:18:07.085 INFO:teuthology.orchestra.run.smithi134.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-24T23:18:07.085 INFO:teuthology.orchestra.run.smithi134.stdout: flex-2.6.4-9.el9.x86_64 2024-04-24T23:18:07.085 INFO:teuthology.orchestra.run.smithi134.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-24T23:18:07.085 INFO:teuthology.orchestra.run.smithi134.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-24T23:18:07.085 INFO:teuthology.orchestra.run.smithi134.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-24T23:18:07.086 INFO:teuthology.orchestra.run.smithi134.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-24T23:18:07.086 INFO:teuthology.orchestra.run.smithi134.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-24T23:18:07.086 INFO:teuthology.orchestra.run.smithi134.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-24T23:18:07.086 INFO:teuthology.orchestra.run.smithi134.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-24T23:18:07.086 INFO:teuthology.orchestra.run.smithi134.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-24T23:18:07.086 INFO:teuthology.orchestra.run.smithi134.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-24T23:18:07.086 INFO:teuthology.orchestra.run.smithi134.stdout: libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:18:07.086 INFO:teuthology.orchestra.run.smithi134.stdout: libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:18:07.086 INFO:teuthology.orchestra.run.smithi134.stdout: libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:18:07.086 INFO:teuthology.orchestra.run.smithi134.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-24T23:18:07.086 INFO:teuthology.orchestra.run.smithi134.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-24T23:18:07.086 INFO:teuthology.orchestra.run.smithi134.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-24T23:18:07.086 INFO:teuthology.orchestra.run.smithi134.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-24T23:18:07.087 INFO:teuthology.orchestra.run.smithi134.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-24T23:18:07.087 INFO:teuthology.orchestra.run.smithi134.stdout: librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:18:07.087 INFO:teuthology.orchestra.run.smithi134.stdout: libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:18:07.087 INFO:teuthology.orchestra.run.smithi134.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-24T23:18:07.087 INFO:teuthology.orchestra.run.smithi134.stdout: librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:18:07.087 INFO:teuthology.orchestra.run.smithi134.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-24T23:18:07.087 INFO:teuthology.orchestra.run.smithi134.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-24T23:18:07.087 INFO:teuthology.orchestra.run.smithi134.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-24T23:18:07.087 INFO:teuthology.orchestra.run.smithi134.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-24T23:18:07.087 INFO:teuthology.orchestra.run.smithi134.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-24T23:18:07.087 INFO:teuthology.orchestra.run.smithi134.stdout: lua-5.4.4-4.el9.x86_64 2024-04-24T23:18:07.087 INFO:teuthology.orchestra.run.smithi134.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-24T23:18:07.087 INFO:teuthology.orchestra.run.smithi134.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-24T23:18:07.087 INFO:teuthology.orchestra.run.smithi134.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-24T23:18:07.088 INFO:teuthology.orchestra.run.smithi134.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-24T23:18:07.088 INFO:teuthology.orchestra.run.smithi134.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-24T23:18:07.088 INFO:teuthology.orchestra.run.smithi134.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-24T23:18:07.088 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-13.14-1.el9.x86_64 2024-04-24T23:18:07.088 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-24T23:18:07.088 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-24T23:18:07.088 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-24T23:18:07.088 INFO:teuthology.orchestra.run.smithi134.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-24T23:18:07.088 INFO:teuthology.orchestra.run.smithi134.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-24T23:18:07.088 INFO:teuthology.orchestra.run.smithi134.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-24T23:18:07.088 INFO:teuthology.orchestra.run.smithi134.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-24T23:18:07.088 INFO:teuthology.orchestra.run.smithi134.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-24T23:18:07.088 INFO:teuthology.orchestra.run.smithi134.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-24T23:18:07.089 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-24T23:18:07.089 INFO:teuthology.orchestra.run.smithi134.stdout: python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:18:07.089 INFO:teuthology.orchestra.run.smithi134.stdout: python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:18:07.089 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:18:07.089 INFO:teuthology.orchestra.run.smithi134.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-24T23:18:07.089 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-24T23:18:07.089 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-24T23:18:07.089 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-24T23:18:07.089 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-24T23:18:07.089 INFO:teuthology.orchestra.run.smithi134.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-24T23:18:07.089 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-24T23:18:07.089 INFO:teuthology.orchestra.run.smithi134.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-24T23:18:07.089 INFO:teuthology.orchestra.run.smithi134.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-24T23:18:07.089 INFO:teuthology.orchestra.run.smithi134.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-24T23:18:07.089 INFO:teuthology.orchestra.run.smithi134.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-24T23:18:07.090 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-24T23:18:07.090 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-24T23:18:07.090 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-24T23:18:07.090 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-24T23:18:07.090 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-24T23:18:07.090 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-24T23:18:07.090 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-24T23:18:07.090 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-24T23:18:07.090 INFO:teuthology.orchestra.run.smithi134.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-24T23:18:07.090 INFO:teuthology.orchestra.run.smithi134.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-24T23:18:07.090 INFO:teuthology.orchestra.run.smithi134.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-24T23:18:07.090 INFO:teuthology.orchestra.run.smithi134.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-24T23:18:07.090 INFO:teuthology.orchestra.run.smithi134.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-24T23:18:07.091 INFO:teuthology.orchestra.run.smithi134.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-24T23:18:07.091 INFO:teuthology.orchestra.run.smithi134.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-24T23:18:07.091 INFO:teuthology.orchestra.run.smithi134.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-24T23:18:07.091 INFO:teuthology.orchestra.run.smithi134.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-24T23:18:07.091 INFO:teuthology.orchestra.run.smithi134.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-24T23:18:07.091 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-24T23:18:07.091 INFO:teuthology.orchestra.run.smithi134.stdout: python3-ply-3.11-14.el9.noarch 2024-04-24T23:18:07.091 INFO:teuthology.orchestra.run.smithi134.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-24T23:18:07.091 INFO:teuthology.orchestra.run.smithi134.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-24T23:18:07.091 INFO:teuthology.orchestra.run.smithi134.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-24T23:18:07.091 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-24T23:18:07.091 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-24T23:18:07.091 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-24T23:18:07.091 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-24T23:18:07.091 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-24T23:18:07.092 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-24T23:18:07.092 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:18:07.092 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:18:07.092 INFO:teuthology.orchestra.run.smithi134.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-24T23:18:07.092 INFO:teuthology.orchestra.run.smithi134.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-24T23:18:07.092 INFO:teuthology.orchestra.run.smithi134.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-24T23:18:07.092 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:18:07.092 INFO:teuthology.orchestra.run.smithi134.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-24T23:18:07.092 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-24T23:18:07.092 INFO:teuthology.orchestra.run.smithi134.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-24T23:18:07.092 INFO:teuthology.orchestra.run.smithi134.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-24T23:18:07.093 INFO:teuthology.orchestra.run.smithi134.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-24T23:18:07.093 INFO:teuthology.orchestra.run.smithi134.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-24T23:18:07.093 INFO:teuthology.orchestra.run.smithi134.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-24T23:18:07.093 INFO:teuthology.orchestra.run.smithi134.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-24T23:18:07.093 INFO:teuthology.orchestra.run.smithi134.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-24T23:18:07.093 INFO:teuthology.orchestra.run.smithi134.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-24T23:18:07.093 INFO:teuthology.orchestra.run.smithi134.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-24T23:18:07.093 INFO:teuthology.orchestra.run.smithi134.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-24T23:18:07.093 INFO:teuthology.orchestra.run.smithi134.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-24T23:18:07.093 INFO:teuthology.orchestra.run.smithi134.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-24T23:18:07.093 INFO:teuthology.orchestra.run.smithi134.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-24T23:18:07.093 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:18:07.093 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:18:07.094 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:18:07.094 INFO:teuthology.orchestra.run.smithi134.stdout: re2-1:20211101-3.el9.x86_64 2024-04-24T23:18:07.094 INFO:teuthology.orchestra.run.smithi134.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-24T23:18:07.094 INFO:teuthology.orchestra.run.smithi134.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-24T23:18:07.094 INFO:teuthology.orchestra.run.smithi134.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-24T23:18:07.094 INFO:teuthology.orchestra.run.smithi134.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-24T23:18:07.094 INFO:teuthology.orchestra.run.smithi134.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-24T23:18:07.094 INFO:teuthology.orchestra.run.smithi134.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-24T23:18:07.094 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-24T23:18:07.095 INFO:teuthology.orchestra.run.smithi134.stdout:Complete! 2024-04-24T23:18:07.506 DEBUG:teuthology.parallel:result is None 2024-04-24T23:18:08.059 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T23:18:08.059 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-24T23:18:08.059 INFO:teuthology.orchestra.run.smithi080.stdout:Upgraded: 2024-04-24T23:18:08.059 INFO:teuthology.orchestra.run.smithi080.stdout: librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:18:08.059 INFO:teuthology.orchestra.run.smithi080.stdout: librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:18:08.059 INFO:teuthology.orchestra.run.smithi080.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-24T23:18:08.059 INFO:teuthology.orchestra.run.smithi080.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-24T23:18:08.059 INFO:teuthology.orchestra.run.smithi080.stdout:Installed: 2024-04-24T23:18:08.060 INFO:teuthology.orchestra.run.smithi080.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-24T23:18:08.060 INFO:teuthology.orchestra.run.smithi080.stdout: automake-1.16.2-8.el9.noarch 2024-04-24T23:18:08.060 INFO:teuthology.orchestra.run.smithi080.stdout: bison-3.7.4-5.el9.x86_64 2024-04-24T23:18:08.060 INFO:teuthology.orchestra.run.smithi080.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-24T23:18:08.060 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:18:08.060 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:18:08.060 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:18:08.060 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:18:08.060 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:18:08.060 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-immutable-object-cache-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:18:08.060 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:18:08.060 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:18:08.060 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:18:08.061 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:18:08.061 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:18:08.061 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:18:08.061 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:18:08.061 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:18:08.061 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:18:08.061 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:18:08.061 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:18:08.061 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:18:08.061 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:18:08.061 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:18:08.061 INFO:teuthology.orchestra.run.smithi080.stdout: cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:18:08.061 INFO:teuthology.orchestra.run.smithi080.stdout: cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:18:08.061 INFO:teuthology.orchestra.run.smithi080.stdout: cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:18:08.062 INFO:teuthology.orchestra.run.smithi080.stdout: cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:18:08.062 INFO:teuthology.orchestra.run.smithi080.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-24T23:18:08.062 INFO:teuthology.orchestra.run.smithi080.stdout: flex-2.6.4-9.el9.x86_64 2024-04-24T23:18:08.062 INFO:teuthology.orchestra.run.smithi080.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-24T23:18:08.062 INFO:teuthology.orchestra.run.smithi080.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-24T23:18:08.062 INFO:teuthology.orchestra.run.smithi080.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-24T23:18:08.062 INFO:teuthology.orchestra.run.smithi080.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-24T23:18:08.062 INFO:teuthology.orchestra.run.smithi080.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-24T23:18:08.062 INFO:teuthology.orchestra.run.smithi080.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-24T23:18:08.062 INFO:teuthology.orchestra.run.smithi080.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-24T23:18:08.062 INFO:teuthology.orchestra.run.smithi080.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-24T23:18:08.062 INFO:teuthology.orchestra.run.smithi080.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-24T23:18:08.062 INFO:teuthology.orchestra.run.smithi080.stdout: libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:18:08.063 INFO:teuthology.orchestra.run.smithi080.stdout: libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:18:08.063 INFO:teuthology.orchestra.run.smithi080.stdout: libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:18:08.063 INFO:teuthology.orchestra.run.smithi080.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-24T23:18:08.063 INFO:teuthology.orchestra.run.smithi080.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-24T23:18:08.063 INFO:teuthology.orchestra.run.smithi080.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-24T23:18:08.063 INFO:teuthology.orchestra.run.smithi080.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-24T23:18:08.063 INFO:teuthology.orchestra.run.smithi080.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-24T23:18:08.063 INFO:teuthology.orchestra.run.smithi080.stdout: librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:18:08.063 INFO:teuthology.orchestra.run.smithi080.stdout: libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:18:08.063 INFO:teuthology.orchestra.run.smithi080.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-24T23:18:08.063 INFO:teuthology.orchestra.run.smithi080.stdout: librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:18:08.063 INFO:teuthology.orchestra.run.smithi080.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-24T23:18:08.063 INFO:teuthology.orchestra.run.smithi080.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-24T23:18:08.064 INFO:teuthology.orchestra.run.smithi080.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-24T23:18:08.064 INFO:teuthology.orchestra.run.smithi080.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-24T23:18:08.064 INFO:teuthology.orchestra.run.smithi080.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-24T23:18:08.064 INFO:teuthology.orchestra.run.smithi080.stdout: lua-5.4.4-4.el9.x86_64 2024-04-24T23:18:08.064 INFO:teuthology.orchestra.run.smithi080.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-24T23:18:08.064 INFO:teuthology.orchestra.run.smithi080.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-24T23:18:08.064 INFO:teuthology.orchestra.run.smithi080.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-24T23:18:08.064 INFO:teuthology.orchestra.run.smithi080.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-24T23:18:08.064 INFO:teuthology.orchestra.run.smithi080.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-24T23:18:08.064 INFO:teuthology.orchestra.run.smithi080.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-24T23:18:08.064 INFO:teuthology.orchestra.run.smithi080.stdout: postgresql-13.14-1.el9.x86_64 2024-04-24T23:18:08.064 INFO:teuthology.orchestra.run.smithi080.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-24T23:18:08.064 INFO:teuthology.orchestra.run.smithi080.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-24T23:18:08.064 INFO:teuthology.orchestra.run.smithi080.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-24T23:18:08.065 INFO:teuthology.orchestra.run.smithi080.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-24T23:18:08.065 INFO:teuthology.orchestra.run.smithi080.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-24T23:18:08.065 INFO:teuthology.orchestra.run.smithi080.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-24T23:18:08.065 INFO:teuthology.orchestra.run.smithi080.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-24T23:18:08.065 INFO:teuthology.orchestra.run.smithi080.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-24T23:18:08.065 INFO:teuthology.orchestra.run.smithi080.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-24T23:18:08.065 INFO:teuthology.orchestra.run.smithi080.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-24T23:18:08.065 INFO:teuthology.orchestra.run.smithi080.stdout: python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:18:08.065 INFO:teuthology.orchestra.run.smithi080.stdout: python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:18:08.065 INFO:teuthology.orchestra.run.smithi080.stdout: python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:18:08.065 INFO:teuthology.orchestra.run.smithi080.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-24T23:18:08.065 INFO:teuthology.orchestra.run.smithi080.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-24T23:18:08.065 INFO:teuthology.orchestra.run.smithi080.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-24T23:18:08.065 INFO:teuthology.orchestra.run.smithi080.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-24T23:18:08.066 INFO:teuthology.orchestra.run.smithi080.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-24T23:18:08.066 INFO:teuthology.orchestra.run.smithi080.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-24T23:18:08.066 INFO:teuthology.orchestra.run.smithi080.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-24T23:18:08.066 INFO:teuthology.orchestra.run.smithi080.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-24T23:18:08.066 INFO:teuthology.orchestra.run.smithi080.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-24T23:18:08.066 INFO:teuthology.orchestra.run.smithi080.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-24T23:18:08.066 INFO:teuthology.orchestra.run.smithi080.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-24T23:18:08.066 INFO:teuthology.orchestra.run.smithi080.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-24T23:18:08.066 INFO:teuthology.orchestra.run.smithi080.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-24T23:18:08.066 INFO:teuthology.orchestra.run.smithi080.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-24T23:18:08.066 INFO:teuthology.orchestra.run.smithi080.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-24T23:18:08.066 INFO:teuthology.orchestra.run.smithi080.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-24T23:18:08.066 INFO:teuthology.orchestra.run.smithi080.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-24T23:18:08.067 INFO:teuthology.orchestra.run.smithi080.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-24T23:18:08.067 INFO:teuthology.orchestra.run.smithi080.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-24T23:18:08.067 INFO:teuthology.orchestra.run.smithi080.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-24T23:18:08.067 INFO:teuthology.orchestra.run.smithi080.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-24T23:18:08.067 INFO:teuthology.orchestra.run.smithi080.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-24T23:18:08.067 INFO:teuthology.orchestra.run.smithi080.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-24T23:18:08.067 INFO:teuthology.orchestra.run.smithi080.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-24T23:18:08.067 INFO:teuthology.orchestra.run.smithi080.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-24T23:18:08.067 INFO:teuthology.orchestra.run.smithi080.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-24T23:18:08.067 INFO:teuthology.orchestra.run.smithi080.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-24T23:18:08.067 INFO:teuthology.orchestra.run.smithi080.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-24T23:18:08.067 INFO:teuthology.orchestra.run.smithi080.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-24T23:18:08.067 INFO:teuthology.orchestra.run.smithi080.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-24T23:18:08.067 INFO:teuthology.orchestra.run.smithi080.stdout: python3-ply-3.11-14.el9.noarch 2024-04-24T23:18:08.068 INFO:teuthology.orchestra.run.smithi080.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-24T23:18:08.068 INFO:teuthology.orchestra.run.smithi080.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-24T23:18:08.068 INFO:teuthology.orchestra.run.smithi080.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-24T23:18:08.068 INFO:teuthology.orchestra.run.smithi080.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-24T23:18:08.068 INFO:teuthology.orchestra.run.smithi080.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-24T23:18:08.068 INFO:teuthology.orchestra.run.smithi080.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-24T23:18:08.068 INFO:teuthology.orchestra.run.smithi080.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-24T23:18:08.068 INFO:teuthology.orchestra.run.smithi080.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-24T23:18:08.068 INFO:teuthology.orchestra.run.smithi080.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-24T23:18:08.068 INFO:teuthology.orchestra.run.smithi080.stdout: python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:18:08.068 INFO:teuthology.orchestra.run.smithi080.stdout: python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:18:08.068 INFO:teuthology.orchestra.run.smithi080.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-24T23:18:08.068 INFO:teuthology.orchestra.run.smithi080.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-24T23:18:08.068 INFO:teuthology.orchestra.run.smithi080.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-24T23:18:08.069 INFO:teuthology.orchestra.run.smithi080.stdout: python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:18:08.069 INFO:teuthology.orchestra.run.smithi080.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-24T23:18:08.069 INFO:teuthology.orchestra.run.smithi080.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-24T23:18:08.069 INFO:teuthology.orchestra.run.smithi080.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-24T23:18:08.069 INFO:teuthology.orchestra.run.smithi080.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-24T23:18:08.069 INFO:teuthology.orchestra.run.smithi080.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-24T23:18:08.069 INFO:teuthology.orchestra.run.smithi080.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-24T23:18:08.069 INFO:teuthology.orchestra.run.smithi080.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-24T23:18:08.069 INFO:teuthology.orchestra.run.smithi080.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-24T23:18:08.069 INFO:teuthology.orchestra.run.smithi080.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-24T23:18:08.069 INFO:teuthology.orchestra.run.smithi080.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-24T23:18:08.069 INFO:teuthology.orchestra.run.smithi080.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-24T23:18:08.069 INFO:teuthology.orchestra.run.smithi080.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-24T23:18:08.069 INFO:teuthology.orchestra.run.smithi080.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-24T23:18:08.070 INFO:teuthology.orchestra.run.smithi080.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-24T23:18:08.070 INFO:teuthology.orchestra.run.smithi080.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-24T23:18:08.070 INFO:teuthology.orchestra.run.smithi080.stdout: rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:18:08.070 INFO:teuthology.orchestra.run.smithi080.stdout: rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:18:08.070 INFO:teuthology.orchestra.run.smithi080.stdout: rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:18:08.070 INFO:teuthology.orchestra.run.smithi080.stdout: re2-1:20211101-3.el9.x86_64 2024-04-24T23:18:08.070 INFO:teuthology.orchestra.run.smithi080.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-24T23:18:08.070 INFO:teuthology.orchestra.run.smithi080.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-24T23:18:08.070 INFO:teuthology.orchestra.run.smithi080.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-24T23:18:08.070 INFO:teuthology.orchestra.run.smithi080.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-24T23:18:08.070 INFO:teuthology.orchestra.run.smithi080.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-24T23:18:08.071 INFO:teuthology.orchestra.run.smithi080.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-24T23:18:08.071 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-24T23:18:08.071 INFO:teuthology.orchestra.run.smithi080.stdout:Complete! 2024-04-24T23:18:08.446 DEBUG:teuthology.parallel:result is None 2024-04-24T23:18:09.653 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T23:18:09.653 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 1/156 2024-04-24T23:18:09.654 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 2/156 2024-04-24T23:18:09.654 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 3/156 2024-04-24T23:18:09.654 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 4/156 2024-04-24T23:18:09.654 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 5/156 2024-04-24T23:18:09.654 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 6/156 2024-04-24T23:18:09.654 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 7/156 2024-04-24T23:18:09.654 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-24T23:18:09.654 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-24T23:18:09.654 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 10/156 2024-04-24T23:18:09.654 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 11/156 2024-04-24T23:18:09.654 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 12/156 2024-04-24T23:18:09.654 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-24T23:18:09.654 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 14/156 2024-04-24T23:18:09.655 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_ 15/156 2024-04-24T23:18:09.655 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 16/156 2024-04-24T23:18:09.655 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 17/156 2024-04-24T23:18:09.655 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_6 18/156 2024-04-24T23:18:09.655 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 19/156 2024-04-24T23:18:09.655 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 20/156 2024-04-24T23:18:09.655 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el 21/156 2024-04-24T23:18:09.655 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9. 22/156 2024-04-24T23:18:09.655 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_6 23/156 2024-04-24T23:18:09.655 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 24/156 2024-04-24T23:18:09.655 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 25/156 2024-04-24T23:18:09.655 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 26/156 2024-04-24T23:18:09.655 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 27/156 2024-04-24T23:18:09.655 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 28/156 2024-04-24T23:18:09.656 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 29/156 2024-04-24T23:18:09.656 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97. 30/156 2024-04-24T23:18:09.656 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 31/156 2024-04-24T23:18:09.656 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 32/156 2024-04-24T23:18:09.656 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 33/156 2024-04-24T23:18:09.656 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el 34/156 2024-04-24T23:18:09.656 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 35/156 2024-04-24T23:18:09.656 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.e 36/156 2024-04-24T23:18:09.656 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 37/156 2024-04-24T23:18:09.656 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 38/156 2024-04-24T23:18:09.656 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 39/156 2024-04-24T23:18:09.656 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-24T23:18:09.656 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-24T23:18:09.656 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-24T23:18:09.657 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-24T23:18:09.657 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-24T23:18:09.657 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-24T23:18:09.657 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-24T23:18:09.657 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-24T23:18:09.657 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-24T23:18:09.657 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-24T23:18:09.657 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-24T23:18:09.657 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-24T23:18:09.657 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-24T23:18:09.657 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-24T23:18:09.657 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-24T23:18:09.657 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-24T23:18:09.657 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-24T23:18:09.658 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-24T23:18:09.658 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-24T23:18:09.658 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-24T23:18:09.658 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-24T23:18:09.658 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-24T23:18:09.658 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-24T23:18:09.658 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-24T23:18:09.658 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-24T23:18:09.658 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-24T23:18:09.658 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-24T23:18:09.658 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-24T23:18:09.658 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-24T23:18:09.658 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-24T23:18:09.658 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-24T23:18:09.659 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-24T23:18:09.659 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-24T23:18:09.659 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-24T23:18:09.659 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-24T23:18:09.659 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-24T23:18:09.659 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-24T23:18:09.659 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-24T23:18:09.659 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-24T23:18:09.659 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-24T23:18:09.659 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-24T23:18:09.659 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-24T23:18:09.659 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-24T23:18:09.659 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-24T23:18:09.659 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-24T23:18:09.660 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-24T23:18:09.660 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-24T23:18:09.660 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-24T23:18:09.660 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-24T23:18:09.660 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-24T23:18:09.660 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-24T23:18:09.660 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-24T23:18:09.660 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-24T23:18:09.660 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-24T23:18:09.660 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-24T23:18:09.660 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-24T23:18:09.660 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-24T23:18:09.660 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-24T23:18:09.660 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-24T23:18:09.660 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-24T23:18:09.660 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-24T23:18:09.661 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-24T23:18:09.661 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-24T23:18:09.661 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-24T23:18:09.661 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-24T23:18:09.661 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-24T23:18:09.661 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-24T23:18:09.661 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-24T23:18:09.661 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-24T23:18:09.661 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-24T23:18:09.661 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-24T23:18:09.661 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-24T23:18:09.661 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-24T23:18:09.661 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-24T23:18:09.661 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-24T23:18:09.661 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-24T23:18:09.662 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-24T23:18:09.662 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-24T23:18:09.662 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-24T23:18:09.662 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-24T23:18:09.662 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-24T23:18:09.662 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-24T23:18:09.662 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-24T23:18:09.662 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-24T23:18:09.662 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-24T23:18:09.662 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-24T23:18:09.662 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-24T23:18:09.662 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-24T23:18:09.662 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-24T23:18:09.662 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-24T23:18:09.663 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-24T23:18:09.663 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-24T23:18:09.663 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-24T23:18:09.663 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-24T23:18:09.663 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-24T23:18:09.663 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-24T23:18:09.663 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-24T23:18:09.663 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-24T23:18:09.663 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-24T23:18:09.663 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-24T23:18:09.663 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-24T23:18:09.663 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-24T23:18:09.663 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-24T23:18:09.663 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-24T23:18:09.664 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-24T23:18:09.664 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-24T23:18:09.664 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-24T23:18:09.664 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-24T23:18:09.664 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-24T23:18:09.664 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 149/156 2024-04-24T23:18:09.664 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-24T23:18:09.664 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 151/156 2024-04-24T23:18:09.664 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-24T23:18:09.664 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-24T23:18:09.664 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-24T23:18:09.664 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-24T23:18:11.823 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T23:18:11.823 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-24T23:18:11.824 INFO:teuthology.orchestra.run.smithi148.stdout:Upgraded: 2024-04-24T23:18:11.824 INFO:teuthology.orchestra.run.smithi148.stdout: librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:18:11.824 INFO:teuthology.orchestra.run.smithi148.stdout: librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:18:11.824 INFO:teuthology.orchestra.run.smithi148.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-24T23:18:11.824 INFO:teuthology.orchestra.run.smithi148.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-24T23:18:11.824 INFO:teuthology.orchestra.run.smithi148.stdout:Installed: 2024-04-24T23:18:11.824 INFO:teuthology.orchestra.run.smithi148.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-24T23:18:11.824 INFO:teuthology.orchestra.run.smithi148.stdout: automake-1.16.2-8.el9.noarch 2024-04-24T23:18:11.824 INFO:teuthology.orchestra.run.smithi148.stdout: bison-3.7.4-5.el9.x86_64 2024-04-24T23:18:11.824 INFO:teuthology.orchestra.run.smithi148.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-24T23:18:11.824 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:18:11.824 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:18:11.824 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:18:11.824 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:18:11.825 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:18:11.825 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-immutable-object-cache-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:18:11.825 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:18:11.825 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:18:11.825 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:18:11.825 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:18:11.825 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:18:11.825 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:18:11.825 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:18:11.825 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:18:11.825 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:18:11.825 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:18:11.825 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:18:11.826 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:18:11.826 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:18:11.826 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:18:11.826 INFO:teuthology.orchestra.run.smithi148.stdout: cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:18:11.826 INFO:teuthology.orchestra.run.smithi148.stdout: cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:18:11.826 INFO:teuthology.orchestra.run.smithi148.stdout: cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:18:11.826 INFO:teuthology.orchestra.run.smithi148.stdout: cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:18:11.826 INFO:teuthology.orchestra.run.smithi148.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-24T23:18:11.826 INFO:teuthology.orchestra.run.smithi148.stdout: flex-2.6.4-9.el9.x86_64 2024-04-24T23:18:11.826 INFO:teuthology.orchestra.run.smithi148.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-24T23:18:11.826 INFO:teuthology.orchestra.run.smithi148.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-24T23:18:11.826 INFO:teuthology.orchestra.run.smithi148.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-24T23:18:11.826 INFO:teuthology.orchestra.run.smithi148.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-24T23:18:11.827 INFO:teuthology.orchestra.run.smithi148.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-24T23:18:11.827 INFO:teuthology.orchestra.run.smithi148.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-24T23:18:11.827 INFO:teuthology.orchestra.run.smithi148.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-24T23:18:11.827 INFO:teuthology.orchestra.run.smithi148.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-24T23:18:11.827 INFO:teuthology.orchestra.run.smithi148.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-24T23:18:11.827 INFO:teuthology.orchestra.run.smithi148.stdout: libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:18:11.827 INFO:teuthology.orchestra.run.smithi148.stdout: libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:18:11.827 INFO:teuthology.orchestra.run.smithi148.stdout: libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:18:11.827 INFO:teuthology.orchestra.run.smithi148.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-24T23:18:11.827 INFO:teuthology.orchestra.run.smithi148.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-24T23:18:11.827 INFO:teuthology.orchestra.run.smithi148.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-24T23:18:11.827 INFO:teuthology.orchestra.run.smithi148.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-24T23:18:11.827 INFO:teuthology.orchestra.run.smithi148.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-24T23:18:11.827 INFO:teuthology.orchestra.run.smithi148.stdout: librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:18:11.828 INFO:teuthology.orchestra.run.smithi148.stdout: libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:18:11.828 INFO:teuthology.orchestra.run.smithi148.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-24T23:18:11.828 INFO:teuthology.orchestra.run.smithi148.stdout: librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:18:11.828 INFO:teuthology.orchestra.run.smithi148.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-24T23:18:11.828 INFO:teuthology.orchestra.run.smithi148.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-24T23:18:11.828 INFO:teuthology.orchestra.run.smithi148.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-24T23:18:11.828 INFO:teuthology.orchestra.run.smithi148.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-24T23:18:11.828 INFO:teuthology.orchestra.run.smithi148.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-24T23:18:11.828 INFO:teuthology.orchestra.run.smithi148.stdout: lua-5.4.4-4.el9.x86_64 2024-04-24T23:18:11.828 INFO:teuthology.orchestra.run.smithi148.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-24T23:18:11.828 INFO:teuthology.orchestra.run.smithi148.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-24T23:18:11.828 INFO:teuthology.orchestra.run.smithi148.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-24T23:18:11.828 INFO:teuthology.orchestra.run.smithi148.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-24T23:18:11.828 INFO:teuthology.orchestra.run.smithi148.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-24T23:18:11.828 INFO:teuthology.orchestra.run.smithi148.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-24T23:18:11.829 INFO:teuthology.orchestra.run.smithi148.stdout: postgresql-13.14-1.el9.x86_64 2024-04-24T23:18:11.829 INFO:teuthology.orchestra.run.smithi148.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-24T23:18:11.829 INFO:teuthology.orchestra.run.smithi148.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-24T23:18:11.829 INFO:teuthology.orchestra.run.smithi148.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-24T23:18:11.829 INFO:teuthology.orchestra.run.smithi148.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-24T23:18:11.829 INFO:teuthology.orchestra.run.smithi148.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-24T23:18:11.829 INFO:teuthology.orchestra.run.smithi148.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-24T23:18:11.829 INFO:teuthology.orchestra.run.smithi148.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-24T23:18:11.829 INFO:teuthology.orchestra.run.smithi148.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-24T23:18:11.829 INFO:teuthology.orchestra.run.smithi148.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-24T23:18:11.829 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-24T23:18:11.829 INFO:teuthology.orchestra.run.smithi148.stdout: python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:18:11.829 INFO:teuthology.orchestra.run.smithi148.stdout: python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:18:11.829 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:18:11.830 INFO:teuthology.orchestra.run.smithi148.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-24T23:18:11.830 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-24T23:18:11.830 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-24T23:18:11.830 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-24T23:18:11.830 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-24T23:18:11.830 INFO:teuthology.orchestra.run.smithi148.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-24T23:18:11.830 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-24T23:18:11.830 INFO:teuthology.orchestra.run.smithi148.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-24T23:18:11.830 INFO:teuthology.orchestra.run.smithi148.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-24T23:18:11.830 INFO:teuthology.orchestra.run.smithi148.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-24T23:18:11.830 INFO:teuthology.orchestra.run.smithi148.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-24T23:18:11.830 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-24T23:18:11.830 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-24T23:18:11.830 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-24T23:18:11.831 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-24T23:18:11.831 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-24T23:18:11.831 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-24T23:18:11.831 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-24T23:18:11.831 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-24T23:18:11.831 INFO:teuthology.orchestra.run.smithi148.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-24T23:18:11.831 INFO:teuthology.orchestra.run.smithi148.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-24T23:18:11.831 INFO:teuthology.orchestra.run.smithi148.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-24T23:18:11.831 INFO:teuthology.orchestra.run.smithi148.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-24T23:18:11.831 INFO:teuthology.orchestra.run.smithi148.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-24T23:18:11.831 INFO:teuthology.orchestra.run.smithi148.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-24T23:18:11.831 INFO:teuthology.orchestra.run.smithi148.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-24T23:18:11.831 INFO:teuthology.orchestra.run.smithi148.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-24T23:18:11.831 INFO:teuthology.orchestra.run.smithi148.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-24T23:18:11.831 INFO:teuthology.orchestra.run.smithi148.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-24T23:18:11.832 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-24T23:18:11.832 INFO:teuthology.orchestra.run.smithi148.stdout: python3-ply-3.11-14.el9.noarch 2024-04-24T23:18:11.832 INFO:teuthology.orchestra.run.smithi148.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-24T23:18:11.832 INFO:teuthology.orchestra.run.smithi148.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-24T23:18:11.832 INFO:teuthology.orchestra.run.smithi148.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-24T23:18:11.832 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-24T23:18:11.832 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-24T23:18:11.832 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-24T23:18:11.832 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-24T23:18:11.832 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-24T23:18:11.832 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-24T23:18:11.832 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:18:11.832 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:18:11.832 INFO:teuthology.orchestra.run.smithi148.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-24T23:18:11.832 INFO:teuthology.orchestra.run.smithi148.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-24T23:18:11.833 INFO:teuthology.orchestra.run.smithi148.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-24T23:18:11.833 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:18:11.833 INFO:teuthology.orchestra.run.smithi148.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-24T23:18:11.833 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-24T23:18:11.833 INFO:teuthology.orchestra.run.smithi148.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-24T23:18:11.833 INFO:teuthology.orchestra.run.smithi148.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-24T23:18:11.833 INFO:teuthology.orchestra.run.smithi148.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-24T23:18:11.833 INFO:teuthology.orchestra.run.smithi148.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-24T23:18:11.833 INFO:teuthology.orchestra.run.smithi148.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-24T23:18:11.833 INFO:teuthology.orchestra.run.smithi148.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-24T23:18:11.833 INFO:teuthology.orchestra.run.smithi148.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-24T23:18:11.833 INFO:teuthology.orchestra.run.smithi148.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-24T23:18:11.834 INFO:teuthology.orchestra.run.smithi148.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-24T23:18:11.834 INFO:teuthology.orchestra.run.smithi148.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-24T23:18:11.834 INFO:teuthology.orchestra.run.smithi148.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-24T23:18:11.834 INFO:teuthology.orchestra.run.smithi148.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-24T23:18:11.834 INFO:teuthology.orchestra.run.smithi148.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-24T23:18:11.834 INFO:teuthology.orchestra.run.smithi148.stdout: rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:18:11.834 INFO:teuthology.orchestra.run.smithi148.stdout: rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:18:11.834 INFO:teuthology.orchestra.run.smithi148.stdout: rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:18:11.834 INFO:teuthology.orchestra.run.smithi148.stdout: re2-1:20211101-3.el9.x86_64 2024-04-24T23:18:11.834 INFO:teuthology.orchestra.run.smithi148.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-24T23:18:11.834 INFO:teuthology.orchestra.run.smithi148.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-24T23:18:11.834 INFO:teuthology.orchestra.run.smithi148.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-24T23:18:11.834 INFO:teuthology.orchestra.run.smithi148.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-24T23:18:11.835 INFO:teuthology.orchestra.run.smithi148.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-24T23:18:11.835 INFO:teuthology.orchestra.run.smithi148.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-24T23:18:11.835 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-24T23:18:11.835 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-04-24T23:18:12.222 DEBUG:teuthology.parallel:result is None 2024-04-24T23:18:12.223 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T23:18:12.379 DEBUG:teuthology.orchestra.run.smithi080:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-24T23:18:12.405 INFO:teuthology.orchestra.run.smithi080.stdout:19.0.0-2445.gd2c45d97.el9 2024-04-24T23:18:12.406 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2445.gd2c45d97.el9 2024-04-24T23:18:12.406 INFO:teuthology.task.install:The correct ceph version 19.0.0-2445.gd2c45d97 is installed. 2024-04-24T23:18:12.408 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T23:18:12.563 DEBUG:teuthology.orchestra.run.smithi134:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-24T23:18:12.588 INFO:teuthology.orchestra.run.smithi134.stdout:19.0.0-2445.gd2c45d97.el9 2024-04-24T23:18:12.589 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2445.gd2c45d97.el9 2024-04-24T23:18:12.589 INFO:teuthology.task.install:The correct ceph version 19.0.0-2445.gd2c45d97 is installed. 2024-04-24T23:18:12.590 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T23:18:12.737 DEBUG:teuthology.orchestra.run.smithi148:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-24T23:18:12.760 INFO:teuthology.orchestra.run.smithi148.stdout:19.0.0-2445.gd2c45d97.el9 2024-04-24T23:18:12.760 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2445.gd2c45d97.el9 2024-04-24T23:18:12.760 INFO:teuthology.task.install:The correct ceph version 19.0.0-2445.gd2c45d97 is installed. 2024-04-24T23:18:12.762 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-24T23:18:12.762 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-24T23:18:12.762 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-24T23:18:12.792 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-24T23:18:12.792 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-24T23:18:12.823 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-24T23:18:12.823 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-24T23:18:12.849 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-24T23:18:12.849 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-24T23:18:12.850 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd of=/usr/bin/daemon-helper 2024-04-24T23:18:12.876 DEBUG:teuthology.orchestra.run.smithi080:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-24T23:18:12.941 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-24T23:18:12.941 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/usr/bin/daemon-helper 2024-04-24T23:18:12.970 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-24T23:18:13.037 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-24T23:18:13.037 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/usr/bin/daemon-helper 2024-04-24T23:18:13.062 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-24T23:18:13.126 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-24T23:18:13.126 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-24T23:18:13.126 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-24T23:18:13.151 DEBUG:teuthology.orchestra.run.smithi080:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-24T23:18:13.215 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-24T23:18:13.215 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-24T23:18:13.244 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-24T23:18:13.312 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-24T23:18:13.313 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-24T23:18:13.337 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-24T23:18:13.400 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-24T23:18:13.401 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-24T23:18:13.401 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd of=/usr/bin/stdin-killer 2024-04-24T23:18:13.425 DEBUG:teuthology.orchestra.run.smithi080:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-24T23:18:13.489 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-24T23:18:13.489 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/usr/bin/stdin-killer 2024-04-24T23:18:13.517 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-24T23:18:13.585 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-24T23:18:13.585 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/usr/bin/stdin-killer 2024-04-24T23:18:13.608 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-24T23:18:13.671 INFO:teuthology.run_tasks:Running task cephadm... 2024-04-24T23:18:13.763 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-04-24T23:18:13.763 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T23:18:13.764 INFO:tasks.cephadm:Cluster fsid is ec7885aa-0290-11ef-bc93-c7b262605968 2024-04-24T23:18:13.764 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-04-24T23:18:13.764 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.80', 'mon.b': '172.21.15.134', 'mon.c': '172.21.15.148'} 2024-04-24T23:18:13.764 INFO:tasks.cephadm:First mon is mon.a on smithi080 2024-04-24T23:18:13.764 INFO:tasks.cephadm:First mgr is x 2024-04-24T23:18:13.764 INFO:tasks.cephadm:Normalizing hostnames... 2024-04-24T23:18:13.765 DEBUG:teuthology.orchestra.run.smithi080:> sudo hostname $(hostname -s) 2024-04-24T23:18:13.790 DEBUG:teuthology.orchestra.run.smithi134:> sudo hostname $(hostname -s) 2024-04-24T23:18:13.818 DEBUG:teuthology.orchestra.run.smithi148:> sudo hostname $(hostname -s) 2024-04-24T23:18:13.841 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-04-24T23:18:13.841 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T23:18:13.990 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/54987/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-2445-gd2c45d97', 'node_name': '172.21.5.36+adami06', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-2445.gd2c45d97'}, 'url': 'https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-04-24 22:37:46.422067', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}, {'status': 'ready', 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/54981/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-2445-gd2c45d97', 'node_name': '172.21.5.36+adami06', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-2445.gd2c45d97'}, 'url': 'https://1.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-04-24 17:00:00.872117', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-04-24T23:18:14.139 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref squid, sha1 d2c45d977142e024a59b68a8f1dce8f0b9840551 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T23:18:14.140 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/x86_64/flavors/default/cephadm 2024-04-24T23:18:14.140 DEBUG:teuthology.orchestra.run.smithi080:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-24T23:18:14.555 INFO:teuthology.orchestra.run.smithi080.stdout:-rw-r--r--. 1 ubuntu ubuntu 781409 Apr 24 23:18 /home/ubuntu/cephtest/cephadm 2024-04-24T23:18:14.556 DEBUG:teuthology.orchestra.run.smithi134:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-24T23:18:14.937 INFO:teuthology.orchestra.run.smithi134.stdout:-rw-r--r--. 1 ubuntu ubuntu 781409 Apr 24 23:18 /home/ubuntu/cephtest/cephadm 2024-04-24T23:18:14.937 DEBUG:teuthology.orchestra.run.smithi148:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-24T23:18:15.345 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r--. 1 ubuntu ubuntu 781409 Apr 24 23:18 /home/ubuntu/cephtest/cephadm 2024-04-24T23:18:15.345 DEBUG:teuthology.orchestra.run.smithi080:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-24T23:18:15.363 DEBUG:teuthology.orchestra.run.smithi134:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-24T23:18:15.382 DEBUG:teuthology.orchestra.run.smithi148:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-24T23:18:15.415 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 on all hosts... 2024-04-24T23:18:15.415 DEBUG:teuthology.orchestra.run.smithi080:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 pull 2024-04-24T23:18:15.418 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 pull 2024-04-24T23:18:15.425 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 pull 2024-04-24T23:18:17.012 INFO:teuthology.orchestra.run.smithi080.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551... 2024-04-24T23:18:17.019 INFO:teuthology.orchestra.run.smithi134.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551... 2024-04-24T23:18:17.136 INFO:teuthology.orchestra.run.smithi148.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551... 2024-04-24T23:18:45.121 INFO:teuthology.orchestra.run.smithi080.stdout:{ 2024-04-24T23:18:45.121 INFO:teuthology.orchestra.run.smithi080.stdout: "ceph_version": "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)", 2024-04-24T23:18:45.121 INFO:teuthology.orchestra.run.smithi080.stdout: "image_id": "f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e", 2024-04-24T23:18:45.122 INFO:teuthology.orchestra.run.smithi080.stdout: "repo_digests": [ 2024-04-24T23:18:45.122 INFO:teuthology.orchestra.run.smithi080.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901" 2024-04-24T23:18:45.122 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-24T23:18:45.122 INFO:teuthology.orchestra.run.smithi080.stdout:} 2024-04-24T23:18:46.413 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-24T23:18:46.413 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_version": "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)", 2024-04-24T23:18:46.413 INFO:teuthology.orchestra.run.smithi148.stdout: "image_id": "f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e", 2024-04-24T23:18:46.414 INFO:teuthology.orchestra.run.smithi148.stdout: "repo_digests": [ 2024-04-24T23:18:46.414 INFO:teuthology.orchestra.run.smithi148.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901" 2024-04-24T23:18:46.414 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-04-24T23:18:46.414 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-24T23:18:52.614 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-04-24T23:18:52.614 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_version": "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)", 2024-04-24T23:18:52.615 INFO:teuthology.orchestra.run.smithi134.stdout: "image_id": "f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e", 2024-04-24T23:18:52.615 INFO:teuthology.orchestra.run.smithi134.stdout: "repo_digests": [ 2024-04-24T23:18:52.615 INFO:teuthology.orchestra.run.smithi134.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901" 2024-04-24T23:18:52.615 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-04-24T23:18:52.615 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-04-24T23:18:52.640 DEBUG:teuthology.orchestra.run.smithi080:> sudo mkdir -p /etc/ceph 2024-04-24T23:18:52.672 DEBUG:teuthology.orchestra.run.smithi134:> sudo mkdir -p /etc/ceph 2024-04-24T23:18:52.699 DEBUG:teuthology.orchestra.run.smithi148:> sudo mkdir -p /etc/ceph 2024-04-24T23:18:52.731 DEBUG:teuthology.orchestra.run.smithi080:> sudo chmod 777 /etc/ceph 2024-04-24T23:18:52.758 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod 777 /etc/ceph 2024-04-24T23:18:52.781 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod 777 /etc/ceph 2024-04-24T23:18:52.809 INFO:tasks.cephadm:Writing seed config... 2024-04-24T23:18:52.810 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-04-24T23:18:52.810 INFO:tasks.cephadm: override: [client] debug client = 20 2024-04-24T23:18:52.810 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-04-24T23:18:52.810 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-04-24T23:18:52.810 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-04-24T23:18:52.810 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-04-24T23:18:52.811 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-04-24T23:18:52.811 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-04-24T23:18:52.811 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-04-24T23:18:52.811 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-04-24T23:18:52.811 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-04-24T23:18:52.811 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-04-24T23:18:52.811 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-04-24T23:18:52.811 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-04-24T23:18:52.811 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-04-24T23:18:52.811 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-04-24T23:18:52.811 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-04-24T23:18:52.811 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-04-24T23:18:52.811 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-04-24T23:18:52.811 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-04-24T23:18:52.811 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-04-24T23:18:52.811 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-04-24T23:18:52.812 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-04-24T23:18:52.812 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-04-24T23:18:52.812 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-04-24T23:18:52.812 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-04-24T23:18:52.812 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-04-24T23:18:52.812 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-04-24T23:18:52.812 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-04-24T23:18:52.812 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-04-24T23:18:52.812 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-04-24T23:18:52.812 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-04-24T23:18:52.812 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-04-24T23:18:52.812 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-04-24T23:18:52.812 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-04-24T23:18:52.812 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-04-24T23:18:52.812 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-04-24T23:18:52.812 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-04-24T23:18:52.812 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-04-24T23:18:52.813 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-04-24T23:18:52.813 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-04-24T23:18:52.813 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-04-24T23:18:52.813 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-04-24T23:18:52.813 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-04-24T23:18:52.813 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-04-24T23:18:52.814 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-24T23:18:52.814 DEBUG:teuthology.orchestra.run.smithi080:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-04-24T23:18:52.831 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = ec7885aa-0290-11ef-bc93-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-04-24T23:18:52.832 DEBUG:teuthology.orchestra.run.smithi080:mon.a> sudo journalctl -f -n 0 -u ceph-ec7885aa-0290-11ef-bc93-c7b262605968@mon.a.service 2024-04-24T23:18:52.874 DEBUG:teuthology.orchestra.run.smithi080:mgr.x> sudo journalctl -f -n 0 -u ceph-ec7885aa-0290-11ef-bc93-c7b262605968@mgr.x.service 2024-04-24T23:18:52.917 INFO:tasks.cephadm:Bootstrapping... 2024-04-24T23:18:52.917 DEBUG:teuthology.orchestra.run.smithi080:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 -v bootstrap --fsid ec7885aa-0290-11ef-bc93-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.80 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-04-24T23:18:53.116 INFO:teuthology.orchestra.run.smithi080.stdout:-------------------------------------------------------------------------------- 2024-04-24T23:18:53.116 INFO:teuthology.orchestra.run.smithi080.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551', '-v', 'bootstrap', '--fsid', 'ec7885aa-0290-11ef-bc93-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.80', '--skip-admin-label'] 2024-04-24T23:18:53.117 INFO:teuthology.orchestra.run.smithi080.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-04-24T23:18:53.117 INFO:teuthology.orchestra.run.smithi080.stdout:Verifying podman|docker is present... 2024-04-24T23:18:53.142 INFO:teuthology.orchestra.run.smithi080.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-24T23:18:53.142 INFO:teuthology.orchestra.run.smithi080.stdout:Verifying lvm2 is present... 2024-04-24T23:18:53.142 INFO:teuthology.orchestra.run.smithi080.stdout:Verifying time synchronization is in place... 2024-04-24T23:18:53.150 INFO:teuthology.orchestra.run.smithi080.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-24T23:18:53.150 INFO:teuthology.orchestra.run.smithi080.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-24T23:18:53.156 INFO:teuthology.orchestra.run.smithi080.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-24T23:18:53.156 INFO:teuthology.orchestra.run.smithi080.stdout:systemctl: stdout inactive 2024-04-24T23:18:53.162 INFO:teuthology.orchestra.run.smithi080.stdout:systemctl: stdout enabled 2024-04-24T23:18:53.168 INFO:teuthology.orchestra.run.smithi080.stdout:systemctl: stdout active 2024-04-24T23:18:53.168 INFO:teuthology.orchestra.run.smithi080.stdout:Unit chronyd.service is enabled and running 2024-04-24T23:18:53.169 INFO:teuthology.orchestra.run.smithi080.stdout:Repeating the final host check... 2024-04-24T23:18:53.192 INFO:teuthology.orchestra.run.smithi080.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-24T23:18:53.193 INFO:teuthology.orchestra.run.smithi080.stdout:podman (/bin/podman) version 4.9.4 is present 2024-04-24T23:18:53.193 INFO:teuthology.orchestra.run.smithi080.stdout:systemctl is present 2024-04-24T23:18:53.193 INFO:teuthology.orchestra.run.smithi080.stdout:lvcreate is present 2024-04-24T23:18:53.199 INFO:teuthology.orchestra.run.smithi080.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-24T23:18:53.199 INFO:teuthology.orchestra.run.smithi080.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-24T23:18:53.205 INFO:teuthology.orchestra.run.smithi080.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-24T23:18:53.205 INFO:teuthology.orchestra.run.smithi080.stdout:systemctl: stdout inactive 2024-04-24T23:18:53.211 INFO:teuthology.orchestra.run.smithi080.stdout:systemctl: stdout enabled 2024-04-24T23:18:53.217 INFO:teuthology.orchestra.run.smithi080.stdout:systemctl: stdout active 2024-04-24T23:18:53.217 INFO:teuthology.orchestra.run.smithi080.stdout:Unit chronyd.service is enabled and running 2024-04-24T23:18:53.217 INFO:teuthology.orchestra.run.smithi080.stdout:Host looks OK 2024-04-24T23:18:53.217 INFO:teuthology.orchestra.run.smithi080.stdout:Cluster fsid: ec7885aa-0290-11ef-bc93-c7b262605968 2024-04-24T23:18:53.217 INFO:teuthology.orchestra.run.smithi080.stdout:Acquiring lock 140655544852000 on /run/cephadm/ec7885aa-0290-11ef-bc93-c7b262605968.lock 2024-04-24T23:18:53.217 INFO:teuthology.orchestra.run.smithi080.stdout:Lock 140655544852000 acquired on /run/cephadm/ec7885aa-0290-11ef-bc93-c7b262605968.lock 2024-04-24T23:18:53.218 INFO:teuthology.orchestra.run.smithi080.stdout:Verifying IP 172.21.15.80 port 3300 ... 2024-04-24T23:18:53.218 INFO:teuthology.orchestra.run.smithi080.stdout:Verifying IP 172.21.15.80 port 6789 ... 2024-04-24T23:18:53.218 INFO:teuthology.orchestra.run.smithi080.stdout:Base mon IP(s) is [172.21.15.80:3300, 172.21.15.80:6789], mon addrv is [v2:172.21.15.80:3300,v1:172.21.15.80:6789] 2024-04-24T23:18:53.221 INFO:teuthology.orchestra.run.smithi080.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.80 metric 100 2024-04-24T23:18:53.221 INFO:teuthology.orchestra.run.smithi080.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.80 metric 100 2024-04-24T23:18:53.224 INFO:teuthology.orchestra.run.smithi080.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-04-24T23:18:53.224 INFO:teuthology.orchestra.run.smithi080.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2024-04-24T23:18:53.224 INFO:teuthology.orchestra.run.smithi080.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2024-04-24T23:18:53.226 INFO:teuthology.orchestra.run.smithi080.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-04-24T23:18:53.226 INFO:teuthology.orchestra.run.smithi080.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-04-24T23:18:53.226 INFO:teuthology.orchestra.run.smithi080.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-24T23:18:53.226 INFO:teuthology.orchestra.run.smithi080.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-04-24T23:18:53.227 INFO:teuthology.orchestra.run.smithi080.stdout:/sbin/ip: stdout inet6 fe80::477b:93ee:6d5c:3d53/64 scope link noprefixroute 2024-04-24T23:18:53.227 INFO:teuthology.orchestra.run.smithi080.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-24T23:18:53.227 INFO:teuthology.orchestra.run.smithi080.stdout:Mon IP `172.21.15.80` is in CIDR network `172.21.0.0/20` 2024-04-24T23:18:53.227 INFO:teuthology.orchestra.run.smithi080.stdout:Mon IP `172.21.15.80` is in CIDR network `172.21.0.0/20` 2024-04-24T23:18:53.227 INFO:teuthology.orchestra.run.smithi080.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-04-24T23:18:53.228 INFO:teuthology.orchestra.run.smithi080.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-04-24T23:18:53.228 INFO:teuthology.orchestra.run.smithi080.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551... 2024-04-24T23:18:53.704 INFO:teuthology.orchestra.run.smithi080.stdout:/bin/podman: stdout f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e 2024-04-24T23:18:53.704 INFO:teuthology.orchestra.run.smithi080.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551... 2024-04-24T23:18:53.705 INFO:teuthology.orchestra.run.smithi080.stdout:/bin/podman: stderr Getting image source signatures 2024-04-24T23:18:53.705 INFO:teuthology.orchestra.run.smithi080.stdout:/bin/podman: stderr Copying blob sha256:58a99251f41e8a467bafa0ec849428709568906ca732f10972304f37056be6c5 2024-04-24T23:18:53.705 INFO:teuthology.orchestra.run.smithi080.stdout:/bin/podman: stderr Copying blob sha256:3d665f4c032644be9cae748af04bee69f2bfe96cdc3616dabe0ab17899633ab5 2024-04-24T23:18:53.705 INFO:teuthology.orchestra.run.smithi080.stdout:/bin/podman: stderr Copying config sha256:f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e 2024-04-24T23:18:53.705 INFO:teuthology.orchestra.run.smithi080.stdout:/bin/podman: stderr Writing manifest to image destination 2024-04-24T23:18:54.650 INFO:teuthology.orchestra.run.smithi080.stdout:ceph: stdout ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev) 2024-04-24T23:18:54.650 INFO:teuthology.orchestra.run.smithi080.stdout:Ceph version: ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev) 2024-04-24T23:18:54.650 INFO:teuthology.orchestra.run.smithi080.stdout:Extracting ceph user uid/gid from container image... 2024-04-24T23:18:55.614 INFO:teuthology.orchestra.run.smithi080.stdout:stat: stdout 167 167 2024-04-24T23:18:55.614 INFO:teuthology.orchestra.run.smithi080.stdout:Creating initial keys... 2024-04-24T23:18:56.695 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph-authtool: stdout AQBfkylmbyxoNxAACpNFnAnpYaADNm910z/+gA== 2024-04-24T23:18:57.617 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph-authtool: stdout AQBhkylm2V8qABAACCNqqVgBVXY9Dch0r4DFDg== 2024-04-24T23:18:58.493 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph-authtool: stdout AQBhkylmWH5iNxAA16l3/ycyNMZ2thWhClVCSA== 2024-04-24T23:18:58.494 INFO:teuthology.orchestra.run.smithi080.stdout:Creating initial monmap... 2024-04-24T23:18:59.434 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-24T23:18:59.435 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-04-24T23:18:59.435 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to ec7885aa-0290-11ef-bc93-c7b262605968 2024-04-24T23:18:59.435 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-24T23:18:59.435 INFO:teuthology.orchestra.run.smithi080.stdout:monmaptool for a [v2:172.21.15.80:3300,v1:172.21.15.80:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-24T23:18:59.435 INFO:teuthology.orchestra.run.smithi080.stdout:setting min_mon_release = quincy 2024-04-24T23:18:59.435 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/monmaptool: set fsid to ec7885aa-0290-11ef-bc93-c7b262605968 2024-04-24T23:18:59.435 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-24T23:18:59.435 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-24T23:18:59.435 INFO:teuthology.orchestra.run.smithi080.stdout:Creating mon... 2024-04-24T23:19:00.502 INFO:teuthology.orchestra.run.smithi080.stdout:create mon.a on 2024-04-24T23:19:00.693 INFO:teuthology.orchestra.run.smithi080.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-04-24T23:19:00.880 INFO:teuthology.orchestra.run.smithi080.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-04-24T23:19:01.273 INFO:teuthology.orchestra.run.smithi080.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-ec7885aa-0290-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-ec7885aa-0290-11ef-bc93-c7b262605968.target. 2024-04-24T23:19:01.273 INFO:teuthology.orchestra.run.smithi080.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-ec7885aa-0290-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-ec7885aa-0290-11ef-bc93-c7b262605968.target. 2024-04-24T23:19:02.117 INFO:teuthology.orchestra.run.smithi080.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-ec7885aa-0290-11ef-bc93-c7b262605968@mon.a 2024-04-24T23:19:02.117 INFO:teuthology.orchestra.run.smithi080.stdout:systemctl: stderr Failed to reset failed state of unit ceph-ec7885aa-0290-11ef-bc93-c7b262605968@mon.a.service: Unit ceph-ec7885aa-0290-11ef-bc93-c7b262605968@mon.a.service not loaded. 2024-04-24T23:19:02.384 INFO:teuthology.orchestra.run.smithi080.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-ec7885aa-0290-11ef-bc93-c7b262605968.target.wants/ceph-ec7885aa-0290-11ef-bc93-c7b262605968@mon.a.service → /etc/systemd/system/ceph-ec7885aa-0290-11ef-bc93-c7b262605968@.service. 2024-04-24T23:19:03.040 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:02 smithi080 podman[28763]: 2024-04-24 23:19:02.687296896 +0000 UTC m=+0.017928637 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T23:19:04.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:03 smithi080 podman[28763]: 2024-04-24 23:19:03.720564665 +0000 UTC m=+1.051196400 container create 3c65024875ee8831d9df16a6a3f0e13ee336c36645a6be4e9f5789e2b0a8a460 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mon-a, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, ceph=True, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T23:19:05.790 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:05 smithi080 podman[28763]: 2024-04-24 23:19:05.538210594 +0000 UTC m=+2.868842331 container init 3c65024875ee8831d9df16a6a3f0e13ee336c36645a6be4e9f5789e2b0a8a460 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mon-a, ceph=True, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424) 2024-04-24T23:19:05.790 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:05 smithi080 podman[28763]: 2024-04-24 23:19:05.542008562 +0000 UTC m=+2.872640298 container start 3c65024875ee8831d9df16a6a3f0e13ee336c36645a6be4e9f5789e2b0a8a460 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mon-a, io.buildah.version=1.33.5, ceph=True, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-04-24T23:19:05.790 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:05 smithi080 ceph-mon[28781]: mkfs ec7885aa-0290-11ef-bc93-c7b262605968 2024-04-24T23:19:05.790 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:05 smithi080 ceph-mon[28781]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-24T23:19:06.481 INFO:teuthology.orchestra.run.smithi080.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-24T23:19:06.481 INFO:teuthology.orchestra.run.smithi080.stdout:systemctl: stdout disabled 2024-04-24T23:19:06.489 INFO:teuthology.orchestra.run.smithi080.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-24T23:19:06.489 INFO:teuthology.orchestra.run.smithi080.stdout:systemctl: stdout inactive 2024-04-24T23:19:06.489 INFO:teuthology.orchestra.run.smithi080.stdout:firewalld.service is not enabled 2024-04-24T23:19:06.489 INFO:teuthology.orchestra.run.smithi080.stdout:Not possible to enable service . firewalld.service is not available 2024-04-24T23:19:06.490 INFO:teuthology.orchestra.run.smithi080.stdout:Waiting for mon to start... 2024-04-24T23:19:06.490 INFO:teuthology.orchestra.run.smithi080.stdout:Waiting for mon... 2024-04-24T23:19:06.790 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:06 smithi080 bash[28763]: 3c65024875ee8831d9df16a6a3f0e13ee336c36645a6be4e9f5789e2b0a8a460 2024-04-24T23:19:06.790 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:06 smithi080 systemd[1]: Started Ceph mon.a for ec7885aa-0290-11ef-bc93-c7b262605968. 2024-04-24T23:19:06.790 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:06 smithi080 ceph-mon[28781]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-24T23:19:06.790 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:06 smithi080 ceph-mon[28781]: monmap epoch 1 2024-04-24T23:19:06.790 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:06 smithi080 ceph-mon[28781]: fsid ec7885aa-0290-11ef-bc93-c7b262605968 2024-04-24T23:19:06.790 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:06 smithi080 ceph-mon[28781]: last_changed 2024-04-24T23:18:58.816011+0000 2024-04-24T23:19:06.790 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:06 smithi080 ceph-mon[28781]: created 2024-04-24T23:18:58.816011+0000 2024-04-24T23:19:06.790 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:06 smithi080 ceph-mon[28781]: min_mon_release 19 (squid) 2024-04-24T23:19:06.790 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:06 smithi080 ceph-mon[28781]: election_strategy: 1 2024-04-24T23:19:06.791 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:06 smithi080 ceph-mon[28781]: 0: [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] mon.a 2024-04-24T23:19:06.791 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:06 smithi080 ceph-mon[28781]: fsmap 2024-04-24T23:19:06.791 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:06 smithi080 ceph-mon[28781]: osdmap e1: 0 total, 0 up, 0 in 2024-04-24T23:19:06.791 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:06 smithi080 ceph-mon[28781]: mgrmap e1: no daemons active 2024-04-24T23:19:10.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:09 smithi080 ceph-mon[28781]: from='client.? 172.21.15.80:0/966285899' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-04-24T23:19:14.461 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout cluster: 2024-04-24T23:19:14.467 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout id: ec7885aa-0290-11ef-bc93-c7b262605968 2024-04-24T23:19:14.468 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-04-24T23:19:14.468 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout 2024-04-24T23:19:14.468 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout services: 2024-04-24T23:19:14.468 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 4s) 2024-04-24T23:19:14.468 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-04-24T23:19:14.468 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-04-24T23:19:14.468 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout 2024-04-24T23:19:14.468 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout data: 2024-04-24T23:19:14.468 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-04-24T23:19:14.468 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-04-24T23:19:14.468 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-04-24T23:19:14.468 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout pgs: 2024-04-24T23:19:14.468 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout 2024-04-24T23:19:14.469 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:09.710+0000 7f9bdddc6700 1 Processor -- start 2024-04-24T23:19:14.469 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:09.710+0000 7f9bdddc6700 1 -- start start 2024-04-24T23:19:14.469 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:09.710+0000 7f9bdddc6700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9bd8107bd0 0x7f9bd8107ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:14.469 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:09.710+0000 7f9bdddc6700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd81085c0 con 0x7f9bd8107bd0 2024-04-24T23:19:14.469 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:09.710+0000 7f9bd77fe700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9bd8107bd0 0x7f9bd8107ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:14.469 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:09.710+0000 7f9bd77fe700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9bd8107bd0 0x7f9bd8107ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:56746/0 (socket says 172.21.15.80:56746) 2024-04-24T23:19:14.469 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:09.710+0000 7f9bd77fe700 1 -- 172.21.15.80:0/240078840 learned_addr learned my addr 172.21.15.80:0/240078840 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:19:14.469 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:09.711+0000 7f9bd77fe700 1 -- 172.21.15.80:0/240078840 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bd8108d80 con 0x7f9bd8107bd0 2024-04-24T23:19:14.469 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:09.711+0000 7f9bd77fe700 1 --2- 172.21.15.80:0/240078840 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9bd8107bd0 0x7f9bd8107ff0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f9bd0009e80 tx=0x7f9bd000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b3ee6306ffbb5e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:14.469 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:09.711+0000 7f9bd67fc700 1 -- 172.21.15.80:0/240078840 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9bd0017070 con 0x7f9bd8107bd0 2024-04-24T23:19:14.469 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:09.711+0000 7f9bd67fc700 1 -- 172.21.15.80:0/240078840 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9bd0003a20 con 0x7f9bd8107bd0 2024-04-24T23:19:14.469 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:09.711+0000 7f9bd67fc700 1 -- 172.21.15.80:0/240078840 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9bd0003d20 con 0x7f9bd8107bd0 2024-04-24T23:19:14.469 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:09.711+0000 7f9bdddc6700 1 -- 172.21.15.80:0/240078840 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9bd8107bd0 msgr2=0x7f9bd8107ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:14.469 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:09.711+0000 7f9bdddc6700 1 --2- 172.21.15.80:0/240078840 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9bd8107bd0 0x7f9bd8107ff0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f9bd0009e80 tx=0x7f9bd000b330 comp rx=0 tx=0).stop 2024-04-24T23:19:14.470 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:09.711+0000 7f9bdddc6700 1 -- 172.21.15.80:0/240078840 shutdown_connections 2024-04-24T23:19:14.470 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:09.711+0000 7f9bdddc6700 1 --2- 172.21.15.80:0/240078840 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9bd8107bd0 0x7f9bd8107ff0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:14.470 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:09.711+0000 7f9bdddc6700 1 -- 172.21.15.80:0/240078840 >> 172.21.15.80:0/240078840 conn(0x7f9bd8103170 msgr2=0x7f9bd81055b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:14.470 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:09.712+0000 7f9bdddc6700 1 -- 172.21.15.80:0/240078840 shutdown_connections 2024-04-24T23:19:14.470 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:09.712+0000 7f9bdddc6700 1 -- 172.21.15.80:0/240078840 wait complete. 2024-04-24T23:19:14.470 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:09.712+0000 7f9bdddc6700 1 Processor -- start 2024-04-24T23:19:14.470 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:09.712+0000 7f9bdddc6700 1 -- start start 2024-04-24T23:19:14.470 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:09.712+0000 7f9bdddc6700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9bd8107bd0 0x7f9bd819b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:14.470 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:09.712+0000 7f9bdddc6700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd81092b0 con 0x7f9bd8107bd0 2024-04-24T23:19:14.470 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:09.713+0000 7f9bd77fe700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9bd8107bd0 0x7f9bd819b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:14.470 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:09.713+0000 7f9bd77fe700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9bd8107bd0 0x7f9bd819b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:56748/0 (socket says 172.21.15.80:56748) 2024-04-24T23:19:14.470 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:09.713+0000 7f9bd77fe700 1 -- 172.21.15.80:0/966285899 learned_addr learned my addr 172.21.15.80:0/966285899 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:19:14.470 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:09.713+0000 7f9bd77fe700 1 -- 172.21.15.80:0/966285899 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bd819b8d0 con 0x7f9bd8107bd0 2024-04-24T23:19:14.471 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:09.713+0000 7f9bd77fe700 1 --2- 172.21.15.80:0/966285899 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9bd8107bd0 0x7f9bd819b390 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f9bd0004000 tx=0x7f9bd0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:14.471 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:09.713+0000 7f9bd4ff9700 1 -- 172.21.15.80:0/966285899 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9bd0022020 con 0x7f9bd8107bd0 2024-04-24T23:19:14.471 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:09.713+0000 7f9bd4ff9700 1 -- 172.21.15.80:0/966285899 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9bd0018040 con 0x7f9bd8107bd0 2024-04-24T23:19:14.471 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:09.713+0000 7f9bd4ff9700 1 -- 172.21.15.80:0/966285899 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9bd0017070 con 0x7f9bd8107bd0 2024-04-24T23:19:14.471 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:09.713+0000 7f9bdddc6700 1 -- 172.21.15.80:0/966285899 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bd819bb60 con 0x7f9bd8107bd0 2024-04-24T23:19:14.471 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:09.713+0000 7f9bdddc6700 1 -- 172.21.15.80:0/966285899 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bd819e800 con 0x7f9bd8107bd0 2024-04-24T23:19:14.471 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:09.713+0000 7f9bd4ff9700 1 -- 172.21.15.80:0/966285899 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f9bd0025020 con 0x7f9bd8107bd0 2024-04-24T23:19:14.471 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:09.713+0000 7f9bd4ff9700 1 -- 172.21.15.80:0/966285899 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f9bd000f050 con 0x7f9bd8107bd0 2024-04-24T23:19:14.471 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:09.714+0000 7f9bdddc6700 1 -- 172.21.15.80:0/966285899 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9bd8063890 con 0x7f9bd8107bd0 2024-04-24T23:19:14.471 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:09.716+0000 7f9bd4ff9700 1 -- 172.21.15.80:0/966285899 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f9bd002c030 con 0x7f9bd8107bd0 2024-04-24T23:19:14.471 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:09.786+0000 7f9bdddc6700 1 -- 172.21.15.80:0/966285899 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f9bd802dcc0 con 0x7f9bd8107bd0 2024-04-24T23:19:14.471 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:09.786+0000 7f9bd4ff9700 1 -- 172.21.15.80:0/966285899 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+310 (secure 0 0 0) 0x7f9bd001e480 con 0x7f9bd8107bd0 2024-04-24T23:19:14.471 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:09.787+0000 7f9bdddc6700 1 -- 172.21.15.80:0/966285899 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9bd8107bd0 msgr2=0x7f9bd819b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:14.472 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:09.787+0000 7f9bdddc6700 1 --2- 172.21.15.80:0/966285899 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9bd8107bd0 0x7f9bd819b390 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f9bd0004000 tx=0x7f9bd0004ad0 comp rx=0 tx=0).stop 2024-04-24T23:19:14.472 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:09.787+0000 7f9bdddc6700 1 -- 172.21.15.80:0/966285899 shutdown_connections 2024-04-24T23:19:14.472 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:09.787+0000 7f9bdddc6700 1 --2- 172.21.15.80:0/966285899 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9bd8107bd0 0x7f9bd819b390 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:14.472 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:09.787+0000 7f9bdddc6700 1 -- 172.21.15.80:0/966285899 >> 172.21.15.80:0/966285899 conn(0x7f9bd8103170 msgr2=0x7f9bd8191fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:14.472 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:09.787+0000 7f9bdddc6700 1 -- 172.21.15.80:0/966285899 shutdown_connections 2024-04-24T23:19:14.472 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:09.787+0000 7f9bdddc6700 1 -- 172.21.15.80:0/966285899 wait complete. 2024-04-24T23:19:14.472 INFO:teuthology.orchestra.run.smithi080.stdout:mon is available 2024-04-24T23:19:14.472 INFO:teuthology.orchestra.run.smithi080.stdout:Assimilating anything we can from ceph.conf... 2024-04-24T23:19:15.238 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:15 smithi080 ceph-mon[28781]: from='client.? 172.21.15.80:0/1410042348' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-24T23:19:15.239 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:15 smithi080 ceph-mon[28781]: from='client.? 172.21.15.80:0/1410042348' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-04-24T23:19:15.531 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout 2024-04-24T23:19:15.531 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout [global] 2024-04-24T23:19:15.531 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout fsid = ec7885aa-0290-11ef-bc93-c7b262605968 2024-04-24T23:19:15.531 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.80:3300,v1:172.21.15.80:6789] 2024-04-24T23:19:15.532 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-24T23:19:15.532 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-24T23:19:15.532 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-24T23:19:15.532 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-24T23:19:15.532 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout 2024-04-24T23:19:15.532 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-24T23:19:15.532 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-24T23:19:15.532 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout 2024-04-24T23:19:15.532 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout [osd] 2024-04-24T23:19:15.532 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-24T23:19:15.532 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-24T23:19:15.532 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-24T23:19:15.532 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-24T23:19:15.532 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-24T23:19:15.533 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:14.910+0000 7f78142fd700 1 Processor -- start 2024-04-24T23:19:15.533 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:14.910+0000 7f78142fd700 1 -- start start 2024-04-24T23:19:15.533 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:14.911+0000 7f78142fd700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f780c1081a0 0x7f780c1085c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:15.533 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:14.911+0000 7f78142fd700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f780c1066e0 con 0x7f780c1081a0 2024-04-24T23:19:15.533 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:14.911+0000 7f7812099700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f780c1081a0 0x7f780c1085c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:15.533 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:14.911+0000 7f7812099700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f780c1081a0 0x7f780c1085c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:51480/0 (socket says 172.21.15.80:51480) 2024-04-24T23:19:15.533 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:14.911+0000 7f7812099700 1 -- 172.21.15.80:0/187945430 learned_addr learned my addr 172.21.15.80:0/187945430 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:19:15.533 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:14.911+0000 7f7812099700 1 -- 172.21.15.80:0/187945430 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f780c106860 con 0x7f780c1081a0 2024-04-24T23:19:15.533 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:14.911+0000 7f7812099700 1 --2- 172.21.15.80:0/187945430 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f780c1081a0 0x7f780c1085c0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f77fc00bb40 tx=0x7f77fc00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d972097e35fa121 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:15.533 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:14.911+0000 7f7811097700 1 -- 172.21.15.80:0/187945430 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f77fc00dbe0 con 0x7f780c1081a0 2024-04-24T23:19:15.533 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:14.911+0000 7f7811097700 1 -- 172.21.15.80:0/187945430 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77fc00dd80 con 0x7f780c1081a0 2024-04-24T23:19:15.533 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:14.911+0000 7f7811097700 1 -- 172.21.15.80:0/187945430 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f77fc004180 con 0x7f780c1081a0 2024-04-24T23:19:15.533 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:14.912+0000 7f78142fd700 1 -- 172.21.15.80:0/187945430 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f780c1081a0 msgr2=0x7f780c1085c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:15.534 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:14.912+0000 7f78142fd700 1 --2- 172.21.15.80:0/187945430 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f780c1081a0 0x7f780c1085c0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f77fc00bb40 tx=0x7f77fc00be50 comp rx=0 tx=0).stop 2024-04-24T23:19:15.534 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:14.912+0000 7f78142fd700 1 -- 172.21.15.80:0/187945430 shutdown_connections 2024-04-24T23:19:15.534 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:14.912+0000 7f78142fd700 1 --2- 172.21.15.80:0/187945430 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f780c1081a0 0x7f780c1085c0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:15.534 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:14.912+0000 7f78142fd700 1 -- 172.21.15.80:0/187945430 >> 172.21.15.80:0/187945430 conn(0x7f780c102040 msgr2=0x7f780c1044a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:15.534 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:14.912+0000 7f78142fd700 1 -- 172.21.15.80:0/187945430 shutdown_connections 2024-04-24T23:19:15.534 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:14.912+0000 7f78142fd700 1 -- 172.21.15.80:0/187945430 wait complete. 2024-04-24T23:19:15.534 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:14.913+0000 7f78142fd700 1 Processor -- start 2024-04-24T23:19:15.534 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:14.913+0000 7f78142fd700 1 -- start start 2024-04-24T23:19:15.534 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:14.913+0000 7f78142fd700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f780c1081a0 0x7f780c199210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:15.534 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:14.913+0000 7f78142fd700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f780c106c70 con 0x7f780c1081a0 2024-04-24T23:19:15.534 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:14.913+0000 7f7812099700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f780c1081a0 0x7f780c199210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:15.534 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:14.913+0000 7f7812099700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f780c1081a0 0x7f780c199210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:51482/0 (socket says 172.21.15.80:51482) 2024-04-24T23:19:15.534 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:14.913+0000 7f7812099700 1 -- 172.21.15.80:0/1410042348 learned_addr learned my addr 172.21.15.80:0/1410042348 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:19:15.535 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:14.913+0000 7f7812099700 1 -- 172.21.15.80:0/1410042348 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f780c199750 con 0x7f780c1081a0 2024-04-24T23:19:15.535 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:14.913+0000 7f7812099700 1 --2- 172.21.15.80:0/1410042348 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f780c1081a0 0x7f780c199210 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f77fc00b340 tx=0x7f77fc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:15.535 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:14.913+0000 7f78037fe700 1 -- 172.21.15.80:0/1410042348 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f77fc029070 con 0x7f780c1081a0 2024-04-24T23:19:15.535 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:14.913+0000 7f78037fe700 1 -- 172.21.15.80:0/1410042348 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77fc0044e0 con 0x7f780c1081a0 2024-04-24T23:19:15.535 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:14.913+0000 7f78142fd700 1 -- 172.21.15.80:0/1410042348 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f780c1999e0 con 0x7f780c1081a0 2024-04-24T23:19:15.535 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:14.913+0000 7f78037fe700 1 -- 172.21.15.80:0/1410042348 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f77fc0105a0 con 0x7f780c1081a0 2024-04-24T23:19:15.535 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:14.914+0000 7f78142fd700 1 -- 172.21.15.80:0/1410042348 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f780c19c6d0 con 0x7f780c1081a0 2024-04-24T23:19:15.535 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:14.914+0000 7f78037fe700 1 -- 172.21.15.80:0/1410042348 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f77fc02c070 con 0x7f780c1081a0 2024-04-24T23:19:15.535 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:14.914+0000 7f78037fe700 1 -- 172.21.15.80:0/1410042348 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f77fc021070 con 0x7f780c1081a0 2024-04-24T23:19:15.535 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:14.914+0000 7f78142fd700 1 -- 172.21.15.80:0/1410042348 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f780c106a90 con 0x7f780c1081a0 2024-04-24T23:19:15.535 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:14.917+0000 7f78037fe700 1 -- 172.21.15.80:0/1410042348 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f780c106a90 con 0x7f780c1081a0 2024-04-24T23:19:15.535 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:14.983+0000 7f78142fd700 1 -- 172.21.15.80:0/1410042348 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f780c02db60 con 0x7f780c1081a0 2024-04-24T23:19:15.535 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:14.986+0000 7f78037fe700 1 -- 172.21.15.80:0/1410042348 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+429 (secure 0 0 0) 0x7f77fc015070 con 0x7f780c1081a0 2024-04-24T23:19:15.536 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:14.987+0000 7f78142fd700 1 -- 172.21.15.80:0/1410042348 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f780c1081a0 msgr2=0x7f780c199210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:15.536 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:14.987+0000 7f78142fd700 1 --2- 172.21.15.80:0/1410042348 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f780c1081a0 0x7f780c199210 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f77fc00b340 tx=0x7f77fc004750 comp rx=0 tx=0).stop 2024-04-24T23:19:15.536 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:14.987+0000 7f78142fd700 1 -- 172.21.15.80:0/1410042348 shutdown_connections 2024-04-24T23:19:15.536 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:14.987+0000 7f78142fd700 1 --2- 172.21.15.80:0/1410042348 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f780c1081a0 0x7f780c199210 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:15.536 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:14.987+0000 7f78142fd700 1 -- 172.21.15.80:0/1410042348 >> 172.21.15.80:0/1410042348 conn(0x7f780c102040 msgr2=0x7f780c18feb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:15.536 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:14.987+0000 7f78142fd700 1 -- 172.21.15.80:0/1410042348 shutdown_connections 2024-04-24T23:19:15.536 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:14.987+0000 7f78142fd700 1 -- 172.21.15.80:0/1410042348 wait complete. 2024-04-24T23:19:15.536 INFO:teuthology.orchestra.run.smithi080.stdout:Generating new minimal ceph.conf... 2024-04-24T23:19:16.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:16 smithi080 ceph-mon[28781]: from='client.? 172.21.15.80:0/1818773668' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:19:16.555 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:15.914+0000 7fd0a8e5c700 1 Processor -- start 2024-04-24T23:19:16.555 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:15.914+0000 7fd0a8e5c700 1 -- start start 2024-04-24T23:19:16.555 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:15.914+0000 7fd0a8e5c700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd0a41081b0 0x7fd0a41085d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:16.555 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:15.915+0000 7fd0a8e5c700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a41066f0 con 0x7fd0a41081b0 2024-04-24T23:19:16.555 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:15.915+0000 7fd0a259c700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd0a41081b0 0x7fd0a41085d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:16.555 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:15.915+0000 7fd0a259c700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd0a41081b0 0x7fd0a41085d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:51488/0 (socket says 172.21.15.80:51488) 2024-04-24T23:19:16.555 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:15.915+0000 7fd0a259c700 1 -- 172.21.15.80:0/779637908 learned_addr learned my addr 172.21.15.80:0/779637908 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:19:16.555 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:15.915+0000 7fd0a259c700 1 -- 172.21.15.80:0/779637908 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0a4106870 con 0x7fd0a41081b0 2024-04-24T23:19:16.555 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:15.915+0000 7fd0a259c700 1 --2- 172.21.15.80:0/779637908 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd0a41081b0 0x7fd0a41085d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fd0980098a0 tx=0x7fd098009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8470a2b9ebd64983 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:16.556 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:15.915+0000 7fd0a159a700 1 -- 172.21.15.80:0/779637908 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd098004160 con 0x7fd0a41081b0 2024-04-24T23:19:16.556 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:15.915+0000 7fd0a159a700 1 -- 172.21.15.80:0/779637908 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd098004300 con 0x7fd0a41081b0 2024-04-24T23:19:16.556 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:15.915+0000 7fd0a159a700 1 -- 172.21.15.80:0/779637908 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd098011b50 con 0x7fd0a41081b0 2024-04-24T23:19:16.556 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:15.916+0000 7fd0a8e5c700 1 -- 172.21.15.80:0/779637908 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd0a41081b0 msgr2=0x7fd0a41085d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:16.556 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:15.916+0000 7fd0a8e5c700 1 --2- 172.21.15.80:0/779637908 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd0a41081b0 0x7fd0a41085d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fd0980098a0 tx=0x7fd098009bb0 comp rx=0 tx=0).stop 2024-04-24T23:19:16.556 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:15.916+0000 7fd0a8e5c700 1 -- 172.21.15.80:0/779637908 shutdown_connections 2024-04-24T23:19:16.556 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:15.916+0000 7fd0a8e5c700 1 --2- 172.21.15.80:0/779637908 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd0a41081b0 0x7fd0a41085d0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:16.556 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:15.916+0000 7fd0a8e5c700 1 -- 172.21.15.80:0/779637908 >> 172.21.15.80:0/779637908 conn(0x7fd0a41020b0 msgr2=0x7fd0a41044d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:16.556 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:15.916+0000 7fd0a8e5c700 1 -- 172.21.15.80:0/779637908 shutdown_connections 2024-04-24T23:19:16.556 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:15.916+0000 7fd0a8e5c700 1 -- 172.21.15.80:0/779637908 wait complete. 2024-04-24T23:19:16.556 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:15.917+0000 7fd0a8e5c700 1 Processor -- start 2024-04-24T23:19:16.556 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:15.917+0000 7fd0a8e5c700 1 -- start start 2024-04-24T23:19:16.556 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:15.917+0000 7fd0a8e5c700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd0a41081b0 0x7fd0a4191d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:16.556 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:15.917+0000 7fd0a8e5c700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a410b080 con 0x7fd0a41081b0 2024-04-24T23:19:16.557 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:15.917+0000 7fd0a259c700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd0a41081b0 0x7fd0a4191d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:16.557 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:15.917+0000 7fd0a259c700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd0a41081b0 0x7fd0a4191d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:51490/0 (socket says 172.21.15.80:51490) 2024-04-24T23:19:16.557 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:15.917+0000 7fd0a259c700 1 -- 172.21.15.80:0/1818773668 learned_addr learned my addr 172.21.15.80:0/1818773668 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:19:16.557 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:15.917+0000 7fd0a259c700 1 -- 172.21.15.80:0/1818773668 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0a4192280 con 0x7fd0a41081b0 2024-04-24T23:19:16.557 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:15.917+0000 7fd0a259c700 1 --2- 172.21.15.80:0/1818773668 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd0a41081b0 0x7fd0a4191d40 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fd098011850 tx=0x7fd098011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:16.557 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:15.917+0000 7fd0937fe700 1 -- 172.21.15.80:0/1818773668 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd098003af0 con 0x7fd0a41081b0 2024-04-24T23:19:16.557 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:15.918+0000 7fd0937fe700 1 -- 172.21.15.80:0/1818773668 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd098020070 con 0x7fd0a41081b0 2024-04-24T23:19:16.557 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:15.918+0000 7fd0a8e5c700 1 -- 172.21.15.80:0/1818773668 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0a41904c0 con 0x7fd0a41081b0 2024-04-24T23:19:16.557 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:15.918+0000 7fd0937fe700 1 -- 172.21.15.80:0/1818773668 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd0980195a0 con 0x7fd0a41081b0 2024-04-24T23:19:16.557 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:15.918+0000 7fd0a8e5c700 1 -- 172.21.15.80:0/1818773668 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0a41909a0 con 0x7fd0a41081b0 2024-04-24T23:19:16.557 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:15.918+0000 7fd0937fe700 1 -- 172.21.15.80:0/1818773668 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd098019c30 con 0x7fd0a41081b0 2024-04-24T23:19:16.557 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:15.918+0000 7fd0937fe700 1 -- 172.21.15.80:0/1818773668 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd098011e00 con 0x7fd0a41081b0 2024-04-24T23:19:16.557 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:15.919+0000 7fd0a8e5c700 1 -- 172.21.15.80:0/1818773668 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0a4063890 con 0x7fd0a41081b0 2024-04-24T23:19:16.557 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:15.921+0000 7fd0937fe700 1 -- 172.21.15.80:0/1818773668 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fd098003c90 con 0x7fd0a41081b0 2024-04-24T23:19:16.557 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:15.987+0000 7fd0a8e5c700 1 -- 172.21.15.80:0/1818773668 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fd0a4195940 con 0x7fd0a41081b0 2024-04-24T23:19:16.558 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:15.988+0000 7fd0937fe700 1 -- 172.21.15.80:0/1818773668 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7fd09801a430 con 0x7fd0a41081b0 2024-04-24T23:19:16.558 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:15.988+0000 7fd0a8e5c700 1 -- 172.21.15.80:0/1818773668 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd0a41081b0 msgr2=0x7fd0a4191d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:16.558 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:15.988+0000 7fd0a8e5c700 1 --2- 172.21.15.80:0/1818773668 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd0a41081b0 0x7fd0a4191d40 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fd098011850 tx=0x7fd098011c10 comp rx=0 tx=0).stop 2024-04-24T23:19:16.558 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:15.988+0000 7fd0a8e5c700 1 -- 172.21.15.80:0/1818773668 shutdown_connections 2024-04-24T23:19:16.558 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:15.988+0000 7fd0a8e5c700 1 --2- 172.21.15.80:0/1818773668 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd0a41081b0 0x7fd0a4191d40 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:16.558 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:15.988+0000 7fd0a8e5c700 1 -- 172.21.15.80:0/1818773668 >> 172.21.15.80:0/1818773668 conn(0x7fd0a41020b0 msgr2=0x7fd0a4102fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:16.558 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:15.988+0000 7fd0a8e5c700 1 -- 172.21.15.80:0/1818773668 shutdown_connections 2024-04-24T23:19:16.558 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:15.989+0000 7fd0a8e5c700 1 -- 172.21.15.80:0/1818773668 wait complete. 2024-04-24T23:19:16.558 INFO:teuthology.orchestra.run.smithi080.stdout:Restarting the monitor... 2024-04-24T23:19:16.826 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:16 smithi080 systemd[1]: Stopping Ceph mon.a for ec7885aa-0290-11ef-bc93-c7b262605968... 2024-04-24T23:19:17.193 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:16 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mon-a[28777]: 2024-04-24T23:19:16.824+0000 7f941d98c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:19:17.193 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:16 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mon-a[28777]: 2024-04-24T23:19:16.824+0000 7f941d98c700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-04-24T23:19:17.193 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:16 smithi080 podman[29027]: 2024-04-24 23:19:16.957883072 +0000 UTC m=+0.244591729 container died 3c65024875ee8831d9df16a6a3f0e13ee336c36645a6be4e9f5789e2b0a8a460 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mon-a, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-04-24T23:19:17.476 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:17 smithi080 podman[29027]: 2024-04-24 23:19:17.266104818 +0000 UTC m=+0.552813475 container cleanup 3c65024875ee8831d9df16a6a3f0e13ee336c36645a6be4e9f5789e2b0a8a460 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mon-a, ceph=True, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2) 2024-04-24T23:19:17.476 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:17 smithi080 bash[29027]: ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mon-a 2024-04-24T23:19:17.789 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:17 smithi080 podman[29041]: 2024-04-24 23:19:17.475201376 +0000 UTC m=+0.512758973 container remove 3c65024875ee8831d9df16a6a3f0e13ee336c36645a6be4e9f5789e2b0a8a460 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-24T23:19:17.789 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:17 smithi080 systemd[1]: ceph-ec7885aa-0290-11ef-bc93-c7b262605968@mon.a.service: Deactivated successfully. 2024-04-24T23:19:17.790 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:17 smithi080 systemd[1]: Stopped Ceph mon.a for ec7885aa-0290-11ef-bc93-c7b262605968. 2024-04-24T23:19:17.790 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:17 smithi080 systemd[1]: Starting Ceph mon.a for ec7885aa-0290-11ef-bc93-c7b262605968... 2024-04-24T23:19:18.096 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:17 smithi080 podman[29134]: 2024-04-24 23:19:17.727816445 +0000 UTC m=+0.018134338 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T23:19:18.096 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:17 smithi080 podman[29134]: 2024-04-24 23:19:17.859750709 +0000 UTC m=+0.150068594 container create 789fd293dfe79fb3cb1d886b44b1ee6d45add80bc3ec75b005da5e8f3b3fefb6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mon-a, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-24T23:19:18.096 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 podman[29134]: 2024-04-24 23:19:18.036816343 +0000 UTC m=+0.327134230 container init 789fd293dfe79fb3cb1d886b44b1ee6d45add80bc3ec75b005da5e8f3b3fefb6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mon-a, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T23:19:18.097 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 podman[29134]: 2024-04-24 23:19:18.03978815 +0000 UTC m=+0.330106033 container start 789fd293dfe79fb3cb1d886b44b1ee6d45add80bc3ec75b005da5e8f3b3fefb6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mon-a, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-04-24T23:19:18.097 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: set uid:gid to 167:167 (ceph:ceph) 2024-04-24T23:19:18.097 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev), process ceph-mon, pid 2 2024-04-24T23:19:18.097 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: pidfile_write: ignore empty --pid-file 2024-04-24T23:19:18.097 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: load: jerasure load: lrc 2024-04-24T23:19:18.097 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: RocksDB version: 7.9.2 2024-04-24T23:19:18.097 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Git sha 0 2024-04-24T23:19:18.097 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Compile date 2024-04-24 16:41:23 2024-04-24T23:19:18.097 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: DB SUMMARY 2024-04-24T23:19:18.097 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: DB Session ID: P9GVAGHO0PH3OT0D9JX4 2024-04-24T23:19:18.097 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: CURRENT file: CURRENT 2024-04-24T23:19:18.097 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: IDENTITY file: IDENTITY 2024-04-24T23:19:18.097 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-04-24T23:19:18.097 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-04-24T23:19:18.098 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 104571 ; 2024-04-24T23:19:18.098 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.error_if_exists: 0 2024-04-24T23:19:18.099 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.create_if_missing: 0 2024-04-24T23:19:18.099 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.paranoid_checks: 1 2024-04-24T23:19:18.099 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-24T23:19:18.099 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-24T23:19:18.099 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-24T23:19:18.099 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.env: 0x558e40f5cc60 2024-04-24T23:19:18.100 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.fs: PosixFileSystem 2024-04-24T23:19:18.100 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.info_log: 0x558e43623aa0 2024-04-24T23:19:18.100 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.max_file_opening_threads: 16 2024-04-24T23:19:18.100 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.statistics: (nil) 2024-04-24T23:19:18.100 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.use_fsync: 0 2024-04-24T23:19:18.100 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.max_log_file_size: 0 2024-04-24T23:19:18.100 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-24T23:19:18.100 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-24T23:19:18.100 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.keep_log_file_num: 1000 2024-04-24T23:19:18.100 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.recycle_log_file_num: 0 2024-04-24T23:19:18.100 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.allow_fallocate: 1 2024-04-24T23:19:18.100 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.allow_mmap_reads: 0 2024-04-24T23:19:18.100 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.allow_mmap_writes: 0 2024-04-24T23:19:18.100 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.use_direct_reads: 0 2024-04-24T23:19:18.100 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-24T23:19:18.101 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.create_missing_column_families: 0 2024-04-24T23:19:18.101 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.db_log_dir: 2024-04-24T23:19:18.101 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.wal_dir: 2024-04-24T23:19:18.101 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-24T23:19:18.101 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-24T23:19:18.101 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-24T23:19:18.101 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-24T23:19:18.101 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-24T23:19:18.101 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-24T23:19:18.101 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.advise_random_on_open: 1 2024-04-24T23:19:18.101 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.db_write_buffer_size: 0 2024-04-24T23:19:18.101 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.write_buffer_manager: 0x558e435f61e0 2024-04-24T23:19:18.101 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-24T23:19:18.101 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-24T23:19:18.102 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-24T23:19:18.102 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.rate_limiter: (nil) 2024-04-24T23:19:18.102 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-24T23:19:18.102 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.wal_recovery_mode: 2 2024-04-24T23:19:18.102 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.enable_thread_tracking: 0 2024-04-24T23:19:18.102 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.enable_pipelined_write: 0 2024-04-24T23:19:18.102 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.unordered_write: 0 2024-04-24T23:19:18.102 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-24T23:19:18.102 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-24T23:19:18.102 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-24T23:19:18.102 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-24T23:19:18.102 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.row_cache: None 2024-04-24T23:19:18.102 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.wal_filter: None 2024-04-24T23:19:18.102 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-24T23:19:18.103 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.allow_ingest_behind: 0 2024-04-24T23:19:18.103 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.two_write_queues: 0 2024-04-24T23:19:18.103 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.manual_wal_flush: 0 2024-04-24T23:19:18.103 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.wal_compression: 0 2024-04-24T23:19:18.103 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.atomic_flush: 0 2024-04-24T23:19:18.103 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-24T23:19:18.103 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-24T23:19:18.103 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-24T23:19:18.103 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.log_readahead_size: 0 2024-04-24T23:19:18.103 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-24T23:19:18.103 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.best_efforts_recovery: 0 2024-04-24T23:19:18.103 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-24T23:19:18.103 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-24T23:19:18.103 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.allow_data_in_errors: 0 2024-04-24T23:19:18.103 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.db_host_id: __hostname__ 2024-04-24T23:19:18.104 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-24T23:19:18.104 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.max_background_jobs: 2 2024-04-24T23:19:18.104 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.max_background_compactions: -1 2024-04-24T23:19:18.104 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.max_subcompactions: 1 2024-04-24T23:19:18.104 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-24T23:19:18.104 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-24T23:19:18.104 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-24T23:19:18.104 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.max_total_wal_size: 0 2024-04-24T23:19:18.104 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-24T23:19:18.104 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-24T23:19:18.104 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-24T23:19:18.104 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-24T23:19:18.104 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.max_open_files: -1 2024-04-24T23:19:18.104 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.bytes_per_sync: 0 2024-04-24T23:19:18.104 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-24T23:19:18.105 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-24T23:19:18.105 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.compaction_readahead_size: 0 2024-04-24T23:19:18.105 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.max_background_flushes: -1 2024-04-24T23:19:18.105 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Compression algorithms supported: 2024-04-24T23:19:18.105 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-24T23:19:18.105 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: kZSTD supported: 0 2024-04-24T23:19:18.105 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: kXpressCompression supported: 0 2024-04-24T23:19:18.105 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: kLZ4HCCompression supported: 1 2024-04-24T23:19:18.105 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: kZlibCompression supported: 1 2024-04-24T23:19:18.105 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: kSnappyCompression supported: 1 2024-04-24T23:19:18.105 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: kLZ4Compression supported: 1 2024-04-24T23:19:18.105 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: kBZip2Compression supported: 0 2024-04-24T23:19:18.105 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-24T23:19:18.105 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-24T23:19:18.106 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-04-24T23:19:18.106 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-24T23:19:18.106 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-24T23:19:18.106 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.merge_operator: 2024-04-24T23:19:18.106 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.compaction_filter: None 2024-04-24T23:19:18.106 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.compaction_filter_factory: None 2024-04-24T23:19:18.106 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.sst_partitioner_factory: None 2024-04-24T23:19:18.106 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-24T23:19:18.106 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-24T23:19:18.106 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x558e43623ba0) 2024-04-24T23:19:18.106 INFO:journalctl@ceph.mon.a.smithi080.stdout: cache_index_and_filter_blocks: 1 2024-04-24T23:19:18.106 INFO:journalctl@ceph.mon.a.smithi080.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-24T23:19:18.106 INFO:journalctl@ceph.mon.a.smithi080.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-24T23:19:18.107 INFO:journalctl@ceph.mon.a.smithi080.stdout: pin_top_level_index_and_filter: 1 2024-04-24T23:19:18.107 INFO:journalctl@ceph.mon.a.smithi080.stdout: index_type: 0 2024-04-24T23:19:18.107 INFO:journalctl@ceph.mon.a.smithi080.stdout: data_block_index_type: 0 2024-04-24T23:19:18.107 INFO:journalctl@ceph.mon.a.smithi080.stdout: index_shortening: 1 2024-04-24T23:19:18.107 INFO:journalctl@ceph.mon.a.smithi080.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-24T23:19:18.107 INFO:journalctl@ceph.mon.a.smithi080.stdout: checksum: 4 2024-04-24T23:19:18.107 INFO:journalctl@ceph.mon.a.smithi080.stdout: no_block_cache: 0 2024-04-24T23:19:18.107 INFO:journalctl@ceph.mon.a.smithi080.stdout: block_cache: 0x558e43627090 2024-04-24T23:19:18.107 INFO:journalctl@ceph.mon.a.smithi080.stdout: block_cache_name: BinnedLRUCache 2024-04-24T23:19:18.107 INFO:journalctl@ceph.mon.a.smithi080.stdout: block_cache_options: 2024-04-24T23:19:18.107 INFO:journalctl@ceph.mon.a.smithi080.stdout: capacity : 536870912 2024-04-24T23:19:18.107 INFO:journalctl@ceph.mon.a.smithi080.stdout: num_shard_bits : 4 2024-04-24T23:19:18.107 INFO:journalctl@ceph.mon.a.smithi080.stdout: strict_capacity_limit : 0 2024-04-24T23:19:18.107 INFO:journalctl@ceph.mon.a.smithi080.stdout: high_pri_pool_ratio: 0.000 2024-04-24T23:19:18.107 INFO:journalctl@ceph.mon.a.smithi080.stdout: block_cache_compressed: (nil) 2024-04-24T23:19:18.108 INFO:journalctl@ceph.mon.a.smithi080.stdout: persistent_cache: (nil) 2024-04-24T23:19:18.108 INFO:journalctl@ceph.mon.a.smithi080.stdout: block_size: 4096 2024-04-24T23:19:18.108 INFO:journalctl@ceph.mon.a.smithi080.stdout: block_size_deviation: 10 2024-04-24T23:19:18.108 INFO:journalctl@ceph.mon.a.smithi080.stdout: block_restart_interval: 16 2024-04-24T23:19:18.108 INFO:journalctl@ceph.mon.a.smithi080.stdout: index_block_restart_interval: 1 2024-04-24T23:19:18.108 INFO:journalctl@ceph.mon.a.smithi080.stdout: metadata_block_size: 4096 2024-04-24T23:19:18.108 INFO:journalctl@ceph.mon.a.smithi080.stdout: partition_filters: 0 2024-04-24T23:19:18.108 INFO:journalctl@ceph.mon.a.smithi080.stdout: use_delta_encoding: 1 2024-04-24T23:19:18.108 INFO:journalctl@ceph.mon.a.smithi080.stdout: filter_policy: bloomfilter 2024-04-24T23:19:18.108 INFO:journalctl@ceph.mon.a.smithi080.stdout: whole_key_filtering: 1 2024-04-24T23:19:18.108 INFO:journalctl@ceph.mon.a.smithi080.stdout: verify_compression: 0 2024-04-24T23:19:18.108 INFO:journalctl@ceph.mon.a.smithi080.stdout: read_amp_bytes_per_bit: 0 2024-04-24T23:19:18.108 INFO:journalctl@ceph.mon.a.smithi080.stdout: format_version: 5 2024-04-24T23:19:18.108 INFO:journalctl@ceph.mon.a.smithi080.stdout: enable_index_compression: 1 2024-04-24T23:19:18.108 INFO:journalctl@ceph.mon.a.smithi080.stdout: block_align: 0 2024-04-24T23:19:18.109 INFO:journalctl@ceph.mon.a.smithi080.stdout: max_auto_readahead_size: 262144 2024-04-24T23:19:18.109 INFO:journalctl@ceph.mon.a.smithi080.stdout: prepopulate_block_cache: 0 2024-04-24T23:19:18.109 INFO:journalctl@ceph.mon.a.smithi080.stdout: initial_auto_readahead_size: 8192 2024-04-24T23:19:18.109 INFO:journalctl@ceph.mon.a.smithi080.stdout: num_file_reads_for_auto_readahead: 2 2024-04-24T23:19:18.109 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.write_buffer_size: 33554432 2024-04-24T23:19:18.109 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.max_write_buffer_number: 2 2024-04-24T23:19:18.109 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.compression: NoCompression 2024-04-24T23:19:18.109 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.bottommost_compression: Disabled 2024-04-24T23:19:18.109 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.prefix_extractor: nullptr 2024-04-24T23:19:18.110 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-24T23:19:18.110 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.num_levels: 7 2024-04-24T23:19:18.110 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-24T23:19:18.110 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-24T23:19:18.110 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-24T23:19:18.110 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-24T23:19:18.110 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-24T23:19:18.110 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-24T23:19:18.110 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-24T23:19:18.110 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-24T23:19:18.110 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-24T23:19:18.110 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-24T23:19:18.110 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-24T23:19:18.110 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-24T23:19:18.111 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-24T23:19:18.111 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.compression_opts.level: 32767 2024-04-24T23:19:18.111 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.compression_opts.strategy: 0 2024-04-24T23:19:18.111 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-24T23:19:18.111 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-24T23:19:18.111 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-24T23:19:18.111 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-24T23:19:18.111 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.compression_opts.enabled: false 2024-04-24T23:19:18.111 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-24T23:19:18.111 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-24T23:19:18.111 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-24T23:19:18.111 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-24T23:19:18.111 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.target_file_size_base: 67108864 2024-04-24T23:19:18.111 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-24T23:19:18.111 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-24T23:19:18.112 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-24T23:19:18.112 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-24T23:19:18.112 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-24T23:19:18.112 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-24T23:19:18.112 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-24T23:19:18.112 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-24T23:19:18.112 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-24T23:19:18.112 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-24T23:19:18.112 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-24T23:19:18.112 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-24T23:19:18.112 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-24T23:19:18.112 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-24T23:19:18.112 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.arena_block_size: 1048576 2024-04-24T23:19:18.112 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-24T23:19:18.113 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-24T23:19:18.113 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.disable_auto_compactions: 0 2024-04-24T23:19:18.113 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-24T23:19:18.113 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-24T23:19:18.113 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-24T23:19:18.113 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-24T23:19:18.113 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-24T23:19:18.113 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-24T23:19:18.113 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-24T23:19:18.113 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-24T23:19:18.113 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-24T23:19:18.113 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-24T23:19:18.113 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.table_properties_collectors: 2024-04-24T23:19:18.113 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.inplace_update_support: 0 2024-04-24T23:19:18.114 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-24T23:19:18.114 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-24T23:19:18.114 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-24T23:19:18.114 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-24T23:19:18.114 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.bloom_locality: 0 2024-04-24T23:19:18.114 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.max_successive_merges: 0 2024-04-24T23:19:18.114 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-24T23:19:18.114 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.paranoid_file_checks: 0 2024-04-24T23:19:18.114 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.force_consistency_checks: 1 2024-04-24T23:19:18.114 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.report_bg_io_stats: 0 2024-04-24T23:19:18.114 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.ttl: 2592000 2024-04-24T23:19:18.114 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-24T23:19:18.114 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-24T23:19:18.114 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-24T23:19:18.115 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.enable_blob_files: false 2024-04-24T23:19:18.115 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.min_blob_size: 0 2024-04-24T23:19:18.115 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.blob_file_size: 268435456 2024-04-24T23:19:18.115 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-24T23:19:18.115 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-24T23:19:18.115 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-24T23:19:18.115 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-24T23:19:18.115 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-24T23:19:18.115 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.blob_file_starting_level: 0 2024-04-24T23:19:18.115 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-24T23:19:18.115 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-04-24T23:19:18.115 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-04-24T23:19:18.115 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: befac428-e188-4a8e-8270-11066de0a7d6 2024-04-24T23:19:18.115 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714000758068825, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-04-24T23:19:18.115 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-04-24T23:19:18.116 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714000758070234, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 99582, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 330, "table_properties": {"data_size": 97490, "index_size": 310, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14084, "raw_average_key_size": 49, "raw_value_size": 89838, "raw_average_value_size": 315, "num_data_blocks": 12, "num_entries": 285, "num_filter_entries": 285, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1714000758, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "befac428-e188-4a8e-8270-11066de0a7d6", "db_session_id": "P9GVAGHO0PH3OT0D9JX4", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-04-24T23:19:18.116 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714000758070347, "job": 1, "event": "recovery_finished"} 2024-04-24T23:19:18.116 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-04-24T23:19:18.116 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-24T23:19:18.116 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x558e43718000 2024-04-24T23:19:18.116 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: DB pointer 0x558e43702000 2024-04-24T23:19:18.116 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-24T23:19:18.116 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-24T23:19:18.116 INFO:journalctl@ceph.mon.a.smithi080.stdout: ** DB Stats ** 2024-04-24T23:19:18.116 INFO:journalctl@ceph.mon.a.smithi080.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-24T23:19:18.116 INFO:journalctl@ceph.mon.a.smithi080.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-24T23:19:18.116 INFO:journalctl@ceph.mon.a.smithi080.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-24T23:19:18.116 INFO:journalctl@ceph.mon.a.smithi080.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-24T23:19:18.116 INFO:journalctl@ceph.mon.a.smithi080.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-24T23:19:18.117 INFO:journalctl@ceph.mon.a.smithi080.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-24T23:19:18.117 INFO:journalctl@ceph.mon.a.smithi080.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-24T23:19:18.117 INFO:journalctl@ceph.mon.a.smithi080.stdout: 2024-04-24T23:19:18.117 INFO:journalctl@ceph.mon.a.smithi080.stdout: ** Compaction Stats [default] ** 2024-04-24T23:19:18.117 INFO:journalctl@ceph.mon.a.smithi080.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-24T23:19:18.117 INFO:journalctl@ceph.mon.a.smithi080.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-24T23:19:18.117 INFO:journalctl@ceph.mon.a.smithi080.stdout: L0 2/0 99.08 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 117.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-24T23:19:18.117 INFO:journalctl@ceph.mon.a.smithi080.stdout: Sum 2/0 99.08 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 117.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-24T23:19:18.117 INFO:journalctl@ceph.mon.a.smithi080.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 117.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-24T23:19:18.117 INFO:journalctl@ceph.mon.a.smithi080.stdout: 2024-04-24T23:19:18.117 INFO:journalctl@ceph.mon.a.smithi080.stdout: ** Compaction Stats [default] ** 2024-04-24T23:19:18.117 INFO:journalctl@ceph.mon.a.smithi080.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-24T23:19:18.117 INFO:journalctl@ceph.mon.a.smithi080.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-24T23:19:18.117 INFO:journalctl@ceph.mon.a.smithi080.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 117.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-24T23:19:18.117 INFO:journalctl@ceph.mon.a.smithi080.stdout: 2024-04-24T23:19:18.118 INFO:journalctl@ceph.mon.a.smithi080.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-24T23:19:18.118 INFO:journalctl@ceph.mon.a.smithi080.stdout: 2024-04-24T23:19:18.118 INFO:journalctl@ceph.mon.a.smithi080.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-24T23:19:18.118 INFO:journalctl@ceph.mon.a.smithi080.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-24T23:19:18.118 INFO:journalctl@ceph.mon.a.smithi080.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-24T23:19:18.118 INFO:journalctl@ceph.mon.a.smithi080.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-24T23:19:18.118 INFO:journalctl@ceph.mon.a.smithi080.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-24T23:19:18.118 INFO:journalctl@ceph.mon.a.smithi080.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-24T23:19:18.118 INFO:journalctl@ceph.mon.a.smithi080.stdout: Cumulative compaction: 0.00 GB write, 22.84 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-24T23:19:18.118 INFO:journalctl@ceph.mon.a.smithi080.stdout: Interval compaction: 0.00 GB write, 22.84 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-24T23:19:18.118 INFO:journalctl@ceph.mon.a.smithi080.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-24T23:19:18.118 INFO:journalctl@ceph.mon.a.smithi080.stdout: Block cache BinnedLRUCache@0x558e43627090#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-04-24T23:19:18.118 INFO:journalctl@ceph.mon.a.smithi080.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-04-24T23:19:18.118 INFO:journalctl@ceph.mon.a.smithi080.stdout: 2024-04-24T23:19:18.119 INFO:journalctl@ceph.mon.a.smithi080.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-24T23:19:18.119 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: starting mon.a rank 0 at public addrs [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] at bind addrs [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid ec7885aa-0290-11ef-bc93-c7b262605968 2024-04-24T23:19:18.119 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: mon.a@-1(???) e1 preinit fsid ec7885aa-0290-11ef-bc93-c7b262605968 2024-04-24T23:19:18.119 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: mon.a@-1(???).mds e1 new map 2024-04-24T23:19:18.119 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: mon.a@-1(???).mds e1 print_map 2024-04-24T23:19:18.119 INFO:journalctl@ceph.mon.a.smithi080.stdout: e1 2024-04-24T23:19:18.119 INFO:journalctl@ceph.mon.a.smithi080.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-24T23:19:18.119 INFO:journalctl@ceph.mon.a.smithi080.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-24T23:19:18.119 INFO:journalctl@ceph.mon.a.smithi080.stdout: legacy client fscid: -1 2024-04-24T23:19:18.119 INFO:journalctl@ceph.mon.a.smithi080.stdout: 2024-04-24T23:19:18.119 INFO:journalctl@ceph.mon.a.smithi080.stdout: No filesystems configured 2024-04-24T23:19:18.119 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-24T23:19:18.119 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-24T23:19:18.119 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-24T23:19:18.120 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-24T23:19:18.120 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-04-24T23:19:18.120 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 bash[29134]: 789fd293dfe79fb3cb1d886b44b1ee6d45add80bc3ec75b005da5e8f3b3fefb6 2024-04-24T23:19:18.120 INFO:teuthology.orchestra.run.smithi080.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-04-24T23:19:18.390 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 systemd[1]: Started Ceph mon.a for ec7885aa-0290-11ef-bc93-c7b262605968. 2024-04-24T23:19:18.391 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-24T23:19:18.391 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: monmap epoch 1 2024-04-24T23:19:18.391 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: fsid ec7885aa-0290-11ef-bc93-c7b262605968 2024-04-24T23:19:18.391 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: last_changed 2024-04-24T23:18:58.816011+0000 2024-04-24T23:19:18.391 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: created 2024-04-24T23:18:58.816011+0000 2024-04-24T23:19:18.391 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: min_mon_release 19 (squid) 2024-04-24T23:19:18.391 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: election_strategy: 1 2024-04-24T23:19:18.391 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: 0: [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] mon.a 2024-04-24T23:19:18.391 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: fsmap 2024-04-24T23:19:18.391 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: osdmap e1: 0 total, 0 up, 0 in 2024-04-24T23:19:18.391 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:18 smithi080 ceph-mon[29152]: mgrmap e1: no daemons active 2024-04-24T23:19:19.144 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:18.488+0000 7f9d65d82700 1 Processor -- start 2024-04-24T23:19:19.144 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:18.488+0000 7f9d65d82700 1 -- start start 2024-04-24T23:19:19.144 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:18.488+0000 7f9d65d82700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9d60108700 0x7f9d6010aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:19.144 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:18.488+0000 7f9d65d82700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f9d6007be60 con 0x7f9d60108700 2024-04-24T23:19:19.144 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:18.489+0000 7f9d5f7fe700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9d60108700 0x7f9d6010aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:19.144 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:18.489+0000 7f9d5f7fe700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9d60108700 0x7f9d6010aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:51496/0 (socket says 172.21.15.80:51496) 2024-04-24T23:19:19.144 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:18.489+0000 7f9d5f7fe700 1 -- 172.21.15.80:0/3381756411 learned_addr learned my addr 172.21.15.80:0/3381756411 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:19:19.144 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:18.489+0000 7f9d5f7fe700 1 -- 172.21.15.80:0/3381756411 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d6010b030 con 0x7f9d60108700 2024-04-24T23:19:19.144 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:18.489+0000 7f9d5f7fe700 1 --2- 172.21.15.80:0/3381756411 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9d60108700 0x7f9d6010aaf0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f9d500098a0 tx=0x7f9d50009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5adb5baf03547161 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:19.144 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:18.490+0000 7f9d5effd700 1 -- 172.21.15.80:0/3381756411 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9d50004160 con 0x7f9d60108700 2024-04-24T23:19:19.144 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:18.490+0000 7f9d5effd700 1 -- 172.21.15.80:0/3381756411 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9d50004300 con 0x7f9d60108700 2024-04-24T23:19:19.145 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:18.490+0000 7f9d5effd700 1 -- 172.21.15.80:0/3381756411 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9d50011b50 con 0x7f9d60108700 2024-04-24T23:19:19.145 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:18.490+0000 7f9d65d82700 1 -- 172.21.15.80:0/3381756411 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9d60108700 msgr2=0x7f9d6010aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:19.145 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:18.490+0000 7f9d65d82700 1 --2- 172.21.15.80:0/3381756411 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9d60108700 0x7f9d6010aaf0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f9d500098a0 tx=0x7f9d50009bb0 comp rx=0 tx=0).stop 2024-04-24T23:19:19.145 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:18.490+0000 7f9d65d82700 1 -- 172.21.15.80:0/3381756411 shutdown_connections 2024-04-24T23:19:19.145 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:18.490+0000 7f9d65d82700 1 --2- 172.21.15.80:0/3381756411 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9d60108700 0x7f9d6010aaf0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:19.145 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:18.490+0000 7f9d65d82700 1 -- 172.21.15.80:0/3381756411 >> 172.21.15.80:0/3381756411 conn(0x7f9d60102030 msgr2=0x7f9d60104490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:19.145 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:18.490+0000 7f9d65d82700 1 -- 172.21.15.80:0/3381756411 shutdown_connections 2024-04-24T23:19:19.145 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:18.490+0000 7f9d65d82700 1 -- 172.21.15.80:0/3381756411 wait complete. 2024-04-24T23:19:19.145 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:18.491+0000 7f9d65d82700 1 Processor -- start 2024-04-24T23:19:19.145 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:18.491+0000 7f9d65d82700 1 -- start start 2024-04-24T23:19:19.145 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:18.491+0000 7f9d65d82700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9d60108700 0x7f9d60107f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:19.145 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:18.491+0000 7f9d65d82700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f9d6010c0b0 con 0x7f9d60108700 2024-04-24T23:19:19.145 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:18.491+0000 7f9d5f7fe700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9d60108700 0x7f9d60107f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:19.145 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:18.491+0000 7f9d5f7fe700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9d60108700 0x7f9d60107f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:51502/0 (socket says 172.21.15.80:51502) 2024-04-24T23:19:19.146 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:18.491+0000 7f9d5f7fe700 1 -- 172.21.15.80:0/3165244082 learned_addr learned my addr 172.21.15.80:0/3165244082 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:19:19.146 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:18.492+0000 7f9d5f7fe700 1 -- 172.21.15.80:0/3165244082 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d601084c0 con 0x7f9d60108700 2024-04-24T23:19:19.146 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:18.492+0000 7f9d5f7fe700 1 --2- 172.21.15.80:0/3165244082 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9d60108700 0x7f9d60107f80 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f9d50011670 tx=0x7f9d50011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:19.146 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:18.492+0000 7f9d5d7fa700 1 -- 172.21.15.80:0/3165244082 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9d500039c0 con 0x7f9d60108700 2024-04-24T23:19:19.146 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:18.492+0000 7f9d5d7fa700 1 -- 172.21.15.80:0/3165244082 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9d50011e50 con 0x7f9d60108700 2024-04-24T23:19:19.146 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:18.492+0000 7f9d65d82700 1 -- 172.21.15.80:0/3165244082 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d601066b0 con 0x7f9d60108700 2024-04-24T23:19:19.146 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:18.492+0000 7f9d5d7fa700 1 -- 172.21.15.80:0/3165244082 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9d5001a5c0 con 0x7f9d60108700 2024-04-24T23:19:19.146 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:18.492+0000 7f9d65d82700 1 -- 172.21.15.80:0/3165244082 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d60106bf0 con 0x7f9d60108700 2024-04-24T23:19:19.146 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:18.492+0000 7f9d5d7fa700 1 -- 172.21.15.80:0/3165244082 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f9d5001ac00 con 0x7f9d60108700 2024-04-24T23:19:19.146 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:18.492+0000 7f9d5d7fa700 1 -- 172.21.15.80:0/3165244082 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f9d50019700 con 0x7f9d60108700 2024-04-24T23:19:19.146 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:18.493+0000 7f9d65d82700 1 -- 172.21.15.80:0/3165244082 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d60063890 con 0x7f9d60108700 2024-04-24T23:19:19.146 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:18.495+0000 7f9d5d7fa700 1 -- 172.21.15.80:0/3165244082 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f9d50003b60 con 0x7f9d60108700 2024-04-24T23:19:19.146 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:18.562+0000 7f9d65d82700 1 -- 172.21.15.80:0/3165244082 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f9d60106e70 con 0x7f9d60108700 2024-04-24T23:19:19.146 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:18.566+0000 7f9d5d7fa700 1 -- 172.21.15.80:0/3165244082 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f9d50019900 con 0x7f9d60108700 2024-04-24T23:19:19.147 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:18.567+0000 7f9d65d82700 1 -- 172.21.15.80:0/3165244082 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9d60108700 msgr2=0x7f9d60107f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:19.147 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:18.567+0000 7f9d65d82700 1 --2- 172.21.15.80:0/3165244082 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9d60108700 0x7f9d60107f80 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f9d50011670 tx=0x7f9d50011b60 comp rx=0 tx=0).stop 2024-04-24T23:19:19.147 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:18.567+0000 7f9d65d82700 1 -- 172.21.15.80:0/3165244082 shutdown_connections 2024-04-24T23:19:19.147 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:18.567+0000 7f9d65d82700 1 --2- 172.21.15.80:0/3165244082 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9d60108700 0x7f9d60107f80 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:19.147 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:18.567+0000 7f9d65d82700 1 -- 172.21.15.80:0/3165244082 >> 172.21.15.80:0/3165244082 conn(0x7f9d60102030 msgr2=0x7f9d60102b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:19.147 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:18.568+0000 7f9d65d82700 1 -- 172.21.15.80:0/3165244082 shutdown_connections 2024-04-24T23:19:19.147 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:18.568+0000 7f9d65d82700 1 -- 172.21.15.80:0/3165244082 wait complete. 2024-04-24T23:19:19.147 INFO:teuthology.orchestra.run.smithi080.stdout:Wrote config to /etc/ceph/ceph.conf 2024-04-24T23:19:19.169 INFO:teuthology.orchestra.run.smithi080.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-04-24T23:19:19.169 INFO:teuthology.orchestra.run.smithi080.stdout:Creating mgr... 2024-04-24T23:19:19.169 INFO:teuthology.orchestra.run.smithi080.stdout:Verifying port 0.0.0.0:9283 ... 2024-04-24T23:19:19.169 INFO:teuthology.orchestra.run.smithi080.stdout:Verifying port 0.0.0.0:8765 ... 2024-04-24T23:19:19.964 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:19 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/3165244082' entity='client.admin' 2024-04-24T23:19:20.166 INFO:teuthology.orchestra.run.smithi080.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-ec7885aa-0290-11ef-bc93-c7b262605968@mgr.x 2024-04-24T23:19:20.166 INFO:teuthology.orchestra.run.smithi080.stdout:systemctl: stderr Failed to reset failed state of unit ceph-ec7885aa-0290-11ef-bc93-c7b262605968@mgr.x.service: Unit ceph-ec7885aa-0290-11ef-bc93-c7b262605968@mgr.x.service not loaded. 2024-04-24T23:19:20.338 INFO:teuthology.orchestra.run.smithi080.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-ec7885aa-0290-11ef-bc93-c7b262605968.target.wants/ceph-ec7885aa-0290-11ef-bc93-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-ec7885aa-0290-11ef-bc93-c7b262605968@.service. 2024-04-24T23:19:20.895 INFO:teuthology.orchestra.run.smithi080.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-24T23:19:20.895 INFO:teuthology.orchestra.run.smithi080.stdout:systemctl: stdout disabled 2024-04-24T23:19:20.903 INFO:teuthology.orchestra.run.smithi080.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-24T23:19:20.904 INFO:teuthology.orchestra.run.smithi080.stdout:systemctl: stdout inactive 2024-04-24T23:19:20.904 INFO:teuthology.orchestra.run.smithi080.stdout:firewalld.service is not enabled 2024-04-24T23:19:20.904 INFO:teuthology.orchestra.run.smithi080.stdout:Not possible to enable service . firewalld.service is not available 2024-04-24T23:19:20.912 INFO:teuthology.orchestra.run.smithi080.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-24T23:19:20.912 INFO:teuthology.orchestra.run.smithi080.stdout:systemctl: stdout disabled 2024-04-24T23:19:20.919 INFO:teuthology.orchestra.run.smithi080.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-24T23:19:20.919 INFO:teuthology.orchestra.run.smithi080.stdout:systemctl: stdout inactive 2024-04-24T23:19:20.919 INFO:teuthology.orchestra.run.smithi080.stdout:firewalld.service is not enabled 2024-04-24T23:19:20.919 INFO:teuthology.orchestra.run.smithi080.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-04-24T23:19:20.919 INFO:teuthology.orchestra.run.smithi080.stdout:Waiting for mgr to start... 2024-04-24T23:19:20.919 INFO:teuthology.orchestra.run.smithi080.stdout:Waiting for mgr... 2024-04-24T23:19:21.662 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:21 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/1650300422' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-24T23:19:21.663 INFO:journalctl@ceph.mgr.x.smithi080.stdout:Apr 24 23:19:21 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-x[29381]: 2024-04-24T23:19:21.573+0000 7fafac3b9200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-24T23:19:21.663 INFO:journalctl@ceph.mgr.x.smithi080.stdout:Apr 24 23:19:21 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-x[29381]: 2024-04-24T23:19:21.660+0000 7fafac3b9200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-24T23:19:22.041 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout 2024-04-24T23:19:22.041 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout { 2024-04-24T23:19:22.041 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "fsid": "ec7885aa-0290-11ef-bc93-c7b262605968", 2024-04-24T23:19:22.041 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "health": { 2024-04-24T23:19:22.041 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-24T23:19:22.041 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-24T23:19:22.041 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-24T23:19:22.041 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:19:22.041 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-24T23:19:22.041 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-24T23:19:22.041 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout 0 2024-04-24T23:19:22.042 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout ], 2024-04-24T23:19:22.042 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-24T23:19:22.042 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "a" 2024-04-24T23:19:22.042 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout ], 2024-04-24T23:19:22.042 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2024-04-24T23:19:22.042 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-24T23:19:22.042 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T23:19:22.042 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-24T23:19:22.042 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-24T23:19:22.042 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:19:22.042 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-24T23:19:22.042 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T23:19:22.042 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-24T23:19:22.042 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-24T23:19:22.043 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-24T23:19:22.053 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-24T23:19:22.053 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-24T23:19:22.053 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-24T23:19:22.054 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:19:22.054 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-24T23:19:22.054 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-24T23:19:22.054 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-24T23:19:22.054 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-24T23:19:22.054 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-24T23:19:22.054 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-24T23:19:22.054 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-24T23:19:22.054 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-24T23:19:22.054 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-24T23:19:22.054 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:19:22.054 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-24T23:19:22.054 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T23:19:22.054 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-24T23:19:22.055 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-24T23:19:22.055 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:19:22.055 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-24T23:19:22.055 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-24T23:19:22.055 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-24T23:19:22.055 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-24T23:19:22.055 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-24T23:19:22.055 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-24T23:19:22.055 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "restful" 2024-04-24T23:19:22.055 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout ], 2024-04-24T23:19:22.055 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-24T23:19:22.055 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:19:22.055 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-24T23:19:22.055 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T23:19:22.056 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "modified": "2024-04-24T23:19:05.586911+0000", 2024-04-24T23:19:22.056 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-24T23:19:22.056 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:19:22.056 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-24T23:19:22.056 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout } 2024-04-24T23:19:22.056 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:21.354+0000 7f4eead0f700 1 Processor -- start 2024-04-24T23:19:22.056 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:21.355+0000 7f4eead0f700 1 -- start start 2024-04-24T23:19:22.056 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:21.355+0000 7f4eead0f700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4ee4106460 0x7f4ee4106880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:22.056 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:21.355+0000 7f4eead0f700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee4106dc0 con 0x7f4ee4106460 2024-04-24T23:19:22.056 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:21.355+0000 7f4ee8aab700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4ee4106460 0x7f4ee4106880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:22.056 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:21.355+0000 7f4ee8aab700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4ee4106460 0x7f4ee4106880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:51526/0 (socket says 172.21.15.80:51526) 2024-04-24T23:19:22.056 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:21.355+0000 7f4ee8aab700 1 -- 172.21.15.80:0/3128368133 learned_addr learned my addr 172.21.15.80:0/3128368133 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:19:22.056 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:21.355+0000 7f4ee8aab700 1 -- 172.21.15.80:0/3128368133 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ee4106f40 con 0x7f4ee4106460 2024-04-24T23:19:22.056 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:21.355+0000 7f4ee8aab700 1 --2- 172.21.15.80:0/3128368133 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4ee4106460 0x7f4ee4106880 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f4ed80098a0 tx=0x7f4ed8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a36278129dfc102 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:22.057 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:21.356+0000 7f4ee37fe700 1 -- 172.21.15.80:0/3128368133 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4ed8004160 con 0x7f4ee4106460 2024-04-24T23:19:22.057 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:21.356+0000 7f4ee37fe700 1 -- 172.21.15.80:0/3128368133 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4ed8004300 con 0x7f4ee4106460 2024-04-24T23:19:22.057 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:21.356+0000 7f4ee37fe700 1 -- 172.21.15.80:0/3128368133 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4ed8011b50 con 0x7f4ee4106460 2024-04-24T23:19:22.057 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:21.356+0000 7f4eead0f700 1 -- 172.21.15.80:0/3128368133 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4ee4106460 msgr2=0x7f4ee4106880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:22.057 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:21.356+0000 7f4eead0f700 1 --2- 172.21.15.80:0/3128368133 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4ee4106460 0x7f4ee4106880 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f4ed80098a0 tx=0x7f4ed8009bb0 comp rx=0 tx=0).stop 2024-04-24T23:19:22.057 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:21.356+0000 7f4eead0f700 1 -- 172.21.15.80:0/3128368133 shutdown_connections 2024-04-24T23:19:22.057 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:21.356+0000 7f4eead0f700 1 --2- 172.21.15.80:0/3128368133 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4ee4106460 0x7f4ee4106880 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:22.057 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:21.356+0000 7f4eead0f700 1 -- 172.21.15.80:0/3128368133 >> 172.21.15.80:0/3128368133 conn(0x7f4ee4102040 msgr2=0x7f4ee41044a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:22.057 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:21.356+0000 7f4eead0f700 1 -- 172.21.15.80:0/3128368133 shutdown_connections 2024-04-24T23:19:22.057 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:21.356+0000 7f4eead0f700 1 -- 172.21.15.80:0/3128368133 wait complete. 2024-04-24T23:19:22.057 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:21.357+0000 7f4eead0f700 1 Processor -- start 2024-04-24T23:19:22.057 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:21.357+0000 7f4eead0f700 1 -- start start 2024-04-24T23:19:22.057 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:21.357+0000 7f4eead0f700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4ee4106460 0x7f4ee419deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:22.057 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:21.357+0000 7f4eead0f700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee4107fd0 con 0x7f4ee4106460 2024-04-24T23:19:22.058 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:21.357+0000 7f4ee8aab700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4ee4106460 0x7f4ee419deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:22.058 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:21.357+0000 7f4ee8aab700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4ee4106460 0x7f4ee419deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:51530/0 (socket says 172.21.15.80:51530) 2024-04-24T23:19:22.058 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:21.357+0000 7f4ee8aab700 1 -- 172.21.15.80:0/1650300422 learned_addr learned my addr 172.21.15.80:0/1650300422 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:19:22.058 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:21.358+0000 7f4ee8aab700 1 -- 172.21.15.80:0/1650300422 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ee419e3f0 con 0x7f4ee4106460 2024-04-24T23:19:22.058 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:21.358+0000 7f4ee8aab700 1 --2- 172.21.15.80:0/1650300422 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4ee4106460 0x7f4ee419deb0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4ed8011b60 tx=0x7f4ed8011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:22.058 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:21.358+0000 7f4ee1ffb700 1 -- 172.21.15.80:0/1650300422 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4ed8003970 con 0x7f4ee4106460 2024-04-24T23:19:22.058 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:21.358+0000 7f4ee1ffb700 1 -- 172.21.15.80:0/1650300422 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4ed8003b10 con 0x7f4ee4106460 2024-04-24T23:19:22.058 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:21.358+0000 7f4ee1ffb700 1 -- 172.21.15.80:0/1650300422 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4ed8024610 con 0x7f4ee4106460 2024-04-24T23:19:22.058 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:21.358+0000 7f4eead0f700 1 -- 172.21.15.80:0/1650300422 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ee419e680 con 0x7f4ee4106460 2024-04-24T23:19:22.058 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:21.358+0000 7f4eead0f700 1 -- 172.21.15.80:0/1650300422 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ee41a1370 con 0x7f4ee4106460 2024-04-24T23:19:22.058 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:21.358+0000 7f4ee1ffb700 1 -- 172.21.15.80:0/1650300422 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f4ed8024c50 con 0x7f4ee4106460 2024-04-24T23:19:22.058 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:21.358+0000 7f4ee1ffb700 1 -- 172.21.15.80:0/1650300422 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f4ed8011e00 con 0x7f4ee4106460 2024-04-24T23:19:22.058 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:21.359+0000 7f4eead0f700 1 -- 172.21.15.80:0/1650300422 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ee4063890 con 0x7f4ee4106460 2024-04-24T23:19:22.059 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:21.361+0000 7f4ee1ffb700 1 -- 172.21.15.80:0/1650300422 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f4ed801a9f0 con 0x7f4ee4106460 2024-04-24T23:19:22.059 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:21.433+0000 7f4eead0f700 1 -- 172.21.15.80:0/1650300422 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f4ee402dcf0 con 0x7f4ee4106460 2024-04-24T23:19:22.059 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:21.433+0000 7f4ee1ffb700 1 -- 172.21.15.80:0/1650300422 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f4ed801a430 con 0x7f4ee4106460 2024-04-24T23:19:22.059 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:21.435+0000 7f4eead0f700 1 -- 172.21.15.80:0/1650300422 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4ee4106460 msgr2=0x7f4ee419deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:22.059 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:21.435+0000 7f4eead0f700 1 --2- 172.21.15.80:0/1650300422 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4ee4106460 0x7f4ee419deb0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4ed8011b60 tx=0x7f4ed8011c40 comp rx=0 tx=0).stop 2024-04-24T23:19:22.059 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:21.435+0000 7f4eead0f700 1 -- 172.21.15.80:0/1650300422 shutdown_connections 2024-04-24T23:19:22.059 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:21.435+0000 7f4eead0f700 1 --2- 172.21.15.80:0/1650300422 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4ee4106460 0x7f4ee419deb0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:22.059 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:21.435+0000 7f4eead0f700 1 -- 172.21.15.80:0/1650300422 >> 172.21.15.80:0/1650300422 conn(0x7f4ee4102040 msgr2=0x7f4ee4194b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:22.059 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:21.435+0000 7f4eead0f700 1 -- 172.21.15.80:0/1650300422 shutdown_connections 2024-04-24T23:19:22.059 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:21.435+0000 7f4eead0f700 1 -- 172.21.15.80:0/1650300422 wait complete. 2024-04-24T23:19:22.059 INFO:teuthology.orchestra.run.smithi080.stdout:mgr not available, waiting (1/15)... 2024-04-24T23:19:23.194 INFO:journalctl@ceph.mgr.x.smithi080.stdout:Apr 24 23:19:22 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-x[29381]: 2024-04-24T23:19:22.857+0000 7fafac3b9200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-24T23:19:23.194 INFO:journalctl@ceph.mgr.x.smithi080.stdout:Apr 24 23:19:23 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-x[29381]: 2024-04-24T23:19:23.025+0000 7fafac3b9200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-24T23:19:23.194 INFO:journalctl@ceph.mgr.x.smithi080.stdout:Apr 24 23:19:23 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-x[29381]: 2024-04-24T23:19:23.192+0000 7fafac3b9200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-24T23:19:23.770 INFO:journalctl@ceph.mgr.x.smithi080.stdout:Apr 24 23:19:23 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-x[29381]: 2024-04-24T23:19:23.475+0000 7fafac3b9200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-24T23:19:23.771 INFO:journalctl@ceph.mgr.x.smithi080.stdout:Apr 24 23:19:23 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-x[29381]: 2024-04-24T23:19:23.566+0000 7fafac3b9200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-24T23:19:23.771 INFO:journalctl@ceph.mgr.x.smithi080.stdout:Apr 24 23:19:23 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-x[29381]: 2024-04-24T23:19:23.768+0000 7fafac3b9200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-24T23:19:24.039 INFO:journalctl@ceph.mgr.x.smithi080.stdout:Apr 24 23:19:23 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-x[29381]: 2024-04-24T23:19:23.957+0000 7fafac3b9200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-24T23:19:24.789 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:24 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/270110040' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-24T23:19:24.790 INFO:journalctl@ceph.mgr.x.smithi080.stdout:Apr 24 23:19:24 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-x[29381]: 2024-04-24T23:19:24.590+0000 7fafac3b9200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-24T23:19:25.116 INFO:journalctl@ceph.mgr.x.smithi080.stdout:Apr 24 23:19:25 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-x[29381]: 2024-04-24T23:19:25.022+0000 7fafac3b9200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-24T23:19:25.131 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout 2024-04-24T23:19:25.132 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout { 2024-04-24T23:19:25.132 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "fsid": "ec7885aa-0290-11ef-bc93-c7b262605968", 2024-04-24T23:19:25.132 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "health": { 2024-04-24T23:19:25.132 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-24T23:19:25.132 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-24T23:19:25.132 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-24T23:19:25.133 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:19:25.133 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-24T23:19:25.133 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-24T23:19:25.133 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout 0 2024-04-24T23:19:25.133 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout ], 2024-04-24T23:19:25.133 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-24T23:19:25.134 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "a" 2024-04-24T23:19:25.134 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout ], 2024-04-24T23:19:25.134 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "quorum_age": 6, 2024-04-24T23:19:25.134 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-24T23:19:25.134 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T23:19:25.134 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-24T23:19:25.134 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-24T23:19:25.134 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:19:25.134 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-24T23:19:25.135 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T23:19:25.135 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-24T23:19:25.135 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-24T23:19:25.135 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-24T23:19:25.135 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-24T23:19:25.135 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-24T23:19:25.135 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-24T23:19:25.135 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:19:25.135 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-24T23:19:25.135 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-24T23:19:25.135 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-24T23:19:25.135 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-24T23:19:25.135 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-24T23:19:25.136 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-24T23:19:25.136 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-24T23:19:25.136 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-24T23:19:25.136 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-24T23:19:25.136 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:19:25.136 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-24T23:19:25.136 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T23:19:25.136 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-24T23:19:25.136 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-24T23:19:25.136 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:19:25.136 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-24T23:19:25.136 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-24T23:19:25.136 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-24T23:19:25.136 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-24T23:19:25.137 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-24T23:19:25.137 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-24T23:19:25.137 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "restful" 2024-04-24T23:19:25.137 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout ], 2024-04-24T23:19:25.137 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-24T23:19:25.137 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:19:25.137 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-24T23:19:25.137 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T23:19:25.137 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "modified": "2024-04-24T23:19:05.586911+0000", 2024-04-24T23:19:25.137 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-24T23:19:25.137 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:19:25.137 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-24T23:19:25.137 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout } 2024-04-24T23:19:25.138 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:24.391+0000 7fac53b6c700 1 Processor -- start 2024-04-24T23:19:25.138 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:24.391+0000 7fac53b6c700 1 -- start start 2024-04-24T23:19:25.138 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:24.391+0000 7fac53b6c700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fac4c1092d0 0x7fac4c1096f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:25.138 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:24.391+0000 7fac53b6c700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fac4c109cc0 con 0x7fac4c1092d0 2024-04-24T23:19:25.138 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:24.392+0000 7fac51908700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fac4c1092d0 0x7fac4c1096f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:25.138 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:24.392+0000 7fac51908700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fac4c1092d0 0x7fac4c1096f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53220/0 (socket says 172.21.15.80:53220) 2024-04-24T23:19:25.138 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:24.392+0000 7fac51908700 1 -- 172.21.15.80:0/89194370 learned_addr learned my addr 172.21.15.80:0/89194370 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:19:25.138 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:24.392+0000 7fac51908700 1 -- 172.21.15.80:0/89194370 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac4c105b20 con 0x7fac4c1092d0 2024-04-24T23:19:25.138 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:24.392+0000 7fac51908700 1 --2- 172.21.15.80:0/89194370 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fac4c1092d0 0x7fac4c1096f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fac480098a0 tx=0x7fac48009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a16267af818379f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:25.138 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:24.392+0000 7fac50906700 1 -- 172.21.15.80:0/89194370 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fac48004160 con 0x7fac4c1092d0 2024-04-24T23:19:25.138 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:24.392+0000 7fac50906700 1 -- 172.21.15.80:0/89194370 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fac48004300 con 0x7fac4c1092d0 2024-04-24T23:19:25.138 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:24.392+0000 7fac50906700 1 -- 172.21.15.80:0/89194370 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fac48011b50 con 0x7fac4c1092d0 2024-04-24T23:19:25.138 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:24.393+0000 7fac53b6c700 1 -- 172.21.15.80:0/89194370 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fac4c1092d0 msgr2=0x7fac4c1096f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:25.139 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:24.393+0000 7fac53b6c700 1 --2- 172.21.15.80:0/89194370 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fac4c1092d0 0x7fac4c1096f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fac480098a0 tx=0x7fac48009bb0 comp rx=0 tx=0).stop 2024-04-24T23:19:25.139 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:24.393+0000 7fac53b6c700 1 -- 172.21.15.80:0/89194370 shutdown_connections 2024-04-24T23:19:25.139 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:24.393+0000 7fac53b6c700 1 --2- 172.21.15.80:0/89194370 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fac4c1092d0 0x7fac4c1096f0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:25.139 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:24.393+0000 7fac53b6c700 1 -- 172.21.15.80:0/89194370 >> 172.21.15.80:0/89194370 conn(0x7fac4c1017e0 msgr2=0x7fac4c103c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:25.139 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:24.393+0000 7fac53b6c700 1 -- 172.21.15.80:0/89194370 shutdown_connections 2024-04-24T23:19:25.139 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:24.393+0000 7fac53b6c700 1 -- 172.21.15.80:0/89194370 wait complete. 2024-04-24T23:19:25.139 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:24.394+0000 7fac53b6c700 1 Processor -- start 2024-04-24T23:19:25.139 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:24.394+0000 7fac53b6c700 1 -- start start 2024-04-24T23:19:25.139 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:24.394+0000 7fac53b6c700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fac4c1092d0 0x7fac4c19bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:25.139 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:24.394+0000 7fac53b6c700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fac4c10c1f0 con 0x7fac4c1092d0 2024-04-24T23:19:25.139 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:24.394+0000 7fac51908700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fac4c1092d0 0x7fac4c19bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:25.139 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:24.394+0000 7fac51908700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fac4c1092d0 0x7fac4c19bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53224/0 (socket says 172.21.15.80:53224) 2024-04-24T23:19:25.139 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:24.394+0000 7fac51908700 1 -- 172.21.15.80:0/270110040 learned_addr learned my addr 172.21.15.80:0/270110040 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:19:25.140 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:24.394+0000 7fac51908700 1 -- 172.21.15.80:0/270110040 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac4c19c250 con 0x7fac4c1092d0 2024-04-24T23:19:25.140 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:24.394+0000 7fac51908700 1 --2- 172.21.15.80:0/270110040 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fac4c1092d0 0x7fac4c19bd10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fac48011850 tx=0x7fac48011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:25.140 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:24.395+0000 7fac42ffd700 1 -- 172.21.15.80:0/270110040 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fac48003af0 con 0x7fac4c1092d0 2024-04-24T23:19:25.140 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:24.395+0000 7fac53b6c700 1 -- 172.21.15.80:0/270110040 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac4c19c4e0 con 0x7fac4c1092d0 2024-04-24T23:19:25.140 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:24.395+0000 7fac42ffd700 1 -- 172.21.15.80:0/270110040 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fac48020070 con 0x7fac4c1092d0 2024-04-24T23:19:25.140 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:24.395+0000 7fac42ffd700 1 -- 172.21.15.80:0/270110040 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fac480195a0 con 0x7fac4c1092d0 2024-04-24T23:19:25.140 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:24.395+0000 7fac53b6c700 1 -- 172.21.15.80:0/270110040 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac4c19f1c0 con 0x7fac4c1092d0 2024-04-24T23:19:25.140 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:24.395+0000 7fac42ffd700 1 -- 172.21.15.80:0/270110040 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fac48019c30 con 0x7fac4c1092d0 2024-04-24T23:19:25.140 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:24.395+0000 7fac42ffd700 1 -- 172.21.15.80:0/270110040 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fac48011e00 con 0x7fac4c1092d0 2024-04-24T23:19:25.140 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:24.396+0000 7fac53b6c700 1 -- 172.21.15.80:0/270110040 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac4c10c370 con 0x7fac4c1092d0 2024-04-24T23:19:25.140 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:24.398+0000 7fac42ffd700 1 -- 172.21.15.80:0/270110040 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fac48003c90 con 0x7fac4c1092d0 2024-04-24T23:19:25.140 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:24.470+0000 7fac53b6c700 1 -- 172.21.15.80:0/270110040 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fac4c063890 con 0x7fac4c1092d0 2024-04-24T23:19:25.140 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:24.471+0000 7fac42ffd700 1 -- 172.21.15.80:0/270110040 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fac4801a430 con 0x7fac4c1092d0 2024-04-24T23:19:25.140 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:24.472+0000 7fac53b6c700 1 -- 172.21.15.80:0/270110040 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fac4c1092d0 msgr2=0x7fac4c19bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:25.140 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:24.472+0000 7fac53b6c700 1 --2- 172.21.15.80:0/270110040 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fac4c1092d0 0x7fac4c19bd10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fac48011850 tx=0x7fac48011c10 comp rx=0 tx=0).stop 2024-04-24T23:19:25.141 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:24.472+0000 7fac53b6c700 1 -- 172.21.15.80:0/270110040 shutdown_connections 2024-04-24T23:19:25.141 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:24.472+0000 7fac53b6c700 1 --2- 172.21.15.80:0/270110040 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fac4c1092d0 0x7fac4c19bd10 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:25.141 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:24.472+0000 7fac53b6c700 1 -- 172.21.15.80:0/270110040 >> 172.21.15.80:0/270110040 conn(0x7fac4c1017e0 msgr2=0x7fac4c1023e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:25.141 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:24.472+0000 7fac53b6c700 1 -- 172.21.15.80:0/270110040 shutdown_connections 2024-04-24T23:19:25.141 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:24.472+0000 7fac53b6c700 1 -- 172.21.15.80:0/270110040 wait complete. 2024-04-24T23:19:25.141 INFO:teuthology.orchestra.run.smithi080.stdout:mgr not available, waiting (2/15)... 2024-04-24T23:19:25.539 INFO:journalctl@ceph.mgr.x.smithi080.stdout:Apr 24 23:19:25 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-x[29381]: 2024-04-24T23:19:25.120+0000 7fafac3b9200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-24T23:19:25.539 INFO:journalctl@ceph.mgr.x.smithi080.stdout:Apr 24 23:19:25 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-x[29381]: 2024-04-24T23:19:25.320+0000 7fafac3b9200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-24T23:19:26.180 INFO:journalctl@ceph.mgr.x.smithi080.stdout:Apr 24 23:19:25 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-x[29381]: 2024-04-24T23:19:25.850+0000 7fafac3b9200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-24T23:19:26.180 INFO:journalctl@ceph.mgr.x.smithi080.stdout:Apr 24 23:19:25 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-x[29381]: 2024-04-24T23:19:25.939+0000 7fafac3b9200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-24T23:19:26.455 INFO:journalctl@ceph.mgr.x.smithi080.stdout:Apr 24 23:19:26 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-x[29381]: 2024-04-24T23:19:26.178+0000 7fafac3b9200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-24T23:19:26.455 INFO:journalctl@ceph.mgr.x.smithi080.stdout:Apr 24 23:19:26 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-x[29381]: 2024-04-24T23:19:26.318+0000 7fafac3b9200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-24T23:19:26.455 INFO:journalctl@ceph.mgr.x.smithi080.stdout:Apr 24 23:19:26 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-x[29381]: 2024-04-24T23:19:26.416+0000 7fafac3b9200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-24T23:19:26.789 INFO:journalctl@ceph.mgr.x.smithi080.stdout:Apr 24 23:19:26 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-x[29381]: 2024-04-24T23:19:26.537+0000 7fafac3b9200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-24T23:19:26.789 INFO:journalctl@ceph.mgr.x.smithi080.stdout:Apr 24 23:19:26 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-x[29381]: 2024-04-24T23:19:26.634+0000 7fafac3b9200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-24T23:19:27.528 INFO:journalctl@ceph.mgr.x.smithi080.stdout:Apr 24 23:19:27 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-x[29381]: 2024-04-24T23:19:27.489+0000 7fafac3b9200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-24T23:19:27.789 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:27 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/3592082867' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-24T23:19:27.790 INFO:journalctl@ceph.mgr.x.smithi080.stdout:Apr 24 23:19:27 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-x[29381]: 2024-04-24T23:19:27.711+0000 7fafac3b9200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-24T23:19:28.199 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout 2024-04-24T23:19:28.202 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout { 2024-04-24T23:19:28.202 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "fsid": "ec7885aa-0290-11ef-bc93-c7b262605968", 2024-04-24T23:19:28.202 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "health": { 2024-04-24T23:19:28.202 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-24T23:19:28.202 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-24T23:19:28.202 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-24T23:19:28.202 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:19:28.203 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-24T23:19:28.203 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-24T23:19:28.203 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout 0 2024-04-24T23:19:28.203 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout ], 2024-04-24T23:19:28.203 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-24T23:19:28.203 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "a" 2024-04-24T23:19:28.203 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout ], 2024-04-24T23:19:28.203 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-04-24T23:19:28.203 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-24T23:19:28.203 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T23:19:28.203 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-24T23:19:28.203 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-24T23:19:28.203 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:19:28.203 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-24T23:19:28.204 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T23:19:28.204 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-24T23:19:28.204 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-24T23:19:28.204 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-24T23:19:28.215 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-24T23:19:28.215 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-24T23:19:28.215 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-24T23:19:28.215 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:19:28.215 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-24T23:19:28.215 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-24T23:19:28.216 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-24T23:19:28.216 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-24T23:19:28.216 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-24T23:19:28.216 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-24T23:19:28.216 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-24T23:19:28.216 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-24T23:19:28.216 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-24T23:19:28.216 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:19:28.216 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-24T23:19:28.216 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T23:19:28.216 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-24T23:19:28.216 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-24T23:19:28.216 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:19:28.216 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-24T23:19:28.217 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-24T23:19:28.217 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-24T23:19:28.217 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-24T23:19:28.217 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-24T23:19:28.217 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-24T23:19:28.217 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "restful" 2024-04-24T23:19:28.217 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout ], 2024-04-24T23:19:28.217 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-24T23:19:28.217 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:19:28.217 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-24T23:19:28.217 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T23:19:28.217 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "modified": "2024-04-24T23:19:05.586911+0000", 2024-04-24T23:19:28.217 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-24T23:19:28.217 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:19:28.218 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-24T23:19:28.218 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout } 2024-04-24T23:19:28.218 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:27.526+0000 7f22ab1a4700 1 Processor -- start 2024-04-24T23:19:28.218 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:27.526+0000 7f22ab1a4700 1 -- start start 2024-04-24T23:19:28.218 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:27.526+0000 7f22ab1a4700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f22a41092a0 0x7f22a41096c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:28.218 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:27.526+0000 7f22ab1a4700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f22a4109c90 con 0x7f22a41092a0 2024-04-24T23:19:28.218 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:27.527+0000 7f22a8f40700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f22a41092a0 0x7f22a41096c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:28.218 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:27.527+0000 7f22a8f40700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f22a41092a0 0x7f22a41096c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53234/0 (socket says 172.21.15.80:53234) 2024-04-24T23:19:28.218 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:27.527+0000 7f22a8f40700 1 -- 172.21.15.80:0/2112237057 learned_addr learned my addr 172.21.15.80:0/2112237057 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:19:28.218 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:27.527+0000 7f22a8f40700 1 -- 172.21.15.80:0/2112237057 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22a4105b40 con 0x7f22a41092a0 2024-04-24T23:19:28.218 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:27.527+0000 7f22a8f40700 1 --2- 172.21.15.80:0/2112237057 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f22a41092a0 0x7f22a41096c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f22980098a0 tx=0x7f2298009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72e8b761cd1de671 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:28.218 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:27.527+0000 7f22a37fe700 1 -- 172.21.15.80:0/2112237057 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2298004160 con 0x7f22a41092a0 2024-04-24T23:19:28.218 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:27.527+0000 7f22a37fe700 1 -- 172.21.15.80:0/2112237057 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2298004300 con 0x7f22a41092a0 2024-04-24T23:19:28.219 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:27.527+0000 7f22a37fe700 1 -- 172.21.15.80:0/2112237057 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2298011b50 con 0x7f22a41092a0 2024-04-24T23:19:28.219 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:27.528+0000 7f22ab1a4700 1 -- 172.21.15.80:0/2112237057 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f22a41092a0 msgr2=0x7f22a41096c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:28.219 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:27.528+0000 7f22ab1a4700 1 --2- 172.21.15.80:0/2112237057 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f22a41092a0 0x7f22a41096c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f22980098a0 tx=0x7f2298009bb0 comp rx=0 tx=0).stop 2024-04-24T23:19:28.219 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:27.528+0000 7f22ab1a4700 1 -- 172.21.15.80:0/2112237057 shutdown_connections 2024-04-24T23:19:28.219 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:27.528+0000 7f22ab1a4700 1 --2- 172.21.15.80:0/2112237057 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f22a41092a0 0x7f22a41096c0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:28.219 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:27.528+0000 7f22ab1a4700 1 -- 172.21.15.80:0/2112237057 >> 172.21.15.80:0/2112237057 conn(0x7f22a41017e0 msgr2=0x7f22a4103c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:28.219 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:27.528+0000 7f22ab1a4700 1 -- 172.21.15.80:0/2112237057 shutdown_connections 2024-04-24T23:19:28.219 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:27.528+0000 7f22ab1a4700 1 -- 172.21.15.80:0/2112237057 wait complete. 2024-04-24T23:19:28.219 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:27.529+0000 7f22ab1a4700 1 Processor -- start 2024-04-24T23:19:28.219 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:27.529+0000 7f22ab1a4700 1 -- start start 2024-04-24T23:19:28.219 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:27.529+0000 7f22ab1a4700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f22a41092a0 0x7f22a41086e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:28.219 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:27.529+0000 7f22ab1a4700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f22a410c210 con 0x7f22a41092a0 2024-04-24T23:19:28.219 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:27.529+0000 7f22a8f40700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f22a41092a0 0x7f22a41086e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:28.219 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:27.529+0000 7f22a8f40700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f22a41092a0 0x7f22a41086e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53246/0 (socket says 172.21.15.80:53246) 2024-04-24T23:19:28.220 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:27.529+0000 7f22a8f40700 1 -- 172.21.15.80:0/3592082867 learned_addr learned my addr 172.21.15.80:0/3592082867 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:19:28.220 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:27.530+0000 7f22a8f40700 1 -- 172.21.15.80:0/3592082867 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22a4108c20 con 0x7f22a41092a0 2024-04-24T23:19:28.220 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:27.530+0000 7f22a8f40700 1 --2- 172.21.15.80:0/3592082867 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f22a41092a0 0x7f22a41086e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2298011850 tx=0x7f2298011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:28.220 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:27.530+0000 7f22a1ffb700 1 -- 172.21.15.80:0/3592082867 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2298003af0 con 0x7f22a41092a0 2024-04-24T23:19:28.220 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:27.530+0000 7f22a1ffb700 1 -- 172.21.15.80:0/3592082867 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2298020070 con 0x7f22a41092a0 2024-04-24T23:19:28.220 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:27.530+0000 7f22ab1a4700 1 -- 172.21.15.80:0/3592082867 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22a4106e10 con 0x7f22a41092a0 2024-04-24T23:19:28.220 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:27.530+0000 7f22a1ffb700 1 -- 172.21.15.80:0/3592082867 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f22980195a0 con 0x7f22a41092a0 2024-04-24T23:19:28.220 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:27.530+0000 7f22ab1a4700 1 -- 172.21.15.80:0/3592082867 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22a4107350 con 0x7f22a41092a0 2024-04-24T23:19:28.220 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:27.530+0000 7f22a1ffb700 1 -- 172.21.15.80:0/3592082867 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f2298019c30 con 0x7f22a41092a0 2024-04-24T23:19:28.220 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:27.530+0000 7f22a1ffb700 1 -- 172.21.15.80:0/3592082867 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f2298011e00 con 0x7f22a41092a0 2024-04-24T23:19:28.220 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:27.531+0000 7f22ab1a4700 1 -- 172.21.15.80:0/3592082867 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22a4063890 con 0x7f22a41092a0 2024-04-24T23:19:28.220 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:27.533+0000 7f22a1ffb700 1 -- 172.21.15.80:0/3592082867 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f2298003c90 con 0x7f22a41092a0 2024-04-24T23:19:28.220 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:27.606+0000 7f22ab1a4700 1 -- 172.21.15.80:0/3592082867 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f22a4107980 con 0x7f22a41092a0 2024-04-24T23:19:28.221 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:27.606+0000 7f22a1ffb700 1 -- 172.21.15.80:0/3592082867 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f229801a430 con 0x7f22a41092a0 2024-04-24T23:19:28.221 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:27.607+0000 7f22ab1a4700 1 -- 172.21.15.80:0/3592082867 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f22a41092a0 msgr2=0x7f22a41086e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:28.221 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:27.607+0000 7f22ab1a4700 1 --2- 172.21.15.80:0/3592082867 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f22a41092a0 0x7f22a41086e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2298011850 tx=0x7f2298011c10 comp rx=0 tx=0).stop 2024-04-24T23:19:28.221 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:27.607+0000 7f22ab1a4700 1 -- 172.21.15.80:0/3592082867 shutdown_connections 2024-04-24T23:19:28.221 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:27.607+0000 7f22ab1a4700 1 --2- 172.21.15.80:0/3592082867 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f22a41092a0 0x7f22a41086e0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:28.221 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:27.607+0000 7f22ab1a4700 1 -- 172.21.15.80:0/3592082867 >> 172.21.15.80:0/3592082867 conn(0x7f22a41017e0 msgr2=0x7f22a4102320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:28.221 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:27.608+0000 7f22ab1a4700 1 -- 172.21.15.80:0/3592082867 shutdown_connections 2024-04-24T23:19:28.221 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:27.608+0000 7f22ab1a4700 1 -- 172.21.15.80:0/3592082867 wait complete. 2024-04-24T23:19:28.221 INFO:teuthology.orchestra.run.smithi080.stdout:mgr not available, waiting (3/15)... 2024-04-24T23:19:29.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:28 smithi080 ceph-mon[29152]: Activating manager daemon x 2024-04-24T23:19:29.040 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:28 smithi080 ceph-mon[29152]: mgrmap e2: x(active, starting, since 0.00298561s) 2024-04-24T23:19:29.040 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:28 smithi080 ceph-mon[29152]: from='mgr.14100 172.21.15.80:0/712979829' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-24T23:19:29.040 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:28 smithi080 ceph-mon[29152]: from='mgr.14100 172.21.15.80:0/712979829' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-24T23:19:29.040 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:28 smithi080 ceph-mon[29152]: from='mgr.14100 172.21.15.80:0/712979829' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-24T23:19:29.040 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:28 smithi080 ceph-mon[29152]: from='mgr.14100 172.21.15.80:0/712979829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T23:19:29.040 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:28 smithi080 ceph-mon[29152]: from='mgr.14100 172.21.15.80:0/712979829' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-24T23:19:29.040 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:28 smithi080 ceph-mon[29152]: Manager daemon x is now available 2024-04-24T23:19:29.040 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:28 smithi080 ceph-mon[29152]: from='mgr.14100 172.21.15.80:0/712979829' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-24T23:19:29.040 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:28 smithi080 ceph-mon[29152]: from='mgr.14100 172.21.15.80:0/712979829' entity='mgr.x' 2024-04-24T23:19:29.040 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:28 smithi080 ceph-mon[29152]: from='mgr.14100 172.21.15.80:0/712979829' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-24T23:19:29.040 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:28 smithi080 ceph-mon[29152]: from='mgr.14100 172.21.15.80:0/712979829' entity='mgr.x' 2024-04-24T23:19:29.040 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:28 smithi080 ceph-mon[29152]: from='mgr.14100 172.21.15.80:0/712979829' entity='mgr.x' 2024-04-24T23:19:30.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:29 smithi080 ceph-mon[29152]: mgrmap e3: x(active, since 1.00677s) 2024-04-24T23:19:30.902 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:30 smithi080 ceph-mon[29152]: mgrmap e4: x(active, since 2s) 2024-04-24T23:19:31.553 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout 2024-04-24T23:19:31.553 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout { 2024-04-24T23:19:31.553 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "fsid": "ec7885aa-0290-11ef-bc93-c7b262605968", 2024-04-24T23:19:31.553 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "health": { 2024-04-24T23:19:31.554 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-24T23:19:31.554 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-24T23:19:31.554 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-24T23:19:31.554 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:19:31.554 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-24T23:19:31.554 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-24T23:19:31.554 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout 0 2024-04-24T23:19:31.554 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout ], 2024-04-24T23:19:31.554 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-24T23:19:31.555 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "a" 2024-04-24T23:19:31.555 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout ], 2024-04-24T23:19:31.555 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "quorum_age": 12, 2024-04-24T23:19:31.555 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-24T23:19:31.555 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T23:19:31.555 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-24T23:19:31.555 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-24T23:19:31.555 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:19:31.556 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-24T23:19:31.556 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T23:19:31.556 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-24T23:19:31.556 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-24T23:19:31.556 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-24T23:19:31.556 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-24T23:19:31.556 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-24T23:19:31.556 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-24T23:19:31.556 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:19:31.556 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-24T23:19:31.556 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-24T23:19:31.557 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-24T23:19:31.557 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-24T23:19:31.557 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-24T23:19:31.557 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-24T23:19:31.557 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-24T23:19:31.557 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-24T23:19:31.557 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-24T23:19:31.557 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:19:31.557 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-24T23:19:31.557 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T23:19:31.557 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-24T23:19:31.558 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-24T23:19:31.558 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:19:31.558 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-24T23:19:31.558 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-24T23:19:31.558 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-24T23:19:31.558 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-24T23:19:31.558 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-24T23:19:31.558 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-24T23:19:31.558 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "restful" 2024-04-24T23:19:31.558 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout ], 2024-04-24T23:19:31.558 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-24T23:19:31.559 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:19:31.559 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-24T23:19:31.559 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T23:19:31.559 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "modified": "2024-04-24T23:19:05.586911+0000", 2024-04-24T23:19:31.559 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-24T23:19:31.559 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:19:31.559 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-24T23:19:31.559 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout } 2024-04-24T23:19:31.559 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:30.617+0000 7f8438832700 1 Processor -- start 2024-04-24T23:19:31.559 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:30.617+0000 7f8438832700 1 -- start start 2024-04-24T23:19:31.559 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:30.617+0000 7f8438832700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8434108690 0x7f843410aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:31.560 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:30.617+0000 7f8438832700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f843407bda0 con 0x7f8434108690 2024-04-24T23:19:31.560 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:30.617+0000 7f843259c700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8434108690 0x7f843410aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:31.560 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:30.617+0000 7f843259c700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8434108690 0x7f843410aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53318/0 (socket says 172.21.15.80:53318) 2024-04-24T23:19:31.560 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:30.617+0000 7f843259c700 1 -- 172.21.15.80:0/2488796755 learned_addr learned my addr 172.21.15.80:0/2488796755 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:19:31.560 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:30.618+0000 7f843259c700 1 -- 172.21.15.80:0/2488796755 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f843410afc0 con 0x7f8434108690 2024-04-24T23:19:31.560 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:30.618+0000 7f843259c700 1 --2- 172.21.15.80:0/2488796755 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8434108690 0x7f843410aa80 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f84240098a0 tx=0x7f8424009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4514c2359762d16c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:31.560 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:30.618+0000 7f8431d9b700 1 -- 172.21.15.80:0/2488796755 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8424004160 con 0x7f8434108690 2024-04-24T23:19:31.560 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:30.618+0000 7f8431d9b700 1 -- 172.21.15.80:0/2488796755 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8424004300 con 0x7f8434108690 2024-04-24T23:19:31.560 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:30.618+0000 7f8431d9b700 1 -- 172.21.15.80:0/2488796755 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8424011b50 con 0x7f8434108690 2024-04-24T23:19:31.560 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:30.619+0000 7f8438832700 1 -- 172.21.15.80:0/2488796755 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8434108690 msgr2=0x7f843410aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:31.560 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:30.619+0000 7f8438832700 1 --2- 172.21.15.80:0/2488796755 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8434108690 0x7f843410aa80 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f84240098a0 tx=0x7f8424009bb0 comp rx=0 tx=0).stop 2024-04-24T23:19:31.561 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:30.619+0000 7f8438832700 1 -- 172.21.15.80:0/2488796755 shutdown_connections 2024-04-24T23:19:31.561 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:30.619+0000 7f8438832700 1 --2- 172.21.15.80:0/2488796755 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8434108690 0x7f843410aa80 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:31.561 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:30.619+0000 7f8438832700 1 -- 172.21.15.80:0/2488796755 >> 172.21.15.80:0/2488796755 conn(0x7f8434101fe0 msgr2=0x7f8434104460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:31.561 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:30.619+0000 7f8438832700 1 -- 172.21.15.80:0/2488796755 shutdown_connections 2024-04-24T23:19:31.561 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:30.619+0000 7f8438832700 1 -- 172.21.15.80:0/2488796755 wait complete. 2024-04-24T23:19:31.561 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:30.619+0000 7f8438832700 1 Processor -- start 2024-04-24T23:19:31.561 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:30.620+0000 7f8438832700 1 -- start start 2024-04-24T23:19:31.561 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:30.620+0000 7f8438832700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8434108690 0x7f843419bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:31.561 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:30.620+0000 7f8438832700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f843410bf80 con 0x7f8434108690 2024-04-24T23:19:31.561 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:30.620+0000 7f843259c700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8434108690 0x7f843419bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:31.561 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:30.620+0000 7f843259c700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8434108690 0x7f843419bb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53328/0 (socket says 172.21.15.80:53328) 2024-04-24T23:19:31.561 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:30.620+0000 7f843259c700 1 -- 172.21.15.80:0/2971419345 learned_addr learned my addr 172.21.15.80:0/2971419345 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:19:31.562 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:30.620+0000 7f843259c700 1 -- 172.21.15.80:0/2971419345 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f843419c090 con 0x7f8434108690 2024-04-24T23:19:31.562 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:30.620+0000 7f843259c700 1 --2- 172.21.15.80:0/2971419345 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8434108690 0x7f843419bb50 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8424011b60 tx=0x7f8424011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:31.562 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:30.620+0000 7f842bfff700 1 -- 172.21.15.80:0/2971419345 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8424003b20 con 0x7f8434108690 2024-04-24T23:19:31.562 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:30.620+0000 7f8438832700 1 -- 172.21.15.80:0/2971419345 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f843419c320 con 0x7f8434108690 2024-04-24T23:19:31.562 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:30.620+0000 7f842bfff700 1 -- 172.21.15.80:0/2971419345 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8424003cc0 con 0x7f8434108690 2024-04-24T23:19:31.562 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:30.620+0000 7f842bfff700 1 -- 172.21.15.80:0/2971419345 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8424024840 con 0x7f8434108690 2024-04-24T23:19:31.562 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:30.621+0000 7f8438832700 1 -- 172.21.15.80:0/2971419345 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f843419f010 con 0x7f8434108690 2024-04-24T23:19:31.562 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:30.621+0000 7f842bfff700 1 -- 172.21.15.80:0/2971419345 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f8424022020 con 0x7f8434108690 2024-04-24T23:19:31.563 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:30.621+0000 7f8438832700 1 -- 172.21.15.80:0/2971419345 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8434063890 con 0x7f8434108690 2024-04-24T23:19:31.563 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:30.622+0000 7f842bfff700 1 --2- 172.21.15.80:0/2971419345 >> [v2:172.21.15.80:6800/1614691410,v1:172.21.15.80:6801/1614691410] conn(0x7f8420039000 0x7f842003b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:31.563 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:30.622+0000 7f842bfff700 1 -- 172.21.15.80:0/2971419345 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f8424011e00 con 0x7f8434108690 2024-04-24T23:19:31.563 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:30.622+0000 7f8429d9b700 1 --2- 172.21.15.80:0/2971419345 >> [v2:172.21.15.80:6800/1614691410,v1:172.21.15.80:6801/1614691410] conn(0x7f8420039000 0x7f842003b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:31.563 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:30.622+0000 7f8429d9b700 1 --2- 172.21.15.80:0/2971419345 >> [v2:172.21.15.80:6800/1614691410,v1:172.21.15.80:6801/1614691410] conn(0x7f8420039000 0x7f842003b4c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f841c006fd0 tx=0x7f841c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:31.563 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:30.627+0000 7f842bfff700 1 -- 172.21.15.80:0/2971419345 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f84240172b0 con 0x7f8434108690 2024-04-24T23:19:31.563 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:30.901+0000 7f8438832700 1 -- 172.21.15.80:0/2971419345 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f843419f1e0 con 0x7f8434108690 2024-04-24T23:19:31.563 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:30.901+0000 7f842bfff700 1 -- 172.21.15.80:0/2971419345 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f843419f1e0 con 0x7f8434108690 2024-04-24T23:19:31.563 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:30.904+0000 7f8438832700 1 -- 172.21.15.80:0/2971419345 >> [v2:172.21.15.80:6800/1614691410,v1:172.21.15.80:6801/1614691410] conn(0x7f8420039000 msgr2=0x7f842003b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:31.563 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:30.904+0000 7f8438832700 1 --2- 172.21.15.80:0/2971419345 >> [v2:172.21.15.80:6800/1614691410,v1:172.21.15.80:6801/1614691410] conn(0x7f8420039000 0x7f842003b4c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f841c006fd0 tx=0x7f841c006e40 comp rx=0 tx=0).stop 2024-04-24T23:19:31.564 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:30.904+0000 7f8438832700 1 -- 172.21.15.80:0/2971419345 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8434108690 msgr2=0x7f843419bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:31.564 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:30.904+0000 7f8438832700 1 --2- 172.21.15.80:0/2971419345 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8434108690 0x7f843419bb50 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8424011b60 tx=0x7f8424011c40 comp rx=0 tx=0).stop 2024-04-24T23:19:31.564 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:30.904+0000 7f8438832700 1 -- 172.21.15.80:0/2971419345 shutdown_connections 2024-04-24T23:19:31.564 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:30.904+0000 7f8438832700 1 --2- 172.21.15.80:0/2971419345 >> [v2:172.21.15.80:6800/1614691410,v1:172.21.15.80:6801/1614691410] conn(0x7f8420039000 0x7f842003b4c0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:31.564 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:30.904+0000 7f8438832700 1 --2- 172.21.15.80:0/2971419345 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8434108690 0x7f843419bb50 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:31.564 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:30.904+0000 7f8438832700 1 -- 172.21.15.80:0/2971419345 >> 172.21.15.80:0/2971419345 conn(0x7f8434101fe0 msgr2=0x7f8434102bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:31.564 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:30.904+0000 7f8438832700 1 -- 172.21.15.80:0/2971419345 shutdown_connections 2024-04-24T23:19:31.564 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:30.904+0000 7f8438832700 1 -- 172.21.15.80:0/2971419345 wait complete. 2024-04-24T23:19:31.564 INFO:teuthology.orchestra.run.smithi080.stdout:mgr is available 2024-04-24T23:19:31.939 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:31 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/2971419345' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-24T23:19:32.742 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:32 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/1826476559' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-24T23:19:32.752 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout 2024-04-24T23:19:32.752 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout [global] 2024-04-24T23:19:32.752 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout fsid = ec7885aa-0290-11ef-bc93-c7b262605968 2024-04-24T23:19:32.752 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-24T23:19:32.752 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-24T23:19:32.752 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-24T23:19:32.752 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-24T23:19:32.752 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout 2024-04-24T23:19:32.752 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-24T23:19:32.752 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-24T23:19:32.752 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout 2024-04-24T23:19:32.752 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout [osd] 2024-04-24T23:19:32.753 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-24T23:19:32.753 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-24T23:19:32.753 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-24T23:19:32.753 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-24T23:19:32.753 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-24T23:19:32.753 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:31.935+0000 7fb4aac3c700 1 Processor -- start 2024-04-24T23:19:32.753 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:31.935+0000 7fb4aac3c700 1 -- start start 2024-04-24T23:19:32.753 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:31.936+0000 7fb4aac3c700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb4a4106200 0x7fb4a4106620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:32.753 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:31.936+0000 7fb4aac3c700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a4106bf0 con 0x7fb4a4106200 2024-04-24T23:19:32.753 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:31.936+0000 7fb4a89d8700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb4a4106200 0x7fb4a4106620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:32.753 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:31.936+0000 7fb4a89d8700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb4a4106200 0x7fb4a4106620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53340/0 (socket says 172.21.15.80:53340) 2024-04-24T23:19:32.753 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:31.936+0000 7fb4a89d8700 1 -- 172.21.15.80:0/3599620322 learned_addr learned my addr 172.21.15.80:0/3599620322 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:19:32.753 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:31.936+0000 7fb4a89d8700 1 -- 172.21.15.80:0/3599620322 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4a4107400 con 0x7fb4a4106200 2024-04-24T23:19:32.753 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:31.936+0000 7fb4a89d8700 1 --2- 172.21.15.80:0/3599620322 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb4a4106200 0x7fb4a4106620 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fb49400aa40 tx=0x7fb49400ad50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=486ef3abb6c482f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:32.754 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:31.936+0000 7fb4a37fe700 1 -- 172.21.15.80:0/3599620322 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb49400b040 con 0x7fb4a4106200 2024-04-24T23:19:32.754 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:31.936+0000 7fb4a37fe700 1 -- 172.21.15.80:0/3599620322 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb49400c8f0 con 0x7fb4a4106200 2024-04-24T23:19:32.754 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:31.936+0000 7fb4a37fe700 1 -- 172.21.15.80:0/3599620322 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb494003c10 con 0x7fb4a4106200 2024-04-24T23:19:32.754 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:31.937+0000 7fb4aac3c700 1 -- 172.21.15.80:0/3599620322 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb4a4106200 msgr2=0x7fb4a4106620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:32.754 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:31.937+0000 7fb4aac3c700 1 --2- 172.21.15.80:0/3599620322 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb4a4106200 0x7fb4a4106620 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fb49400aa40 tx=0x7fb49400ad50 comp rx=0 tx=0).stop 2024-04-24T23:19:32.754 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:31.937+0000 7fb4aac3c700 1 -- 172.21.15.80:0/3599620322 shutdown_connections 2024-04-24T23:19:32.754 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:31.937+0000 7fb4aac3c700 1 --2- 172.21.15.80:0/3599620322 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb4a4106200 0x7fb4a4106620 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:32.754 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:31.937+0000 7fb4aac3c700 1 -- 172.21.15.80:0/3599620322 >> 172.21.15.80:0/3599620322 conn(0x7fb4a4101780 msgr2=0x7fb4a4103be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:32.754 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:31.937+0000 7fb4aac3c700 1 -- 172.21.15.80:0/3599620322 shutdown_connections 2024-04-24T23:19:32.754 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:31.937+0000 7fb4aac3c700 1 -- 172.21.15.80:0/3599620322 wait complete. 2024-04-24T23:19:32.754 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:31.938+0000 7fb4aac3c700 1 Processor -- start 2024-04-24T23:19:32.754 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:31.938+0000 7fb4aac3c700 1 -- start start 2024-04-24T23:19:32.754 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:31.938+0000 7fb4aac3c700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb4a4106200 0x7fb4a419bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:32.754 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:31.938+0000 7fb4aac3c700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a4107b80 con 0x7fb4a4106200 2024-04-24T23:19:32.755 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:31.939+0000 7fb4a89d8700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb4a4106200 0x7fb4a419bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:32.755 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:31.939+0000 7fb4a89d8700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb4a4106200 0x7fb4a419bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53348/0 (socket says 172.21.15.80:53348) 2024-04-24T23:19:32.755 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:31.939+0000 7fb4a89d8700 1 -- 172.21.15.80:0/1826476559 learned_addr learned my addr 172.21.15.80:0/1826476559 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:19:32.755 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:31.939+0000 7fb4a89d8700 1 -- 172.21.15.80:0/1826476559 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4a419c000 con 0x7fb4a4106200 2024-04-24T23:19:32.755 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:31.939+0000 7fb4a89d8700 1 --2- 172.21.15.80:0/1826476559 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb4a4106200 0x7fb4a419bac0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fb49400afd0 tx=0x7fb494006c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:32.755 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:31.939+0000 7fb4a1ffb700 1 -- 172.21.15.80:0/1826476559 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb494025070 con 0x7fb4a4106200 2024-04-24T23:19:32.755 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:31.939+0000 7fb4a1ffb700 1 -- 172.21.15.80:0/1826476559 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb494004340 con 0x7fb4a4106200 2024-04-24T23:19:32.755 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:31.939+0000 7fb4aac3c700 1 -- 172.21.15.80:0/1826476559 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4a419c290 con 0x7fb4a4106200 2024-04-24T23:19:32.755 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:31.939+0000 7fb4a1ffb700 1 -- 172.21.15.80:0/1826476559 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb494010740 con 0x7fb4a4106200 2024-04-24T23:19:32.755 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:31.939+0000 7fb4aac3c700 1 -- 172.21.15.80:0/1826476559 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4a419ef80 con 0x7fb4a4106200 2024-04-24T23:19:32.755 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:31.940+0000 7fb4a1ffb700 1 -- 172.21.15.80:0/1826476559 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7fb494028070 con 0x7fb4a4106200 2024-04-24T23:19:32.755 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:31.940+0000 7fb4a1ffb700 1 --2- 172.21.15.80:0/1826476559 >> [v2:172.21.15.80:6800/1614691410,v1:172.21.15.80:6801/1614691410] conn(0x7fb48c039050 0x7fb48c03b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:32.755 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:31.940+0000 7fb4aac3c700 1 -- 172.21.15.80:0/1826476559 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4a4063890 con 0x7fb4a4106200 2024-04-24T23:19:32.755 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:31.941+0000 7fb4a1ffb700 1 -- 172.21.15.80:0/1826476559 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb49404ccc0 con 0x7fb4a4106200 2024-04-24T23:19:32.756 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:31.941+0000 7fb4a3fff700 1 --2- 172.21.15.80:0/1826476559 >> [v2:172.21.15.80:6800/1614691410,v1:172.21.15.80:6801/1614691410] conn(0x7fb48c039050 0x7fb48c03b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:32.756 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:31.941+0000 7fb4a3fff700 1 --2- 172.21.15.80:0/1826476559 >> [v2:172.21.15.80:6800/1614691410,v1:172.21.15.80:6801/1614691410] conn(0x7fb48c039050 0x7fb48c03b510 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb498006fd0 tx=0x7fb498006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:32.756 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:31.945+0000 7fb4a1ffb700 1 -- 172.21.15.80:0/1826476559 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fb494014070 con 0x7fb4a4106200 2024-04-24T23:19:32.756 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:32.136+0000 7fb4aac3c700 1 -- 172.21.15.80:0/1826476559 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fb4a402de20 con 0x7fb4a4106200 2024-04-24T23:19:32.756 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:32.137+0000 7fb4a1ffb700 1 -- 172.21.15.80:0/1826476559 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+373 (secure 0 0 0) 0x7fb494010e00 con 0x7fb4a4106200 2024-04-24T23:19:32.756 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:32.139+0000 7fb4aac3c700 1 -- 172.21.15.80:0/1826476559 >> [v2:172.21.15.80:6800/1614691410,v1:172.21.15.80:6801/1614691410] conn(0x7fb48c039050 msgr2=0x7fb48c03b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:32.756 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:32.139+0000 7fb4aac3c700 1 --2- 172.21.15.80:0/1826476559 >> [v2:172.21.15.80:6800/1614691410,v1:172.21.15.80:6801/1614691410] conn(0x7fb48c039050 0x7fb48c03b510 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb498006fd0 tx=0x7fb498006e40 comp rx=0 tx=0).stop 2024-04-24T23:19:32.756 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:32.139+0000 7fb4aac3c700 1 -- 172.21.15.80:0/1826476559 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb4a4106200 msgr2=0x7fb4a419bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:32.756 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:32.139+0000 7fb4aac3c700 1 --2- 172.21.15.80:0/1826476559 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb4a4106200 0x7fb4a419bac0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fb49400afd0 tx=0x7fb494006c80 comp rx=0 tx=0).stop 2024-04-24T23:19:32.756 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:32.139+0000 7fb4aac3c700 1 -- 172.21.15.80:0/1826476559 shutdown_connections 2024-04-24T23:19:32.756 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:32.139+0000 7fb4aac3c700 1 --2- 172.21.15.80:0/1826476559 >> [v2:172.21.15.80:6800/1614691410,v1:172.21.15.80:6801/1614691410] conn(0x7fb48c039050 0x7fb48c03b510 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:32.756 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:32.139+0000 7fb4aac3c700 1 --2- 172.21.15.80:0/1826476559 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb4a4106200 0x7fb4a419bac0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:32.756 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:32.139+0000 7fb4aac3c700 1 -- 172.21.15.80:0/1826476559 >> 172.21.15.80:0/1826476559 conn(0x7fb4a4101780 msgr2=0x7fb4a41905b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:32.756 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:32.140+0000 7fb4aac3c700 1 -- 172.21.15.80:0/1826476559 shutdown_connections 2024-04-24T23:19:32.756 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:32.140+0000 7fb4aac3c700 1 -- 172.21.15.80:0/1826476559 wait complete. 2024-04-24T23:19:32.757 INFO:teuthology.orchestra.run.smithi080.stdout:Enabling cephadm module... 2024-04-24T23:19:34.006 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:33 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/1796632525' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-04-24T23:19:34.007 INFO:journalctl@ceph.mgr.x.smithi080.stdout:Apr 24 23:19:33 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-x[29381]: ignoring --setuser ceph since I am not root 2024-04-24T23:19:34.007 INFO:journalctl@ceph.mgr.x.smithi080.stdout:Apr 24 23:19:33 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-x[29381]: ignoring --setgroup ceph since I am not root 2024-04-24T23:19:34.007 INFO:journalctl@ceph.mgr.x.smithi080.stdout:Apr 24 23:19:33 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-x[29381]: 2024-04-24T23:19:33.892+0000 7f75f7728200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-24T23:19:34.282 INFO:journalctl@ceph.mgr.x.smithi080.stdout:Apr 24 23:19:34 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-x[29381]: 2024-04-24T23:19:34.075+0000 7f75f7728200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-24T23:19:34.282 INFO:journalctl@ceph.mgr.x.smithi080.stdout:Apr 24 23:19:34 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-x[29381]: 2024-04-24T23:19:34.175+0000 7f75f7728200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-24T23:19:34.289 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:33.116+0000 7fb186790700 1 Processor -- start 2024-04-24T23:19:34.289 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:33.116+0000 7fb186790700 1 -- start start 2024-04-24T23:19:34.289 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:33.116+0000 7fb186790700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb180107c10 0x7fb180108030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:34.290 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:33.116+0000 7fb186790700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fb180108600 con 0x7fb180107c10 2024-04-24T23:19:34.290 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:33.117+0000 7fb17ffff700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb180107c10 0x7fb180108030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:34.290 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:33.117+0000 7fb17ffff700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb180107c10 0x7fb180108030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:59750/0 (socket says 172.21.15.80:59750) 2024-04-24T23:19:34.290 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:33.117+0000 7fb17ffff700 1 -- 172.21.15.80:0/3901099097 learned_addr learned my addr 172.21.15.80:0/3901099097 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:19:34.290 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:33.117+0000 7fb17ffff700 1 -- 172.21.15.80:0/3901099097 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb180108dc0 con 0x7fb180107c10 2024-04-24T23:19:34.290 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:33.117+0000 7fb17ffff700 1 --2- 172.21.15.80:0/3901099097 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb180107c10 0x7fb180108030 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fb1700098a0 tx=0x7fb170009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efa740365c383b37 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:34.290 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:33.117+0000 7fb17effd700 1 -- 172.21.15.80:0/3901099097 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb170004160 con 0x7fb180107c10 2024-04-24T23:19:34.290 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:33.117+0000 7fb17effd700 1 -- 172.21.15.80:0/3901099097 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb170004300 con 0x7fb180107c10 2024-04-24T23:19:34.290 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:33.117+0000 7fb17effd700 1 -- 172.21.15.80:0/3901099097 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb170011b50 con 0x7fb180107c10 2024-04-24T23:19:34.290 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:33.118+0000 7fb186790700 1 -- 172.21.15.80:0/3901099097 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb180107c10 msgr2=0x7fb180108030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:34.290 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:33.118+0000 7fb186790700 1 --2- 172.21.15.80:0/3901099097 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb180107c10 0x7fb180108030 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fb1700098a0 tx=0x7fb170009bb0 comp rx=0 tx=0).stop 2024-04-24T23:19:34.291 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:33.118+0000 7fb186790700 1 -- 172.21.15.80:0/3901099097 shutdown_connections 2024-04-24T23:19:34.291 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:33.118+0000 7fb186790700 1 --2- 172.21.15.80:0/3901099097 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb180107c10 0x7fb180108030 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:34.291 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:33.118+0000 7fb186790700 1 -- 172.21.15.80:0/3901099097 >> 172.21.15.80:0/3901099097 conn(0x7fb1801031d0 msgr2=0x7fb1801055f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:34.291 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:33.118+0000 7fb186790700 1 -- 172.21.15.80:0/3901099097 shutdown_connections 2024-04-24T23:19:34.291 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:33.118+0000 7fb186790700 1 -- 172.21.15.80:0/3901099097 wait complete. 2024-04-24T23:19:34.291 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:33.118+0000 7fb186790700 1 Processor -- start 2024-04-24T23:19:34.291 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:33.118+0000 7fb186790700 1 -- start start 2024-04-24T23:19:34.291 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:33.119+0000 7fb186790700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb180107c10 0x7fb18007db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:34.291 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:33.119+0000 7fb186790700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fb18010c170 con 0x7fb180107c10 2024-04-24T23:19:34.291 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:33.119+0000 7fb17ffff700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb180107c10 0x7fb18007db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:34.291 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:33.119+0000 7fb17ffff700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb180107c10 0x7fb18007db00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:59752/0 (socket says 172.21.15.80:59752) 2024-04-24T23:19:34.291 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:33.119+0000 7fb17ffff700 1 -- 172.21.15.80:0/1796632525 learned_addr learned my addr 172.21.15.80:0/1796632525 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:19:34.291 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:33.119+0000 7fb17ffff700 1 -- 172.21.15.80:0/1796632525 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb18007e040 con 0x7fb180107c10 2024-04-24T23:19:34.291 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:33.119+0000 7fb17ffff700 1 --2- 172.21.15.80:0/1796632525 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb180107c10 0x7fb18007db00 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb170009fb0 tx=0x7fb170011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:34.292 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:33.119+0000 7fb17d7fa700 1 -- 172.21.15.80:0/1796632525 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb1700038b0 con 0x7fb180107c10 2024-04-24T23:19:34.292 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:33.119+0000 7fb17d7fa700 1 -- 172.21.15.80:0/1796632525 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb170011e50 con 0x7fb180107c10 2024-04-24T23:19:34.292 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:33.119+0000 7fb186790700 1 -- 172.21.15.80:0/1796632525 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb18007c230 con 0x7fb180107c10 2024-04-24T23:19:34.292 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:33.119+0000 7fb17d7fa700 1 -- 172.21.15.80:0/1796632525 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb170019550 con 0x7fb180107c10 2024-04-24T23:19:34.292 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:33.119+0000 7fb186790700 1 -- 172.21.15.80:0/1796632525 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb18007c770 con 0x7fb180107c10 2024-04-24T23:19:34.292 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:33.120+0000 7fb17d7fa700 1 -- 172.21.15.80:0/1796632525 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7fb170019800 con 0x7fb180107c10 2024-04-24T23:19:34.292 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:33.120+0000 7fb186790700 1 -- 172.21.15.80:0/1796632525 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb180063890 con 0x7fb180107c10 2024-04-24T23:19:34.292 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:33.120+0000 7fb17d7fa700 1 --2- 172.21.15.80:0/1796632525 >> [v2:172.21.15.80:6800/1614691410,v1:172.21.15.80:6801/1614691410] conn(0x7fb168039000 0x7fb16803b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:34.292 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:33.120+0000 7fb17d7fa700 1 -- 172.21.15.80:0/1796632525 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb17004d2a0 con 0x7fb180107c10 2024-04-24T23:19:34.292 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:33.121+0000 7fb17f7fe700 1 --2- 172.21.15.80:0/1796632525 >> [v2:172.21.15.80:6800/1614691410,v1:172.21.15.80:6801/1614691410] conn(0x7fb168039000 0x7fb16803b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:34.292 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:33.121+0000 7fb17f7fe700 1 --2- 172.21.15.80:0/1796632525 >> [v2:172.21.15.80:6800/1614691410,v1:172.21.15.80:6801/1614691410] conn(0x7fb168039000 0x7fb16803b4c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb174006fd0 tx=0x7fb174006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:34.292 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:33.126+0000 7fb17d7fa700 1 -- 172.21.15.80:0/1796632525 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fb17002d960 con 0x7fb180107c10 2024-04-24T23:19:34.292 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:33.375+0000 7fb186790700 1 -- 172.21.15.80:0/1796632525 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7fb18007c9f0 con 0x7fb180107c10 2024-04-24T23:19:34.293 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:33.724+0000 7fb17d7fa700 1 -- 172.21.15.80:0/1796632525 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7fb17002d440 con 0x7fb180107c10 2024-04-24T23:19:34.293 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:33.725+0000 7fb17d7fa700 1 -- 172.21.15.80:0/1796632525 <== mon.0 v2:172.21.15.80:3300/0 8 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7fb170018350 con 0x7fb180107c10 2024-04-24T23:19:34.293 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:33.727+0000 7fb186790700 1 -- 172.21.15.80:0/1796632525 >> [v2:172.21.15.80:6800/1614691410,v1:172.21.15.80:6801/1614691410] conn(0x7fb168039000 msgr2=0x7fb16803b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:34.293 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:33.727+0000 7fb186790700 1 --2- 172.21.15.80:0/1796632525 >> [v2:172.21.15.80:6800/1614691410,v1:172.21.15.80:6801/1614691410] conn(0x7fb168039000 0x7fb16803b4c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb174006fd0 tx=0x7fb174006e40 comp rx=0 tx=0).stop 2024-04-24T23:19:34.293 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:33.727+0000 7fb186790700 1 -- 172.21.15.80:0/1796632525 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb180107c10 msgr2=0x7fb18007db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:34.293 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:33.727+0000 7fb186790700 1 --2- 172.21.15.80:0/1796632525 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb180107c10 0x7fb18007db00 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb170009fb0 tx=0x7fb170011bf0 comp rx=0 tx=0).stop 2024-04-24T23:19:34.293 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:33.727+0000 7fb186790700 1 -- 172.21.15.80:0/1796632525 shutdown_connections 2024-04-24T23:19:34.293 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:33.727+0000 7fb186790700 1 --2- 172.21.15.80:0/1796632525 >> [v2:172.21.15.80:6800/1614691410,v1:172.21.15.80:6801/1614691410] conn(0x7fb168039000 0x7fb16803b4c0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:34.293 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:33.727+0000 7fb186790700 1 --2- 172.21.15.80:0/1796632525 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb180107c10 0x7fb18007db00 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:34.293 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:33.727+0000 7fb186790700 1 -- 172.21.15.80:0/1796632525 >> 172.21.15.80:0/1796632525 conn(0x7fb1801031d0 msgr2=0x7fb180103d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:34.293 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:33.727+0000 7fb186790700 1 -- 172.21.15.80:0/1796632525 shutdown_connections 2024-04-24T23:19:34.293 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:33.728+0000 7fb186790700 1 -- 172.21.15.80:0/1796632525 wait complete. 2024-04-24T23:19:34.535 INFO:journalctl@ceph.mgr.x.smithi080.stdout:Apr 24 23:19:34 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-x[29381]: 2024-04-24T23:19:34.491+0000 7f75f7728200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-24T23:19:34.790 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:34 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/1796632525' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-04-24T23:19:34.790 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:34 smithi080 ceph-mon[29152]: mgrmap e5: x(active, since 6s) 2024-04-24T23:19:34.790 INFO:journalctl@ceph.mgr.x.smithi080.stdout:Apr 24 23:19:34 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-x[29381]: 2024-04-24T23:19:34.578+0000 7f75f7728200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-24T23:19:35.547 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout { 2024-04-24T23:19:35.547 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-04-24T23:19:35.547 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-24T23:19:35.547 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-24T23:19:35.547 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-24T23:19:35.547 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout } 2024-04-24T23:19:35.547 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:34.707+0000 7f3f07953700 1 Processor -- start 2024-04-24T23:19:35.547 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:34.707+0000 7f3f07953700 1 -- start start 2024-04-24T23:19:35.547 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:34.707+0000 7f3f07953700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3f00108190 0x7f3f001085b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:35.547 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:34.707+0000 7f3f07953700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f3f001066d0 con 0x7f3f00108190 2024-04-24T23:19:35.547 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:34.708+0000 7f3f056ef700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3f00108190 0x7f3f001085b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:35.548 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:34.708+0000 7f3f056ef700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3f00108190 0x7f3f001085b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:59778/0 (socket says 172.21.15.80:59778) 2024-04-24T23:19:35.548 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:34.708+0000 7f3f056ef700 1 -- 172.21.15.80:0/3301052526 learned_addr learned my addr 172.21.15.80:0/3301052526 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:19:35.548 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:34.708+0000 7f3f056ef700 1 -- 172.21.15.80:0/3301052526 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f00106850 con 0x7f3f00108190 2024-04-24T23:19:35.548 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:34.708+0000 7f3f056ef700 1 --2- 172.21.15.80:0/3301052526 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3f00108190 0x7f3f001085b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f3ef00098a0 tx=0x7f3ef0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a0611dd80e6d50e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:35.548 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:34.708+0000 7f3ef7fff700 1 -- 172.21.15.80:0/3301052526 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3ef0004160 con 0x7f3f00108190 2024-04-24T23:19:35.548 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:34.708+0000 7f3ef7fff700 1 -- 172.21.15.80:0/3301052526 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3ef0004300 con 0x7f3f00108190 2024-04-24T23:19:35.548 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:34.708+0000 7f3ef7fff700 1 -- 172.21.15.80:0/3301052526 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3ef0011b50 con 0x7f3f00108190 2024-04-24T23:19:35.548 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:34.709+0000 7f3f07953700 1 -- 172.21.15.80:0/3301052526 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3f00108190 msgr2=0x7f3f001085b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:35.548 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:34.709+0000 7f3f07953700 1 --2- 172.21.15.80:0/3301052526 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3f00108190 0x7f3f001085b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f3ef00098a0 tx=0x7f3ef0009bb0 comp rx=0 tx=0).stop 2024-04-24T23:19:35.548 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:34.709+0000 7f3f07953700 1 -- 172.21.15.80:0/3301052526 shutdown_connections 2024-04-24T23:19:35.548 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:34.709+0000 7f3f07953700 1 --2- 172.21.15.80:0/3301052526 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3f00108190 0x7f3f001085b0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:35.548 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:34.709+0000 7f3f07953700 1 -- 172.21.15.80:0/3301052526 >> 172.21.15.80:0/3301052526 conn(0x7f3f00102030 msgr2=0x7f3f00104490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:35.548 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:34.709+0000 7f3f07953700 1 -- 172.21.15.80:0/3301052526 shutdown_connections 2024-04-24T23:19:35.549 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:34.709+0000 7f3f07953700 1 -- 172.21.15.80:0/3301052526 wait complete. 2024-04-24T23:19:35.549 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:34.710+0000 7f3f07953700 1 Processor -- start 2024-04-24T23:19:35.549 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:34.710+0000 7f3f07953700 1 -- start start 2024-04-24T23:19:35.549 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:34.710+0000 7f3f07953700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3f00108190 0x7f3f0007b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:35.549 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:34.710+0000 7f3f07953700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f3f0010b060 con 0x7f3f00108190 2024-04-24T23:19:35.549 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:34.710+0000 7f3f056ef700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3f00108190 0x7f3f0007b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:35.549 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:34.710+0000 7f3f056ef700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3f00108190 0x7f3f0007b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:59786/0 (socket says 172.21.15.80:59786) 2024-04-24T23:19:35.549 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:34.710+0000 7f3f056ef700 1 -- 172.21.15.80:0/3105509875 learned_addr learned my addr 172.21.15.80:0/3105509875 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:19:35.549 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:34.710+0000 7f3f056ef700 1 -- 172.21.15.80:0/3105509875 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f0019ea30 con 0x7f3f00108190 2024-04-24T23:19:35.549 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:34.710+0000 7f3f056ef700 1 --2- 172.21.15.80:0/3105509875 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3f00108190 0x7f3f0007b9e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3ef0011670 tx=0x7f3ef0011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:35.549 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:34.710+0000 7f3ef67fc700 1 -- 172.21.15.80:0/3105509875 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3ef00039c0 con 0x7f3f00108190 2024-04-24T23:19:35.549 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:34.710+0000 7f3ef67fc700 1 -- 172.21.15.80:0/3105509875 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3ef0011e50 con 0x7f3f00108190 2024-04-24T23:19:35.549 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:34.710+0000 7f3f07953700 1 -- 172.21.15.80:0/3105509875 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f0019ec00 con 0x7f3f00108190 2024-04-24T23:19:35.549 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:34.711+0000 7f3f07953700 1 -- 172.21.15.80:0/3105509875 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f0019efa0 con 0x7f3f00108190 2024-04-24T23:19:35.550 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:34.711+0000 7f3ef67fc700 1 -- 172.21.15.80:0/3105509875 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3ef001a5c0 con 0x7f3f00108190 2024-04-24T23:19:35.550 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:34.712+0000 7f3ef67fc700 1 -- 172.21.15.80:0/3105509875 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f3ef001a900 con 0x7f3f00108190 2024-04-24T23:19:35.550 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:34.712+0000 7f3f07953700 1 -- 172.21.15.80:0/3105509875 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ee4005360 con 0x7f3f00108190 2024-04-24T23:19:35.550 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:34.712+0000 7f3ef67fc700 1 --2- 172.21.15.80:0/3105509875 >> [v2:172.21.15.80:6800/1614691410,v1:172.21.15.80:6801/1614691410] conn(0x7f3eec039000 0x7f3eec03b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:35.550 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:34.712+0000 7f3f04eee700 1 -- 172.21.15.80:0/3105509875 >> [v2:172.21.15.80:6800/1614691410,v1:172.21.15.80:6801/1614691410] conn(0x7f3eec039000 msgr2=0x7f3eec03b4c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.80:6800/1614691410 2024-04-24T23:19:35.550 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:34.712+0000 7f3ef67fc700 1 -- 172.21.15.80:0/3105509875 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f3ef004d3f0 con 0x7f3f00108190 2024-04-24T23:19:35.550 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:34.712+0000 7f3f04eee700 1 --2- 172.21.15.80:0/3105509875 >> [v2:172.21.15.80:6800/1614691410,v1:172.21.15.80:6801/1614691410] conn(0x7f3eec039000 0x7f3eec03b4c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T23:19:35.550 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:34.717+0000 7f3ef67fc700 1 -- 172.21.15.80:0/3105509875 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f3ef002cc00 con 0x7f3f00108190 2024-04-24T23:19:35.550 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:34.913+0000 7f3f04eee700 1 -- 172.21.15.80:0/3105509875 >> [v2:172.21.15.80:6800/1614691410,v1:172.21.15.80:6801/1614691410] conn(0x7f3eec039000 msgr2=0x7f3eec03b4c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.80:6800/1614691410 2024-04-24T23:19:35.550 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:34.913+0000 7f3f04eee700 1 --2- 172.21.15.80:0/3105509875 >> [v2:172.21.15.80:6800/1614691410,v1:172.21.15.80:6801/1614691410] conn(0x7f3eec039000 0x7f3eec03b4c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-24T23:19:35.550 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:34.993+0000 7f3f07953700 1 -- 172.21.15.80:0/3105509875 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f3ee4005e60 con 0x7f3f00108190 2024-04-24T23:19:35.550 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:34.994+0000 7f3ef67fc700 1 -- 172.21.15.80:0/3105509875 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f3ef004ce10 con 0x7f3f00108190 2024-04-24T23:19:35.550 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:34.997+0000 7f3f07953700 1 -- 172.21.15.80:0/3105509875 >> [v2:172.21.15.80:6800/1614691410,v1:172.21.15.80:6801/1614691410] conn(0x7f3eec039000 msgr2=0x7f3eec03b4c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T23:19:35.550 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:34.997+0000 7f3f07953700 1 --2- 172.21.15.80:0/3105509875 >> [v2:172.21.15.80:6800/1614691410,v1:172.21.15.80:6801/1614691410] conn(0x7f3eec039000 0x7f3eec03b4c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:35.551 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:34.997+0000 7f3f07953700 1 -- 172.21.15.80:0/3105509875 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3f00108190 msgr2=0x7f3f0007b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:35.551 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:34.997+0000 7f3f07953700 1 --2- 172.21.15.80:0/3105509875 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3f00108190 0x7f3f0007b9e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3ef0011670 tx=0x7f3ef0011b60 comp rx=0 tx=0).stop 2024-04-24T23:19:35.551 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:34.997+0000 7f3f07953700 1 -- 172.21.15.80:0/3105509875 shutdown_connections 2024-04-24T23:19:35.551 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:34.997+0000 7f3f07953700 1 --2- 172.21.15.80:0/3105509875 >> [v2:172.21.15.80:6800/1614691410,v1:172.21.15.80:6801/1614691410] conn(0x7f3eec039000 0x7f3eec03b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:35.551 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:34.997+0000 7f3f07953700 1 --2- 172.21.15.80:0/3105509875 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3f00108190 0x7f3f0007b9e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:35.551 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:34.997+0000 7f3f07953700 1 -- 172.21.15.80:0/3105509875 >> 172.21.15.80:0/3105509875 conn(0x7f3f00102030 msgr2=0x7f3f00102c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:35.551 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:34.999+0000 7f3f07953700 1 -- 172.21.15.80:0/3105509875 shutdown_connections 2024-04-24T23:19:35.551 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:34.999+0000 7f3f07953700 1 -- 172.21.15.80:0/3105509875 wait complete. 2024-04-24T23:19:35.551 INFO:teuthology.orchestra.run.smithi080.stdout:Waiting for the mgr to restart... 2024-04-24T23:19:35.551 INFO:teuthology.orchestra.run.smithi080.stdout:Waiting for mgr epoch 5... 2024-04-24T23:19:35.982 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:35 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/3105509875' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-24T23:19:35.983 INFO:journalctl@ceph.mgr.x.smithi080.stdout:Apr 24 23:19:35 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-x[29381]: 2024-04-24T23:19:35.782+0000 7f75f7728200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-24T23:19:35.983 INFO:journalctl@ceph.mgr.x.smithi080.stdout:Apr 24 23:19:35 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-x[29381]: 2024-04-24T23:19:35.954+0000 7f75f7728200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-24T23:19:36.289 INFO:journalctl@ceph.mgr.x.smithi080.stdout:Apr 24 23:19:36 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-x[29381]: 2024-04-24T23:19:36.123+0000 7f75f7728200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-24T23:19:36.694 INFO:journalctl@ceph.mgr.x.smithi080.stdout:Apr 24 23:19:36 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-x[29381]: 2024-04-24T23:19:36.405+0000 7f75f7728200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-24T23:19:36.694 INFO:journalctl@ceph.mgr.x.smithi080.stdout:Apr 24 23:19:36 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-x[29381]: 2024-04-24T23:19:36.495+0000 7f75f7728200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-24T23:19:36.694 INFO:journalctl@ceph.mgr.x.smithi080.stdout:Apr 24 23:19:36 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-x[29381]: 2024-04-24T23:19:36.693+0000 7f75f7728200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-24T23:19:37.039 INFO:journalctl@ceph.mgr.x.smithi080.stdout:Apr 24 23:19:36 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-x[29381]: 2024-04-24T23:19:36.883+0000 7f75f7728200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-24T23:19:37.789 INFO:journalctl@ceph.mgr.x.smithi080.stdout:Apr 24 23:19:37 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-x[29381]: 2024-04-24T23:19:37.473+0000 7f75f7728200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-24T23:19:38.180 INFO:journalctl@ceph.mgr.x.smithi080.stdout:Apr 24 23:19:37 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-x[29381]: 2024-04-24T23:19:37.888+0000 7f75f7728200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-24T23:19:38.180 INFO:journalctl@ceph.mgr.x.smithi080.stdout:Apr 24 23:19:37 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-x[29381]: 2024-04-24T23:19:37.981+0000 7f75f7728200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-24T23:19:38.180 INFO:journalctl@ceph.mgr.x.smithi080.stdout:Apr 24 23:19:38 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-x[29381]: 2024-04-24T23:19:38.177+0000 7f75f7728200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-24T23:19:38.992 INFO:journalctl@ceph.mgr.x.smithi080.stdout:Apr 24 23:19:38 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-x[29381]: 2024-04-24T23:19:38.718+0000 7f75f7728200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-24T23:19:38.992 INFO:journalctl@ceph.mgr.x.smithi080.stdout:Apr 24 23:19:38 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-x[29381]: 2024-04-24T23:19:38.809+0000 7f75f7728200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-24T23:19:39.279 INFO:journalctl@ceph.mgr.x.smithi080.stdout:Apr 24 23:19:39 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-x[29381]: 2024-04-24T23:19:39.052+0000 7f75f7728200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-24T23:19:39.279 INFO:journalctl@ceph.mgr.x.smithi080.stdout:Apr 24 23:19:39 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-x[29381]: 2024-04-24T23:19:39.183+0000 7f75f7728200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-24T23:19:39.279 INFO:journalctl@ceph.mgr.x.smithi080.stdout:Apr 24 23:19:39 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-x[29381]: 2024-04-24T23:19:39.276+0000 7f75f7728200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-24T23:19:39.539 INFO:journalctl@ceph.mgr.x.smithi080.stdout:Apr 24 23:19:39 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-x[29381]: 2024-04-24T23:19:39.390+0000 7f75f7728200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-24T23:19:39.539 INFO:journalctl@ceph.mgr.x.smithi080.stdout:Apr 24 23:19:39 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-x[29381]: 2024-04-24T23:19:39.480+0000 7f75f7728200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-24T23:19:40.539 INFO:journalctl@ceph.mgr.x.smithi080.stdout:Apr 24 23:19:40 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-x[29381]: 2024-04-24T23:19:40.278+0000 7f75f7728200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-24T23:19:40.539 INFO:journalctl@ceph.mgr.x.smithi080.stdout:Apr 24 23:19:40 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-x[29381]: 2024-04-24T23:19:40.488+0000 7f75f7728200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-24T23:19:41.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:40 smithi080 ceph-mon[29152]: Active manager daemon x restarted 2024-04-24T23:19:41.040 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:40 smithi080 ceph-mon[29152]: Activating manager daemon x 2024-04-24T23:19:41.040 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:40 smithi080 ceph-mon[29152]: osdmap e2: 0 total, 0 up, 0 in 2024-04-24T23:19:41.040 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:40 smithi080 ceph-mon[29152]: mgrmap e6: x(active, starting, since 0.00333909s) 2024-04-24T23:19:41.040 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:40 smithi080 ceph-mon[29152]: from='mgr.14120 172.21.15.80:0/1767820559' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T23:19:41.040 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:40 smithi080 ceph-mon[29152]: from='mgr.14120 172.21.15.80:0/1767820559' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-24T23:19:41.040 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:40 smithi080 ceph-mon[29152]: from='mgr.14120 172.21.15.80:0/1767820559' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-24T23:19:41.040 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:40 smithi080 ceph-mon[29152]: from='mgr.14120 172.21.15.80:0/1767820559' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-24T23:19:41.040 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:40 smithi080 ceph-mon[29152]: from='mgr.14120 172.21.15.80:0/1767820559' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-24T23:19:41.040 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:40 smithi080 ceph-mon[29152]: Manager daemon x is now available 2024-04-24T23:19:41.040 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:40 smithi080 ceph-mon[29152]: from='mgr.14120 172.21.15.80:0/1767820559' entity='mgr.x' 2024-04-24T23:19:41.040 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:40 smithi080 ceph-mon[29152]: from='mgr.14120 172.21.15.80:0/1767820559' entity='mgr.x' 2024-04-24T23:19:41.040 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:40 smithi080 ceph-mon[29152]: from='mgr.14120 172.21.15.80:0/1767820559' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:19:41.041 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:40 smithi080 ceph-mon[29152]: from='mgr.14120 172.21.15.80:0/1767820559' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:19:41.041 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:40 smithi080 ceph-mon[29152]: from='mgr.14120 172.21.15.80:0/1767820559' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-24T23:19:41.041 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:40 smithi080 ceph-mon[29152]: from='mgr.14120 172.21.15.80:0/1767820559' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-24T23:19:41.789 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:41 smithi080 ceph-mon[29152]: Found migration_current of "None". Setting to last migration. 2024-04-24T23:19:41.789 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:41 smithi080 ceph-mon[29152]: mgrmap e7: x(active, since 1.00531s) 2024-04-24T23:19:41.789 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:41 smithi080 ceph-mon[29152]: from='mgr.14120 172.21.15.80:0/1767820559' entity='mgr.x' 2024-04-24T23:19:41.789 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:41 smithi080 ceph-mon[29152]: from='mgr.14120 172.21.15.80:0/1767820559' entity='mgr.x' 2024-04-24T23:19:42.055 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout { 2024-04-24T23:19:42.055 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-04-24T23:19:42.055 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-24T23:19:42.055 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout } 2024-04-24T23:19:42.055 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:35.981+0000 7f6489998700 1 Processor -- start 2024-04-24T23:19:42.055 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:35.981+0000 7f6489998700 1 -- start start 2024-04-24T23:19:42.055 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:35.981+0000 7f6489998700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f64841078e0 0x7f6484107d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:42.055 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:35.981+0000 7f6489998700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f6484105e20 con 0x7f64841078e0 2024-04-24T23:19:42.055 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:35.982+0000 7f6482ffd700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f64841078e0 0x7f6484107d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:42.055 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:35.982+0000 7f6482ffd700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f64841078e0 0x7f6484107d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:59800/0 (socket says 172.21.15.80:59800) 2024-04-24T23:19:42.055 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:35.982+0000 7f6482ffd700 1 -- 172.21.15.80:0/576013720 learned_addr learned my addr 172.21.15.80:0/576013720 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:19:42.056 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:35.982+0000 7f6482ffd700 1 -- 172.21.15.80:0/576013720 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6484105fa0 con 0x7f64841078e0 2024-04-24T23:19:42.056 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:35.982+0000 7f6482ffd700 1 --2- 172.21.15.80:0/576013720 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f64841078e0 0x7f6484107d00 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f647800bb40 tx=0x7f647800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ed188ad2184b91b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:42.056 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:35.982+0000 7f6481ffb700 1 -- 172.21.15.80:0/576013720 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f647800dad0 con 0x7f64841078e0 2024-04-24T23:19:42.056 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:35.982+0000 7f6481ffb700 1 -- 172.21.15.80:0/576013720 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f64780044f0 con 0x7f64841078e0 2024-04-24T23:19:42.056 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:35.982+0000 7f6481ffb700 1 -- 172.21.15.80:0/576013720 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6478017410 con 0x7f64841078e0 2024-04-24T23:19:42.056 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:35.983+0000 7f6489998700 1 -- 172.21.15.80:0/576013720 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f64841078e0 msgr2=0x7f6484107d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:42.056 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:35.983+0000 7f6489998700 1 --2- 172.21.15.80:0/576013720 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f64841078e0 0x7f6484107d00 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f647800bb40 tx=0x7f647800be50 comp rx=0 tx=0).stop 2024-04-24T23:19:42.056 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:35.983+0000 7f6489998700 1 -- 172.21.15.80:0/576013720 shutdown_connections 2024-04-24T23:19:42.056 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:35.983+0000 7f6489998700 1 --2- 172.21.15.80:0/576013720 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f64841078e0 0x7f6484107d00 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.056 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:35.983+0000 7f6489998700 1 -- 172.21.15.80:0/576013720 >> 172.21.15.80:0/576013720 conn(0x7f64841017a0 msgr2=0x7f6484103be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:42.056 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:35.983+0000 7f6489998700 1 -- 172.21.15.80:0/576013720 shutdown_connections 2024-04-24T23:19:42.056 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:35.983+0000 7f6489998700 1 -- 172.21.15.80:0/576013720 wait complete. 2024-04-24T23:19:42.056 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:35.984+0000 7f6489998700 1 Processor -- start 2024-04-24T23:19:42.057 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:35.984+0000 7f6489998700 1 -- start start 2024-04-24T23:19:42.057 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:35.984+0000 7f6489998700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f64841078e0 0x7f6484199a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:42.057 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:35.984+0000 7f6489998700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f64841066c0 con 0x7f64841078e0 2024-04-24T23:19:42.057 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:35.984+0000 7f6482ffd700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f64841078e0 0x7f6484199a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:42.057 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:35.984+0000 7f6482ffd700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f64841078e0 0x7f6484199a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:59816/0 (socket says 172.21.15.80:59816) 2024-04-24T23:19:42.057 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:35.984+0000 7f6482ffd700 1 -- 172.21.15.80:0/2052409655 learned_addr learned my addr 172.21.15.80:0/2052409655 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:19:42.058 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:35.984+0000 7f6482ffd700 1 -- 172.21.15.80:0/2052409655 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6484199fc0 con 0x7f64841078e0 2024-04-24T23:19:42.058 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:35.985+0000 7f6482ffd700 1 --2- 172.21.15.80:0/2052409655 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f64841078e0 0x7f6484199a80 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f647800bb10 tx=0x7f64780047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:42.058 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:35.985+0000 7f6488996700 1 -- 172.21.15.80:0/2052409655 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6478004090 con 0x7f64841078e0 2024-04-24T23:19:42.058 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:35.985+0000 7f6488996700 1 -- 172.21.15.80:0/2052409655 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6478004230 con 0x7f64841078e0 2024-04-24T23:19:42.059 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:35.985+0000 7f6489998700 1 -- 172.21.15.80:0/2052409655 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f648419a250 con 0x7f64841078e0 2024-04-24T23:19:42.059 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:35.985+0000 7f6489998700 1 -- 172.21.15.80:0/2052409655 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f648419cf40 con 0x7f64841078e0 2024-04-24T23:19:42.059 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:35.985+0000 7f6488996700 1 -- 172.21.15.80:0/2052409655 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6478021c00 con 0x7f64841078e0 2024-04-24T23:19:42.059 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:35.986+0000 7f6488996700 1 -- 172.21.15.80:0/2052409655 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f647802a400 con 0x7f64841078e0 2024-04-24T23:19:42.059 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:35.986+0000 7f6488996700 1 --2- 172.21.15.80:0/2052409655 >> [v2:172.21.15.80:6800/1614691410,v1:172.21.15.80:6801/1614691410] conn(0x7f646c039000 0x7f646c03b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:42.059 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:35.986+0000 7f6488996700 1 -- 172.21.15.80:0/2052409655 --> [v2:172.21.15.80:6800/1614691410,v1:172.21.15.80:6801/1614691410] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f646c03bbd0 con 0x7f646c039000 2024-04-24T23:19:42.059 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:35.986+0000 7f6488996700 1 -- 172.21.15.80:0/2052409655 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f64780509b0 con 0x7f64841078e0 2024-04-24T23:19:42.059 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:35.986+0000 7f64827fc700 1 -- 172.21.15.80:0/2052409655 >> [v2:172.21.15.80:6800/1614691410,v1:172.21.15.80:6801/1614691410] conn(0x7f646c039000 msgr2=0x7f646c03b4c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.80:6800/1614691410 2024-04-24T23:19:42.059 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:35.986+0000 7f64827fc700 1 --2- 172.21.15.80:0/2052409655 >> [v2:172.21.15.80:6800/1614691410,v1:172.21.15.80:6801/1614691410] conn(0x7f646c039000 0x7f646c03b4c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T23:19:42.059 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:36.187+0000 7f64827fc700 1 -- 172.21.15.80:0/2052409655 >> [v2:172.21.15.80:6800/1614691410,v1:172.21.15.80:6801/1614691410] conn(0x7f646c039000 msgr2=0x7f646c03b4c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.80:6800/1614691410 2024-04-24T23:19:42.059 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:36.187+0000 7f64827fc700 1 --2- 172.21.15.80:0/2052409655 >> [v2:172.21.15.80:6800/1614691410,v1:172.21.15.80:6801/1614691410] conn(0x7f646c039000 0x7f646c03b4c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-24T23:19:42.059 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:36.588+0000 7f64827fc700 1 -- 172.21.15.80:0/2052409655 >> [v2:172.21.15.80:6800/1614691410,v1:172.21.15.80:6801/1614691410] conn(0x7f646c039000 msgr2=0x7f646c03b4c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.80:6800/1614691410 2024-04-24T23:19:42.059 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:36.588+0000 7f64827fc700 1 --2- 172.21.15.80:0/2052409655 >> [v2:172.21.15.80:6800/1614691410,v1:172.21.15.80:6801/1614691410] conn(0x7f646c039000 0x7f646c03b4c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-24T23:19:42.060 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:37.388+0000 7f64827fc700 1 -- 172.21.15.80:0/2052409655 >> [v2:172.21.15.80:6800/1614691410,v1:172.21.15.80:6801/1614691410] conn(0x7f646c039000 msgr2=0x7f646c03b4c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.80:6800/1614691410 2024-04-24T23:19:42.060 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:37.388+0000 7f64827fc700 1 --2- 172.21.15.80:0/2052409655 >> [v2:172.21.15.80:6800/1614691410,v1:172.21.15.80:6801/1614691410] conn(0x7f646c039000 0x7f646c03b4c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-24T23:19:42.060 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:38.989+0000 7f64827fc700 1 -- 172.21.15.80:0/2052409655 >> [v2:172.21.15.80:6800/1614691410,v1:172.21.15.80:6801/1614691410] conn(0x7f646c039000 msgr2=0x7f646c03b4c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.80:6800/1614691410 2024-04-24T23:19:42.060 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:38.989+0000 7f64827fc700 1 --2- 172.21.15.80:0/2052409655 >> [v2:172.21.15.80:6800/1614691410,v1:172.21.15.80:6801/1614691410] conn(0x7f646c039000 0x7f646c03b4c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-24T23:19:42.060 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:40.491+0000 7f6488996700 1 -- 172.21.15.80:0/2052409655 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mgrmap(e 6) ==== 46054+0+0 (secure 0 0 0) 0x7f647802ad00 con 0x7f64841078e0 2024-04-24T23:19:42.060 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:40.491+0000 7f6488996700 1 -- 172.21.15.80:0/2052409655 >> [v2:172.21.15.80:6800/1614691410,v1:172.21.15.80:6801/1614691410] conn(0x7f646c039000 msgr2=0x7f646c03b4c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T23:19:42.060 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:40.491+0000 7f6488996700 1 --2- 172.21.15.80:0/2052409655 >> [v2:172.21.15.80:6800/1614691410,v1:172.21.15.80:6801/1614691410] conn(0x7f646c039000 0x7f646c03b4c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.060 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:41.492+0000 7f6488996700 1 -- 172.21.15.80:0/2052409655 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7f6478028350 con 0x7f64841078e0 2024-04-24T23:19:42.060 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:41.493+0000 7f6488996700 1 --2- 172.21.15.80:0/2052409655 >> [v2:172.21.15.80:6800/233364790,v1:172.21.15.80:6801/233364790] conn(0x7f646c03c930 0x7f646c03ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:42.060 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:41.493+0000 7f6488996700 1 -- 172.21.15.80:0/2052409655 --> [v2:172.21.15.80:6800/233364790,v1:172.21.15.80:6801/233364790] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f647801de50 con 0x7f646c03c930 2024-04-24T23:19:42.060 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:41.493+0000 7f64827fc700 1 --2- 172.21.15.80:0/2052409655 >> [v2:172.21.15.80:6800/233364790,v1:172.21.15.80:6801/233364790] conn(0x7f646c03c930 0x7f646c03ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:42.060 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:41.494+0000 7f64827fc700 1 --2- 172.21.15.80:0/2052409655 >> [v2:172.21.15.80:6800/233364790,v1:172.21.15.80:6801/233364790] conn(0x7f646c03c930 0x7f646c03ed20 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f6474003a10 tx=0x7f6474009290 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:42.060 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:41.494+0000 7f6488996700 1 -- 172.21.15.80:0/2052409655 <== mgr.14120 v2:172.21.15.80:6800/233364790 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f647801de50 con 0x7f646c03c930 2024-04-24T23:19:42.060 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:41.505+0000 7f6489998700 1 -- 172.21.15.80:0/2052409655 --> [v2:172.21.15.80:6800/233364790,v1:172.21.15.80:6801/233364790] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f6470002630 con 0x7f646c03c930 2024-04-24T23:19:42.061 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:41.506+0000 7f6488996700 1 -- 172.21.15.80:0/2052409655 <== mgr.14120 v2:172.21.15.80:6800/233364790 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f6470002630 con 0x7f646c03c930 2024-04-24T23:19:42.061 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:41.506+0000 7f6489998700 1 -- 172.21.15.80:0/2052409655 >> [v2:172.21.15.80:6800/233364790,v1:172.21.15.80:6801/233364790] conn(0x7f646c03c930 msgr2=0x7f646c03ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:42.061 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:41.506+0000 7f6489998700 1 --2- 172.21.15.80:0/2052409655 >> [v2:172.21.15.80:6800/233364790,v1:172.21.15.80:6801/233364790] conn(0x7f646c03c930 0x7f646c03ed20 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f6474003a10 tx=0x7f6474009290 comp rx=0 tx=0).stop 2024-04-24T23:19:42.061 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:41.506+0000 7f6489998700 1 -- 172.21.15.80:0/2052409655 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f64841078e0 msgr2=0x7f6484199a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:42.061 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:41.506+0000 7f6489998700 1 --2- 172.21.15.80:0/2052409655 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f64841078e0 0x7f6484199a80 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f647800bb10 tx=0x7f64780047c0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.061 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:41.506+0000 7f6489998700 1 -- 172.21.15.80:0/2052409655 shutdown_connections 2024-04-24T23:19:42.061 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:41.506+0000 7f6489998700 1 --2- 172.21.15.80:0/2052409655 >> [v2:172.21.15.80:6800/233364790,v1:172.21.15.80:6801/233364790] conn(0x7f646c03c930 0x7f646c03ed20 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.061 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:41.506+0000 7f6489998700 1 --2- 172.21.15.80:0/2052409655 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f64841078e0 0x7f6484199a80 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.061 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:41.506+0000 7f6489998700 1 --2- 172.21.15.80:0/2052409655 >> [v2:172.21.15.80:6800/1614691410,v1:172.21.15.80:6801/1614691410] conn(0x7f646c039000 0x7f646c03b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:42.061 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:41.506+0000 7f6489998700 1 -- 172.21.15.80:0/2052409655 >> 172.21.15.80:0/2052409655 conn(0x7f64841017a0 msgr2=0x7f64841906f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:42.061 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:41.506+0000 7f6489998700 1 -- 172.21.15.80:0/2052409655 shutdown_connections 2024-04-24T23:19:42.061 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:41.506+0000 7f6489998700 1 -- 172.21.15.80:0/2052409655 wait complete. 2024-04-24T23:19:42.061 INFO:teuthology.orchestra.run.smithi080.stdout:mgr epoch 5 is available 2024-04-24T23:19:42.061 INFO:teuthology.orchestra.run.smithi080.stdout:Setting orchestrator backend to cephadm... 2024-04-24T23:19:42.789 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:42 smithi080 ceph-mon[29152]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-24T23:19:42.789 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:42 smithi080 ceph-mon[29152]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-24T23:19:42.789 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:42 smithi080 ceph-mon[29152]: from='mgr.14120 172.21.15.80:0/1767820559' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:19:43.217 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:42.468+0000 7f1d0be27700 1 Processor -- start 2024-04-24T23:19:43.217 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:42.468+0000 7f1d0be27700 1 -- start start 2024-04-24T23:19:43.217 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:42.469+0000 7f1d0be27700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1d04108120 0x7f1d04108540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:43.218 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:42.469+0000 7f1d0be27700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f1d04108b10 con 0x7f1d04108120 2024-04-24T23:19:43.218 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:42.469+0000 7f1d09bc3700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1d04108120 0x7f1d04108540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:43.218 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:42.469+0000 7f1d09bc3700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1d04108120 0x7f1d04108540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:43276/0 (socket says 172.21.15.80:43276) 2024-04-24T23:19:43.218 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:42.469+0000 7f1d09bc3700 1 -- 172.21.15.80:0/917345868 learned_addr learned my addr 172.21.15.80:0/917345868 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:19:43.218 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:42.469+0000 7f1d09bc3700 1 -- 172.21.15.80:0/917345868 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d04105c20 con 0x7f1d04108120 2024-04-24T23:19:43.218 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:42.469+0000 7f1d09bc3700 1 --2- 172.21.15.80:0/917345868 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1d04108120 0x7f1d04108540 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1d000098a0 tx=0x7f1d00009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6eb43b69ba81e2cb server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:43.218 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:42.469+0000 7f1d08bc1700 1 -- 172.21.15.80:0/917345868 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1d00004160 con 0x7f1d04108120 2024-04-24T23:19:43.218 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:42.469+0000 7f1d08bc1700 1 -- 172.21.15.80:0/917345868 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1d00004300 con 0x7f1d04108120 2024-04-24T23:19:43.218 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:42.469+0000 7f1d08bc1700 1 -- 172.21.15.80:0/917345868 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1d00011b50 con 0x7f1d04108120 2024-04-24T23:19:43.218 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:42.470+0000 7f1d0be27700 1 -- 172.21.15.80:0/917345868 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1d04108120 msgr2=0x7f1d04108540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:43.218 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:42.470+0000 7f1d0be27700 1 --2- 172.21.15.80:0/917345868 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1d04108120 0x7f1d04108540 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1d000098a0 tx=0x7f1d00009bb0 comp rx=0 tx=0).stop 2024-04-24T23:19:43.218 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:42.470+0000 7f1d0be27700 1 -- 172.21.15.80:0/917345868 shutdown_connections 2024-04-24T23:19:43.219 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:42.470+0000 7f1d0be27700 1 --2- 172.21.15.80:0/917345868 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1d04108120 0x7f1d04108540 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:43.219 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:42.470+0000 7f1d0be27700 1 -- 172.21.15.80:0/917345868 >> 172.21.15.80:0/917345868 conn(0x7f1d041017e0 msgr2=0x7f1d04103c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:43.219 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:42.470+0000 7f1d0be27700 1 -- 172.21.15.80:0/917345868 shutdown_connections 2024-04-24T23:19:43.219 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:42.470+0000 7f1d0be27700 1 -- 172.21.15.80:0/917345868 wait complete. 2024-04-24T23:19:43.219 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:42.470+0000 7f1d0be27700 1 Processor -- start 2024-04-24T23:19:43.219 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:42.471+0000 7f1d0be27700 1 -- start start 2024-04-24T23:19:43.219 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:42.471+0000 7f1d0be27700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1d04108120 0x7f1d0419dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:43.219 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:42.471+0000 7f1d0be27700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f1d04106ca0 con 0x7f1d04108120 2024-04-24T23:19:43.219 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:42.471+0000 7f1d09bc3700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1d04108120 0x7f1d0419dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:43.219 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:42.471+0000 7f1d09bc3700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1d04108120 0x7f1d0419dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:43286/0 (socket says 172.21.15.80:43286) 2024-04-24T23:19:43.219 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:42.471+0000 7f1d09bc3700 1 -- 172.21.15.80:0/712005158 learned_addr learned my addr 172.21.15.80:0/712005158 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:19:43.219 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:42.471+0000 7f1d09bc3700 1 -- 172.21.15.80:0/712005158 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d0419e250 con 0x7f1d04108120 2024-04-24T23:19:43.219 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:42.471+0000 7f1d09bc3700 1 --2- 172.21.15.80:0/712005158 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1d04108120 0x7f1d0419dd10 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1d00011670 tx=0x7f1d00011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:43.220 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:42.471+0000 7f1cfaffd700 1 -- 172.21.15.80:0/712005158 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1d000039c0 con 0x7f1d04108120 2024-04-24T23:19:43.220 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:42.471+0000 7f1cfaffd700 1 -- 172.21.15.80:0/712005158 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1d00011e50 con 0x7f1d04108120 2024-04-24T23:19:43.220 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:42.471+0000 7f1d0be27700 1 -- 172.21.15.80:0/712005158 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d0419e4e0 con 0x7f1d04108120 2024-04-24T23:19:43.220 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:42.471+0000 7f1cfaffd700 1 -- 172.21.15.80:0/712005158 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1d0001a5c0 con 0x7f1d04108120 2024-04-24T23:19:43.220 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:42.471+0000 7f1d0be27700 1 -- 172.21.15.80:0/712005158 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d041a11c0 con 0x7f1d04108120 2024-04-24T23:19:43.220 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:42.472+0000 7f1cfaffd700 1 -- 172.21.15.80:0/712005158 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7f1d0001a870 con 0x7f1d04108120 2024-04-24T23:19:43.220 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:42.472+0000 7f1d0be27700 1 -- 172.21.15.80:0/712005158 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d04063890 con 0x7f1d04108120 2024-04-24T23:19:43.220 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:42.472+0000 7f1cfaffd700 1 --2- 172.21.15.80:0/712005158 >> [v2:172.21.15.80:6800/233364790,v1:172.21.15.80:6801/233364790] conn(0x7f1cf0038f40 0x7f1cf003b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:43.220 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:42.472+0000 7f1cfaffd700 1 -- 172.21.15.80:0/712005158 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f1d0004d1c0 con 0x7f1d04108120 2024-04-24T23:19:43.220 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:42.472+0000 7f1d093c2700 1 --2- 172.21.15.80:0/712005158 >> [v2:172.21.15.80:6800/233364790,v1:172.21.15.80:6801/233364790] conn(0x7f1cf0038f40 0x7f1cf003b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:43.220 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:42.473+0000 7f1d093c2700 1 --2- 172.21.15.80:0/712005158 >> [v2:172.21.15.80:6800/233364790,v1:172.21.15.80:6801/233364790] conn(0x7f1cf0038f40 0x7f1cf003b400 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1cf4006fd0 tx=0x7f1cf4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:43.220 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:42.477+0000 7f1cfaffd700 1 -- 172.21.15.80:0/712005158 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f1d0002cc00 con 0x7f1d04108120 2024-04-24T23:19:43.220 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:42.549+0000 7f1cfaffd700 1 -- 172.21.15.80:0/712005158 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f1d0002d910 con 0x7f1d04108120 2024-04-24T23:19:43.220 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:42.694+0000 7f1d0be27700 1 -- 172.21.15.80:0/712005158 --> [v2:172.21.15.80:6800/233364790,v1:172.21.15.80:6801/233364790] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f1d04062670 con 0x7f1cf0038f40 2024-04-24T23:19:43.221 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:42.702+0000 7f1cfaffd700 1 -- 172.21.15.80:0/712005158 <== mgr.14120 v2:172.21.15.80:6800/233364790 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f1d04062670 con 0x7f1cf0038f40 2024-04-24T23:19:43.221 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:42.704+0000 7f1d0be27700 1 -- 172.21.15.80:0/712005158 >> [v2:172.21.15.80:6800/233364790,v1:172.21.15.80:6801/233364790] conn(0x7f1cf0038f40 msgr2=0x7f1cf003b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:43.221 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:42.704+0000 7f1d0be27700 1 --2- 172.21.15.80:0/712005158 >> [v2:172.21.15.80:6800/233364790,v1:172.21.15.80:6801/233364790] conn(0x7f1cf0038f40 0x7f1cf003b400 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1cf4006fd0 tx=0x7f1cf4006e40 comp rx=0 tx=0).stop 2024-04-24T23:19:43.221 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:42.704+0000 7f1d0be27700 1 -- 172.21.15.80:0/712005158 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1d04108120 msgr2=0x7f1d0419dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:43.221 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:42.704+0000 7f1d0be27700 1 --2- 172.21.15.80:0/712005158 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1d04108120 0x7f1d0419dd10 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1d00011670 tx=0x7f1d00011b60 comp rx=0 tx=0).stop 2024-04-24T23:19:43.221 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:42.704+0000 7f1d0be27700 1 -- 172.21.15.80:0/712005158 shutdown_connections 2024-04-24T23:19:43.221 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:42.704+0000 7f1d0be27700 1 --2- 172.21.15.80:0/712005158 >> [v2:172.21.15.80:6800/233364790,v1:172.21.15.80:6801/233364790] conn(0x7f1cf0038f40 0x7f1cf003b400 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:43.221 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:42.704+0000 7f1d0be27700 1 --2- 172.21.15.80:0/712005158 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1d04108120 0x7f1d0419dd10 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:43.221 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:42.704+0000 7f1d0be27700 1 -- 172.21.15.80:0/712005158 >> 172.21.15.80:0/712005158 conn(0x7f1d041017e0 msgr2=0x7f1d041943d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:43.221 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:42.705+0000 7f1d0be27700 1 -- 172.21.15.80:0/712005158 shutdown_connections 2024-04-24T23:19:43.221 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:42.705+0000 7f1d0be27700 1 -- 172.21.15.80:0/712005158 wait complete. 2024-04-24T23:19:43.595 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:43 smithi080 ceph-mon[29152]: [24/Apr/2024:23:19:42] ENGINE Bus STARTING 2024-04-24T23:19:43.596 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:43 smithi080 ceph-mon[29152]: [24/Apr/2024:23:19:42] ENGINE Serving on https://172.21.15.80:7150 2024-04-24T23:19:43.596 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:43 smithi080 ceph-mon[29152]: [24/Apr/2024:23:19:42] ENGINE Serving on http://172.21.15.80:8765 2024-04-24T23:19:43.596 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:43 smithi080 ceph-mon[29152]: [24/Apr/2024:23:19:42] ENGINE Bus STARTED 2024-04-24T23:19:43.596 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:43 smithi080 ceph-mon[29152]: mgrmap e8: x(active, since 2s) 2024-04-24T23:19:43.596 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:43 smithi080 ceph-mon[29152]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:19:43.596 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:43 smithi080 ceph-mon[29152]: from='mgr.14120 172.21.15.80:0/1767820559' entity='mgr.x' 2024-04-24T23:19:43.596 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:43 smithi080 ceph-mon[29152]: from='mgr.14120 172.21.15.80:0/1767820559' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:19:44.431 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout value unchanged 2024-04-24T23:19:44.431 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:43.695+0000 7f9578b23700 1 Processor -- start 2024-04-24T23:19:44.431 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:43.695+0000 7f9578b23700 1 -- start start 2024-04-24T23:19:44.431 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:43.696+0000 7f9578b23700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9574108660 0x7f957410aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:44.431 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:43.696+0000 7f9578b23700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f957410af90 con 0x7f9574108660 2024-04-24T23:19:44.431 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:43.696+0000 7f957259c700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9574108660 0x7f957410aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:44.431 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:43.696+0000 7f957259c700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9574108660 0x7f957410aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:43302/0 (socket says 172.21.15.80:43302) 2024-04-24T23:19:44.431 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:43.696+0000 7f957259c700 1 -- 172.21.15.80:0/3846243649 learned_addr learned my addr 172.21.15.80:0/3846243649 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:19:44.431 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:43.696+0000 7f957259c700 1 -- 172.21.15.80:0/3846243649 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f957410b110 con 0x7f9574108660 2024-04-24T23:19:44.431 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:43.696+0000 7f957259c700 1 --2- 172.21.15.80:0/3846243649 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9574108660 0x7f957410aa50 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f95680098a0 tx=0x7f9568009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d08bc4214dc73c97 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:44.432 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:43.696+0000 7f957159a700 1 -- 172.21.15.80:0/3846243649 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9568004160 con 0x7f9574108660 2024-04-24T23:19:44.432 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:43.696+0000 7f957159a700 1 -- 172.21.15.80:0/3846243649 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9568004300 con 0x7f9574108660 2024-04-24T23:19:44.432 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:43.696+0000 7f957159a700 1 -- 172.21.15.80:0/3846243649 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9568011b50 con 0x7f9574108660 2024-04-24T23:19:44.432 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:43.697+0000 7f9578b23700 1 -- 172.21.15.80:0/3846243649 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9574108660 msgr2=0x7f957410aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:44.432 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:43.697+0000 7f9578b23700 1 --2- 172.21.15.80:0/3846243649 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9574108660 0x7f957410aa50 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f95680098a0 tx=0x7f9568009bb0 comp rx=0 tx=0).stop 2024-04-24T23:19:44.432 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:43.697+0000 7f9578b23700 1 -- 172.21.15.80:0/3846243649 shutdown_connections 2024-04-24T23:19:44.432 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:43.697+0000 7f9578b23700 1 --2- 172.21.15.80:0/3846243649 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9574108660 0x7f957410aa50 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:44.432 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:43.697+0000 7f9578b23700 1 -- 172.21.15.80:0/3846243649 >> 172.21.15.80:0/3846243649 conn(0x7f9574102090 msgr2=0x7f95741044b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:44.432 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:43.697+0000 7f9578b23700 1 -- 172.21.15.80:0/3846243649 shutdown_connections 2024-04-24T23:19:44.432 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:43.697+0000 7f9578b23700 1 -- 172.21.15.80:0/3846243649 wait complete. 2024-04-24T23:19:44.432 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:43.698+0000 7f9578b23700 1 Processor -- start 2024-04-24T23:19:44.432 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:43.698+0000 7f9578b23700 1 -- start start 2024-04-24T23:19:44.432 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:43.698+0000 7f9578b23700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9574108660 0x7f957419bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:44.432 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:43.698+0000 7f9578b23700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f957410c0d0 con 0x7f9574108660 2024-04-24T23:19:44.433 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:43.698+0000 7f957259c700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9574108660 0x7f957419bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:44.433 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:43.698+0000 7f957259c700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9574108660 0x7f957419bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:43308/0 (socket says 172.21.15.80:43308) 2024-04-24T23:19:44.433 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:43.698+0000 7f957259c700 1 -- 172.21.15.80:0/1188761350 learned_addr learned my addr 172.21.15.80:0/1188761350 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:19:44.433 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:43.698+0000 7f957259c700 1 -- 172.21.15.80:0/1188761350 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f957419c1a0 con 0x7f9574108660 2024-04-24T23:19:44.433 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:43.698+0000 7f957259c700 1 --2- 172.21.15.80:0/1188761350 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9574108660 0x7f957419bc60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9568011b60 tx=0x7f9568011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:44.433 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:43.698+0000 7f95637fe700 1 -- 172.21.15.80:0/1188761350 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9568003b20 con 0x7f9574108660 2024-04-24T23:19:44.433 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:43.699+0000 7f9578b23700 1 -- 172.21.15.80:0/1188761350 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f957419c430 con 0x7f9574108660 2024-04-24T23:19:44.433 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:43.699+0000 7f95637fe700 1 -- 172.21.15.80:0/1188761350 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9568003cc0 con 0x7f9574108660 2024-04-24T23:19:44.433 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:43.699+0000 7f95637fe700 1 -- 172.21.15.80:0/1188761350 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9568024840 con 0x7f9574108660 2024-04-24T23:19:44.433 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:43.699+0000 7f9578b23700 1 -- 172.21.15.80:0/1188761350 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f957419f120 con 0x7f9574108660 2024-04-24T23:19:44.433 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:43.699+0000 7f95637fe700 1 -- 172.21.15.80:0/1188761350 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f9568022020 con 0x7f9574108660 2024-04-24T23:19:44.433 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:43.699+0000 7f9578b23700 1 -- 172.21.15.80:0/1188761350 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9574063890 con 0x7f9574108660 2024-04-24T23:19:44.433 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:43.700+0000 7f95637fe700 1 --2- 172.21.15.80:0/1188761350 >> [v2:172.21.15.80:6800/233364790,v1:172.21.15.80:6801/233364790] conn(0x7f955c039050 0x7f955c03b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:44.433 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:43.700+0000 7f95637fe700 1 -- 172.21.15.80:0/1188761350 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f9568011e00 con 0x7f9574108660 2024-04-24T23:19:44.434 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:43.700+0000 7f9571d9b700 1 --2- 172.21.15.80:0/1188761350 >> [v2:172.21.15.80:6800/233364790,v1:172.21.15.80:6801/233364790] conn(0x7f955c039050 0x7f955c03b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:44.434 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:43.700+0000 7f9571d9b700 1 --2- 172.21.15.80:0/1188761350 >> [v2:172.21.15.80:6800/233364790,v1:172.21.15.80:6801/233364790] conn(0x7f955c039050 0x7f955c03b510 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9564006fd0 tx=0x7f9564006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:44.434 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:43.705+0000 7f95637fe700 1 -- 172.21.15.80:0/1188761350 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f95680172b0 con 0x7f9574108660 2024-04-24T23:19:44.434 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:43.902+0000 7f9578b23700 1 -- 172.21.15.80:0/1188761350 --> [v2:172.21.15.80:6800/233364790,v1:172.21.15.80:6801/233364790] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f95741032b0 con 0x7f955c039050 2024-04-24T23:19:44.434 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:43.903+0000 7f95637fe700 1 -- 172.21.15.80:0/1188761350 <== mgr.14120 v2:172.21.15.80:6800/233364790 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f95741032b0 con 0x7f955c039050 2024-04-24T23:19:44.434 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:43.906+0000 7f9578b23700 1 -- 172.21.15.80:0/1188761350 >> [v2:172.21.15.80:6800/233364790,v1:172.21.15.80:6801/233364790] conn(0x7f955c039050 msgr2=0x7f955c03b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:44.434 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:43.906+0000 7f9578b23700 1 --2- 172.21.15.80:0/1188761350 >> [v2:172.21.15.80:6800/233364790,v1:172.21.15.80:6801/233364790] conn(0x7f955c039050 0x7f955c03b510 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9564006fd0 tx=0x7f9564006e40 comp rx=0 tx=0).stop 2024-04-24T23:19:44.434 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:43.906+0000 7f9578b23700 1 -- 172.21.15.80:0/1188761350 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9574108660 msgr2=0x7f957419bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:44.434 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:43.906+0000 7f9578b23700 1 --2- 172.21.15.80:0/1188761350 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9574108660 0x7f957419bc60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9568011b60 tx=0x7f9568011c40 comp rx=0 tx=0).stop 2024-04-24T23:19:44.434 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:43.906+0000 7f9578b23700 1 -- 172.21.15.80:0/1188761350 shutdown_connections 2024-04-24T23:19:44.434 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:43.906+0000 7f9578b23700 1 --2- 172.21.15.80:0/1188761350 >> [v2:172.21.15.80:6800/233364790,v1:172.21.15.80:6801/233364790] conn(0x7f955c039050 0x7f955c03b510 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:44.434 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:43.906+0000 7f9578b23700 1 --2- 172.21.15.80:0/1188761350 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9574108660 0x7f957419bc60 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:44.434 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:43.906+0000 7f9578b23700 1 -- 172.21.15.80:0/1188761350 >> 172.21.15.80:0/1188761350 conn(0x7f9574102090 msgr2=0x7f9574102ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:44.435 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:43.906+0000 7f9578b23700 1 -- 172.21.15.80:0/1188761350 shutdown_connections 2024-04-24T23:19:44.435 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:43.906+0000 7f9578b23700 1 -- 172.21.15.80:0/1188761350 wait complete. 2024-04-24T23:19:44.435 INFO:teuthology.orchestra.run.smithi080.stdout:Generating ssh key... 2024-04-24T23:19:45.084 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:44 smithi080 ceph-mon[29152]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:19:45.670 INFO:journalctl@ceph.mgr.x.smithi080.stdout:Apr 24 23:19:45 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-x[29381]: Generating public/private rsa key pair. 2024-04-24T23:19:45.670 INFO:journalctl@ceph.mgr.x.smithi080.stdout:Apr 24 23:19:45 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-x[29381]: Your identification has been saved in /tmp/tmpzro1ximq/key. 2024-04-24T23:19:45.670 INFO:journalctl@ceph.mgr.x.smithi080.stdout:Apr 24 23:19:45 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-x[29381]: Your public key has been saved in /tmp/tmpzro1ximq/key.pub. 2024-04-24T23:19:45.670 INFO:journalctl@ceph.mgr.x.smithi080.stdout:Apr 24 23:19:45 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-x[29381]: The key fingerprint is: 2024-04-24T23:19:45.670 INFO:journalctl@ceph.mgr.x.smithi080.stdout:Apr 24 23:19:45 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-x[29381]: SHA256:aKMdmG5ejE8t08bqPgMMN0YxAlZQYxno1PxaaS6xskA ceph-ec7885aa-0290-11ef-bc93-c7b262605968 2024-04-24T23:19:45.670 INFO:journalctl@ceph.mgr.x.smithi080.stdout:Apr 24 23:19:45 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-x[29381]: The key's randomart image is: 2024-04-24T23:19:45.671 INFO:journalctl@ceph.mgr.x.smithi080.stdout:Apr 24 23:19:45 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-x[29381]: +---[RSA 3072]----+ 2024-04-24T23:19:45.671 INFO:journalctl@ceph.mgr.x.smithi080.stdout:Apr 24 23:19:45 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-x[29381]: |.+O*=. | 2024-04-24T23:19:45.671 INFO:journalctl@ceph.mgr.x.smithi080.stdout:Apr 24 23:19:45 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-x[29381]: |.o.=o. | 2024-04-24T23:19:45.671 INFO:journalctl@ceph.mgr.x.smithi080.stdout:Apr 24 23:19:45 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-x[29381]: |o .. . | 2024-04-24T23:19:45.671 INFO:journalctl@ceph.mgr.x.smithi080.stdout:Apr 24 23:19:45 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-x[29381]: | E..+B . | 2024-04-24T23:19:45.671 INFO:journalctl@ceph.mgr.x.smithi080.stdout:Apr 24 23:19:45 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-x[29381]: |. =X.= S | 2024-04-24T23:19:45.671 INFO:journalctl@ceph.mgr.x.smithi080.stdout:Apr 24 23:19:45 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-x[29381]: |.. =oB * | 2024-04-24T23:19:45.671 INFO:journalctl@ceph.mgr.x.smithi080.stdout:Apr 24 23:19:45 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-x[29381]: |. o *.B = | 2024-04-24T23:19:45.671 INFO:journalctl@ceph.mgr.x.smithi080.stdout:Apr 24 23:19:45 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-x[29381]: | . o +o= | 2024-04-24T23:19:45.671 INFO:journalctl@ceph.mgr.x.smithi080.stdout:Apr 24 23:19:45 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-x[29381]: | .o=o | 2024-04-24T23:19:45.671 INFO:journalctl@ceph.mgr.x.smithi080.stdout:Apr 24 23:19:45 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-x[29381]: +----[SHA256]-----+ 2024-04-24T23:19:45.913 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:44.876+0000 7f0a96a17700 1 Processor -- start 2024-04-24T23:19:45.913 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:44.876+0000 7f0a96a17700 1 -- start start 2024-04-24T23:19:45.913 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:44.876+0000 7f0a96a17700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0a90106200 0x7f0a90106620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:45.913 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:44.876+0000 7f0a96a17700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f0a90106bf0 con 0x7f0a90106200 2024-04-24T23:19:45.913 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:44.876+0000 7f0a8ffff700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0a90106200 0x7f0a90106620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:45.913 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:44.876+0000 7f0a8ffff700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0a90106200 0x7f0a90106620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:43320/0 (socket says 172.21.15.80:43320) 2024-04-24T23:19:45.914 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:44.876+0000 7f0a8ffff700 1 -- 172.21.15.80:0/852651555 learned_addr learned my addr 172.21.15.80:0/852651555 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:19:45.914 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:44.876+0000 7f0a8ffff700 1 -- 172.21.15.80:0/852651555 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a90107400 con 0x7f0a90106200 2024-04-24T23:19:45.914 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:44.877+0000 7f0a8ffff700 1 --2- 172.21.15.80:0/852651555 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0a90106200 0x7f0a90106620 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f0a8000aa40 tx=0x7f0a8000ad50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f20c65c94faf532a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:45.914 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:44.877+0000 7f0a8effd700 1 -- 172.21.15.80:0/852651555 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a8000b040 con 0x7f0a90106200 2024-04-24T23:19:45.914 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:44.877+0000 7f0a8effd700 1 -- 172.21.15.80:0/852651555 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0a8000c8f0 con 0x7f0a90106200 2024-04-24T23:19:45.914 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:44.877+0000 7f0a8effd700 1 -- 172.21.15.80:0/852651555 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a80003c10 con 0x7f0a90106200 2024-04-24T23:19:45.914 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:44.877+0000 7f0a96a17700 1 -- 172.21.15.80:0/852651555 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0a90106200 msgr2=0x7f0a90106620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:45.914 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:44.877+0000 7f0a96a17700 1 --2- 172.21.15.80:0/852651555 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0a90106200 0x7f0a90106620 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f0a8000aa40 tx=0x7f0a8000ad50 comp rx=0 tx=0).stop 2024-04-24T23:19:45.914 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:44.877+0000 7f0a96a17700 1 -- 172.21.15.80:0/852651555 shutdown_connections 2024-04-24T23:19:45.914 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:44.877+0000 7f0a96a17700 1 --2- 172.21.15.80:0/852651555 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0a90106200 0x7f0a90106620 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:45.914 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:44.877+0000 7f0a96a17700 1 -- 172.21.15.80:0/852651555 >> 172.21.15.80:0/852651555 conn(0x7f0a90101780 msgr2=0x7f0a90103be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:45.914 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:44.877+0000 7f0a96a17700 1 -- 172.21.15.80:0/852651555 shutdown_connections 2024-04-24T23:19:45.914 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:44.877+0000 7f0a96a17700 1 -- 172.21.15.80:0/852651555 wait complete. 2024-04-24T23:19:45.914 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:44.878+0000 7f0a96a17700 1 Processor -- start 2024-04-24T23:19:45.915 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:44.878+0000 7f0a96a17700 1 -- start start 2024-04-24T23:19:45.915 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:44.878+0000 7f0a96a17700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0a90106200 0x7f0a9019bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:45.915 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:44.878+0000 7f0a96a17700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f0a90107b80 con 0x7f0a90106200 2024-04-24T23:19:45.915 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:44.878+0000 7f0a8ffff700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0a90106200 0x7f0a9019bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:45.915 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:44.878+0000 7f0a8ffff700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0a90106200 0x7f0a9019bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:43324/0 (socket says 172.21.15.80:43324) 2024-04-24T23:19:45.915 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:44.878+0000 7f0a8ffff700 1 -- 172.21.15.80:0/863802024 learned_addr learned my addr 172.21.15.80:0/863802024 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:19:45.915 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:44.879+0000 7f0a8ffff700 1 -- 172.21.15.80:0/863802024 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a9019c020 con 0x7f0a90106200 2024-04-24T23:19:45.915 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:44.879+0000 7f0a8ffff700 1 --2- 172.21.15.80:0/863802024 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0a90106200 0x7f0a9019bae0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0a8000afd0 tx=0x7f0a80006c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:45.915 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:44.879+0000 7f0a8d7fa700 1 -- 172.21.15.80:0/863802024 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a80025070 con 0x7f0a90106200 2024-04-24T23:19:45.916 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:44.879+0000 7f0a96a17700 1 -- 172.21.15.80:0/863802024 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a9019c2b0 con 0x7f0a90106200 2024-04-24T23:19:45.916 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:44.879+0000 7f0a8d7fa700 1 -- 172.21.15.80:0/863802024 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0a80004340 con 0x7f0a90106200 2024-04-24T23:19:45.916 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:44.879+0000 7f0a8d7fa700 1 -- 172.21.15.80:0/863802024 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a80010740 con 0x7f0a90106200 2024-04-24T23:19:45.916 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:44.879+0000 7f0a96a17700 1 -- 172.21.15.80:0/863802024 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a9019efa0 con 0x7f0a90106200 2024-04-24T23:19:45.916 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:44.880+0000 7f0a8d7fa700 1 -- 172.21.15.80:0/863802024 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f0a80028070 con 0x7f0a90106200 2024-04-24T23:19:45.916 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:44.880+0000 7f0a96a17700 1 -- 172.21.15.80:0/863802024 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a90063890 con 0x7f0a90106200 2024-04-24T23:19:45.916 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:44.880+0000 7f0a8d7fa700 1 --2- 172.21.15.80:0/863802024 >> [v2:172.21.15.80:6800/233364790,v1:172.21.15.80:6801/233364790] conn(0x7f0a780390a0 0x7f0a7803b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:45.916 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:44.880+0000 7f0a8d7fa700 1 -- 172.21.15.80:0/863802024 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f0a8004ce40 con 0x7f0a90106200 2024-04-24T23:19:45.917 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:44.880+0000 7f0a8f7fe700 1 --2- 172.21.15.80:0/863802024 >> [v2:172.21.15.80:6800/233364790,v1:172.21.15.80:6801/233364790] conn(0x7f0a780390a0 0x7f0a7803b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:45.917 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:44.881+0000 7f0a8f7fe700 1 --2- 172.21.15.80:0/863802024 >> [v2:172.21.15.80:6800/233364790,v1:172.21.15.80:6801/233364790] conn(0x7f0a780390a0 0x7f0a7803b560 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f0a84006fd0 tx=0x7f0a84006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:45.917 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:44.885+0000 7f0a8d7fa700 1 -- 172.21.15.80:0/863802024 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f0a80014070 con 0x7f0a90106200 2024-04-24T23:19:45.917 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:45.081+0000 7f0a96a17700 1 -- 172.21.15.80:0/863802024 --> [v2:172.21.15.80:6800/233364790,v1:172.21.15.80:6801/233364790] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f0a900008d0 con 0x7f0a780390a0 2024-04-24T23:19:45.917 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:45.383+0000 7f0a8d7fa700 1 -- 172.21.15.80:0/863802024 <== mgr.14120 v2:172.21.15.80:6800/233364790 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f0a900008d0 con 0x7f0a780390a0 2024-04-24T23:19:45.917 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:45.385+0000 7f0a96a17700 1 -- 172.21.15.80:0/863802024 >> [v2:172.21.15.80:6800/233364790,v1:172.21.15.80:6801/233364790] conn(0x7f0a780390a0 msgr2=0x7f0a7803b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:45.917 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:45.385+0000 7f0a96a17700 1 --2- 172.21.15.80:0/863802024 >> [v2:172.21.15.80:6800/233364790,v1:172.21.15.80:6801/233364790] conn(0x7f0a780390a0 0x7f0a7803b560 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f0a84006fd0 tx=0x7f0a84006e40 comp rx=0 tx=0).stop 2024-04-24T23:19:45.917 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:45.385+0000 7f0a96a17700 1 -- 172.21.15.80:0/863802024 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0a90106200 msgr2=0x7f0a9019bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:45.917 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:45.385+0000 7f0a96a17700 1 --2- 172.21.15.80:0/863802024 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0a90106200 0x7f0a9019bae0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0a8000afd0 tx=0x7f0a80006c80 comp rx=0 tx=0).stop 2024-04-24T23:19:45.917 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:45.385+0000 7f0a96a17700 1 -- 172.21.15.80:0/863802024 shutdown_connections 2024-04-24T23:19:45.917 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:45.385+0000 7f0a96a17700 1 --2- 172.21.15.80:0/863802024 >> [v2:172.21.15.80:6800/233364790,v1:172.21.15.80:6801/233364790] conn(0x7f0a780390a0 0x7f0a7803b560 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:45.917 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:45.385+0000 7f0a96a17700 1 --2- 172.21.15.80:0/863802024 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0a90106200 0x7f0a9019bae0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:45.917 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:45.385+0000 7f0a96a17700 1 -- 172.21.15.80:0/863802024 >> 172.21.15.80:0/863802024 conn(0x7f0a90101780 msgr2=0x7f0a901926d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:45.917 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:45.385+0000 7f0a96a17700 1 -- 172.21.15.80:0/863802024 shutdown_connections 2024-04-24T23:19:45.917 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:45.385+0000 7f0a96a17700 1 -- 172.21.15.80:0/863802024 wait complete. 2024-04-24T23:19:46.789 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:46 smithi080 ceph-mon[29152]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:19:46.796 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:46 smithi080 ceph-mon[29152]: Generating ssh key... 2024-04-24T23:19:46.796 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:46 smithi080 ceph-mon[29152]: from='mgr.14120 172.21.15.80:0/1767820559' entity='mgr.x' 2024-04-24T23:19:46.797 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:46 smithi080 ceph-mon[29152]: from='mgr.14120 172.21.15.80:0/1767820559' entity='mgr.x' 2024-04-24T23:19:47.052 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDbNYYWP8+Aw48kkQl7GjpK+h7iccwwpwIixWyTdY4ltK8vJHs+d5mPs9p75pmCGMWht6/+QHNE8RcZcRP+w5k5tz6QxN8FaOwONv6M0vqSuGLA9twcdK1J3NtnUYULZEeCrr8+Hzq89pr5pHHVMEckiJVRu+REn0mcBbJZNKDnAridhebH0g6XaSujuKxNjhgm3mA7/DBpP7hzZJqD7V1umVkQLbaTNJ/0LZ9yH82D4X05UgAq7W1cDQl4DZEpyjZHiWWvcwkQNmMeQYfqcncS9ZSRoRY2Iz/mJZbuQNvqLb/QluOHe1mR9OeLH9QRsxyu5KaMNar0B3rtJBJa5GNDoFwDaYHkkxywRt/IMPzr9jYOvM14xorPMpHpjwV0OI59kx3EtM7wLJNGb1KKc6vM9morpK3KHWL5QoKZwveV0rl8sUkHcQLUV3H83flTCBEcS6XrjXng2aqscbmio6WrPhXGVwDl5QGrnsqjK5k7gQVcXToIeLhEzuEQSLzlvpM= ceph-ec7885aa-0290-11ef-bc93-c7b262605968 2024-04-24T23:19:47.052 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:46.312+0000 7f4b94e81700 1 Processor -- start 2024-04-24T23:19:47.052 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:46.312+0000 7f4b94e81700 1 -- start start 2024-04-24T23:19:47.052 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:46.313+0000 7f4b94e81700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4b90108190 0x7f4b901085b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:47.052 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:46.313+0000 7f4b94e81700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f4b901066d0 con 0x7f4b90108190 2024-04-24T23:19:47.052 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:46.313+0000 7f4b8e59c700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4b90108190 0x7f4b901085b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:47.052 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:46.313+0000 7f4b8e59c700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4b90108190 0x7f4b901085b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:43338/0 (socket says 172.21.15.80:43338) 2024-04-24T23:19:47.052 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:46.313+0000 7f4b8e59c700 1 -- 172.21.15.80:0/1873636896 learned_addr learned my addr 172.21.15.80:0/1873636896 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:19:47.053 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:46.313+0000 7f4b8e59c700 1 -- 172.21.15.80:0/1873636896 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b90106850 con 0x7f4b90108190 2024-04-24T23:19:47.053 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:46.313+0000 7f4b8e59c700 1 --2- 172.21.15.80:0/1873636896 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4b90108190 0x7f4b901085b0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4b800098a0 tx=0x7f4b80009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb9d1bef20b86f5e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:47.053 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:46.313+0000 7f4b8d59a700 1 -- 172.21.15.80:0/1873636896 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4b80004160 con 0x7f4b90108190 2024-04-24T23:19:47.053 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:46.313+0000 7f4b8d59a700 1 -- 172.21.15.80:0/1873636896 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4b80004300 con 0x7f4b90108190 2024-04-24T23:19:47.053 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:46.313+0000 7f4b8d59a700 1 -- 172.21.15.80:0/1873636896 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4b80011b50 con 0x7f4b90108190 2024-04-24T23:19:47.053 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:46.314+0000 7f4b94e81700 1 -- 172.21.15.80:0/1873636896 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4b90108190 msgr2=0x7f4b901085b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:47.053 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:46.314+0000 7f4b94e81700 1 --2- 172.21.15.80:0/1873636896 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4b90108190 0x7f4b901085b0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4b800098a0 tx=0x7f4b80009bb0 comp rx=0 tx=0).stop 2024-04-24T23:19:47.053 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:46.314+0000 7f4b94e81700 1 -- 172.21.15.80:0/1873636896 shutdown_connections 2024-04-24T23:19:47.053 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:46.314+0000 7f4b94e81700 1 --2- 172.21.15.80:0/1873636896 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4b90108190 0x7f4b901085b0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:47.053 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:46.314+0000 7f4b94e81700 1 -- 172.21.15.80:0/1873636896 >> 172.21.15.80:0/1873636896 conn(0x7f4b90102030 msgr2=0x7f4b90104490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:47.053 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:46.314+0000 7f4b94e81700 1 -- 172.21.15.80:0/1873636896 shutdown_connections 2024-04-24T23:19:47.053 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:46.314+0000 7f4b94e81700 1 -- 172.21.15.80:0/1873636896 wait complete. 2024-04-24T23:19:47.053 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:46.315+0000 7f4b94e81700 1 Processor -- start 2024-04-24T23:19:47.053 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:46.315+0000 7f4b94e81700 1 -- start start 2024-04-24T23:19:47.054 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:46.315+0000 7f4b94e81700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4b90108190 0x7f4b90191d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:47.054 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:46.315+0000 7f4b94e81700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f4b9010b060 con 0x7f4b90108190 2024-04-24T23:19:47.054 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:46.315+0000 7f4b8e59c700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4b90108190 0x7f4b90191d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:47.054 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:46.315+0000 7f4b8e59c700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4b90108190 0x7f4b90191d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:43340/0 (socket says 172.21.15.80:43340) 2024-04-24T23:19:47.054 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:46.315+0000 7f4b8e59c700 1 -- 172.21.15.80:0/2029264737 learned_addr learned my addr 172.21.15.80:0/2029264737 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:19:47.054 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:46.315+0000 7f4b8e59c700 1 -- 172.21.15.80:0/2029264737 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b901922b0 con 0x7f4b90108190 2024-04-24T23:19:47.054 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:46.315+0000 7f4b8e59c700 1 --2- 172.21.15.80:0/2029264737 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4b90108190 0x7f4b90191d70 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f4b80011670 tx=0x7f4b80011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:47.054 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:46.316+0000 7f4b877fe700 1 -- 172.21.15.80:0/2029264737 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4b800039c0 con 0x7f4b90108190 2024-04-24T23:19:47.054 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:46.316+0000 7f4b94e81700 1 -- 172.21.15.80:0/2029264737 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b901904a0 con 0x7f4b90108190 2024-04-24T23:19:47.054 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:46.316+0000 7f4b877fe700 1 -- 172.21.15.80:0/2029264737 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4b80011e50 con 0x7f4b90108190 2024-04-24T23:19:47.054 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:46.316+0000 7f4b877fe700 1 -- 172.21.15.80:0/2029264737 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4b8001a5c0 con 0x7f4b90108190 2024-04-24T23:19:47.054 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:46.316+0000 7f4b94e81700 1 -- 172.21.15.80:0/2029264737 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b901909e0 con 0x7f4b90108190 2024-04-24T23:19:47.054 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:46.317+0000 7f4b877fe700 1 -- 172.21.15.80:0/2029264737 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f4b8001a8c0 con 0x7f4b90108190 2024-04-24T23:19:47.054 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:46.317+0000 7f4b94e81700 1 -- 172.21.15.80:0/2029264737 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b90063890 con 0x7f4b90108190 2024-04-24T23:19:47.055 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:46.317+0000 7f4b877fe700 1 --2- 172.21.15.80:0/2029264737 >> [v2:172.21.15.80:6800/233364790,v1:172.21.15.80:6801/233364790] conn(0x7f4b7c039000 0x7f4b7c03b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:47.055 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:46.317+0000 7f4b877fe700 1 -- 172.21.15.80:0/2029264737 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f4b8004d580 con 0x7f4b90108190 2024-04-24T23:19:47.055 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:46.317+0000 7f4b8dd9b700 1 --2- 172.21.15.80:0/2029264737 >> [v2:172.21.15.80:6800/233364790,v1:172.21.15.80:6801/233364790] conn(0x7f4b7c039000 0x7f4b7c03b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:47.055 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:46.317+0000 7f4b8dd9b700 1 --2- 172.21.15.80:0/2029264737 >> [v2:172.21.15.80:6800/233364790,v1:172.21.15.80:6801/233364790] conn(0x7f4b7c039000 0x7f4b7c03b4c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4b78006fd0 tx=0x7f4b78006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:47.055 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:46.322+0000 7f4b877fe700 1 -- 172.21.15.80:0/2029264737 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f4b80019c70 con 0x7f4b90108190 2024-04-24T23:19:47.055 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:46.520+0000 7f4b94e81700 1 -- 172.21.15.80:0/2029264737 --> [v2:172.21.15.80:6800/233364790,v1:172.21.15.80:6801/233364790] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f4b90195a70 con 0x7f4b7c039000 2024-04-24T23:19:47.055 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:46.520+0000 7f4b877fe700 1 -- 172.21.15.80:0/2029264737 <== mgr.14120 v2:172.21.15.80:6800/233364790 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f4b90195a70 con 0x7f4b7c039000 2024-04-24T23:19:47.055 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:46.522+0000 7f4b94e81700 1 -- 172.21.15.80:0/2029264737 >> [v2:172.21.15.80:6800/233364790,v1:172.21.15.80:6801/233364790] conn(0x7f4b7c039000 msgr2=0x7f4b7c03b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:47.055 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:46.522+0000 7f4b94e81700 1 --2- 172.21.15.80:0/2029264737 >> [v2:172.21.15.80:6800/233364790,v1:172.21.15.80:6801/233364790] conn(0x7f4b7c039000 0x7f4b7c03b4c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4b78006fd0 tx=0x7f4b78006e40 comp rx=0 tx=0).stop 2024-04-24T23:19:47.055 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:46.522+0000 7f4b94e81700 1 -- 172.21.15.80:0/2029264737 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4b90108190 msgr2=0x7f4b90191d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:47.056 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:46.522+0000 7f4b94e81700 1 --2- 172.21.15.80:0/2029264737 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4b90108190 0x7f4b90191d70 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f4b80011670 tx=0x7f4b80011b60 comp rx=0 tx=0).stop 2024-04-24T23:19:47.056 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:46.522+0000 7f4b94e81700 1 -- 172.21.15.80:0/2029264737 shutdown_connections 2024-04-24T23:19:47.056 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:46.522+0000 7f4b94e81700 1 --2- 172.21.15.80:0/2029264737 >> [v2:172.21.15.80:6800/233364790,v1:172.21.15.80:6801/233364790] conn(0x7f4b7c039000 0x7f4b7c03b4c0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:47.056 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:46.522+0000 7f4b94e81700 1 --2- 172.21.15.80:0/2029264737 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4b90108190 0x7f4b90191d70 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:47.056 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:46.522+0000 7f4b94e81700 1 -- 172.21.15.80:0/2029264737 >> 172.21.15.80:0/2029264737 conn(0x7f4b90102030 msgr2=0x7f4b90102f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:47.056 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:46.523+0000 7f4b94e81700 1 -- 172.21.15.80:0/2029264737 shutdown_connections 2024-04-24T23:19:47.056 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:46.523+0000 7f4b94e81700 1 -- 172.21.15.80:0/2029264737 wait complete. 2024-04-24T23:19:47.056 INFO:teuthology.orchestra.run.smithi080.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-04-24T23:19:47.056 INFO:teuthology.orchestra.run.smithi080.stdout:Adding key to root@localhost authorized_keys... 2024-04-24T23:19:47.056 INFO:teuthology.orchestra.run.smithi080.stdout:Adding host smithi080... 2024-04-24T23:19:47.654 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:47 smithi080 ceph-mon[29152]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:19:48.470 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:48 smithi080 ceph-mon[29152]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi080", "addr": "172.21.15.80", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:19:49.789 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:49 smithi080 ceph-mon[29152]: Deploying cephadm binary to smithi080 2024-04-24T23:19:52.253 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout Added host 'smithi080' with addr '172.21.15.80' 2024-04-24T23:19:52.253 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:47.445+0000 7f8134b86700 1 Processor -- start 2024-04-24T23:19:52.253 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:47.445+0000 7f8134b86700 1 -- start start 2024-04-24T23:19:52.254 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:47.446+0000 7f8134b86700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8130106250 0x7f8130106670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:52.254 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:47.446+0000 7f8134b86700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f8130106c40 con 0x7f8130106250 2024-04-24T23:19:52.254 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:47.446+0000 7f812e59c700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8130106250 0x7f8130106670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:52.254 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:47.446+0000 7f812e59c700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8130106250 0x7f8130106670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:43352/0 (socket says 172.21.15.80:43352) 2024-04-24T23:19:52.254 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:47.446+0000 7f812e59c700 1 -- 172.21.15.80:0/2882530236 learned_addr learned my addr 172.21.15.80:0/2882530236 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:19:52.255 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:47.446+0000 7f812e59c700 1 -- 172.21.15.80:0/2882530236 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8130107450 con 0x7f8130106250 2024-04-24T23:19:52.255 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:47.446+0000 7f812e59c700 1 --2- 172.21.15.80:0/2882530236 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8130106250 0x7f8130106670 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f81200098a0 tx=0x7f8120009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef8171baf8a158dd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:52.255 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:47.446+0000 7f812d59a700 1 -- 172.21.15.80:0/2882530236 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8120004160 con 0x7f8130106250 2024-04-24T23:19:52.255 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:47.446+0000 7f812d59a700 1 -- 172.21.15.80:0/2882530236 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8120004300 con 0x7f8130106250 2024-04-24T23:19:52.255 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:47.446+0000 7f812d59a700 1 -- 172.21.15.80:0/2882530236 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8120011b50 con 0x7f8130106250 2024-04-24T23:19:52.255 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:47.447+0000 7f8134b86700 1 -- 172.21.15.80:0/2882530236 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8130106250 msgr2=0x7f8130106670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:52.255 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:47.447+0000 7f8134b86700 1 --2- 172.21.15.80:0/2882530236 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8130106250 0x7f8130106670 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f81200098a0 tx=0x7f8120009bb0 comp rx=0 tx=0).stop 2024-04-24T23:19:52.255 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:47.447+0000 7f8134b86700 1 -- 172.21.15.80:0/2882530236 shutdown_connections 2024-04-24T23:19:52.256 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:47.447+0000 7f8134b86700 1 --2- 172.21.15.80:0/2882530236 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8130106250 0x7f8130106670 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:52.256 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:47.447+0000 7f8134b86700 1 -- 172.21.15.80:0/2882530236 >> 172.21.15.80:0/2882530236 conn(0x7f8130101830 msgr2=0x7f8130103c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:52.256 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:47.447+0000 7f8134b86700 1 -- 172.21.15.80:0/2882530236 shutdown_connections 2024-04-24T23:19:52.256 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:47.447+0000 7f8134b86700 1 -- 172.21.15.80:0/2882530236 wait complete. 2024-04-24T23:19:52.256 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:47.448+0000 7f8134b86700 1 Processor -- start 2024-04-24T23:19:52.256 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:47.448+0000 7f8134b86700 1 -- start start 2024-04-24T23:19:52.256 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:47.448+0000 7f8134b86700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8130106250 0x7f8130102c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:52.256 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:47.448+0000 7f8134b86700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f813010a800 con 0x7f8130106250 2024-04-24T23:19:52.256 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:47.448+0000 7f812e59c700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8130106250 0x7f8130102c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:52.257 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:47.448+0000 7f812e59c700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8130106250 0x7f8130102c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:43360/0 (socket says 172.21.15.80:43360) 2024-04-24T23:19:52.257 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:47.448+0000 7f812e59c700 1 -- 172.21.15.80:0/3478966485 learned_addr learned my addr 172.21.15.80:0/3478966485 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:19:52.257 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:47.448+0000 7f812e59c700 1 -- 172.21.15.80:0/3478966485 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f813019e8f0 con 0x7f8130106250 2024-04-24T23:19:52.257 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:47.448+0000 7f812e59c700 1 --2- 172.21.15.80:0/3478966485 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8130106250 0x7f8130102c90 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8120011670 tx=0x7f8120011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:52.257 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:47.449+0000 7f811f7fe700 1 -- 172.21.15.80:0/3478966485 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f81200039c0 con 0x7f8130106250 2024-04-24T23:19:52.257 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:47.449+0000 7f811f7fe700 1 -- 172.21.15.80:0/3478966485 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8120011e50 con 0x7f8130106250 2024-04-24T23:19:52.257 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:47.449+0000 7f8134b86700 1 -- 172.21.15.80:0/3478966485 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f813019eac0 con 0x7f8130106250 2024-04-24T23:19:52.257 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:47.449+0000 7f811f7fe700 1 -- 172.21.15.80:0/3478966485 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f812001a5c0 con 0x7f8130106250 2024-04-24T23:19:52.257 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:47.449+0000 7f8134b86700 1 -- 172.21.15.80:0/3478966485 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f813019ee60 con 0x7f8130106250 2024-04-24T23:19:52.257 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:47.449+0000 7f811f7fe700 1 -- 172.21.15.80:0/3478966485 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f812001a8c0 con 0x7f8130106250 2024-04-24T23:19:52.258 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:47.449+0000 7f8134b86700 1 -- 172.21.15.80:0/3478966485 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8110005330 con 0x7f8130106250 2024-04-24T23:19:52.258 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:47.450+0000 7f811f7fe700 1 --2- 172.21.15.80:0/3478966485 >> [v2:172.21.15.80:6800/233364790,v1:172.21.15.80:6801/233364790] conn(0x7f8118039000 0x7f811803b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:52.258 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:47.450+0000 7f811f7fe700 1 -- 172.21.15.80:0/3478966485 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f812004d490 con 0x7f8130106250 2024-04-24T23:19:52.258 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:47.450+0000 7f812dd9b700 1 --2- 172.21.15.80:0/3478966485 >> [v2:172.21.15.80:6800/233364790,v1:172.21.15.80:6801/233364790] conn(0x7f8118039000 0x7f811803b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:52.258 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:47.450+0000 7f812dd9b700 1 --2- 172.21.15.80:0/3478966485 >> [v2:172.21.15.80:6800/233364790,v1:172.21.15.80:6801/233364790] conn(0x7f8118039000 0x7f811803b4c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8124006fd0 tx=0x7f8124006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:52.258 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:47.454+0000 7f811f7fe700 1 -- 172.21.15.80:0/3478966485 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f8120019c70 con 0x7f8130106250 2024-04-24T23:19:52.258 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:47.652+0000 7f8134b86700 1 -- 172.21.15.80:0/3478966485 --> [v2:172.21.15.80:6800/233364790,v1:172.21.15.80:6801/233364790] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi080", "addr": "172.21.15.80", "target": ["mon-mgr", ""]}) -- 0x7f8110000bc0 con 0x7f8118039000 2024-04-24T23:19:52.258 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:51.660+0000 7f811f7fe700 1 -- 172.21.15.80:0/3478966485 <== mgr.14120 v2:172.21.15.80:6800/233364790 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f8110000bc0 con 0x7f8118039000 2024-04-24T23:19:52.259 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:51.663+0000 7f8134b86700 1 -- 172.21.15.80:0/3478966485 >> [v2:172.21.15.80:6800/233364790,v1:172.21.15.80:6801/233364790] conn(0x7f8118039000 msgr2=0x7f811803b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:52.259 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:51.663+0000 7f8134b86700 1 --2- 172.21.15.80:0/3478966485 >> [v2:172.21.15.80:6800/233364790,v1:172.21.15.80:6801/233364790] conn(0x7f8118039000 0x7f811803b4c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8124006fd0 tx=0x7f8124006e40 comp rx=0 tx=0).stop 2024-04-24T23:19:52.259 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:51.663+0000 7f8134b86700 1 -- 172.21.15.80:0/3478966485 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8130106250 msgr2=0x7f8130102c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:52.259 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:51.663+0000 7f8134b86700 1 --2- 172.21.15.80:0/3478966485 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8130106250 0x7f8130102c90 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8120011670 tx=0x7f8120011b60 comp rx=0 tx=0).stop 2024-04-24T23:19:52.259 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:51.663+0000 7f8134b86700 1 -- 172.21.15.80:0/3478966485 shutdown_connections 2024-04-24T23:19:52.259 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:51.663+0000 7f8134b86700 1 --2- 172.21.15.80:0/3478966485 >> [v2:172.21.15.80:6800/233364790,v1:172.21.15.80:6801/233364790] conn(0x7f8118039000 0x7f811803b4c0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:52.259 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:51.663+0000 7f8134b86700 1 --2- 172.21.15.80:0/3478966485 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8130106250 0x7f8130102c90 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:52.259 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:51.663+0000 7f8134b86700 1 -- 172.21.15.80:0/3478966485 >> 172.21.15.80:0/3478966485 conn(0x7f8130101830 msgr2=0x7f8130109ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:52.259 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:51.663+0000 7f8134b86700 1 -- 172.21.15.80:0/3478966485 shutdown_connections 2024-04-24T23:19:52.259 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:51.663+0000 7f8134b86700 1 -- 172.21.15.80:0/3478966485 wait complete. 2024-04-24T23:19:52.259 INFO:teuthology.orchestra.run.smithi080.stdout:Deploying unmanaged mon service... 2024-04-24T23:19:52.892 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:52 smithi080 ceph-mon[29152]: from='mgr.14120 172.21.15.80:0/1767820559' entity='mgr.x' 2024-04-24T23:19:52.892 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:52 smithi080 ceph-mon[29152]: Added host smithi080 2024-04-24T23:19:52.892 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:52 smithi080 ceph-mon[29152]: from='mgr.14120 172.21.15.80:0/1767820559' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:19:53.520 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-04-24T23:19:53.521 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:52.666+0000 7fe200598700 1 Processor -- start 2024-04-24T23:19:53.521 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:52.666+0000 7fe200598700 1 -- start start 2024-04-24T23:19:53.521 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:52.666+0000 7fe200598700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe1f8106fb0 0x7fe1f81073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:53.521 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:52.666+0000 7fe200598700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f81079a0 con 0x7fe1f8106fb0 2024-04-24T23:19:53.521 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:52.666+0000 7fe1fe334700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe1f8106fb0 0x7fe1f81073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:53.521 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:52.666+0000 7fe1fe334700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe1f8106fb0 0x7fe1f81073d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:41280/0 (socket says 172.21.15.80:41280) 2024-04-24T23:19:53.521 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:52.666+0000 7fe1fe334700 1 -- 172.21.15.80:0/211307600 learned_addr learned my addr 172.21.15.80:0/211307600 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:19:53.521 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:52.667+0000 7fe1fe334700 1 -- 172.21.15.80:0/211307600 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1f8107cd0 con 0x7fe1f8106fb0 2024-04-24T23:19:53.521 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:52.667+0000 7fe1fe334700 1 --2- 172.21.15.80:0/211307600 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe1f8106fb0 0x7fe1f81073d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe1f40098a0 tx=0x7fe1f4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f441a9ac34f604e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:53.521 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:52.667+0000 7fe1fd332700 1 -- 172.21.15.80:0/211307600 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe1f4004160 con 0x7fe1f8106fb0 2024-04-24T23:19:53.521 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:52.667+0000 7fe1fd332700 1 -- 172.21.15.80:0/211307600 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe1f4004300 con 0x7fe1f8106fb0 2024-04-24T23:19:53.521 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:52.667+0000 7fe1fd332700 1 -- 172.21.15.80:0/211307600 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe1f4011b50 con 0x7fe1f8106fb0 2024-04-24T23:19:53.522 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:52.668+0000 7fe200598700 1 -- 172.21.15.80:0/211307600 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe1f8106fb0 msgr2=0x7fe1f81073d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:53.522 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:52.668+0000 7fe200598700 1 --2- 172.21.15.80:0/211307600 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe1f8106fb0 0x7fe1f81073d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe1f40098a0 tx=0x7fe1f4009bb0 comp rx=0 tx=0).stop 2024-04-24T23:19:53.522 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:52.668+0000 7fe200598700 1 -- 172.21.15.80:0/211307600 shutdown_connections 2024-04-24T23:19:53.522 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:52.668+0000 7fe200598700 1 --2- 172.21.15.80:0/211307600 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe1f8106fb0 0x7fe1f81073d0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:53.522 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:52.668+0000 7fe200598700 1 -- 172.21.15.80:0/211307600 >> 172.21.15.80:0/211307600 conn(0x7fe1f8101810 msgr2=0x7fe1f8103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:53.522 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:52.668+0000 7fe200598700 1 -- 172.21.15.80:0/211307600 shutdown_connections 2024-04-24T23:19:53.522 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:52.668+0000 7fe200598700 1 -- 172.21.15.80:0/211307600 wait complete. 2024-04-24T23:19:53.522 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:52.668+0000 7fe200598700 1 Processor -- start 2024-04-24T23:19:53.522 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:52.668+0000 7fe200598700 1 -- start start 2024-04-24T23:19:53.522 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:52.668+0000 7fe200598700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe1f8106fb0 0x7fe1f8191c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:53.522 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:52.668+0000 7fe200598700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f810b080 con 0x7fe1f8106fb0 2024-04-24T23:19:53.522 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:52.669+0000 7fe1fe334700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe1f8106fb0 0x7fe1f8191c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:53.522 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:52.669+0000 7fe1fe334700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe1f8106fb0 0x7fe1f8191c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:41288/0 (socket says 172.21.15.80:41288) 2024-04-24T23:19:53.522 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:52.669+0000 7fe1fe334700 1 -- 172.21.15.80:0/3536105151 learned_addr learned my addr 172.21.15.80:0/3536105151 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:19:53.523 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:52.669+0000 7fe1fe334700 1 -- 172.21.15.80:0/3536105151 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1f8192180 con 0x7fe1f8106fb0 2024-04-24T23:19:53.523 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:52.669+0000 7fe1fe334700 1 --2- 172.21.15.80:0/3536105151 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe1f8106fb0 0x7fe1f8191c40 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe1f4011670 tx=0x7fe1f4011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:53.523 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:52.669+0000 7fe1ef7fe700 1 -- 172.21.15.80:0/3536105151 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe1f40039c0 con 0x7fe1f8106fb0 2024-04-24T23:19:53.523 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:52.669+0000 7fe1ef7fe700 1 -- 172.21.15.80:0/3536105151 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe1f4011e50 con 0x7fe1f8106fb0 2024-04-24T23:19:53.523 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:52.669+0000 7fe200598700 1 -- 172.21.15.80:0/3536105151 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1f8190370 con 0x7fe1f8106fb0 2024-04-24T23:19:53.523 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:52.669+0000 7fe1ef7fe700 1 -- 172.21.15.80:0/3536105151 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe1f401a5c0 con 0x7fe1f8106fb0 2024-04-24T23:19:53.523 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:52.669+0000 7fe200598700 1 -- 172.21.15.80:0/3536105151 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1f81908b0 con 0x7fe1f8106fb0 2024-04-24T23:19:53.523 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:52.670+0000 7fe200598700 1 -- 172.21.15.80:0/3536105151 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1f8063890 con 0x7fe1f8106fb0 2024-04-24T23:19:53.523 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:52.670+0000 7fe1ef7fe700 1 -- 172.21.15.80:0/3536105151 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fe1f4024430 con 0x7fe1f8106fb0 2024-04-24T23:19:53.523 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:52.670+0000 7fe1ef7fe700 1 --2- 172.21.15.80:0/3536105151 >> [v2:172.21.15.80:6800/233364790,v1:172.21.15.80:6801/233364790] conn(0x7fe1e4039050 0x7fe1e403b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:53.523 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:52.670+0000 7fe1ef7fe700 1 -- 172.21.15.80:0/3536105151 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fe1f404d3f0 con 0x7fe1f8106fb0 2024-04-24T23:19:53.523 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:52.670+0000 7fe1fdb33700 1 --2- 172.21.15.80:0/3536105151 >> [v2:172.21.15.80:6800/233364790,v1:172.21.15.80:6801/233364790] conn(0x7fe1e4039050 0x7fe1e403b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:53.523 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:52.671+0000 7fe1fdb33700 1 --2- 172.21.15.80:0/3536105151 >> [v2:172.21.15.80:6800/233364790,v1:172.21.15.80:6801/233364790] conn(0x7fe1e4039050 0x7fe1e403b510 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fe1e8006fd0 tx=0x7fe1e8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:53.524 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:52.675+0000 7fe1ef7fe700 1 -- 172.21.15.80:0/3536105151 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fe1f404cd40 con 0x7fe1f8106fb0 2024-04-24T23:19:53.524 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:52.889+0000 7fe200598700 1 -- 172.21.15.80:0/3536105151 --> [v2:172.21.15.80:6800/233364790,v1:172.21.15.80:6801/233364790] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fe1f8195a00 con 0x7fe1e4039050 2024-04-24T23:19:53.524 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:52.897+0000 7fe1ef7fe700 1 -- 172.21.15.80:0/3536105151 <== mgr.14120 v2:172.21.15.80:6800/233364790 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fe1f8195a00 con 0x7fe1e4039050 2024-04-24T23:19:53.524 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:52.899+0000 7fe200598700 1 -- 172.21.15.80:0/3536105151 >> [v2:172.21.15.80:6800/233364790,v1:172.21.15.80:6801/233364790] conn(0x7fe1e4039050 msgr2=0x7fe1e403b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:53.524 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:52.899+0000 7fe200598700 1 --2- 172.21.15.80:0/3536105151 >> [v2:172.21.15.80:6800/233364790,v1:172.21.15.80:6801/233364790] conn(0x7fe1e4039050 0x7fe1e403b510 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fe1e8006fd0 tx=0x7fe1e8006e40 comp rx=0 tx=0).stop 2024-04-24T23:19:53.524 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:52.899+0000 7fe200598700 1 -- 172.21.15.80:0/3536105151 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe1f8106fb0 msgr2=0x7fe1f8191c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:53.524 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:52.899+0000 7fe200598700 1 --2- 172.21.15.80:0/3536105151 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe1f8106fb0 0x7fe1f8191c40 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe1f4011670 tx=0x7fe1f4011b60 comp rx=0 tx=0).stop 2024-04-24T23:19:53.524 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:52.899+0000 7fe200598700 1 -- 172.21.15.80:0/3536105151 shutdown_connections 2024-04-24T23:19:53.524 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:52.899+0000 7fe200598700 1 --2- 172.21.15.80:0/3536105151 >> [v2:172.21.15.80:6800/233364790,v1:172.21.15.80:6801/233364790] conn(0x7fe1e4039050 0x7fe1e403b510 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:53.524 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:52.899+0000 7fe200598700 1 --2- 172.21.15.80:0/3536105151 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe1f8106fb0 0x7fe1f8191c40 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:53.524 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:52.899+0000 7fe200598700 1 -- 172.21.15.80:0/3536105151 >> 172.21.15.80:0/3536105151 conn(0x7fe1f8101810 msgr2=0x7fe1f81026c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:53.524 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:52.899+0000 7fe200598700 1 -- 172.21.15.80:0/3536105151 shutdown_connections 2024-04-24T23:19:53.524 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:52.899+0000 7fe200598700 1 -- 172.21.15.80:0/3536105151 wait complete. 2024-04-24T23:19:53.524 INFO:teuthology.orchestra.run.smithi080.stdout:Deploying unmanaged mgr service... 2024-04-24T23:19:54.153 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:53 smithi080 ceph-mon[29152]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:19:54.154 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:53 smithi080 ceph-mon[29152]: Saving service mon spec with placement count:5 2024-04-24T23:19:54.154 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:53 smithi080 ceph-mon[29152]: from='mgr.14120 172.21.15.80:0/1767820559' entity='mgr.x' 2024-04-24T23:19:55.422 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:55 smithi080 ceph-mon[29152]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:19:55.423 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:55 smithi080 ceph-mon[29152]: Saving service mgr spec with placement count:2 2024-04-24T23:19:55.423 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:55 smithi080 ceph-mon[29152]: from='mgr.14120 172.21.15.80:0/1767820559' entity='mgr.x' 2024-04-24T23:19:55.423 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:55 smithi080 ceph-mon[29152]: from='mgr.14120 172.21.15.80:0/1767820559' entity='mgr.x' 2024-04-24T23:19:55.433 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-04-24T23:19:55.434 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:53.927+0000 7f120e842700 1 Processor -- start 2024-04-24T23:19:55.434 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:53.927+0000 7f120e842700 1 -- start start 2024-04-24T23:19:55.434 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:53.927+0000 7f120e842700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1208108620 0x7f120810aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:55.434 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:53.927+0000 7f120e842700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f120810af50 con 0x7f1208108620 2024-04-24T23:19:55.434 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:53.927+0000 7f1207fff700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1208108620 0x7f120810aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:55.434 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:53.928+0000 7f1207fff700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1208108620 0x7f120810aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:41290/0 (socket says 172.21.15.80:41290) 2024-04-24T23:19:55.434 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:53.928+0000 7f1207fff700 1 -- 172.21.15.80:0/2807655873 learned_addr learned my addr 172.21.15.80:0/2807655873 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:19:55.434 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:53.928+0000 7f1207fff700 1 -- 172.21.15.80:0/2807655873 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f120810b0d0 con 0x7f1208108620 2024-04-24T23:19:55.434 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:53.928+0000 7f1207fff700 1 --2- 172.21.15.80:0/2807655873 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1208108620 0x7f120810aa10 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f11f80098a0 tx=0x7f11f8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d32919ff9232ab71 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:55.434 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:53.928+0000 7f1206ffd700 1 -- 172.21.15.80:0/2807655873 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f11f8004160 con 0x7f1208108620 2024-04-24T23:19:55.434 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:53.928+0000 7f1206ffd700 1 -- 172.21.15.80:0/2807655873 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f11f8004300 con 0x7f1208108620 2024-04-24T23:19:55.434 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:53.928+0000 7f1206ffd700 1 -- 172.21.15.80:0/2807655873 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f11f8011b50 con 0x7f1208108620 2024-04-24T23:19:55.434 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:53.929+0000 7f120e842700 1 -- 172.21.15.80:0/2807655873 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1208108620 msgr2=0x7f120810aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:55.435 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:53.929+0000 7f120e842700 1 --2- 172.21.15.80:0/2807655873 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1208108620 0x7f120810aa10 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f11f80098a0 tx=0x7f11f8009bb0 comp rx=0 tx=0).stop 2024-04-24T23:19:55.435 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:53.929+0000 7f120e842700 1 -- 172.21.15.80:0/2807655873 shutdown_connections 2024-04-24T23:19:55.435 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:53.929+0000 7f120e842700 1 --2- 172.21.15.80:0/2807655873 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1208108620 0x7f120810aa10 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:55.435 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:53.929+0000 7f120e842700 1 -- 172.21.15.80:0/2807655873 >> 172.21.15.80:0/2807655873 conn(0x7f1208102030 msgr2=0x7f1208104490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:55.435 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:53.929+0000 7f120e842700 1 -- 172.21.15.80:0/2807655873 shutdown_connections 2024-04-24T23:19:55.435 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:53.929+0000 7f120e842700 1 -- 172.21.15.80:0/2807655873 wait complete. 2024-04-24T23:19:55.435 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:53.929+0000 7f120e842700 1 Processor -- start 2024-04-24T23:19:55.435 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:53.929+0000 7f120e842700 1 -- start start 2024-04-24T23:19:55.435 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:53.930+0000 7f120e842700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1208108620 0x7f120819bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:55.435 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:53.930+0000 7f120e842700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f120810c090 con 0x7f1208108620 2024-04-24T23:19:55.435 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:53.930+0000 7f1207fff700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1208108620 0x7f120819bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:55.435 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:53.930+0000 7f1207fff700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1208108620 0x7f120819bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:41292/0 (socket says 172.21.15.80:41292) 2024-04-24T23:19:55.435 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:53.930+0000 7f1207fff700 1 -- 172.21.15.80:0/4241041985 learned_addr learned my addr 172.21.15.80:0/4241041985 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:19:55.436 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:53.930+0000 7f1207fff700 1 -- 172.21.15.80:0/4241041985 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f120819c1d0 con 0x7f1208108620 2024-04-24T23:19:55.436 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:53.930+0000 7f1207fff700 1 --2- 172.21.15.80:0/4241041985 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1208108620 0x7f120819bc90 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f11f8011670 tx=0x7f11f8011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:55.436 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:53.930+0000 7f12057fa700 1 -- 172.21.15.80:0/4241041985 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f11f80039c0 con 0x7f1208108620 2024-04-24T23:19:55.436 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:53.930+0000 7f120e842700 1 -- 172.21.15.80:0/4241041985 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f120819c460 con 0x7f1208108620 2024-04-24T23:19:55.436 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:53.930+0000 7f12057fa700 1 -- 172.21.15.80:0/4241041985 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f11f8011e50 con 0x7f1208108620 2024-04-24T23:19:55.436 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:53.930+0000 7f12057fa700 1 -- 172.21.15.80:0/4241041985 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f11f801a5c0 con 0x7f1208108620 2024-04-24T23:19:55.436 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:53.930+0000 7f120e842700 1 -- 172.21.15.80:0/4241041985 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f120819f150 con 0x7f1208108620 2024-04-24T23:19:55.436 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:53.931+0000 7f12057fa700 1 -- 172.21.15.80:0/4241041985 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f11f801a8c0 con 0x7f1208108620 2024-04-24T23:19:55.436 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:53.931+0000 7f120e842700 1 -- 172.21.15.80:0/4241041985 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1208063890 con 0x7f1208108620 2024-04-24T23:19:55.436 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:53.931+0000 7f12057fa700 1 --2- 172.21.15.80:0/4241041985 >> [v2:172.21.15.80:6800/233364790,v1:172.21.15.80:6801/233364790] conn(0x7f11f0039000 0x7f11f003b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:55.436 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:53.931+0000 7f12057fa700 1 -- 172.21.15.80:0/4241041985 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f11f804d630 con 0x7f1208108620 2024-04-24T23:19:55.436 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:53.931+0000 7f12077fe700 1 --2- 172.21.15.80:0/4241041985 >> [v2:172.21.15.80:6800/233364790,v1:172.21.15.80:6801/233364790] conn(0x7f11f0039000 0x7f11f003b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:55.436 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:53.932+0000 7f12077fe700 1 --2- 172.21.15.80:0/4241041985 >> [v2:172.21.15.80:6800/233364790,v1:172.21.15.80:6801/233364790] conn(0x7f11f0039000 0x7f11f003b4c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f11fc006fd0 tx=0x7f11fc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:55.436 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:53.936+0000 7f12057fa700 1 -- 172.21.15.80:0/4241041985 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f11f8003b60 con 0x7f1208108620 2024-04-24T23:19:55.436 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:54.159+0000 7f120e842700 1 -- 172.21.15.80:0/4241041985 --> [v2:172.21.15.80:6800/233364790,v1:172.21.15.80:6801/233364790] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f1208103310 con 0x7f11f0039000 2024-04-24T23:19:55.437 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:54.166+0000 7f12057fa700 1 -- 172.21.15.80:0/4241041985 <== mgr.14120 v2:172.21.15.80:6800/233364790 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f1208103310 con 0x7f11f0039000 2024-04-24T23:19:55.437 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:54.168+0000 7f120e842700 1 -- 172.21.15.80:0/4241041985 >> [v2:172.21.15.80:6800/233364790,v1:172.21.15.80:6801/233364790] conn(0x7f11f0039000 msgr2=0x7f11f003b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:55.437 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:54.168+0000 7f120e842700 1 --2- 172.21.15.80:0/4241041985 >> [v2:172.21.15.80:6800/233364790,v1:172.21.15.80:6801/233364790] conn(0x7f11f0039000 0x7f11f003b4c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f11fc006fd0 tx=0x7f11fc006e40 comp rx=0 tx=0).stop 2024-04-24T23:19:55.437 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:54.168+0000 7f120e842700 1 -- 172.21.15.80:0/4241041985 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1208108620 msgr2=0x7f120819bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:55.437 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:54.168+0000 7f120e842700 1 --2- 172.21.15.80:0/4241041985 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1208108620 0x7f120819bc90 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f11f8011670 tx=0x7f11f8011b60 comp rx=0 tx=0).stop 2024-04-24T23:19:55.437 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:54.168+0000 7f120e842700 1 -- 172.21.15.80:0/4241041985 shutdown_connections 2024-04-24T23:19:55.437 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:54.168+0000 7f120e842700 1 --2- 172.21.15.80:0/4241041985 >> [v2:172.21.15.80:6800/233364790,v1:172.21.15.80:6801/233364790] conn(0x7f11f0039000 0x7f11f003b4c0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:55.437 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:54.168+0000 7f120e842700 1 --2- 172.21.15.80:0/4241041985 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1208108620 0x7f120819bc90 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:55.437 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:54.168+0000 7f120e842700 1 -- 172.21.15.80:0/4241041985 >> 172.21.15.80:0/4241041985 conn(0x7f1208102030 msgr2=0x7f1208102c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:55.437 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:54.168+0000 7f120e842700 1 -- 172.21.15.80:0/4241041985 shutdown_connections 2024-04-24T23:19:55.437 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:54.168+0000 7f120e842700 1 -- 172.21.15.80:0/4241041985 wait complete. 2024-04-24T23:19:56.603 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:55.836+0000 7f8261414700 1 Processor -- start 2024-04-24T23:19:56.604 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:55.836+0000 7f8261414700 1 -- start start 2024-04-24T23:19:56.604 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:55.836+0000 7f8261414700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f825c108630 0x7f825c10aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:56.604 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:55.836+0000 7f8261414700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f825c10af60 con 0x7f825c108630 2024-04-24T23:19:56.604 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:55.836+0000 7f825affd700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f825c108630 0x7f825c10aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:56.604 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:55.836+0000 7f825affd700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f825c108630 0x7f825c10aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:41302/0 (socket says 172.21.15.80:41302) 2024-04-24T23:19:56.604 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:55.836+0000 7f825affd700 1 -- 172.21.15.80:0/1868935653 learned_addr learned my addr 172.21.15.80:0/1868935653 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:19:56.604 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:55.837+0000 7f825affd700 1 -- 172.21.15.80:0/1868935653 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f825c10b0e0 con 0x7f825c108630 2024-04-24T23:19:56.604 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:55.837+0000 7f825affd700 1 --2- 172.21.15.80:0/1868935653 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f825c108630 0x7f825c10aa20 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f82500098a0 tx=0x7f8250009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2773ba809fe45669 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:56.604 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:55.837+0000 7f8259ffb700 1 -- 172.21.15.80:0/1868935653 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8250004160 con 0x7f825c108630 2024-04-24T23:19:56.604 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:55.837+0000 7f8259ffb700 1 -- 172.21.15.80:0/1868935653 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8250004300 con 0x7f825c108630 2024-04-24T23:19:56.604 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:55.837+0000 7f8259ffb700 1 -- 172.21.15.80:0/1868935653 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8250011b20 con 0x7f825c108630 2024-04-24T23:19:56.604 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:55.838+0000 7f8261414700 1 -- 172.21.15.80:0/1868935653 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f825c108630 msgr2=0x7f825c10aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:56.605 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:55.838+0000 7f8261414700 1 --2- 172.21.15.80:0/1868935653 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f825c108630 0x7f825c10aa20 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f82500098a0 tx=0x7f8250009bb0 comp rx=0 tx=0).stop 2024-04-24T23:19:56.605 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:55.838+0000 7f8261414700 1 -- 172.21.15.80:0/1868935653 shutdown_connections 2024-04-24T23:19:56.605 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:55.838+0000 7f8261414700 1 --2- 172.21.15.80:0/1868935653 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f825c108630 0x7f825c10aa20 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:56.605 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:55.838+0000 7f8261414700 1 -- 172.21.15.80:0/1868935653 >> 172.21.15.80:0/1868935653 conn(0x7f825c102040 msgr2=0x7f825c104480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:56.605 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:55.838+0000 7f8261414700 1 -- 172.21.15.80:0/1868935653 shutdown_connections 2024-04-24T23:19:56.605 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:55.838+0000 7f8261414700 1 -- 172.21.15.80:0/1868935653 wait complete. 2024-04-24T23:19:56.605 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:55.838+0000 7f8261414700 1 Processor -- start 2024-04-24T23:19:56.605 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:55.838+0000 7f8261414700 1 -- start start 2024-04-24T23:19:56.605 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:55.839+0000 7f8261414700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f825c108630 0x7f825c107eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:56.605 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:55.839+0000 7f8261414700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f825c10c0e0 con 0x7f825c108630 2024-04-24T23:19:56.605 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:55.839+0000 7f825affd700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f825c108630 0x7f825c107eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:56.605 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:55.839+0000 7f825affd700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f825c108630 0x7f825c107eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:41308/0 (socket says 172.21.15.80:41308) 2024-04-24T23:19:56.605 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:55.839+0000 7f825affd700 1 -- 172.21.15.80:0/4252090332 learned_addr learned my addr 172.21.15.80:0/4252090332 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:19:56.605 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:55.839+0000 7f825affd700 1 -- 172.21.15.80:0/4252090332 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f825c1083f0 con 0x7f825c108630 2024-04-24T23:19:56.606 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:55.839+0000 7f825affd700 1 --2- 172.21.15.80:0/4252090332 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f825c108630 0x7f825c107eb0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f82500117e0 tx=0x7f8250011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:56.606 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:55.839+0000 7f8243fff700 1 -- 172.21.15.80:0/4252090332 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8250003ba0 con 0x7f825c108630 2024-04-24T23:19:56.606 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:55.839+0000 7f8261414700 1 -- 172.21.15.80:0/4252090332 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f825c1065e0 con 0x7f825c108630 2024-04-24T23:19:56.606 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:55.839+0000 7f8243fff700 1 -- 172.21.15.80:0/4252090332 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f825001a910 con 0x7f825c108630 2024-04-24T23:19:56.606 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:55.839+0000 7f8243fff700 1 -- 172.21.15.80:0/4252090332 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f82500248f0 con 0x7f825c108630 2024-04-24T23:19:56.606 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:55.839+0000 7f8261414700 1 -- 172.21.15.80:0/4252090332 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f825c106aa0 con 0x7f825c108630 2024-04-24T23:19:56.606 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:55.840+0000 7f8243fff700 1 -- 172.21.15.80:0/4252090332 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f8250022020 con 0x7f825c108630 2024-04-24T23:19:56.606 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:55.841+0000 7f8243fff700 1 --2- 172.21.15.80:0/4252090332 >> [v2:172.21.15.80:6800/233364790,v1:172.21.15.80:6801/233364790] conn(0x7f8244039050 0x7f824403b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:56.606 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:55.841+0000 7f8243fff700 1 -- 172.21.15.80:0/4252090332 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f825004d980 con 0x7f825c108630 2024-04-24T23:19:56.606 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:55.841+0000 7f825a7fc700 1 --2- 172.21.15.80:0/4252090332 >> [v2:172.21.15.80:6800/233364790,v1:172.21.15.80:6801/233364790] conn(0x7f8244039050 0x7f824403b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:56.606 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:55.841+0000 7f8261414700 1 -- 172.21.15.80:0/4252090332 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f825c063890 con 0x7f825c108630 2024-04-24T23:19:56.606 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:55.841+0000 7f825a7fc700 1 --2- 172.21.15.80:0/4252090332 >> [v2:172.21.15.80:6800/233364790,v1:172.21.15.80:6801/233364790] conn(0x7f8244039050 0x7f824403b510 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f824c006fd0 tx=0x7f824c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:56.606 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:55.850+0000 7f8243fff700 1 -- 172.21.15.80:0/4252090332 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f8250020c10 con 0x7f825c108630 2024-04-24T23:19:56.606 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:56.083+0000 7f8261414700 1 -- 172.21.15.80:0/4252090332 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f825c107010 con 0x7f825c108630 2024-04-24T23:19:56.607 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:56.086+0000 7f8243fff700 1 -- 172.21.15.80:0/4252090332 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f825004d2d0 con 0x7f825c108630 2024-04-24T23:19:56.607 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:56.088+0000 7f8261414700 1 -- 172.21.15.80:0/4252090332 >> [v2:172.21.15.80:6800/233364790,v1:172.21.15.80:6801/233364790] conn(0x7f8244039050 msgr2=0x7f824403b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:56.607 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:56.088+0000 7f8261414700 1 --2- 172.21.15.80:0/4252090332 >> [v2:172.21.15.80:6800/233364790,v1:172.21.15.80:6801/233364790] conn(0x7f8244039050 0x7f824403b510 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f824c006fd0 tx=0x7f824c006e40 comp rx=0 tx=0).stop 2024-04-24T23:19:56.607 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:56.088+0000 7f8261414700 1 -- 172.21.15.80:0/4252090332 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f825c108630 msgr2=0x7f825c107eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:56.607 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:56.088+0000 7f8261414700 1 --2- 172.21.15.80:0/4252090332 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f825c108630 0x7f825c107eb0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f82500117e0 tx=0x7f8250011ba0 comp rx=0 tx=0).stop 2024-04-24T23:19:56.607 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:56.088+0000 7f8261414700 1 -- 172.21.15.80:0/4252090332 shutdown_connections 2024-04-24T23:19:56.607 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:56.088+0000 7f8261414700 1 --2- 172.21.15.80:0/4252090332 >> [v2:172.21.15.80:6800/233364790,v1:172.21.15.80:6801/233364790] conn(0x7f8244039050 0x7f824403b510 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:56.607 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:56.088+0000 7f8261414700 1 --2- 172.21.15.80:0/4252090332 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f825c108630 0x7f825c107eb0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:56.607 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:56.088+0000 7f8261414700 1 -- 172.21.15.80:0/4252090332 >> 172.21.15.80:0/4252090332 conn(0x7f825c102040 msgr2=0x7f825c102ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:56.607 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:56.089+0000 7f8261414700 1 -- 172.21.15.80:0/4252090332 shutdown_connections 2024-04-24T23:19:56.607 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:56.089+0000 7f8261414700 1 -- 172.21.15.80:0/4252090332 wait complete. 2024-04-24T23:19:56.906 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:56 smithi080 ceph-mon[29152]: from='mgr.14120 172.21.15.80:0/1767820559' entity='mgr.x' 2024-04-24T23:19:56.907 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:56 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/4252090332' entity='client.admin' 2024-04-24T23:19:58.337 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:57.008+0000 7f137ce21700 1 Processor -- start 2024-04-24T23:19:58.337 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:57.008+0000 7f137ce21700 1 -- start start 2024-04-24T23:19:58.337 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:57.008+0000 7f137ce21700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f137807dde0 0x7f137807c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:58.337 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:57.008+0000 7f137ce21700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f137807c860 con 0x7f137807dde0 2024-04-24T23:19:58.337 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:57.009+0000 7f137659c700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f137807dde0 0x7f137807c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:58.337 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:57.009+0000 7f137659c700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f137807dde0 0x7f137807c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:41318/0 (socket says 172.21.15.80:41318) 2024-04-24T23:19:58.337 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:57.009+0000 7f137659c700 1 -- 172.21.15.80:0/3193202583 learned_addr learned my addr 172.21.15.80:0/3193202583 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:19:58.337 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:57.009+0000 7f137659c700 1 -- 172.21.15.80:0/3193202583 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f137807c9e0 con 0x7f137807dde0 2024-04-24T23:19:58.337 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:57.009+0000 7f137659c700 1 --2- 172.21.15.80:0/3193202583 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f137807dde0 0x7f137807c290 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f136c0098a0 tx=0x7f136c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8871dd16b50ec381 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:58.337 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:57.010+0000 7f137559a700 1 -- 172.21.15.80:0/3193202583 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f136c004160 con 0x7f137807dde0 2024-04-24T23:19:58.337 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:57.010+0000 7f137559a700 1 -- 172.21.15.80:0/3193202583 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f136c004300 con 0x7f137807dde0 2024-04-24T23:19:58.337 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:57.010+0000 7f137559a700 1 -- 172.21.15.80:0/3193202583 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f136c011b20 con 0x7f137807dde0 2024-04-24T23:19:58.338 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:57.010+0000 7f137ce21700 1 -- 172.21.15.80:0/3193202583 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f137807dde0 msgr2=0x7f137807c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:58.338 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:57.010+0000 7f137ce21700 1 --2- 172.21.15.80:0/3193202583 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f137807dde0 0x7f137807c290 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f136c0098a0 tx=0x7f136c009bb0 comp rx=0 tx=0).stop 2024-04-24T23:19:58.338 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:57.010+0000 7f137ce21700 1 -- 172.21.15.80:0/3193202583 shutdown_connections 2024-04-24T23:19:58.338 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:57.010+0000 7f137ce21700 1 --2- 172.21.15.80:0/3193202583 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f137807dde0 0x7f137807c290 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:58.338 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:57.010+0000 7f137ce21700 1 -- 172.21.15.80:0/3193202583 >> 172.21.15.80:0/3193202583 conn(0x7f13781031a0 msgr2=0x7f13781055c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:58.338 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:57.010+0000 7f137ce21700 1 -- 172.21.15.80:0/3193202583 shutdown_connections 2024-04-24T23:19:58.338 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:57.010+0000 7f137ce21700 1 -- 172.21.15.80:0/3193202583 wait complete. 2024-04-24T23:19:58.338 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:57.011+0000 7f137ce21700 1 Processor -- start 2024-04-24T23:19:58.338 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:57.011+0000 7f137ce21700 1 -- start start 2024-04-24T23:19:58.338 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:57.011+0000 7f137ce21700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f137807dde0 0x7f137819dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:58.338 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:57.011+0000 7f137ce21700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f137810e1c0 con 0x7f137807dde0 2024-04-24T23:19:58.338 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:57.011+0000 7f137659c700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f137807dde0 0x7f137819dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:58.338 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:57.011+0000 7f137659c700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f137807dde0 0x7f137819dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:41334/0 (socket says 172.21.15.80:41334) 2024-04-24T23:19:58.338 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:57.011+0000 7f137659c700 1 -- 172.21.15.80:0/2636352933 learned_addr learned my addr 172.21.15.80:0/2636352933 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:19:58.339 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:57.012+0000 7f137659c700 1 -- 172.21.15.80:0/2636352933 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f137819e200 con 0x7f137807dde0 2024-04-24T23:19:58.339 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:57.012+0000 7f137659c700 1 --2- 172.21.15.80:0/2636352933 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f137807dde0 0x7f137819dcc0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f136c0117e0 tx=0x7f136c011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:58.339 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:57.012+0000 7f13677fe700 1 -- 172.21.15.80:0/2636352933 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f136c003bf0 con 0x7f137807dde0 2024-04-24T23:19:58.339 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:57.012+0000 7f13677fe700 1 -- 172.21.15.80:0/2636352933 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f136c01a910 con 0x7f137807dde0 2024-04-24T23:19:58.339 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:57.012+0000 7f137ce21700 1 -- 172.21.15.80:0/2636352933 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f137819e490 con 0x7f137807dde0 2024-04-24T23:19:58.339 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:57.012+0000 7f13677fe700 1 -- 172.21.15.80:0/2636352933 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f136c024970 con 0x7f137807dde0 2024-04-24T23:19:58.339 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:57.012+0000 7f137ce21700 1 -- 172.21.15.80:0/2636352933 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f137819e8f0 con 0x7f137807dde0 2024-04-24T23:19:58.339 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:57.013+0000 7f13677fe700 1 -- 172.21.15.80:0/2636352933 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f136c022020 con 0x7f137807dde0 2024-04-24T23:19:58.339 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:57.013+0000 7f13677fe700 1 --2- 172.21.15.80:0/2636352933 >> [v2:172.21.15.80:6800/233364790,v1:172.21.15.80:6801/233364790] conn(0x7f1360039050 0x7f136003b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:58.339 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:57.013+0000 7f137ce21700 1 -- 172.21.15.80:0/2636352933 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1378063890 con 0x7f137807dde0 2024-04-24T23:19:58.339 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:57.013+0000 7f13677fe700 1 -- 172.21.15.80:0/2636352933 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f136c04d380 con 0x7f137807dde0 2024-04-24T23:19:58.339 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:57.013+0000 7f1375d9b700 1 --2- 172.21.15.80:0/2636352933 >> [v2:172.21.15.80:6800/233364790,v1:172.21.15.80:6801/233364790] conn(0x7f1360039050 0x7f136003b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:58.339 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:57.014+0000 7f1375d9b700 1 --2- 172.21.15.80:0/2636352933 >> [v2:172.21.15.80:6800/233364790,v1:172.21.15.80:6801/233364790] conn(0x7f1360039050 0x7f136003b510 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f1368006fd0 tx=0x7f1368006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:58.339 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:57.018+0000 7f13677fe700 1 -- 172.21.15.80:0/2636352933 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f136c011e20 con 0x7f137807dde0 2024-04-24T23:19:58.340 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:57.238+0000 7f137ce21700 1 -- 172.21.15.80:0/2636352933 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f13781a1360 con 0x7f137807dde0 2024-04-24T23:19:58.340 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:57.241+0000 7f13677fe700 1 -- 172.21.15.80:0/2636352933 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f136c00ba00 con 0x7f137807dde0 2024-04-24T23:19:58.340 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:57.243+0000 7f137ce21700 1 -- 172.21.15.80:0/2636352933 >> [v2:172.21.15.80:6800/233364790,v1:172.21.15.80:6801/233364790] conn(0x7f1360039050 msgr2=0x7f136003b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:58.340 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:57.243+0000 7f137ce21700 1 --2- 172.21.15.80:0/2636352933 >> [v2:172.21.15.80:6800/233364790,v1:172.21.15.80:6801/233364790] conn(0x7f1360039050 0x7f136003b510 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f1368006fd0 tx=0x7f1368006e40 comp rx=0 tx=0).stop 2024-04-24T23:19:58.340 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:57.243+0000 7f137ce21700 1 -- 172.21.15.80:0/2636352933 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f137807dde0 msgr2=0x7f137819dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:58.340 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:57.243+0000 7f137ce21700 1 --2- 172.21.15.80:0/2636352933 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f137807dde0 0x7f137819dcc0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f136c0117e0 tx=0x7f136c011c10 comp rx=0 tx=0).stop 2024-04-24T23:19:58.340 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:57.243+0000 7f137ce21700 1 -- 172.21.15.80:0/2636352933 shutdown_connections 2024-04-24T23:19:58.340 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:57.243+0000 7f137ce21700 1 --2- 172.21.15.80:0/2636352933 >> [v2:172.21.15.80:6800/233364790,v1:172.21.15.80:6801/233364790] conn(0x7f1360039050 0x7f136003b510 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:58.340 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:57.243+0000 7f137ce21700 1 --2- 172.21.15.80:0/2636352933 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f137807dde0 0x7f137819dcc0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:58.340 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:57.243+0000 7f137ce21700 1 -- 172.21.15.80:0/2636352933 >> 172.21.15.80:0/2636352933 conn(0x7f13781031a0 msgr2=0x7f1378103c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:58.340 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:57.243+0000 7f137ce21700 1 -- 172.21.15.80:0/2636352933 shutdown_connections 2024-04-24T23:19:58.340 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:57.243+0000 7f137ce21700 1 -- 172.21.15.80:0/2636352933 wait complete. 2024-04-24T23:19:58.340 INFO:teuthology.orchestra.run.smithi080.stdout:Enabling the dashboard module... 2024-04-24T23:19:58.355 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:58 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/2636352933' entity='client.admin' 2024-04-24T23:19:58.356 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:19:58 smithi080 ceph-mon[29152]: from='mgr.14120 172.21.15.80:0/1767820559' entity='mgr.x' 2024-04-24T23:20:00.208 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:00 smithi080 ceph-mon[29152]: from='mgr.14120 172.21.15.80:0/1767820559' entity='mgr.x' 2024-04-24T23:20:00.208 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:00 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/4178028968' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-04-24T23:20:00.208 INFO:journalctl@ceph.mgr.x.smithi080.stdout:Apr 24 23:20:00 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-x[29381]: ignoring --setuser ceph since I am not root 2024-04-24T23:20:00.208 INFO:journalctl@ceph.mgr.x.smithi080.stdout:Apr 24 23:20:00 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-x[29381]: ignoring --setgroup ceph since I am not root 2024-04-24T23:20:00.522 INFO:journalctl@ceph.mgr.x.smithi080.stdout:Apr 24 23:20:00 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-x[29381]: 2024-04-24T23:20:00.238+0000 7f5db40a6200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-24T23:20:00.522 INFO:journalctl@ceph.mgr.x.smithi080.stdout:Apr 24 23:20:00 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-x[29381]: 2024-04-24T23:20:00.419+0000 7f5db40a6200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-24T23:20:00.523 INFO:journalctl@ceph.mgr.x.smithi080.stdout:Apr 24 23:20:00 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-x[29381]: 2024-04-24T23:20:00.520+0000 7f5db40a6200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-24T23:20:01.101 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:01 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/4178028968' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-04-24T23:20:01.101 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:01 smithi080 ceph-mon[29152]: mgrmap e9: x(active, since 19s) 2024-04-24T23:20:01.101 INFO:journalctl@ceph.mgr.x.smithi080.stdout:Apr 24 23:20:00 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-x[29381]: 2024-04-24T23:20:00.829+0000 7f5db40a6200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-24T23:20:01.101 INFO:journalctl@ceph.mgr.x.smithi080.stdout:Apr 24 23:20:00 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-x[29381]: 2024-04-24T23:20:00.913+0000 7f5db40a6200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-24T23:20:01.109 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:58.877+0000 7f656bad1700 1 Processor -- start 2024-04-24T23:20:01.109 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:58.877+0000 7f656bad1700 1 -- start start 2024-04-24T23:20:01.109 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:58.877+0000 7f656bad1700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f65641086a0 0x7f656410aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:01.109 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:58.877+0000 7f656bad1700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f656407bee0 con 0x7f65641086a0 2024-04-24T23:20:01.109 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:58.878+0000 7f656986d700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f65641086a0 0x7f656410aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:01.109 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:58.878+0000 7f656986d700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f65641086a0 0x7f656410aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:41338/0 (socket says 172.21.15.80:41338) 2024-04-24T23:20:01.110 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:58.878+0000 7f656986d700 1 -- 172.21.15.80:0/885067182 learned_addr learned my addr 172.21.15.80:0/885067182 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:20:01.110 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:58.878+0000 7f656986d700 1 -- 172.21.15.80:0/885067182 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f656410afd0 con 0x7f65641086a0 2024-04-24T23:20:01.110 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:58.878+0000 7f656986d700 1 --2- 172.21.15.80:0/885067182 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f65641086a0 0x7f656410aa90 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f65540098a0 tx=0x7f6554009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16cadca0e58a8292 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:01.110 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:58.878+0000 7f656886b700 1 -- 172.21.15.80:0/885067182 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6554004160 con 0x7f65641086a0 2024-04-24T23:20:01.110 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:58.878+0000 7f656886b700 1 -- 172.21.15.80:0/885067182 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6554004300 con 0x7f65641086a0 2024-04-24T23:20:01.110 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:58.878+0000 7f656886b700 1 -- 172.21.15.80:0/885067182 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6554011b20 con 0x7f65641086a0 2024-04-24T23:20:01.110 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:58.879+0000 7f656bad1700 1 -- 172.21.15.80:0/885067182 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f65641086a0 msgr2=0x7f656410aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:01.110 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:58.879+0000 7f656bad1700 1 --2- 172.21.15.80:0/885067182 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f65641086a0 0x7f656410aa90 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f65540098a0 tx=0x7f6554009bb0 comp rx=0 tx=0).stop 2024-04-24T23:20:01.110 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:58.879+0000 7f656bad1700 1 -- 172.21.15.80:0/885067182 shutdown_connections 2024-04-24T23:20:01.110 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:58.879+0000 7f656bad1700 1 --2- 172.21.15.80:0/885067182 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f65641086a0 0x7f656410aa90 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:01.110 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:58.879+0000 7f656bad1700 1 -- 172.21.15.80:0/885067182 >> 172.21.15.80:0/885067182 conn(0x7f65641020b0 msgr2=0x7f65641044d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:01.110 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:58.879+0000 7f656bad1700 1 -- 172.21.15.80:0/885067182 shutdown_connections 2024-04-24T23:20:01.110 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:58.879+0000 7f656bad1700 1 -- 172.21.15.80:0/885067182 wait complete. 2024-04-24T23:20:01.111 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:58.880+0000 7f656bad1700 1 Processor -- start 2024-04-24T23:20:01.111 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:58.880+0000 7f656bad1700 1 -- start start 2024-04-24T23:20:01.111 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:58.880+0000 7f656bad1700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f65641086a0 0x7f656419bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:01.111 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:58.880+0000 7f656bad1700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f656410c0f0 con 0x7f65641086a0 2024-04-24T23:20:01.111 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:58.880+0000 7f656986d700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f65641086a0 0x7f656419bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:01.111 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:58.880+0000 7f656986d700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f65641086a0 0x7f656419bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:41350/0 (socket says 172.21.15.80:41350) 2024-04-24T23:20:01.111 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:58.880+0000 7f656986d700 1 -- 172.21.15.80:0/4178028968 learned_addr learned my addr 172.21.15.80:0/4178028968 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:20:01.111 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:58.880+0000 7f656986d700 1 -- 172.21.15.80:0/4178028968 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f656419c190 con 0x7f65641086a0 2024-04-24T23:20:01.111 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:58.881+0000 7f656986d700 1 --2- 172.21.15.80:0/4178028968 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f65641086a0 0x7f656419bc50 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f65540116a0 tx=0x7f6554011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:01.111 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:58.881+0000 7f655affd700 1 -- 172.21.15.80:0/4178028968 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6554003a40 con 0x7f65641086a0 2024-04-24T23:20:01.111 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:58.881+0000 7f656bad1700 1 -- 172.21.15.80:0/4178028968 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f656419c420 con 0x7f65641086a0 2024-04-24T23:20:01.111 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:58.881+0000 7f656bad1700 1 -- 172.21.15.80:0/4178028968 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f656419c880 con 0x7f65641086a0 2024-04-24T23:20:01.111 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:58.881+0000 7f655affd700 1 -- 172.21.15.80:0/4178028968 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6554003be0 con 0x7f65641086a0 2024-04-24T23:20:01.111 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:58.881+0000 7f655affd700 1 -- 172.21.15.80:0/4178028968 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f655401a750 con 0x7f65641086a0 2024-04-24T23:20:01.112 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:58.882+0000 7f655affd700 1 -- 172.21.15.80:0/4178028968 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f655401aa80 con 0x7f65641086a0 2024-04-24T23:20:01.112 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:58.882+0000 7f656bad1700 1 -- 172.21.15.80:0/4178028968 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6548005360 con 0x7f65641086a0 2024-04-24T23:20:01.112 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:58.882+0000 7f655affd700 1 --2- 172.21.15.80:0/4178028968 >> [v2:172.21.15.80:6800/233364790,v1:172.21.15.80:6801/233364790] conn(0x7f6550039050 0x7f655003b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:01.112 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:58.882+0000 7f655affd700 1 -- 172.21.15.80:0/4178028968 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f655404d600 con 0x7f65641086a0 2024-04-24T23:20:01.112 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:58.882+0000 7f656906c700 1 --2- 172.21.15.80:0/4178028968 >> [v2:172.21.15.80:6800/233364790,v1:172.21.15.80:6801/233364790] conn(0x7f6550039050 0x7f655003b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:01.112 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:58.882+0000 7f656906c700 1 --2- 172.21.15.80:0/4178028968 >> [v2:172.21.15.80:6800/233364790,v1:172.21.15.80:6801/233364790] conn(0x7f6550039050 0x7f655003b510 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f6560006fd0 tx=0x7f6560006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:01.112 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:58.887+0000 7f655affd700 1 -- 172.21.15.80:0/4178028968 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f6554011e20 con 0x7f65641086a0 2024-04-24T23:20:01.112 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:19:59.156+0000 7f656bad1700 1 -- 172.21.15.80:0/4178028968 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f6548005650 con 0x7f65641086a0 2024-04-24T23:20:01.112 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:00.060+0000 7f655affd700 1 -- 172.21.15.80:0/4178028968 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f6554019c70 con 0x7f65641086a0 2024-04-24T23:20:01.112 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:00.062+0000 7f655affd700 1 -- 172.21.15.80:0/4178028968 <== mon.0 v2:172.21.15.80:3300/0 8 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f655402d3f0 con 0x7f65641086a0 2024-04-24T23:20:01.112 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:00.063+0000 7f656bad1700 1 -- 172.21.15.80:0/4178028968 >> [v2:172.21.15.80:6800/233364790,v1:172.21.15.80:6801/233364790] conn(0x7f6550039050 msgr2=0x7f655003b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:01.112 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:00.063+0000 7f656bad1700 1 --2- 172.21.15.80:0/4178028968 >> [v2:172.21.15.80:6800/233364790,v1:172.21.15.80:6801/233364790] conn(0x7f6550039050 0x7f655003b510 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f6560006fd0 tx=0x7f6560006e40 comp rx=0 tx=0).stop 2024-04-24T23:20:01.113 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:00.063+0000 7f656bad1700 1 -- 172.21.15.80:0/4178028968 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f65641086a0 msgr2=0x7f656419bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:01.113 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:00.063+0000 7f656bad1700 1 --2- 172.21.15.80:0/4178028968 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f65641086a0 0x7f656419bc50 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f65540116a0 tx=0x7f6554011ba0 comp rx=0 tx=0).stop 2024-04-24T23:20:01.113 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:00.063+0000 7f656bad1700 1 -- 172.21.15.80:0/4178028968 shutdown_connections 2024-04-24T23:20:01.113 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:00.063+0000 7f656bad1700 1 --2- 172.21.15.80:0/4178028968 >> [v2:172.21.15.80:6800/233364790,v1:172.21.15.80:6801/233364790] conn(0x7f6550039050 0x7f655003b510 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:01.113 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:00.063+0000 7f656bad1700 1 --2- 172.21.15.80:0/4178028968 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f65641086a0 0x7f656419bc50 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:01.113 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:00.063+0000 7f656bad1700 1 -- 172.21.15.80:0/4178028968 >> 172.21.15.80:0/4178028968 conn(0x7f65641020b0 msgr2=0x7f6564102be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:01.113 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:00.063+0000 7f656bad1700 1 -- 172.21.15.80:0/4178028968 shutdown_connections 2024-04-24T23:20:01.113 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:00.063+0000 7f656bad1700 1 -- 172.21.15.80:0/4178028968 wait complete. 2024-04-24T23:20:02.400 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:02 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/1964661900' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-24T23:20:02.400 INFO:journalctl@ceph.mgr.x.smithi080.stdout:Apr 24 23:20:02 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-x[29381]: 2024-04-24T23:20:02.111+0000 7f5db40a6200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-24T23:20:02.401 INFO:journalctl@ceph.mgr.x.smithi080.stdout:Apr 24 23:20:02 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-x[29381]: 2024-04-24T23:20:02.282+0000 7f5db40a6200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-24T23:20:02.409 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout { 2024-04-24T23:20:02.409 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-04-24T23:20:02.409 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-24T23:20:02.410 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-24T23:20:02.410 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-24T23:20:02.410 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout } 2024-04-24T23:20:02.410 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:01.534+0000 7fdea5c0a700 1 Processor -- start 2024-04-24T23:20:02.410 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:01.534+0000 7fdea5c0a700 1 -- start start 2024-04-24T23:20:02.410 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:01.534+0000 7fdea5c0a700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdea0107c10 0x7fdea0108030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:02.410 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:01.534+0000 7fdea5c0a700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fdea0108600 con 0x7fdea0107c10 2024-04-24T23:20:02.410 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:01.534+0000 7fde9f7fe700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdea0107c10 0x7fdea0108030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:02.410 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:01.534+0000 7fde9f7fe700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdea0107c10 0x7fdea0108030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:41380/0 (socket says 172.21.15.80:41380) 2024-04-24T23:20:02.410 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:01.534+0000 7fde9f7fe700 1 -- 172.21.15.80:0/1140905372 learned_addr learned my addr 172.21.15.80:0/1140905372 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:20:02.410 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:01.534+0000 7fde9f7fe700 1 -- 172.21.15.80:0/1140905372 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdea0108dc0 con 0x7fdea0107c10 2024-04-24T23:20:02.411 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:01.535+0000 7fde9f7fe700 1 --2- 172.21.15.80:0/1140905372 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdea0107c10 0x7fdea0108030 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fde900098a0 tx=0x7fde90009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19d1d14554bd6aba server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:02.411 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:01.535+0000 7fde9e7fc700 1 -- 172.21.15.80:0/1140905372 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fde90004160 con 0x7fdea0107c10 2024-04-24T23:20:02.411 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:01.535+0000 7fde9e7fc700 1 -- 172.21.15.80:0/1140905372 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde90004300 con 0x7fdea0107c10 2024-04-24T23:20:02.411 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:01.535+0000 7fde9e7fc700 1 -- 172.21.15.80:0/1140905372 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fde90011b20 con 0x7fdea0107c10 2024-04-24T23:20:02.411 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:01.535+0000 7fdea5c0a700 1 -- 172.21.15.80:0/1140905372 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdea0107c10 msgr2=0x7fdea0108030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:02.411 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:01.535+0000 7fdea5c0a700 1 --2- 172.21.15.80:0/1140905372 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdea0107c10 0x7fdea0108030 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fde900098a0 tx=0x7fde90009bb0 comp rx=0 tx=0).stop 2024-04-24T23:20:02.411 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:01.535+0000 7fdea5c0a700 1 -- 172.21.15.80:0/1140905372 shutdown_connections 2024-04-24T23:20:02.411 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:01.535+0000 7fdea5c0a700 1 --2- 172.21.15.80:0/1140905372 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdea0107c10 0x7fdea0108030 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:02.411 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:01.535+0000 7fdea5c0a700 1 -- 172.21.15.80:0/1140905372 >> 172.21.15.80:0/1140905372 conn(0x7fdea01031d0 msgr2=0x7fdea01055f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:02.411 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:01.536+0000 7fdea5c0a700 1 -- 172.21.15.80:0/1140905372 shutdown_connections 2024-04-24T23:20:02.411 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:01.536+0000 7fdea5c0a700 1 -- 172.21.15.80:0/1140905372 wait complete. 2024-04-24T23:20:02.412 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:01.536+0000 7fdea5c0a700 1 Processor -- start 2024-04-24T23:20:02.412 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:01.536+0000 7fdea5c0a700 1 -- start start 2024-04-24T23:20:02.412 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:01.536+0000 7fdea5c0a700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdea0107c10 0x7fdea019bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:02.412 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:01.536+0000 7fdea5c0a700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fdea010c170 con 0x7fdea0107c10 2024-04-24T23:20:02.412 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:01.537+0000 7fde9f7fe700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdea0107c10 0x7fdea019bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:02.412 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:01.537+0000 7fde9f7fe700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdea0107c10 0x7fdea019bb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:41384/0 (socket says 172.21.15.80:41384) 2024-04-24T23:20:02.412 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:01.537+0000 7fde9f7fe700 1 -- 172.21.15.80:0/1964661900 learned_addr learned my addr 172.21.15.80:0/1964661900 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:20:02.412 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:01.537+0000 7fde9f7fe700 1 -- 172.21.15.80:0/1964661900 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdea019c0b0 con 0x7fdea0107c10 2024-04-24T23:20:02.412 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:01.537+0000 7fde9f7fe700 1 --2- 172.21.15.80:0/1964661900 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdea0107c10 0x7fdea019bb70 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fde900116a0 tx=0x7fde90011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:02.412 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:01.537+0000 7fde9cff9700 1 -- 172.21.15.80:0/1964661900 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fde90003a40 con 0x7fdea0107c10 2024-04-24T23:20:02.412 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:01.537+0000 7fdea5c0a700 1 -- 172.21.15.80:0/1964661900 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdea019c340 con 0x7fdea0107c10 2024-04-24T23:20:02.412 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:01.537+0000 7fde9cff9700 1 -- 172.21.15.80:0/1964661900 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde90003be0 con 0x7fdea0107c10 2024-04-24T23:20:02.413 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:01.537+0000 7fde9cff9700 1 -- 172.21.15.80:0/1964661900 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fde9001a720 con 0x7fdea0107c10 2024-04-24T23:20:02.413 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:01.537+0000 7fdea5c0a700 1 -- 172.21.15.80:0/1964661900 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdea019c7a0 con 0x7fdea0107c10 2024-04-24T23:20:02.413 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:01.538+0000 7fde9cff9700 1 -- 172.21.15.80:0/1964661900 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7fde9001aa20 con 0x7fdea0107c10 2024-04-24T23:20:02.413 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:01.538+0000 7fde9cff9700 1 --2- 172.21.15.80:0/1964661900 >> [v2:172.21.15.80:6800/233364790,v1:172.21.15.80:6801/233364790] conn(0x7fde8c039050 0x7fde8c03b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:02.413 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:01.538+0000 7fde9cff9700 1 -- 172.21.15.80:0/1964661900 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fde9004d800 con 0x7fdea0107c10 2024-04-24T23:20:02.413 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:01.538+0000 7fde9effd700 1 -- 172.21.15.80:0/1964661900 >> [v2:172.21.15.80:6800/233364790,v1:172.21.15.80:6801/233364790] conn(0x7fde8c039050 msgr2=0x7fde8c03b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.80:6800/233364790 2024-04-24T23:20:02.413 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:01.538+0000 7fde9effd700 1 --2- 172.21.15.80:0/1964661900 >> [v2:172.21.15.80:6800/233364790,v1:172.21.15.80:6801/233364790] conn(0x7fde8c039050 0x7fde8c03b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T23:20:02.413 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:01.539+0000 7fdea5c0a700 1 -- 172.21.15.80:0/1964661900 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdea0063890 con 0x7fdea0107c10 2024-04-24T23:20:02.413 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:01.548+0000 7fde9cff9700 1 -- 172.21.15.80:0/1964661900 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fde90011e20 con 0x7fdea0107c10 2024-04-24T23:20:02.413 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:01.739+0000 7fde9effd700 1 -- 172.21.15.80:0/1964661900 >> [v2:172.21.15.80:6800/233364790,v1:172.21.15.80:6801/233364790] conn(0x7fde8c039050 msgr2=0x7fde8c03b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.80:6800/233364790 2024-04-24T23:20:02.413 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:01.739+0000 7fde9effd700 1 --2- 172.21.15.80:0/1964661900 >> [v2:172.21.15.80:6800/233364790,v1:172.21.15.80:6801/233364790] conn(0x7fde8c039050 0x7fde8c03b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-24T23:20:02.413 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:01.802+0000 7fdea5c0a700 1 -- 172.21.15.80:0/1964661900 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fdea019f0d0 con 0x7fdea0107c10 2024-04-24T23:20:02.414 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:01.802+0000 7fde9cff9700 1 -- 172.21.15.80:0/1964661900 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7fde9004d130 con 0x7fdea0107c10 2024-04-24T23:20:02.414 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:01.805+0000 7fdea5c0a700 1 -- 172.21.15.80:0/1964661900 >> [v2:172.21.15.80:6800/233364790,v1:172.21.15.80:6801/233364790] conn(0x7fde8c039050 msgr2=0x7fde8c03b510 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T23:20:02.414 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:01.805+0000 7fdea5c0a700 1 --2- 172.21.15.80:0/1964661900 >> [v2:172.21.15.80:6800/233364790,v1:172.21.15.80:6801/233364790] conn(0x7fde8c039050 0x7fde8c03b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:02.414 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:01.805+0000 7fdea5c0a700 1 -- 172.21.15.80:0/1964661900 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdea0107c10 msgr2=0x7fdea019bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:02.414 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:01.805+0000 7fdea5c0a700 1 --2- 172.21.15.80:0/1964661900 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdea0107c10 0x7fdea019bb70 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fde900116a0 tx=0x7fde90011ba0 comp rx=0 tx=0).stop 2024-04-24T23:20:02.414 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:01.805+0000 7fdea5c0a700 1 -- 172.21.15.80:0/1964661900 shutdown_connections 2024-04-24T23:20:02.414 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:01.805+0000 7fdea5c0a700 1 --2- 172.21.15.80:0/1964661900 >> [v2:172.21.15.80:6800/233364790,v1:172.21.15.80:6801/233364790] conn(0x7fde8c039050 0x7fde8c03b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:02.414 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:01.805+0000 7fdea5c0a700 1 --2- 172.21.15.80:0/1964661900 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdea0107c10 0x7fdea019bb70 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:02.414 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:01.805+0000 7fdea5c0a700 1 -- 172.21.15.80:0/1964661900 >> 172.21.15.80:0/1964661900 conn(0x7fdea01031d0 msgr2=0x7fdea0103cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:02.414 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:01.805+0000 7fdea5c0a700 1 -- 172.21.15.80:0/1964661900 shutdown_connections 2024-04-24T23:20:02.414 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:01.805+0000 7fdea5c0a700 1 -- 172.21.15.80:0/1964661900 wait complete. 2024-04-24T23:20:02.414 INFO:teuthology.orchestra.run.smithi080.stdout:Waiting for the mgr to restart... 2024-04-24T23:20:02.414 INFO:teuthology.orchestra.run.smithi080.stdout:Waiting for mgr epoch 9... 2024-04-24T23:20:02.744 INFO:journalctl@ceph.mgr.x.smithi080.stdout:Apr 24 23:20:02 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-x[29381]: 2024-04-24T23:20:02.456+0000 7f5db40a6200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-24T23:20:03.039 INFO:journalctl@ceph.mgr.x.smithi080.stdout:Apr 24 23:20:02 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-x[29381]: 2024-04-24T23:20:02.747+0000 7f5db40a6200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-24T23:20:03.039 INFO:journalctl@ceph.mgr.x.smithi080.stdout:Apr 24 23:20:02 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-x[29381]: 2024-04-24T23:20:02.843+0000 7f5db40a6200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-24T23:20:03.451 INFO:journalctl@ceph.mgr.x.smithi080.stdout:Apr 24 23:20:03 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-x[29381]: 2024-04-24T23:20:03.043+0000 7f5db40a6200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-24T23:20:03.451 INFO:journalctl@ceph.mgr.x.smithi080.stdout:Apr 24 23:20:03 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-x[29381]: 2024-04-24T23:20:03.234+0000 7f5db40a6200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-24T23:20:04.232 INFO:journalctl@ceph.mgr.x.smithi080.stdout:Apr 24 23:20:03 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-x[29381]: 2024-04-24T23:20:03.818+0000 7f5db40a6200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-24T23:20:04.232 INFO:journalctl@ceph.mgr.x.smithi080.stdout:Apr 24 23:20:04 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-x[29381]: 2024-04-24T23:20:04.230+0000 7f5db40a6200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-24T23:20:04.524 INFO:journalctl@ceph.mgr.x.smithi080.stdout:Apr 24 23:20:04 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-x[29381]: 2024-04-24T23:20:04.325+0000 7f5db40a6200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-24T23:20:04.524 INFO:journalctl@ceph.mgr.x.smithi080.stdout:Apr 24 23:20:04 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-x[29381]: 2024-04-24T23:20:04.522+0000 7f5db40a6200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-24T23:20:05.289 INFO:journalctl@ceph.mgr.x.smithi080.stdout:Apr 24 23:20:05 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-x[29381]: 2024-04-24T23:20:05.032+0000 7f5db40a6200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-24T23:20:05.289 INFO:journalctl@ceph.mgr.x.smithi080.stdout:Apr 24 23:20:05 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-x[29381]: 2024-04-24T23:20:05.122+0000 7f5db40a6200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-24T23:20:05.697 INFO:journalctl@ceph.mgr.x.smithi080.stdout:Apr 24 23:20:05 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-x[29381]: 2024-04-24T23:20:05.359+0000 7f5db40a6200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-24T23:20:05.697 INFO:journalctl@ceph.mgr.x.smithi080.stdout:Apr 24 23:20:05 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-x[29381]: 2024-04-24T23:20:05.489+0000 7f5db40a6200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-24T23:20:05.697 INFO:journalctl@ceph.mgr.x.smithi080.stdout:Apr 24 23:20:05 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-x[29381]: 2024-04-24T23:20:05.582+0000 7f5db40a6200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-24T23:20:05.697 INFO:journalctl@ceph.mgr.x.smithi080.stdout:Apr 24 23:20:05 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-x[29381]: 2024-04-24T23:20:05.694+0000 7f5db40a6200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-24T23:20:06.039 INFO:journalctl@ceph.mgr.x.smithi080.stdout:Apr 24 23:20:05 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-x[29381]: 2024-04-24T23:20:05.784+0000 7f5db40a6200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-24T23:20:06.846 INFO:journalctl@ceph.mgr.x.smithi080.stdout:Apr 24 23:20:06 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-x[29381]: 2024-04-24T23:20:06.578+0000 7f5db40a6200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-24T23:20:06.846 INFO:journalctl@ceph.mgr.x.smithi080.stdout:Apr 24 23:20:06 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-x[29381]: 2024-04-24T23:20:06.791+0000 7f5db40a6200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-24T23:20:07.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:06 smithi080 ceph-mon[29152]: Active manager daemon x restarted 2024-04-24T23:20:07.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:06 smithi080 ceph-mon[29152]: Activating manager daemon x 2024-04-24T23:20:07.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:06 smithi080 ceph-mon[29152]: osdmap e3: 0 total, 0 up, 0 in 2024-04-24T23:20:07.290 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:06 smithi080 ceph-mon[29152]: mgrmap e10: x(active, starting, since 0.00378326s) 2024-04-24T23:20:07.290 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:06 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T23:20:07.290 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:06 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-24T23:20:07.290 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:06 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-24T23:20:07.290 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:06 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-24T23:20:07.290 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:06 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-24T23:20:07.290 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:06 smithi080 ceph-mon[29152]: Manager daemon x is now available 2024-04-24T23:20:07.290 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:06 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:20:07.290 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:06 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-24T23:20:08.037 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:07 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-24T23:20:08.037 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:07 smithi080 ceph-mon[29152]: [24/Apr/2024:23:20:07] ENGINE Bus STARTING 2024-04-24T23:20:08.038 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:07 smithi080 ceph-mon[29152]: [24/Apr/2024:23:20:07] ENGINE Serving on http://172.21.15.80:8765 2024-04-24T23:20:08.038 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:07 smithi080 ceph-mon[29152]: mgrmap e11: x(active, since 1.00626s) 2024-04-24T23:20:08.399 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout { 2024-04-24T23:20:08.400 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-04-24T23:20:08.400 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-24T23:20:08.400 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout } 2024-04-24T23:20:08.400 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:02.842+0000 7ff78c70f700 1 Processor -- start 2024-04-24T23:20:08.400 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:02.842+0000 7ff78c70f700 1 -- start start 2024-04-24T23:20:08.400 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:02.842+0000 7ff78c70f700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff7841064d0 0x7ff7841068f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:08.400 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:02.842+0000 7ff78c70f700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7ff78407bea0 con 0x7ff7841064d0 2024-04-24T23:20:08.400 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:02.842+0000 7ff78a4ab700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff7841064d0 0x7ff7841068f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:08.400 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:02.842+0000 7ff78a4ab700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff7841064d0 0x7ff7841068f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:45990/0 (socket says 172.21.15.80:45990) 2024-04-24T23:20:08.400 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:02.842+0000 7ff78a4ab700 1 -- 172.21.15.80:0/2647328547 learned_addr learned my addr 172.21.15.80:0/2647328547 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:20:08.400 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:02.843+0000 7ff78a4ab700 1 -- 172.21.15.80:0/2647328547 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff784106e30 con 0x7ff7841064d0 2024-04-24T23:20:08.400 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:02.843+0000 7ff78a4ab700 1 --2- 172.21.15.80:0/2647328547 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff7841064d0 0x7ff7841068f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff7740098a0 tx=0x7ff774009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e852ab8c75b23c8e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:08.400 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:02.843+0000 7ff7894a9700 1 -- 172.21.15.80:0/2647328547 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff774004160 con 0x7ff7841064d0 2024-04-24T23:20:08.400 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:02.843+0000 7ff7894a9700 1 -- 172.21.15.80:0/2647328547 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff774004300 con 0x7ff7841064d0 2024-04-24T23:20:08.401 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:02.843+0000 7ff7894a9700 1 -- 172.21.15.80:0/2647328547 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff774011b20 con 0x7ff7841064d0 2024-04-24T23:20:08.401 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:02.844+0000 7ff78c70f700 1 -- 172.21.15.80:0/2647328547 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff7841064d0 msgr2=0x7ff7841068f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:08.401 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:02.844+0000 7ff78c70f700 1 --2- 172.21.15.80:0/2647328547 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff7841064d0 0x7ff7841068f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff7740098a0 tx=0x7ff774009bb0 comp rx=0 tx=0).stop 2024-04-24T23:20:08.401 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:02.844+0000 7ff78c70f700 1 -- 172.21.15.80:0/2647328547 shutdown_connections 2024-04-24T23:20:08.401 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:02.844+0000 7ff78c70f700 1 --2- 172.21.15.80:0/2647328547 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff7841064d0 0x7ff7841068f0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:08.401 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:02.844+0000 7ff78c70f700 1 -- 172.21.15.80:0/2647328547 >> 172.21.15.80:0/2647328547 conn(0x7ff7841020b0 msgr2=0x7ff7841044d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:08.401 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:02.844+0000 7ff78c70f700 1 -- 172.21.15.80:0/2647328547 shutdown_connections 2024-04-24T23:20:08.401 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:02.844+0000 7ff78c70f700 1 -- 172.21.15.80:0/2647328547 wait complete. 2024-04-24T23:20:08.401 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:02.844+0000 7ff78c70f700 1 Processor -- start 2024-04-24T23:20:08.401 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:02.844+0000 7ff78c70f700 1 -- start start 2024-04-24T23:20:08.401 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:02.845+0000 7ff78c70f700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff7841064d0 0x7ff78419de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:08.401 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:02.845+0000 7ff78c70f700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7ff784107f00 con 0x7ff7841064d0 2024-04-24T23:20:08.401 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:02.845+0000 7ff78a4ab700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff7841064d0 0x7ff78419de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:08.401 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:02.845+0000 7ff78a4ab700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff7841064d0 0x7ff78419de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:45994/0 (socket says 172.21.15.80:45994) 2024-04-24T23:20:08.402 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:02.845+0000 7ff78a4ab700 1 -- 172.21.15.80:0/2959155873 learned_addr learned my addr 172.21.15.80:0/2959155873 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:20:08.402 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:02.845+0000 7ff78a4ab700 1 -- 172.21.15.80:0/2959155873 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff78419e390 con 0x7ff7841064d0 2024-04-24T23:20:08.402 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:02.845+0000 7ff78a4ab700 1 --2- 172.21.15.80:0/2959155873 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff7841064d0 0x7ff78419de50 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff7740116a0 tx=0x7ff774011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:08.402 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:02.845+0000 7ff77b7fe700 1 -- 172.21.15.80:0/2959155873 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff774003a40 con 0x7ff7841064d0 2024-04-24T23:20:08.402 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:02.845+0000 7ff78c70f700 1 -- 172.21.15.80:0/2959155873 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff78419e620 con 0x7ff7841064d0 2024-04-24T23:20:08.402 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:02.845+0000 7ff77b7fe700 1 -- 172.21.15.80:0/2959155873 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff774003be0 con 0x7ff7841064d0 2024-04-24T23:20:08.402 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:02.845+0000 7ff77b7fe700 1 -- 172.21.15.80:0/2959155873 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff77401a750 con 0x7ff7841064d0 2024-04-24T23:20:08.402 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:02.845+0000 7ff78c70f700 1 -- 172.21.15.80:0/2959155873 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff78419ea80 con 0x7ff7841064d0 2024-04-24T23:20:08.402 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:02.846+0000 7ff77b7fe700 1 -- 172.21.15.80:0/2959155873 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7ff77401aa80 con 0x7ff7841064d0 2024-04-24T23:20:08.402 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:02.846+0000 7ff77b7fe700 1 --2- 172.21.15.80:0/2959155873 >> [v2:172.21.15.80:6800/233364790,v1:172.21.15.80:6801/233364790] conn(0x7ff7700390b0 0x7ff77003b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:08.402 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:02.846+0000 7ff77b7fe700 1 -- 172.21.15.80:0/2959155873 --> [v2:172.21.15.80:6800/233364790,v1:172.21.15.80:6801/233364790] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ff77003bc80 con 0x7ff7700390b0 2024-04-24T23:20:08.402 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:02.847+0000 7ff77b7fe700 1 -- 172.21.15.80:0/2959155873 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7ff77404e570 con 0x7ff7841064d0 2024-04-24T23:20:08.402 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:02.847+0000 7ff789caa700 1 -- 172.21.15.80:0/2959155873 >> [v2:172.21.15.80:6800/233364790,v1:172.21.15.80:6801/233364790] conn(0x7ff7700390b0 msgr2=0x7ff77003b570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.80:6800/233364790 2024-04-24T23:20:08.403 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:02.847+0000 7ff789caa700 1 --2- 172.21.15.80:0/2959155873 >> [v2:172.21.15.80:6800/233364790,v1:172.21.15.80:6801/233364790] conn(0x7ff7700390b0 0x7ff77003b570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T23:20:08.403 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:03.048+0000 7ff789caa700 1 -- 172.21.15.80:0/2959155873 >> [v2:172.21.15.80:6800/233364790,v1:172.21.15.80:6801/233364790] conn(0x7ff7700390b0 msgr2=0x7ff77003b570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.80:6800/233364790 2024-04-24T23:20:08.403 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:03.048+0000 7ff789caa700 1 --2- 172.21.15.80:0/2959155873 >> [v2:172.21.15.80:6800/233364790,v1:172.21.15.80:6801/233364790] conn(0x7ff7700390b0 0x7ff77003b570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-24T23:20:08.403 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:03.449+0000 7ff789caa700 1 -- 172.21.15.80:0/2959155873 >> [v2:172.21.15.80:6800/233364790,v1:172.21.15.80:6801/233364790] conn(0x7ff7700390b0 msgr2=0x7ff77003b570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.80:6800/233364790 2024-04-24T23:20:08.403 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:03.449+0000 7ff789caa700 1 --2- 172.21.15.80:0/2959155873 >> [v2:172.21.15.80:6800/233364790,v1:172.21.15.80:6801/233364790] conn(0x7ff7700390b0 0x7ff77003b570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-24T23:20:08.403 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:04.250+0000 7ff789caa700 1 -- 172.21.15.80:0/2959155873 >> [v2:172.21.15.80:6800/233364790,v1:172.21.15.80:6801/233364790] conn(0x7ff7700390b0 msgr2=0x7ff77003b570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.80:6800/233364790 2024-04-24T23:20:08.403 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:04.250+0000 7ff789caa700 1 --2- 172.21.15.80:0/2959155873 >> [v2:172.21.15.80:6800/233364790,v1:172.21.15.80:6801/233364790] conn(0x7ff7700390b0 0x7ff77003b570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-24T23:20:08.403 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:05.851+0000 7ff789caa700 1 -- 172.21.15.80:0/2959155873 >> [v2:172.21.15.80:6800/233364790,v1:172.21.15.80:6801/233364790] conn(0x7ff7700390b0 msgr2=0x7ff77003b570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.80:6800/233364790 2024-04-24T23:20:08.403 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:05.851+0000 7ff789caa700 1 --2- 172.21.15.80:0/2959155873 >> [v2:172.21.15.80:6800/233364790,v1:172.21.15.80:6801/233364790] conn(0x7ff7700390b0 0x7ff77003b570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-24T23:20:08.403 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:06.794+0000 7ff77b7fe700 1 -- 172.21.15.80:0/2959155873 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mgrmap(e 10) ==== 46067+0+0 (secure 0 0 0) 0x7ff77402d910 con 0x7ff7841064d0 2024-04-24T23:20:08.403 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:06.794+0000 7ff77b7fe700 1 -- 172.21.15.80:0/2959155873 >> [v2:172.21.15.80:6800/233364790,v1:172.21.15.80:6801/233364790] conn(0x7ff7700390b0 msgr2=0x7ff77003b570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T23:20:08.403 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:06.794+0000 7ff77b7fe700 1 --2- 172.21.15.80:0/2959155873 >> [v2:172.21.15.80:6800/233364790,v1:172.21.15.80:6801/233364790] conn(0x7ff7700390b0 0x7ff77003b570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:08.403 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:07.796+0000 7ff77b7fe700 1 -- 172.21.15.80:0/2959155873 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mgrmap(e 11) ==== 46194+0+0 (secure 0 0 0) 0x7ff77403cc60 con 0x7ff7841064d0 2024-04-24T23:20:08.403 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:07.796+0000 7ff77b7fe700 1 --2- 172.21.15.80:0/2959155873 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7ff77003cb70 0x7ff77003ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:08.403 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:07.796+0000 7ff77b7fe700 1 -- 172.21.15.80:0/2959155873 --> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ff774047690 con 0x7ff77003cb70 2024-04-24T23:20:08.404 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:07.797+0000 7ff789caa700 1 --2- 172.21.15.80:0/2959155873 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7ff77003cb70 0x7ff77003ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:08.404 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:07.797+0000 7ff789caa700 1 --2- 172.21.15.80:0/2959155873 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7ff77003cb70 0x7ff77003ef60 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7ff780003a10 tx=0x7ff780009270 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:08.404 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:07.798+0000 7ff77b7fe700 1 -- 172.21.15.80:0/2959155873 <== mgr.14152 v2:172.21.15.80:6800/3830596922 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7ff774047690 con 0x7ff77003cb70 2024-04-24T23:20:08.404 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:07.810+0000 7ff78c70f700 1 -- 172.21.15.80:0/2959155873 --> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7ff784063890 con 0x7ff77003cb70 2024-04-24T23:20:08.404 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:07.810+0000 7ff77b7fe700 1 -- 172.21.15.80:0/2959155873 <== mgr.14152 v2:172.21.15.80:6800/3830596922 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7ff784063890 con 0x7ff77003cb70 2024-04-24T23:20:08.404 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:07.811+0000 7ff78c70f700 1 -- 172.21.15.80:0/2959155873 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7ff77003cb70 msgr2=0x7ff77003ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:08.404 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:07.811+0000 7ff78c70f700 1 --2- 172.21.15.80:0/2959155873 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7ff77003cb70 0x7ff77003ef60 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7ff780003a10 tx=0x7ff780009270 comp rx=0 tx=0).stop 2024-04-24T23:20:08.404 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:07.811+0000 7ff78c70f700 1 -- 172.21.15.80:0/2959155873 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff7841064d0 msgr2=0x7ff78419de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:08.404 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:07.811+0000 7ff78c70f700 1 --2- 172.21.15.80:0/2959155873 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff7841064d0 0x7ff78419de50 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff7740116a0 tx=0x7ff774011ba0 comp rx=0 tx=0).stop 2024-04-24T23:20:08.404 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:07.811+0000 7ff78c70f700 1 -- 172.21.15.80:0/2959155873 shutdown_connections 2024-04-24T23:20:08.404 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:07.811+0000 7ff78c70f700 1 --2- 172.21.15.80:0/2959155873 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7ff77003cb70 0x7ff77003ef60 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:08.404 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:07.811+0000 7ff78c70f700 1 --2- 172.21.15.80:0/2959155873 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff7841064d0 0x7ff78419de50 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:08.404 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:07.811+0000 7ff78c70f700 1 --2- 172.21.15.80:0/2959155873 >> [v2:172.21.15.80:6800/233364790,v1:172.21.15.80:6801/233364790] conn(0x7ff7700390b0 0x7ff77003b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:08.404 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:07.811+0000 7ff78c70f700 1 -- 172.21.15.80:0/2959155873 >> 172.21.15.80:0/2959155873 conn(0x7ff7841020b0 msgr2=0x7ff784194a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:08.405 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:07.811+0000 7ff78c70f700 1 -- 172.21.15.80:0/2959155873 shutdown_connections 2024-04-24T23:20:08.405 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:07.811+0000 7ff78c70f700 1 -- 172.21.15.80:0/2959155873 wait complete. 2024-04-24T23:20:08.405 INFO:teuthology.orchestra.run.smithi080.stdout:mgr epoch 9 is available 2024-04-24T23:20:08.405 INFO:teuthology.orchestra.run.smithi080.stdout:Generating a dashboard self-signed certificate... 2024-04-24T23:20:09.009 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:08 smithi080 ceph-mon[29152]: [24/Apr/2024:23:20:07] ENGINE Serving on https://172.21.15.80:7150 2024-04-24T23:20:09.009 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:08 smithi080 ceph-mon[29152]: [24/Apr/2024:23:20:07] ENGINE Bus STARTED 2024-04-24T23:20:09.009 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:08 smithi080 ceph-mon[29152]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-24T23:20:09.010 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:08 smithi080 ceph-mon[29152]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-24T23:20:09.765 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-04-24T23:20:09.765 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:08.789+0000 7f471789e700 1 Processor -- start 2024-04-24T23:20:09.765 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:08.789+0000 7f471789e700 1 -- start start 2024-04-24T23:20:09.765 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:08.789+0000 7f471789e700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4710108680 0x7f471010aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:09.765 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:08.789+0000 7f471789e700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f471010afb0 con 0x7f4710108680 2024-04-24T23:20:09.765 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:08.790+0000 7f471563a700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4710108680 0x7f471010aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:09.765 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:08.790+0000 7f471563a700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4710108680 0x7f471010aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:46074/0 (socket says 172.21.15.80:46074) 2024-04-24T23:20:09.765 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:08.790+0000 7f471563a700 1 -- 172.21.15.80:0/1420336934 learned_addr learned my addr 172.21.15.80:0/1420336934 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:20:09.765 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:08.790+0000 7f471563a700 1 -- 172.21.15.80:0/1420336934 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f471010b130 con 0x7f4710108680 2024-04-24T23:20:09.766 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:08.790+0000 7f471563a700 1 --2- 172.21.15.80:0/1420336934 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4710108680 0x7f471010aa70 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f470c0098a0 tx=0x7f470c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4221a2a753f5c3b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:09.766 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:08.790+0000 7f4707fff700 1 -- 172.21.15.80:0/1420336934 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f470c004160 con 0x7f4710108680 2024-04-24T23:20:09.766 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:08.790+0000 7f4707fff700 1 -- 172.21.15.80:0/1420336934 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f470c004300 con 0x7f4710108680 2024-04-24T23:20:09.766 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:08.790+0000 7f4707fff700 1 -- 172.21.15.80:0/1420336934 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f470c011b20 con 0x7f4710108680 2024-04-24T23:20:09.766 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:08.791+0000 7f471789e700 1 -- 172.21.15.80:0/1420336934 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4710108680 msgr2=0x7f471010aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:09.766 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:08.791+0000 7f471789e700 1 --2- 172.21.15.80:0/1420336934 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4710108680 0x7f471010aa70 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f470c0098a0 tx=0x7f470c009bb0 comp rx=0 tx=0).stop 2024-04-24T23:20:09.766 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:08.791+0000 7f471789e700 1 -- 172.21.15.80:0/1420336934 shutdown_connections 2024-04-24T23:20:09.766 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:08.791+0000 7f471789e700 1 --2- 172.21.15.80:0/1420336934 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4710108680 0x7f471010aa70 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:09.766 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:08.791+0000 7f471789e700 1 -- 172.21.15.80:0/1420336934 >> 172.21.15.80:0/1420336934 conn(0x7f47101020b0 msgr2=0x7f47101044d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:09.766 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:08.791+0000 7f471789e700 1 -- 172.21.15.80:0/1420336934 shutdown_connections 2024-04-24T23:20:09.766 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:08.791+0000 7f471789e700 1 -- 172.21.15.80:0/1420336934 wait complete. 2024-04-24T23:20:09.766 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:08.792+0000 7f471789e700 1 Processor -- start 2024-04-24T23:20:09.766 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:08.792+0000 7f471789e700 1 -- start start 2024-04-24T23:20:09.766 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:08.792+0000 7f471789e700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4710108680 0x7f4710107f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:09.767 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:08.792+0000 7f471789e700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f471010c0e0 con 0x7f4710108680 2024-04-24T23:20:09.767 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:08.792+0000 7f471563a700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4710108680 0x7f4710107f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:09.767 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:08.792+0000 7f471563a700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4710108680 0x7f4710107f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:46082/0 (socket says 172.21.15.80:46082) 2024-04-24T23:20:09.767 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:08.792+0000 7f471563a700 1 -- 172.21.15.80:0/181510529 learned_addr learned my addr 172.21.15.80:0/181510529 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:20:09.767 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:08.792+0000 7f471563a700 1 -- 172.21.15.80:0/181510529 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4710108440 con 0x7f4710108680 2024-04-24T23:20:09.767 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:08.792+0000 7f471563a700 1 --2- 172.21.15.80:0/181510529 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4710108680 0x7f4710107f00 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f470c0116a0 tx=0x7f470c011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:09.767 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:08.792+0000 7f47067fc700 1 -- 172.21.15.80:0/181510529 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f470c003a40 con 0x7f4710108680 2024-04-24T23:20:09.767 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:08.793+0000 7f47067fc700 1 -- 172.21.15.80:0/181510529 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f470c003be0 con 0x7f4710108680 2024-04-24T23:20:09.767 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:08.793+0000 7f471789e700 1 -- 172.21.15.80:0/181510529 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4710106630 con 0x7f4710108680 2024-04-24T23:20:09.767 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:08.793+0000 7f47067fc700 1 -- 172.21.15.80:0/181510529 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f470c01a750 con 0x7f4710108680 2024-04-24T23:20:09.767 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:08.793+0000 7f471789e700 1 -- 172.21.15.80:0/181510529 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4710106af0 con 0x7f4710108680 2024-04-24T23:20:09.767 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:08.794+0000 7f47067fc700 1 -- 172.21.15.80:0/181510529 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 11) ==== 46194+0+0 (secure 0 0 0) 0x7f470c01aa00 con 0x7f4710108680 2024-04-24T23:20:09.767 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:08.794+0000 7f471789e700 1 -- 172.21.15.80:0/181510529 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4710063890 con 0x7f4710108680 2024-04-24T23:20:09.767 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:08.794+0000 7f47067fc700 1 --2- 172.21.15.80:0/181510529 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f46fc038f90 0x7f46fc03b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:09.768 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:08.794+0000 7f47067fc700 1 -- 172.21.15.80:0/181510529 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f470c04d6f0 con 0x7f4710108680 2024-04-24T23:20:09.768 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:08.794+0000 7f4714e39700 1 --2- 172.21.15.80:0/181510529 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f46fc038f90 0x7f46fc03b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:09.768 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:08.794+0000 7f4714e39700 1 --2- 172.21.15.80:0/181510529 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f46fc038f90 0x7f46fc03b450 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4700006fd0 tx=0x7f4700006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:09.768 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:08.799+0000 7f47067fc700 1 -- 172.21.15.80:0/181510529 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f470c011e20 con 0x7f4710108680 2024-04-24T23:20:09.768 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:08.854+0000 7f47067fc700 1 -- 172.21.15.80:0/181510529 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f470c02d910 con 0x7f4710108680 2024-04-24T23:20:09.768 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:09.007+0000 7f471789e700 1 -- 172.21.15.80:0/181510529 --> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f47101071b0 con 0x7f46fc038f90 2024-04-24T23:20:09.768 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:09.207+0000 7f47067fc700 1 -- 172.21.15.80:0/181510529 <== mgr.14152 v2:172.21.15.80:6800/3830596922 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f47101071b0 con 0x7f46fc038f90 2024-04-24T23:20:09.768 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:09.209+0000 7f471789e700 1 -- 172.21.15.80:0/181510529 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f46fc038f90 msgr2=0x7f46fc03b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:09.768 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:09.209+0000 7f471789e700 1 --2- 172.21.15.80:0/181510529 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f46fc038f90 0x7f46fc03b450 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4700006fd0 tx=0x7f4700006e40 comp rx=0 tx=0).stop 2024-04-24T23:20:09.768 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:09.209+0000 7f471789e700 1 -- 172.21.15.80:0/181510529 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4710108680 msgr2=0x7f4710107f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:09.768 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:09.209+0000 7f471789e700 1 --2- 172.21.15.80:0/181510529 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4710108680 0x7f4710107f00 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f470c0116a0 tx=0x7f470c011ba0 comp rx=0 tx=0).stop 2024-04-24T23:20:09.768 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:09.209+0000 7f471789e700 1 -- 172.21.15.80:0/181510529 shutdown_connections 2024-04-24T23:20:09.768 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:09.209+0000 7f471789e700 1 --2- 172.21.15.80:0/181510529 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f46fc038f90 0x7f46fc03b450 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:09.769 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:09.209+0000 7f471789e700 1 --2- 172.21.15.80:0/181510529 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4710108680 0x7f4710107f00 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:09.769 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:09.209+0000 7f471789e700 1 -- 172.21.15.80:0/181510529 >> 172.21.15.80:0/181510529 conn(0x7f47101020b0 msgr2=0x7f4710102ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:09.769 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:09.209+0000 7f471789e700 1 -- 172.21.15.80:0/181510529 shutdown_connections 2024-04-24T23:20:09.769 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:09.210+0000 7f471789e700 1 -- 172.21.15.80:0/181510529 wait complete. 2024-04-24T23:20:09.769 INFO:teuthology.orchestra.run.smithi080.stdout:Creating initial admin user... 2024-04-24T23:20:10.161 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:09 smithi080 ceph-mon[29152]: mgrmap e12: x(active, since 2s) 2024-04-24T23:20:10.161 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:09 smithi080 ceph-mon[29152]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:20:10.161 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:09 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:20:10.161 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:09 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:20:10.161 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:09 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:20:11.306 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$Kk10vsEfv2Qn.4DM4DJnOeKeOigFDu0bzUzdMmkQs5bIfIYQrfIn6", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1714000810, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-04-24T23:20:11.306 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:10.197+0000 7fce86f16700 1 Processor -- start 2024-04-24T23:20:11.306 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:10.197+0000 7fce86f16700 1 -- start start 2024-04-24T23:20:11.306 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:10.197+0000 7fce86f16700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fce80108700 0x7fce8010aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:11.306 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:10.197+0000 7fce86f16700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fce8007be60 con 0x7fce80108700 2024-04-24T23:20:11.307 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:10.198+0000 7fce84cb2700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fce80108700 0x7fce8010aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:11.307 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:10.198+0000 7fce84cb2700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fce80108700 0x7fce8010aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:46086/0 (socket says 172.21.15.80:46086) 2024-04-24T23:20:11.307 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:10.198+0000 7fce84cb2700 1 -- 172.21.15.80:0/1193893857 learned_addr learned my addr 172.21.15.80:0/1193893857 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:20:11.307 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:10.198+0000 7fce84cb2700 1 -- 172.21.15.80:0/1193893857 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce8010b030 con 0x7fce80108700 2024-04-24T23:20:11.307 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:10.198+0000 7fce84cb2700 1 --2- 172.21.15.80:0/1193893857 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fce80108700 0x7fce8010aaf0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fce740098a0 tx=0x7fce74009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13cf0741aa8ef367 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:11.307 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:10.198+0000 7fce7f7fe700 1 -- 172.21.15.80:0/1193893857 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fce74004160 con 0x7fce80108700 2024-04-24T23:20:11.307 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:10.198+0000 7fce7f7fe700 1 -- 172.21.15.80:0/1193893857 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce74004300 con 0x7fce80108700 2024-04-24T23:20:11.307 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:10.198+0000 7fce7f7fe700 1 -- 172.21.15.80:0/1193893857 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fce74011b20 con 0x7fce80108700 2024-04-24T23:20:11.307 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:10.199+0000 7fce86f16700 1 -- 172.21.15.80:0/1193893857 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fce80108700 msgr2=0x7fce8010aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:11.307 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:10.199+0000 7fce86f16700 1 --2- 172.21.15.80:0/1193893857 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fce80108700 0x7fce8010aaf0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fce740098a0 tx=0x7fce74009bb0 comp rx=0 tx=0).stop 2024-04-24T23:20:11.307 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:10.199+0000 7fce86f16700 1 -- 172.21.15.80:0/1193893857 shutdown_connections 2024-04-24T23:20:11.307 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:10.199+0000 7fce86f16700 1 --2- 172.21.15.80:0/1193893857 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fce80108700 0x7fce8010aaf0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:11.307 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:10.199+0000 7fce86f16700 1 -- 172.21.15.80:0/1193893857 >> 172.21.15.80:0/1193893857 conn(0x7fce80102030 msgr2=0x7fce80104490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:11.308 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:10.199+0000 7fce86f16700 1 -- 172.21.15.80:0/1193893857 shutdown_connections 2024-04-24T23:20:11.308 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:10.199+0000 7fce86f16700 1 -- 172.21.15.80:0/1193893857 wait complete. 2024-04-24T23:20:11.308 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:10.200+0000 7fce86f16700 1 Processor -- start 2024-04-24T23:20:11.308 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:10.200+0000 7fce86f16700 1 -- start start 2024-04-24T23:20:11.308 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:10.200+0000 7fce86f16700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fce80108700 0x7fce80107f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:11.308 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:10.200+0000 7fce86f16700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fce8010c0f0 con 0x7fce80108700 2024-04-24T23:20:11.308 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:10.200+0000 7fce84cb2700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fce80108700 0x7fce80107f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:11.308 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:10.200+0000 7fce84cb2700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fce80108700 0x7fce80107f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:46094/0 (socket says 172.21.15.80:46094) 2024-04-24T23:20:11.308 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:10.200+0000 7fce84cb2700 1 -- 172.21.15.80:0/4142716517 learned_addr learned my addr 172.21.15.80:0/4142716517 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:20:11.308 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:10.200+0000 7fce84cb2700 1 -- 172.21.15.80:0/4142716517 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce801084c0 con 0x7fce80108700 2024-04-24T23:20:11.308 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:10.200+0000 7fce84cb2700 1 --2- 172.21.15.80:0/4142716517 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fce80108700 0x7fce80107f80 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fce740117e0 tx=0x7fce74011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:11.308 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:10.200+0000 7fce7dffb700 1 -- 172.21.15.80:0/4142716517 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fce74003ba0 con 0x7fce80108700 2024-04-24T23:20:11.308 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:10.200+0000 7fce7dffb700 1 -- 172.21.15.80:0/4142716517 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce7401a910 con 0x7fce80108700 2024-04-24T23:20:11.308 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:10.200+0000 7fce7dffb700 1 -- 172.21.15.80:0/4142716517 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fce740248f0 con 0x7fce80108700 2024-04-24T23:20:11.309 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:10.200+0000 7fce86f16700 1 -- 172.21.15.80:0/4142716517 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce801066b0 con 0x7fce80108700 2024-04-24T23:20:11.309 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:10.201+0000 7fce86f16700 1 -- 172.21.15.80:0/4142716517 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce80106b70 con 0x7fce80108700 2024-04-24T23:20:11.309 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:10.202+0000 7fce7dffb700 1 -- 172.21.15.80:0/4142716517 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7fce74022020 con 0x7fce80108700 2024-04-24T23:20:11.309 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:10.202+0000 7fce86f16700 1 -- 172.21.15.80:0/4142716517 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce80063890 con 0x7fce80108700 2024-04-24T23:20:11.309 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:10.202+0000 7fce7dffb700 1 --2- 172.21.15.80:0/4142716517 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fce70039100 0x7fce7003b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:11.309 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:10.202+0000 7fce7dffb700 1 -- 172.21.15.80:0/4142716517 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fce7404d5d0 con 0x7fce80108700 2024-04-24T23:20:11.309 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:10.202+0000 7fce7ffff700 1 --2- 172.21.15.80:0/4142716517 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fce70039100 0x7fce7003b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:11.309 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:10.202+0000 7fce7ffff700 1 --2- 172.21.15.80:0/4142716517 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fce70039100 0x7fce7003b5c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fce6c006fd0 tx=0x7fce6c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:11.309 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:10.207+0000 7fce7dffb700 1 -- 172.21.15.80:0/4142716517 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fce7401ab70 con 0x7fce80108700 2024-04-24T23:20:11.309 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:10.428+0000 7fce86f16700 1 -- 172.21.15.80:0/4142716517 --> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7fce801071d0 con 0x7fce70039100 2024-04-24T23:20:11.309 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:10.656+0000 7fce7dffb700 1 -- 172.21.15.80:0/4142716517 <== mgr.14152 v2:172.21.15.80:6800/3830596922 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7fce801071d0 con 0x7fce70039100 2024-04-24T23:20:11.309 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:10.658+0000 7fce86f16700 1 -- 172.21.15.80:0/4142716517 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fce70039100 msgr2=0x7fce7003b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:11.309 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:10.658+0000 7fce86f16700 1 --2- 172.21.15.80:0/4142716517 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fce70039100 0x7fce7003b5c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fce6c006fd0 tx=0x7fce6c006e40 comp rx=0 tx=0).stop 2024-04-24T23:20:11.309 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:10.658+0000 7fce86f16700 1 -- 172.21.15.80:0/4142716517 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fce80108700 msgr2=0x7fce80107f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:11.310 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:10.658+0000 7fce86f16700 1 --2- 172.21.15.80:0/4142716517 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fce80108700 0x7fce80107f80 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fce740117e0 tx=0x7fce74011ba0 comp rx=0 tx=0).stop 2024-04-24T23:20:11.310 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:10.658+0000 7fce86f16700 1 -- 172.21.15.80:0/4142716517 shutdown_connections 2024-04-24T23:20:11.310 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:10.658+0000 7fce86f16700 1 --2- 172.21.15.80:0/4142716517 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fce70039100 0x7fce7003b5c0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:11.310 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:10.658+0000 7fce86f16700 1 --2- 172.21.15.80:0/4142716517 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fce80108700 0x7fce80107f80 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:11.310 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:10.658+0000 7fce86f16700 1 -- 172.21.15.80:0/4142716517 >> 172.21.15.80:0/4142716517 conn(0x7fce80102030 msgr2=0x7fce80102af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:11.310 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:10.658+0000 7fce86f16700 1 -- 172.21.15.80:0/4142716517 shutdown_connections 2024-04-24T23:20:11.310 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:10.658+0000 7fce86f16700 1 -- 172.21.15.80:0/4142716517 wait complete. 2024-04-24T23:20:11.310 INFO:teuthology.orchestra.run.smithi080.stdout:Fetching dashboard port number... 2024-04-24T23:20:11.634 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:11 smithi080 ceph-mon[29152]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:20:11.634 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:11 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:20:11.634 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:11 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:20:12.540 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:12 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/329590630' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-04-24T23:20:12.549 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stdout 8443 2024-04-24T23:20:12.549 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:11.732+0000 7fe3d4a42700 1 Processor -- start 2024-04-24T23:20:12.549 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:11.732+0000 7fe3d4a42700 1 -- start start 2024-04-24T23:20:12.549 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:11.732+0000 7fe3d4a42700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe3d0108190 0x7fe3d01085b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:12.549 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:11.732+0000 7fe3d4a42700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d01066d0 con 0x7fe3d0108190 2024-04-24T23:20:12.550 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:11.732+0000 7fe3ce59c700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe3d0108190 0x7fe3d01085b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:12.550 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:11.732+0000 7fe3ce59c700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe3d0108190 0x7fe3d01085b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:46104/0 (socket says 172.21.15.80:46104) 2024-04-24T23:20:12.550 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:11.732+0000 7fe3ce59c700 1 -- 172.21.15.80:0/3030360190 learned_addr learned my addr 172.21.15.80:0/3030360190 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:20:12.550 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:11.732+0000 7fe3ce59c700 1 -- 172.21.15.80:0/3030360190 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3d0106850 con 0x7fe3d0108190 2024-04-24T23:20:12.550 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:11.733+0000 7fe3ce59c700 1 --2- 172.21.15.80:0/3030360190 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe3d0108190 0x7fe3d01085b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe3c00098a0 tx=0x7fe3c0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=894f244fb07785de server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:12.550 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:11.733+0000 7fe3cd59a700 1 -- 172.21.15.80:0/3030360190 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe3c0004160 con 0x7fe3d0108190 2024-04-24T23:20:12.550 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:11.733+0000 7fe3cd59a700 1 -- 172.21.15.80:0/3030360190 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3c0004300 con 0x7fe3d0108190 2024-04-24T23:20:12.550 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:11.733+0000 7fe3cd59a700 1 -- 172.21.15.80:0/3030360190 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe3c0011b20 con 0x7fe3d0108190 2024-04-24T23:20:12.550 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:11.733+0000 7fe3d4a42700 1 -- 172.21.15.80:0/3030360190 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe3d0108190 msgr2=0x7fe3d01085b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:12.550 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:11.733+0000 7fe3d4a42700 1 --2- 172.21.15.80:0/3030360190 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe3d0108190 0x7fe3d01085b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe3c00098a0 tx=0x7fe3c0009bb0 comp rx=0 tx=0).stop 2024-04-24T23:20:12.550 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:11.733+0000 7fe3d4a42700 1 -- 172.21.15.80:0/3030360190 shutdown_connections 2024-04-24T23:20:12.550 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:11.733+0000 7fe3d4a42700 1 --2- 172.21.15.80:0/3030360190 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe3d0108190 0x7fe3d01085b0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:12.550 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:11.733+0000 7fe3d4a42700 1 -- 172.21.15.80:0/3030360190 >> 172.21.15.80:0/3030360190 conn(0x7fe3d0102030 msgr2=0x7fe3d0104490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:12.551 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:11.733+0000 7fe3d4a42700 1 -- 172.21.15.80:0/3030360190 shutdown_connections 2024-04-24T23:20:12.551 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:11.733+0000 7fe3d4a42700 1 -- 172.21.15.80:0/3030360190 wait complete. 2024-04-24T23:20:12.551 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:11.734+0000 7fe3d4a42700 1 Processor -- start 2024-04-24T23:20:12.551 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:11.734+0000 7fe3d4a42700 1 -- start start 2024-04-24T23:20:12.551 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:11.734+0000 7fe3d4a42700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe3d0108190 0x7fe3d019bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:12.551 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:11.734+0000 7fe3d4a42700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d010b060 con 0x7fe3d0108190 2024-04-24T23:20:12.551 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:11.734+0000 7fe3ce59c700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe3d0108190 0x7fe3d019bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:12.551 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:11.734+0000 7fe3ce59c700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe3d0108190 0x7fe3d019bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:46120/0 (socket says 172.21.15.80:46120) 2024-04-24T23:20:12.551 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:11.734+0000 7fe3ce59c700 1 -- 172.21.15.80:0/329590630 learned_addr learned my addr 172.21.15.80:0/329590630 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:20:12.551 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:11.735+0000 7fe3ce59c700 1 -- 172.21.15.80:0/329590630 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3d019c1b0 con 0x7fe3d0108190 2024-04-24T23:20:12.551 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:11.735+0000 7fe3ce59c700 1 --2- 172.21.15.80:0/329590630 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe3d0108190 0x7fe3d019bc70 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe3c00117e0 tx=0x7fe3c0011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:12.551 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:11.735+0000 7fe3bf7fe700 1 -- 172.21.15.80:0/329590630 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe3c0003bf0 con 0x7fe3d0108190 2024-04-24T23:20:12.551 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:11.735+0000 7fe3bf7fe700 1 -- 172.21.15.80:0/329590630 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3c001a910 con 0x7fe3d0108190 2024-04-24T23:20:12.551 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:11.735+0000 7fe3d4a42700 1 -- 172.21.15.80:0/329590630 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3d019c440 con 0x7fe3d0108190 2024-04-24T23:20:12.551 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:11.735+0000 7fe3bf7fe700 1 -- 172.21.15.80:0/329590630 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe3c00249a0 con 0x7fe3d0108190 2024-04-24T23:20:12.552 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:11.735+0000 7fe3d4a42700 1 -- 172.21.15.80:0/329590630 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3d019c8a0 con 0x7fe3d0108190 2024-04-24T23:20:12.552 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:11.736+0000 7fe3bf7fe700 1 -- 172.21.15.80:0/329590630 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7fe3c0022020 con 0x7fe3d0108190 2024-04-24T23:20:12.552 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:11.736+0000 7fe3d4a42700 1 -- 172.21.15.80:0/329590630 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3d0063890 con 0x7fe3d0108190 2024-04-24T23:20:12.552 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:11.736+0000 7fe3bf7fe700 1 --2- 172.21.15.80:0/329590630 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fe3b80390b0 0x7fe3b803b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:12.552 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:11.736+0000 7fe3bf7fe700 1 -- 172.21.15.80:0/329590630 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fe3c004d5f0 con 0x7fe3d0108190 2024-04-24T23:20:12.552 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:11.736+0000 7fe3cdd9b700 1 --2- 172.21.15.80:0/329590630 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fe3b80390b0 0x7fe3b803b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:12.552 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:11.737+0000 7fe3cdd9b700 1 --2- 172.21.15.80:0/329590630 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fe3b80390b0 0x7fe3b803b570 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe3c4006fd0 tx=0x7fe3c4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:12.552 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:11.741+0000 7fe3bf7fe700 1 -- 172.21.15.80:0/329590630 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fe3c0011e20 con 0x7fe3d0108190 2024-04-24T23:20:12.552 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:11.948+0000 7fe3d4a42700 1 -- 172.21.15.80:0/329590630 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7fe3d002de70 con 0x7fe3d0108190 2024-04-24T23:20:12.552 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:11.949+0000 7fe3bf7fe700 1 -- 172.21.15.80:0/329590630 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7fe3c000ba00 con 0x7fe3d0108190 2024-04-24T23:20:12.552 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:11.952+0000 7fe3d4a42700 1 -- 172.21.15.80:0/329590630 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fe3b80390b0 msgr2=0x7fe3b803b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:12.552 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:11.952+0000 7fe3d4a42700 1 --2- 172.21.15.80:0/329590630 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fe3b80390b0 0x7fe3b803b570 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe3c4006fd0 tx=0x7fe3c4006e40 comp rx=0 tx=0).stop 2024-04-24T23:20:12.552 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:11.952+0000 7fe3d4a42700 1 -- 172.21.15.80:0/329590630 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe3d0108190 msgr2=0x7fe3d019bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:12.553 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:11.952+0000 7fe3d4a42700 1 --2- 172.21.15.80:0/329590630 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe3d0108190 0x7fe3d019bc70 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe3c00117e0 tx=0x7fe3c0011c10 comp rx=0 tx=0).stop 2024-04-24T23:20:12.553 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:11.952+0000 7fe3d4a42700 1 -- 172.21.15.80:0/329590630 shutdown_connections 2024-04-24T23:20:12.553 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:11.952+0000 7fe3d4a42700 1 --2- 172.21.15.80:0/329590630 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fe3b80390b0 0x7fe3b803b570 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:12.553 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:11.952+0000 7fe3d4a42700 1 --2- 172.21.15.80:0/329590630 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe3d0108190 0x7fe3d019bc70 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:12.553 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:11.952+0000 7fe3d4a42700 1 -- 172.21.15.80:0/329590630 >> 172.21.15.80:0/329590630 conn(0x7fe3d0102030 msgr2=0x7fe3d0102ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:12.553 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:11.952+0000 7fe3d4a42700 1 -- 172.21.15.80:0/329590630 shutdown_connections 2024-04-24T23:20:12.553 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:11.952+0000 7fe3d4a42700 1 -- 172.21.15.80:0/329590630 wait complete. 2024-04-24T23:20:12.557 INFO:teuthology.orchestra.run.smithi080.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-24T23:20:12.557 INFO:teuthology.orchestra.run.smithi080.stdout:systemctl: stdout disabled 2024-04-24T23:20:12.564 INFO:teuthology.orchestra.run.smithi080.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-24T23:20:12.565 INFO:teuthology.orchestra.run.smithi080.stdout:systemctl: stdout inactive 2024-04-24T23:20:12.565 INFO:teuthology.orchestra.run.smithi080.stdout:firewalld.service is not enabled 2024-04-24T23:20:12.565 INFO:teuthology.orchestra.run.smithi080.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-04-24T23:20:12.566 INFO:teuthology.orchestra.run.smithi080.stdout:Ceph Dashboard is now available at: 2024-04-24T23:20:12.566 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-24T23:20:12.566 INFO:teuthology.orchestra.run.smithi080.stdout: URL: https://smithi080:8443/ 2024-04-24T23:20:12.567 INFO:teuthology.orchestra.run.smithi080.stdout: User: admin 2024-04-24T23:20:12.567 INFO:teuthology.orchestra.run.smithi080.stdout: Password: 00ekebr3tr 2024-04-24T23:20:12.567 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-24T23:20:12.567 INFO:teuthology.orchestra.run.smithi080.stdout:Saving cluster configuration to /var/lib/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/config directory 2024-04-24T23:20:12.567 INFO:teuthology.orchestra.run.smithi080.stdout:Enabling autotune for osd_memory_target 2024-04-24T23:20:13.925 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:12.996+0000 7f53e44a9700 1 Processor -- start 2024-04-24T23:20:13.925 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:12.996+0000 7f53e44a9700 1 -- start start 2024-04-24T23:20:13.926 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:12.996+0000 7f53e44a9700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f53dc1092d0 0x7f53dc1096f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:13.926 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:12.996+0000 7f53e44a9700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f53dc109cc0 con 0x7f53dc1092d0 2024-04-24T23:20:13.926 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:12.996+0000 7f53e2245700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f53dc1092d0 0x7f53dc1096f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:13.926 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:12.996+0000 7f53e2245700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f53dc1092d0 0x7f53dc1096f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:55536/0 (socket says 172.21.15.80:55536) 2024-04-24T23:20:13.926 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:12.996+0000 7f53e2245700 1 -- 172.21.15.80:0/3352241434 learned_addr learned my addr 172.21.15.80:0/3352241434 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:20:13.926 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:12.996+0000 7f53e2245700 1 -- 172.21.15.80:0/3352241434 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53dc105b20 con 0x7f53dc1092d0 2024-04-24T23:20:13.926 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:12.996+0000 7f53e2245700 1 --2- 172.21.15.80:0/3352241434 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f53dc1092d0 0x7f53dc1096f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f53d00098a0 tx=0x7f53d0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=764216c000c0950c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:13.926 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:12.996+0000 7f53e1243700 1 -- 172.21.15.80:0/3352241434 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f53d0004160 con 0x7f53dc1092d0 2024-04-24T23:20:13.926 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:12.996+0000 7f53e1243700 1 -- 172.21.15.80:0/3352241434 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f53d0004300 con 0x7f53dc1092d0 2024-04-24T23:20:13.926 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:12.996+0000 7f53e1243700 1 -- 172.21.15.80:0/3352241434 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f53d0011b20 con 0x7f53dc1092d0 2024-04-24T23:20:13.926 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:12.997+0000 7f53e44a9700 1 -- 172.21.15.80:0/3352241434 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f53dc1092d0 msgr2=0x7f53dc1096f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:13.926 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:12.997+0000 7f53e44a9700 1 --2- 172.21.15.80:0/3352241434 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f53dc1092d0 0x7f53dc1096f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f53d00098a0 tx=0x7f53d0009bb0 comp rx=0 tx=0).stop 2024-04-24T23:20:13.926 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:12.997+0000 7f53e44a9700 1 -- 172.21.15.80:0/3352241434 shutdown_connections 2024-04-24T23:20:13.927 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:12.997+0000 7f53e44a9700 1 --2- 172.21.15.80:0/3352241434 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f53dc1092d0 0x7f53dc1096f0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:13.927 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:12.997+0000 7f53e44a9700 1 -- 172.21.15.80:0/3352241434 >> 172.21.15.80:0/3352241434 conn(0x7f53dc1017e0 msgr2=0x7f53dc103c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:13.927 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:12.997+0000 7f53e44a9700 1 -- 172.21.15.80:0/3352241434 shutdown_connections 2024-04-24T23:20:13.927 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:12.997+0000 7f53e44a9700 1 -- 172.21.15.80:0/3352241434 wait complete. 2024-04-24T23:20:13.927 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:12.998+0000 7f53e44a9700 1 Processor -- start 2024-04-24T23:20:13.927 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:12.998+0000 7f53e44a9700 1 -- start start 2024-04-24T23:20:13.927 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:12.998+0000 7f53e44a9700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f53dc1092d0 0x7f53dc19bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:13.927 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:12.998+0000 7f53e44a9700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f53dc10c1f0 con 0x7f53dc1092d0 2024-04-24T23:20:13.927 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:12.998+0000 7f53e2245700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f53dc1092d0 0x7f53dc19bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:13.927 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:12.998+0000 7f53e2245700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f53dc1092d0 0x7f53dc19bca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:55552/0 (socket says 172.21.15.80:55552) 2024-04-24T23:20:13.927 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:12.998+0000 7f53e2245700 1 -- 172.21.15.80:0/3224342916 learned_addr learned my addr 172.21.15.80:0/3224342916 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:20:13.927 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:12.998+0000 7f53e2245700 1 -- 172.21.15.80:0/3224342916 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53dc19c1e0 con 0x7f53dc1092d0 2024-04-24T23:20:13.927 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:12.998+0000 7f53e2245700 1 --2- 172.21.15.80:0/3224342916 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f53dc1092d0 0x7f53dc19bca0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f53d00116a0 tx=0x7f53d0011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:13.927 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:12.998+0000 7f53cf7fe700 1 -- 172.21.15.80:0/3224342916 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f53d0003a40 con 0x7f53dc1092d0 2024-04-24T23:20:13.927 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:12.998+0000 7f53cf7fe700 1 -- 172.21.15.80:0/3224342916 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f53d0003be0 con 0x7f53dc1092d0 2024-04-24T23:20:13.928 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:12.998+0000 7f53e44a9700 1 -- 172.21.15.80:0/3224342916 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53dc19c470 con 0x7f53dc1092d0 2024-04-24T23:20:13.928 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:12.998+0000 7f53cf7fe700 1 -- 172.21.15.80:0/3224342916 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f53d001a720 con 0x7f53dc1092d0 2024-04-24T23:20:13.928 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:12.999+0000 7f53e44a9700 1 -- 172.21.15.80:0/3224342916 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53dc19c8d0 con 0x7f53dc1092d0 2024-04-24T23:20:13.928 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:12.999+0000 7f53cf7fe700 1 -- 172.21.15.80:0/3224342916 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f53d001aa40 con 0x7f53dc1092d0 2024-04-24T23:20:13.928 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:12.999+0000 7f53e44a9700 1 -- 172.21.15.80:0/3224342916 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53dc063890 con 0x7f53dc1092d0 2024-04-24T23:20:13.928 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:13.000+0000 7f53cf7fe700 1 --2- 172.21.15.80:0/3224342916 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f53c8039050 0x7f53c803b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:13.928 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:13.000+0000 7f53cf7fe700 1 -- 172.21.15.80:0/3224342916 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f53d004d940 con 0x7f53dc1092d0 2024-04-24T23:20:13.928 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:13.000+0000 7f53e1a44700 1 --2- 172.21.15.80:0/3224342916 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f53c8039050 0x7f53c803b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:13.928 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:13.000+0000 7f53e1a44700 1 --2- 172.21.15.80:0/3224342916 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f53c8039050 0x7f53c803b510 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f53d8006fd0 tx=0x7f53d8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:13.928 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:13.004+0000 7f53cf7fe700 1 -- 172.21.15.80:0/3224342916 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f53d0011e20 con 0x7f53dc1092d0 2024-04-24T23:20:13.928 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:13.205+0000 7f53e44a9700 1 -- 172.21.15.80:0/3224342916 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7f53dc19f220 con 0x7f53dc1092d0 2024-04-24T23:20:13.928 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:13.206+0000 7f53cf7fe700 1 -- 172.21.15.80:0/3224342916 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7f53d0024430 con 0x7f53dc1092d0 2024-04-24T23:20:13.928 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:13.208+0000 7f53e44a9700 1 -- 172.21.15.80:0/3224342916 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f53c8039050 msgr2=0x7f53c803b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:13.928 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:13.208+0000 7f53e44a9700 1 --2- 172.21.15.80:0/3224342916 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f53c8039050 0x7f53c803b510 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f53d8006fd0 tx=0x7f53d8006e40 comp rx=0 tx=0).stop 2024-04-24T23:20:13.928 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:13.208+0000 7f53e44a9700 1 -- 172.21.15.80:0/3224342916 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f53dc1092d0 msgr2=0x7f53dc19bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:13.929 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:13.208+0000 7f53e44a9700 1 --2- 172.21.15.80:0/3224342916 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f53dc1092d0 0x7f53dc19bca0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f53d00116a0 tx=0x7f53d0011ba0 comp rx=0 tx=0).stop 2024-04-24T23:20:13.929 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:13.208+0000 7f53e44a9700 1 -- 172.21.15.80:0/3224342916 shutdown_connections 2024-04-24T23:20:13.929 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:13.208+0000 7f53e44a9700 1 --2- 172.21.15.80:0/3224342916 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f53c8039050 0x7f53c803b510 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:13.929 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:13.208+0000 7f53e44a9700 1 --2- 172.21.15.80:0/3224342916 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f53dc1092d0 0x7f53dc19bca0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:13.929 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:13.208+0000 7f53e44a9700 1 -- 172.21.15.80:0/3224342916 >> 172.21.15.80:0/3224342916 conn(0x7f53dc1017e0 msgr2=0x7f53dc102340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:13.929 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:13.208+0000 7f53e44a9700 1 -- 172.21.15.80:0/3224342916 shutdown_connections 2024-04-24T23:20:13.929 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:13.208+0000 7f53e44a9700 1 -- 172.21.15.80:0/3224342916 wait complete. 2024-04-24T23:20:14.539 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:14 smithi080 ceph-mon[29152]: mgrmap e13: x(active, since 6s) 2024-04-24T23:20:15.369 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:14.477+0000 7f06ba2d3700 1 Processor -- start 2024-04-24T23:20:15.369 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:14.477+0000 7f06ba2d3700 1 -- start start 2024-04-24T23:20:15.370 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:14.477+0000 7f06ba2d3700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f06b4107bc0 0x7f06b4107fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:15.370 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:14.477+0000 7f06ba2d3700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f06b41085b0 con 0x7f06b4107bc0 2024-04-24T23:20:15.370 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:14.478+0000 7f06b3fff700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f06b4107bc0 0x7f06b4107fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:15.370 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:14.478+0000 7f06b3fff700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f06b4107bc0 0x7f06b4107fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:55568/0 (socket says 172.21.15.80:55568) 2024-04-24T23:20:15.370 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:14.478+0000 7f06b3fff700 1 -- 172.21.15.80:0/972736770 learned_addr learned my addr 172.21.15.80:0/972736770 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:20:15.370 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:14.478+0000 7f06b3fff700 1 -- 172.21.15.80:0/972736770 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06b4108d70 con 0x7f06b4107bc0 2024-04-24T23:20:15.370 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:14.478+0000 7f06b3fff700 1 --2- 172.21.15.80:0/972736770 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f06b4107bc0 0x7f06b4107fe0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f06a40098a0 tx=0x7f06a4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8805c03e430ba2fd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:15.370 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:14.478+0000 7f06b2ffd700 1 -- 172.21.15.80:0/972736770 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f06a4004160 con 0x7f06b4107bc0 2024-04-24T23:20:15.371 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:14.478+0000 7f06b2ffd700 1 -- 172.21.15.80:0/972736770 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06a4004300 con 0x7f06b4107bc0 2024-04-24T23:20:15.371 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:14.478+0000 7f06b2ffd700 1 -- 172.21.15.80:0/972736770 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f06a4011b20 con 0x7f06b4107bc0 2024-04-24T23:20:15.371 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:14.479+0000 7f06ba2d3700 1 -- 172.21.15.80:0/972736770 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f06b4107bc0 msgr2=0x7f06b4107fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:15.371 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:14.479+0000 7f06ba2d3700 1 --2- 172.21.15.80:0/972736770 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f06b4107bc0 0x7f06b4107fe0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f06a40098a0 tx=0x7f06a4009bb0 comp rx=0 tx=0).stop 2024-04-24T23:20:15.371 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:14.479+0000 7f06ba2d3700 1 -- 172.21.15.80:0/972736770 shutdown_connections 2024-04-24T23:20:15.371 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:14.479+0000 7f06ba2d3700 1 --2- 172.21.15.80:0/972736770 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f06b4107bc0 0x7f06b4107fe0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:15.371 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:14.479+0000 7f06ba2d3700 1 -- 172.21.15.80:0/972736770 >> 172.21.15.80:0/972736770 conn(0x7f06b41031a0 msgr2=0x7f06b41055c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:15.371 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:14.479+0000 7f06ba2d3700 1 -- 172.21.15.80:0/972736770 shutdown_connections 2024-04-24T23:20:15.371 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:14.479+0000 7f06ba2d3700 1 -- 172.21.15.80:0/972736770 wait complete. 2024-04-24T23:20:15.371 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:14.479+0000 7f06ba2d3700 1 Processor -- start 2024-04-24T23:20:15.371 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:14.479+0000 7f06ba2d3700 1 -- start start 2024-04-24T23:20:15.371 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:14.480+0000 7f06ba2d3700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f06b4107bc0 0x7f06b419bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:15.371 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:14.480+0000 7f06ba2d3700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f06b410c120 con 0x7f06b4107bc0 2024-04-24T23:20:15.371 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:14.480+0000 7f06b3fff700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f06b4107bc0 0x7f06b419bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:15.372 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:14.480+0000 7f06b3fff700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f06b4107bc0 0x7f06b419bad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:55574/0 (socket says 172.21.15.80:55574) 2024-04-24T23:20:15.372 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:14.480+0000 7f06b3fff700 1 -- 172.21.15.80:0/4141060208 learned_addr learned my addr 172.21.15.80:0/4141060208 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:20:15.372 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:14.480+0000 7f06b3fff700 1 -- 172.21.15.80:0/4141060208 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06b419c010 con 0x7f06b4107bc0 2024-04-24T23:20:15.372 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:14.480+0000 7f06b3fff700 1 --2- 172.21.15.80:0/4141060208 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f06b4107bc0 0x7f06b419bad0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f06a40117e0 tx=0x7f06a4011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:15.372 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:14.480+0000 7f06b17fa700 1 -- 172.21.15.80:0/4141060208 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f06a4003bf0 con 0x7f06b4107bc0 2024-04-24T23:20:15.373 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:14.480+0000 7f06ba2d3700 1 -- 172.21.15.80:0/4141060208 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06b419c2a0 con 0x7f06b4107bc0 2024-04-24T23:20:15.373 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:14.480+0000 7f06b17fa700 1 -- 172.21.15.80:0/4141060208 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06a401a910 con 0x7f06b4107bc0 2024-04-24T23:20:15.373 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:14.480+0000 7f06b17fa700 1 -- 172.21.15.80:0/4141060208 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f06a4024970 con 0x7f06b4107bc0 2024-04-24T23:20:15.373 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:14.481+0000 7f06ba2d3700 1 -- 172.21.15.80:0/4141060208 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06b419c700 con 0x7f06b4107bc0 2024-04-24T23:20:15.373 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:14.481+0000 7f06b17fa700 1 -- 172.21.15.80:0/4141060208 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f06a4022020 con 0x7f06b4107bc0 2024-04-24T23:20:15.373 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:14.481+0000 7f06b17fa700 1 --2- 172.21.15.80:0/4141060208 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f069c039140 0x7f069c03b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:15.373 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:14.482+0000 7f06b17fa700 1 -- 172.21.15.80:0/4141060208 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f06a404da40 con 0x7f06b4107bc0 2024-04-24T23:20:15.373 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:14.482+0000 7f06ba2d3700 1 -- 172.21.15.80:0/4141060208 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06b4063890 con 0x7f06b4107bc0 2024-04-24T23:20:15.373 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:14.482+0000 7f06b37fe700 1 --2- 172.21.15.80:0/4141060208 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f069c039140 0x7f069c03b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:15.373 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:14.482+0000 7f06b37fe700 1 --2- 172.21.15.80:0/4141060208 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f069c039140 0x7f069c03b600 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f06a8006fd0 tx=0x7f06a8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:15.373 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:14.487+0000 7f06b17fa700 1 -- 172.21.15.80:0/4141060208 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f06a400b3e0 con 0x7f06b4107bc0 2024-04-24T23:20:15.373 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:14.800+0000 7f06ba2d3700 1 -- 172.21.15.80:0/4141060208 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f06b419f450 con 0x7f06b4107bc0 2024-04-24T23:20:15.373 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:14.802+0000 7f06b17fa700 1 -- 172.21.15.80:0/4141060208 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7f06a40172b0 con 0x7f06b4107bc0 2024-04-24T23:20:15.373 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:14.804+0000 7f06ba2d3700 1 -- 172.21.15.80:0/4141060208 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f069c039140 msgr2=0x7f069c03b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:15.374 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:14.804+0000 7f06ba2d3700 1 --2- 172.21.15.80:0/4141060208 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f069c039140 0x7f069c03b600 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f06a8006fd0 tx=0x7f06a8006e40 comp rx=0 tx=0).stop 2024-04-24T23:20:15.374 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:14.804+0000 7f06ba2d3700 1 -- 172.21.15.80:0/4141060208 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f06b4107bc0 msgr2=0x7f06b419bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:15.374 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:14.804+0000 7f06ba2d3700 1 --2- 172.21.15.80:0/4141060208 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f06b4107bc0 0x7f06b419bad0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f06a40117e0 tx=0x7f06a4011ba0 comp rx=0 tx=0).stop 2024-04-24T23:20:15.374 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:14.805+0000 7f06ba2d3700 1 -- 172.21.15.80:0/4141060208 shutdown_connections 2024-04-24T23:20:15.374 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:14.805+0000 7f06ba2d3700 1 --2- 172.21.15.80:0/4141060208 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f069c039140 0x7f069c03b600 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:15.374 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:14.805+0000 7f06ba2d3700 1 --2- 172.21.15.80:0/4141060208 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f06b4107bc0 0x7f06b419bad0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:15.374 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:14.805+0000 7f06ba2d3700 1 -- 172.21.15.80:0/4141060208 >> 172.21.15.80:0/4141060208 conn(0x7f06b41031a0 msgr2=0x7f06b4103c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:15.374 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:14.805+0000 7f06ba2d3700 1 -- 172.21.15.80:0/4141060208 shutdown_connections 2024-04-24T23:20:15.374 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr 2024-04-24T23:20:14.805+0000 7f06ba2d3700 1 -- 172.21.15.80:0/4141060208 wait complete. 2024-04-24T23:20:15.374 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-04-24T23:20:15.374 INFO:teuthology.orchestra.run.smithi080.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-04-24T23:20:15.374 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-24T23:20:15.374 INFO:teuthology.orchestra.run.smithi080.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid ec7885aa-0290-11ef-bc93-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-04-24T23:20:15.374 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-24T23:20:15.375 INFO:teuthology.orchestra.run.smithi080.stdout:Or, if you are only running a single cluster on this host: 2024-04-24T23:20:15.375 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-24T23:20:15.375 INFO:teuthology.orchestra.run.smithi080.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-04-24T23:20:15.375 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-24T23:20:15.375 INFO:teuthology.orchestra.run.smithi080.stdout:Please consider enabling telemetry to help improve Ceph: 2024-04-24T23:20:15.375 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-24T23:20:15.375 INFO:teuthology.orchestra.run.smithi080.stdout: ceph telemetry on 2024-04-24T23:20:15.375 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-24T23:20:15.375 INFO:teuthology.orchestra.run.smithi080.stdout:For more information see: 2024-04-24T23:20:15.375 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-24T23:20:15.375 INFO:teuthology.orchestra.run.smithi080.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-04-24T23:20:15.375 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-24T23:20:15.375 INFO:teuthology.orchestra.run.smithi080.stdout:Bootstrap complete. 2024-04-24T23:20:15.408 INFO:tasks.cephadm:Fetching config... 2024-04-24T23:20:15.409 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-24T23:20:15.409 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-04-24T23:20:15.424 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-04-24T23:20:15.424 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-24T23:20:15.425 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-04-24T23:20:15.480 INFO:tasks.cephadm:Fetching mon keyring... 2024-04-24T23:20:15.480 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-24T23:20:15.480 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd if=/var/lib/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/mon.a/keyring of=/dev/stdout 2024-04-24T23:20:15.544 INFO:tasks.cephadm:Fetching pub ssh key... 2024-04-24T23:20:15.544 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-24T23:20:15.544 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-04-24T23:20:15.599 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-04-24T23:20:15.600 DEBUG:teuthology.orchestra.run.smithi080:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDbNYYWP8+Aw48kkQl7GjpK+h7iccwwpwIixWyTdY4ltK8vJHs+d5mPs9p75pmCGMWht6/+QHNE8RcZcRP+w5k5tz6QxN8FaOwONv6M0vqSuGLA9twcdK1J3NtnUYULZEeCrr8+Hzq89pr5pHHVMEckiJVRu+REn0mcBbJZNKDnAridhebH0g6XaSujuKxNjhgm3mA7/DBpP7hzZJqD7V1umVkQLbaTNJ/0LZ9yH82D4X05UgAq7W1cDQl4DZEpyjZHiWWvcwkQNmMeQYfqcncS9ZSRoRY2Iz/mJZbuQNvqLb/QluOHe1mR9OeLH9QRsxyu5KaMNar0B3rtJBJa5GNDoFwDaYHkkxywRt/IMPzr9jYOvM14xorPMpHpjwV0OI59kx3EtM7wLJNGb1KKc6vM9morpK3KHWL5QoKZwveV0rl8sUkHcQLUV3H83flTCBEcS6XrjXng2aqscbmio6WrPhXGVwDl5QGrnsqjK5k7gQVcXToIeLhEzuEQSLzlvpM= ceph-ec7885aa-0290-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-24T23:20:15.681 INFO:teuthology.orchestra.run.smithi080.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDbNYYWP8+Aw48kkQl7GjpK+h7iccwwpwIixWyTdY4ltK8vJHs+d5mPs9p75pmCGMWht6/+QHNE8RcZcRP+w5k5tz6QxN8FaOwONv6M0vqSuGLA9twcdK1J3NtnUYULZEeCrr8+Hzq89pr5pHHVMEckiJVRu+REn0mcBbJZNKDnAridhebH0g6XaSujuKxNjhgm3mA7/DBpP7hzZJqD7V1umVkQLbaTNJ/0LZ9yH82D4X05UgAq7W1cDQl4DZEpyjZHiWWvcwkQNmMeQYfqcncS9ZSRoRY2Iz/mJZbuQNvqLb/QluOHe1mR9OeLH9QRsxyu5KaMNar0B3rtJBJa5GNDoFwDaYHkkxywRt/IMPzr9jYOvM14xorPMpHpjwV0OI59kx3EtM7wLJNGb1KKc6vM9morpK3KHWL5QoKZwveV0rl8sUkHcQLUV3H83flTCBEcS6XrjXng2aqscbmio6WrPhXGVwDl5QGrnsqjK5k7gQVcXToIeLhEzuEQSLzlvpM= ceph-ec7885aa-0290-11ef-bc93-c7b262605968 2024-04-24T23:20:15.697 DEBUG:teuthology.orchestra.run.smithi134:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDbNYYWP8+Aw48kkQl7GjpK+h7iccwwpwIixWyTdY4ltK8vJHs+d5mPs9p75pmCGMWht6/+QHNE8RcZcRP+w5k5tz6QxN8FaOwONv6M0vqSuGLA9twcdK1J3NtnUYULZEeCrr8+Hzq89pr5pHHVMEckiJVRu+REn0mcBbJZNKDnAridhebH0g6XaSujuKxNjhgm3mA7/DBpP7hzZJqD7V1umVkQLbaTNJ/0LZ9yH82D4X05UgAq7W1cDQl4DZEpyjZHiWWvcwkQNmMeQYfqcncS9ZSRoRY2Iz/mJZbuQNvqLb/QluOHe1mR9OeLH9QRsxyu5KaMNar0B3rtJBJa5GNDoFwDaYHkkxywRt/IMPzr9jYOvM14xorPMpHpjwV0OI59kx3EtM7wLJNGb1KKc6vM9morpK3KHWL5QoKZwveV0rl8sUkHcQLUV3H83flTCBEcS6XrjXng2aqscbmio6WrPhXGVwDl5QGrnsqjK5k7gQVcXToIeLhEzuEQSLzlvpM= ceph-ec7885aa-0290-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-24T23:20:15.757 INFO:teuthology.orchestra.run.smithi134.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDbNYYWP8+Aw48kkQl7GjpK+h7iccwwpwIixWyTdY4ltK8vJHs+d5mPs9p75pmCGMWht6/+QHNE8RcZcRP+w5k5tz6QxN8FaOwONv6M0vqSuGLA9twcdK1J3NtnUYULZEeCrr8+Hzq89pr5pHHVMEckiJVRu+REn0mcBbJZNKDnAridhebH0g6XaSujuKxNjhgm3mA7/DBpP7hzZJqD7V1umVkQLbaTNJ/0LZ9yH82D4X05UgAq7W1cDQl4DZEpyjZHiWWvcwkQNmMeQYfqcncS9ZSRoRY2Iz/mJZbuQNvqLb/QluOHe1mR9OeLH9QRsxyu5KaMNar0B3rtJBJa5GNDoFwDaYHkkxywRt/IMPzr9jYOvM14xorPMpHpjwV0OI59kx3EtM7wLJNGb1KKc6vM9morpK3KHWL5QoKZwveV0rl8sUkHcQLUV3H83flTCBEcS6XrjXng2aqscbmio6WrPhXGVwDl5QGrnsqjK5k7gQVcXToIeLhEzuEQSLzlvpM= ceph-ec7885aa-0290-11ef-bc93-c7b262605968 2024-04-24T23:20:15.769 DEBUG:teuthology.orchestra.run.smithi148:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDbNYYWP8+Aw48kkQl7GjpK+h7iccwwpwIixWyTdY4ltK8vJHs+d5mPs9p75pmCGMWht6/+QHNE8RcZcRP+w5k5tz6QxN8FaOwONv6M0vqSuGLA9twcdK1J3NtnUYULZEeCrr8+Hzq89pr5pHHVMEckiJVRu+REn0mcBbJZNKDnAridhebH0g6XaSujuKxNjhgm3mA7/DBpP7hzZJqD7V1umVkQLbaTNJ/0LZ9yH82D4X05UgAq7W1cDQl4DZEpyjZHiWWvcwkQNmMeQYfqcncS9ZSRoRY2Iz/mJZbuQNvqLb/QluOHe1mR9OeLH9QRsxyu5KaMNar0B3rtJBJa5GNDoFwDaYHkkxywRt/IMPzr9jYOvM14xorPMpHpjwV0OI59kx3EtM7wLJNGb1KKc6vM9morpK3KHWL5QoKZwveV0rl8sUkHcQLUV3H83flTCBEcS6XrjXng2aqscbmio6WrPhXGVwDl5QGrnsqjK5k7gQVcXToIeLhEzuEQSLzlvpM= ceph-ec7885aa-0290-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-24T23:20:15.809 INFO:teuthology.orchestra.run.smithi148.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDbNYYWP8+Aw48kkQl7GjpK+h7iccwwpwIixWyTdY4ltK8vJHs+d5mPs9p75pmCGMWht6/+QHNE8RcZcRP+w5k5tz6QxN8FaOwONv6M0vqSuGLA9twcdK1J3NtnUYULZEeCrr8+Hzq89pr5pHHVMEckiJVRu+REn0mcBbJZNKDnAridhebH0g6XaSujuKxNjhgm3mA7/DBpP7hzZJqD7V1umVkQLbaTNJ/0LZ9yH82D4X05UgAq7W1cDQl4DZEpyjZHiWWvcwkQNmMeQYfqcncS9ZSRoRY2Iz/mJZbuQNvqLb/QluOHe1mR9OeLH9QRsxyu5KaMNar0B3rtJBJa5GNDoFwDaYHkkxywRt/IMPzr9jYOvM14xorPMpHpjwV0OI59kx3EtM7wLJNGb1KKc6vM9morpK3KHWL5QoKZwveV0rl8sUkHcQLUV3H83flTCBEcS6XrjXng2aqscbmio6WrPhXGVwDl5QGrnsqjK5k7gQVcXToIeLhEzuEQSLzlvpM= ceph-ec7885aa-0290-11ef-bc93-c7b262605968 2024-04-24T23:20:15.821 DEBUG:teuthology.orchestra.run.smithi080:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ec7885aa-0290-11ef-bc93-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-04-24T23:20:16.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:15 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/4141060208' entity='client.admin' 2024-04-24T23:20:16.062 INFO:teuthology.orchestra.run.smithi080.stderr:Inferring config /var/lib/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:20:18.651 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:18.628+0000 7fc3af914700 1 -- 172.21.15.80:0/4288768431 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc3a8101c90 msgr2=0x7fc3a81020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:18.651 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:18.628+0000 7fc3af914700 1 --2- 172.21.15.80:0/4288768431 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc3a8101c90 0x7fc3a81020b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc3a4009910 tx=0x7fc3a4009c20 comp rx=0 tx=0).stop 2024-04-24T23:20:18.651 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:18.628+0000 7fc3af914700 1 -- 172.21.15.80:0/4288768431 shutdown_connections 2024-04-24T23:20:18.651 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:18.628+0000 7fc3af914700 1 --2- 172.21.15.80:0/4288768431 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc3a8101c90 0x7fc3a81020b0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:18.651 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:18.628+0000 7fc3af914700 1 -- 172.21.15.80:0/4288768431 >> 172.21.15.80:0/4288768431 conn(0x7fc3a80fd230 msgr2=0x7fc3a80ff670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:18.651 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:18.628+0000 7fc3af914700 1 -- 172.21.15.80:0/4288768431 shutdown_connections 2024-04-24T23:20:18.651 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:18.628+0000 7fc3af914700 1 -- 172.21.15.80:0/4288768431 wait complete. 2024-04-24T23:20:18.651 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:18.628+0000 7fc3af914700 1 Processor -- start 2024-04-24T23:20:18.651 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:18.629+0000 7fc3af914700 1 -- start start 2024-04-24T23:20:18.651 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:18.629+0000 7fc3af914700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc3a8101c90 0x7fc3a81995c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:18.651 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:18.629+0000 7fc3af914700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fc3a8107d20 con 0x7fc3a8101c90 2024-04-24T23:20:18.651 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:18.629+0000 7fc3ad6b0700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc3a8101c90 0x7fc3a81995c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:18.652 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:18.629+0000 7fc3ad6b0700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc3a8101c90 0x7fc3a81995c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:55606/0 (socket says 172.21.15.80:55606) 2024-04-24T23:20:18.652 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:18.629+0000 7fc3ad6b0700 1 -- 172.21.15.80:0/221665078 learned_addr learned my addr 172.21.15.80:0/221665078 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:20:18.652 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:18.629+0000 7fc3ad6b0700 1 -- 172.21.15.80:0/221665078 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3a8199b00 con 0x7fc3a8101c90 2024-04-24T23:20:18.652 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:18.629+0000 7fc3ad6b0700 1 --2- 172.21.15.80:0/221665078 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc3a8101c90 0x7fc3a81995c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc3a4005850 tx=0x7fc3a4005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:18.652 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:18.629+0000 7fc39e7fc700 1 -- 172.21.15.80:0/221665078 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc3a4026070 con 0x7fc3a8101c90 2024-04-24T23:20:18.652 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:18.629+0000 7fc3af914700 1 -- 172.21.15.80:0/221665078 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3a8199d90 con 0x7fc3a8101c90 2024-04-24T23:20:18.652 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:18.629+0000 7fc39e7fc700 1 -- 172.21.15.80:0/221665078 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3a40044a0 con 0x7fc3a8101c90 2024-04-24T23:20:18.652 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:18.629+0000 7fc39e7fc700 1 -- 172.21.15.80:0/221665078 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc3a4003ab0 con 0x7fc3a8101c90 2024-04-24T23:20:18.652 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:18.629+0000 7fc3af914700 1 -- 172.21.15.80:0/221665078 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3a819ca80 con 0x7fc3a8101c90 2024-04-24T23:20:18.652 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:18.630+0000 7fc39e7fc700 1 -- 172.21.15.80:0/221665078 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fc3a4029070 con 0x7fc3a8101c90 2024-04-24T23:20:18.652 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:18.630+0000 7fc39e7fc700 1 --2- 172.21.15.80:0/221665078 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fc3940390f0 0x7fc39403b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:18.652 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:18.630+0000 7fc39e7fc700 1 -- 172.21.15.80:0/221665078 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fc3a401e070 con 0x7fc3a8101c90 2024-04-24T23:20:18.652 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:18.630+0000 7fc3af914700 1 -- 172.21.15.80:0/221665078 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3a81084a0 con 0x7fc3a8101c90 2024-04-24T23:20:18.652 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:18.631+0000 7fc3aceaf700 1 --2- 172.21.15.80:0/221665078 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fc3940390f0 0x7fc39403b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:18.653 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:18.631+0000 7fc3aceaf700 1 --2- 172.21.15.80:0/221665078 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fc3940390f0 0x7fc39403b5b0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc398006fd0 tx=0x7fc398006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:18.653 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:18.636+0000 7fc39e7fc700 1 -- 172.21.15.80:0/221665078 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fc3a4030020 con 0x7fc3a8101c90 2024-04-24T23:20:18.866 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:18.865+0000 7fc3af914700 1 -- 172.21.15.80:0/221665078 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7fc3a802dba0 con 0x7fc3a8101c90 2024-04-24T23:20:18.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:18.867+0000 7fc39e7fc700 1 -- 172.21.15.80:0/221665078 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7fc3a4013070 con 0x7fc3a8101c90 2024-04-24T23:20:18.871 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:18.870+0000 7fc3af914700 1 -- 172.21.15.80:0/221665078 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fc3940390f0 msgr2=0x7fc39403b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:18.871 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:18.870+0000 7fc3af914700 1 --2- 172.21.15.80:0/221665078 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fc3940390f0 0x7fc39403b5b0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc398006fd0 tx=0x7fc398006e40 comp rx=0 tx=0).stop 2024-04-24T23:20:18.871 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:18.870+0000 7fc3af914700 1 -- 172.21.15.80:0/221665078 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc3a8101c90 msgr2=0x7fc3a81995c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:18.871 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:18.870+0000 7fc3af914700 1 --2- 172.21.15.80:0/221665078 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc3a8101c90 0x7fc3a81995c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc3a4005850 tx=0x7fc3a4005030 comp rx=0 tx=0).stop 2024-04-24T23:20:18.871 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:18.870+0000 7fc3af914700 1 -- 172.21.15.80:0/221665078 shutdown_connections 2024-04-24T23:20:18.871 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:18.870+0000 7fc3af914700 1 --2- 172.21.15.80:0/221665078 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fc3940390f0 0x7fc39403b5b0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:18.871 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:18.870+0000 7fc3af914700 1 --2- 172.21.15.80:0/221665078 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc3a8101c90 0x7fc3a81995c0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:18.871 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:18.870+0000 7fc3af914700 1 -- 172.21.15.80:0/221665078 >> 172.21.15.80:0/221665078 conn(0x7fc3a80fd230 msgr2=0x7fc3a80ff670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:18.871 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:18.870+0000 7fc3af914700 1 -- 172.21.15.80:0/221665078 shutdown_connections 2024-04-24T23:20:18.872 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:18.870+0000 7fc3af914700 1 -- 172.21.15.80:0/221665078 wait complete. 2024-04-24T23:20:19.952 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-04-24T23:20:19.952 DEBUG:teuthology.orchestra.run.smithi080:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ec7885aa-0290-11ef-bc93-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-04-24T23:20:20.185 INFO:teuthology.orchestra.run.smithi080.stderr:Inferring config /var/lib/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:20:20.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:19 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/221665078' entity='client.admin' 2024-04-24T23:20:20.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:19 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:20:20.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:19 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:20:20.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:19 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi080", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:20:20.290 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:19 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:20:20.290 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:19 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:20:20.290 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:19 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:20:21.683 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:21.680+0000 7ff741805700 1 -- 172.21.15.80:0/2476741478 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff73c100b90 msgr2=0x7ff73c100fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:21.683 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:21.680+0000 7ff741805700 1 --2- 172.21.15.80:0/2476741478 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff73c100b90 0x7ff73c100fb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff72c009910 tx=0x7ff72c009c20 comp rx=0 tx=0).stop 2024-04-24T23:20:21.683 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:21.680+0000 7ff741805700 1 -- 172.21.15.80:0/2476741478 shutdown_connections 2024-04-24T23:20:21.683 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:21.680+0000 7ff741805700 1 --2- 172.21.15.80:0/2476741478 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff73c100b90 0x7ff73c100fb0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:21.683 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:21.680+0000 7ff741805700 1 -- 172.21.15.80:0/2476741478 >> 172.21.15.80:0/2476741478 conn(0x7ff73c0fc130 msgr2=0x7ff73c0fe570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:21.683 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:21.680+0000 7ff741805700 1 -- 172.21.15.80:0/2476741478 shutdown_connections 2024-04-24T23:20:21.683 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:21.680+0000 7ff741805700 1 -- 172.21.15.80:0/2476741478 wait complete. 2024-04-24T23:20:21.683 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:21.680+0000 7ff741805700 1 Processor -- start 2024-04-24T23:20:21.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:21.680+0000 7ff741805700 1 -- start start 2024-04-24T23:20:21.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:21.681+0000 7ff741805700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff73c100b90 0x7ff73c06a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:21.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:21.681+0000 7ff741805700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7ff73c106c20 con 0x7ff73c100b90 2024-04-24T23:20:21.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:21.681+0000 7ff73affd700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff73c100b90 0x7ff73c06a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:21.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:21.681+0000 7ff73affd700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff73c100b90 0x7ff73c06a0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:55626/0 (socket says 172.21.15.80:55626) 2024-04-24T23:20:21.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:21.681+0000 7ff73affd700 1 -- 172.21.15.80:0/3911833597 learned_addr learned my addr 172.21.15.80:0/3911833597 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:20:21.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:21.681+0000 7ff73affd700 1 -- 172.21.15.80:0/3911833597 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff73c19a0f0 con 0x7ff73c100b90 2024-04-24T23:20:21.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:21.681+0000 7ff73affd700 1 --2- 172.21.15.80:0/3911833597 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff73c100b90 0x7ff73c06a0d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff72c005850 tx=0x7ff72c005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:21.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:21.681+0000 7ff740803700 1 -- 172.21.15.80:0/3911833597 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff72c026070 con 0x7ff73c100b90 2024-04-24T23:20:21.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:21.681+0000 7ff741805700 1 -- 172.21.15.80:0/3911833597 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff73c19a2c0 con 0x7ff73c100b90 2024-04-24T23:20:21.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:21.681+0000 7ff740803700 1 -- 172.21.15.80:0/3911833597 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff72c0044a0 con 0x7ff73c100b90 2024-04-24T23:20:21.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:21.681+0000 7ff740803700 1 -- 172.21.15.80:0/3911833597 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff72c003ab0 con 0x7ff73c100b90 2024-04-24T23:20:21.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:21.682+0000 7ff741805700 1 -- 172.21.15.80:0/3911833597 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff73c19a660 con 0x7ff73c100b90 2024-04-24T23:20:21.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:21.682+0000 7ff740803700 1 -- 172.21.15.80:0/3911833597 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7ff72c029070 con 0x7ff73c100b90 2024-04-24T23:20:21.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:21.682+0000 7ff740803700 1 --2- 172.21.15.80:0/3911833597 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7ff7280390f0 0x7ff72803b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:21.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:21.682+0000 7ff741805700 1 -- 172.21.15.80:0/3911833597 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff73c1073a0 con 0x7ff73c100b90 2024-04-24T23:20:21.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:21.683+0000 7ff740803700 1 -- 172.21.15.80:0/3911833597 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff72c01e070 con 0x7ff73c100b90 2024-04-24T23:20:21.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:21.683+0000 7ff7327fc700 1 --2- 172.21.15.80:0/3911833597 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7ff7280390f0 0x7ff72803b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:21.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:21.683+0000 7ff7327fc700 1 --2- 172.21.15.80:0/3911833597 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7ff7280390f0 0x7ff72803b5b0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff724006fd0 tx=0x7ff724006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:21.688 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:21.688+0000 7ff740803700 1 -- 172.21.15.80:0/3911833597 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff72c013070 con 0x7ff73c100b90 2024-04-24T23:20:21.888 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:21.887+0000 7ff741805700 1 -- 172.21.15.80:0/3911833597 --> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7ff73c02ddf0 con 0x7ff7280390f0 2024-04-24T23:20:21.891 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:21.890+0000 7ff740803700 1 -- 172.21.15.80:0/3911833597 <== mgr.14152 v2:172.21.15.80:6800/3830596922 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff73c02ddf0 con 0x7ff7280390f0 2024-04-24T23:20:21.894 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:21.892+0000 7ff741805700 1 -- 172.21.15.80:0/3911833597 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7ff7280390f0 msgr2=0x7ff72803b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:21.895 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:21.892+0000 7ff741805700 1 --2- 172.21.15.80:0/3911833597 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7ff7280390f0 0x7ff72803b5b0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff724006fd0 tx=0x7ff724006e40 comp rx=0 tx=0).stop 2024-04-24T23:20:21.895 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:21.892+0000 7ff741805700 1 -- 172.21.15.80:0/3911833597 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff73c100b90 msgr2=0x7ff73c06a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:21.895 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:21.892+0000 7ff741805700 1 --2- 172.21.15.80:0/3911833597 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff73c100b90 0x7ff73c06a0d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff72c005850 tx=0x7ff72c005030 comp rx=0 tx=0).stop 2024-04-24T23:20:21.895 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:21.892+0000 7ff741805700 1 -- 172.21.15.80:0/3911833597 shutdown_connections 2024-04-24T23:20:21.895 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:21.892+0000 7ff741805700 1 --2- 172.21.15.80:0/3911833597 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7ff7280390f0 0x7ff72803b5b0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:21.895 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:21.892+0000 7ff741805700 1 --2- 172.21.15.80:0/3911833597 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff73c100b90 0x7ff73c06a0d0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:21.895 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:21.892+0000 7ff741805700 1 -- 172.21.15.80:0/3911833597 >> 172.21.15.80:0/3911833597 conn(0x7ff73c0fc130 msgr2=0x7ff73c0fcbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:21.895 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:21.893+0000 7ff741805700 1 -- 172.21.15.80:0/3911833597 shutdown_connections 2024-04-24T23:20:21.895 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:21.893+0000 7ff741805700 1 -- 172.21.15.80:0/3911833597 wait complete. 2024-04-24T23:20:22.648 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi134 2024-04-24T23:20:22.649 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-24T23:20:22.649 DEBUG:teuthology.orchestra.run.smithi134:> dd of=/etc/ceph/ceph.conf 2024-04-24T23:20:22.668 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-24T23:20:22.668 DEBUG:teuthology.orchestra.run.smithi134:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-24T23:20:22.727 INFO:tasks.cephadm:Adding host smithi134 to orchestrator... 2024-04-24T23:20:22.727 DEBUG:teuthology.orchestra.run.smithi080:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ec7885aa-0290-11ef-bc93-c7b262605968 -- ceph orch host add smithi134 2024-04-24T23:20:22.965 INFO:teuthology.orchestra.run.smithi080.stderr:Inferring config /var/lib/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:20:23.205 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:22 smithi080 ceph-mon[29152]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:20:23.205 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:22 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:20:23.205 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:22 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:20:23.205 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:22 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:20:23.206 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:22 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:20:23.206 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:22 smithi080 ceph-mon[29152]: Updating smithi080:/etc/ceph/ceph.conf 2024-04-24T23:20:23.206 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:22 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:20:23.206 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:22 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:20:23.206 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:22 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:20:24.152 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:23 smithi080 ceph-mon[29152]: Updating smithi080:/var/lib/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T23:20:24.153 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:23 smithi080 ceph-mon[29152]: Updating smithi080:/etc/ceph/ceph.client.admin.keyring 2024-04-24T23:20:24.153 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:23 smithi080 ceph-mon[29152]: Updating smithi080:/var/lib/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T23:20:24.380 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:24.377+0000 7f5850932700 1 -- 172.21.15.80:0/1835016589 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f584c106bf0 msgr2=0x7f584c106fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:24.380 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:24.377+0000 7f5850932700 1 --2- 172.21.15.80:0/1835016589 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f584c106bf0 0x7f584c106fd0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5840009910 tx=0x7f5840009c20 comp rx=0 tx=0).stop 2024-04-24T23:20:24.380 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:24.377+0000 7f5850932700 1 -- 172.21.15.80:0/1835016589 shutdown_connections 2024-04-24T23:20:24.380 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:24.377+0000 7f5850932700 1 --2- 172.21.15.80:0/1835016589 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f584c106bf0 0x7f584c106fd0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:24.380 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:24.377+0000 7f5850932700 1 -- 172.21.15.80:0/1835016589 >> 172.21.15.80:0/1835016589 conn(0x7f584c0fc630 msgr2=0x7f584c0fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:24.380 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:24.377+0000 7f5850932700 1 -- 172.21.15.80:0/1835016589 shutdown_connections 2024-04-24T23:20:24.381 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:24.377+0000 7f5850932700 1 -- 172.21.15.80:0/1835016589 wait complete. 2024-04-24T23:20:24.383 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:24.378+0000 7f5850932700 1 Processor -- start 2024-04-24T23:20:24.383 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:24.378+0000 7f5850932700 1 -- start start 2024-04-24T23:20:24.383 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:24.378+0000 7f5850932700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f584c106bf0 0x7f584c19d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:24.383 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:24.378+0000 7f5850932700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f584c10b1b0 con 0x7f584c106bf0 2024-04-24T23:20:24.383 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:24.378+0000 7f584a59c700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f584c106bf0 0x7f584c19d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:24.383 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:24.378+0000 7f584a59c700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f584c106bf0 0x7f584c19d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:45148/0 (socket says 172.21.15.80:45148) 2024-04-24T23:20:24.383 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:24.378+0000 7f584a59c700 1 -- 172.21.15.80:0/973594404 learned_addr learned my addr 172.21.15.80:0/973594404 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:20:24.383 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:24.379+0000 7f584a59c700 1 -- 172.21.15.80:0/973594404 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f584c19d990 con 0x7f584c106bf0 2024-04-24T23:20:24.383 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:24.379+0000 7f584a59c700 1 --2- 172.21.15.80:0/973594404 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f584c106bf0 0x7f584c19d450 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5840006010 tx=0x7f5840005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:24.384 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:24.379+0000 7f583b7fe700 1 -- 172.21.15.80:0/973594404 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5840026070 con 0x7f584c106bf0 2024-04-24T23:20:24.384 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:24.379+0000 7f583b7fe700 1 -- 172.21.15.80:0/973594404 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f584000bc30 con 0x7f584c106bf0 2024-04-24T23:20:24.384 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:24.379+0000 7f583b7fe700 1 -- 172.21.15.80:0/973594404 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f58400039d0 con 0x7f584c106bf0 2024-04-24T23:20:24.384 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:24.379+0000 7f5850932700 1 -- 172.21.15.80:0/973594404 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f584c19dc20 con 0x7f584c106bf0 2024-04-24T23:20:24.384 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:24.379+0000 7f5850932700 1 -- 172.21.15.80:0/973594404 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f584c197610 con 0x7f584c106bf0 2024-04-24T23:20:24.384 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:24.380+0000 7f583b7fe700 1 -- 172.21.15.80:0/973594404 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f5840029070 con 0x7f584c106bf0 2024-04-24T23:20:24.384 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:24.380+0000 7f583b7fe700 1 --2- 172.21.15.80:0/973594404 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f58340390f0 0x7f583403b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:24.384 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:24.380+0000 7f5850932700 1 -- 172.21.15.80:0/973594404 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f584c1021f0 con 0x7f584c106bf0 2024-04-24T23:20:24.384 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:24.380+0000 7f583b7fe700 1 -- 172.21.15.80:0/973594404 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f584001e070 con 0x7f584c106bf0 2024-04-24T23:20:24.384 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:24.380+0000 7f5849d9b700 1 --2- 172.21.15.80:0/973594404 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f58340390f0 0x7f583403b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:24.384 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:24.380+0000 7f5849d9b700 1 --2- 172.21.15.80:0/973594404 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f58340390f0 0x7f583403b5b0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f583c006fd0 tx=0x7f583c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:24.387 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:24.385+0000 7f583b7fe700 1 -- 172.21.15.80:0/973594404 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f5840013070 con 0x7f584c106bf0 2024-04-24T23:20:24.587 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:24.585+0000 7f5850932700 1 -- 172.21.15.80:0/973594404 --> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi134", "target": ["mon-mgr", ""]}) -- 0x7f584c063890 con 0x7f58340390f0 2024-04-24T23:20:25.539 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:25 smithi080 ceph-mon[29152]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi134", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:20:26.440 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:26 smithi080 ceph-mon[29152]: Deploying cephadm binary to smithi134 2024-04-24T23:20:28.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:27 smithi080 ceph-mon[29152]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:20:28.702 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:28.699+0000 7f583b7fe700 1 -- 172.21.15.80:0/973594404 <== mgr.14152 v2:172.21.15.80:6800/3830596922 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f584c063890 con 0x7f58340390f0 2024-04-24T23:20:28.702 INFO:teuthology.orchestra.run.smithi080.stdout:Added host 'smithi134' with addr '172.21.15.134' 2024-04-24T23:20:28.704 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:28.702+0000 7f5850932700 1 -- 172.21.15.80:0/973594404 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f58340390f0 msgr2=0x7f583403b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:28.704 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:28.702+0000 7f5850932700 1 --2- 172.21.15.80:0/973594404 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f58340390f0 0x7f583403b5b0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f583c006fd0 tx=0x7f583c006e40 comp rx=0 tx=0).stop 2024-04-24T23:20:28.704 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:28.702+0000 7f5850932700 1 -- 172.21.15.80:0/973594404 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f584c106bf0 msgr2=0x7f584c19d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:28.704 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:28.702+0000 7f5850932700 1 --2- 172.21.15.80:0/973594404 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f584c106bf0 0x7f584c19d450 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5840006010 tx=0x7f5840005030 comp rx=0 tx=0).stop 2024-04-24T23:20:28.704 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:28.702+0000 7f5850932700 1 -- 172.21.15.80:0/973594404 shutdown_connections 2024-04-24T23:20:28.704 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:28.702+0000 7f5850932700 1 --2- 172.21.15.80:0/973594404 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f58340390f0 0x7f583403b5b0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:28.705 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:28.702+0000 7f5850932700 1 --2- 172.21.15.80:0/973594404 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f584c106bf0 0x7f584c19d450 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:28.705 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:28.702+0000 7f5850932700 1 -- 172.21.15.80:0/973594404 >> 172.21.15.80:0/973594404 conn(0x7f584c0fc630 msgr2=0x7f584c069ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:28.705 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:28.702+0000 7f5850932700 1 -- 172.21.15.80:0/973594404 shutdown_connections 2024-04-24T23:20:28.705 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:28.702+0000 7f5850932700 1 -- 172.21.15.80:0/973594404 wait complete. 2024-04-24T23:20:29.273 DEBUG:teuthology.orchestra.run.smithi080:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ec7885aa-0290-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-24T23:20:29.510 INFO:teuthology.orchestra.run.smithi080.stderr:Inferring config /var/lib/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:20:29.884 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:29 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:20:29.884 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:29 smithi080 ceph-mon[29152]: Added host smithi134 2024-04-24T23:20:29.885 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:29 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:20:29.885 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:29 smithi080 ceph-mon[29152]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:20:29.885 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:29 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:20:30.917 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:30 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:20:30.945 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:30.942+0000 7f1ffa1ae700 1 -- 172.21.15.80:0/1097652614 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1ff4101eb0 msgr2=0x7f1ff4102290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:30.945 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:30.942+0000 7f1ffa1ae700 1 --2- 172.21.15.80:0/1097652614 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1ff4101eb0 0x7f1ff4102290 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1fe4009910 tx=0x7f1fe4009c20 comp rx=0 tx=0).stop 2024-04-24T23:20:30.945 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:30.942+0000 7f1ffa1ae700 1 -- 172.21.15.80:0/1097652614 shutdown_connections 2024-04-24T23:20:30.945 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:30.942+0000 7f1ffa1ae700 1 --2- 172.21.15.80:0/1097652614 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1ff4101eb0 0x7f1ff4102290 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:30.945 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:30.942+0000 7f1ffa1ae700 1 -- 172.21.15.80:0/1097652614 >> 172.21.15.80:0/1097652614 conn(0x7f1ff40fd760 msgr2=0x7f1ff40ffb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:30.945 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:30.942+0000 7f1ffa1ae700 1 -- 172.21.15.80:0/1097652614 shutdown_connections 2024-04-24T23:20:30.945 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:30.943+0000 7f1ffa1ae700 1 -- 172.21.15.80:0/1097652614 wait complete. 2024-04-24T23:20:30.947 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:30.943+0000 7f1ffa1ae700 1 Processor -- start 2024-04-24T23:20:30.947 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:30.943+0000 7f1ffa1ae700 1 -- start start 2024-04-24T23:20:30.947 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:30.944+0000 7f1ffa1ae700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1ff4101eb0 0x7f1ff419d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:30.947 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:30.944+0000 7f1ffa1ae700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f1ff410c280 con 0x7f1ff4101eb0 2024-04-24T23:20:30.947 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:30.944+0000 7f1ff37fe700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1ff4101eb0 0x7f1ff419d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:30.947 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:30.944+0000 7f1ff37fe700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1ff4101eb0 0x7f1ff419d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:45172/0 (socket says 172.21.15.80:45172) 2024-04-24T23:20:30.947 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:30.944+0000 7f1ff37fe700 1 -- 172.21.15.80:0/1293692934 learned_addr learned my addr 172.21.15.80:0/1293692934 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:20:30.948 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:30.944+0000 7f1ff37fe700 1 -- 172.21.15.80:0/1293692934 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ff419d930 con 0x7f1ff4101eb0 2024-04-24T23:20:30.948 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:30.944+0000 7f1ff37fe700 1 --2- 172.21.15.80:0/1293692934 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1ff4101eb0 0x7f1ff419d3f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1fe4006010 tx=0x7f1fe4005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:30.948 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:30.944+0000 7f1ff0ff9700 1 -- 172.21.15.80:0/1293692934 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1fe4026070 con 0x7f1ff4101eb0 2024-04-24T23:20:30.948 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:30.944+0000 7f1ffa1ae700 1 -- 172.21.15.80:0/1293692934 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ff419dc20 con 0x7f1ff4101eb0 2024-04-24T23:20:30.948 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:30.944+0000 7f1ff0ff9700 1 -- 172.21.15.80:0/1293692934 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1fe400bbd0 con 0x7f1ff4101eb0 2024-04-24T23:20:30.948 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:30.944+0000 7f1ff0ff9700 1 -- 172.21.15.80:0/1293692934 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1fe4003910 con 0x7f1ff4101eb0 2024-04-24T23:20:30.948 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:30.944+0000 7f1ffa1ae700 1 -- 172.21.15.80:0/1293692934 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ff41975b0 con 0x7f1ff4101eb0 2024-04-24T23:20:30.948 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:30.945+0000 7f1ffa1ae700 1 -- 172.21.15.80:0/1293692934 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ff4103390 con 0x7f1ff4101eb0 2024-04-24T23:20:30.948 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:30.945+0000 7f1ff0ff9700 1 -- 172.21.15.80:0/1293692934 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f1fe4029070 con 0x7f1ff4101eb0 2024-04-24T23:20:30.949 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:30.945+0000 7f1ff0ff9700 1 --2- 172.21.15.80:0/1293692934 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f1fdc0390f0 0x7f1fdc03b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:30.949 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:30.946+0000 7f1ff0ff9700 1 -- 172.21.15.80:0/1293692934 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f1fe401e070 con 0x7f1ff4101eb0 2024-04-24T23:20:30.949 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:30.946+0000 7f1ff2ffd700 1 --2- 172.21.15.80:0/1293692934 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f1fdc0390f0 0x7f1fdc03b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:30.949 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:30.946+0000 7f1ff2ffd700 1 --2- 172.21.15.80:0/1293692934 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f1fdc0390f0 0x7f1fdc03b5b0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f1fe8006fd0 tx=0x7f1fe8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:30.952 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:30.950+0000 7f1ff0ff9700 1 -- 172.21.15.80:0/1293692934 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f1fe4013070 con 0x7f1ff4101eb0 2024-04-24T23:20:31.151 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:31.149+0000 7f1ffa1ae700 1 -- 172.21.15.80:0/1293692934 --> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1ff4198ae0 con 0x7f1fdc0390f0 2024-04-24T23:20:31.152 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:31.150+0000 7f1ff0ff9700 1 -- 172.21.15.80:0/1293692934 <== mgr.14152 v2:172.21.15.80:6800/3830596922 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+160 (secure 0 0 0) 0x7f1ff4198ae0 con 0x7f1fdc0390f0 2024-04-24T23:20:31.152 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-24T23:20:31.153 INFO:teuthology.orchestra.run.smithi080.stdout:[{"addr": "172.21.15.80", "hostname": "smithi080", "labels": [], "status": ""}, {"addr": "172.21.15.134", "hostname": "smithi134", "labels": [], "status": ""}] 2024-04-24T23:20:31.155 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:31.152+0000 7f1ffa1ae700 1 -- 172.21.15.80:0/1293692934 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f1fdc0390f0 msgr2=0x7f1fdc03b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:31.155 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:31.152+0000 7f1ffa1ae700 1 --2- 172.21.15.80:0/1293692934 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f1fdc0390f0 0x7f1fdc03b5b0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f1fe8006fd0 tx=0x7f1fe8006e40 comp rx=0 tx=0).stop 2024-04-24T23:20:31.155 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:31.152+0000 7f1ffa1ae700 1 -- 172.21.15.80:0/1293692934 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1ff4101eb0 msgr2=0x7f1ff419d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:31.155 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:31.152+0000 7f1ffa1ae700 1 --2- 172.21.15.80:0/1293692934 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1ff4101eb0 0x7f1ff419d3f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1fe4006010 tx=0x7f1fe4005030 comp rx=0 tx=0).stop 2024-04-24T23:20:31.155 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:31.152+0000 7f1ffa1ae700 1 -- 172.21.15.80:0/1293692934 shutdown_connections 2024-04-24T23:20:31.156 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:31.152+0000 7f1ffa1ae700 1 --2- 172.21.15.80:0/1293692934 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f1fdc0390f0 0x7f1fdc03b5b0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:31.156 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:31.152+0000 7f1ffa1ae700 1 --2- 172.21.15.80:0/1293692934 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1ff4101eb0 0x7f1ff419d3f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:31.156 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:31.152+0000 7f1ffa1ae700 1 -- 172.21.15.80:0/1293692934 >> 172.21.15.80:0/1293692934 conn(0x7f1ff40fd760 msgr2=0x7f1ff4103880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:31.156 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:31.153+0000 7f1ffa1ae700 1 -- 172.21.15.80:0/1293692934 shutdown_connections 2024-04-24T23:20:31.156 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:31.153+0000 7f1ffa1ae700 1 -- 172.21.15.80:0/1293692934 wait complete. 2024-04-24T23:20:31.789 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:31 smithi080 ceph-mon[29152]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:20:31.789 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:31 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:20:31.825 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi148 2024-04-24T23:20:31.826 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-24T23:20:31.826 DEBUG:teuthology.orchestra.run.smithi148:> dd of=/etc/ceph/ceph.conf 2024-04-24T23:20:31.844 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-24T23:20:31.844 DEBUG:teuthology.orchestra.run.smithi148:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-24T23:20:31.902 INFO:tasks.cephadm:Adding host smithi148 to orchestrator... 2024-04-24T23:20:31.902 DEBUG:teuthology.orchestra.run.smithi080:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ec7885aa-0290-11ef-bc93-c7b262605968 -- ceph orch host add smithi148 2024-04-24T23:20:32.135 INFO:teuthology.orchestra.run.smithi080.stderr:Inferring config /var/lib/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:20:32.789 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:32 smithi080 ceph-mon[29152]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T23:20:33.641 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:33.638+0000 7fda399f7700 1 -- 172.21.15.80:0/4209798622 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fda34102b30 msgr2=0x7fda34102f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:33.641 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:33.638+0000 7fda399f7700 1 --2- 172.21.15.80:0/4209798622 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fda34102b30 0x7fda34102f10 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fda24009910 tx=0x7fda24009c20 comp rx=0 tx=0).stop 2024-04-24T23:20:33.641 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:33.638+0000 7fda399f7700 1 -- 172.21.15.80:0/4209798622 shutdown_connections 2024-04-24T23:20:33.641 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:33.638+0000 7fda399f7700 1 --2- 172.21.15.80:0/4209798622 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fda34102b30 0x7fda34102f10 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:33.641 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:33.638+0000 7fda399f7700 1 -- 172.21.15.80:0/4209798622 >> 172.21.15.80:0/4209798622 conn(0x7fda340fbda0 msgr2=0x7fda340fe1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:33.641 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:33.638+0000 7fda399f7700 1 -- 172.21.15.80:0/4209798622 shutdown_connections 2024-04-24T23:20:33.641 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:33.638+0000 7fda399f7700 1 -- 172.21.15.80:0/4209798622 wait complete. 2024-04-24T23:20:33.642 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:33.639+0000 7fda399f7700 1 Processor -- start 2024-04-24T23:20:33.642 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:33.639+0000 7fda399f7700 1 -- start start 2024-04-24T23:20:33.643 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:33.639+0000 7fda399f7700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fda34102b30 0x7fda34198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:33.643 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:33.639+0000 7fda399f7700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fda3410b2e0 con 0x7fda34102b30 2024-04-24T23:20:33.644 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:33.639+0000 7fda32ffd700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fda34102b30 0x7fda34198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:33.644 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:33.639+0000 7fda32ffd700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fda34102b30 0x7fda34198670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:58356/0 (socket says 172.21.15.80:58356) 2024-04-24T23:20:33.644 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:33.639+0000 7fda32ffd700 1 -- 172.21.15.80:0/1853878915 learned_addr learned my addr 172.21.15.80:0/1853878915 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:20:33.644 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:33.640+0000 7fda32ffd700 1 -- 172.21.15.80:0/1853878915 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda34198bb0 con 0x7fda34102b30 2024-04-24T23:20:33.644 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:33.640+0000 7fda32ffd700 1 --2- 172.21.15.80:0/1853878915 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fda34102b30 0x7fda34198670 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fda24004f80 tx=0x7fda24005060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:33.644 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:33.640+0000 7fda389f5700 1 -- 172.21.15.80:0/1853878915 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fda24026070 con 0x7fda34102b30 2024-04-24T23:20:33.644 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:33.640+0000 7fda389f5700 1 -- 172.21.15.80:0/1853878915 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda2400b750 con 0x7fda34102b30 2024-04-24T23:20:33.644 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:33.640+0000 7fda399f7700 1 -- 172.21.15.80:0/1853878915 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda34198e40 con 0x7fda34102b30 2024-04-24T23:20:33.644 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:33.640+0000 7fda389f5700 1 -- 172.21.15.80:0/1853878915 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fda240038d0 con 0x7fda34102b30 2024-04-24T23:20:33.644 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:33.640+0000 7fda399f7700 1 -- 172.21.15.80:0/1853878915 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda341992a0 con 0x7fda34102b30 2024-04-24T23:20:33.644 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:33.641+0000 7fda389f5700 1 -- 172.21.15.80:0/1853878915 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fda24029070 con 0x7fda34102b30 2024-04-24T23:20:33.644 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:33.641+0000 7fda389f5700 1 --2- 172.21.15.80:0/1853878915 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fda1c039150 0x7fda1c03b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:33.644 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:33.641+0000 7fda389f5700 1 -- 172.21.15.80:0/1853878915 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fda2401e070 con 0x7fda34102b30 2024-04-24T23:20:33.645 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:33.641+0000 7fda327fc700 1 --2- 172.21.15.80:0/1853878915 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fda1c039150 0x7fda1c03b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:33.645 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:33.641+0000 7fda399f7700 1 -- 172.21.15.80:0/1853878915 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda341022c0 con 0x7fda34102b30 2024-04-24T23:20:33.645 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:33.642+0000 7fda327fc700 1 --2- 172.21.15.80:0/1853878915 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fda1c039150 0x7fda1c03b610 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fda28006fd0 tx=0x7fda28006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:33.648 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:33.646+0000 7fda389f5700 1 -- 172.21.15.80:0/1853878915 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fda24030070 con 0x7fda34102b30 2024-04-24T23:20:33.853 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:33.851+0000 7fda399f7700 1 -- 172.21.15.80:0/1853878915 --> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi148", "target": ["mon-mgr", ""]}) -- 0x7fda34063890 con 0x7fda1c039150 2024-04-24T23:20:34.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:33 smithi080 ceph-mon[29152]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:20:35.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:34 smithi080 ceph-mon[29152]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi148", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:20:35.040 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:34 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:20:35.040 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:34 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:20:35.040 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:34 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:20:35.040 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:34 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:20:35.040 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:34 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi134", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:20:35.040 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:34 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:20:35.040 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:34 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:20:36.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:35 smithi080 ceph-mon[29152]: Deploying cephadm binary to smithi148 2024-04-24T23:20:36.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:35 smithi080 ceph-mon[29152]: Updating smithi134:/etc/ceph/ceph.conf 2024-04-24T23:20:36.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:35 smithi080 ceph-mon[29152]: Updating smithi134:/var/lib/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T23:20:36.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:35 smithi080 ceph-mon[29152]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:20:36.040 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:35 smithi080 ceph-mon[29152]: Updating smithi134:/etc/ceph/ceph.client.admin.keyring 2024-04-24T23:20:37.900 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:37.897+0000 7fda389f5700 1 -- 172.21.15.80:0/1853878915 <== mgr.14152 v2:172.21.15.80:6800/3830596922 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fda34063890 con 0x7fda1c039150 2024-04-24T23:20:37.900 INFO:teuthology.orchestra.run.smithi080.stdout:Added host 'smithi148' with addr '172.21.15.148' 2024-04-24T23:20:37.902 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:37.899+0000 7fda399f7700 1 -- 172.21.15.80:0/1853878915 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fda1c039150 msgr2=0x7fda1c03b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:37.902 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:37.899+0000 7fda399f7700 1 --2- 172.21.15.80:0/1853878915 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fda1c039150 0x7fda1c03b610 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fda28006fd0 tx=0x7fda28006e40 comp rx=0 tx=0).stop 2024-04-24T23:20:37.902 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:37.899+0000 7fda399f7700 1 -- 172.21.15.80:0/1853878915 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fda34102b30 msgr2=0x7fda34198670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:37.902 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:37.899+0000 7fda399f7700 1 --2- 172.21.15.80:0/1853878915 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fda34102b30 0x7fda34198670 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fda24004f80 tx=0x7fda24005060 comp rx=0 tx=0).stop 2024-04-24T23:20:37.902 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:37.900+0000 7fda399f7700 1 -- 172.21.15.80:0/1853878915 shutdown_connections 2024-04-24T23:20:37.902 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:37.900+0000 7fda399f7700 1 --2- 172.21.15.80:0/1853878915 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fda1c039150 0x7fda1c03b610 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:37.902 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:37.900+0000 7fda399f7700 1 --2- 172.21.15.80:0/1853878915 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fda34102b30 0x7fda34198670 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:37.903 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:37.900+0000 7fda399f7700 1 -- 172.21.15.80:0/1853878915 >> 172.21.15.80:0/1853878915 conn(0x7fda340fbda0 msgr2=0x7fda34108740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:37.903 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:37.900+0000 7fda399f7700 1 -- 172.21.15.80:0/1853878915 shutdown_connections 2024-04-24T23:20:37.903 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:37.900+0000 7fda399f7700 1 -- 172.21.15.80:0/1853878915 wait complete. 2024-04-24T23:20:38.216 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:37 smithi080 ceph-mon[29152]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:20:38.216 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:37 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:20:38.217 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:37 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:20:38.217 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:37 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:20:38.510 DEBUG:teuthology.orchestra.run.smithi080:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ec7885aa-0290-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-24T23:20:38.740 INFO:teuthology.orchestra.run.smithi080.stderr:Inferring config /var/lib/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:20:39.204 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:38 smithi080 ceph-mon[29152]: Updating smithi134:/var/lib/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T23:20:39.205 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:38 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:20:39.205 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:38 smithi080 ceph-mon[29152]: Added host smithi148 2024-04-24T23:20:39.205 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:38 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:20:39.205 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:38 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:20:40.261 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:39 smithi080 ceph-mon[29152]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:20:40.261 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:39 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:20:40.278 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:40.276+0000 7fcddb980700 1 -- 172.21.15.80:0/2282039708 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcdd4106c00 msgr2=0x7fcdd4106fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:40.279 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:40.276+0000 7fcddb980700 1 --2- 172.21.15.80:0/2282039708 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcdd4106c00 0x7fcdd4106fe0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fcdc4009910 tx=0x7fcdc4009c20 comp rx=0 tx=0).stop 2024-04-24T23:20:40.279 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:40.276+0000 7fcddb980700 1 -- 172.21.15.80:0/2282039708 shutdown_connections 2024-04-24T23:20:40.279 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:40.276+0000 7fcddb980700 1 --2- 172.21.15.80:0/2282039708 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcdd4106c00 0x7fcdd4106fe0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:40.279 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:40.276+0000 7fcddb980700 1 -- 172.21.15.80:0/2282039708 >> 172.21.15.80:0/2282039708 conn(0x7fcdd4076690 msgr2=0x7fcdd4076aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:40.279 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:40.276+0000 7fcddb980700 1 -- 172.21.15.80:0/2282039708 shutdown_connections 2024-04-24T23:20:40.279 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:40.276+0000 7fcddb980700 1 -- 172.21.15.80:0/2282039708 wait complete. 2024-04-24T23:20:40.281 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:40.277+0000 7fcddb980700 1 Processor -- start 2024-04-24T23:20:40.281 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:40.277+0000 7fcddb980700 1 -- start start 2024-04-24T23:20:40.281 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:40.277+0000 7fcddb980700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcdd4106c00 0x7fcdd41976f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:40.281 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:40.277+0000 7fcddb980700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd410b1c0 con 0x7fcdd4106c00 2024-04-24T23:20:40.282 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:40.277+0000 7fcdd971c700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcdd4106c00 0x7fcdd41976f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:40.282 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:40.277+0000 7fcdd971c700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcdd4106c00 0x7fcdd41976f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:58378/0 (socket says 172.21.15.80:58378) 2024-04-24T23:20:40.282 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:40.277+0000 7fcdd971c700 1 -- 172.21.15.80:0/1102820915 learned_addr learned my addr 172.21.15.80:0/1102820915 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:20:40.282 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:40.277+0000 7fcdd971c700 1 -- 172.21.15.80:0/1102820915 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdd4197c30 con 0x7fcdd4106c00 2024-04-24T23:20:40.282 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:40.278+0000 7fcdd971c700 1 --2- 172.21.15.80:0/1102820915 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcdd4106c00 0x7fcdd41976f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fcdc4005850 tx=0x7fcdc4005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:40.282 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:40.278+0000 7fcdca7fc700 1 -- 172.21.15.80:0/1102820915 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcdc4026070 con 0x7fcdd4106c00 2024-04-24T23:20:40.282 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:40.278+0000 7fcdca7fc700 1 -- 172.21.15.80:0/1102820915 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcdc400bc30 con 0x7fcdd4106c00 2024-04-24T23:20:40.282 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:40.278+0000 7fcddb980700 1 -- 172.21.15.80:0/1102820915 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdd4197ec0 con 0x7fcdd4106c00 2024-04-24T23:20:40.282 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:40.278+0000 7fcdca7fc700 1 -- 172.21.15.80:0/1102820915 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcdc4003a20 con 0x7fcdd4106c00 2024-04-24T23:20:40.282 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:40.278+0000 7fcddb980700 1 -- 172.21.15.80:0/1102820915 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdd419bfc0 con 0x7fcdd4106c00 2024-04-24T23:20:40.282 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:40.279+0000 7fcdca7fc700 1 -- 172.21.15.80:0/1102820915 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fcdc4029070 con 0x7fcdd4106c00 2024-04-24T23:20:40.283 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:40.279+0000 7fcddb980700 1 -- 172.21.15.80:0/1102820915 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcdd4102200 con 0x7fcdd4106c00 2024-04-24T23:20:40.283 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:40.279+0000 7fcdca7fc700 1 --2- 172.21.15.80:0/1102820915 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fcdc0039150 0x7fcdc003b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:40.283 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:40.279+0000 7fcdca7fc700 1 -- 172.21.15.80:0/1102820915 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fcdc401e070 con 0x7fcdd4106c00 2024-04-24T23:20:40.283 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:40.279+0000 7fcdd8f1b700 1 --2- 172.21.15.80:0/1102820915 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fcdc0039150 0x7fcdc003b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:40.283 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:40.279+0000 7fcdd8f1b700 1 --2- 172.21.15.80:0/1102820915 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fcdc0039150 0x7fcdc003b610 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fcdd0006fd0 tx=0x7fcdd0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:40.286 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:40.284+0000 7fcdca7fc700 1 -- 172.21.15.80:0/1102820915 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fcdc4013070 con 0x7fcdd4106c00 2024-04-24T23:20:40.486 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:40.484+0000 7fcddb980700 1 -- 172.21.15.80:0/1102820915 --> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fcdd4198a50 con 0x7fcdc0039150 2024-04-24T23:20:40.487 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:40.485+0000 7fcdca7fc700 1 -- 172.21.15.80:0/1102820915 <== mgr.14152 v2:172.21.15.80:6800/3830596922 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+240 (secure 0 0 0) 0x7fcdd4198a50 con 0x7fcdc0039150 2024-04-24T23:20:40.487 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-24T23:20:40.487 INFO:teuthology.orchestra.run.smithi080.stdout:[{"addr": "172.21.15.80", "hostname": "smithi080", "labels": [], "status": ""}, {"addr": "172.21.15.134", "hostname": "smithi134", "labels": [], "status": ""}, {"addr": "172.21.15.148", "hostname": "smithi148", "labels": [], "status": ""}] 2024-04-24T23:20:40.489 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:40.487+0000 7fcddb980700 1 -- 172.21.15.80:0/1102820915 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fcdc0039150 msgr2=0x7fcdc003b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:40.489 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:40.487+0000 7fcddb980700 1 --2- 172.21.15.80:0/1102820915 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fcdc0039150 0x7fcdc003b610 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fcdd0006fd0 tx=0x7fcdd0006e40 comp rx=0 tx=0).stop 2024-04-24T23:20:40.489 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:40.487+0000 7fcddb980700 1 -- 172.21.15.80:0/1102820915 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcdd4106c00 msgr2=0x7fcdd41976f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:40.489 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:40.487+0000 7fcddb980700 1 --2- 172.21.15.80:0/1102820915 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcdd4106c00 0x7fcdd41976f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fcdc4005850 tx=0x7fcdc4005030 comp rx=0 tx=0).stop 2024-04-24T23:20:40.490 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:40.487+0000 7fcddb980700 1 -- 172.21.15.80:0/1102820915 shutdown_connections 2024-04-24T23:20:40.490 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:40.487+0000 7fcddb980700 1 --2- 172.21.15.80:0/1102820915 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fcdc0039150 0x7fcdc003b610 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:40.490 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:40.487+0000 7fcddb980700 1 --2- 172.21.15.80:0/1102820915 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcdd4106c00 0x7fcdd41976f0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:40.490 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:40.487+0000 7fcddb980700 1 -- 172.21.15.80:0/1102820915 >> 172.21.15.80:0/1102820915 conn(0x7fcdd4076690 msgr2=0x7fcdd4104700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:40.490 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:40.487+0000 7fcddb980700 1 -- 172.21.15.80:0/1102820915 shutdown_connections 2024-04-24T23:20:40.490 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:40.487+0000 7fcddb980700 1 -- 172.21.15.80:0/1102820915 wait complete. 2024-04-24T23:20:41.050 INFO:tasks.cephadm:Setting crush tunables to default 2024-04-24T23:20:41.051 DEBUG:teuthology.orchestra.run.smithi080:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ec7885aa-0290-11ef-bc93-c7b262605968 -- ceph osd crush tunables default 2024-04-24T23:20:41.281 INFO:teuthology.orchestra.run.smithi080.stderr:Inferring config /var/lib/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:20:41.718 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:41 smithi080 ceph-mon[29152]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T23:20:41.719 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:41 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:20:41.719 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:41 smithi080 ceph-mon[29152]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:20:42.839 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:42.837+0000 7fdf42100700 1 -- 172.21.15.80:0/1278371877 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdf3c102ee0 msgr2=0x7fdf3c1032c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:42.839 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:42.837+0000 7fdf42100700 1 --2- 172.21.15.80:0/1278371877 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdf3c102ee0 0x7fdf3c1032c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c009910 tx=0x7fdf2c009c20 comp rx=0 tx=0).stop 2024-04-24T23:20:42.839 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:42.837+0000 7fdf42100700 1 -- 172.21.15.80:0/1278371877 shutdown_connections 2024-04-24T23:20:42.840 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:42.837+0000 7fdf42100700 1 --2- 172.21.15.80:0/1278371877 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdf3c102ee0 0x7fdf3c1032c0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:42.840 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:42.837+0000 7fdf42100700 1 -- 172.21.15.80:0/1278371877 >> 172.21.15.80:0/1278371877 conn(0x7fdf3c0fc690 msgr2=0x7fdf3c0feab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:42.840 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:42.837+0000 7fdf42100700 1 -- 172.21.15.80:0/1278371877 shutdown_connections 2024-04-24T23:20:42.840 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:42.837+0000 7fdf42100700 1 -- 172.21.15.80:0/1278371877 wait complete. 2024-04-24T23:20:42.840 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:42.837+0000 7fdf42100700 1 Processor -- start 2024-04-24T23:20:42.840 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:42.837+0000 7fdf42100700 1 -- start start 2024-04-24T23:20:42.842 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:42.838+0000 7fdf42100700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdf3c102ee0 0x7fdf3c19d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:42.842 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:42.838+0000 7fdf42100700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fdf3c10c330 con 0x7fdf3c102ee0 2024-04-24T23:20:42.842 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:42.838+0000 7fdf3b7fe700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdf3c102ee0 0x7fdf3c19d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:42.842 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:42.838+0000 7fdf3b7fe700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdf3c102ee0 0x7fdf3c19d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:43142/0 (socket says 172.21.15.80:43142) 2024-04-24T23:20:42.842 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:42.838+0000 7fdf3b7fe700 1 -- 172.21.15.80:0/965511150 learned_addr learned my addr 172.21.15.80:0/965511150 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:20:42.842 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:42.838+0000 7fdf3b7fe700 1 -- 172.21.15.80:0/965511150 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf3c19d9a0 con 0x7fdf3c102ee0 2024-04-24T23:20:42.842 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:42.838+0000 7fdf3b7fe700 1 --2- 172.21.15.80:0/965511150 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdf3c102ee0 0x7fdf3c19d460 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c006010 tx=0x7fdf2c005050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:42.842 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:42.838+0000 7fdf397fa700 1 -- 172.21.15.80:0/965511150 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdf2c026070 con 0x7fdf3c102ee0 2024-04-24T23:20:42.843 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:42.838+0000 7fdf397fa700 1 -- 172.21.15.80:0/965511150 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf2c00bc30 con 0x7fdf3c102ee0 2024-04-24T23:20:42.843 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:42.838+0000 7fdf42100700 1 -- 172.21.15.80:0/965511150 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf3c19dc30 con 0x7fdf3c102ee0 2024-04-24T23:20:42.843 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:42.838+0000 7fdf397fa700 1 -- 172.21.15.80:0/965511150 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdf2c003ab0 con 0x7fdf3c102ee0 2024-04-24T23:20:42.843 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:42.838+0000 7fdf42100700 1 -- 172.21.15.80:0/965511150 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf3c197620 con 0x7fdf3c102ee0 2024-04-24T23:20:42.843 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:42.839+0000 7fdf397fa700 1 -- 172.21.15.80:0/965511150 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fdf2c029070 con 0x7fdf3c102ee0 2024-04-24T23:20:42.843 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:42.839+0000 7fdf397fa700 1 --2- 172.21.15.80:0/965511150 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fdf280390f0 0x7fdf2803b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:42.843 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:42.839+0000 7fdf397fa700 1 -- 172.21.15.80:0/965511150 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fdf2c01e070 con 0x7fdf3c102ee0 2024-04-24T23:20:42.843 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:42.840+0000 7fdf32ffd700 1 --2- 172.21.15.80:0/965511150 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fdf280390f0 0x7fdf2803b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:42.843 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:42.840+0000 7fdf42100700 1 -- 172.21.15.80:0/965511150 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf3c06a8e0 con 0x7fdf3c102ee0 2024-04-24T23:20:42.843 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:42.840+0000 7fdf32ffd700 1 --2- 172.21.15.80:0/965511150 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fdf280390f0 0x7fdf2803b5b0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fdf24006fd0 tx=0x7fdf24006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:42.846 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:42.844+0000 7fdf397fa700 1 -- 172.21.15.80:0/965511150 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fdf2c030020 con 0x7fdf3c102ee0 2024-04-24T23:20:43.046 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:43.044+0000 7fdf42100700 1 -- 172.21.15.80:0/965511150 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7fdf3c063890 con 0x7fdf3c102ee0 2024-04-24T23:20:43.859 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:43.857+0000 7fdf397fa700 1 -- 172.21.15.80:0/965511150 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7fdf2c013070 con 0x7fdf3c102ee0 2024-04-24T23:20:43.862 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:43.859+0000 7fdf42100700 1 -- 172.21.15.80:0/965511150 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fdf280390f0 msgr2=0x7fdf2803b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:43.862 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:43.859+0000 7fdf42100700 1 --2- 172.21.15.80:0/965511150 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fdf280390f0 0x7fdf2803b5b0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fdf24006fd0 tx=0x7fdf24006e40 comp rx=0 tx=0).stop 2024-04-24T23:20:43.862 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:43.859+0000 7fdf42100700 1 -- 172.21.15.80:0/965511150 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdf3c102ee0 msgr2=0x7fdf3c19d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:43.862 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:43.859+0000 7fdf42100700 1 --2- 172.21.15.80:0/965511150 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdf3c102ee0 0x7fdf3c19d460 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c006010 tx=0x7fdf2c005050 comp rx=0 tx=0).stop 2024-04-24T23:20:43.862 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:43.860+0000 7fdf42100700 1 -- 172.21.15.80:0/965511150 shutdown_connections 2024-04-24T23:20:43.862 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:43.860+0000 7fdf42100700 1 --2- 172.21.15.80:0/965511150 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fdf280390f0 0x7fdf2803b5b0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:43.862 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:43.860+0000 7fdf42100700 1 --2- 172.21.15.80:0/965511150 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdf3c102ee0 0x7fdf3c19d460 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:43.862 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:43.860+0000 7fdf42100700 1 -- 172.21.15.80:0/965511150 >> 172.21.15.80:0/965511150 conn(0x7fdf3c0fc690 msgr2=0x7fdf3c103c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:43.862 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:43.860+0000 7fdf42100700 1 -- 172.21.15.80:0/965511150 shutdown_connections 2024-04-24T23:20:43.862 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:20:43.860+0000 7fdf42100700 1 -- 172.21.15.80:0/965511150 wait complete. 2024-04-24T23:20:43.863 INFO:teuthology.orchestra.run.smithi080.stderr:adjusted tunables profile to default 2024-04-24T23:20:44.161 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:43 smithi080 ceph-mon[29152]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:20:44.162 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:43 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/965511150' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-24T23:20:44.162 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:43 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:20:44.162 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:43 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:20:44.162 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:43 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:20:44.162 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:43 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:20:44.162 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:43 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi148", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:20:44.162 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:43 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:20:44.162 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:43 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:20:44.483 INFO:tasks.cephadm:Adding mon.a on smithi080 2024-04-24T23:20:44.483 INFO:tasks.cephadm:Adding mon.b on smithi134 2024-04-24T23:20:44.483 INFO:tasks.cephadm:Adding mon.c on smithi148 2024-04-24T23:20:44.483 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ec7885aa-0290-11ef-bc93-c7b262605968 -- ceph orch apply mon '3;smithi080:172.21.15.80=a;smithi134:172.21.15.134=b;smithi148:172.21.15.148=c' 2024-04-24T23:20:44.709 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T23:20:45.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:44 smithi080 ceph-mon[29152]: Updating smithi148:/etc/ceph/ceph.conf 2024-04-24T23:20:45.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:44 smithi080 ceph-mon[29152]: Updating smithi148:/var/lib/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T23:20:45.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:44 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/965511150' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-24T23:20:45.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:44 smithi080 ceph-mon[29152]: osdmap e4: 0 total, 0 up, 0 in 2024-04-24T23:20:45.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:44 smithi080 ceph-mon[29152]: Updating smithi148:/etc/ceph/ceph.client.admin.keyring 2024-04-24T23:20:45.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:44 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:20:45.290 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:44 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:20:45.290 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:44 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:20:46.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:46.219+0000 7f15c2ec9700 1 -- 172.21.15.148:0/1602688347 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f15bc101ea0 msgr2=0x7f15bc102280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:46.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:46.219+0000 7f15c2ec9700 1 --2- 172.21.15.148:0/1602688347 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f15bc101ea0 0x7f15bc102280 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f15b0009910 tx=0x7f15b0009c20 comp rx=0 tx=0).stop 2024-04-24T23:20:46.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:46.219+0000 7f15c2ec9700 1 -- 172.21.15.148:0/1602688347 shutdown_connections 2024-04-24T23:20:46.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:46.219+0000 7f15c2ec9700 1 --2- 172.21.15.148:0/1602688347 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f15bc101ea0 0x7f15bc102280 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:46.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:46.219+0000 7f15c2ec9700 1 -- 172.21.15.148:0/1602688347 >> 172.21.15.148:0/1602688347 conn(0x7f15bc0fd770 msgr2=0x7f15bc0ffb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:46.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:46.219+0000 7f15c2ec9700 1 -- 172.21.15.148:0/1602688347 shutdown_connections 2024-04-24T23:20:46.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:46.219+0000 7f15c2ec9700 1 -- 172.21.15.148:0/1602688347 wait complete. 2024-04-24T23:20:46.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:46.220+0000 7f15c2ec9700 1 Processor -- start 2024-04-24T23:20:46.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:46.220+0000 7f15c2ec9700 1 -- start start 2024-04-24T23:20:46.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:46.220+0000 7f15c2ec9700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f15bc101ea0 0x7f15bc074530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:46.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:46.220+0000 7f15c2ec9700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f15bc10c270 con 0x7f15bc101ea0 2024-04-24T23:20:46.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:46.221+0000 7f15c0c65700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f15bc101ea0 0x7f15bc074530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:46.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:46.221+0000 7f15c0c65700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f15bc101ea0 0x7f15bc074530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.148:56396/0 (socket says 172.21.15.148:56396) 2024-04-24T23:20:46.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:46.221+0000 7f15c0c65700 1 -- 172.21.15.148:0/600282415 learned_addr learned my addr 172.21.15.148:0/600282415 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-24T23:20:46.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:46.221+0000 7f15c0c65700 1 -- 172.21.15.148:0/600282415 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15bc076420 con 0x7f15bc101ea0 2024-04-24T23:20:46.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:46.221+0000 7f15c0c65700 1 --2- 172.21.15.148:0/600282415 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f15bc101ea0 0x7f15bc074530 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f15b0005850 tx=0x7f15b0005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:46.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:46.221+0000 7f15b9ffb700 1 -- 172.21.15.148:0/600282415 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f15b0026070 con 0x7f15bc101ea0 2024-04-24T23:20:46.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:46.221+0000 7f15b9ffb700 1 -- 172.21.15.148:0/600282415 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15b000bc30 con 0x7f15bc101ea0 2024-04-24T23:20:46.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:46.221+0000 7f15c2ec9700 1 -- 172.21.15.148:0/600282415 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15bc074ad0 con 0x7f15bc101ea0 2024-04-24T23:20:46.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:46.221+0000 7f15b9ffb700 1 -- 172.21.15.148:0/600282415 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f15b0003a20 con 0x7f15bc101ea0 2024-04-24T23:20:46.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:46.222+0000 7f15c2ec9700 1 -- 172.21.15.148:0/600282415 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15bc074f90 con 0x7f15bc101ea0 2024-04-24T23:20:46.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:46.222+0000 7f15c2ec9700 1 -- 172.21.15.148:0/600282415 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15bc103380 con 0x7f15bc101ea0 2024-04-24T23:20:46.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:46.222+0000 7f15b9ffb700 1 -- 172.21.15.148:0/600282415 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f15b0029070 con 0x7f15bc101ea0 2024-04-24T23:20:46.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:46.223+0000 7f15b9ffb700 1 --2- 172.21.15.148:0/600282415 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f15ac0390f0 0x7f15ac03b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:46.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:46.223+0000 7f15b9ffb700 1 -- 172.21.15.148:0/600282415 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f15b001e070 con 0x7f15bc101ea0 2024-04-24T23:20:46.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:46.223+0000 7f15bbfff700 1 --2- 172.21.15.148:0/600282415 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f15ac0390f0 0x7f15ac03b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:46.226 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:46.223+0000 7f15bbfff700 1 --2- 172.21.15.148:0/600282415 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f15ac0390f0 0x7f15ac03b5b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f15a8006fd0 tx=0x7f15a8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:46.230 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:46.228+0000 7f15b9ffb700 1 -- 172.21.15.148:0/600282415 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f15b0013070 con 0x7f15bc101ea0 2024-04-24T23:20:46.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:45 smithi080 ceph-mon[29152]: Updating smithi148:/var/lib/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T23:20:46.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:45 smithi080 ceph-mon[29152]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:20:46.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:46.433+0000 7f15c2ec9700 1 -- 172.21.15.148:0/600282415 --> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi080:172.21.15.80=a;smithi134:172.21.15.134=b;smithi148:172.21.15.148=c", "target": ["mon-mgr", ""]}) -- 0x7f15bc075a60 con 0x7f15ac0390f0 2024-04-24T23:20:46.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:46.439+0000 7f15b9ffb700 1 -- 172.21.15.148:0/600282415 <== mgr.14152 v2:172.21.15.80:6800/3830596922 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f15bc075a60 con 0x7f15ac0390f0 2024-04-24T23:20:46.441 INFO:teuthology.orchestra.run.smithi148.stdout:Scheduled mon update... 2024-04-24T23:20:46.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:46.441+0000 7f15c2ec9700 1 -- 172.21.15.148:0/600282415 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f15ac0390f0 msgr2=0x7f15ac03b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:46.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:46.441+0000 7f15c2ec9700 1 --2- 172.21.15.148:0/600282415 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f15ac0390f0 0x7f15ac03b5b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f15a8006fd0 tx=0x7f15a8006e40 comp rx=0 tx=0).stop 2024-04-24T23:20:46.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:46.441+0000 7f15c2ec9700 1 -- 172.21.15.148:0/600282415 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f15bc101ea0 msgr2=0x7f15bc074530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:46.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:46.441+0000 7f15c2ec9700 1 --2- 172.21.15.148:0/600282415 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f15bc101ea0 0x7f15bc074530 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f15b0005850 tx=0x7f15b0005030 comp rx=0 tx=0).stop 2024-04-24T23:20:46.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:46.441+0000 7f15c2ec9700 1 -- 172.21.15.148:0/600282415 shutdown_connections 2024-04-24T23:20:46.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:46.441+0000 7f15c2ec9700 1 --2- 172.21.15.148:0/600282415 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f15ac0390f0 0x7f15ac03b5b0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:46.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:46.441+0000 7f15c2ec9700 1 --2- 172.21.15.148:0/600282415 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f15bc101ea0 0x7f15bc074530 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:46.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:46.441+0000 7f15c2ec9700 1 -- 172.21.15.148:0/600282415 >> 172.21.15.148:0/600282415 conn(0x7f15bc0fd770 msgr2=0x7f15bc103870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:46.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:46.441+0000 7f15c2ec9700 1 -- 172.21.15.148:0/600282415 shutdown_connections 2024-04-24T23:20:46.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:46.441+0000 7f15c2ec9700 1 -- 172.21.15.148:0/600282415 wait complete. 2024-04-24T23:20:47.035 DEBUG:teuthology.orchestra.run.smithi134:mon.b> sudo journalctl -f -n 0 -u ceph-ec7885aa-0290-11ef-bc93-c7b262605968@mon.b.service 2024-04-24T23:20:47.038 DEBUG:teuthology.orchestra.run.smithi148:mon.c> sudo journalctl -f -n 0 -u ceph-ec7885aa-0290-11ef-bc93-c7b262605968@mon.c.service 2024-04-24T23:20:47.040 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-24T23:20:47.041 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ec7885aa-0290-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-24T23:20:47.789 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:47 smithi080 ceph-mon[29152]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi080:172.21.15.80=a;smithi134:172.21.15.134=b;smithi148:172.21.15.148=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:20:47.790 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:47 smithi080 ceph-mon[29152]: Saving service mon spec with placement smithi080:172.21.15.80=a;smithi134:172.21.15.134=b;smithi148:172.21.15.148=c;count:3 2024-04-24T23:20:47.790 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:47 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:20:47.790 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:47 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:20:47.790 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:47 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:20:47.790 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:47 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:20:47.790 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:47 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:20:47.790 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:47 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T23:20:47.790 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:47 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:20:47.790 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:47 smithi080 ceph-mon[29152]: Deploying daemon mon.c on smithi148 2024-04-24T23:20:47.790 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:47 smithi080 ceph-mon[29152]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:20:47.875 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T23:20:50.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:49 smithi080 ceph-mon[29152]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:20:50.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:50.662+0000 7f5225924700 1 -- 172.21.15.148:0/1253494847 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5220106c00 msgr2=0x7f5220106fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:50.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:50.662+0000 7f5225924700 1 --2- 172.21.15.148:0/1253494847 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5220106c00 0x7f5220106fe0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f521000b0a0 tx=0x7f521000b3b0 comp rx=0 tx=0).stop 2024-04-24T23:20:50.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:50.662+0000 7f5225924700 1 -- 172.21.15.148:0/1253494847 shutdown_connections 2024-04-24T23:20:50.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:50.662+0000 7f5225924700 1 --2- 172.21.15.148:0/1253494847 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5220106c00 0x7f5220106fe0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:50.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:50.662+0000 7f5225924700 1 -- 172.21.15.148:0/1253494847 >> 172.21.15.148:0/1253494847 conn(0x7f5220076690 msgr2=0x7f5220076aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:50.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:50.662+0000 7f5225924700 1 -- 172.21.15.148:0/1253494847 shutdown_connections 2024-04-24T23:20:50.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:50.662+0000 7f5225924700 1 -- 172.21.15.148:0/1253494847 wait complete. 2024-04-24T23:20:50.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:50.663+0000 7f5225924700 1 Processor -- start 2024-04-24T23:20:50.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:50.663+0000 7f5225924700 1 -- start start 2024-04-24T23:20:50.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:50.663+0000 7f5225924700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5220106c00 0x7f5220197650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:50.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:50.663+0000 7f5225924700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f522010b1c0 con 0x7f5220106c00 2024-04-24T23:20:50.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:50.664+0000 7f521effd700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5220106c00 0x7f5220197650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:50.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:50.664+0000 7f521effd700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5220106c00 0x7f5220197650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.148:56412/0 (socket says 172.21.15.148:56412) 2024-04-24T23:20:50.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:50.664+0000 7f521effd700 1 -- 172.21.15.148:0/437264072 learned_addr learned my addr 172.21.15.148:0/437264072 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-24T23:20:50.668 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:50.664+0000 7f521effd700 1 -- 172.21.15.148:0/437264072 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5220197b90 con 0x7f5220106c00 2024-04-24T23:20:50.668 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:50.664+0000 7f521effd700 1 --2- 172.21.15.148:0/437264072 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5220106c00 0x7f5220197650 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f52100062a0 tx=0x7f5210003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:50.668 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:50.664+0000 7f5224922700 1 -- 172.21.15.148:0/437264072 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5210017040 con 0x7f5220106c00 2024-04-24T23:20:50.668 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:50.664+0000 7f5224922700 1 -- 172.21.15.148:0/437264072 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5210022070 con 0x7f5220106c00 2024-04-24T23:20:50.668 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:50.664+0000 7f5225924700 1 -- 172.21.15.148:0/437264072 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5220197e20 con 0x7f5220106c00 2024-04-24T23:20:50.668 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:50.664+0000 7f5224922700 1 -- 172.21.15.148:0/437264072 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f521001d440 con 0x7f5220106c00 2024-04-24T23:20:50.668 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:50.664+0000 7f5225924700 1 -- 172.21.15.148:0/437264072 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f522019bf20 con 0x7f5220106c00 2024-04-24T23:20:50.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:50.665+0000 7f5224922700 1 -- 172.21.15.148:0/437264072 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f5210004000 con 0x7f5220106c00 2024-04-24T23:20:50.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:50.665+0000 7f5225924700 1 -- 172.21.15.148:0/437264072 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5220102200 con 0x7f5220106c00 2024-04-24T23:20:50.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:50.665+0000 7f5224922700 1 --2- 172.21.15.148:0/437264072 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f520c0390f0 0x7f520c03b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:50.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:50.665+0000 7f5224922700 1 -- 172.21.15.148:0/437264072 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f521000f050 con 0x7f5220106c00 2024-04-24T23:20:50.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:50.666+0000 7f521e7fc700 1 --2- 172.21.15.148:0/437264072 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f520c0390f0 0x7f520c03b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:50.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:50.666+0000 7f521e7fc700 1 --2- 172.21.15.148:0/437264072 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f520c0390f0 0x7f520c03b5b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f5208006fd0 tx=0x7f5208006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:50.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:50.671+0000 7f5224922700 1 -- 172.21.15.148:0/437264072 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f5210007e20 con 0x7f5220106c00 2024-04-24T23:20:50.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:50.984+0000 7f5225924700 1 -- 172.21.15.148:0/437264072 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f52201989b0 con 0x7f5220106c00 2024-04-24T23:20:50.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:50.984+0000 7f5224922700 1 -- 172.21.15.148:0/437264072 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7f5210025ba0 con 0x7f5220106c00 2024-04-24T23:20:50.986 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-24T23:20:50.987 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":1,"fsid":"ec7885aa-0290-11ef-bc93-c7b262605968","modified":"2024-04-24T23:18:58.816011Z","created":"2024-04-24T23:18:58.816011Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:3300","nonce":0},{"type":"v1","addr":"172.21.15.80:6789","nonce":0}]},"addr":"172.21.15.80:6789/0","public_addr":"172.21.15.80:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-24T23:20:50.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:50.987+0000 7f5225924700 1 -- 172.21.15.148:0/437264072 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f520c0390f0 msgr2=0x7f520c03b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:50.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:50.987+0000 7f5225924700 1 --2- 172.21.15.148:0/437264072 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f520c0390f0 0x7f520c03b5b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f5208006fd0 tx=0x7f5208006e40 comp rx=0 tx=0).stop 2024-04-24T23:20:50.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:50.987+0000 7f5225924700 1 -- 172.21.15.148:0/437264072 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5220106c00 msgr2=0x7f5220197650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:50.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:50.987+0000 7f5225924700 1 --2- 172.21.15.148:0/437264072 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5220106c00 0x7f5220197650 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f52100062a0 tx=0x7f5210003ce0 comp rx=0 tx=0).stop 2024-04-24T23:20:50.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:50.987+0000 7f5225924700 1 -- 172.21.15.148:0/437264072 shutdown_connections 2024-04-24T23:20:50.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:50.987+0000 7f5225924700 1 --2- 172.21.15.148:0/437264072 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f520c0390f0 0x7f520c03b5b0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:50.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:50.987+0000 7f5225924700 1 --2- 172.21.15.148:0/437264072 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5220106c00 0x7f5220197650 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:50.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:50.987+0000 7f5225924700 1 -- 172.21.15.148:0/437264072 >> 172.21.15.148:0/437264072 conn(0x7f5220076690 msgr2=0x7f5220069650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:50.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:50.987+0000 7f5225924700 1 -- 172.21.15.148:0/437264072 shutdown_connections 2024-04-24T23:20:50.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:50.987+0000 7f5225924700 1 -- 172.21.15.148:0/437264072 wait complete. 2024-04-24T23:20:50.990 INFO:teuthology.orchestra.run.smithi148.stderr:dumped monmap epoch 1 2024-04-24T23:20:51.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 systemd[1]: Starting Ceph mon.c for ec7885aa-0290-11ef-bc93-c7b262605968... 2024-04-24T23:20:51.967 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 podman[32509]: 2024-04-24 23:20:51.702616497 +0000 UTC m=+0.107516630 container create 24ebdbf62b15b1033b329f642fc39a8fd54a71232db0bd11bc5db0e6184d3ca1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mon-c, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-04-24T23:20:51.967 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 podman[32509]: 2024-04-24 23:20:51.612855845 +0000 UTC m=+0.017755935 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:20:51.967 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 podman[32509]: 2024-04-24 23:20:51.888382121 +0000 UTC m=+0.293282203 container init 24ebdbf62b15b1033b329f642fc39a8fd54a71232db0bd11bc5db0e6184d3ca1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mon-c, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T23:20:51.967 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 podman[32509]: 2024-04-24 23:20:51.891333601 +0000 UTC m=+0.296233687 container start 24ebdbf62b15b1033b329f642fc39a8fd54a71232db0bd11bc5db0e6184d3ca1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mon-c, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-04-24T23:20:51.967 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: set uid:gid to 167:167 (ceph:ceph) 2024-04-24T23:20:51.967 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev), process ceph-mon, pid 2 2024-04-24T23:20:51.968 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: pidfile_write: ignore empty --pid-file 2024-04-24T23:20:51.968 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: load: jerasure load: lrc 2024-04-24T23:20:51.968 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: RocksDB version: 7.9.2 2024-04-24T23:20:51.968 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Git sha 0 2024-04-24T23:20:51.968 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Compile date 2024-04-24 16:41:23 2024-04-24T23:20:51.968 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: DB SUMMARY 2024-04-24T23:20:51.968 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: DB Session ID: Y6VIN4290RBWLN5NUL9T 2024-04-24T23:20:51.968 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: CURRENT file: CURRENT 2024-04-24T23:20:51.968 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: IDENTITY file: IDENTITY 2024-04-24T23:20:51.968 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-04-24T23:20:51.968 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-04-24T23:20:51.968 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-04-24T23:20:51.969 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.error_if_exists: 0 2024-04-24T23:20:51.969 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.create_if_missing: 0 2024-04-24T23:20:51.969 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.paranoid_checks: 1 2024-04-24T23:20:51.969 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-24T23:20:51.970 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-24T23:20:51.970 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-24T23:20:51.970 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.env: 0x5636eb6fac60 2024-04-24T23:20:51.970 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.fs: PosixFileSystem 2024-04-24T23:20:51.970 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.info_log: 0x5636ec501aa0 2024-04-24T23:20:51.970 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.max_file_opening_threads: 16 2024-04-24T23:20:51.970 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.statistics: (nil) 2024-04-24T23:20:51.970 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.use_fsync: 0 2024-04-24T23:20:51.970 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.max_log_file_size: 0 2024-04-24T23:20:51.970 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-24T23:20:51.970 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-24T23:20:51.970 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.keep_log_file_num: 1000 2024-04-24T23:20:51.971 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.recycle_log_file_num: 0 2024-04-24T23:20:51.974 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.allow_fallocate: 1 2024-04-24T23:20:51.974 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.allow_mmap_reads: 0 2024-04-24T23:20:51.974 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.allow_mmap_writes: 0 2024-04-24T23:20:51.974 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.use_direct_reads: 0 2024-04-24T23:20:51.974 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-24T23:20:51.974 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.create_missing_column_families: 0 2024-04-24T23:20:51.974 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.db_log_dir: 2024-04-24T23:20:51.974 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.wal_dir: 2024-04-24T23:20:51.974 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-24T23:20:51.975 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-24T23:20:51.975 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-24T23:20:51.975 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-24T23:20:51.975 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-24T23:20:51.975 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-24T23:20:51.975 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.advise_random_on_open: 1 2024-04-24T23:20:51.975 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.db_write_buffer_size: 0 2024-04-24T23:20:51.975 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.write_buffer_manager: 0x5636ec4d41e0 2024-04-24T23:20:51.975 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-24T23:20:51.975 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-24T23:20:51.975 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-24T23:20:51.975 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.rate_limiter: (nil) 2024-04-24T23:20:51.975 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-24T23:20:51.976 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.wal_recovery_mode: 2 2024-04-24T23:20:51.976 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.enable_thread_tracking: 0 2024-04-24T23:20:51.976 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.enable_pipelined_write: 0 2024-04-24T23:20:51.976 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.unordered_write: 0 2024-04-24T23:20:51.976 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-24T23:20:51.976 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-24T23:20:51.976 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-24T23:20:51.976 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-24T23:20:51.976 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.row_cache: None 2024-04-24T23:20:51.976 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.wal_filter: None 2024-04-24T23:20:51.976 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-24T23:20:51.976 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.allow_ingest_behind: 0 2024-04-24T23:20:51.976 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.two_write_queues: 0 2024-04-24T23:20:51.976 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.manual_wal_flush: 0 2024-04-24T23:20:51.977 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.wal_compression: 0 2024-04-24T23:20:51.977 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.atomic_flush: 0 2024-04-24T23:20:51.977 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-24T23:20:51.977 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-24T23:20:51.977 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-24T23:20:51.977 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.log_readahead_size: 0 2024-04-24T23:20:51.977 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-24T23:20:51.977 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.best_efforts_recovery: 0 2024-04-24T23:20:51.977 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-24T23:20:51.977 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-24T23:20:51.977 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.allow_data_in_errors: 0 2024-04-24T23:20:51.977 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.db_host_id: __hostname__ 2024-04-24T23:20:51.977 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-24T23:20:51.978 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.max_background_jobs: 2 2024-04-24T23:20:51.978 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.max_background_compactions: -1 2024-04-24T23:20:51.978 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.max_subcompactions: 1 2024-04-24T23:20:51.978 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-24T23:20:51.978 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-24T23:20:51.978 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-24T23:20:51.978 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.max_total_wal_size: 0 2024-04-24T23:20:51.978 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-24T23:20:51.978 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-24T23:20:51.978 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-24T23:20:51.978 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-24T23:20:51.978 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.max_open_files: -1 2024-04-24T23:20:51.978 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.bytes_per_sync: 0 2024-04-24T23:20:51.979 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-24T23:20:51.979 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-24T23:20:51.979 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.compaction_readahead_size: 0 2024-04-24T23:20:51.979 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.max_background_flushes: -1 2024-04-24T23:20:51.979 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Compression algorithms supported: 2024-04-24T23:20:51.979 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-24T23:20:51.979 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: kZSTD supported: 0 2024-04-24T23:20:51.979 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: kXpressCompression supported: 0 2024-04-24T23:20:51.979 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: kLZ4HCCompression supported: 1 2024-04-24T23:20:51.979 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: kZlibCompression supported: 1 2024-04-24T23:20:51.979 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: kSnappyCompression supported: 1 2024-04-24T23:20:51.979 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: kLZ4Compression supported: 1 2024-04-24T23:20:51.980 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: kBZip2Compression supported: 0 2024-04-24T23:20:51.980 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-24T23:20:51.980 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-24T23:20:51.980 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-04-24T23:20:51.980 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-24T23:20:51.980 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-24T23:20:51.980 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.merge_operator: 2024-04-24T23:20:51.980 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.compaction_filter: None 2024-04-24T23:20:51.980 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.compaction_filter_factory: None 2024-04-24T23:20:51.980 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.sst_partitioner_factory: None 2024-04-24T23:20:51.980 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-24T23:20:51.980 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-24T23:20:51.980 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5636ec501ba0) 2024-04-24T23:20:51.980 INFO:journalctl@ceph.mon.c.smithi148.stdout: cache_index_and_filter_blocks: 1 2024-04-24T23:20:51.981 INFO:journalctl@ceph.mon.c.smithi148.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-24T23:20:51.981 INFO:journalctl@ceph.mon.c.smithi148.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-24T23:20:51.981 INFO:journalctl@ceph.mon.c.smithi148.stdout: pin_top_level_index_and_filter: 1 2024-04-24T23:20:51.981 INFO:journalctl@ceph.mon.c.smithi148.stdout: index_type: 0 2024-04-24T23:20:51.981 INFO:journalctl@ceph.mon.c.smithi148.stdout: data_block_index_type: 0 2024-04-24T23:20:51.981 INFO:journalctl@ceph.mon.c.smithi148.stdout: index_shortening: 1 2024-04-24T23:20:51.981 INFO:journalctl@ceph.mon.c.smithi148.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-24T23:20:51.981 INFO:journalctl@ceph.mon.c.smithi148.stdout: checksum: 4 2024-04-24T23:20:51.981 INFO:journalctl@ceph.mon.c.smithi148.stdout: no_block_cache: 0 2024-04-24T23:20:51.981 INFO:journalctl@ceph.mon.c.smithi148.stdout: block_cache: 0x5636ec505090 2024-04-24T23:20:51.981 INFO:journalctl@ceph.mon.c.smithi148.stdout: block_cache_name: BinnedLRUCache 2024-04-24T23:20:51.981 INFO:journalctl@ceph.mon.c.smithi148.stdout: block_cache_options: 2024-04-24T23:20:51.981 INFO:journalctl@ceph.mon.c.smithi148.stdout: capacity : 536870912 2024-04-24T23:20:51.982 INFO:journalctl@ceph.mon.c.smithi148.stdout: num_shard_bits : 4 2024-04-24T23:20:51.982 INFO:journalctl@ceph.mon.c.smithi148.stdout: strict_capacity_limit : 0 2024-04-24T23:20:51.982 INFO:journalctl@ceph.mon.c.smithi148.stdout: high_pri_pool_ratio: 0.000 2024-04-24T23:20:51.982 INFO:journalctl@ceph.mon.c.smithi148.stdout: block_cache_compressed: (nil) 2024-04-24T23:20:51.982 INFO:journalctl@ceph.mon.c.smithi148.stdout: persistent_cache: (nil) 2024-04-24T23:20:51.982 INFO:journalctl@ceph.mon.c.smithi148.stdout: block_size: 4096 2024-04-24T23:20:51.982 INFO:journalctl@ceph.mon.c.smithi148.stdout: block_size_deviation: 10 2024-04-24T23:20:51.982 INFO:journalctl@ceph.mon.c.smithi148.stdout: block_restart_interval: 16 2024-04-24T23:20:51.982 INFO:journalctl@ceph.mon.c.smithi148.stdout: index_block_restart_interval: 1 2024-04-24T23:20:51.982 INFO:journalctl@ceph.mon.c.smithi148.stdout: metadata_block_size: 4096 2024-04-24T23:20:51.982 INFO:journalctl@ceph.mon.c.smithi148.stdout: partition_filters: 0 2024-04-24T23:20:51.982 INFO:journalctl@ceph.mon.c.smithi148.stdout: use_delta_encoding: 1 2024-04-24T23:20:51.982 INFO:journalctl@ceph.mon.c.smithi148.stdout: filter_policy: bloomfilter 2024-04-24T23:20:51.982 INFO:journalctl@ceph.mon.c.smithi148.stdout: whole_key_filtering: 1 2024-04-24T23:20:51.983 INFO:journalctl@ceph.mon.c.smithi148.stdout: verify_compression: 0 2024-04-24T23:20:51.983 INFO:journalctl@ceph.mon.c.smithi148.stdout: read_amp_bytes_per_bit: 0 2024-04-24T23:20:51.983 INFO:journalctl@ceph.mon.c.smithi148.stdout: format_version: 5 2024-04-24T23:20:51.983 INFO:journalctl@ceph.mon.c.smithi148.stdout: enable_index_compression: 1 2024-04-24T23:20:51.983 INFO:journalctl@ceph.mon.c.smithi148.stdout: block_align: 0 2024-04-24T23:20:51.983 INFO:journalctl@ceph.mon.c.smithi148.stdout: max_auto_readahead_size: 262144 2024-04-24T23:20:51.983 INFO:journalctl@ceph.mon.c.smithi148.stdout: prepopulate_block_cache: 0 2024-04-24T23:20:51.983 INFO:journalctl@ceph.mon.c.smithi148.stdout: initial_auto_readahead_size: 8192 2024-04-24T23:20:51.983 INFO:journalctl@ceph.mon.c.smithi148.stdout: num_file_reads_for_auto_readahead: 2 2024-04-24T23:20:51.983 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.write_buffer_size: 33554432 2024-04-24T23:20:51.983 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.max_write_buffer_number: 2 2024-04-24T23:20:51.983 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.compression: NoCompression 2024-04-24T23:20:51.983 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.bottommost_compression: Disabled 2024-04-24T23:20:51.983 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.prefix_extractor: nullptr 2024-04-24T23:20:51.984 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-24T23:20:51.984 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.num_levels: 7 2024-04-24T23:20:51.984 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-24T23:20:51.984 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-24T23:20:51.984 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-24T23:20:51.984 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-24T23:20:51.984 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-24T23:20:51.984 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-24T23:20:51.984 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-24T23:20:51.984 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-24T23:20:51.984 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-24T23:20:51.984 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-24T23:20:51.984 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-24T23:20:51.984 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-24T23:20:51.985 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-24T23:20:51.985 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.compression_opts.level: 32767 2024-04-24T23:20:51.985 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.compression_opts.strategy: 0 2024-04-24T23:20:51.985 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-24T23:20:51.985 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-24T23:20:51.985 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-24T23:20:51.985 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-24T23:20:51.985 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.compression_opts.enabled: false 2024-04-24T23:20:51.986 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-24T23:20:51.986 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-24T23:20:51.987 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-24T23:20:51.987 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-24T23:20:51.987 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.target_file_size_base: 67108864 2024-04-24T23:20:51.987 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-24T23:20:51.987 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-24T23:20:51.987 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-24T23:20:51.987 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-24T23:20:51.987 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-24T23:20:51.987 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-24T23:20:51.987 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-24T23:20:51.987 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-24T23:20:51.987 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-24T23:20:51.987 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-24T23:20:51.988 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-24T23:20:51.988 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-24T23:20:51.988 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-24T23:20:51.988 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-24T23:20:51.988 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.arena_block_size: 1048576 2024-04-24T23:20:51.988 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-24T23:20:51.988 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-24T23:20:51.988 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.disable_auto_compactions: 0 2024-04-24T23:20:51.988 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-24T23:20:51.988 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-24T23:20:51.988 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-24T23:20:51.988 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-24T23:20:51.988 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-24T23:20:51.988 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-24T23:20:51.988 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-24T23:20:51.989 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-24T23:20:51.989 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-24T23:20:51.989 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-24T23:20:51.989 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.table_properties_collectors: 2024-04-24T23:20:51.989 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.inplace_update_support: 0 2024-04-24T23:20:51.989 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-24T23:20:51.989 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-24T23:20:51.989 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-24T23:20:51.989 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-24T23:20:51.989 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.bloom_locality: 0 2024-04-24T23:20:51.989 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.max_successive_merges: 0 2024-04-24T23:20:51.989 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-24T23:20:51.989 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.paranoid_file_checks: 0 2024-04-24T23:20:51.989 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.force_consistency_checks: 1 2024-04-24T23:20:51.989 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.report_bg_io_stats: 0 2024-04-24T23:20:51.990 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.ttl: 2592000 2024-04-24T23:20:51.990 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-24T23:20:51.990 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-24T23:20:51.990 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-24T23:20:51.990 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.enable_blob_files: false 2024-04-24T23:20:51.990 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.min_blob_size: 0 2024-04-24T23:20:51.990 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.blob_file_size: 268435456 2024-04-24T23:20:51.990 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-24T23:20:51.990 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-24T23:20:51.990 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-24T23:20:51.990 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-24T23:20:51.990 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-24T23:20:51.990 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.blob_file_starting_level: 0 2024-04-24T23:20:51.990 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-24T23:20:51.991 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-04-24T23:20:51.991 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-04-24T23:20:51.991 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: f0616d05-0af1-4808-bec9-8f13f5ab3cac 2024-04-24T23:20:51.991 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714000851931968, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-04-24T23:20:51.991 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-04-24T23:20:51.991 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714000851932398, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1714000851, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "f0616d05-0af1-4808-bec9-8f13f5ab3cac", "db_session_id": "Y6VIN4290RBWLN5NUL9T", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-04-24T23:20:51.991 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714000851932475, "job": 1, "event": "recovery_finished"} 2024-04-24T23:20:51.991 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-04-24T23:20:51.991 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-24T23:20:51.991 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5636ec5f6000 2024-04-24T23:20:51.991 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: DB pointer 0x5636ec5e0000 2024-04-24T23:20:51.991 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-04-24T23:20:51.991 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-24T23:20:51.991 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-24T23:20:51.992 INFO:journalctl@ceph.mon.c.smithi148.stdout: ** DB Stats ** 2024-04-24T23:20:51.992 INFO:journalctl@ceph.mon.c.smithi148.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-24T23:20:51.992 INFO:journalctl@ceph.mon.c.smithi148.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-24T23:20:51.992 INFO:journalctl@ceph.mon.c.smithi148.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-24T23:20:51.992 INFO:journalctl@ceph.mon.c.smithi148.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-24T23:20:51.992 INFO:journalctl@ceph.mon.c.smithi148.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-24T23:20:51.992 INFO:journalctl@ceph.mon.c.smithi148.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-24T23:20:51.992 INFO:journalctl@ceph.mon.c.smithi148.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-24T23:20:51.992 INFO:journalctl@ceph.mon.c.smithi148.stdout: 2024-04-24T23:20:51.992 INFO:journalctl@ceph.mon.c.smithi148.stdout: ** Compaction Stats [default] ** 2024-04-24T23:20:51.992 INFO:journalctl@ceph.mon.c.smithi148.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-24T23:20:51.992 INFO:journalctl@ceph.mon.c.smithi148.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-24T23:20:51.993 INFO:journalctl@ceph.mon.c.smithi148.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-24T23:20:51.993 INFO:journalctl@ceph.mon.c.smithi148.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-24T23:20:51.993 INFO:journalctl@ceph.mon.c.smithi148.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-24T23:20:51.993 INFO:journalctl@ceph.mon.c.smithi148.stdout: 2024-04-24T23:20:51.993 INFO:journalctl@ceph.mon.c.smithi148.stdout: ** Compaction Stats [default] ** 2024-04-24T23:20:51.993 INFO:journalctl@ceph.mon.c.smithi148.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-24T23:20:51.993 INFO:journalctl@ceph.mon.c.smithi148.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-24T23:20:51.993 INFO:journalctl@ceph.mon.c.smithi148.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-24T23:20:51.993 INFO:journalctl@ceph.mon.c.smithi148.stdout: 2024-04-24T23:20:51.993 INFO:journalctl@ceph.mon.c.smithi148.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-24T23:20:51.993 INFO:journalctl@ceph.mon.c.smithi148.stdout: 2024-04-24T23:20:51.993 INFO:journalctl@ceph.mon.c.smithi148.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-24T23:20:51.993 INFO:journalctl@ceph.mon.c.smithi148.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-24T23:20:51.993 INFO:journalctl@ceph.mon.c.smithi148.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-24T23:20:51.994 INFO:journalctl@ceph.mon.c.smithi148.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-24T23:20:51.994 INFO:journalctl@ceph.mon.c.smithi148.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-24T23:20:51.994 INFO:journalctl@ceph.mon.c.smithi148.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-24T23:20:51.994 INFO:journalctl@ceph.mon.c.smithi148.stdout: Cumulative compaction: 0.00 GB write, 0.12 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-24T23:20:51.994 INFO:journalctl@ceph.mon.c.smithi148.stdout: Interval compaction: 0.00 GB write, 0.12 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-24T23:20:51.994 INFO:journalctl@ceph.mon.c.smithi148.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-24T23:20:51.994 INFO:journalctl@ceph.mon.c.smithi148.stdout: Block cache BinnedLRUCache@0x5636ec505090#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-04-24T23:20:51.994 INFO:journalctl@ceph.mon.c.smithi148.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-04-24T23:20:51.994 INFO:journalctl@ceph.mon.c.smithi148.stdout: 2024-04-24T23:20:51.994 INFO:journalctl@ceph.mon.c.smithi148.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-24T23:20:51.994 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: using public_addr v2:172.21.15.148:0/0 -> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] 2024-04-24T23:20:51.994 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: starting mon.c rank -1 at public addrs [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] at bind addrs [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid ec7885aa-0290-11ef-bc93-c7b262605968 2024-04-24T23:20:51.994 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: mon.c@-1(???) e0 preinit fsid ec7885aa-0290-11ef-bc93-c7b262605968 2024-04-24T23:20:51.995 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: mon.c@-1(synchronizing).mds e1 new map 2024-04-24T23:20:51.995 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: mon.c@-1(synchronizing).mds e1 print_map 2024-04-24T23:20:51.995 INFO:journalctl@ceph.mon.c.smithi148.stdout: e1 2024-04-24T23:20:51.995 INFO:journalctl@ceph.mon.c.smithi148.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-24T23:20:51.995 INFO:journalctl@ceph.mon.c.smithi148.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-24T23:20:51.995 INFO:journalctl@ceph.mon.c.smithi148.stdout: legacy client fscid: -1 2024-04-24T23:20:51.995 INFO:journalctl@ceph.mon.c.smithi148.stdout: 2024-04-24T23:20:51.995 INFO:journalctl@ceph.mon.c.smithi148.stdout: No filesystems configured 2024-04-24T23:20:51.995 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-04-24T23:20:51.995 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-04-24T23:20:51.995 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-04-24T23:20:51.995 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-04-24T23:20:51.995 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-04-24T23:20:51.995 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-04-24T23:20:51.996 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-24T23:20:51.996 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-24T23:20:51.996 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-24T23:20:51.996 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-24T23:20:51.996 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:20:51.996 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:20:51.996 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:20:51.996 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T23:20:51.996 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:20:51.996 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi148", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:20:51.996 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:20:51.996 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:20:51.996 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:20:51.997 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:20:51.997 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi134", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:20:51.997 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:20:51.997 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:20:51.997 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: Deploying cephadm binary to smithi148 2024-04-24T23:20:51.997 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: Updating smithi134:/etc/ceph/ceph.conf 2024-04-24T23:20:51.997 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: Updating smithi134:/var/lib/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T23:20:51.997 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:20:51.997 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: Updating smithi134:/etc/ceph/ceph.client.admin.keyring 2024-04-24T23:20:51.997 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:20:51.997 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:20:51.997 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:20:51.997 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:20:51.997 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: Updating smithi134:/var/lib/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T23:20:51.997 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:20:51.998 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: Added host smithi148 2024-04-24T23:20:51.998 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:20:51.998 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:20:51.998 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:20:51.998 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:20:51.998 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T23:20:51.998 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:20:51.998 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:20:51.998 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:20:51.998 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/965511150' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-24T23:20:51.998 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:20:51.998 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:20:51.998 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:20:51.998 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:20:51.999 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi148", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:20:51.999 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:20:51.999 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:20:51.999 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: Updating smithi148:/etc/ceph/ceph.conf 2024-04-24T23:20:51.999 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: Updating smithi148:/var/lib/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T23:20:51.999 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/965511150' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-24T23:20:51.999 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: osdmap e4: 0 total, 0 up, 0 in 2024-04-24T23:20:51.999 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: Updating smithi148:/etc/ceph/ceph.client.admin.keyring 2024-04-24T23:20:51.999 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:20:51.999 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:20:51.999 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:20:51.999 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: Updating smithi148:/var/lib/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T23:20:51.999 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:20:51.999 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi080:172.21.15.80=a;smithi134:172.21.15.134=b;smithi148:172.21.15.148=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:20:51.999 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: Saving service mon spec with placement smithi080:172.21.15.80=a;smithi134:172.21.15.134=b;smithi148:172.21.15.148=c;count:3 2024-04-24T23:20:51.999 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:20:52.000 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:20:52.000 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:20:52.000 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:20:52.000 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:20:52.000 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T23:20:52.000 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:20:52.000 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: Deploying daemon mon.c on smithi148 2024-04-24T23:20:52.000 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:20:52.000 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:20:52.000 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:20:52.000 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: from='client.? 172.21.15.148:0/437264072' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T23:20:52.000 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 ceph-mon[32528]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-04-24T23:20:52.000 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 bash[32509]: 24ebdbf62b15b1033b329f642fc39a8fd54a71232db0bd11bc5db0e6184d3ca1 2024-04-24T23:20:52.000 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:51 smithi148 systemd[1]: Started Ceph mon.c for ec7885aa-0290-11ef-bc93-c7b262605968. 2024-04-24T23:20:52.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:51 smithi080 ceph-mon[29152]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:20:52.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:51 smithi080 ceph-mon[29152]: from='client.? 172.21.15.148:0/437264072' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T23:20:52.522 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-24T23:20:52.522 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ec7885aa-0290-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-24T23:20:52.763 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T23:20:57.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:57 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T23:20:57.290 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:57 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:20:57.290 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:57 smithi080 ceph-mon[29152]: mon.a calling monitor election 2024-04-24T23:20:57.290 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:57 smithi080 ceph-mon[29152]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:20:57.290 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:57 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:20:57.290 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:57 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:20:57.290 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:57 smithi080 ceph-mon[29152]: mon.c calling monitor election 2024-04-24T23:20:57.290 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:57 smithi080 ceph-mon[29152]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:20:57.290 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:57 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:20:57.290 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:57 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:20:57.290 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:57 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:20:57.290 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:57 smithi080 ceph-mon[29152]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-24T23:20:57.291 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:57 smithi080 ceph-mon[29152]: monmap epoch 2 2024-04-24T23:20:57.291 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:57 smithi080 ceph-mon[29152]: fsid ec7885aa-0290-11ef-bc93-c7b262605968 2024-04-24T23:20:57.293 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:57 smithi080 ceph-mon[29152]: last_changed 2024-04-24T23:20:51.955535+0000 2024-04-24T23:20:57.293 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:57 smithi080 ceph-mon[29152]: created 2024-04-24T23:18:58.816011+0000 2024-04-24T23:20:57.293 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:57 smithi080 ceph-mon[29152]: min_mon_release 19 (squid) 2024-04-24T23:20:57.294 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:57 smithi080 ceph-mon[29152]: election_strategy: 1 2024-04-24T23:20:57.294 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:57 smithi080 ceph-mon[29152]: 0: [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] mon.a 2024-04-24T23:20:57.294 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:57 smithi080 ceph-mon[29152]: 1: [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] mon.c 2024-04-24T23:20:57.294 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:57 smithi080 ceph-mon[29152]: fsmap 2024-04-24T23:20:57.294 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:57 smithi080 ceph-mon[29152]: osdmap e4: 0 total, 0 up, 0 in 2024-04-24T23:20:57.294 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:57 smithi080 ceph-mon[29152]: mgrmap e13: x(active, since 50s) 2024-04-24T23:20:57.294 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:57 smithi080 ceph-mon[29152]: overall HEALTH_OK 2024-04-24T23:20:57.294 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:57 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:20:57.294 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:57 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:20:57.295 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:57 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:20:57.295 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:57 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T23:20:57.295 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:57 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:20:57.311 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:57.308+0000 7fcd8b81c700 1 -- 172.21.15.148:0/1066525187 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcd700035c0 msgr2=0x7fcd70005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:57.311 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:57.308+0000 7fcd8b81c700 1 --2- 172.21.15.148:0/1066525187 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcd700035c0 0x7fcd70005a50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fcd84101c90 tx=0x7fcd7400feb0 comp rx=0 tx=0).stop 2024-04-24T23:20:57.311 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:57.308+0000 7fcd8b81c700 1 -- 172.21.15.148:0/1066525187 shutdown_connections 2024-04-24T23:20:57.311 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:57.308+0000 7fcd8b81c700 1 --2- 172.21.15.148:0/1066525187 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcd700035c0 0x7fcd70005a50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:57.311 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:57.308+0000 7fcd8b81c700 1 -- 172.21.15.148:0/1066525187 >> 172.21.15.148:0/1066525187 conn(0x7fcd840fd850 msgr2=0x7fcd840ffc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:57.311 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:57.308+0000 7fcd8b81c700 1 -- 172.21.15.148:0/1066525187 shutdown_connections 2024-04-24T23:20:57.311 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:57.308+0000 7fcd8b81c700 1 -- 172.21.15.148:0/1066525187 wait complete. 2024-04-24T23:20:57.312 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:57 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T23:20:57.312 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:57 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:20:57.312 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:57 smithi148 ceph-mon[32528]: mon.a calling monitor election 2024-04-24T23:20:57.312 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:57 smithi148 ceph-mon[32528]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:20:57.312 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:57 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:20:57.312 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:57 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:20:57.312 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:57 smithi148 ceph-mon[32528]: mon.c calling monitor election 2024-04-24T23:20:57.312 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:57 smithi148 ceph-mon[32528]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:20:57.312 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:57 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:20:57.312 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:57 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:20:57.312 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:57 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:20:57.312 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:57 smithi148 ceph-mon[32528]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-24T23:20:57.313 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:57 smithi148 ceph-mon[32528]: monmap epoch 2 2024-04-24T23:20:57.313 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:57 smithi148 ceph-mon[32528]: fsid ec7885aa-0290-11ef-bc93-c7b262605968 2024-04-24T23:20:57.313 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:57 smithi148 ceph-mon[32528]: last_changed 2024-04-24T23:20:51.955535+0000 2024-04-24T23:20:57.313 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:57 smithi148 ceph-mon[32528]: created 2024-04-24T23:18:58.816011+0000 2024-04-24T23:20:57.313 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:57 smithi148 ceph-mon[32528]: min_mon_release 19 (squid) 2024-04-24T23:20:57.313 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:57 smithi148 ceph-mon[32528]: election_strategy: 1 2024-04-24T23:20:57.313 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:57 smithi148 ceph-mon[32528]: 0: [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] mon.a 2024-04-24T23:20:57.313 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:57 smithi148 ceph-mon[32528]: 1: [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] mon.c 2024-04-24T23:20:57.313 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:57 smithi148 ceph-mon[32528]: fsmap 2024-04-24T23:20:57.313 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:57 smithi148 ceph-mon[32528]: osdmap e4: 0 total, 0 up, 0 in 2024-04-24T23:20:57.313 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:57 smithi148 ceph-mon[32528]: mgrmap e13: x(active, since 50s) 2024-04-24T23:20:57.313 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:57 smithi148 ceph-mon[32528]: overall HEALTH_OK 2024-04-24T23:20:57.313 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:57 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:20:57.313 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:57 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:20:57.313 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:57 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:20:57.314 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:57 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T23:20:57.314 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:57 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:20:57.316 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:57.309+0000 7fcd8b81c700 1 Processor -- start 2024-04-24T23:20:57.316 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:57.309+0000 7fcd8b81c700 1 -- start start 2024-04-24T23:20:57.317 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:57.310+0000 7fcd8b81c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd700035c0 0x7fcd8419d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:57.317 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:57.310+0000 7fcd8b81c700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcd8419db10 0x7fcd841976a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:57.317 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:57.310+0000 7fcd8b81c700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fcd84109df0 con 0x7fcd8419db10 2024-04-24T23:20:57.317 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:57.310+0000 7fcd8b81c700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fcd84109c70 con 0x7fcd700035c0 2024-04-24T23:20:57.317 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:57.310+0000 7fcd895b8700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd700035c0 0x7fcd8419d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:57.317 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:57.310+0000 7fcd88db7700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcd8419db10 0x7fcd841976a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:57.317 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:57.310+0000 7fcd895b8700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd700035c0 0x7fcd8419d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:52712/0 (socket says 172.21.15.148:52712) 2024-04-24T23:20:57.317 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:57.310+0000 7fcd895b8700 1 -- 172.21.15.148:0/2712305800 learned_addr learned my addr 172.21.15.148:0/2712305800 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-24T23:20:57.317 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:57.310+0000 7fcd895b8700 1 -- 172.21.15.148:0/2712305800 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd700035c0 msgr2=0x7fcd8419d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-24T23:20:57.317 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:57.310+0000 7fcd895b8700 1 -- 172.21.15.148:0/2712305800 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd700035c0 msgr2=0x7fcd8419d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-24T23:20:57.317 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:57.310+0000 7fcd895b8700 1 --2- 172.21.15.148:0/2712305800 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd700035c0 0x7fcd8419d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-24T23:20:57.317 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:57.310+0000 7fcd895b8700 1 --2- 172.21.15.148:0/2712305800 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd700035c0 0x7fcd8419d5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T23:20:57.317 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:57.310+0000 7fcd88db7700 1 -- 172.21.15.148:0/2712305800 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd700035c0 msgr2=0x7fcd8419d5d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T23:20:57.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:57.310+0000 7fcd88db7700 1 --2- 172.21.15.148:0/2712305800 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd700035c0 0x7fcd8419d5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:57.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:57.310+0000 7fcd88db7700 1 -- 172.21.15.148:0/2712305800 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd8419e090 con 0x7fcd8419db10 2024-04-24T23:20:57.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:57.311+0000 7fcd88db7700 1 --2- 172.21.15.148:0/2712305800 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcd8419db10 0x7fcd841976a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fcd8000b4d0 tx=0x7fcd8000b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:57.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:57.311+0000 7fcd7a7fc700 1 -- 172.21.15.148:0/2712305800 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fcd80011630 con 0x7fcd8419db10 2024-04-24T23:20:57.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:57.311+0000 7fcd7a7fc700 1 -- 172.21.15.148:0/2712305800 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd80011cb0 con 0x7fcd8419db10 2024-04-24T23:20:57.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:57.311+0000 7fcd8b81c700 1 -- 172.21.15.148:0/2712305800 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd84197c40 con 0x7fcd8419db10 2024-04-24T23:20:57.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:57.311+0000 7fcd7a7fc700 1 -- 172.21.15.148:0/2712305800 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fcd80010e60 con 0x7fcd8419db10 2024-04-24T23:20:57.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:57.311+0000 7fcd8b81c700 1 -- 172.21.15.148:0/2712305800 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd841981d0 con 0x7fcd8419db10 2024-04-24T23:20:57.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:57.312+0000 7fcd7a7fc700 1 -- 172.21.15.148:0/2712305800 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fcd80018020 con 0x7fcd8419db10 2024-04-24T23:20:57.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:57.312+0000 7fcd8b81c700 1 -- 172.21.15.148:0/2712305800 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd841034a0 con 0x7fcd8419db10 2024-04-24T23:20:57.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:57.312+0000 7fcd7a7fc700 1 --2- 172.21.15.148:0/2712305800 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fcd68038e90 0x7fcd6803b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:57.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:57.312+0000 7fcd7a7fc700 1 -- 172.21.15.148:0/2712305800 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fcd8004cc60 con 0x7fcd8419db10 2024-04-24T23:20:57.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:57.312+0000 7fcd895b8700 1 --2- 172.21.15.148:0/2712305800 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fcd68038e90 0x7fcd6803b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:57.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:57.313+0000 7fcd895b8700 1 --2- 172.21.15.148:0/2712305800 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fcd68038e90 0x7fcd6803b350 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fcd7400fa60 tx=0x7fcd74005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:57.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:57.318+0000 7fcd7a7fc700 1 -- 172.21.15.148:0/2712305800 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fcd8001f020 con 0x7fcd8419db10 2024-04-24T23:20:57.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:57.603+0000 7fcd8b81c700 1 -- 172.21.15.148:0/2712305800 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fcd84198540 con 0x7fcd8419db10 2024-04-24T23:20:57.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:57.604+0000 7fcd7a7fc700 1 -- 172.21.15.148:0/2712305800 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7fcd80018570 con 0x7fcd8419db10 2024-04-24T23:20:57.605 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-24T23:20:57.605 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":2,"fsid":"ec7885aa-0290-11ef-bc93-c7b262605968","modified":"2024-04-24T23:20:51.955535Z","created":"2024-04-24T23:18:58.816011Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:3300","nonce":0},{"type":"v1","addr":"172.21.15.80:6789","nonce":0}]},"addr":"172.21.15.80:6789/0","public_addr":"172.21.15.80:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:3300","nonce":0},{"type":"v1","addr":"172.21.15.148:6789","nonce":0}]},"addr":"172.21.15.148:6789/0","public_addr":"172.21.15.148:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-24T23:20:57.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:57.606+0000 7fcd8b81c700 1 -- 172.21.15.148:0/2712305800 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fcd68038e90 msgr2=0x7fcd6803b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:57.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:57.606+0000 7fcd8b81c700 1 --2- 172.21.15.148:0/2712305800 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fcd68038e90 0x7fcd6803b350 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fcd7400fa60 tx=0x7fcd74005960 comp rx=0 tx=0).stop 2024-04-24T23:20:57.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:57.606+0000 7fcd8b81c700 1 -- 172.21.15.148:0/2712305800 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcd8419db10 msgr2=0x7fcd841976a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:57.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:57.606+0000 7fcd8b81c700 1 --2- 172.21.15.148:0/2712305800 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcd8419db10 0x7fcd841976a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fcd8000b4d0 tx=0x7fcd8000b890 comp rx=0 tx=0).stop 2024-04-24T23:20:57.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:57.606+0000 7fcd8b81c700 1 -- 172.21.15.148:0/2712305800 shutdown_connections 2024-04-24T23:20:57.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:57.606+0000 7fcd8b81c700 1 --2- 172.21.15.148:0/2712305800 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd700035c0 0x7fcd8419d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:57.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:57.606+0000 7fcd8b81c700 1 --2- 172.21.15.148:0/2712305800 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fcd68038e90 0x7fcd6803b350 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:57.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:57.606+0000 7fcd8b81c700 1 --2- 172.21.15.148:0/2712305800 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcd8419db10 0x7fcd841976a0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:57.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:57.606+0000 7fcd8b81c700 1 -- 172.21.15.148:0/2712305800 >> 172.21.15.148:0/2712305800 conn(0x7fcd840fd850 msgr2=0x7fcd840feb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:57.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:57.606+0000 7fcd8b81c700 1 -- 172.21.15.148:0/2712305800 shutdown_connections 2024-04-24T23:20:57.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:20:57.606+0000 7fcd8b81c700 1 -- 172.21.15.148:0/2712305800 wait complete. 2024-04-24T23:20:57.609 INFO:teuthology.orchestra.run.smithi148.stderr:dumped monmap epoch 2 2024-04-24T23:20:58.165 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:58 smithi148 ceph-mon[32528]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:20:58.165 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:58 smithi148 ceph-mon[32528]: Deploying daemon mon.b on smithi134 2024-04-24T23:20:58.165 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:58 smithi148 ceph-mon[32528]: from='client.? 172.21.15.148:0/2712305800' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T23:20:58.165 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:20:58 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:20:58.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:58 smithi080 ceph-mon[29152]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:20:58.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:58 smithi080 ceph-mon[29152]: Deploying daemon mon.b on smithi134 2024-04-24T23:20:58.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:58 smithi080 ceph-mon[29152]: from='client.? 172.21.15.148:0/2712305800' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T23:20:58.290 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:20:58 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:20:59.191 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-24T23:20:59.192 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ec7885aa-0290-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-24T23:20:59.434 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T23:21:00.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:00 smithi148 ceph-mon[32528]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:21:00.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:00 smithi080 ceph-mon[29152]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:21:00.948 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:00 smithi134 systemd[1]: Started Ceph mon.b for ec7885aa-0290-11ef-bc93-c7b262605968. 2024-04-24T23:21:06.040 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:05 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T23:21:06.040 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:05 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:21:06.040 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:05 smithi080 ceph-mon[29152]: mon.a calling monitor election 2024-04-24T23:21:06.040 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:05 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:21:06.040 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:05 smithi080 ceph-mon[29152]: mon.c calling monitor election 2024-04-24T23:21:06.040 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:05 smithi080 ceph-mon[29152]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:21:06.040 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:05 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:21:06.040 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:05 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:21:06.040 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:05 smithi080 ceph-mon[29152]: mon.b calling monitor election 2024-04-24T23:21:06.041 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:05 smithi080 ceph-mon[29152]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:21:06.041 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:05 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:21:06.041 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:05 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:21:06.041 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:05 smithi080 ceph-mon[29152]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:21:06.041 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:05 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:21:06.041 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:05 smithi080 ceph-mon[29152]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-24T23:21:06.041 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:05 smithi080 ceph-mon[29152]: monmap epoch 3 2024-04-24T23:21:06.041 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:05 smithi080 ceph-mon[29152]: fsid ec7885aa-0290-11ef-bc93-c7b262605968 2024-04-24T23:21:06.041 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:05 smithi080 ceph-mon[29152]: last_changed 2024-04-24T23:21:00.713440+0000 2024-04-24T23:21:06.041 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:05 smithi080 ceph-mon[29152]: created 2024-04-24T23:18:58.816011+0000 2024-04-24T23:21:06.041 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:05 smithi080 ceph-mon[29152]: min_mon_release 19 (squid) 2024-04-24T23:21:06.041 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:05 smithi080 ceph-mon[29152]: election_strategy: 1 2024-04-24T23:21:06.041 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:05 smithi080 ceph-mon[29152]: 0: [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] mon.a 2024-04-24T23:21:06.041 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:05 smithi080 ceph-mon[29152]: 1: [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] mon.c 2024-04-24T23:21:06.041 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:05 smithi080 ceph-mon[29152]: 2: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.b 2024-04-24T23:21:06.042 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:05 smithi080 ceph-mon[29152]: fsmap 2024-04-24T23:21:06.042 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:05 smithi080 ceph-mon[29152]: osdmap e4: 0 total, 0 up, 0 in 2024-04-24T23:21:06.042 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:05 smithi080 ceph-mon[29152]: mgrmap e13: x(active, since 58s) 2024-04-24T23:21:06.042 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:05 smithi080 ceph-mon[29152]: overall HEALTH_OK 2024-04-24T23:21:06.042 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:05 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:06.042 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:05 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:06.042 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:05 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:06.042 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:05 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:06.199 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:05 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T23:21:06.199 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:05 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:21:06.199 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:05 smithi134 ceph-mon[32619]: mon.a calling monitor election 2024-04-24T23:21:06.199 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:05 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:21:06.199 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:05 smithi134 ceph-mon[32619]: mon.c calling monitor election 2024-04-24T23:21:06.199 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:05 smithi134 ceph-mon[32619]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:21:06.199 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:05 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:21:06.199 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:05 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:21:06.199 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:05 smithi134 ceph-mon[32619]: mon.b calling monitor election 2024-04-24T23:21:06.199 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:05 smithi134 ceph-mon[32619]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:21:06.199 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:05 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:21:06.199 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:05 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:21:06.200 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:05 smithi134 ceph-mon[32619]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:21:06.200 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:05 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:21:06.200 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:05 smithi134 ceph-mon[32619]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-24T23:21:06.200 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:05 smithi134 ceph-mon[32619]: monmap epoch 3 2024-04-24T23:21:06.200 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:05 smithi134 ceph-mon[32619]: fsid ec7885aa-0290-11ef-bc93-c7b262605968 2024-04-24T23:21:06.200 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:05 smithi134 ceph-mon[32619]: last_changed 2024-04-24T23:21:00.713440+0000 2024-04-24T23:21:06.200 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:05 smithi134 ceph-mon[32619]: created 2024-04-24T23:18:58.816011+0000 2024-04-24T23:21:06.200 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:05 smithi134 ceph-mon[32619]: min_mon_release 19 (squid) 2024-04-24T23:21:06.200 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:05 smithi134 ceph-mon[32619]: election_strategy: 1 2024-04-24T23:21:06.200 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:05 smithi134 ceph-mon[32619]: 0: [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] mon.a 2024-04-24T23:21:06.200 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:05 smithi134 ceph-mon[32619]: 1: [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] mon.c 2024-04-24T23:21:06.200 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:05 smithi134 ceph-mon[32619]: 2: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.b 2024-04-24T23:21:06.200 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:05 smithi134 ceph-mon[32619]: fsmap 2024-04-24T23:21:06.200 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:05 smithi134 ceph-mon[32619]: osdmap e4: 0 total, 0 up, 0 in 2024-04-24T23:21:06.200 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:05 smithi134 ceph-mon[32619]: mgrmap e13: x(active, since 58s) 2024-04-24T23:21:06.200 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:05 smithi134 ceph-mon[32619]: overall HEALTH_OK 2024-04-24T23:21:06.201 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:05 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:06.201 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:05 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:06.201 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:05 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:06.201 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:05 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:06.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:05 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T23:21:06.258 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:05 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:21:06.258 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:05 smithi148 ceph-mon[32528]: mon.a calling monitor election 2024-04-24T23:21:06.258 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:05 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:21:06.258 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:05 smithi148 ceph-mon[32528]: mon.c calling monitor election 2024-04-24T23:21:06.258 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:05 smithi148 ceph-mon[32528]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:21:06.258 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:05 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:21:06.258 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:05 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:21:06.258 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:05 smithi148 ceph-mon[32528]: mon.b calling monitor election 2024-04-24T23:21:06.258 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:05 smithi148 ceph-mon[32528]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:21:06.258 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:05 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:21:06.258 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:05 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:21:06.259 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:05 smithi148 ceph-mon[32528]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:21:06.259 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:05 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:21:06.259 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:05 smithi148 ceph-mon[32528]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-24T23:21:06.259 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:05 smithi148 ceph-mon[32528]: monmap epoch 3 2024-04-24T23:21:06.259 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:05 smithi148 ceph-mon[32528]: fsid ec7885aa-0290-11ef-bc93-c7b262605968 2024-04-24T23:21:06.259 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:05 smithi148 ceph-mon[32528]: last_changed 2024-04-24T23:21:00.713440+0000 2024-04-24T23:21:06.259 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:05 smithi148 ceph-mon[32528]: created 2024-04-24T23:18:58.816011+0000 2024-04-24T23:21:06.259 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:05 smithi148 ceph-mon[32528]: min_mon_release 19 (squid) 2024-04-24T23:21:06.259 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:05 smithi148 ceph-mon[32528]: election_strategy: 1 2024-04-24T23:21:06.259 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:05 smithi148 ceph-mon[32528]: 0: [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] mon.a 2024-04-24T23:21:06.259 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:05 smithi148 ceph-mon[32528]: 1: [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] mon.c 2024-04-24T23:21:06.259 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:05 smithi148 ceph-mon[32528]: 2: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.b 2024-04-24T23:21:06.259 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:05 smithi148 ceph-mon[32528]: fsmap 2024-04-24T23:21:06.259 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:05 smithi148 ceph-mon[32528]: osdmap e4: 0 total, 0 up, 0 in 2024-04-24T23:21:06.259 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:05 smithi148 ceph-mon[32528]: mgrmap e13: x(active, since 58s) 2024-04-24T23:21:06.260 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:05 smithi148 ceph-mon[32528]: overall HEALTH_OK 2024-04-24T23:21:06.260 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:05 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:06.260 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:05 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:06.260 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:05 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:06.260 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:05 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:06.893 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:06 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:21:06.893 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:06 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:21:06.948 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:06 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:21:06.948 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:06 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:21:07.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:21:07.006+0000 7f1f1040f700 1 -- 172.21.15.148:0/1347019808 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1ef4006240 msgr2=0x7f1ef40086b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:07.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:21:07.006+0000 7f1f1040f700 1 --2- 172.21.15.148:0/1347019808 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1ef4006240 0x7f1ef40086b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1f08101480 tx=0x7f1f0400fe90 comp rx=0 tx=0).stop 2024-04-24T23:21:07.009 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:21:07.006+0000 7f1f1040f700 1 -- 172.21.15.148:0/1347019808 shutdown_connections 2024-04-24T23:21:07.009 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:21:07.006+0000 7f1f1040f700 1 --2- 172.21.15.148:0/1347019808 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1ef4006240 0x7f1ef40086b0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:07.009 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:21:07.006+0000 7f1f1040f700 1 -- 172.21.15.148:0/1347019808 >> 172.21.15.148:0/1347019808 conn(0x7f1f08075f90 msgr2=0x7f1f080763a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:07.009 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:21:07.006+0000 7f1f1040f700 1 -- 172.21.15.148:0/1347019808 shutdown_connections 2024-04-24T23:21:07.009 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:21:07.007+0000 7f1f1040f700 1 -- 172.21.15.148:0/1347019808 wait complete. 2024-04-24T23:21:07.012 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:21:07.007+0000 7f1f1040f700 1 Processor -- start 2024-04-24T23:21:07.012 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:21:07.007+0000 7f1f1040f700 1 -- start start 2024-04-24T23:21:07.012 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:21:07.008+0000 7f1f1040f700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1ef4006240 0x7f1f08101650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:07.012 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:21:07.008+0000 7f1f1040f700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1f08101b90 0x7f1f08102010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:07.012 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:21:07.008+0000 7f1f1040f700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f081a3bd0 0x7f1f081a5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:07.012 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:21:07.008+0000 7f1f1040f700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f1f08109db0 con 0x7f1ef4006240 2024-04-24T23:21:07.012 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:21:07.008+0000 7f1f1040f700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1f08109c30 con 0x7f1f08101b90 2024-04-24T23:21:07.012 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:21:07.008+0000 7f1f1040f700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1f08109f30 con 0x7f1f081a3bd0 2024-04-24T23:21:07.012 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:21:07.008+0000 7f1f0e9ac700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f081a3bd0 0x7f1f081a5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:07.012 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:21:07.008+0000 7f1f0e9ac700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f081a3bd0 0x7f1f081a5fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42878/0 (socket says 172.21.15.148:42878) 2024-04-24T23:21:07.012 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:21:07.008+0000 7f1f0e9ac700 1 -- 172.21.15.148:0/1328010531 learned_addr learned my addr 172.21.15.148:0/1328010531 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-24T23:21:07.013 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:21:07.008+0000 7f1f0d9aa700 1 --2- 172.21.15.148:0/1328010531 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1f08101b90 0x7f1f08102010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:07.013 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:21:07.008+0000 7f1f0e1ab700 1 --2- 172.21.15.148:0/1328010531 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1ef4006240 0x7f1f08101650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:07.013 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:21:07.008+0000 7f1f0e9ac700 1 -- 172.21.15.148:0/1328010531 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1f08101b90 msgr2=0x7f1f08102010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:07.013 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:21:07.008+0000 7f1f0e9ac700 1 --2- 172.21.15.148:0/1328010531 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1f08101b90 0x7f1f08102010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:07.013 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:21:07.008+0000 7f1f0e9ac700 1 -- 172.21.15.148:0/1328010531 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1ef4006240 msgr2=0x7f1f08101650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:07.013 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:21:07.008+0000 7f1f0e9ac700 1 --2- 172.21.15.148:0/1328010531 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1ef4006240 0x7f1f08101650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:07.013 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:21:07.008+0000 7f1f0e9ac700 1 -- 172.21.15.148:0/1328010531 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f081a6690 con 0x7f1f081a3bd0 2024-04-24T23:21:07.013 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:21:07.008+0000 7f1f0e9ac700 1 --2- 172.21.15.148:0/1328010531 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f081a3bd0 0x7f1f081a5fc0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1f0000ba80 tx=0x7f1f0000bd90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:07.013 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:21:07.008+0000 7f1eff7fe700 1 -- 172.21.15.148:0/1328010531 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f00019070 con 0x7f1f081a3bd0 2024-04-24T23:21:07.013 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:21:07.008+0000 7f1eff7fe700 1 -- 172.21.15.148:0/1328010531 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f00014440 con 0x7f1f081a3bd0 2024-04-24T23:21:07.013 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:21:07.008+0000 7f1f1040f700 1 -- 172.21.15.148:0/1328010531 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f081a6980 con 0x7f1f081a3bd0 2024-04-24T23:21:07.013 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:21:07.008+0000 7f1eff7fe700 1 -- 172.21.15.148:0/1328010531 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f000075b0 con 0x7f1f081a3bd0 2024-04-24T23:21:07.013 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:21:07.009+0000 7f1f1040f700 1 -- 172.21.15.148:0/1328010531 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f081a6e70 con 0x7f1f081a3bd0 2024-04-24T23:21:07.013 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:21:07.010+0000 7f1eff7fe700 1 -- 172.21.15.148:0/1328010531 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f1f0001c070 con 0x7f1f081a3bd0 2024-04-24T23:21:07.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:21:07.010+0000 7f1f1040f700 1 -- 172.21.15.148:0/1328010531 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f08063890 con 0x7f1f081a3bd0 2024-04-24T23:21:07.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:21:07.010+0000 7f1eff7fe700 1 --2- 172.21.15.148:0/1328010531 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f1eec039580 0x7f1eec03ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:07.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:21:07.010+0000 7f1eff7fe700 1 -- 172.21.15.148:0/1328010531 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f1f0004b9a0 con 0x7f1f081a3bd0 2024-04-24T23:21:07.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:21:07.010+0000 7f1f0e1ab700 1 --2- 172.21.15.148:0/1328010531 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f1eec039580 0x7f1eec03ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:07.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:21:07.011+0000 7f1f0e1ab700 1 --2- 172.21.15.148:0/1328010531 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f1eec039580 0x7f1eec03ba40 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1f0400fa40 tx=0x7f1f04005a90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:07.016 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:21:07.015+0000 7f1eff7fe700 1 -- 172.21.15.148:0/1328010531 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f1f00012500 con 0x7f1f081a3bd0 2024-04-24T23:21:07.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:06 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:21:07.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:06 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:21:07.315 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:21:07.313+0000 7f1f1040f700 1 -- 172.21.15.148:0/1328010531 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f1f0802dcf0 con 0x7f1f081a3bd0 2024-04-24T23:21:07.315 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:21:07.313+0000 7f1eff7fe700 1 -- 172.21.15.148:0/1328010531 <== mon.1 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1285 (secure 0 0 0) 0x7f1f00012760 con 0x7f1f081a3bd0 2024-04-24T23:21:07.315 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-24T23:21:07.316 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":3,"fsid":"ec7885aa-0290-11ef-bc93-c7b262605968","modified":"2024-04-24T23:21:00.713440Z","created":"2024-04-24T23:18:58.816011Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:3300","nonce":0},{"type":"v1","addr":"172.21.15.80:6789","nonce":0}]},"addr":"172.21.15.80:6789/0","public_addr":"172.21.15.80:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:3300","nonce":0},{"type":"v1","addr":"172.21.15.148:6789","nonce":0}]},"addr":"172.21.15.148:6789/0","public_addr":"172.21.15.148:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-04-24T23:21:07.317 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:21:07.316+0000 7f1f1040f700 1 -- 172.21.15.148:0/1328010531 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f1eec039580 msgr2=0x7f1eec03ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:07.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:21:07.316+0000 7f1f1040f700 1 --2- 172.21.15.148:0/1328010531 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f1eec039580 0x7f1eec03ba40 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1f0400fa40 tx=0x7f1f04005a90 comp rx=0 tx=0).stop 2024-04-24T23:21:07.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:21:07.316+0000 7f1f1040f700 1 -- 172.21.15.148:0/1328010531 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f081a3bd0 msgr2=0x7f1f081a5fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:07.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:21:07.316+0000 7f1f1040f700 1 --2- 172.21.15.148:0/1328010531 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f081a3bd0 0x7f1f081a5fc0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1f0000ba80 tx=0x7f1f0000bd90 comp rx=0 tx=0).stop 2024-04-24T23:21:07.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:21:07.316+0000 7f1f1040f700 1 -- 172.21.15.148:0/1328010531 shutdown_connections 2024-04-24T23:21:07.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:21:07.316+0000 7f1f1040f700 1 --2- 172.21.15.148:0/1328010531 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f081a3bd0 0x7f1f081a5fc0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:07.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:21:07.316+0000 7f1f1040f700 1 --2- 172.21.15.148:0/1328010531 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f1eec039580 0x7f1eec03ba40 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:07.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:21:07.316+0000 7f1f1040f700 1 --2- 172.21.15.148:0/1328010531 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1ef4006240 0x7f1f08101650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:07.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:21:07.316+0000 7f1f1040f700 1 --2- 172.21.15.148:0/1328010531 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1f08101b90 0x7f1f08102010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:07.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:21:07.316+0000 7f1f1040f700 1 -- 172.21.15.148:0/1328010531 >> 172.21.15.148:0/1328010531 conn(0x7f1f08075f90 msgr2=0x7f1f08103ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:07.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:21:07.316+0000 7f1f1040f700 1 -- 172.21.15.148:0/1328010531 shutdown_connections 2024-04-24T23:21:07.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:21:07.316+0000 7f1f1040f700 1 -- 172.21.15.148:0/1328010531 wait complete. 2024-04-24T23:21:07.319 INFO:teuthology.orchestra.run.smithi148.stderr:dumped monmap epoch 3 2024-04-24T23:21:08.008 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:07 smithi148 ceph-mon[32528]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:21:08.008 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:07 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:08.009 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:07 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:08.009 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:07 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:08.009 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:07 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:08.009 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:07 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:08.009 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:07 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:21:08.009 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:07 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:21:08.009 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:07 smithi148 ceph-mon[32528]: from='client.? 172.21.15.148:0/1328010531' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T23:21:08.009 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:07 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:08.009 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:07 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:08.009 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:07 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:08.009 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:07 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:08.009 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:07 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:08.009 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:07 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:08.009 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:07 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:08.010 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:07 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:08.010 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:07 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:08.010 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:07 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:08.010 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:07 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:08.010 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:07 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T23:21:08.010 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:07 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T23:21:08.010 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:07 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:21:08.150 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-04-24T23:21:08.150 DEBUG:teuthology.orchestra.run.smithi080:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ec7885aa-0290-11ef-bc93-c7b262605968 -- ceph config generate-minimal-conf 2024-04-24T23:21:08.172 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:07 smithi080 ceph-mon[29152]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:21:08.172 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:07 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:08.172 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:07 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:08.172 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:07 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:08.172 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:07 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:08.172 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:07 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:08.172 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:07 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:21:08.172 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:07 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:21:08.172 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:07 smithi080 ceph-mon[29152]: from='client.? 172.21.15.148:0/1328010531' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T23:21:08.172 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:07 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:08.173 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:07 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:08.173 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:07 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:08.173 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:07 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:08.173 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:07 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:08.173 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:07 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:08.173 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:07 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:08.173 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:07 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:08.173 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:07 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:08.173 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:07 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:08.173 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:07 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:08.173 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:07 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T23:21:08.173 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:07 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T23:21:08.173 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:07 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:21:08.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:07 smithi134 ceph-mon[32619]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:21:08.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:07 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:08.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:07 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:08.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:07 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:08.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:07 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:08.199 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:07 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:08.199 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:07 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:21:08.199 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:07 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:21:08.199 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:07 smithi134 ceph-mon[32619]: from='client.? 172.21.15.148:0/1328010531' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T23:21:08.199 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:07 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:08.199 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:07 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:08.199 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:07 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:08.199 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:07 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:08.199 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:07 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:08.199 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:07 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:08.199 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:07 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:08.199 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:07 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:08.199 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:07 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:08.199 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:07 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:08.199 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:07 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:08.200 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:07 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T23:21:08.200 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:07 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T23:21:08.200 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:07 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:21:08.965 INFO:teuthology.orchestra.run.smithi080.stderr:Inferring config /var/lib/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:21:09.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:08 smithi080 ceph-mon[29152]: Updating smithi080:/etc/ceph/ceph.conf 2024-04-24T23:21:09.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:08 smithi080 ceph-mon[29152]: Updating smithi134:/etc/ceph/ceph.conf 2024-04-24T23:21:09.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:08 smithi080 ceph-mon[29152]: Updating smithi148:/etc/ceph/ceph.conf 2024-04-24T23:21:09.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:08 smithi080 ceph-mon[29152]: Updating smithi080:/var/lib/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T23:21:09.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:08 smithi080 ceph-mon[29152]: Updating smithi134:/var/lib/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T23:21:09.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:08 smithi080 ceph-mon[29152]: Updating smithi148:/var/lib/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T23:21:09.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:08 smithi080 ceph-mon[29152]: Reconfiguring mon.a (unknown last config time)... 2024-04-24T23:21:09.040 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:08 smithi080 ceph-mon[29152]: Reconfiguring daemon mon.a on smithi080 2024-04-24T23:21:09.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:08 smithi134 ceph-mon[32619]: Updating smithi080:/etc/ceph/ceph.conf 2024-04-24T23:21:09.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:08 smithi134 ceph-mon[32619]: Updating smithi134:/etc/ceph/ceph.conf 2024-04-24T23:21:09.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:08 smithi134 ceph-mon[32619]: Updating smithi148:/etc/ceph/ceph.conf 2024-04-24T23:21:09.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:08 smithi134 ceph-mon[32619]: Updating smithi080:/var/lib/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T23:21:09.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:08 smithi134 ceph-mon[32619]: Updating smithi134:/var/lib/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T23:21:09.199 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:08 smithi134 ceph-mon[32619]: Updating smithi148:/var/lib/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T23:21:09.199 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:08 smithi134 ceph-mon[32619]: Reconfiguring mon.a (unknown last config time)... 2024-04-24T23:21:09.199 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:08 smithi134 ceph-mon[32619]: Reconfiguring daemon mon.a on smithi080 2024-04-24T23:21:09.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:08 smithi148 ceph-mon[32528]: Updating smithi080:/etc/ceph/ceph.conf 2024-04-24T23:21:09.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:08 smithi148 ceph-mon[32528]: Updating smithi134:/etc/ceph/ceph.conf 2024-04-24T23:21:09.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:08 smithi148 ceph-mon[32528]: Updating smithi148:/etc/ceph/ceph.conf 2024-04-24T23:21:09.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:08 smithi148 ceph-mon[32528]: Updating smithi080:/var/lib/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T23:21:09.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:08 smithi148 ceph-mon[32528]: Updating smithi134:/var/lib/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T23:21:09.258 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:08 smithi148 ceph-mon[32528]: Updating smithi148:/var/lib/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T23:21:09.258 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:08 smithi148 ceph-mon[32528]: Reconfiguring mon.a (unknown last config time)... 2024-04-24T23:21:09.258 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:08 smithi148 ceph-mon[32528]: Reconfiguring daemon mon.a on smithi080 2024-04-24T23:21:10.126 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:09 smithi134 ceph-mon[32619]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:21:10.126 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:09 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:10.126 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:09 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:10.126 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:09 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T23:21:10.126 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:09 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T23:21:10.127 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:09 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:21:10.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:09 smithi148 ceph-mon[32528]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:21:10.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:09 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:10.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:09 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:10.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:09 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T23:21:10.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:09 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T23:21:10.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:09 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:21:10.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:09 smithi080 ceph-mon[29152]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:21:10.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:09 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:10.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:09 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:10.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:09 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T23:21:10.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:09 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T23:21:10.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:09 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:21:10.997 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:10 smithi134 ceph-mon[32619]: Reconfiguring mon.b (monmap changed)... 2024-04-24T23:21:10.997 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:10 smithi134 ceph-mon[32619]: Reconfiguring daemon mon.b on smithi134 2024-04-24T23:21:10.997 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:10 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:10.998 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:10 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:11.013 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:10 smithi080 ceph-mon[29152]: Reconfiguring mon.b (monmap changed)... 2024-04-24T23:21:11.013 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:10 smithi080 ceph-mon[29152]: Reconfiguring daemon mon.b on smithi134 2024-04-24T23:21:11.013 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:10 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:11.013 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:10 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:11.014 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:21:11.011+0000 7f9f20972700 1 -- 172.21.15.80:0/2501846715 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9f1c100780 msgr2=0x7f9f1c100c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:11.014 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:21:11.011+0000 7f9f20972700 1 --2- 172.21.15.80:0/2501846715 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9f1c100780 0x7f9f1c100c00 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9f0c0099c0 tx=0x7f9f0c009cd0 comp rx=0 tx=0).stop 2024-04-24T23:21:11.014 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:21:11.011+0000 7f9f20972700 1 -- 172.21.15.80:0/2501846715 shutdown_connections 2024-04-24T23:21:11.014 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:21:11.011+0000 7f9f20972700 1 --2- 172.21.15.80:0/2501846715 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f1c101140 0x7f9f1c10f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:11.014 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:21:11.011+0000 7f9f20972700 1 --2- 172.21.15.80:0/2501846715 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9f1c107250 0x7f9f1c107630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:11.014 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:21:11.011+0000 7f9f20972700 1 --2- 172.21.15.80:0/2501846715 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9f1c100780 0x7f9f1c100c00 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:11.014 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:21:11.011+0000 7f9f20972700 1 -- 172.21.15.80:0/2501846715 >> 172.21.15.80:0/2501846715 conn(0x7f9f1c077730 msgr2=0x7f9f1c0feab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:11.014 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:21:11.011+0000 7f9f20972700 1 -- 172.21.15.80:0/2501846715 shutdown_connections 2024-04-24T23:21:11.014 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:21:11.011+0000 7f9f20972700 1 -- 172.21.15.80:0/2501846715 wait complete. 2024-04-24T23:21:11.015 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:21:11.012+0000 7f9f20972700 1 Processor -- start 2024-04-24T23:21:11.016 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:21:11.012+0000 7f9f20972700 1 -- start start 2024-04-24T23:21:11.016 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:21:11.012+0000 7f9f20972700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f1c100780 0x7f9f1c19e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:11.016 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:21:11.012+0000 7f9f20972700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9f1c101140 0x7f9f1c19eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:11.016 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:21:11.012+0000 7f9f20972700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9f1c107250 0x7f9f1c198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:11.016 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:21:11.012+0000 7f9f20972700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f9f1c112680 con 0x7f9f1c107250 2024-04-24T23:21:11.016 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:21:11.012+0000 7f9f20972700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9f1c112500 con 0x7f9f1c101140 2024-04-24T23:21:11.016 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:21:11.013+0000 7f9f20972700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f9f1c112800 con 0x7f9f1c100780 2024-04-24T23:21:11.016 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:21:11.013+0000 7f9f1ad9d700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9f1c107250 0x7f9f1c198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:11.017 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:21:11.013+0000 7f9f1a59c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f1c100780 0x7f9f1c19e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:11.017 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:21:11.013+0000 7f9f1a59c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f1c100780 0x7f9f1c19e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.80:53394/0 (socket says 172.21.15.80:53394) 2024-04-24T23:21:11.017 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:21:11.013+0000 7f9f1ad9d700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9f1c107250 0x7f9f1c198920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:45088/0 (socket says 172.21.15.80:45088) 2024-04-24T23:21:11.017 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:21:11.013+0000 7f9f1a59c700 1 -- 172.21.15.80:0/1843618857 learned_addr learned my addr 172.21.15.80:0/1843618857 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:21:11.017 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:21:11.013+0000 7f9f19d9b700 1 --2- 172.21.15.80:0/1843618857 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9f1c101140 0x7f9f1c19eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:11.017 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:21:11.013+0000 7f9f1a59c700 1 -- 172.21.15.80:0/1843618857 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9f1c101140 msgr2=0x7f9f1c19eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:11.017 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:21:11.013+0000 7f9f1a59c700 1 --2- 172.21.15.80:0/1843618857 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9f1c101140 0x7f9f1c19eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:11.017 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:21:11.013+0000 7f9f1a59c700 1 -- 172.21.15.80:0/1843618857 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9f1c107250 msgr2=0x7f9f1c198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:11.017 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:21:11.013+0000 7f9f1a59c700 1 --2- 172.21.15.80:0/1843618857 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9f1c107250 0x7f9f1c198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:11.017 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:21:11.013+0000 7f9f1a59c700 1 -- 172.21.15.80:0/1843618857 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f1c199080 con 0x7f9f1c100780 2024-04-24T23:21:11.017 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:21:11.013+0000 7f9f1ad9d700 1 --2- 172.21.15.80:0/1843618857 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9f1c107250 0x7f9f1c198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:21:11.017 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:21:11.013+0000 7f9f1a59c700 1 --2- 172.21.15.80:0/1843618857 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f1c100780 0x7f9f1c19e760 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9f1000d950 tx=0x7f9f1000dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:11.017 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:21:11.014+0000 7f9f0b7fe700 1 -- 172.21.15.80:0/1843618857 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f10016070 con 0x7f9f1c100780 2024-04-24T23:21:11.018 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:21:11.014+0000 7f9f0b7fe700 1 -- 172.21.15.80:0/1843618857 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f1000fd70 con 0x7f9f1c100780 2024-04-24T23:21:11.018 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:21:11.014+0000 7f9f20972700 1 -- 172.21.15.80:0/1843618857 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f1c199370 con 0x7f9f1c100780 2024-04-24T23:21:11.018 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:21:11.014+0000 7f9f0b7fe700 1 -- 172.21.15.80:0/1843618857 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f1000e610 con 0x7f9f1c100780 2024-04-24T23:21:11.018 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:21:11.014+0000 7f9f20972700 1 -- 172.21.15.80:0/1843618857 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f1c101890 con 0x7f9f1c100780 2024-04-24T23:21:11.019 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:21:11.015+0000 7f9f20972700 1 -- 172.21.15.80:0/1843618857 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f1c199e50 con 0x7f9f1c100780 2024-04-24T23:21:11.019 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:21:11.015+0000 7f9f0b7fe700 1 -- 172.21.15.80:0/1843618857 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f9f10019070 con 0x7f9f1c100780 2024-04-24T23:21:11.019 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:21:11.015+0000 7f9f0b7fe700 1 --2- 172.21.15.80:0/1843618857 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f9f04039330 0x7f9f0403b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:11.019 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:21:11.015+0000 7f9f0b7fe700 1 -- 172.21.15.80:0/1843618857 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f9f1004bd50 con 0x7f9f1c100780 2024-04-24T23:21:11.019 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:21:11.015+0000 7f9f19d9b700 1 --2- 172.21.15.80:0/1843618857 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f9f04039330 0x7f9f0403b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:11.019 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:21:11.016+0000 7f9f19d9b700 1 --2- 172.21.15.80:0/1843618857 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f9f04039330 0x7f9f0403b7f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c0ffcf0 tx=0x7f9f0c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:11.022 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:21:11.021+0000 7f9f0b7fe700 1 -- 172.21.15.80:0/1843618857 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f9f10010e10 con 0x7f9f1c100780 2024-04-24T23:21:11.228 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:21:11.226+0000 7f9f20972700 1 -- 172.21.15.80:0/1843618857 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f9f1c02dcf0 con 0x7f9f1c100780 2024-04-24T23:21:11.228 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:21:11.226+0000 7f9f0b7fe700 1 -- 172.21.15.80:0/1843618857 <== mon.1 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+275 (secure 0 0 0) 0x7f9f1000ed70 con 0x7f9f1c100780 2024-04-24T23:21:11.228 INFO:teuthology.orchestra.run.smithi080.stdout:# minimal ceph.conf for ec7885aa-0290-11ef-bc93-c7b262605968 2024-04-24T23:21:11.228 INFO:teuthology.orchestra.run.smithi080.stdout:[global] 2024-04-24T23:21:11.229 INFO:teuthology.orchestra.run.smithi080.stdout: fsid = ec7885aa-0290-11ef-bc93-c7b262605968 2024-04-24T23:21:11.229 INFO:teuthology.orchestra.run.smithi080.stdout: mon_host = [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] 2024-04-24T23:21:11.230 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:21:11.228+0000 7f9f20972700 1 -- 172.21.15.80:0/1843618857 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f9f04039330 msgr2=0x7f9f0403b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:11.230 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:21:11.228+0000 7f9f20972700 1 --2- 172.21.15.80:0/1843618857 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f9f04039330 0x7f9f0403b7f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c0ffcf0 tx=0x7f9f0c005fb0 comp rx=0 tx=0).stop 2024-04-24T23:21:11.230 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:21:11.228+0000 7f9f20972700 1 -- 172.21.15.80:0/1843618857 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f1c100780 msgr2=0x7f9f1c19e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:11.231 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:21:11.228+0000 7f9f20972700 1 --2- 172.21.15.80:0/1843618857 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f1c100780 0x7f9f1c19e760 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9f1000d950 tx=0x7f9f1000dc60 comp rx=0 tx=0).stop 2024-04-24T23:21:11.231 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:21:11.228+0000 7f9f20972700 1 -- 172.21.15.80:0/1843618857 shutdown_connections 2024-04-24T23:21:11.231 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:21:11.228+0000 7f9f20972700 1 --2- 172.21.15.80:0/1843618857 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f9f04039330 0x7f9f0403b7f0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:11.231 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:21:11.228+0000 7f9f20972700 1 --2- 172.21.15.80:0/1843618857 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9f1c107250 0x7f9f1c198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:11.231 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:21:11.228+0000 7f9f20972700 1 --2- 172.21.15.80:0/1843618857 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f1c100780 0x7f9f1c19e760 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:11.231 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:21:11.228+0000 7f9f20972700 1 --2- 172.21.15.80:0/1843618857 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9f1c101140 0x7f9f1c19eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:11.231 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:21:11.228+0000 7f9f20972700 1 -- 172.21.15.80:0/1843618857 >> 172.21.15.80:0/1843618857 conn(0x7f9f1c077730 msgr2=0x7f9f1c0fcce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:11.231 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:21:11.228+0000 7f9f20972700 1 -- 172.21.15.80:0/1843618857 shutdown_connections 2024-04-24T23:21:11.231 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:21:11.228+0000 7f9f20972700 1 -- 172.21.15.80:0/1843618857 wait complete. 2024-04-24T23:21:11.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:10 smithi148 ceph-mon[32528]: Reconfiguring mon.b (monmap changed)... 2024-04-24T23:21:11.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:10 smithi148 ceph-mon[32528]: Reconfiguring daemon mon.b on smithi134 2024-04-24T23:21:11.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:10 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:11.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:10 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:11.861 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-04-24T23:21:11.861 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-24T23:21:11.861 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd of=/etc/ceph/ceph.conf 2024-04-24T23:21:11.928 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-24T23:21:11.928 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-24T23:21:11.991 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-24T23:21:11.991 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/etc/ceph/ceph.conf 2024-04-24T23:21:12.007 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:11 smithi148 ceph-mon[32528]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:21:12.007 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:11 smithi148 ceph-mon[32528]: Reconfiguring mon.c (monmap changed)... 2024-04-24T23:21:12.007 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:11 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T23:21:12.007 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:11 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T23:21:12.007 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:11 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:21:12.008 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:11 smithi148 ceph-mon[32528]: Reconfiguring daemon mon.c on smithi148 2024-04-24T23:21:12.008 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:11 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/1843618857' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:21:12.020 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-24T23:21:12.021 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-24T23:21:12.086 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-24T23:21:12.086 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/etc/ceph/ceph.conf 2024-04-24T23:21:12.114 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-24T23:21:12.114 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-24T23:21:12.178 INFO:tasks.cephadm:Adding mgr.x on smithi080 2024-04-24T23:21:12.178 INFO:tasks.cephadm:Adding mgr.y on smithi134 2024-04-24T23:21:12.178 INFO:tasks.cephadm:Adding mgr.z on smithi148 2024-04-24T23:21:12.179 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ec7885aa-0290-11ef-bc93-c7b262605968 -- ceph orch apply mgr '3;smithi080=x;smithi134=y;smithi148=z' 2024-04-24T23:21:12.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:11 smithi134 ceph-mon[32619]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:21:12.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:11 smithi134 ceph-mon[32619]: Reconfiguring mon.c (monmap changed)... 2024-04-24T23:21:12.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:11 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T23:21:12.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:11 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T23:21:12.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:11 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:21:12.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:11 smithi134 ceph-mon[32619]: Reconfiguring daemon mon.c on smithi148 2024-04-24T23:21:12.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:11 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/1843618857' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:21:12.221 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:11 smithi080 ceph-mon[29152]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:21:12.222 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:11 smithi080 ceph-mon[29152]: Reconfiguring mon.c (monmap changed)... 2024-04-24T23:21:12.222 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:11 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T23:21:12.222 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:11 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T23:21:12.222 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:11 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:21:12.222 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:11 smithi080 ceph-mon[29152]: Reconfiguring daemon mon.c on smithi148 2024-04-24T23:21:12.222 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:11 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/1843618857' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:21:12.629 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T23:21:13.863 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:13 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:13.863 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:13 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:13.863 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:13 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:21:13.863 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:13 smithi148 ceph-mon[32528]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:21:13.863 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:13 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:21:13.863 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:13 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:21:13.863 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:13 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:13.948 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:13 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:13.948 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:13 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:13.948 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:13 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:21:13.949 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:13 smithi134 ceph-mon[32619]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:21:13.949 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:13 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:21:13.949 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:13 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:21:13.949 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:13 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:14.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:13 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:14.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:13 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:14.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:13 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:21:14.040 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:13 smithi080 ceph-mon[29152]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:21:14.040 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:13 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:21:14.040 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:13 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:21:14.040 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:13 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:14.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:21:14.054+0000 7fdbb3ddf700 1 -- 172.21.15.148:0/1594375700 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdbac102d20 msgr2=0x7fdbac105110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:14.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:21:14.054+0000 7fdbb3ddf700 1 --2- 172.21.15.148:0/1594375700 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdbac102d20 0x7fdbac105110 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fdba8009990 tx=0x7fdba8009ca0 comp rx=0 tx=0).stop 2024-04-24T23:21:14.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:21:14.054+0000 7fdbb3ddf700 1 -- 172.21.15.148:0/1594375700 shutdown_connections 2024-04-24T23:21:14.061 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:21:14.054+0000 7fdbb3ddf700 1 --2- 172.21.15.148:0/1594375700 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdbac107f80 0x7fdbac10a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:14.061 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:21:14.054+0000 7fdbb3ddf700 1 --2- 172.21.15.148:0/1594375700 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdbac102d20 0x7fdbac105110 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:14.061 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:21:14.054+0000 7fdbb3ddf700 1 --2- 172.21.15.148:0/1594375700 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdbac105650 0x7fdbac107a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:14.061 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:21:14.054+0000 7fdbb3ddf700 1 -- 172.21.15.148:0/1594375700 >> 172.21.15.148:0/1594375700 conn(0x7fdbac0fc730 msgr2=0x7fdbac0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:14.061 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:21:14.054+0000 7fdbb3ddf700 1 -- 172.21.15.148:0/1594375700 shutdown_connections 2024-04-24T23:21:14.061 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:21:14.055+0000 7fdbb3ddf700 1 -- 172.21.15.148:0/1594375700 wait complete. 2024-04-24T23:21:14.061 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:21:14.055+0000 7fdbb3ddf700 1 Processor -- start 2024-04-24T23:21:14.061 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:21:14.055+0000 7fdbb3ddf700 1 -- start start 2024-04-24T23:21:14.061 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:21:14.055+0000 7fdbb3ddf700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdbac102d20 0x7fdbac102680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:14.061 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:21:14.055+0000 7fdbb3ddf700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdbac105650 0x7fdbac100cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:14.061 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:21:14.055+0000 7fdbb3ddf700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdbac107f80 0x7fdbac101210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:14.061 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:21:14.055+0000 7fdbb3ddf700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fdbac10dea0 con 0x7fdbac102d20 2024-04-24T23:21:14.061 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:21:14.055+0000 7fdbb3ddf700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fdbac10dd20 con 0x7fdbac105650 2024-04-24T23:21:14.062 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:21:14.055+0000 7fdbb3ddf700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fdbac10e020 con 0x7fdbac107f80 2024-04-24T23:21:14.062 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:21:14.055+0000 7fdbb237c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdbac107f80 0x7fdbac101210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:14.062 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:21:14.055+0000 7fdbb237c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdbac107f80 0x7fdbac101210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:35346/0 (socket says 172.21.15.148:35346) 2024-04-24T23:21:14.062 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:21:14.055+0000 7fdbb237c700 1 -- 172.21.15.148:0/1975592894 learned_addr learned my addr 172.21.15.148:0/1975592894 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-24T23:21:14.062 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:21:14.056+0000 7fdbb137a700 1 --2- 172.21.15.148:0/1975592894 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdbac105650 0x7fdbac100cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:14.062 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:21:14.056+0000 7fdbb1b7b700 1 --2- 172.21.15.148:0/1975592894 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdbac102d20 0x7fdbac102680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:14.062 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:21:14.056+0000 7fdbb237c700 1 -- 172.21.15.148:0/1975592894 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdbac105650 msgr2=0x7fdbac100cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:14.062 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:21:14.056+0000 7fdbb237c700 1 --2- 172.21.15.148:0/1975592894 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdbac105650 0x7fdbac100cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:14.062 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:21:14.056+0000 7fdbb237c700 1 -- 172.21.15.148:0/1975592894 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdbac102d20 msgr2=0x7fdbac102680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:14.062 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:21:14.056+0000 7fdbb237c700 1 --2- 172.21.15.148:0/1975592894 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdbac102d20 0x7fdbac102680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:14.062 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:21:14.056+0000 7fdbb237c700 1 -- 172.21.15.148:0/1975592894 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbac101ac0 con 0x7fdbac107f80 2024-04-24T23:21:14.062 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:21:14.056+0000 7fdbb1b7b700 1 --2- 172.21.15.148:0/1975592894 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdbac102d20 0x7fdbac102680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:21:14.062 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:21:14.056+0000 7fdbb237c700 1 --2- 172.21.15.148:0/1975592894 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdbac107f80 0x7fdbac101210 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fdba400ada0 tx=0x7fdba400c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:14.063 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:21:14.056+0000 7fdba2ffd700 1 -- 172.21.15.148:0/1975592894 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba4017020 con 0x7fdbac107f80 2024-04-24T23:21:14.063 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:21:14.056+0000 7fdbb3ddf700 1 -- 172.21.15.148:0/1975592894 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbac1a2680 con 0x7fdbac107f80 2024-04-24T23:21:14.063 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:21:14.056+0000 7fdba2ffd700 1 -- 172.21.15.148:0/1975592894 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdba400a0d0 con 0x7fdbac107f80 2024-04-24T23:21:14.063 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:21:14.056+0000 7fdba2ffd700 1 -- 172.21.15.148:0/1975592894 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba4005440 con 0x7fdbac107f80 2024-04-24T23:21:14.063 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:21:14.056+0000 7fdbb3ddf700 1 -- 172.21.15.148:0/1975592894 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbac1a2b90 con 0x7fdbac107f80 2024-04-24T23:21:14.063 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:21:14.057+0000 7fdba2ffd700 1 -- 172.21.15.148:0/1975592894 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fdba40055e0 con 0x7fdbac107f80 2024-04-24T23:21:14.063 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:21:14.057+0000 7fdbb3ddf700 1 -- 172.21.15.148:0/1975592894 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbac068f20 con 0x7fdbac107f80 2024-04-24T23:21:14.063 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:21:14.057+0000 7fdba2ffd700 1 --2- 172.21.15.148:0/1975592894 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fdb98039330 0x7fdb9803b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:14.063 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:21:14.057+0000 7fdba2ffd700 1 -- 172.21.15.148:0/1975592894 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fdba404bbd0 con 0x7fdbac107f80 2024-04-24T23:21:14.063 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:21:14.057+0000 7fdbb1b7b700 1 --2- 172.21.15.148:0/1975592894 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fdb98039330 0x7fdb9803b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:14.064 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:21:14.058+0000 7fdbb1b7b700 1 --2- 172.21.15.148:0/1975592894 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fdb98039330 0x7fdb9803b7f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fdba80039a0 tx=0x7fdba800b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:14.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:21:14.062+0000 7fdba2ffd700 1 -- 172.21.15.148:0/1975592894 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fdba4021020 con 0x7fdbac107f80 2024-04-24T23:21:14.272 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:21:14.269+0000 7fdbb3ddf700 1 -- 172.21.15.148:0/1975592894 --> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi080=x;smithi134=y;smithi148=z", "target": ["mon-mgr", ""]}) -- 0x7fdbac062670 con 0x7fdb98039330 2024-04-24T23:21:14.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:21:14.275+0000 7fdba2ffd700 1 -- 172.21.15.148:0/1975592894 <== mgr.14152 v2:172.21.15.80:6800/3830596922 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fdbac062670 con 0x7fdb98039330 2024-04-24T23:21:14.278 INFO:teuthology.orchestra.run.smithi148.stdout:Scheduled mgr update... 2024-04-24T23:21:14.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:21:14.277+0000 7fdbb3ddf700 1 -- 172.21.15.148:0/1975592894 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fdb98039330 msgr2=0x7fdb9803b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:14.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:21:14.277+0000 7fdbb3ddf700 1 --2- 172.21.15.148:0/1975592894 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fdb98039330 0x7fdb9803b7f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fdba80039a0 tx=0x7fdba800b540 comp rx=0 tx=0).stop 2024-04-24T23:21:14.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:21:14.277+0000 7fdbb3ddf700 1 -- 172.21.15.148:0/1975592894 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdbac107f80 msgr2=0x7fdbac101210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:14.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:21:14.277+0000 7fdbb3ddf700 1 --2- 172.21.15.148:0/1975592894 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdbac107f80 0x7fdbac101210 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fdba400ada0 tx=0x7fdba400c330 comp rx=0 tx=0).stop 2024-04-24T23:21:14.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:21:14.277+0000 7fdbb3ddf700 1 -- 172.21.15.148:0/1975592894 shutdown_connections 2024-04-24T23:21:14.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:21:14.277+0000 7fdbb3ddf700 1 --2- 172.21.15.148:0/1975592894 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdbac107f80 0x7fdbac101210 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:14.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:21:14.277+0000 7fdbb3ddf700 1 --2- 172.21.15.148:0/1975592894 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fdb98039330 0x7fdb9803b7f0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:14.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:21:14.277+0000 7fdbb3ddf700 1 --2- 172.21.15.148:0/1975592894 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdbac102d20 0x7fdbac102680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:14.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:21:14.277+0000 7fdbb3ddf700 1 --2- 172.21.15.148:0/1975592894 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdbac105650 0x7fdbac100cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:14.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:21:14.277+0000 7fdbb3ddf700 1 -- 172.21.15.148:0/1975592894 >> 172.21.15.148:0/1975592894 conn(0x7fdbac0fc730 msgr2=0x7fdbac1035e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:14.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:21:14.278+0000 7fdbb3ddf700 1 -- 172.21.15.148:0/1975592894 shutdown_connections 2024-04-24T23:21:14.282 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:21:14.278+0000 7fdbb3ddf700 1 -- 172.21.15.148:0/1975592894 wait complete. 2024-04-24T23:21:14.876 DEBUG:teuthology.orchestra.run.smithi134:mgr.y> sudo journalctl -f -n 0 -u ceph-ec7885aa-0290-11ef-bc93-c7b262605968@mgr.y.service 2024-04-24T23:21:14.879 DEBUG:teuthology.orchestra.run.smithi148:mgr.z> sudo journalctl -f -n 0 -u ceph-ec7885aa-0290-11ef-bc93-c7b262605968@mgr.z.service 2024-04-24T23:21:14.881 INFO:tasks.cephadm:Deploying OSDs... 2024-04-24T23:21:14.881 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-24T23:21:14.881 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/scratch_devs of=/dev/stdout 2024-04-24T23:21:14.900 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-24T23:21:14.900 DEBUG:teuthology.orchestra.run.smithi080:> stat /dev/vg_nvme/lv_1 2024-04-24T23:21:14.958 INFO:teuthology.orchestra.run.smithi080.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-24T23:21:14.958 INFO:teuthology.orchestra.run.smithi080.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T23:21:14.958 INFO:teuthology.orchestra.run.smithi080.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-04-24T23:21:14.958 INFO:teuthology.orchestra.run.smithi080.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T23:21:14.958 INFO:teuthology.orchestra.run.smithi080.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T23:21:14.958 INFO:teuthology.orchestra.run.smithi080.stdout:Access: 2024-04-24 23:20:18.536110106 +0000 2024-04-24T23:21:14.958 INFO:teuthology.orchestra.run.smithi080.stdout:Modify: 2024-04-24 23:15:40.752587396 +0000 2024-04-24T23:21:14.958 INFO:teuthology.orchestra.run.smithi080.stdout:Change: 2024-04-24 23:15:40.752587396 +0000 2024-04-24T23:21:14.958 INFO:teuthology.orchestra.run.smithi080.stdout: Birth: 2024-04-24 23:15:40.752587396 +0000 2024-04-24T23:21:14.959 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-24T23:21:15.024 INFO:teuthology.orchestra.run.smithi080.stderr:1+0 records in 2024-04-24T23:21:15.024 INFO:teuthology.orchestra.run.smithi080.stderr:1+0 records out 2024-04-24T23:21:15.024 INFO:teuthology.orchestra.run.smithi080.stderr:512 bytes copied, 0.000232093 s, 2.2 MB/s 2024-04-24T23:21:15.025 DEBUG:teuthology.orchestra.run.smithi080:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-24T23:21:15.083 DEBUG:teuthology.orchestra.run.smithi080:> stat /dev/vg_nvme/lv_2 2024-04-24T23:21:15.140 INFO:teuthology.orchestra.run.smithi080.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-24T23:21:15.140 INFO:teuthology.orchestra.run.smithi080.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T23:21:15.140 INFO:teuthology.orchestra.run.smithi080.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-04-24T23:21:15.140 INFO:teuthology.orchestra.run.smithi080.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T23:21:15.141 INFO:teuthology.orchestra.run.smithi080.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T23:21:15.141 INFO:teuthology.orchestra.run.smithi080.stdout:Access: 2024-04-24 23:20:18.562109537 +0000 2024-04-24T23:21:15.141 INFO:teuthology.orchestra.run.smithi080.stdout:Modify: 2024-04-24 23:15:41.106579591 +0000 2024-04-24T23:21:15.141 INFO:teuthology.orchestra.run.smithi080.stdout:Change: 2024-04-24 23:15:41.106579591 +0000 2024-04-24T23:21:15.141 INFO:teuthology.orchestra.run.smithi080.stdout: Birth: 2024-04-24 23:15:41.106579591 +0000 2024-04-24T23:21:15.141 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-24T23:21:15.205 INFO:teuthology.orchestra.run.smithi080.stderr:1+0 records in 2024-04-24T23:21:15.205 INFO:teuthology.orchestra.run.smithi080.stderr:1+0 records out 2024-04-24T23:21:15.206 INFO:teuthology.orchestra.run.smithi080.stderr:512 bytes copied, 0.000207087 s, 2.5 MB/s 2024-04-24T23:21:15.207 DEBUG:teuthology.orchestra.run.smithi080:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-24T23:21:15.264 DEBUG:teuthology.orchestra.run.smithi080:> stat /dev/vg_nvme/lv_3 2024-04-24T23:21:15.321 INFO:teuthology.orchestra.run.smithi080.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-24T23:21:15.321 INFO:teuthology.orchestra.run.smithi080.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T23:21:15.321 INFO:teuthology.orchestra.run.smithi080.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-04-24T23:21:15.321 INFO:teuthology.orchestra.run.smithi080.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T23:21:15.321 INFO:teuthology.orchestra.run.smithi080.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T23:21:15.322 INFO:teuthology.orchestra.run.smithi080.stdout:Access: 2024-04-24 23:20:18.586109011 +0000 2024-04-24T23:21:15.322 INFO:teuthology.orchestra.run.smithi080.stdout:Modify: 2024-04-24 23:15:41.432572404 +0000 2024-04-24T23:21:15.322 INFO:teuthology.orchestra.run.smithi080.stdout:Change: 2024-04-24 23:15:41.432572404 +0000 2024-04-24T23:21:15.322 INFO:teuthology.orchestra.run.smithi080.stdout: Birth: 2024-04-24 23:15:41.432572404 +0000 2024-04-24T23:21:15.322 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-24T23:21:15.385 INFO:teuthology.orchestra.run.smithi080.stderr:1+0 records in 2024-04-24T23:21:15.385 INFO:teuthology.orchestra.run.smithi080.stderr:1+0 records out 2024-04-24T23:21:15.385 INFO:teuthology.orchestra.run.smithi080.stderr:512 bytes copied, 0.000199687 s, 2.6 MB/s 2024-04-24T23:21:15.386 DEBUG:teuthology.orchestra.run.smithi080:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-24T23:21:15.443 DEBUG:teuthology.orchestra.run.smithi080:> stat /dev/vg_nvme/lv_4 2024-04-24T23:21:15.499 INFO:teuthology.orchestra.run.smithi080.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-24T23:21:15.499 INFO:teuthology.orchestra.run.smithi080.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T23:21:15.499 INFO:teuthology.orchestra.run.smithi080.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-04-24T23:21:15.499 INFO:teuthology.orchestra.run.smithi080.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T23:21:15.499 INFO:teuthology.orchestra.run.smithi080.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T23:21:15.499 INFO:teuthology.orchestra.run.smithi080.stdout:Access: 2024-04-24 23:20:18.611108464 +0000 2024-04-24T23:21:15.499 INFO:teuthology.orchestra.run.smithi080.stdout:Modify: 2024-04-24 23:15:41.756565261 +0000 2024-04-24T23:21:15.499 INFO:teuthology.orchestra.run.smithi080.stdout:Change: 2024-04-24 23:15:41.756565261 +0000 2024-04-24T23:21:15.500 INFO:teuthology.orchestra.run.smithi080.stdout: Birth: 2024-04-24 23:15:41.756565261 +0000 2024-04-24T23:21:15.500 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-24T23:21:15.509 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:15 smithi148 ceph-mon[32528]: from='client.24112 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi080=x;smithi134=y;smithi148=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:21:15.509 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:15 smithi148 ceph-mon[32528]: Saving service mgr spec with placement smithi080=x;smithi134=y;smithi148=z;count:3 2024-04-24T23:21:15.509 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:15 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:15.509 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:15 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:21:15.509 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:15 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:21:15.510 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:15 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:21:15.510 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:15 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:15.510 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:15 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T23:21:15.510 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:15 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-24T23:21:15.510 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:15 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T23:21:15.510 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:15 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:21:15.510 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:15 smithi148 ceph-mon[32528]: Deploying daemon mgr.z on smithi148 2024-04-24T23:21:15.510 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:15 smithi148 ceph-mon[32528]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:21:15.539 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:15 smithi080 ceph-mon[29152]: from='client.24112 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi080=x;smithi134=y;smithi148=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:21:15.539 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:15 smithi080 ceph-mon[29152]: Saving service mgr spec with placement smithi080=x;smithi134=y;smithi148=z;count:3 2024-04-24T23:21:15.539 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:15 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:15.539 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:15 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:21:15.540 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:15 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:21:15.540 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:15 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:21:15.540 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:15 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:15.540 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:15 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T23:21:15.540 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:15 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-24T23:21:15.540 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:15 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T23:21:15.540 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:15 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:21:15.540 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:15 smithi080 ceph-mon[29152]: Deploying daemon mgr.z on smithi148 2024-04-24T23:21:15.540 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:15 smithi080 ceph-mon[29152]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:21:15.562 INFO:teuthology.orchestra.run.smithi080.stderr:1+0 records in 2024-04-24T23:21:15.562 INFO:teuthology.orchestra.run.smithi080.stderr:1+0 records out 2024-04-24T23:21:15.562 INFO:teuthology.orchestra.run.smithi080.stderr:512 bytes copied, 0.000237495 s, 2.2 MB/s 2024-04-24T23:21:15.564 DEBUG:teuthology.orchestra.run.smithi080:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-24T23:21:15.619 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-24T23:21:15.620 DEBUG:teuthology.orchestra.run.smithi134:> dd if=/scratch_devs of=/dev/stdout 2024-04-24T23:21:15.639 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-24T23:21:15.639 DEBUG:teuthology.orchestra.run.smithi134:> stat /dev/vg_nvme/lv_1 2024-04-24T23:21:15.696 INFO:teuthology.orchestra.run.smithi134.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-24T23:21:15.697 INFO:teuthology.orchestra.run.smithi134.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T23:21:15.697 INFO:teuthology.orchestra.run.smithi134.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-04-24T23:21:15.697 INFO:teuthology.orchestra.run.smithi134.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T23:21:15.697 INFO:teuthology.orchestra.run.smithi134.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T23:21:15.697 INFO:teuthology.orchestra.run.smithi134.stdout:Access: 2024-04-24 23:21:00.133643393 +0000 2024-04-24T23:21:15.697 INFO:teuthology.orchestra.run.smithi134.stdout:Modify: 2024-04-24 23:16:12.138672087 +0000 2024-04-24T23:21:15.697 INFO:teuthology.orchestra.run.smithi134.stdout:Change: 2024-04-24 23:16:12.138672087 +0000 2024-04-24T23:21:15.697 INFO:teuthology.orchestra.run.smithi134.stdout: Birth: 2024-04-24 23:16:12.138672087 +0000 2024-04-24T23:21:15.697 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-24T23:21:15.698 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:15 smithi134 ceph-mon[32619]: from='client.24112 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi080=x;smithi134=y;smithi148=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:21:15.698 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:15 smithi134 ceph-mon[32619]: Saving service mgr spec with placement smithi080=x;smithi134=y;smithi148=z;count:3 2024-04-24T23:21:15.698 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:15 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:15.698 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:15 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:21:15.698 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:15 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:21:15.698 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:15 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:21:15.699 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:15 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:15.699 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:15 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T23:21:15.699 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:15 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-24T23:21:15.699 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:15 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T23:21:15.699 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:15 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:21:15.699 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:15 smithi134 ceph-mon[32619]: Deploying daemon mgr.z on smithi148 2024-04-24T23:21:15.699 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:15 smithi134 ceph-mon[32619]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:21:15.724 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records in 2024-04-24T23:21:15.724 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records out 2024-04-24T23:21:15.724 INFO:teuthology.orchestra.run.smithi134.stderr:512 bytes copied, 0.00024785 s, 2.1 MB/s 2024-04-24T23:21:15.725 DEBUG:teuthology.orchestra.run.smithi134:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-24T23:21:15.783 DEBUG:teuthology.orchestra.run.smithi134:> stat /dev/vg_nvme/lv_2 2024-04-24T23:21:15.839 INFO:teuthology.orchestra.run.smithi134.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-24T23:21:15.840 INFO:teuthology.orchestra.run.smithi134.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T23:21:15.840 INFO:teuthology.orchestra.run.smithi134.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-04-24T23:21:15.840 INFO:teuthology.orchestra.run.smithi134.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T23:21:15.840 INFO:teuthology.orchestra.run.smithi134.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T23:21:15.840 INFO:teuthology.orchestra.run.smithi134.stdout:Access: 2024-04-24 23:21:00.133643393 +0000 2024-04-24T23:21:15.840 INFO:teuthology.orchestra.run.smithi134.stdout:Modify: 2024-04-24 23:16:12.471664336 +0000 2024-04-24T23:21:15.840 INFO:teuthology.orchestra.run.smithi134.stdout:Change: 2024-04-24 23:16:12.471664336 +0000 2024-04-24T23:21:15.840 INFO:teuthology.orchestra.run.smithi134.stdout: Birth: 2024-04-24 23:16:12.471664336 +0000 2024-04-24T23:21:15.840 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-24T23:21:15.903 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records in 2024-04-24T23:21:15.903 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records out 2024-04-24T23:21:15.903 INFO:teuthology.orchestra.run.smithi134.stderr:512 bytes copied, 0.000210322 s, 2.4 MB/s 2024-04-24T23:21:15.905 DEBUG:teuthology.orchestra.run.smithi134:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-24T23:21:15.961 DEBUG:teuthology.orchestra.run.smithi134:> stat /dev/vg_nvme/lv_3 2024-04-24T23:21:16.017 INFO:teuthology.orchestra.run.smithi134.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-24T23:21:16.017 INFO:teuthology.orchestra.run.smithi134.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T23:21:16.017 INFO:teuthology.orchestra.run.smithi134.stdout:Device: 5h/5d Inode: 701 Links: 1 2024-04-24T23:21:16.017 INFO:teuthology.orchestra.run.smithi134.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T23:21:16.017 INFO:teuthology.orchestra.run.smithi134.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T23:21:16.017 INFO:teuthology.orchestra.run.smithi134.stdout:Access: 2024-04-24 23:21:00.134643371 +0000 2024-04-24T23:21:16.017 INFO:teuthology.orchestra.run.smithi134.stdout:Modify: 2024-04-24 23:16:12.811656422 +0000 2024-04-24T23:21:16.018 INFO:teuthology.orchestra.run.smithi134.stdout:Change: 2024-04-24 23:16:12.811656422 +0000 2024-04-24T23:21:16.018 INFO:teuthology.orchestra.run.smithi134.stdout: Birth: 2024-04-24 23:16:12.811656422 +0000 2024-04-24T23:21:16.018 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-24T23:21:16.081 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records in 2024-04-24T23:21:16.081 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records out 2024-04-24T23:21:16.081 INFO:teuthology.orchestra.run.smithi134.stderr:512 bytes copied, 0.000238472 s, 2.1 MB/s 2024-04-24T23:21:16.083 DEBUG:teuthology.orchestra.run.smithi134:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-24T23:21:16.139 DEBUG:teuthology.orchestra.run.smithi134:> stat /dev/vg_nvme/lv_4 2024-04-24T23:21:16.195 INFO:teuthology.orchestra.run.smithi134.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-24T23:21:16.195 INFO:teuthology.orchestra.run.smithi134.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T23:21:16.195 INFO:teuthology.orchestra.run.smithi134.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-04-24T23:21:16.195 INFO:teuthology.orchestra.run.smithi134.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T23:21:16.195 INFO:teuthology.orchestra.run.smithi134.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T23:21:16.195 INFO:teuthology.orchestra.run.smithi134.stdout:Access: 2024-04-24 23:21:00.134643371 +0000 2024-04-24T23:21:16.195 INFO:teuthology.orchestra.run.smithi134.stdout:Modify: 2024-04-24 23:16:13.148648578 +0000 2024-04-24T23:21:16.195 INFO:teuthology.orchestra.run.smithi134.stdout:Change: 2024-04-24 23:16:13.148648578 +0000 2024-04-24T23:21:16.196 INFO:teuthology.orchestra.run.smithi134.stdout: Birth: 2024-04-24 23:16:13.148648578 +0000 2024-04-24T23:21:16.196 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-24T23:21:16.259 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records in 2024-04-24T23:21:16.259 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records out 2024-04-24T23:21:16.259 INFO:teuthology.orchestra.run.smithi134.stderr:512 bytes copied, 0.000240682 s, 2.1 MB/s 2024-04-24T23:21:16.260 DEBUG:teuthology.orchestra.run.smithi134:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-24T23:21:16.316 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-24T23:21:16.316 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/scratch_devs of=/dev/stdout 2024-04-24T23:21:16.335 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-24T23:21:16.335 DEBUG:teuthology.orchestra.run.smithi148:> stat /dev/vg_nvme/lv_1 2024-04-24T23:21:16.391 INFO:teuthology.orchestra.run.smithi148.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-24T23:21:16.392 INFO:teuthology.orchestra.run.smithi148.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T23:21:16.392 INFO:teuthology.orchestra.run.smithi148.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-04-24T23:21:16.392 INFO:teuthology.orchestra.run.smithi148.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T23:21:16.392 INFO:teuthology.orchestra.run.smithi148.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T23:21:16.392 INFO:teuthology.orchestra.run.smithi148.stdout:Access: 2024-04-24 23:21:16.281936854 +0000 2024-04-24T23:21:16.392 INFO:teuthology.orchestra.run.smithi148.stdout:Modify: 2024-04-24 23:16:04.600260032 +0000 2024-04-24T23:21:16.392 INFO:teuthology.orchestra.run.smithi148.stdout:Change: 2024-04-24 23:16:04.600260032 +0000 2024-04-24T23:21:16.392 INFO:teuthology.orchestra.run.smithi148.stdout: Birth: 2024-04-24 23:16:04.600260032 +0000 2024-04-24T23:21:16.392 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-24T23:21:16.454 INFO:journalctl@ceph.mgr.z.smithi148.stdout:Apr 24 23:21:16 smithi148 systemd[1]: Starting Ceph mgr.z for ec7885aa-0290-11ef-bc93-c7b262605968... 2024-04-24T23:21:16.457 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-24T23:21:16.457 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-24T23:21:16.457 INFO:teuthology.orchestra.run.smithi148.stderr:512 bytes copied, 0.000277096 s, 1.8 MB/s 2024-04-24T23:21:16.459 DEBUG:teuthology.orchestra.run.smithi148:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-24T23:21:16.514 DEBUG:teuthology.orchestra.run.smithi148:> stat /dev/vg_nvme/lv_2 2024-04-24T23:21:16.568 INFO:teuthology.orchestra.run.smithi148.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-24T23:21:16.569 INFO:teuthology.orchestra.run.smithi148.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T23:21:16.569 INFO:teuthology.orchestra.run.smithi148.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-04-24T23:21:16.569 INFO:teuthology.orchestra.run.smithi148.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T23:21:16.569 INFO:teuthology.orchestra.run.smithi148.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T23:21:16.569 INFO:teuthology.orchestra.run.smithi148.stdout:Access: 2024-04-24 23:21:16.281936854 +0000 2024-04-24T23:21:16.569 INFO:teuthology.orchestra.run.smithi148.stdout:Modify: 2024-04-24 23:16:04.944251911 +0000 2024-04-24T23:21:16.569 INFO:teuthology.orchestra.run.smithi148.stdout:Change: 2024-04-24 23:16:04.944251911 +0000 2024-04-24T23:21:16.569 INFO:teuthology.orchestra.run.smithi148.stdout: Birth: 2024-04-24 23:16:04.944251911 +0000 2024-04-24T23:21:16.569 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-24T23:21:16.631 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-24T23:21:16.631 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-24T23:21:16.631 INFO:teuthology.orchestra.run.smithi148.stderr:512 bytes copied, 0.000205406 s, 2.5 MB/s 2024-04-24T23:21:16.632 DEBUG:teuthology.orchestra.run.smithi148:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-24T23:21:16.688 DEBUG:teuthology.orchestra.run.smithi148:> stat /dev/vg_nvme/lv_3 2024-04-24T23:21:16.737 INFO:journalctl@ceph.mgr.z.smithi148.stdout:Apr 24 23:21:16 smithi148 podman[34229]: 2024-04-24 23:21:16.465821625 +0000 UTC m=+0.019023196 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:21:16.737 INFO:journalctl@ceph.mgr.z.smithi148.stdout:Apr 24 23:21:16 smithi148 podman[34229]: 2024-04-24 23:21:16.607583597 +0000 UTC m=+0.160785151 container create fd286875e6f7f93993588cf75e70c73f17f7ef907104f4344627932dd722baa4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-z, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T23:21:16.743 INFO:teuthology.orchestra.run.smithi148.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-24T23:21:16.744 INFO:teuthology.orchestra.run.smithi148.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T23:21:16.744 INFO:teuthology.orchestra.run.smithi148.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-04-24T23:21:16.744 INFO:teuthology.orchestra.run.smithi148.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T23:21:16.744 INFO:teuthology.orchestra.run.smithi148.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T23:21:16.744 INFO:teuthology.orchestra.run.smithi148.stdout:Access: 2024-04-24 23:21:16.282936831 +0000 2024-04-24T23:21:16.744 INFO:teuthology.orchestra.run.smithi148.stdout:Modify: 2024-04-24 23:16:05.278244026 +0000 2024-04-24T23:21:16.744 INFO:teuthology.orchestra.run.smithi148.stdout:Change: 2024-04-24 23:16:05.278244026 +0000 2024-04-24T23:21:16.744 INFO:teuthology.orchestra.run.smithi148.stdout: Birth: 2024-04-24 23:16:05.278244026 +0000 2024-04-24T23:21:16.744 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-24T23:21:16.817 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-24T23:21:16.817 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-24T23:21:16.817 INFO:teuthology.orchestra.run.smithi148.stderr:512 bytes copied, 0.000205395 s, 2.5 MB/s 2024-04-24T23:21:16.818 DEBUG:teuthology.orchestra.run.smithi148:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-24T23:21:16.874 DEBUG:teuthology.orchestra.run.smithi148:> stat /dev/vg_nvme/lv_4 2024-04-24T23:21:16.930 INFO:teuthology.orchestra.run.smithi148.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-24T23:21:16.930 INFO:teuthology.orchestra.run.smithi148.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T23:21:16.930 INFO:teuthology.orchestra.run.smithi148.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-04-24T23:21:16.930 INFO:teuthology.orchestra.run.smithi148.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T23:21:16.930 INFO:teuthology.orchestra.run.smithi148.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T23:21:16.930 INFO:teuthology.orchestra.run.smithi148.stdout:Access: 2024-04-24 23:21:16.282936831 +0000 2024-04-24T23:21:16.930 INFO:teuthology.orchestra.run.smithi148.stdout:Modify: 2024-04-24 23:16:05.599236448 +0000 2024-04-24T23:21:16.931 INFO:teuthology.orchestra.run.smithi148.stdout:Change: 2024-04-24 23:16:05.599236448 +0000 2024-04-24T23:21:16.931 INFO:teuthology.orchestra.run.smithi148.stdout: Birth: 2024-04-24 23:16:05.599236448 +0000 2024-04-24T23:21:16.931 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-24T23:21:16.992 INFO:journalctl@ceph.mgr.z.smithi148.stdout:Apr 24 23:21:16 smithi148 podman[34229]: 2024-04-24 23:21:16.835189835 +0000 UTC m=+0.388391385 container init fd286875e6f7f93993588cf75e70c73f17f7ef907104f4344627932dd722baa4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-z, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T23:21:16.993 INFO:journalctl@ceph.mgr.z.smithi148.stdout:Apr 24 23:21:16 smithi148 podman[34229]: 2024-04-24 23:21:16.838007942 +0000 UTC m=+0.391209493 container start fd286875e6f7f93993588cf75e70c73f17f7ef907104f4344627932dd722baa4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-z, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-24T23:21:16.993 INFO:journalctl@ceph.mgr.z.smithi148.stdout:Apr 24 23:21:16 smithi148 ceph-mgr[34359]: -- 172.21.15.148:0/3435527769 <== mon.1 v2:172.21.15.148:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c4cb1ca5a0 con 0x55c4cb19ec00 2024-04-24T23:21:16.993 INFO:journalctl@ceph.mgr.z.smithi148.stdout:Apr 24 23:21:16 smithi148 bash[34229]: fd286875e6f7f93993588cf75e70c73f17f7ef907104f4344627932dd722baa4 2024-04-24T23:21:16.993 INFO:journalctl@ceph.mgr.z.smithi148.stdout:Apr 24 23:21:16 smithi148 systemd[1]: Started Ceph mgr.z for ec7885aa-0290-11ef-bc93-c7b262605968. 2024-04-24T23:21:16.995 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-24T23:21:16.995 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-24T23:21:16.995 INFO:teuthology.orchestra.run.smithi148.stderr:512 bytes copied, 0.000200198 s, 2.6 MB/s 2024-04-24T23:21:16.997 DEBUG:teuthology.orchestra.run.smithi148:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-24T23:21:17.052 INFO:tasks.cephadm:Deploying osd.0 on smithi080 with /dev/vg_nvme/lv_4... 2024-04-24T23:21:17.053 DEBUG:teuthology.orchestra.run.smithi080:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ec7885aa-0290-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-24T23:21:17.257 INFO:journalctl@ceph.mgr.z.smithi148.stdout:Apr 24 23:21:17 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-z[34322]: 2024-04-24T23:21:17.008+0000 7f79d8dc6200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-24T23:21:17.257 INFO:journalctl@ceph.mgr.z.smithi148.stdout:Apr 24 23:21:17 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-z[34322]: 2024-04-24T23:21:17.198+0000 7f79d8dc6200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-24T23:21:17.297 INFO:teuthology.orchestra.run.smithi080.stderr:Inferring config /var/lib/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:21:17.600 INFO:journalctl@ceph.mgr.z.smithi148.stdout:Apr 24 23:21:17 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-z[34322]: 2024-04-24T23:21:17.299+0000 7f79d8dc6200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-24T23:21:17.986 INFO:journalctl@ceph.mgr.z.smithi148.stdout:Apr 24 23:21:17 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-z[34322]: 2024-04-24T23:21:17.611+0000 7f79d8dc6200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-24T23:21:17.986 INFO:journalctl@ceph.mgr.z.smithi148.stdout:Apr 24 23:21:17 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-z[34322]: 2024-04-24T23:21:17.696+0000 7f79d8dc6200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-24T23:21:18.217 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:17 smithi134 ceph-mon[32619]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:21:18.217 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:17 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:18.217 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:17 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:18.217 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:17 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:18.217 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:17 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T23:21:18.217 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:17 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-24T23:21:18.218 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:17 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T23:21:18.218 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:17 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:21:18.218 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:17 smithi134 ceph-mon[32619]: Deploying daemon mgr.y on smithi134 2024-04-24T23:21:18.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:17 smithi148 ceph-mon[32528]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:21:18.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:17 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:18.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:17 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:18.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:17 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:18.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:17 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T23:21:18.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:17 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-24T23:21:18.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:17 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T23:21:18.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:17 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:21:18.258 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:17 smithi148 ceph-mon[32528]: Deploying daemon mgr.y on smithi134 2024-04-24T23:21:18.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:17 smithi080 ceph-mon[29152]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:21:18.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:17 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:18.290 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:17 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:18.290 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:17 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:18.290 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:17 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T23:21:18.290 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:17 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-24T23:21:18.290 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:17 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T23:21:18.290 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:17 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:21:18.290 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:17 smithi080 ceph-mon[29152]: Deploying daemon mgr.y on smithi134 2024-04-24T23:21:19.201 INFO:teuthology.orchestra.run.smithi080.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-24T23:21:19.201 INFO:teuthology.orchestra.run.smithi080.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-24T23:21:19.201 INFO:teuthology.orchestra.run.smithi080.stderr: stderr: 10+0 records in 2024-04-24T23:21:19.201 INFO:teuthology.orchestra.run.smithi080.stderr:10+0 records out 2024-04-24T23:21:19.202 INFO:teuthology.orchestra.run.smithi080.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0113644 s, 923 MB/s 2024-04-24T23:21:19.202 INFO:teuthology.orchestra.run.smithi080.stderr:--> Zapping successful for: 2024-04-24T23:21:19.257 INFO:journalctl@ceph.mgr.z.smithi148.stdout:Apr 24 23:21:18 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-z[34322]: 2024-04-24T23:21:18.905+0000 7f79d8dc6200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-24T23:21:19.257 INFO:journalctl@ceph.mgr.z.smithi148.stdout:Apr 24 23:21:19 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-z[34322]: 2024-04-24T23:21:19.085+0000 7f79d8dc6200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-24T23:21:19.416 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Apr 24 23:21:19 smithi134 systemd[1]: Starting Ceph mgr.y for ec7885aa-0290-11ef-bc93-c7b262605968... 2024-04-24T23:21:19.559 INFO:journalctl@ceph.mgr.z.smithi148.stdout:Apr 24 23:21:19 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-z[34322]: 2024-04-24T23:21:19.262+0000 7f79d8dc6200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-24T23:21:19.698 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Apr 24 23:21:19 smithi134 podman[34027]: 2024-04-24 23:21:19.414522018 +0000 UTC m=+0.114211385 container create bb25bcb9fea2a95e0c824e75824ffd3729f5175c8e9d2485d8763fc06384cc7b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-y, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-04-24T23:21:19.698 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Apr 24 23:21:19 smithi134 podman[34027]: 2024-04-24 23:21:19.318158245 +0000 UTC m=+0.017847613 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:21:19.699 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Apr 24 23:21:19 smithi134 podman[34027]: 2024-04-24 23:21:19.616681897 +0000 UTC m=+0.316371263 container init bb25bcb9fea2a95e0c824e75824ffd3729f5175c8e9d2485d8763fc06384cc7b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-y, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T23:21:19.699 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Apr 24 23:21:19 smithi134 podman[34027]: 2024-04-24 23:21:19.619211262 +0000 UTC m=+0.318900639 container start bb25bcb9fea2a95e0c824e75824ffd3729f5175c8e9d2485d8763fc06384cc7b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-y, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-24T23:21:19.699 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Apr 24 23:21:19 smithi134 ceph-mgr[34046]: -- 172.21.15.134:0/3519281607 <== mon.2 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5601f55645a0 con 0x5601f5539000 2024-04-24T23:21:19.768 DEBUG:teuthology.orchestra.run.smithi080:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ec7885aa-0290-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi080:vg_nvme/lv_4 2024-04-24T23:21:19.846 INFO:journalctl@ceph.mgr.z.smithi148.stdout:Apr 24 23:21:19 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-z[34322]: 2024-04-24T23:21:19.557+0000 7f79d8dc6200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-24T23:21:19.846 INFO:journalctl@ceph.mgr.z.smithi148.stdout:Apr 24 23:21:19 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-z[34322]: 2024-04-24T23:21:19.652+0000 7f79d8dc6200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-24T23:21:19.964 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Apr 24 23:21:19 smithi134 bash[34027]: bb25bcb9fea2a95e0c824e75824ffd3729f5175c8e9d2485d8763fc06384cc7b 2024-04-24T23:21:19.964 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Apr 24 23:21:19 smithi134 systemd[1]: Started Ceph mgr.y for ec7885aa-0290-11ef-bc93-c7b262605968. 2024-04-24T23:21:19.964 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Apr 24 23:21:19 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-y[34042]: 2024-04-24T23:21:19.772+0000 7f976aabf200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-24T23:21:19.964 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Apr 24 23:21:19 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-y[34042]: 2024-04-24T23:21:19.962+0000 7f976aabf200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-24T23:21:20.009 INFO:teuthology.orchestra.run.smithi080.stderr:Inferring config /var/lib/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:21:20.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:19 smithi080 ceph-mon[29152]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:21:20.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:19 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:20.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:19 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:20.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:19 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:20.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:19 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:20.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:19 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:21:20.257 INFO:journalctl@ceph.mgr.z.smithi148.stdout:Apr 24 23:21:19 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-z[34322]: 2024-04-24T23:21:19.866+0000 7f79d8dc6200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-24T23:21:20.257 INFO:journalctl@ceph.mgr.z.smithi148.stdout:Apr 24 23:21:20 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-z[34322]: 2024-04-24T23:21:20.070+0000 7f79d8dc6200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-24T23:21:20.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:19 smithi148 ceph-mon[32528]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:21:20.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:19 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:20.258 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:19 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:20.258 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:19 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:20.258 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:19 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:20.258 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:19 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:21:20.397 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Apr 24 23:21:20 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-y[34042]: 2024-04-24T23:21:20.071+0000 7f976aabf200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-24T23:21:20.397 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Apr 24 23:21:20 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-y[34042]: 2024-04-24T23:21:20.394+0000 7f976aabf200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-24T23:21:20.397 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:19 smithi134 ceph-mon[32619]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:21:20.397 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:19 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:20.398 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:19 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:20.398 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:19 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:20.398 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:19 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:20.398 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:19 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:21:20.698 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Apr 24 23:21:20 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-y[34042]: 2024-04-24T23:21:20.488+0000 7f976aabf200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-24T23:21:21.007 INFO:journalctl@ceph.mgr.z.smithi148.stdout:Apr 24 23:21:20 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-z[34322]: 2024-04-24T23:21:20.698+0000 7f79d8dc6200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-24T23:21:21.449 INFO:journalctl@ceph.mgr.z.smithi148.stdout:Apr 24 23:21:21 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-z[34322]: 2024-04-24T23:21:21.138+0000 7f79d8dc6200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-24T23:21:21.450 INFO:journalctl@ceph.mgr.z.smithi148.stdout:Apr 24 23:21:21 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-z[34322]: 2024-04-24T23:21:21.244+0000 7f79d8dc6200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-24T23:21:21.450 INFO:journalctl@ceph.mgr.z.smithi148.stdout:Apr 24 23:21:21 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-z[34322]: 2024-04-24T23:21:21.448+0000 7f79d8dc6200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-24T23:21:21.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:21:21.570+0000 7f9dd1d17700 1 -- 172.21.15.80:0/3692321984 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9dcc1056d0 msgr2=0x7f9dcc107ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:21.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:21:21.570+0000 7f9dd1d17700 1 --2- 172.21.15.80:0/3692321984 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9dcc1056d0 0x7f9dcc107ac0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc009990 tx=0x7f9dbc009ca0 comp rx=0 tx=0).stop 2024-04-24T23:21:21.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:21:21.570+0000 7f9dd1d17700 1 -- 172.21.15.80:0/3692321984 shutdown_connections 2024-04-24T23:21:21.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:21:21.570+0000 7f9dd1d17700 1 --2- 172.21.15.80:0/3692321984 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9dcc108000 0x7f9dcc10a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:21.573 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:21:21.570+0000 7f9dd1d17700 1 --2- 172.21.15.80:0/3692321984 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9dcc102da0 0x7f9dcc105190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:21.573 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:21:21.570+0000 7f9dd1d17700 1 --2- 172.21.15.80:0/3692321984 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9dcc1056d0 0x7f9dcc107ac0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:21.573 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:21:21.570+0000 7f9dd1d17700 1 -- 172.21.15.80:0/3692321984 >> 172.21.15.80:0/3692321984 conn(0x7f9dcc0fc750 msgr2=0x7f9dcc0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:21.573 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:21:21.570+0000 7f9dd1d17700 1 -- 172.21.15.80:0/3692321984 shutdown_connections 2024-04-24T23:21:21.573 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:21:21.570+0000 7f9dd1d17700 1 -- 172.21.15.80:0/3692321984 wait complete. 2024-04-24T23:21:21.575 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:21:21.571+0000 7f9dd1d17700 1 Processor -- start 2024-04-24T23:21:21.575 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:21:21.571+0000 7f9dd1d17700 1 -- start start 2024-04-24T23:21:21.575 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:21:21.571+0000 7f9dd1d17700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9dcc102da0 0x7f9dcc198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:21.575 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:21:21.571+0000 7f9dd1d17700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9dcc1056d0 0x7f9dcc1989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:21.575 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:21:21.571+0000 7f9dd1d17700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9dcc108000 0x7f9dcc19fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:21.575 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:21:21.571+0000 7f9dd1d17700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f9dcc10def0 con 0x7f9dcc1056d0 2024-04-24T23:21:21.575 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:21:21.571+0000 7f9dd1d17700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9dcc10dd70 con 0x7f9dcc108000 2024-04-24T23:21:21.575 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:21:21.571+0000 7f9dd1d17700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f9dcc10e070 con 0x7f9dcc102da0 2024-04-24T23:21:21.576 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:21:21.571+0000 7f9dcaffd700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9dcc1056d0 0x7f9dcc1989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:21.576 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:21:21.571+0000 7f9dcaffd700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9dcc1056d0 0x7f9dcc1989d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:44590/0 (socket says 172.21.15.80:44590) 2024-04-24T23:21:21.576 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:21:21.571+0000 7f9dcaffd700 1 -- 172.21.15.80:0/2365041686 learned_addr learned my addr 172.21.15.80:0/2365041686 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:21:21.576 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:21:21.571+0000 7f9dcb7fe700 1 --2- 172.21.15.80:0/2365041686 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9dcc102da0 0x7f9dcc198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:21.576 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:21:21.571+0000 7f9dcbfff700 1 --2- 172.21.15.80:0/2365041686 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9dcc108000 0x7f9dcc19fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:21.576 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:21:21.572+0000 7f9dcaffd700 1 -- 172.21.15.80:0/2365041686 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9dcc102da0 msgr2=0x7f9dcc198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:21.576 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:21:21.572+0000 7f9dcaffd700 1 --2- 172.21.15.80:0/2365041686 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9dcc102da0 0x7f9dcc198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:21.576 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:21:21.572+0000 7f9dcaffd700 1 -- 172.21.15.80:0/2365041686 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9dcc108000 msgr2=0x7f9dcc19fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:21.576 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:21:21.572+0000 7f9dcaffd700 1 --2- 172.21.15.80:0/2365041686 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9dcc108000 0x7f9dcc19fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:21.576 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:21:21.572+0000 7f9dcaffd700 1 -- 172.21.15.80:0/2365041686 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dcc1a00f0 con 0x7f9dcc1056d0 2024-04-24T23:21:21.576 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:21:21.572+0000 7f9dcbfff700 1 --2- 172.21.15.80:0/2365041686 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9dcc108000 0x7f9dcc19fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:21:21.576 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:21:21.572+0000 7f9dcaffd700 1 --2- 172.21.15.80:0/2365041686 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9dcc1056d0 0x7f9dcc1989d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc005910 tx=0x7f9dbc0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:21.576 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:21:21.572+0000 7f9dc8ff9700 1 -- 172.21.15.80:0/2365041686 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dbc027070 con 0x7f9dcc1056d0 2024-04-24T23:21:21.576 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:21:21.572+0000 7f9dc8ff9700 1 -- 172.21.15.80:0/2365041686 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9dbc004150 con 0x7f9dcc1056d0 2024-04-24T23:21:21.576 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:21:21.572+0000 7f9dc8ff9700 1 -- 172.21.15.80:0/2365041686 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dbc022e00 con 0x7f9dcc1056d0 2024-04-24T23:21:21.577 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:21:21.572+0000 7f9dd1d17700 1 -- 172.21.15.80:0/2365041686 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9dcc1a0380 con 0x7f9dcc1056d0 2024-04-24T23:21:21.577 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:21:21.572+0000 7f9dd1d17700 1 -- 172.21.15.80:0/2365041686 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9dcc1a07d0 con 0x7f9dcc1056d0 2024-04-24T23:21:21.577 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:21:21.573+0000 7f9dd1d17700 1 -- 172.21.15.80:0/2365041686 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9dcc068f20 con 0x7f9dcc1056d0 2024-04-24T23:21:21.577 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:21:21.573+0000 7f9dc8ff9700 1 -- 172.21.15.80:0/2365041686 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f9dbc02a070 con 0x7f9dcc1056d0 2024-04-24T23:21:21.577 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:21:21.574+0000 7f9dc8ff9700 1 --2- 172.21.15.80:0/2365041686 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f9db4039330 0x7f9db403b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:21.577 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:21:21.574+0000 7f9dc8ff9700 1 -- 172.21.15.80:0/2365041686 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f9dbc04d1a0 con 0x7f9dcc1056d0 2024-04-24T23:21:21.578 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:21:21.574+0000 7f9dcb7fe700 1 --2- 172.21.15.80:0/2365041686 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f9db4039330 0x7f9db403b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:21.578 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:21:21.574+0000 7f9dcb7fe700 1 --2- 172.21.15.80:0/2365041686 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f9db4039330 0x7f9db403b7f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f9dc0007c60 tx=0x7f9dc00073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:21.580 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:21:21.579+0000 7f9dc8ff9700 1 -- 172.21.15.80:0/2365041686 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f9dbc013070 con 0x7f9dcc1056d0 2024-04-24T23:21:21.806 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:21:21.804+0000 7f9dd1d17700 1 -- 172.21.15.80:0/2365041686 --> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi080:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f9dcc062670 con 0x7f9db4039330 2024-04-24T23:21:21.991 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Apr 24 23:21:21 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-y[34042]: 2024-04-24T23:21:21.703+0000 7f976aabf200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-24T23:21:21.991 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Apr 24 23:21:21 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-y[34042]: 2024-04-24T23:21:21.869+0000 7f976aabf200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-24T23:21:22.257 INFO:journalctl@ceph.mgr.z.smithi148.stdout:Apr 24 23:21:21 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-z[34322]: 2024-04-24T23:21:21.970+0000 7f79d8dc6200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-24T23:21:22.257 INFO:journalctl@ceph.mgr.z.smithi148.stdout:Apr 24 23:21:22 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-z[34322]: 2024-04-24T23:21:22.064+0000 7f79d8dc6200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-24T23:21:22.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:21 smithi148 ceph-mon[32528]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:21:22.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:21 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:22.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:21 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:22.258 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:21 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:22.258 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:21 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:22.258 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:21 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:21:22.258 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:21 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:21:22.258 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:21 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:22.258 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:21 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T23:21:22.258 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:21 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T23:21:22.258 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:21 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:21:22.258 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:21 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:21:22.258 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:21 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:21:22.258 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:21 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:21:22.258 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:21 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:22.284 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:21 smithi080 ceph-mon[29152]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:21:22.284 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:21 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:22.284 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:21 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:22.284 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:21 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:22.284 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:21 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:22.284 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:21 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:21:22.284 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:21 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:21:22.284 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:21 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:22.284 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:21 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T23:21:22.284 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:21 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T23:21:22.285 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:21 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:21:22.285 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:21 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:21:22.285 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:21 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:21:22.285 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:21 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:21:22.285 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:21 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:22.322 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Apr 24 23:21:22 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-y[34042]: 2024-04-24T23:21:22.040+0000 7f976aabf200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-24T23:21:22.322 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Apr 24 23:21:22 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-y[34042]: 2024-04-24T23:21:22.320+0000 7f976aabf200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-24T23:21:22.322 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:21 smithi134 ceph-mon[32619]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:21:22.322 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:21 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:22.322 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:21 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:22.322 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:21 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:22.322 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:21 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:22.322 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:21 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:21:22.322 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:21 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:21:22.322 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:21 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:22.322 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:21 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T23:21:22.323 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:21 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T23:21:22.323 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:21 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:21:22.323 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:21 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:21:22.323 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:21 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:21:22.323 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:21 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:21:22.323 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:21 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:22.612 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Apr 24 23:21:22 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-y[34042]: 2024-04-24T23:21:22.410+0000 7f976aabf200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-24T23:21:22.612 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Apr 24 23:21:22 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-y[34042]: 2024-04-24T23:21:22.610+0000 7f976aabf200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-24T23:21:22.647 INFO:journalctl@ceph.mgr.z.smithi148.stdout:Apr 24 23:21:22 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-z[34322]: 2024-04-24T23:21:22.304+0000 7f79d8dc6200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-24T23:21:22.647 INFO:journalctl@ceph.mgr.z.smithi148.stdout:Apr 24 23:21:22 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-z[34322]: 2024-04-24T23:21:22.435+0000 7f79d8dc6200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-24T23:21:22.647 INFO:journalctl@ceph.mgr.z.smithi148.stdout:Apr 24 23:21:22 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-z[34322]: 2024-04-24T23:21:22.530+0000 7f79d8dc6200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-24T23:21:22.647 INFO:journalctl@ceph.mgr.z.smithi148.stdout:Apr 24 23:21:22 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-z[34322]: 2024-04-24T23:21:22.645+0000 7f79d8dc6200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-24T23:21:22.948 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Apr 24 23:21:22 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-y[34042]: 2024-04-24T23:21:22.798+0000 7f976aabf200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-24T23:21:22.996 INFO:journalctl@ceph.mgr.z.smithi148.stdout:Apr 24 23:21:22 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-z[34322]: 2024-04-24T23:21:22.736+0000 7f79d8dc6200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-24T23:21:23.225 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:22 smithi080 ceph-mon[29152]: Reconfiguring mgr.x (unknown last config time)... 2024-04-24T23:21:23.225 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:22 smithi080 ceph-mon[29152]: Reconfiguring daemon mgr.x on smithi080 2024-04-24T23:21:23.225 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:22 smithi080 ceph-mon[29152]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi080:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:21:23.225 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:22 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:23.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:22 smithi148 ceph-mon[32528]: Reconfiguring mgr.x (unknown last config time)... 2024-04-24T23:21:23.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:22 smithi148 ceph-mon[32528]: Reconfiguring daemon mgr.x on smithi080 2024-04-24T23:21:23.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:22 smithi148 ceph-mon[32528]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi080:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:21:23.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:22 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:23.400 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:22 smithi134 ceph-mon[32619]: Reconfiguring mgr.x (unknown last config time)... 2024-04-24T23:21:23.400 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:22 smithi134 ceph-mon[32619]: Reconfiguring daemon mgr.x on smithi080 2024-04-24T23:21:23.400 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:22 smithi134 ceph-mon[32619]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi080:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:21:23.400 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:22 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:23.698 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Apr 24 23:21:23 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-y[34042]: 2024-04-24T23:21:23.399+0000 7f976aabf200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-24T23:21:23.998 INFO:journalctl@ceph.mgr.z.smithi148.stdout:Apr 24 23:21:23 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-z[34322]: 2024-04-24T23:21:23.559+0000 7f79d8dc6200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-24T23:21:23.998 INFO:journalctl@ceph.mgr.z.smithi148.stdout:Apr 24 23:21:23 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-z[34322]: 2024-04-24T23:21:23.772+0000 7f79d8dc6200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-24T23:21:24.043 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:21:24.041+0000 7f9dc8ff9700 1 -- 172.21.15.80:0/2365041686 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mgrmap(e 14) ==== 92060+0+0 (secure 0 0 0) 0x7f9dbc003e50 con 0x7f9dcc1056d0 2024-04-24T23:21:24.113 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Apr 24 23:21:23 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-y[34042]: 2024-04-24T23:21:23.816+0000 7f976aabf200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-24T23:21:24.113 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Apr 24 23:21:23 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-y[34042]: 2024-04-24T23:21:23.912+0000 7f976aabf200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-24T23:21:24.113 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Apr 24 23:21:24 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-y[34042]: 2024-04-24T23:21:24.112+0000 7f976aabf200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-24T23:21:24.113 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:23 smithi134 ceph-mon[32619]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:21:24.113 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:23 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:24.113 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:23 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:21:24.114 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:23 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:21:24.114 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:23 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:21:24.114 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:23 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:24.114 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:23 smithi134 ceph-mon[32619]: Standby manager daemon z started 2024-04-24T23:21:24.114 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:23 smithi134 ceph-mon[32619]: from='mgr.? 172.21.15.148:0/1108083170' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-24T23:21:24.114 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:23 smithi134 ceph-mon[32619]: from='mgr.? 172.21.15.148:0/1108083170' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T23:21:24.114 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:23 smithi134 ceph-mon[32619]: from='mgr.? 172.21.15.148:0/1108083170' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-24T23:21:24.114 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:23 smithi134 ceph-mon[32619]: from='mgr.? 172.21.15.148:0/1108083170' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T23:21:24.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:23 smithi148 ceph-mon[32528]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:21:24.258 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:23 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:24.258 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:23 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:21:24.258 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:23 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:21:24.258 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:23 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:21:24.258 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:23 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:24.258 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:23 smithi148 ceph-mon[32528]: Standby manager daemon z started 2024-04-24T23:21:24.259 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:23 smithi148 ceph-mon[32528]: from='mgr.? 172.21.15.148:0/1108083170' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-24T23:21:24.259 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:23 smithi148 ceph-mon[32528]: from='mgr.? 172.21.15.148:0/1108083170' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T23:21:24.259 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:23 smithi148 ceph-mon[32528]: from='mgr.? 172.21.15.148:0/1108083170' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-24T23:21:24.259 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:23 smithi148 ceph-mon[32528]: from='mgr.? 172.21.15.148:0/1108083170' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T23:21:24.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:23 smithi080 ceph-mon[29152]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:21:24.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:23 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:24.290 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:23 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:21:24.290 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:23 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:21:24.290 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:23 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:21:24.290 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:23 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:24.290 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:23 smithi080 ceph-mon[29152]: Standby manager daemon z started 2024-04-24T23:21:24.290 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:23 smithi080 ceph-mon[29152]: from='mgr.? 172.21.15.148:0/1108083170' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-24T23:21:24.291 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:23 smithi080 ceph-mon[29152]: from='mgr.? 172.21.15.148:0/1108083170' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T23:21:24.291 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:23 smithi080 ceph-mon[29152]: from='mgr.? 172.21.15.148:0/1108083170' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-24T23:21:24.291 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:23 smithi080 ceph-mon[29152]: from='mgr.? 172.21.15.148:0/1108083170' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T23:21:24.948 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Apr 24 23:21:24 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-y[34042]: 2024-04-24T23:21:24.631+0000 7f976aabf200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-24T23:21:24.948 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Apr 24 23:21:24 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-y[34042]: 2024-04-24T23:21:24.722+0000 7f976aabf200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-24T23:21:25.299 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:25 smithi134 ceph-mon[32619]: mgrmap e14: x(active, since 77s), standbys: z 2024-04-24T23:21:25.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:25 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-24T23:21:25.378 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Apr 24 23:21:24 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-y[34042]: 2024-04-24T23:21:24.959+0000 7f976aabf200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-24T23:21:25.379 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Apr 24 23:21:25 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-y[34042]: 2024-04-24T23:21:25.091+0000 7f976aabf200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-24T23:21:25.379 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Apr 24 23:21:25 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-y[34042]: 2024-04-24T23:21:25.185+0000 7f976aabf200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-24T23:21:25.379 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Apr 24 23:21:25 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-y[34042]: 2024-04-24T23:21:25.298+0000 7f976aabf200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-24T23:21:25.403 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:25 smithi080 ceph-mon[29152]: mgrmap e14: x(active, since 77s), standbys: z 2024-04-24T23:21:25.403 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:25 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-24T23:21:25.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:25 smithi148 ceph-mon[32528]: mgrmap e14: x(active, since 77s), standbys: z 2024-04-24T23:21:25.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:25 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-24T23:21:25.698 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Apr 24 23:21:25 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-y[34042]: 2024-04-24T23:21:25.388+0000 7f976aabf200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-24T23:21:26.406 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Apr 24 23:21:26 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-y[34042]: 2024-04-24T23:21:26.193+0000 7f976aabf200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-24T23:21:26.406 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Apr 24 23:21:26 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-y[34042]: 2024-04-24T23:21:26.405+0000 7f976aabf200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-24T23:21:26.406 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:26 smithi134 ceph-mon[32619]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:21:26.406 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:26 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/2706551567' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a525741d-b46c-4eef-aa64-3f057cac1497"}]: dispatch 2024-04-24T23:21:26.407 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:26 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/2706551567' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a525741d-b46c-4eef-aa64-3f057cac1497"}]': finished 2024-04-24T23:21:26.407 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:26 smithi134 ceph-mon[32619]: osdmap e5: 1 total, 0 up, 1 in 2024-04-24T23:21:26.407 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:26 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T23:21:26.407 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:26 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/3433706762' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:21:26.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:26 smithi148 ceph-mon[32528]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:21:26.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:26 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/2706551567' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a525741d-b46c-4eef-aa64-3f057cac1497"}]: dispatch 2024-04-24T23:21:26.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:26 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/2706551567' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a525741d-b46c-4eef-aa64-3f057cac1497"}]': finished 2024-04-24T23:21:26.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:26 smithi148 ceph-mon[32528]: osdmap e5: 1 total, 0 up, 1 in 2024-04-24T23:21:26.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:26 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T23:21:26.508 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:26 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/3433706762' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:21:26.539 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:26 smithi080 ceph-mon[29152]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:21:26.539 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:26 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/2706551567' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a525741d-b46c-4eef-aa64-3f057cac1497"}]: dispatch 2024-04-24T23:21:26.539 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:26 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/2706551567' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a525741d-b46c-4eef-aa64-3f057cac1497"}]': finished 2024-04-24T23:21:26.539 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:26 smithi080 ceph-mon[29152]: osdmap e5: 1 total, 0 up, 1 in 2024-04-24T23:21:26.539 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:26 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T23:21:26.539 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:26 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/3433706762' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:21:27.059 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:21:27.057+0000 7f9dc8ff9700 1 -- 172.21.15.80:0/2365041686 <== mon.0 v2:172.21.15.80:3300/0 8 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7f9dbc00f460 con 0x7f9dcc1056d0 2024-04-24T23:21:27.440 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:27 smithi080 ceph-mon[29152]: Standby manager daemon y started 2024-04-24T23:21:27.441 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:27 smithi080 ceph-mon[29152]: from='mgr.? 172.21.15.134:0/3285366550' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-24T23:21:27.441 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:27 smithi080 ceph-mon[29152]: from='mgr.? 172.21.15.134:0/3285366550' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T23:21:27.441 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:27 smithi080 ceph-mon[29152]: from='mgr.? 172.21.15.134:0/3285366550' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-24T23:21:27.441 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:27 smithi080 ceph-mon[29152]: from='mgr.? 172.21.15.134:0/3285366550' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T23:21:27.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:27 smithi134 ceph-mon[32619]: Standby manager daemon y started 2024-04-24T23:21:27.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:27 smithi134 ceph-mon[32619]: from='mgr.? 172.21.15.134:0/3285366550' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-24T23:21:27.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:27 smithi134 ceph-mon[32619]: from='mgr.? 172.21.15.134:0/3285366550' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T23:21:27.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:27 smithi134 ceph-mon[32619]: from='mgr.? 172.21.15.134:0/3285366550' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-24T23:21:27.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:27 smithi134 ceph-mon[32619]: from='mgr.? 172.21.15.134:0/3285366550' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T23:21:27.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:27 smithi148 ceph-mon[32528]: Standby manager daemon y started 2024-04-24T23:21:27.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:27 smithi148 ceph-mon[32528]: from='mgr.? 172.21.15.134:0/3285366550' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-24T23:21:27.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:27 smithi148 ceph-mon[32528]: from='mgr.? 172.21.15.134:0/3285366550' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T23:21:27.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:27 smithi148 ceph-mon[32528]: from='mgr.? 172.21.15.134:0/3285366550' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-24T23:21:27.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:27 smithi148 ceph-mon[32528]: from='mgr.? 172.21.15.134:0/3285366550' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T23:21:28.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:28 smithi134 ceph-mon[32619]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:21:28.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:28 smithi134 ceph-mon[32619]: mgrmap e15: x(active, since 80s), standbys: z, y 2024-04-24T23:21:28.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:28 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-24T23:21:28.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:28 smithi148 ceph-mon[32528]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:21:28.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:28 smithi148 ceph-mon[32528]: mgrmap e15: x(active, since 80s), standbys: z, y 2024-04-24T23:21:28.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:28 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-24T23:21:28.539 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:28 smithi080 ceph-mon[29152]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:21:28.539 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:28 smithi080 ceph-mon[29152]: mgrmap e15: x(active, since 80s), standbys: z, y 2024-04-24T23:21:28.539 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:28 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-24T23:21:30.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:30 smithi148 ceph-mon[32528]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:21:30.269 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:30 smithi080 ceph-mon[29152]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:21:30.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:30 smithi134 ceph-mon[32619]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:21:32.355 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:32 smithi080 ceph-mon[29152]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:21:32.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:32 smithi134 ceph-mon[32619]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:21:32.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:32 smithi148 ceph-mon[32528]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:21:33.266 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:33 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-24T23:21:33.266 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:33 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:21:33.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:33 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-24T23:21:33.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:33 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:21:33.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:33 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-24T23:21:33.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:33 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:21:34.240 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:34 smithi080 ceph-mon[29152]: Deploying daemon osd.0 on smithi080 2024-04-24T23:21:34.241 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:34 smithi080 ceph-mon[29152]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:21:34.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:34 smithi134 ceph-mon[32619]: Deploying daemon osd.0 on smithi080 2024-04-24T23:21:34.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:34 smithi134 ceph-mon[32619]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:21:34.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:34 smithi148 ceph-mon[32528]: Deploying daemon osd.0 on smithi080 2024-04-24T23:21:34.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:34 smithi148 ceph-mon[32528]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:21:36.406 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:36 smithi080 ceph-mon[29152]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:21:36.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:36 smithi134 ceph-mon[32619]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:21:36.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:36 smithi148 ceph-mon[32528]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:21:38.232 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:38 smithi080 ceph-mon[29152]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:21:38.232 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:38 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:38.232 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:38 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:38.233 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:38 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:21:38.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:38 smithi134 ceph-mon[32619]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:21:38.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:38 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:38.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:38 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:38.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:38 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:21:38.503 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:38 smithi148 ceph-mon[32528]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:21:38.503 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:38 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:38.504 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:38 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:38.504 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:38 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:21:40.222 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:40 smithi080 ceph-mon[29152]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:21:40.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:40 smithi134 ceph-mon[32619]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:21:40.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:40 smithi148 ceph-mon[32528]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:21:40.865 INFO:teuthology.orchestra.run.smithi080.stdout:Created osd(s) 0 on host 'smithi080' 2024-04-24T23:21:40.866 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:21:40.863+0000 7f9dc8ff9700 1 -- 172.21.15.80:0/2365041686 <== mgr.14152 v2:172.21.15.80:6800/3830596922 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f9dcc062670 con 0x7f9db4039330 2024-04-24T23:21:40.867 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:21:40.865+0000 7f9dd1d17700 1 -- 172.21.15.80:0/2365041686 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f9db4039330 msgr2=0x7f9db403b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:40.867 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:21:40.865+0000 7f9dd1d17700 1 --2- 172.21.15.80:0/2365041686 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f9db4039330 0x7f9db403b7f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f9dc0007c60 tx=0x7f9dc00073d0 comp rx=0 tx=0).stop 2024-04-24T23:21:40.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:21:40.866+0000 7f9dd1d17700 1 -- 172.21.15.80:0/2365041686 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9dcc1056d0 msgr2=0x7f9dcc1989d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:40.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:21:40.866+0000 7f9dd1d17700 1 --2- 172.21.15.80:0/2365041686 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9dcc1056d0 0x7f9dcc1989d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc005910 tx=0x7f9dbc0047b0 comp rx=0 tx=0).stop 2024-04-24T23:21:40.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:21:40.866+0000 7f9dd1d17700 1 -- 172.21.15.80:0/2365041686 shutdown_connections 2024-04-24T23:21:40.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:21:40.866+0000 7f9dd1d17700 1 --2- 172.21.15.80:0/2365041686 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9dcc108000 0x7f9dcc19fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:40.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:21:40.866+0000 7f9dd1d17700 1 --2- 172.21.15.80:0/2365041686 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9dcc102da0 0x7f9dcc198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:40.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:21:40.866+0000 7f9dd1d17700 1 --2- 172.21.15.80:0/2365041686 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f9db4039330 0x7f9db403b7f0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:40.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:21:40.866+0000 7f9dd1d17700 1 --2- 172.21.15.80:0/2365041686 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9dcc1056d0 0x7f9dcc1989d0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:40.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:21:40.866+0000 7f9dd1d17700 1 -- 172.21.15.80:0/2365041686 >> 172.21.15.80:0/2365041686 conn(0x7f9dcc0fc750 msgr2=0x7f9dcc103660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:40.869 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:21:40.866+0000 7f9dd1d17700 1 -- 172.21.15.80:0/2365041686 shutdown_connections 2024-04-24T23:21:40.869 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:21:40.866+0000 7f9dd1d17700 1 -- 172.21.15.80:0/2365041686 wait complete. 2024-04-24T23:21:41.550 DEBUG:teuthology.orchestra.run.smithi080:osd.0> sudo journalctl -f -n 0 -u ceph-ec7885aa-0290-11ef-bc93-c7b262605968@osd.0.service 2024-04-24T23:21:41.553 INFO:tasks.cephadm:Deploying osd.1 on smithi134 with /dev/vg_nvme/lv_4... 2024-04-24T23:21:41.553 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ec7885aa-0290-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-24T23:21:41.795 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/mon.b/config 2024-04-24T23:21:42.124 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:41 smithi134 ceph-mon[32619]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:21:42.125 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:41 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:42.125 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:41 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:42.125 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:41 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:42.125 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:41 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:42.125 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:41 smithi134 ceph-mon[32619]: from='osd.0 [v2:172.21.15.80:6802/2597718112,v1:172.21.15.80:6803/2597718112]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-24T23:21:42.133 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:41 smithi080 ceph-mon[29152]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:21:42.133 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:41 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:42.134 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:41 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:42.134 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:41 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:42.134 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:41 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:42.134 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:41 smithi080 ceph-mon[29152]: from='osd.0 [v2:172.21.15.80:6802/2597718112,v1:172.21.15.80:6803/2597718112]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-24T23:21:42.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:41 smithi148 ceph-mon[32528]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:21:42.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:41 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:42.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:41 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:42.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:41 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:42.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:41 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:42.258 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:41 smithi148 ceph-mon[32528]: from='osd.0 [v2:172.21.15.80:6802/2597718112,v1:172.21.15.80:6803/2597718112]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-24T23:21:43.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:43 smithi134 ceph-mon[32619]: from='osd.0 [v2:172.21.15.80:6802/2597718112,v1:172.21.15.80:6803/2597718112]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-24T23:21:43.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:43 smithi134 ceph-mon[32619]: osdmap e6: 1 total, 0 up, 1 in 2024-04-24T23:21:43.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:43 smithi134 ceph-mon[32619]: from='osd.0 [v2:172.21.15.80:6802/2597718112,v1:172.21.15.80:6803/2597718112]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi080", "root=default"]}]: dispatch 2024-04-24T23:21:43.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:43 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T23:21:43.449 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:43 smithi134 ceph-mon[32619]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:21:43.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:43 smithi148 ceph-mon[32528]: from='osd.0 [v2:172.21.15.80:6802/2597718112,v1:172.21.15.80:6803/2597718112]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-24T23:21:43.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:43 smithi148 ceph-mon[32528]: osdmap e6: 1 total, 0 up, 1 in 2024-04-24T23:21:43.508 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:43 smithi148 ceph-mon[32528]: from='osd.0 [v2:172.21.15.80:6802/2597718112,v1:172.21.15.80:6803/2597718112]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi080", "root=default"]}]: dispatch 2024-04-24T23:21:43.508 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:43 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T23:21:43.508 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:43 smithi148 ceph-mon[32528]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:21:43.540 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:43 smithi080 ceph-mon[29152]: from='osd.0 [v2:172.21.15.80:6802/2597718112,v1:172.21.15.80:6803/2597718112]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-24T23:21:43.540 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:43 smithi080 ceph-mon[29152]: osdmap e6: 1 total, 0 up, 1 in 2024-04-24T23:21:43.540 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:43 smithi080 ceph-mon[29152]: from='osd.0 [v2:172.21.15.80:6802/2597718112,v1:172.21.15.80:6803/2597718112]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi080", "root=default"]}]: dispatch 2024-04-24T23:21:43.540 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:43 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T23:21:43.540 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:43 smithi080 ceph-mon[29152]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:21:43.541 INFO:journalctl@ceph.osd.0.smithi080.stdout:Apr 24 23:21:43 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-0[43667]: 2024-04-24T23:21:43.186+0000 7fb3ba062700 -1 osd.0 0 waiting for initial osdmap 2024-04-24T23:21:43.541 INFO:journalctl@ceph.osd.0.smithi080.stdout:Apr 24 23:21:43 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-0[43667]: 2024-04-24T23:21:43.188+0000 7fb3b5e55700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T23:21:43.712 INFO:teuthology.orchestra.run.smithi134.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-24T23:21:43.713 INFO:teuthology.orchestra.run.smithi134.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-24T23:21:43.713 INFO:teuthology.orchestra.run.smithi134.stderr: stderr: 10+0 records in 2024-04-24T23:21:43.713 INFO:teuthology.orchestra.run.smithi134.stderr:10+0 records out 2024-04-24T23:21:43.713 INFO:teuthology.orchestra.run.smithi134.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.011247 s, 932 MB/s 2024-04-24T23:21:43.713 INFO:teuthology.orchestra.run.smithi134.stderr:--> Zapping successful for: 2024-04-24T23:21:44.302 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:44 smithi134 ceph-mon[32619]: from='osd.0 [v2:172.21.15.80:6802/2597718112,v1:172.21.15.80:6803/2597718112]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi080", "root=default"]}]': finished 2024-04-24T23:21:44.302 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:44 smithi134 ceph-mon[32619]: osdmap e7: 1 total, 0 up, 1 in 2024-04-24T23:21:44.303 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:44 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T23:21:44.303 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:44 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T23:21:44.304 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ec7885aa-0290-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi134:vg_nvme/lv_4 2024-04-24T23:21:44.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:44 smithi148 ceph-mon[32528]: from='osd.0 [v2:172.21.15.80:6802/2597718112,v1:172.21.15.80:6803/2597718112]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi080", "root=default"]}]': finished 2024-04-24T23:21:44.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:44 smithi148 ceph-mon[32528]: osdmap e7: 1 total, 0 up, 1 in 2024-04-24T23:21:44.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:44 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T23:21:44.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:44 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T23:21:44.539 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:44 smithi080 ceph-mon[29152]: from='osd.0 [v2:172.21.15.80:6802/2597718112,v1:172.21.15.80:6803/2597718112]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi080", "root=default"]}]': finished 2024-04-24T23:21:44.539 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:44 smithi080 ceph-mon[29152]: osdmap e7: 1 total, 0 up, 1 in 2024-04-24T23:21:44.539 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:44 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T23:21:44.539 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:44 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T23:21:44.547 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/mon.b/config 2024-04-24T23:21:45.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:45 smithi134 ceph-mon[32619]: purged_snaps scrub starts 2024-04-24T23:21:45.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:45 smithi134 ceph-mon[32619]: purged_snaps scrub ok 2024-04-24T23:21:45.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:45 smithi134 ceph-mon[32619]: osd.0 [v2:172.21.15.80:6802/2597718112,v1:172.21.15.80:6803/2597718112] boot 2024-04-24T23:21:45.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:45 smithi134 ceph-mon[32619]: osdmap e8: 1 total, 1 up, 1 in 2024-04-24T23:21:45.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:45 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T23:21:45.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:45 smithi134 ceph-mon[32619]: Detected new or changed devices on smithi080 2024-04-24T23:21:45.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:45 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:45.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:45 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:45.449 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:45 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:21:45.449 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:45 smithi134 ceph-mon[32619]: Adjusting osd_memory_target on smithi080 to 17088M 2024-04-24T23:21:45.449 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:45 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:45.449 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:45 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:21:45.449 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:45 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:21:45.449 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:45 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:45.449 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:45 smithi134 ceph-mon[32619]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:21:45.449 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:45 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:21:45.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:45 smithi148 ceph-mon[32528]: purged_snaps scrub starts 2024-04-24T23:21:45.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:45 smithi148 ceph-mon[32528]: purged_snaps scrub ok 2024-04-24T23:21:45.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:45 smithi148 ceph-mon[32528]: osd.0 [v2:172.21.15.80:6802/2597718112,v1:172.21.15.80:6803/2597718112] boot 2024-04-24T23:21:45.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:45 smithi148 ceph-mon[32528]: osdmap e8: 1 total, 1 up, 1 in 2024-04-24T23:21:45.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:45 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T23:21:45.508 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:45 smithi148 ceph-mon[32528]: Detected new or changed devices on smithi080 2024-04-24T23:21:45.508 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:45 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:45.508 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:45 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:45.508 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:45 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:21:45.508 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:45 smithi148 ceph-mon[32528]: Adjusting osd_memory_target on smithi080 to 17088M 2024-04-24T23:21:45.508 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:45 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:45.508 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:45 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:21:45.508 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:45 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:21:45.508 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:45 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:45.508 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:45 smithi148 ceph-mon[32528]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:21:45.508 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:45 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:21:45.539 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:45 smithi080 ceph-mon[29152]: purged_snaps scrub starts 2024-04-24T23:21:45.539 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:45 smithi080 ceph-mon[29152]: purged_snaps scrub ok 2024-04-24T23:21:45.539 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:45 smithi080 ceph-mon[29152]: osd.0 [v2:172.21.15.80:6802/2597718112,v1:172.21.15.80:6803/2597718112] boot 2024-04-24T23:21:45.539 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:45 smithi080 ceph-mon[29152]: osdmap e8: 1 total, 1 up, 1 in 2024-04-24T23:21:45.539 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:45 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T23:21:45.539 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:45 smithi080 ceph-mon[29152]: Detected new or changed devices on smithi080 2024-04-24T23:21:45.539 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:45 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:45.540 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:45 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:45.540 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:45 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:21:45.540 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:45 smithi080 ceph-mon[29152]: Adjusting osd_memory_target on smithi080 to 17088M 2024-04-24T23:21:45.540 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:45 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:45.540 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:45 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:21:45.540 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:45 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:21:45.540 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:45 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:45.540 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:45 smithi080 ceph-mon[29152]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:21:45.540 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:45 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:21:46.082 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:21:46.078+0000 7f75393a1700 1 -- 172.21.15.134:0/2955367989 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7534102d80 msgr2=0x7f7534105170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:46.083 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:21:46.078+0000 7f75393a1700 1 --2- 172.21.15.134:0/2955367989 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7534102d80 0x7f7534105170 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7524009990 tx=0x7f7524009ca0 comp rx=0 tx=0).stop 2024-04-24T23:21:46.083 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:21:46.078+0000 7f75393a1700 1 -- 172.21.15.134:0/2955367989 shutdown_connections 2024-04-24T23:21:46.083 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:21:46.078+0000 7f75393a1700 1 --2- 172.21.15.134:0/2955367989 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7534107fe0 0x7f753410a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:46.083 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:21:46.078+0000 7f75393a1700 1 --2- 172.21.15.134:0/2955367989 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7534102d80 0x7f7534105170 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:46.083 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:21:46.078+0000 7f75393a1700 1 --2- 172.21.15.134:0/2955367989 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f75341056b0 0x7f7534107aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:46.083 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:21:46.078+0000 7f75393a1700 1 -- 172.21.15.134:0/2955367989 >> 172.21.15.134:0/2955367989 conn(0x7f75340fc750 msgr2=0x7f75340feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:46.083 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:21:46.078+0000 7f75393a1700 1 -- 172.21.15.134:0/2955367989 shutdown_connections 2024-04-24T23:21:46.083 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:21:46.079+0000 7f75393a1700 1 -- 172.21.15.134:0/2955367989 wait complete. 2024-04-24T23:21:46.083 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:21:46.079+0000 7f75393a1700 1 Processor -- start 2024-04-24T23:21:46.083 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:21:46.079+0000 7f75393a1700 1 -- start start 2024-04-24T23:21:46.083 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:21:46.079+0000 7f75393a1700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7534102d80 0x7f75341984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:46.083 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:21:46.079+0000 7f75393a1700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f75341056b0 0x7f7534198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:46.084 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:21:46.079+0000 7f75393a1700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7534107fe0 0x7f753419fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:46.084 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:21:46.080+0000 7f75393a1700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f753410deb0 con 0x7f7534107fe0 2024-04-24T23:21:46.084 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:21:46.080+0000 7f75393a1700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f753410dd30 con 0x7f7534102d80 2024-04-24T23:21:46.084 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:21:46.080+0000 7f75393a1700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f753410e030 con 0x7f75341056b0 2024-04-24T23:21:46.084 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:21:46.080+0000 7f7532ffd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7534102d80 0x7f75341984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:46.084 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:21:46.080+0000 7f7532ffd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7534102d80 0x7f75341984f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:56552/0 (socket says 172.21.15.134:56552) 2024-04-24T23:21:46.084 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:21:46.080+0000 7f7532ffd700 1 -- 172.21.15.134:0/2719608614 learned_addr learned my addr 172.21.15.134:0/2719608614 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-04-24T23:21:46.084 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:21:46.080+0000 7f75337fe700 1 --2- 172.21.15.134:0/2719608614 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7534107fe0 0x7f753419fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:46.084 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:21:46.080+0000 7f75327fc700 1 --2- 172.21.15.134:0/2719608614 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f75341056b0 0x7f7534198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:46.084 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:21:46.080+0000 7f7532ffd700 1 -- 172.21.15.134:0/2719608614 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f75341056b0 msgr2=0x7f7534198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:46.084 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:21:46.080+0000 7f7532ffd700 1 --2- 172.21.15.134:0/2719608614 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f75341056b0 0x7f7534198a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:46.085 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:21:46.080+0000 7f7532ffd700 1 -- 172.21.15.134:0/2719608614 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7534107fe0 msgr2=0x7f753419fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:46.085 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:21:46.080+0000 7f7532ffd700 1 --2- 172.21.15.134:0/2719608614 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7534107fe0 0x7f753419fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:46.085 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:21:46.080+0000 7f7532ffd700 1 -- 172.21.15.134:0/2719608614 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75341a0150 con 0x7f7534102d80 2024-04-24T23:21:46.085 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:21:46.080+0000 7f75337fe700 1 --2- 172.21.15.134:0/2719608614 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7534107fe0 0x7f753419fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:21:46.085 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:21:46.080+0000 7f7532ffd700 1 --2- 172.21.15.134:0/2719608614 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7534102d80 0x7f75341984f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f752400b7b0 tx=0x7f752400bc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:46.085 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:21:46.080+0000 7f751bfff700 1 -- 172.21.15.134:0/2719608614 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7524027070 con 0x7f7534102d80 2024-04-24T23:21:46.085 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:21:46.080+0000 7f751bfff700 1 -- 172.21.15.134:0/2719608614 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7524003b20 con 0x7f7534102d80 2024-04-24T23:21:46.085 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:21:46.080+0000 7f75393a1700 1 -- 172.21.15.134:0/2719608614 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75341a03e0 con 0x7f7534102d80 2024-04-24T23:21:46.085 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:21:46.080+0000 7f751bfff700 1 -- 172.21.15.134:0/2719608614 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f752400f6c0 con 0x7f7534102d80 2024-04-24T23:21:46.085 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:21:46.080+0000 7f75393a1700 1 -- 172.21.15.134:0/2719608614 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75341a0890 con 0x7f7534102d80 2024-04-24T23:21:46.086 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:21:46.081+0000 7f75393a1700 1 -- 172.21.15.134:0/2719608614 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7534068f20 con 0x7f7534102d80 2024-04-24T23:21:46.086 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:21:46.082+0000 7f751bfff700 1 -- 172.21.15.134:0/2719608614 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7f752402a070 con 0x7f7534102d80 2024-04-24T23:21:46.086 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:21:46.083+0000 7f751bfff700 1 --2- 172.21.15.134:0/2719608614 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f751c0a2c10 0x7f751c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:46.086 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:21:46.083+0000 7f751bfff700 1 -- 172.21.15.134:0/2719608614 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7f75240ce670 con 0x7f7534102d80 2024-04-24T23:21:46.086 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:21:46.083+0000 7f75327fc700 1 --2- 172.21.15.134:0/2719608614 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f751c0a2c10 0x7f751c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:46.086 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:21:46.083+0000 7f75327fc700 1 --2- 172.21.15.134:0/2719608614 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f751c0a2c10 0x7f751c0a50d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f7528009bf0 tx=0x7f7528009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:46.089 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:21:46.087+0000 7f751bfff700 1 -- 172.21.15.134:0/2719608614 <== mon.2 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f752405e740 con 0x7f7534102d80 2024-04-24T23:21:46.294 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:21:46.292+0000 7f75393a1700 1 -- 172.21.15.134:0/2719608614 --> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f7534062670 con 0x7f751c0a2c10 2024-04-24T23:21:46.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:46 smithi134 ceph-mon[32619]: osdmap e9: 1 total, 1 up, 1 in 2024-04-24T23:21:46.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:46 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:21:46.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:46 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:21:46.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:46 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:46.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:46 smithi148 ceph-mon[32528]: osdmap e9: 1 total, 1 up, 1 in 2024-04-24T23:21:46.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:46 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:21:46.508 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:46 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:21:46.508 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:46 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:46.539 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:46 smithi080 ceph-mon[29152]: osdmap e9: 1 total, 1 up, 1 in 2024-04-24T23:21:46.540 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:46 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:21:46.540 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:46 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:21:46.540 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:46 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:21:47.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:47 smithi148 ceph-mon[32528]: from='client.24152 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:21:47.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:47 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:21:47.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:47 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:21:47.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:47 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:21:47.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:47 smithi148 ceph-mon[32528]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:21:47.539 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:47 smithi080 ceph-mon[29152]: from='client.24152 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:21:47.539 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:47 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:21:47.539 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:47 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:21:47.539 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:47 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:21:47.539 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:47 smithi080 ceph-mon[29152]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:21:47.583 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:47 smithi134 ceph-mon[32619]: from='client.24152 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:21:47.583 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:47 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:21:47.583 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:47 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:21:47.583 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:47 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:21:47.583 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:47 smithi134 ceph-mon[32619]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:21:50.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:49 smithi134 ceph-mon[32619]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:21:50.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:49 smithi134 ceph-mon[32619]: from='client.? 172.21.15.134:0/3317587608' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2d0f0d67-6db4-40ac-91cc-58604ab6795a"}]: dispatch 2024-04-24T23:21:50.199 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:49 smithi134 ceph-mon[32619]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2d0f0d67-6db4-40ac-91cc-58604ab6795a"}]: dispatch 2024-04-24T23:21:50.199 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:49 smithi134 ceph-mon[32619]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2d0f0d67-6db4-40ac-91cc-58604ab6795a"}]': finished 2024-04-24T23:21:50.199 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:49 smithi134 ceph-mon[32619]: osdmap e10: 2 total, 1 up, 2 in 2024-04-24T23:21:50.199 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:49 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T23:21:50.199 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:49 smithi134 ceph-mon[32619]: from='client.? 172.21.15.134:0/244760770' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:21:50.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:49 smithi148 ceph-mon[32528]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:21:50.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:49 smithi148 ceph-mon[32528]: from='client.? 172.21.15.134:0/3317587608' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2d0f0d67-6db4-40ac-91cc-58604ab6795a"}]: dispatch 2024-04-24T23:21:50.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:49 smithi148 ceph-mon[32528]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2d0f0d67-6db4-40ac-91cc-58604ab6795a"}]: dispatch 2024-04-24T23:21:50.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:49 smithi148 ceph-mon[32528]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2d0f0d67-6db4-40ac-91cc-58604ab6795a"}]': finished 2024-04-24T23:21:50.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:49 smithi148 ceph-mon[32528]: osdmap e10: 2 total, 1 up, 2 in 2024-04-24T23:21:50.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:49 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T23:21:50.258 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:49 smithi148 ceph-mon[32528]: from='client.? 172.21.15.134:0/244760770' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:21:50.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:49 smithi080 ceph-mon[29152]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:21:50.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:49 smithi080 ceph-mon[29152]: from='client.? 172.21.15.134:0/3317587608' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2d0f0d67-6db4-40ac-91cc-58604ab6795a"}]: dispatch 2024-04-24T23:21:50.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:49 smithi080 ceph-mon[29152]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2d0f0d67-6db4-40ac-91cc-58604ab6795a"}]: dispatch 2024-04-24T23:21:50.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:49 smithi080 ceph-mon[29152]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2d0f0d67-6db4-40ac-91cc-58604ab6795a"}]': finished 2024-04-24T23:21:50.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:49 smithi080 ceph-mon[29152]: osdmap e10: 2 total, 1 up, 2 in 2024-04-24T23:21:50.290 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:49 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T23:21:50.290 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:49 smithi080 ceph-mon[29152]: from='client.? 172.21.15.134:0/244760770' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:21:52.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:51 smithi134 ceph-mon[32619]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:21:52.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:51 smithi148 ceph-mon[32528]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:21:52.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:51 smithi080 ceph-mon[29152]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:21:54.053 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:53 smithi134 ceph-mon[32619]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:21:54.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:53 smithi148 ceph-mon[32528]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:21:54.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:53 smithi080 ceph-mon[29152]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:21:56.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:55 smithi134 ceph-mon[32619]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:21:56.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:55 smithi148 ceph-mon[32528]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:21:56.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:55 smithi080 ceph-mon[29152]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:21:57.191 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:56 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-24T23:21:57.191 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:56 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:21:57.191 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:56 smithi080 ceph-mon[29152]: Deploying daemon osd.1 on smithi134 2024-04-24T23:21:57.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:56 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-24T23:21:57.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:56 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:21:57.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:56 smithi134 ceph-mon[32619]: Deploying daemon osd.1 on smithi134 2024-04-24T23:21:57.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:56 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-24T23:21:57.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:56 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:21:57.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:56 smithi148 ceph-mon[32528]: Deploying daemon osd.1 on smithi134 2024-04-24T23:21:58.153 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:57 smithi134 ceph-mon[32619]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:21:58.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:57 smithi148 ceph-mon[32528]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:21:58.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:57 smithi080 ceph-mon[29152]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:22:00.129 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:21:59 smithi134 ceph-mon[32619]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:22:00.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:21:59 smithi148 ceph-mon[32528]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:22:00.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:21:59 smithi080 ceph-mon[29152]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:22:01.948 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:01 smithi134 ceph-mon[32619]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:22:01.948 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:01 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:22:01.948 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:01 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:22:01.948 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:01 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:22:02.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:01 smithi148 ceph-mon[32528]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:22:02.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:01 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:22:02.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:01 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:22:02.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:01 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:22:02.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:01 smithi080 ceph-mon[29152]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:22:02.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:01 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:22:02.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:01 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:22:02.290 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:01 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:22:04.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:03 smithi134 ceph-mon[32619]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:22:04.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:03 smithi148 ceph-mon[32528]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:22:04.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:03 smithi080 ceph-mon[29152]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:22:04.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:22:04.957+0000 7f751bfff700 1 -- 172.21.15.134:0/2719608614 <== mgr.14152 v2:172.21.15.80:6800/3830596922 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f7534062670 con 0x7f751c0a2c10 2024-04-24T23:22:04.959 INFO:teuthology.orchestra.run.smithi134.stdout:Created osd(s) 1 on host 'smithi134' 2024-04-24T23:22:04.961 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:22:04.959+0000 7f75393a1700 1 -- 172.21.15.134:0/2719608614 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f751c0a2c10 msgr2=0x7f751c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:04.961 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:22:04.959+0000 7f75393a1700 1 --2- 172.21.15.134:0/2719608614 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f751c0a2c10 0x7f751c0a50d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f7528009bf0 tx=0x7f7528009260 comp rx=0 tx=0).stop 2024-04-24T23:22:04.961 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:22:04.959+0000 7f75393a1700 1 -- 172.21.15.134:0/2719608614 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7534102d80 msgr2=0x7f75341984f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:04.961 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:22:04.959+0000 7f75393a1700 1 --2- 172.21.15.134:0/2719608614 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7534102d80 0x7f75341984f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f752400b7b0 tx=0x7f752400bc60 comp rx=0 tx=0).stop 2024-04-24T23:22:04.961 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:22:04.959+0000 7f75393a1700 1 -- 172.21.15.134:0/2719608614 shutdown_connections 2024-04-24T23:22:04.961 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:22:04.959+0000 7f75393a1700 1 --2- 172.21.15.134:0/2719608614 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f751c0a2c10 0x7f751c0a50d0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:04.961 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:22:04.959+0000 7f75393a1700 1 --2- 172.21.15.134:0/2719608614 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7534107fe0 0x7f753419fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:04.961 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:22:04.959+0000 7f75393a1700 1 --2- 172.21.15.134:0/2719608614 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7534102d80 0x7f75341984f0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:04.961 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:22:04.959+0000 7f75393a1700 1 --2- 172.21.15.134:0/2719608614 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f75341056b0 0x7f7534198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:04.961 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:22:04.959+0000 7f75393a1700 1 -- 172.21.15.134:0/2719608614 >> 172.21.15.134:0/2719608614 conn(0x7f75340fc750 msgr2=0x7f75341036c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:04.961 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:22:04.959+0000 7f75393a1700 1 -- 172.21.15.134:0/2719608614 shutdown_connections 2024-04-24T23:22:04.962 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:22:04.959+0000 7f75393a1700 1 -- 172.21.15.134:0/2719608614 wait complete. 2024-04-24T23:22:05.167 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:04 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:22:05.168 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:04 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:22:05.168 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:04 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:22:05.168 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:04 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:22:05.168 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:04 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:22:05.168 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:04 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:22:05.168 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:04 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:22:05.168 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:04 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:22:05.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:04 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:22:05.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:04 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:22:05.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:04 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:22:05.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:04 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:22:05.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:04 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:22:05.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:04 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:22:05.258 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:04 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:22:05.258 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:04 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:22:05.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:04 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:22:05.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:04 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:22:05.290 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:04 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:22:05.290 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:04 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:22:05.290 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:04 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:22:05.290 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:04 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:22:05.290 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:04 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:22:05.290 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:04 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:22:05.571 DEBUG:teuthology.orchestra.run.smithi134:osd.1> sudo journalctl -f -n 0 -u ceph-ec7885aa-0290-11ef-bc93-c7b262605968@osd.1.service 2024-04-24T23:22:05.573 INFO:tasks.cephadm:Deploying osd.2 on smithi148 with /dev/vg_nvme/lv_4... 2024-04-24T23:22:05.573 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ec7885aa-0290-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-24T23:22:05.816 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T23:22:06.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:05 smithi134 ceph-mon[32619]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:22:06.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:05 smithi134 ceph-mon[32619]: from='osd.1 [v2:172.21.15.134:6800/2395610201,v1:172.21.15.134:6801/2395610201]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-24T23:22:06.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:05 smithi134 ceph-mon[32619]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-24T23:22:06.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:05 smithi148 ceph-mon[32528]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:22:06.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:05 smithi148 ceph-mon[32528]: from='osd.1 [v2:172.21.15.134:6800/2395610201,v1:172.21.15.134:6801/2395610201]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-24T23:22:06.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:05 smithi148 ceph-mon[32528]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-24T23:22:06.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:05 smithi080 ceph-mon[29152]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:22:06.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:05 smithi080 ceph-mon[29152]: from='osd.1 [v2:172.21.15.134:6800/2395610201,v1:172.21.15.134:6801/2395610201]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-24T23:22:06.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:05 smithi080 ceph-mon[29152]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-24T23:22:07.072 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:07 smithi134 ceph-mon[32619]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-24T23:22:07.072 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:07 smithi134 ceph-mon[32619]: osdmap e11: 2 total, 1 up, 2 in 2024-04-24T23:22:07.072 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:07 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T23:22:07.072 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:07 smithi134 ceph-mon[32619]: from='osd.1 [v2:172.21.15.134:6800/2395610201,v1:172.21.15.134:6801/2395610201]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-04-24T23:22:07.072 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:07 smithi134 ceph-mon[32619]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-04-24T23:22:07.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:07 smithi148 ceph-mon[32528]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-24T23:22:07.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:07 smithi148 ceph-mon[32528]: osdmap e11: 2 total, 1 up, 2 in 2024-04-24T23:22:07.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:07 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T23:22:07.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:07 smithi148 ceph-mon[32528]: from='osd.1 [v2:172.21.15.134:6800/2395610201,v1:172.21.15.134:6801/2395610201]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-04-24T23:22:07.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:07 smithi148 ceph-mon[32528]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-04-24T23:22:07.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:07 smithi080 ceph-mon[29152]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-24T23:22:07.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:07 smithi080 ceph-mon[29152]: osdmap e11: 2 total, 1 up, 2 in 2024-04-24T23:22:07.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:07 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T23:22:07.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:07 smithi080 ceph-mon[29152]: from='osd.1 [v2:172.21.15.134:6800/2395610201,v1:172.21.15.134:6801/2395610201]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-04-24T23:22:07.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:07 smithi080 ceph-mon[29152]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-04-24T23:22:07.863 INFO:teuthology.orchestra.run.smithi148.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-24T23:22:07.863 INFO:teuthology.orchestra.run.smithi148.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-24T23:22:07.864 INFO:teuthology.orchestra.run.smithi148.stderr: stderr: 10+0 records in 2024-04-24T23:22:07.864 INFO:teuthology.orchestra.run.smithi148.stderr:10+0 records out 2024-04-24T23:22:07.864 INFO:teuthology.orchestra.run.smithi148.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113707 s, 922 MB/s 2024-04-24T23:22:07.864 INFO:teuthology.orchestra.run.smithi148.stderr:--> Zapping successful for: 2024-04-24T23:22:07.948 INFO:journalctl@ceph.osd.1.smithi134.stdout:Apr 24 23:22:07 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-1[37250]: 2024-04-24T23:22:07.490+0000 7f9bf3ac3700 -1 osd.1 0 waiting for initial osdmap 2024-04-24T23:22:07.948 INFO:journalctl@ceph.osd.1.smithi134.stdout:Apr 24 23:22:07 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-1[37250]: 2024-04-24T23:22:07.493+0000 7f9beb8ae700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T23:22:08.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:08 smithi080 ceph-mon[29152]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:22:08.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:08 smithi080 ceph-mon[29152]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-04-24T23:22:08.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:08 smithi080 ceph-mon[29152]: osdmap e12: 2 total, 1 up, 2 in 2024-04-24T23:22:08.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:08 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T23:22:08.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:08 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T23:22:08.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:08 smithi080 ceph-mon[29152]: from='osd.1 ' entity='osd.1' 2024-04-24T23:22:08.315 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:08 smithi134 ceph-mon[32619]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:22:08.315 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:08 smithi134 ceph-mon[32619]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-04-24T23:22:08.315 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:08 smithi134 ceph-mon[32619]: osdmap e12: 2 total, 1 up, 2 in 2024-04-24T23:22:08.315 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:08 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T23:22:08.315 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:08 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T23:22:08.315 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:08 smithi134 ceph-mon[32619]: from='osd.1 ' entity='osd.1' 2024-04-24T23:22:08.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:08 smithi148 ceph-mon[32528]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:22:08.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:08 smithi148 ceph-mon[32528]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-04-24T23:22:08.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:08 smithi148 ceph-mon[32528]: osdmap e12: 2 total, 1 up, 2 in 2024-04-24T23:22:08.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:08 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T23:22:08.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:08 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T23:22:08.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:08 smithi148 ceph-mon[32528]: from='osd.1 ' entity='osd.1' 2024-04-24T23:22:08.533 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ec7885aa-0290-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi148:vg_nvme/lv_4 2024-04-24T23:22:08.780 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T23:22:09.110 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:09 smithi148 ceph-mon[32528]: purged_snaps scrub starts 2024-04-24T23:22:09.110 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:09 smithi148 ceph-mon[32528]: purged_snaps scrub ok 2024-04-24T23:22:09.110 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:09 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T23:22:09.110 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:09 smithi148 ceph-mon[32528]: osd.1 [v2:172.21.15.134:6800/2395610201,v1:172.21.15.134:6801/2395610201] boot 2024-04-24T23:22:09.110 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:09 smithi148 ceph-mon[32528]: osdmap e13: 2 total, 2 up, 2 in 2024-04-24T23:22:09.110 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:09 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T23:22:09.110 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:09 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:22:09.111 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:09 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:22:09.111 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:09 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:22:09.111 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:09 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:22:09.111 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:09 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:22:09.111 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:09 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:22:09.111 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:09 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:22:09.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:09 smithi080 ceph-mon[29152]: purged_snaps scrub starts 2024-04-24T23:22:09.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:09 smithi080 ceph-mon[29152]: purged_snaps scrub ok 2024-04-24T23:22:09.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:09 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T23:22:09.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:09 smithi080 ceph-mon[29152]: osd.1 [v2:172.21.15.134:6800/2395610201,v1:172.21.15.134:6801/2395610201] boot 2024-04-24T23:22:09.290 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:09 smithi080 ceph-mon[29152]: osdmap e13: 2 total, 2 up, 2 in 2024-04-24T23:22:09.290 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:09 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T23:22:09.290 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:09 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:22:09.290 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:09 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:22:09.290 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:09 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:22:09.290 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:09 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:22:09.290 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:09 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:22:09.290 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:09 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:22:09.290 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:09 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:22:09.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:09 smithi134 ceph-mon[32619]: purged_snaps scrub starts 2024-04-24T23:22:09.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:09 smithi134 ceph-mon[32619]: purged_snaps scrub ok 2024-04-24T23:22:09.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:09 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T23:22:09.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:09 smithi134 ceph-mon[32619]: osd.1 [v2:172.21.15.134:6800/2395610201,v1:172.21.15.134:6801/2395610201] boot 2024-04-24T23:22:09.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:09 smithi134 ceph-mon[32619]: osdmap e13: 2 total, 2 up, 2 in 2024-04-24T23:22:09.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:09 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T23:22:09.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:09 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:22:09.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:09 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:22:09.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:09 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:22:09.449 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:09 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:22:09.449 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:09 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:22:09.449 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:09 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:22:09.449 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:09 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:22:10.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:10 smithi080 ceph-mon[29152]: Detected new or changed devices on smithi134 2024-04-24T23:22:10.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:10 smithi080 ceph-mon[29152]: Adjusting osd_memory_target on smithi134 to 17088M 2024-04-24T23:22:10.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:10 smithi080 ceph-mon[29152]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:22:10.299 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:10 smithi148 ceph-mon[32528]: Detected new or changed devices on smithi134 2024-04-24T23:22:10.299 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:10 smithi148 ceph-mon[32528]: Adjusting osd_memory_target on smithi134 to 17088M 2024-04-24T23:22:10.299 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:10 smithi148 ceph-mon[32528]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:22:10.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:22:10.316+0000 7f9f2747e700 1 -- 172.21.15.148:0/1033367474 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f20102db0 msgr2=0x7f9f201051a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:10.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:22:10.316+0000 7f9f2747e700 1 --2- 172.21.15.148:0/1033367474 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f20102db0 0x7f9f201051a0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c009990 tx=0x7f9f1c009ca0 comp rx=0 tx=0).stop 2024-04-24T23:22:10.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:22:10.316+0000 7f9f2747e700 1 -- 172.21.15.148:0/1033367474 shutdown_connections 2024-04-24T23:22:10.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:22:10.316+0000 7f9f2747e700 1 --2- 172.21.15.148:0/1033367474 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9f20108010 0x7f9f2010a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:10.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:22:10.316+0000 7f9f2747e700 1 --2- 172.21.15.148:0/1033367474 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f20102db0 0x7f9f201051a0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:10.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:22:10.316+0000 7f9f2747e700 1 --2- 172.21.15.148:0/1033367474 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9f201056e0 0x7f9f20107ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:10.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:22:10.316+0000 7f9f2747e700 1 -- 172.21.15.148:0/1033367474 >> 172.21.15.148:0/1033367474 conn(0x7f9f200fc720 msgr2=0x7f9f200feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:10.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:22:10.316+0000 7f9f2747e700 1 -- 172.21.15.148:0/1033367474 shutdown_connections 2024-04-24T23:22:10.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:22:10.316+0000 7f9f2747e700 1 -- 172.21.15.148:0/1033367474 wait complete. 2024-04-24T23:22:10.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:22:10.317+0000 7f9f2747e700 1 Processor -- start 2024-04-24T23:22:10.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:22:10.317+0000 7f9f2747e700 1 -- start start 2024-04-24T23:22:10.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:22:10.317+0000 7f9f2747e700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9f20102db0 0x7f9f20102710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:10.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:22:10.317+0000 7f9f2747e700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9f201056e0 0x7f9f20100d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:10.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:22:10.317+0000 7f9f2747e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f20108010 0x7f9f201012a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:10.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:22:10.317+0000 7f9f2747e700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f9f2010de60 con 0x7f9f20102db0 2024-04-24T23:22:10.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:22:10.317+0000 7f9f2747e700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9f2010dce0 con 0x7f9f201056e0 2024-04-24T23:22:10.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:22:10.317+0000 7f9f2747e700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f9f2010dfe0 con 0x7f9f20108010 2024-04-24T23:22:10.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:22:10.317+0000 7f9f25a1b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f20108010 0x7f9f201012a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:10.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:22:10.317+0000 7f9f25a1b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f20108010 0x7f9f201012a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:44620/0 (socket says 172.21.15.148:44620) 2024-04-24T23:22:10.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:22:10.317+0000 7f9f25a1b700 1 -- 172.21.15.148:0/1923780880 learned_addr learned my addr 172.21.15.148:0/1923780880 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-24T23:22:10.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:22:10.317+0000 7f9f24a19700 1 --2- 172.21.15.148:0/1923780880 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9f201056e0 0x7f9f20100d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:10.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:22:10.317+0000 7f9f2521a700 1 --2- 172.21.15.148:0/1923780880 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9f20102db0 0x7f9f20102710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:10.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:22:10.317+0000 7f9f25a1b700 1 -- 172.21.15.148:0/1923780880 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9f201056e0 msgr2=0x7f9f20100d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:10.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:22:10.317+0000 7f9f25a1b700 1 --2- 172.21.15.148:0/1923780880 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9f201056e0 0x7f9f20100d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:10.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:22:10.317+0000 7f9f25a1b700 1 -- 172.21.15.148:0/1923780880 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9f20102db0 msgr2=0x7f9f20102710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:10.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:22:10.317+0000 7f9f25a1b700 1 --2- 172.21.15.148:0/1923780880 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9f20102db0 0x7f9f20102710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:10.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:22:10.317+0000 7f9f25a1b700 1 -- 172.21.15.148:0/1923780880 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f20101b50 con 0x7f9f20108010 2024-04-24T23:22:10.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:22:10.318+0000 7f9f25a1b700 1 --2- 172.21.15.148:0/1923780880 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f20108010 0x7f9f201012a0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f9f1800ada0 tx=0x7f9f1800c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:10.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:22:10.318+0000 7f9f167fc700 1 -- 172.21.15.148:0/1923780880 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f18018070 con 0x7f9f20108010 2024-04-24T23:22:10.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:22:10.318+0000 7f9f167fc700 1 -- 172.21.15.148:0/1923780880 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f1800a0d0 con 0x7f9f20108010 2024-04-24T23:22:10.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:22:10.318+0000 7f9f2747e700 1 -- 172.21.15.148:0/1923780880 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f201a2600 con 0x7f9f20108010 2024-04-24T23:22:10.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:22:10.318+0000 7f9f167fc700 1 -- 172.21.15.148:0/1923780880 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f18004290 con 0x7f9f20108010 2024-04-24T23:22:10.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:22:10.318+0000 7f9f2747e700 1 -- 172.21.15.148:0/1923780880 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f201a2b40 con 0x7f9f20108010 2024-04-24T23:22:10.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:22:10.319+0000 7f9f2747e700 1 -- 172.21.15.148:0/1923780880 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f20068f20 con 0x7f9f20108010 2024-04-24T23:22:10.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:22:10.320+0000 7f9f167fc700 1 -- 172.21.15.148:0/1923780880 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7f9f1801b070 con 0x7f9f20108010 2024-04-24T23:22:10.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:22:10.320+0000 7f9f167fc700 1 --2- 172.21.15.148:0/1923780880 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f9f0c0a2c10 0x7f9f0c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:10.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:22:10.321+0000 7f9f167fc700 1 -- 172.21.15.148:0/1923780880 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7f9f180cc750 con 0x7f9f20108010 2024-04-24T23:22:10.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:22:10.321+0000 7f9f2521a700 1 --2- 172.21.15.148:0/1923780880 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f9f0c0a2c10 0x7f9f0c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:10.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:22:10.321+0000 7f9f2521a700 1 --2- 172.21.15.148:0/1923780880 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f9f0c0a2c10 0x7f9f0c0a50d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c00b5c0 tx=0x7f9f1c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:10.326 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:22:10.325+0000 7f9f167fc700 1 -- 172.21.15.148:0/1923780880 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f9f1804f740 con 0x7f9f20108010 2024-04-24T23:22:10.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:10 smithi134 ceph-mon[32619]: Detected new or changed devices on smithi134 2024-04-24T23:22:10.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:10 smithi134 ceph-mon[32619]: Adjusting osd_memory_target on smithi134 to 17088M 2024-04-24T23:22:10.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:10 smithi134 ceph-mon[32619]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:22:10.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:22:10.526+0000 7f9f2747e700 1 -- 172.21.15.148:0/1923780880 --> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f9f20062670 con 0x7f9f0c0a2c10 2024-04-24T23:22:11.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:11 smithi080 ceph-mon[29152]: osdmap e14: 2 total, 2 up, 2 in 2024-04-24T23:22:11.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:11 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:22:11.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:11 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:22:11.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:11 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:22:11.343 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:11 smithi148 ceph-mon[32528]: osdmap e14: 2 total, 2 up, 2 in 2024-04-24T23:22:11.343 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:11 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:22:11.343 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:11 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:22:11.343 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:11 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:22:11.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:11 smithi134 ceph-mon[32619]: osdmap e14: 2 total, 2 up, 2 in 2024-04-24T23:22:11.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:11 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:22:11.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:11 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:22:11.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:11 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:22:12.237 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:12 smithi148 ceph-mon[32528]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:22:12.237 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:12 smithi148 ceph-mon[32528]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:22:12.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:12 smithi080 ceph-mon[29152]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:22:12.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:12 smithi080 ceph-mon[29152]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:22:12.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:12 smithi134 ceph-mon[32619]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:22:12.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:12 smithi134 ceph-mon[32619]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:22:14.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:14 smithi080 ceph-mon[29152]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:22:14.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:14 smithi080 ceph-mon[29152]: from='client.? 172.21.15.148:0/804761619' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "781fb879-fedc-4c84-82fb-59e073bc4cf5"}]: dispatch 2024-04-24T23:22:14.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:14 smithi080 ceph-mon[29152]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "781fb879-fedc-4c84-82fb-59e073bc4cf5"}]: dispatch 2024-04-24T23:22:14.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:14 smithi080 ceph-mon[29152]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "781fb879-fedc-4c84-82fb-59e073bc4cf5"}]': finished 2024-04-24T23:22:14.290 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:14 smithi080 ceph-mon[29152]: osdmap e15: 3 total, 2 up, 3 in 2024-04-24T23:22:14.290 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:14 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T23:22:14.290 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:14 smithi080 ceph-mon[29152]: from='client.? 172.21.15.148:0/2036413789' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:22:14.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:14 smithi134 ceph-mon[32619]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:22:14.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:14 smithi134 ceph-mon[32619]: from='client.? 172.21.15.148:0/804761619' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "781fb879-fedc-4c84-82fb-59e073bc4cf5"}]: dispatch 2024-04-24T23:22:14.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:14 smithi134 ceph-mon[32619]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "781fb879-fedc-4c84-82fb-59e073bc4cf5"}]: dispatch 2024-04-24T23:22:14.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:14 smithi134 ceph-mon[32619]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "781fb879-fedc-4c84-82fb-59e073bc4cf5"}]': finished 2024-04-24T23:22:14.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:14 smithi134 ceph-mon[32619]: osdmap e15: 3 total, 2 up, 3 in 2024-04-24T23:22:14.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:14 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T23:22:14.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:14 smithi134 ceph-mon[32619]: from='client.? 172.21.15.148:0/2036413789' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:22:14.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:14 smithi148 ceph-mon[32528]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:22:14.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:14 smithi148 ceph-mon[32528]: from='client.? 172.21.15.148:0/804761619' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "781fb879-fedc-4c84-82fb-59e073bc4cf5"}]: dispatch 2024-04-24T23:22:14.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:14 smithi148 ceph-mon[32528]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "781fb879-fedc-4c84-82fb-59e073bc4cf5"}]: dispatch 2024-04-24T23:22:14.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:14 smithi148 ceph-mon[32528]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "781fb879-fedc-4c84-82fb-59e073bc4cf5"}]': finished 2024-04-24T23:22:14.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:14 smithi148 ceph-mon[32528]: osdmap e15: 3 total, 2 up, 3 in 2024-04-24T23:22:14.508 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:14 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T23:22:14.508 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:14 smithi148 ceph-mon[32528]: from='client.? 172.21.15.148:0/2036413789' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:22:16.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:16 smithi080 ceph-mon[29152]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:22:16.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:16 smithi134 ceph-mon[32619]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:22:16.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:16 smithi148 ceph-mon[32528]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:22:18.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:18 smithi080 ceph-mon[29152]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:22:18.348 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:18 smithi148 ceph-mon[32528]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:22:18.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:18 smithi134 ceph-mon[32619]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:22:20.325 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:20 smithi148 ceph-mon[32528]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:22:20.436 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:20 smithi134 ceph-mon[32619]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:22:20.539 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:20 smithi080 ceph-mon[29152]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:22:21.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:21 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-24T23:22:21.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:21 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:22:21.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:21 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-24T23:22:21.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:21 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:22:21.539 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:21 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-24T23:22:21.539 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:21 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:22:22.165 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:22 smithi148 ceph-mon[32528]: Deploying daemon osd.2 on smithi148 2024-04-24T23:22:22.165 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:22 smithi148 ceph-mon[32528]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:22:22.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:22 smithi134 ceph-mon[32619]: Deploying daemon osd.2 on smithi148 2024-04-24T23:22:22.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:22 smithi134 ceph-mon[32619]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:22:22.539 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:22 smithi080 ceph-mon[29152]: Deploying daemon osd.2 on smithi148 2024-04-24T23:22:22.539 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:22 smithi080 ceph-mon[29152]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:22:24.134 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:24 smithi148 ceph-mon[32528]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:22:24.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:24 smithi134 ceph-mon[32619]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:22:24.539 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:24 smithi080 ceph-mon[29152]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:22:26.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:26 smithi134 ceph-mon[32619]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:22:26.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:26 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:22:26.449 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:26 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:22:26.449 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:26 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:22:26.466 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:26 smithi148 ceph-mon[32528]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:22:26.467 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:26 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:22:26.467 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:26 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:22:26.467 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:26 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:22:26.539 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:26 smithi080 ceph-mon[29152]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:22:26.539 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:26 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:22:26.539 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:26 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:22:26.539 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:26 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:22:28.341 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:28 smithi080 ceph-mon[29152]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:22:28.385 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:28 smithi148 ceph-mon[32528]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:22:28.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:28 smithi134 ceph-mon[32619]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:22:29.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:22:29.391+0000 7f9f167fc700 1 -- 172.21.15.148:0/1923780880 <== mgr.14152 v2:172.21.15.80:6800/3830596922 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f9f20062670 con 0x7f9f0c0a2c10 2024-04-24T23:22:29.395 INFO:teuthology.orchestra.run.smithi148.stdout:Created osd(s) 2 on host 'smithi148' 2024-04-24T23:22:29.395 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:29 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:22:29.395 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:29 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:22:29.396 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:29 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:22:29.396 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:29 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:22:29.396 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:29 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:22:29.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:22:29.394+0000 7f9f2747e700 1 -- 172.21.15.148:0/1923780880 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f9f0c0a2c10 msgr2=0x7f9f0c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:29.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:22:29.394+0000 7f9f2747e700 1 --2- 172.21.15.148:0/1923780880 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f9f0c0a2c10 0x7f9f0c0a50d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c00b5c0 tx=0x7f9f1c005fb0 comp rx=0 tx=0).stop 2024-04-24T23:22:29.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:22:29.394+0000 7f9f2747e700 1 -- 172.21.15.148:0/1923780880 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f20108010 msgr2=0x7f9f201012a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:29.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:22:29.394+0000 7f9f2747e700 1 --2- 172.21.15.148:0/1923780880 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f20108010 0x7f9f201012a0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f9f1800ada0 tx=0x7f9f1800c330 comp rx=0 tx=0).stop 2024-04-24T23:22:29.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:22:29.394+0000 7f9f2747e700 1 -- 172.21.15.148:0/1923780880 shutdown_connections 2024-04-24T23:22:29.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:22:29.394+0000 7f9f2747e700 1 --2- 172.21.15.148:0/1923780880 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f20108010 0x7f9f201012a0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:29.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:22:29.394+0000 7f9f2747e700 1 --2- 172.21.15.148:0/1923780880 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f9f0c0a2c10 0x7f9f0c0a50d0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:29.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:22:29.394+0000 7f9f2747e700 1 --2- 172.21.15.148:0/1923780880 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9f20102db0 0x7f9f20102710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:29.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:22:29.394+0000 7f9f2747e700 1 --2- 172.21.15.148:0/1923780880 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9f201056e0 0x7f9f20100d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:29.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:22:29.394+0000 7f9f2747e700 1 -- 172.21.15.148:0/1923780880 >> 172.21.15.148:0/1923780880 conn(0x7f9f200fc720 msgr2=0x7f9f201037b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:29.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:22:29.394+0000 7f9f2747e700 1 -- 172.21.15.148:0/1923780880 shutdown_connections 2024-04-24T23:22:29.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:22:29.394+0000 7f9f2747e700 1 -- 172.21.15.148:0/1923780880 wait complete. 2024-04-24T23:22:29.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:29 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:22:29.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:29 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:22:29.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:29 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:22:29.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:29 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:22:29.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:29 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:22:29.539 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:29 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:22:29.539 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:29 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:22:29.539 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:29 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:22:29.539 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:29 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:22:29.539 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:29 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:22:30.034 DEBUG:teuthology.orchestra.run.smithi148:osd.2> sudo journalctl -f -n 0 -u ceph-ec7885aa-0290-11ef-bc93-c7b262605968@osd.2.service 2024-04-24T23:22:30.036 INFO:tasks.cephadm:Deploying osd.3 on smithi080 with /dev/vg_nvme/lv_3... 2024-04-24T23:22:30.037 DEBUG:teuthology.orchestra.run.smithi080:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ec7885aa-0290-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-24T23:22:30.287 INFO:teuthology.orchestra.run.smithi080.stderr:Inferring config /var/lib/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:22:30.311 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:30 smithi148 ceph-mon[32528]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:22:30.311 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:30 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:22:30.311 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:30 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:22:30.311 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:30 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:22:30.312 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:30 smithi148 ceph-mon[32528]: from='osd.2 [v2:172.21.15.148:6800/1629152187,v1:172.21.15.148:6801/1629152187]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-24T23:22:30.312 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:30 smithi148 ceph-mon[32528]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-24T23:22:30.409 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:30 smithi080 ceph-mon[29152]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:22:30.410 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:30 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:22:30.410 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:30 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:22:30.410 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:30 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:22:30.410 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:30 smithi080 ceph-mon[29152]: from='osd.2 [v2:172.21.15.148:6800/1629152187,v1:172.21.15.148:6801/1629152187]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-24T23:22:30.410 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:30 smithi080 ceph-mon[29152]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-24T23:22:30.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:30 smithi134 ceph-mon[32619]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:22:30.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:30 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:22:30.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:30 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:22:30.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:30 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:22:30.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:30 smithi134 ceph-mon[32619]: from='osd.2 [v2:172.21.15.148:6800/1629152187,v1:172.21.15.148:6801/1629152187]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-24T23:22:30.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:30 smithi134 ceph-mon[32619]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-24T23:22:31.498 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:31 smithi148 ceph-mon[32528]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-24T23:22:31.498 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:31 smithi148 ceph-mon[32528]: osdmap e16: 3 total, 2 up, 3 in 2024-04-24T23:22:31.498 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:31 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T23:22:31.498 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:31 smithi148 ceph-mon[32528]: from='osd.2 [v2:172.21.15.148:6800/1629152187,v1:172.21.15.148:6801/1629152187]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-04-24T23:22:31.498 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:31 smithi148 ceph-mon[32528]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-04-24T23:22:31.498 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:31 smithi148 ceph-mon[32528]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:22:31.698 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:31 smithi134 ceph-mon[32619]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-24T23:22:31.698 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:31 smithi134 ceph-mon[32619]: osdmap e16: 3 total, 2 up, 3 in 2024-04-24T23:22:31.698 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:31 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T23:22:31.698 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:31 smithi134 ceph-mon[32619]: from='osd.2 [v2:172.21.15.148:6800/1629152187,v1:172.21.15.148:6801/1629152187]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-04-24T23:22:31.698 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:31 smithi134 ceph-mon[32619]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-04-24T23:22:31.698 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:31 smithi134 ceph-mon[32619]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:22:31.758 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:31 smithi080 ceph-mon[29152]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-24T23:22:31.758 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:31 smithi080 ceph-mon[29152]: osdmap e16: 3 total, 2 up, 3 in 2024-04-24T23:22:31.758 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:31 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T23:22:31.758 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:31 smithi080 ceph-mon[29152]: from='osd.2 [v2:172.21.15.148:6800/1629152187,v1:172.21.15.148:6801/1629152187]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-04-24T23:22:31.759 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:31 smithi080 ceph-mon[29152]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-04-24T23:22:31.759 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:31 smithi080 ceph-mon[29152]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:22:32.007 INFO:journalctl@ceph.osd.2.smithi148.stdout:Apr 24 23:22:31 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-2[37716]: 2024-04-24T23:22:31.819+0000 7f58260d4700 -1 osd.2 0 waiting for initial osdmap 2024-04-24T23:22:32.008 INFO:journalctl@ceph.osd.2.smithi148.stdout:Apr 24 23:22:31 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-2[37716]: 2024-04-24T23:22:31.824+0000 7f581debf700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T23:22:32.353 INFO:teuthology.orchestra.run.smithi080.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-24T23:22:32.353 INFO:teuthology.orchestra.run.smithi080.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-24T23:22:32.353 INFO:teuthology.orchestra.run.smithi080.stderr: stderr: 10+0 records in 2024-04-24T23:22:32.353 INFO:teuthology.orchestra.run.smithi080.stderr:10+0 records out 2024-04-24T23:22:32.353 INFO:teuthology.orchestra.run.smithi080.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0131569 s, 797 MB/s 2024-04-24T23:22:32.353 INFO:teuthology.orchestra.run.smithi080.stderr:--> Zapping successful for: 2024-04-24T23:22:32.653 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:32 smithi080 ceph-mon[29152]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2024-04-24T23:22:32.653 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:32 smithi080 ceph-mon[29152]: osdmap e17: 3 total, 2 up, 3 in 2024-04-24T23:22:32.653 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:32 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T23:22:32.653 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:32 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T23:22:32.653 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:32 smithi080 ceph-mon[29152]: from='osd.2 ' entity='osd.2' 2024-04-24T23:22:32.672 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:32 smithi148 ceph-mon[32528]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2024-04-24T23:22:32.672 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:32 smithi148 ceph-mon[32528]: osdmap e17: 3 total, 2 up, 3 in 2024-04-24T23:22:32.672 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:32 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T23:22:32.672 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:32 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T23:22:32.672 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:32 smithi148 ceph-mon[32528]: from='osd.2 ' entity='osd.2' 2024-04-24T23:22:32.698 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:32 smithi134 ceph-mon[32619]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2024-04-24T23:22:32.698 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:32 smithi134 ceph-mon[32619]: osdmap e17: 3 total, 2 up, 3 in 2024-04-24T23:22:32.698 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:32 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T23:22:32.698 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:32 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T23:22:32.698 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:32 smithi134 ceph-mon[32619]: from='osd.2 ' entity='osd.2' 2024-04-24T23:22:32.995 DEBUG:teuthology.orchestra.run.smithi080:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ec7885aa-0290-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi080:vg_nvme/lv_3 2024-04-24T23:22:33.292 INFO:teuthology.orchestra.run.smithi080.stderr:Inferring config /var/lib/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:22:33.685 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:33 smithi080 ceph-mon[29152]: purged_snaps scrub starts 2024-04-24T23:22:33.686 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:33 smithi080 ceph-mon[29152]: purged_snaps scrub ok 2024-04-24T23:22:33.686 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:33 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T23:22:33.686 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:33 smithi080 ceph-mon[29152]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:22:33.686 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:33 smithi080 ceph-mon[29152]: osd.2 [v2:172.21.15.148:6800/1629152187,v1:172.21.15.148:6801/1629152187] boot 2024-04-24T23:22:33.686 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:33 smithi080 ceph-mon[29152]: osdmap e18: 3 total, 3 up, 3 in 2024-04-24T23:22:33.686 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:33 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T23:22:33.686 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:33 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-24T23:22:33.686 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:33 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:22:33.690 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:33 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:22:33.690 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:33 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:22:33.690 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:33 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:22:33.690 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:33 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:22:33.690 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:33 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:22:33.690 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:33 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:22:33.698 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:33 smithi134 ceph-mon[32619]: purged_snaps scrub starts 2024-04-24T23:22:33.698 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:33 smithi134 ceph-mon[32619]: purged_snaps scrub ok 2024-04-24T23:22:33.698 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:33 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T23:22:33.698 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:33 smithi134 ceph-mon[32619]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:22:33.698 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:33 smithi134 ceph-mon[32619]: osd.2 [v2:172.21.15.148:6800/1629152187,v1:172.21.15.148:6801/1629152187] boot 2024-04-24T23:22:33.698 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:33 smithi134 ceph-mon[32619]: osdmap e18: 3 total, 3 up, 3 in 2024-04-24T23:22:33.698 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:33 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T23:22:33.698 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:33 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-24T23:22:33.698 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:33 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:22:33.699 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:33 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:22:33.699 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:33 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:22:33.699 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:33 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:22:33.699 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:33 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:22:33.699 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:33 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:22:33.699 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:33 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:22:33.757 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:33 smithi148 ceph-mon[32528]: purged_snaps scrub starts 2024-04-24T23:22:33.757 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:33 smithi148 ceph-mon[32528]: purged_snaps scrub ok 2024-04-24T23:22:33.757 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:33 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T23:22:33.757 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:33 smithi148 ceph-mon[32528]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:22:33.757 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:33 smithi148 ceph-mon[32528]: osd.2 [v2:172.21.15.148:6800/1629152187,v1:172.21.15.148:6801/1629152187] boot 2024-04-24T23:22:33.757 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:33 smithi148 ceph-mon[32528]: osdmap e18: 3 total, 3 up, 3 in 2024-04-24T23:22:33.758 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:33 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T23:22:33.758 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:33 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-24T23:22:33.758 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:33 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:22:33.758 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:33 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:22:33.758 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:33 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:22:33.758 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:33 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:22:33.758 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:33 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:22:33.758 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:33 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:22:33.758 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:33 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:22:34.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:22:34.922+0000 7f978f2bd700 1 -- 172.21.15.80:0/3579800272 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9788100900 msgr2=0x7f9788100d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:34.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:22:34.922+0000 7f978f2bd700 1 --2- 172.21.15.80:0/3579800272 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9788100900 0x7f9788100d20 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f978400aad0 tx=0x7f978400ade0 comp rx=0 tx=0).stop 2024-04-24T23:22:34.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:22:34.922+0000 7f978f2bd700 1 -- 172.21.15.80:0/3579800272 shutdown_connections 2024-04-24T23:22:34.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:22:34.922+0000 7f978f2bd700 1 --2- 172.21.15.80:0/3579800272 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97881024a0 0x7f9788108d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:34.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:22:34.922+0000 7f978f2bd700 1 --2- 172.21.15.80:0/3579800272 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9788100900 0x7f9788100d20 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:34.937 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:22:34.922+0000 7f978f2bd700 1 --2- 172.21.15.80:0/3579800272 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9788101b00 0x7f9788101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:34.937 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:22:34.922+0000 7f978f2bd700 1 -- 172.21.15.80:0/3579800272 >> 172.21.15.80:0/3579800272 conn(0x7f97880fbec0 msgr2=0x7f97880fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:34.937 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:22:34.922+0000 7f978f2bd700 1 -- 172.21.15.80:0/3579800272 shutdown_connections 2024-04-24T23:22:34.937 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:22:34.922+0000 7f978f2bd700 1 -- 172.21.15.80:0/3579800272 wait complete. 2024-04-24T23:22:34.937 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:22:34.923+0000 7f978f2bd700 1 Processor -- start 2024-04-24T23:22:34.937 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:22:34.923+0000 7f978f2bd700 1 -- start start 2024-04-24T23:22:34.937 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:22:34.923+0000 7f978f2bd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9788100900 0x7f97881984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:34.937 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:22:34.923+0000 7f978f2bd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9788101b00 0x7f97881989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:34.938 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:22:34.923+0000 7f978f2bd700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f97881024a0 0x7f978819fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:34.938 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:22:34.923+0000 7f978f2bd700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f978810bc30 con 0x7f97881024a0 2024-04-24T23:22:34.938 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:22:34.923+0000 7f978f2bd700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f978810bab0 con 0x7f9788100900 2024-04-24T23:22:34.938 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:22:34.923+0000 7f978f2bd700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f978810bdb0 con 0x7f9788101b00 2024-04-24T23:22:34.938 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:22:34.923+0000 7f978d85a700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f97881024a0 0x7f978819fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:34.938 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:22:34.923+0000 7f978c858700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9788101b00 0x7f97881989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:34.939 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:22:34.923+0000 7f978d85a700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f97881024a0 0x7f978819fa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53318/0 (socket says 172.21.15.80:53318) 2024-04-24T23:22:34.939 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:22:34.923+0000 7f978d85a700 1 -- 172.21.15.80:0/3498659600 learned_addr learned my addr 172.21.15.80:0/3498659600 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:22:34.939 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:22:34.923+0000 7f978d059700 1 --2- 172.21.15.80:0/3498659600 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9788100900 0x7f97881984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:34.939 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:22:34.924+0000 7f978d85a700 1 -- 172.21.15.80:0/3498659600 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9788101b00 msgr2=0x7f97881989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:34.939 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:22:34.924+0000 7f978d85a700 1 --2- 172.21.15.80:0/3498659600 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9788101b00 0x7f97881989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:34.939 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:22:34.924+0000 7f978d85a700 1 -- 172.21.15.80:0/3498659600 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9788100900 msgr2=0x7f97881984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:34.939 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:22:34.924+0000 7f978d85a700 1 --2- 172.21.15.80:0/3498659600 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9788100900 0x7f97881984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:34.940 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:22:34.924+0000 7f978d85a700 1 -- 172.21.15.80:0/3498659600 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97881a0110 con 0x7f97881024a0 2024-04-24T23:22:34.940 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:22:34.924+0000 7f978c858700 1 --2- 172.21.15.80:0/3498659600 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9788101b00 0x7f97881989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:22:34.940 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:22:34.924+0000 7f978d85a700 1 --2- 172.21.15.80:0/3498659600 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f97881024a0 0x7f978819fa70 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f9780007f10 tx=0x7f978000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:34.940 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:22:34.924+0000 7f977e7fc700 1 -- 172.21.15.80:0/3498659600 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f978000dd20 con 0x7f97881024a0 2024-04-24T23:22:34.940 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:22:34.924+0000 7f978f2bd700 1 -- 172.21.15.80:0/3498659600 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97881a0400 con 0x7f97881024a0 2024-04-24T23:22:34.940 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:22:34.924+0000 7f977e7fc700 1 -- 172.21.15.80:0/3498659600 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9780004020 con 0x7f97881024a0 2024-04-24T23:22:34.940 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:22:34.924+0000 7f977e7fc700 1 -- 172.21.15.80:0/3498659600 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9780015660 con 0x7f97881024a0 2024-04-24T23:22:34.940 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:22:34.924+0000 7f978f2bd700 1 -- 172.21.15.80:0/3498659600 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97881a0940 con 0x7f97881024a0 2024-04-24T23:22:34.941 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:22:34.925+0000 7f978f2bd700 1 -- 172.21.15.80:0/3498659600 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9788068f20 con 0x7f97881024a0 2024-04-24T23:22:34.941 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:22:34.927+0000 7f977e7fc700 1 -- 172.21.15.80:0/3498659600 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7f97800041d0 con 0x7f97881024a0 2024-04-24T23:22:34.941 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:22:34.927+0000 7f977e7fc700 1 --2- 172.21.15.80:0/3498659600 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f97740a2b90 0x7f97740a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:34.941 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:22:34.927+0000 7f978d059700 1 --2- 172.21.15.80:0/3498659600 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f97740a2b90 0x7f97740a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:34.941 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:22:34.928+0000 7f977e7fc700 1 -- 172.21.15.80:0/3498659600 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 3237+0+0 (secure 0 0 0) 0x7f97800cddb0 con 0x7f97881024a0 2024-04-24T23:22:34.941 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:22:34.928+0000 7f978d059700 1 --2- 172.21.15.80:0/3498659600 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f97740a2b90 0x7f97740a5050 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f978400c690 tx=0x7f9784005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:34.942 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:22:34.931+0000 7f977e7fc700 1 -- 172.21.15.80:0/3498659600 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f9780010070 con 0x7f97881024a0 2024-04-24T23:22:34.942 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:34 smithi080 ceph-mon[29152]: Detected new or changed devices on smithi148 2024-04-24T23:22:34.942 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:34 smithi080 ceph-mon[29152]: Adjusting osd_memory_target on smithi148 to 17088M 2024-04-24T23:22:34.943 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:34 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-24T23:22:34.943 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:34 smithi080 ceph-mon[29152]: osdmap e19: 3 total, 3 up, 3 in 2024-04-24T23:22:34.944 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:34 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-24T23:22:35.145 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:22:35.144+0000 7f978f2bd700 1 -- 172.21.15.80:0/3498659600 --> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi080:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f9788108750 con 0x7f97740a2b90 2024-04-24T23:22:35.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:34 smithi134 ceph-mon[32619]: Detected new or changed devices on smithi148 2024-04-24T23:22:35.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:34 smithi134 ceph-mon[32619]: Adjusting osd_memory_target on smithi148 to 17088M 2024-04-24T23:22:35.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:34 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-24T23:22:35.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:34 smithi134 ceph-mon[32619]: osdmap e19: 3 total, 3 up, 3 in 2024-04-24T23:22:35.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:34 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-24T23:22:35.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:34 smithi148 ceph-mon[32528]: Detected new or changed devices on smithi148 2024-04-24T23:22:35.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:34 smithi148 ceph-mon[32528]: Adjusting osd_memory_target on smithi148 to 17088M 2024-04-24T23:22:35.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:34 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-24T23:22:35.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:34 smithi148 ceph-mon[32528]: osdmap e19: 3 total, 3 up, 3 in 2024-04-24T23:22:35.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:34 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-24T23:22:35.460 INFO:journalctl@ceph.osd.0.smithi080.stdout:Apr 24 23:22:35 smithi080 sudo[48268]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-24T23:22:35.461 INFO:journalctl@ceph.osd.0.smithi080.stdout:Apr 24 23:22:35 smithi080 sudo[48268]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T23:22:35.461 INFO:journalctl@ceph.osd.0.smithi080.stdout:Apr 24 23:22:35 smithi080 sudo[48268]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T23:22:35.461 INFO:journalctl@ceph.osd.0.smithi080.stdout:Apr 24 23:22:35 smithi080 sudo[48268]: pam_unix(sudo:session): session closed for user root 2024-04-24T23:22:35.767 INFO:journalctl@ceph.osd.0.smithi080.stdout:Apr 24 23:22:35 smithi080 sudo[48298]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-24T23:22:35.767 INFO:journalctl@ceph.osd.0.smithi080.stdout:Apr 24 23:22:35 smithi080 sudo[48298]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T23:22:35.767 INFO:journalctl@ceph.osd.0.smithi080.stdout:Apr 24 23:22:35 smithi080 sudo[48298]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T23:22:35.767 INFO:journalctl@ceph.osd.0.smithi080.stdout:Apr 24 23:22:35 smithi080 sudo[48298]: pam_unix(sudo:session): session closed for user root 2024-04-24T23:22:35.982 INFO:journalctl@ceph.osd.1.smithi134.stdout:Apr 24 23:22:35 smithi134 sudo[41388]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-24T23:22:35.982 INFO:journalctl@ceph.osd.1.smithi134.stdout:Apr 24 23:22:35 smithi134 sudo[41388]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T23:22:35.982 INFO:journalctl@ceph.osd.1.smithi134.stdout:Apr 24 23:22:35 smithi134 sudo[41388]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T23:22:35.982 INFO:journalctl@ceph.osd.1.smithi134.stdout:Apr 24 23:22:35 smithi134 sudo[41388]: pam_unix(sudo:session): session closed for user root 2024-04-24T23:22:35.983 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:35 smithi134 ceph-mon[32619]: pgmap v83: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:22:35.983 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:35 smithi134 ceph-mon[32619]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-24T23:22:35.983 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:35 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-24T23:22:35.983 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:35 smithi134 ceph-mon[32619]: osdmap e20: 3 total, 3 up, 3 in 2024-04-24T23:22:35.983 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:35 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:22:35.983 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:35 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:22:35.983 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:35 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:22:35.983 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:35 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:22:36.018 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:35 smithi080 ceph-mon[29152]: pgmap v83: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:22:36.018 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:35 smithi080 ceph-mon[29152]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-24T23:22:36.018 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:35 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-24T23:22:36.018 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:35 smithi080 ceph-mon[29152]: osdmap e20: 3 total, 3 up, 3 in 2024-04-24T23:22:36.018 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:35 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:22:36.018 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:35 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:22:36.018 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:35 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:22:36.018 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:35 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:22:36.158 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:35 smithi148 ceph-mon[32528]: pgmap v83: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:22:36.159 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:35 smithi148 ceph-mon[32528]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-24T23:22:36.159 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:35 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-24T23:22:36.159 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:35 smithi148 ceph-mon[32528]: osdmap e20: 3 total, 3 up, 3 in 2024-04-24T23:22:36.159 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:35 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:22:36.159 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:35 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:22:36.159 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:35 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:22:36.159 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:35 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:22:36.424 INFO:journalctl@ceph.osd.2.smithi148.stdout:Apr 24 23:22:36 smithi148 sudo[41781]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-24T23:22:36.424 INFO:journalctl@ceph.osd.2.smithi148.stdout:Apr 24 23:22:36 smithi148 sudo[41781]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T23:22:36.424 INFO:journalctl@ceph.osd.2.smithi148.stdout:Apr 24 23:22:36 smithi148 sudo[41781]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T23:22:36.424 INFO:journalctl@ceph.osd.2.smithi148.stdout:Apr 24 23:22:36 smithi148 sudo[41781]: pam_unix(sudo:session): session closed for user root 2024-04-24T23:22:36.448 INFO:journalctl@ceph.osd.1.smithi134.stdout:Apr 24 23:22:35 smithi134 sudo[41391]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-24T23:22:36.448 INFO:journalctl@ceph.osd.1.smithi134.stdout:Apr 24 23:22:35 smithi134 sudo[41391]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T23:22:36.448 INFO:journalctl@ceph.osd.1.smithi134.stdout:Apr 24 23:22:35 smithi134 sudo[41391]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T23:22:36.448 INFO:journalctl@ceph.osd.1.smithi134.stdout:Apr 24 23:22:35 smithi134 sudo[41391]: pam_unix(sudo:session): session closed for user root 2024-04-24T23:22:36.757 INFO:journalctl@ceph.osd.2.smithi148.stdout:Apr 24 23:22:36 smithi148 sudo[41787]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-24T23:22:36.757 INFO:journalctl@ceph.osd.2.smithi148.stdout:Apr 24 23:22:36 smithi148 sudo[41787]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T23:22:36.757 INFO:journalctl@ceph.osd.2.smithi148.stdout:Apr 24 23:22:36 smithi148 sudo[41787]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T23:22:36.757 INFO:journalctl@ceph.osd.2.smithi148.stdout:Apr 24 23:22:36 smithi148 sudo[41787]: pam_unix(sudo:session): session closed for user root 2024-04-24T23:22:37.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:36 smithi080 sudo[48391]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-24T23:22:37.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:36 smithi080 sudo[48391]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T23:22:37.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:36 smithi080 sudo[48391]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T23:22:37.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:36 smithi080 sudo[48391]: pam_unix(sudo:session): session closed for user root 2024-04-24T23:22:37.332 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:37 smithi148 ceph-mon[32528]: from='client.14277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi080:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:22:37.332 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:37 smithi148 ceph-mon[32528]: osdmap e21: 3 total, 3 up, 3 in 2024-04-24T23:22:37.332 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:37 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:22:37.332 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:37 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:22:37.332 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:37 smithi148 sudo[41797]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-24T23:22:37.332 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:37 smithi148 sudo[41797]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T23:22:37.332 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:37 smithi148 sudo[41797]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T23:22:37.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:37 smithi134 ceph-mon[32619]: from='client.14277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi080:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:22:37.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:37 smithi134 ceph-mon[32619]: osdmap e21: 3 total, 3 up, 3 in 2024-04-24T23:22:37.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:37 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:22:37.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:37 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:22:37.539 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:37 smithi080 sudo[48405]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-24T23:22:37.539 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:37 smithi080 sudo[48405]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T23:22:37.539 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:37 smithi080 sudo[48405]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T23:22:37.539 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:37 smithi080 sudo[48405]: pam_unix(sudo:session): session closed for user root 2024-04-24T23:22:37.539 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:37 smithi080 ceph-mon[29152]: from='client.14277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi080:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:22:37.539 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:37 smithi080 ceph-mon[29152]: osdmap e21: 3 total, 3 up, 3 in 2024-04-24T23:22:37.540 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:37 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:22:37.540 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:37 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:22:37.757 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:37 smithi148 sudo[41797]: pam_unix(sudo:session): session closed for user root 2024-04-24T23:22:37.757 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:37 smithi148 sudo[41801]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-24T23:22:37.757 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:37 smithi148 sudo[41801]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T23:22:37.757 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:37 smithi148 sudo[41801]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T23:22:37.757 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:37 smithi148 sudo[41801]: pam_unix(sudo:session): session closed for user root 2024-04-24T23:22:38.063 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:37 smithi134 sudo[41394]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-24T23:22:38.063 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:37 smithi134 sudo[41394]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T23:22:38.063 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:37 smithi134 sudo[41394]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T23:22:38.063 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:37 smithi134 sudo[41394]: pam_unix(sudo:session): session closed for user root 2024-04-24T23:22:38.073 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:22:38.072+0000 7f977e7fc700 1 -- 172.21.15.80:0/3498659600 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f9780098130 con 0x7f97881024a0 2024-04-24T23:22:38.442 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:38 smithi080 ceph-mon[29152]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-24T23:22:38.442 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:38 smithi080 ceph-mon[29152]: pgmap v86: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:22:38.442 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:38 smithi080 ceph-mon[29152]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-24T23:22:38.442 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:38 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T23:22:38.442 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:38 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:22:38.442 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:38 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:22:38.442 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:38 smithi080 ceph-mon[29152]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-24T23:22:38.442 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:38 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T23:22:38.442 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:38 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:22:38.443 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:38 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:22:38.443 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:38 smithi080 ceph-mon[29152]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-24T23:22:38.443 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:38 smithi080 ceph-mon[29152]: Cluster is now healthy 2024-04-24T23:22:38.443 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:38 smithi080 ceph-mon[29152]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-24T23:22:38.443 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:38 smithi080 ceph-mon[29152]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-24T23:22:38.443 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:38 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T23:22:38.443 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:38 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:22:38.443 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:38 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:22:38.443 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:38 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/1363834359' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cd982757-f2f6-4296-b3e7-4cd40fbd8d9b"}]: dispatch 2024-04-24T23:22:38.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:38 smithi134 sudo[41397]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-24T23:22:38.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:38 smithi134 sudo[41397]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T23:22:38.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:38 smithi134 sudo[41397]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T23:22:38.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:38 smithi134 sudo[41397]: pam_unix(sudo:session): session closed for user root 2024-04-24T23:22:38.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:38 smithi134 ceph-mon[32619]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-24T23:22:38.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:38 smithi134 ceph-mon[32619]: pgmap v86: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:22:38.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:38 smithi134 ceph-mon[32619]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-24T23:22:38.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:38 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T23:22:38.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:38 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:22:38.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:38 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:22:38.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:38 smithi134 ceph-mon[32619]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-24T23:22:38.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:38 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T23:22:38.449 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:38 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:22:38.449 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:38 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:22:38.449 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:38 smithi134 ceph-mon[32619]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-24T23:22:38.449 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:38 smithi134 ceph-mon[32619]: Cluster is now healthy 2024-04-24T23:22:38.449 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:38 smithi134 ceph-mon[32619]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-24T23:22:38.449 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:38 smithi134 ceph-mon[32619]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-24T23:22:38.449 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:38 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T23:22:38.449 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:38 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:22:38.449 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:38 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:22:38.449 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:38 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/1363834359' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cd982757-f2f6-4296-b3e7-4cd40fbd8d9b"}]: dispatch 2024-04-24T23:22:38.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:38 smithi148 ceph-mon[32528]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-24T23:22:38.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:38 smithi148 ceph-mon[32528]: pgmap v86: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:22:38.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:38 smithi148 ceph-mon[32528]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-24T23:22:38.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:38 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T23:22:38.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:38 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:22:38.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:38 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:22:38.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:38 smithi148 ceph-mon[32528]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-24T23:22:38.508 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:38 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T23:22:38.508 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:38 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:22:38.508 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:38 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:22:38.508 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:38 smithi148 ceph-mon[32528]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-24T23:22:38.508 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:38 smithi148 ceph-mon[32528]: Cluster is now healthy 2024-04-24T23:22:38.508 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:38 smithi148 ceph-mon[32528]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-24T23:22:38.508 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:38 smithi148 ceph-mon[32528]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-24T23:22:38.508 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:38 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T23:22:38.508 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:38 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:22:38.508 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:38 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:22:38.508 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:38 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/1363834359' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cd982757-f2f6-4296-b3e7-4cd40fbd8d9b"}]: dispatch 2024-04-24T23:22:39.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:39 smithi134 ceph-mon[32619]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-24T23:22:39.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:39 smithi134 ceph-mon[32619]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-24T23:22:39.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:39 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/1363834359' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cd982757-f2f6-4296-b3e7-4cd40fbd8d9b"}]': finished 2024-04-24T23:22:39.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:39 smithi134 ceph-mon[32619]: osdmap e22: 4 total, 3 up, 4 in 2024-04-24T23:22:39.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:39 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T23:22:39.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:39 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/3399524688' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:22:39.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:39 smithi148 ceph-mon[32528]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-24T23:22:39.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:39 smithi148 ceph-mon[32528]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-24T23:22:39.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:39 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/1363834359' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cd982757-f2f6-4296-b3e7-4cd40fbd8d9b"}]': finished 2024-04-24T23:22:39.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:39 smithi148 ceph-mon[32528]: osdmap e22: 4 total, 3 up, 4 in 2024-04-24T23:22:39.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:39 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T23:22:39.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:39 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/3399524688' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:22:39.539 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:39 smithi080 ceph-mon[29152]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-24T23:22:39.539 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:39 smithi080 ceph-mon[29152]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-24T23:22:39.539 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:39 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/1363834359' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cd982757-f2f6-4296-b3e7-4cd40fbd8d9b"}]': finished 2024-04-24T23:22:39.539 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:39 smithi080 ceph-mon[29152]: osdmap e22: 4 total, 3 up, 4 in 2024-04-24T23:22:39.539 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:39 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T23:22:39.539 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:39 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/3399524688' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:22:40.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:40 smithi134 ceph-mon[32619]: pgmap v88: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:22:40.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:40 smithi148 ceph-mon[32528]: pgmap v88: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:22:40.539 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:40 smithi080 ceph-mon[29152]: pgmap v88: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:22:42.414 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:42 smithi080 ceph-mon[29152]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:22:42.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:42 smithi134 ceph-mon[32619]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:22:42.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:42 smithi148 ceph-mon[32528]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:22:44.346 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:44 smithi080 ceph-mon[29152]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:22:44.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:44 smithi134 ceph-mon[32619]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:22:44.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:44 smithi148 ceph-mon[32528]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:22:46.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:46 smithi080 ceph-mon[29152]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:22:46.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:46 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-24T23:22:46.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:46 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:22:46.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:46 smithi134 ceph-mon[32619]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:22:46.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:46 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-24T23:22:46.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:46 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:22:46.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:46 smithi148 ceph-mon[32528]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:22:46.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:46 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-24T23:22:46.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:46 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:22:47.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:47 smithi148 ceph-mon[32528]: Deploying daemon osd.3 on smithi080 2024-04-24T23:22:47.257 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:47 smithi080 ceph-mon[29152]: Deploying daemon osd.3 on smithi080 2024-04-24T23:22:47.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:47 smithi134 ceph-mon[32619]: Deploying daemon osd.3 on smithi080 2024-04-24T23:22:48.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:48 smithi134 ceph-mon[32619]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:22:48.502 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:48 smithi080 ceph-mon[29152]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:22:48.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:48 smithi148 ceph-mon[32528]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:22:50.234 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:50 smithi080 ceph-mon[29152]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:22:50.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:50 smithi134 ceph-mon[32619]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:22:50.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:50 smithi148 ceph-mon[32528]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:22:52.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:51 smithi134 ceph-mon[32619]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:22:52.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:51 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:22:52.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:51 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:22:52.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:51 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:22:52.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:51 smithi148 ceph-mon[32528]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:22:52.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:51 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:22:52.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:51 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:22:52.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:51 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:22:52.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:51 smithi080 ceph-mon[29152]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:22:52.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:51 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:22:52.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:51 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:22:52.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:51 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:22:54.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:53 smithi134 ceph-mon[32619]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:22:54.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:53 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:22:54.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:53 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:22:54.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:53 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:22:54.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:53 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:22:54.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:53 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:22:54.233 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:53 smithi080 ceph-mon[29152]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:22:54.233 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:53 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:22:54.233 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:53 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:22:54.233 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:53 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:22:54.233 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:53 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:22:54.233 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:53 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:22:54.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:53 smithi148 ceph-mon[32528]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:22:54.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:53 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:22:54.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:53 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:22:54.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:53 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:22:54.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:53 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:22:54.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:53 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:22:54.607 INFO:teuthology.orchestra.run.smithi080.stdout:Created osd(s) 3 on host 'smithi080' 2024-04-24T23:22:54.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:22:54.605+0000 7f977e7fc700 1 -- 172.21.15.80:0/3498659600 <== mgr.14152 v2:172.21.15.80:6800/3830596922 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f9788108750 con 0x7f97740a2b90 2024-04-24T23:22:54.609 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:22:54.607+0000 7f978f2bd700 1 -- 172.21.15.80:0/3498659600 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f97740a2b90 msgr2=0x7f97740a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:54.609 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:22:54.607+0000 7f978f2bd700 1 --2- 172.21.15.80:0/3498659600 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f97740a2b90 0x7f97740a5050 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f978400c690 tx=0x7f9784005fb0 comp rx=0 tx=0).stop 2024-04-24T23:22:54.609 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:22:54.607+0000 7f978f2bd700 1 -- 172.21.15.80:0/3498659600 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f97881024a0 msgr2=0x7f978819fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:54.609 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:22:54.607+0000 7f978f2bd700 1 --2- 172.21.15.80:0/3498659600 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f97881024a0 0x7f978819fa70 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f9780007f10 tx=0x7f978000d3b0 comp rx=0 tx=0).stop 2024-04-24T23:22:54.609 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:22:54.607+0000 7f978f2bd700 1 -- 172.21.15.80:0/3498659600 shutdown_connections 2024-04-24T23:22:54.609 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:22:54.607+0000 7f978f2bd700 1 --2- 172.21.15.80:0/3498659600 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f97740a2b90 0x7f97740a5050 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:54.610 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:22:54.607+0000 7f978f2bd700 1 --2- 172.21.15.80:0/3498659600 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f97881024a0 0x7f978819fa70 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:54.610 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:22:54.607+0000 7f978f2bd700 1 --2- 172.21.15.80:0/3498659600 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9788100900 0x7f97881984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:54.610 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:22:54.607+0000 7f978f2bd700 1 --2- 172.21.15.80:0/3498659600 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9788101b00 0x7f97881989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:54.610 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:22:54.607+0000 7f978f2bd700 1 -- 172.21.15.80:0/3498659600 >> 172.21.15.80:0/3498659600 conn(0x7f97880fbec0 msgr2=0x7f9788107030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:54.610 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:22:54.607+0000 7f978f2bd700 1 -- 172.21.15.80:0/3498659600 shutdown_connections 2024-04-24T23:22:54.610 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:22:54.607+0000 7f978f2bd700 1 -- 172.21.15.80:0/3498659600 wait complete. 2024-04-24T23:22:55.310 DEBUG:teuthology.orchestra.run.smithi080:osd.3> sudo journalctl -f -n 0 -u ceph-ec7885aa-0290-11ef-bc93-c7b262605968@osd.3.service 2024-04-24T23:22:55.312 INFO:tasks.cephadm:Deploying osd.4 on smithi134 with /dev/vg_nvme/lv_3... 2024-04-24T23:22:55.313 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ec7885aa-0290-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-24T23:22:55.558 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/mon.b/config 2024-04-24T23:22:55.684 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:55 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:22:55.684 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:55 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:22:55.684 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:55 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:22:55.684 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:55 smithi134 ceph-mon[32619]: from='osd.3 [v2:172.21.15.80:6810/3128298510,v1:172.21.15.80:6811/3128298510]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-24T23:22:55.684 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:55 smithi134 ceph-mon[32619]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:22:55.789 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:55 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:22:55.789 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:55 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:22:55.789 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:55 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:22:55.789 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:55 smithi080 ceph-mon[29152]: from='osd.3 [v2:172.21.15.80:6810/3128298510,v1:172.21.15.80:6811/3128298510]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-24T23:22:55.789 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:55 smithi080 ceph-mon[29152]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:22:56.007 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:55 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:22:56.007 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:55 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:22:56.007 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:55 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:22:56.007 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:55 smithi148 ceph-mon[32528]: from='osd.3 [v2:172.21.15.80:6810/3128298510,v1:172.21.15.80:6811/3128298510]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-24T23:22:56.007 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:55 smithi148 ceph-mon[32528]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:22:56.701 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:56 smithi080 ceph-mon[29152]: from='osd.3 [v2:172.21.15.80:6810/3128298510,v1:172.21.15.80:6811/3128298510]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-24T23:22:56.702 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:56 smithi080 ceph-mon[29152]: osdmap e23: 4 total, 3 up, 4 in 2024-04-24T23:22:56.702 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:56 smithi080 ceph-mon[29152]: from='osd.3 [v2:172.21.15.80:6810/3128298510,v1:172.21.15.80:6811/3128298510]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi080", "root=default"]}]: dispatch 2024-04-24T23:22:56.702 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:56 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T23:22:56.948 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:56 smithi134 ceph-mon[32619]: from='osd.3 [v2:172.21.15.80:6810/3128298510,v1:172.21.15.80:6811/3128298510]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-24T23:22:56.948 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:56 smithi134 ceph-mon[32619]: osdmap e23: 4 total, 3 up, 4 in 2024-04-24T23:22:56.948 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:56 smithi134 ceph-mon[32619]: from='osd.3 [v2:172.21.15.80:6810/3128298510,v1:172.21.15.80:6811/3128298510]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi080", "root=default"]}]: dispatch 2024-04-24T23:22:56.948 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:56 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T23:22:57.007 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:56 smithi148 ceph-mon[32528]: from='osd.3 [v2:172.21.15.80:6810/3128298510,v1:172.21.15.80:6811/3128298510]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-24T23:22:57.007 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:56 smithi148 ceph-mon[32528]: osdmap e23: 4 total, 3 up, 4 in 2024-04-24T23:22:57.007 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:56 smithi148 ceph-mon[32528]: from='osd.3 [v2:172.21.15.80:6810/3128298510,v1:172.21.15.80:6811/3128298510]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi080", "root=default"]}]: dispatch 2024-04-24T23:22:57.007 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:56 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T23:22:57.289 INFO:journalctl@ceph.osd.3.smithi080.stdout:Apr 24 23:22:57 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-3[50507]: 2024-04-24T23:22:57.005+0000 7f269d282700 -1 osd.3 0 waiting for initial osdmap 2024-04-24T23:22:57.289 INFO:journalctl@ceph.osd.3.smithi080.stdout:Apr 24 23:22:57 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-3[50507]: 2024-04-24T23:22:57.010+0000 7f269486c700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T23:22:57.696 INFO:teuthology.orchestra.run.smithi134.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-24T23:22:57.697 INFO:teuthology.orchestra.run.smithi134.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-24T23:22:57.697 INFO:teuthology.orchestra.run.smithi134.stderr: stderr: 10+0 records in 2024-04-24T23:22:57.697 INFO:teuthology.orchestra.run.smithi134.stderr:10+0 records out 2024-04-24T23:22:57.697 INFO:teuthology.orchestra.run.smithi134.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0132615 s, 791 MB/s 2024-04-24T23:22:57.697 INFO:teuthology.orchestra.run.smithi134.stderr:--> Zapping successful for: 2024-04-24T23:22:57.948 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:57 smithi134 ceph-mon[32619]: from='osd.3 [v2:172.21.15.80:6810/3128298510,v1:172.21.15.80:6811/3128298510]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi080", "root=default"]}]': finished 2024-04-24T23:22:57.948 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:57 smithi134 ceph-mon[32619]: osdmap e24: 4 total, 3 up, 4 in 2024-04-24T23:22:57.948 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:57 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T23:22:57.948 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:57 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T23:22:57.948 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:57 smithi134 ceph-mon[32619]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:22:57.948 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:57 smithi134 ceph-mon[32619]: from='osd.3 [v2:172.21.15.80:6810/3128298510,v1:172.21.15.80:6811/3128298510]' entity='osd.3' 2024-04-24T23:22:58.007 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:57 smithi148 ceph-mon[32528]: from='osd.3 [v2:172.21.15.80:6810/3128298510,v1:172.21.15.80:6811/3128298510]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi080", "root=default"]}]': finished 2024-04-24T23:22:58.007 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:57 smithi148 ceph-mon[32528]: osdmap e24: 4 total, 3 up, 4 in 2024-04-24T23:22:58.007 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:57 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T23:22:58.007 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:57 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T23:22:58.007 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:57 smithi148 ceph-mon[32528]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:22:58.007 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:57 smithi148 ceph-mon[32528]: from='osd.3 [v2:172.21.15.80:6810/3128298510,v1:172.21.15.80:6811/3128298510]' entity='osd.3' 2024-04-24T23:22:58.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:57 smithi080 ceph-mon[29152]: from='osd.3 [v2:172.21.15.80:6810/3128298510,v1:172.21.15.80:6811/3128298510]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi080", "root=default"]}]': finished 2024-04-24T23:22:58.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:57 smithi080 ceph-mon[29152]: osdmap e24: 4 total, 3 up, 4 in 2024-04-24T23:22:58.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:57 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T23:22:58.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:57 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T23:22:58.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:57 smithi080 ceph-mon[29152]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:22:58.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:57 smithi080 ceph-mon[29152]: from='osd.3 [v2:172.21.15.80:6810/3128298510,v1:172.21.15.80:6811/3128298510]' entity='osd.3' 2024-04-24T23:22:58.370 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ec7885aa-0290-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi134:vg_nvme/lv_3 2024-04-24T23:22:58.610 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/mon.b/config 2024-04-24T23:22:58.698 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:58 smithi134 ceph-mon[32619]: purged_snaps scrub starts 2024-04-24T23:22:58.698 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:58 smithi134 ceph-mon[32619]: purged_snaps scrub ok 2024-04-24T23:22:58.698 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:58 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T23:22:58.698 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:58 smithi134 ceph-mon[32619]: osd.3 [v2:172.21.15.80:6810/3128298510,v1:172.21.15.80:6811/3128298510] boot 2024-04-24T23:22:58.698 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:58 smithi134 ceph-mon[32619]: osdmap e25: 4 total, 4 up, 4 in 2024-04-24T23:22:58.698 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:58 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T23:22:58.699 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:58 smithi134 ceph-mon[32619]: osdmap e26: 4 total, 4 up, 4 in 2024-04-24T23:22:58.699 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:58 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:22:58.699 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:58 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:22:58.699 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:58 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:22:58.699 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:58 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:22:58.699 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:58 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:22:58.699 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:58 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:22:58.699 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:58 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:22:58.699 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:22:58 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:22:58.789 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:58 smithi080 ceph-mon[29152]: purged_snaps scrub starts 2024-04-24T23:22:58.789 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:58 smithi080 ceph-mon[29152]: purged_snaps scrub ok 2024-04-24T23:22:58.789 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:58 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T23:22:58.789 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:58 smithi080 ceph-mon[29152]: osd.3 [v2:172.21.15.80:6810/3128298510,v1:172.21.15.80:6811/3128298510] boot 2024-04-24T23:22:58.789 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:58 smithi080 ceph-mon[29152]: osdmap e25: 4 total, 4 up, 4 in 2024-04-24T23:22:58.789 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:58 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T23:22:58.789 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:58 smithi080 ceph-mon[29152]: osdmap e26: 4 total, 4 up, 4 in 2024-04-24T23:22:58.790 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:58 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:22:58.790 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:58 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:22:58.790 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:58 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:22:58.790 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:58 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:22:58.790 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:58 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:22:58.790 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:58 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:22:58.790 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:58 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:22:58.790 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:22:58 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:22:59.007 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:58 smithi148 ceph-mon[32528]: purged_snaps scrub starts 2024-04-24T23:22:59.007 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:58 smithi148 ceph-mon[32528]: purged_snaps scrub ok 2024-04-24T23:22:59.007 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:58 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T23:22:59.007 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:58 smithi148 ceph-mon[32528]: osd.3 [v2:172.21.15.80:6810/3128298510,v1:172.21.15.80:6811/3128298510] boot 2024-04-24T23:22:59.007 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:58 smithi148 ceph-mon[32528]: osdmap e25: 4 total, 4 up, 4 in 2024-04-24T23:22:59.007 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:58 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T23:22:59.007 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:58 smithi148 ceph-mon[32528]: osdmap e26: 4 total, 4 up, 4 in 2024-04-24T23:22:59.007 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:58 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:22:59.007 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:58 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:22:59.008 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:58 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:22:59.008 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:58 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:22:59.008 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:58 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:22:59.008 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:58 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:22:59.008 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:58 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:22:59.008 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:22:58 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:23:00.331 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:23:00.329+0000 7fbec2046700 1 -- 172.21.15.134:0/2613494529 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbebc103490 msgr2=0x7fbebc1038f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:00.331 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:23:00.329+0000 7fbec2046700 1 --2- 172.21.15.134:0/2613494529 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbebc103490 0x7fbebc1038f0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fbea40099c0 tx=0x7fbea4009cd0 comp rx=0 tx=0).stop 2024-04-24T23:23:00.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:23:00.329+0000 7fbec2046700 1 -- 172.21.15.134:0/2613494529 shutdown_connections 2024-04-24T23:23:00.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:23:00.329+0000 7fbec2046700 1 --2- 172.21.15.134:0/2613494529 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbebc103e30 0x7fbebc10a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:00.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:23:00.329+0000 7fbec2046700 1 --2- 172.21.15.134:0/2613494529 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbebc102290 0x7fbebc1026b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:00.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:23:00.329+0000 7fbec2046700 1 --2- 172.21.15.134:0/2613494529 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbebc103490 0x7fbebc1038f0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:00.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:23:00.329+0000 7fbec2046700 1 -- 172.21.15.134:0/2613494529 >> 172.21.15.134:0/2613494529 conn(0x7fbebc0fd830 msgr2=0x7fbebc0ffc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:00.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:23:00.329+0000 7fbec2046700 1 -- 172.21.15.134:0/2613494529 shutdown_connections 2024-04-24T23:23:00.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:23:00.329+0000 7fbec2046700 1 -- 172.21.15.134:0/2613494529 wait complete. 2024-04-24T23:23:00.332 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:00 smithi134 ceph-mon[32619]: Detected new or changed devices on smithi080 2024-04-24T23:23:00.332 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:00 smithi134 ceph-mon[32619]: Adjusting osd_memory_target on smithi080 to 8544M 2024-04-24T23:23:00.332 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:00 smithi134 ceph-mon[32619]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-24T23:23:00.333 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:00 smithi134 ceph-mon[32619]: osdmap e27: 4 total, 4 up, 4 in 2024-04-24T23:23:00.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:23:00.330+0000 7fbec2046700 1 Processor -- start 2024-04-24T23:23:00.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:23:00.330+0000 7fbec2046700 1 -- start start 2024-04-24T23:23:00.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:23:00.330+0000 7fbec2046700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbebc102290 0x7fbebc1984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:00.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:23:00.330+0000 7fbec2046700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbebc103490 0x7fbebc198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:00.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:23:00.330+0000 7fbec2046700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbebc103e30 0x7fbebc19faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:00.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:23:00.330+0000 7fbec2046700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fbebc10de70 con 0x7fbebc102290 2024-04-24T23:23:00.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:23:00.330+0000 7fbec2046700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbebc10dcf0 con 0x7fbebc103490 2024-04-24T23:23:00.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:23:00.330+0000 7fbec2046700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fbebc10dff0 con 0x7fbebc103e30 2024-04-24T23:23:00.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:23:00.330+0000 7fbeb2ffd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbebc103490 0x7fbebc198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:00.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:23:00.330+0000 7fbeb2ffd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbebc103490 0x7fbebc198a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:54206/0 (socket says 172.21.15.134:54206) 2024-04-24T23:23:00.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:23:00.330+0000 7fbeb2ffd700 1 -- 172.21.15.134:0/2852595020 learned_addr learned my addr 172.21.15.134:0/2852595020 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-04-24T23:23:00.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:23:00.330+0000 7fbebb7fe700 1 --2- 172.21.15.134:0/2852595020 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbebc102290 0x7fbebc1984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:00.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:23:00.330+0000 7fbebbfff700 1 --2- 172.21.15.134:0/2852595020 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbebc103e30 0x7fbebc19faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:00.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:23:00.331+0000 7fbeb2ffd700 1 -- 172.21.15.134:0/2852595020 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbebc103e30 msgr2=0x7fbebc19faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:00.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:23:00.331+0000 7fbeb2ffd700 1 --2- 172.21.15.134:0/2852595020 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbebc103e30 0x7fbebc19faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:00.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:23:00.331+0000 7fbeb2ffd700 1 -- 172.21.15.134:0/2852595020 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbebc102290 msgr2=0x7fbebc1984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:00.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:23:00.331+0000 7fbeb2ffd700 1 --2- 172.21.15.134:0/2852595020 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbebc102290 0x7fbebc1984e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:00.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:23:00.331+0000 7fbeb2ffd700 1 -- 172.21.15.134:0/2852595020 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbebc1a0140 con 0x7fbebc103490 2024-04-24T23:23:00.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:23:00.331+0000 7fbebb7fe700 1 --2- 172.21.15.134:0/2852595020 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbebc102290 0x7fbebc1984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:23:00.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:23:00.331+0000 7fbeb2ffd700 1 --2- 172.21.15.134:0/2852595020 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbebc103490 0x7fbebc198a20 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fbea4005910 tx=0x7fbea400fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:00.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:23:00.331+0000 7fbeb97fa700 1 -- 172.21.15.134:0/2852595020 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea4027070 con 0x7fbebc103490 2024-04-24T23:23:00.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:23:00.331+0000 7fbeb97fa700 1 -- 172.21.15.134:0/2852595020 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbea400fe40 con 0x7fbebc103490 2024-04-24T23:23:00.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:23:00.331+0000 7fbec2046700 1 -- 172.21.15.134:0/2852595020 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbebc1a03d0 con 0x7fbebc103490 2024-04-24T23:23:00.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:23:00.331+0000 7fbeb97fa700 1 -- 172.21.15.134:0/2852595020 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea40173f0 con 0x7fbebc103490 2024-04-24T23:23:00.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:23:00.331+0000 7fbec2046700 1 -- 172.21.15.134:0/2852595020 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbebc1a0880 con 0x7fbebc103490 2024-04-24T23:23:00.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:23:00.332+0000 7fbec2046700 1 -- 172.21.15.134:0/2852595020 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbebc068f20 con 0x7fbebc103490 2024-04-24T23:23:00.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:23:00.333+0000 7fbeb97fa700 1 -- 172.21.15.134:0/2852595020 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fbea402a070 con 0x7fbebc103490 2024-04-24T23:23:00.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:23:00.334+0000 7fbeb97fa700 1 --2- 172.21.15.134:0/2852595020 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fbea80a2ca0 0x7fbea80a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:00.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:23:00.334+0000 7fbeb97fa700 1 -- 172.21.15.134:0/2852595020 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 3669+0+0 (secure 0 0 0) 0x7fbea40cea50 con 0x7fbebc103490 2024-04-24T23:23:00.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:23:00.334+0000 7fbebb7fe700 1 --2- 172.21.15.134:0/2852595020 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fbea80a2ca0 0x7fbea80a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:00.338 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:23:00.335+0000 7fbebb7fe700 1 --2- 172.21.15.134:0/2852595020 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fbea80a2ca0 0x7fbea80a5160 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fbeac005950 tx=0x7fbeac00a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:00.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:23:00.338+0000 7fbeb97fa700 1 -- 172.21.15.134:0/2852595020 <== mon.2 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fbea4013070 con 0x7fbebc103490 2024-04-24T23:23:00.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:00 smithi148 ceph-mon[32528]: Detected new or changed devices on smithi080 2024-04-24T23:23:00.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:00 smithi148 ceph-mon[32528]: Adjusting osd_memory_target on smithi080 to 8544M 2024-04-24T23:23:00.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:00 smithi148 ceph-mon[32528]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-24T23:23:00.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:00 smithi148 ceph-mon[32528]: osdmap e27: 4 total, 4 up, 4 in 2024-04-24T23:23:00.539 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:00 smithi080 ceph-mon[29152]: Detected new or changed devices on smithi080 2024-04-24T23:23:00.539 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:00 smithi080 ceph-mon[29152]: Adjusting osd_memory_target on smithi080 to 8544M 2024-04-24T23:23:00.539 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:00 smithi080 ceph-mon[29152]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-24T23:23:00.539 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:00 smithi080 ceph-mon[29152]: osdmap e27: 4 total, 4 up, 4 in 2024-04-24T23:23:00.541 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:23:00.540+0000 7fbec2046700 1 -- 172.21.15.134:0/2852595020 --> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fbebc062730 con 0x7fbea80a2ca0 2024-04-24T23:23:01.390 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:01 smithi134 ceph-mon[32619]: osdmap e28: 4 total, 4 up, 4 in 2024-04-24T23:23:01.391 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:01 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:23:01.391 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:01 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:23:01.391 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:01 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:23:01.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:01 smithi148 ceph-mon[32528]: osdmap e28: 4 total, 4 up, 4 in 2024-04-24T23:23:01.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:01 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:23:01.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:01 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:23:01.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:01 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:23:01.539 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:01 smithi080 ceph-mon[29152]: osdmap e28: 4 total, 4 up, 4 in 2024-04-24T23:23:01.539 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:01 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:23:01.539 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:01 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:23:01.539 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:01 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:23:02.418 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:02 smithi134 ceph-mon[32619]: from='client.24212 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:23:02.418 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:02 smithi134 ceph-mon[32619]: pgmap v105: 1 pgs: 1 remapped+peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-24T23:23:02.418 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:02 smithi134 ceph-mon[32619]: osdmap e29: 4 total, 4 up, 4 in 2024-04-24T23:23:02.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:02 smithi148 ceph-mon[32528]: from='client.24212 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:23:02.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:02 smithi148 ceph-mon[32528]: pgmap v105: 1 pgs: 1 remapped+peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-24T23:23:02.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:02 smithi148 ceph-mon[32528]: osdmap e29: 4 total, 4 up, 4 in 2024-04-24T23:23:02.539 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:02 smithi080 ceph-mon[29152]: from='client.24212 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:23:02.539 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:02 smithi080 ceph-mon[29152]: pgmap v105: 1 pgs: 1 remapped+peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-24T23:23:02.539 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:02 smithi080 ceph-mon[29152]: osdmap e29: 4 total, 4 up, 4 in 2024-04-24T23:23:04.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:04 smithi134 ceph-mon[32619]: pgmap v107: 1 pgs: 1 remapped+peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-24T23:23:04.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:04 smithi134 ceph-mon[32619]: from='client.? 172.21.15.134:0/966905826' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dfc55a79-a7e8-417a-b898-ec44171a54af"}]: dispatch 2024-04-24T23:23:04.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:04 smithi134 ceph-mon[32619]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dfc55a79-a7e8-417a-b898-ec44171a54af"}]: dispatch 2024-04-24T23:23:04.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:04 smithi134 ceph-mon[32619]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dfc55a79-a7e8-417a-b898-ec44171a54af"}]': finished 2024-04-24T23:23:04.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:04 smithi134 ceph-mon[32619]: osdmap e30: 5 total, 4 up, 5 in 2024-04-24T23:23:04.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:04 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T23:23:04.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:04 smithi134 ceph-mon[32619]: from='client.? 172.21.15.134:0/662138318' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:23:04.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:04 smithi148 ceph-mon[32528]: pgmap v107: 1 pgs: 1 remapped+peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-24T23:23:04.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:04 smithi148 ceph-mon[32528]: from='client.? 172.21.15.134:0/966905826' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dfc55a79-a7e8-417a-b898-ec44171a54af"}]: dispatch 2024-04-24T23:23:04.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:04 smithi148 ceph-mon[32528]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dfc55a79-a7e8-417a-b898-ec44171a54af"}]: dispatch 2024-04-24T23:23:04.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:04 smithi148 ceph-mon[32528]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dfc55a79-a7e8-417a-b898-ec44171a54af"}]': finished 2024-04-24T23:23:04.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:04 smithi148 ceph-mon[32528]: osdmap e30: 5 total, 4 up, 5 in 2024-04-24T23:23:04.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:04 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T23:23:04.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:04 smithi148 ceph-mon[32528]: from='client.? 172.21.15.134:0/662138318' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:23:04.539 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:04 smithi080 ceph-mon[29152]: pgmap v107: 1 pgs: 1 remapped+peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-24T23:23:04.540 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:04 smithi080 ceph-mon[29152]: from='client.? 172.21.15.134:0/966905826' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dfc55a79-a7e8-417a-b898-ec44171a54af"}]: dispatch 2024-04-24T23:23:04.540 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:04 smithi080 ceph-mon[29152]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dfc55a79-a7e8-417a-b898-ec44171a54af"}]: dispatch 2024-04-24T23:23:04.540 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:04 smithi080 ceph-mon[29152]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dfc55a79-a7e8-417a-b898-ec44171a54af"}]': finished 2024-04-24T23:23:04.540 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:04 smithi080 ceph-mon[29152]: osdmap e30: 5 total, 4 up, 5 in 2024-04-24T23:23:04.540 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:04 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T23:23:04.540 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:04 smithi080 ceph-mon[29152]: from='client.? 172.21.15.134:0/662138318' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:23:05.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:05 smithi148 ceph-mon[32528]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-04-24T23:23:05.539 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:05 smithi080 ceph-mon[29152]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-04-24T23:23:05.698 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:05 smithi134 ceph-mon[32619]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-04-24T23:23:08.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:07 smithi134 ceph-mon[32619]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-04-24T23:23:08.209 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:07 smithi148 ceph-mon[32528]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-04-24T23:23:08.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:07 smithi080 ceph-mon[29152]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-04-24T23:23:09.948 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:09 smithi134 ceph-mon[32619]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-24T23:23:10.243 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:09 smithi080 ceph-mon[29152]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-24T23:23:10.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:09 smithi148 ceph-mon[32528]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-24T23:23:11.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:10 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-24T23:23:11.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:10 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:23:11.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:10 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-24T23:23:11.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:10 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:23:11.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:10 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-24T23:23:11.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:10 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:23:12.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:11 smithi134 ceph-mon[32619]: Deploying daemon osd.4 on smithi134 2024-04-24T23:23:12.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:11 smithi134 ceph-mon[32619]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-04-24T23:23:12.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:11 smithi148 ceph-mon[32528]: Deploying daemon osd.4 on smithi134 2024-04-24T23:23:12.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:11 smithi148 ceph-mon[32528]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-04-24T23:23:12.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:11 smithi080 ceph-mon[29152]: Deploying daemon osd.4 on smithi134 2024-04-24T23:23:12.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:11 smithi080 ceph-mon[29152]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-04-24T23:23:14.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:13 smithi134 ceph-mon[32619]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-24T23:23:14.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:13 smithi148 ceph-mon[32528]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-24T23:23:14.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:13 smithi080 ceph-mon[29152]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-24T23:23:16.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:15 smithi080 ceph-mon[29152]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-04-24T23:23:16.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:15 smithi134 ceph-mon[32619]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-04-24T23:23:16.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:15 smithi148 ceph-mon[32528]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-04-24T23:23:17.698 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:17 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:23:17.698 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:17 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:23:17.698 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:17 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:23:17.698 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:17 smithi134 ceph-mon[32619]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-24T23:23:17.757 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:17 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:23:17.757 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:17 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:23:17.757 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:17 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:23:17.757 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:17 smithi148 ceph-mon[32528]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-24T23:23:17.789 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:17 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:23:17.789 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:17 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:23:17.789 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:17 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:23:17.789 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:17 smithi080 ceph-mon[29152]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-24T23:23:20.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:19 smithi134 ceph-mon[32619]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-24T23:23:20.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:19 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:23:20.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:19 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:23:20.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:19 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:23:20.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:19 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:23:20.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:19 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:23:20.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:19 smithi148 ceph-mon[32528]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-24T23:23:20.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:19 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:23:20.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:19 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:23:20.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:19 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:23:20.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:19 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:23:20.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:19 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:23:20.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:19 smithi080 ceph-mon[29152]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-24T23:23:20.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:19 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:23:20.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:19 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:23:20.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:19 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:23:20.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:19 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:23:20.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:19 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:23:20.357 INFO:teuthology.orchestra.run.smithi134.stdout:Created osd(s) 4 on host 'smithi134' 2024-04-24T23:23:20.357 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:23:20.355+0000 7fbeb97fa700 1 -- 172.21.15.134:0/2852595020 <== mgr.14152 v2:172.21.15.80:6800/3830596922 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fbebc062730 con 0x7fbea80a2ca0 2024-04-24T23:23:20.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:23:20.357+0000 7fbec2046700 1 -- 172.21.15.134:0/2852595020 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fbea80a2ca0 msgr2=0x7fbea80a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:20.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:23:20.357+0000 7fbec2046700 1 --2- 172.21.15.134:0/2852595020 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fbea80a2ca0 0x7fbea80a5160 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fbeac005950 tx=0x7fbeac00a300 comp rx=0 tx=0).stop 2024-04-24T23:23:20.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:23:20.357+0000 7fbec2046700 1 -- 172.21.15.134:0/2852595020 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbebc103490 msgr2=0x7fbebc198a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:20.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:23:20.357+0000 7fbec2046700 1 --2- 172.21.15.134:0/2852595020 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbebc103490 0x7fbebc198a20 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fbea4005910 tx=0x7fbea400fa00 comp rx=0 tx=0).stop 2024-04-24T23:23:20.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:23:20.357+0000 7fbec2046700 1 -- 172.21.15.134:0/2852595020 shutdown_connections 2024-04-24T23:23:20.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:23:20.357+0000 7fbec2046700 1 --2- 172.21.15.134:0/2852595020 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbebc103e30 0x7fbebc19faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:20.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:23:20.357+0000 7fbec2046700 1 --2- 172.21.15.134:0/2852595020 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fbea80a2ca0 0x7fbea80a5160 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:20.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:23:20.357+0000 7fbec2046700 1 --2- 172.21.15.134:0/2852595020 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbebc102290 0x7fbebc1984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:20.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:23:20.357+0000 7fbec2046700 1 --2- 172.21.15.134:0/2852595020 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbebc103490 0x7fbebc198a20 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:20.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:23:20.357+0000 7fbec2046700 1 -- 172.21.15.134:0/2852595020 >> 172.21.15.134:0/2852595020 conn(0x7fbebc0fd830 msgr2=0x7fbebc1067b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:20.360 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:23:20.357+0000 7fbec2046700 1 -- 172.21.15.134:0/2852595020 shutdown_connections 2024-04-24T23:23:20.360 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:23:20.357+0000 7fbec2046700 1 -- 172.21.15.134:0/2852595020 wait complete. 2024-04-24T23:23:20.948 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:20 smithi134 ceph-mon[32619]: from='osd.4 [v2:172.21.15.134:6808/1917556060,v1:172.21.15.134:6809/1917556060]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-24T23:23:20.948 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:20 smithi134 ceph-mon[32619]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-24T23:23:20.948 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:20 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:23:20.948 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:20 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:23:20.948 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:20 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:23:21.032 DEBUG:teuthology.orchestra.run.smithi134:osd.4> sudo journalctl -f -n 0 -u ceph-ec7885aa-0290-11ef-bc93-c7b262605968@osd.4.service 2024-04-24T23:23:21.034 INFO:tasks.cephadm:Deploying osd.5 on smithi148 with /dev/vg_nvme/lv_3... 2024-04-24T23:23:21.034 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ec7885aa-0290-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-24T23:23:21.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:20 smithi148 ceph-mon[32528]: from='osd.4 [v2:172.21.15.134:6808/1917556060,v1:172.21.15.134:6809/1917556060]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-24T23:23:21.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:20 smithi148 ceph-mon[32528]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-24T23:23:21.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:20 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:23:21.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:20 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:23:21.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:20 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:23:21.280 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T23:23:21.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:20 smithi080 ceph-mon[29152]: from='osd.4 [v2:172.21.15.134:6808/1917556060,v1:172.21.15.134:6809/1917556060]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-24T23:23:21.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:20 smithi080 ceph-mon[29152]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-24T23:23:21.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:20 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:23:21.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:20 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:23:21.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:20 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:23:22.007 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:21 smithi148 ceph-mon[32528]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-24T23:23:22.007 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:21 smithi148 ceph-mon[32528]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-24T23:23:22.008 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:21 smithi148 ceph-mon[32528]: from='osd.4 [v2:172.21.15.134:6808/1917556060,v1:172.21.15.134:6809/1917556060]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-04-24T23:23:22.008 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:21 smithi148 ceph-mon[32528]: osdmap e31: 5 total, 4 up, 5 in 2024-04-24T23:23:22.008 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:21 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T23:23:22.008 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:21 smithi148 ceph-mon[32528]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-04-24T23:23:22.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:21 smithi134 ceph-mon[32619]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-24T23:23:22.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:21 smithi134 ceph-mon[32619]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-24T23:23:22.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:21 smithi134 ceph-mon[32619]: from='osd.4 [v2:172.21.15.134:6808/1917556060,v1:172.21.15.134:6809/1917556060]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-04-24T23:23:22.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:21 smithi134 ceph-mon[32619]: osdmap e31: 5 total, 4 up, 5 in 2024-04-24T23:23:22.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:21 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T23:23:22.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:21 smithi134 ceph-mon[32619]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-04-24T23:23:22.198 INFO:journalctl@ceph.osd.4.smithi134.stdout:Apr 24 23:23:21 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-4[44187]: 2024-04-24T23:23:21.897+0000 7f91f463c700 -1 osd.4 0 waiting for initial osdmap 2024-04-24T23:23:22.199 INFO:journalctl@ceph.osd.4.smithi134.stdout:Apr 24 23:23:21 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-4[44187]: 2024-04-24T23:23:21.903+0000 7f91f042f700 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T23:23:22.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:21 smithi080 ceph-mon[29152]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-24T23:23:22.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:21 smithi080 ceph-mon[29152]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-24T23:23:22.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:21 smithi080 ceph-mon[29152]: from='osd.4 [v2:172.21.15.134:6808/1917556060,v1:172.21.15.134:6809/1917556060]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-04-24T23:23:22.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:21 smithi080 ceph-mon[29152]: osdmap e31: 5 total, 4 up, 5 in 2024-04-24T23:23:22.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:21 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T23:23:22.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:21 smithi080 ceph-mon[29152]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-04-24T23:23:23.007 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:22 smithi148 ceph-mon[32528]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-04-24T23:23:23.007 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:22 smithi148 ceph-mon[32528]: osdmap e32: 5 total, 4 up, 5 in 2024-04-24T23:23:23.007 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:22 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T23:23:23.007 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:22 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T23:23:23.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:22 smithi134 ceph-mon[32619]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-04-24T23:23:23.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:22 smithi134 ceph-mon[32619]: osdmap e32: 5 total, 4 up, 5 in 2024-04-24T23:23:23.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:22 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T23:23:23.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:22 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T23:23:23.224 INFO:teuthology.orchestra.run.smithi148.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-24T23:23:23.224 INFO:teuthology.orchestra.run.smithi148.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-24T23:23:23.225 INFO:teuthology.orchestra.run.smithi148.stderr: stderr: 10+0 records in 2024-04-24T23:23:23.225 INFO:teuthology.orchestra.run.smithi148.stderr:10+0 records out 2024-04-24T23:23:23.225 INFO:teuthology.orchestra.run.smithi148.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0142562 s, 736 MB/s 2024-04-24T23:23:23.225 INFO:teuthology.orchestra.run.smithi148.stderr:--> Zapping successful for: 2024-04-24T23:23:23.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:22 smithi080 ceph-mon[29152]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-04-24T23:23:23.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:22 smithi080 ceph-mon[29152]: osdmap e32: 5 total, 4 up, 5 in 2024-04-24T23:23:23.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:22 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T23:23:23.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:22 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T23:23:23.902 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ec7885aa-0290-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi148:vg_nvme/lv_3 2024-04-24T23:23:24.135 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:23 smithi134 ceph-mon[32619]: purged_snaps scrub starts 2024-04-24T23:23:24.136 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:23 smithi134 ceph-mon[32619]: purged_snaps scrub ok 2024-04-24T23:23:24.136 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:23 smithi134 ceph-mon[32619]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-24T23:23:24.136 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:23 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T23:23:24.136 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:23 smithi134 ceph-mon[32619]: osd.4 [v2:172.21.15.134:6808/1917556060,v1:172.21.15.134:6809/1917556060] boot 2024-04-24T23:23:24.136 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:23 smithi134 ceph-mon[32619]: osdmap e33: 5 total, 5 up, 5 in 2024-04-24T23:23:24.136 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:23 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T23:23:24.143 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T23:23:24.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:23 smithi148 ceph-mon[32528]: purged_snaps scrub starts 2024-04-24T23:23:24.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:23 smithi148 ceph-mon[32528]: purged_snaps scrub ok 2024-04-24T23:23:24.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:23 smithi148 ceph-mon[32528]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-24T23:23:24.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:23 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T23:23:24.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:23 smithi148 ceph-mon[32528]: osd.4 [v2:172.21.15.134:6808/1917556060,v1:172.21.15.134:6809/1917556060] boot 2024-04-24T23:23:24.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:23 smithi148 ceph-mon[32528]: osdmap e33: 5 total, 5 up, 5 in 2024-04-24T23:23:24.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:23 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T23:23:24.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:23 smithi080 ceph-mon[29152]: purged_snaps scrub starts 2024-04-24T23:23:24.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:23 smithi080 ceph-mon[29152]: purged_snaps scrub ok 2024-04-24T23:23:24.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:23 smithi080 ceph-mon[29152]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-24T23:23:24.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:23 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T23:23:24.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:23 smithi080 ceph-mon[29152]: osd.4 [v2:172.21.15.134:6808/1917556060,v1:172.21.15.134:6809/1917556060] boot 2024-04-24T23:23:24.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:23 smithi080 ceph-mon[29152]: osdmap e33: 5 total, 5 up, 5 in 2024-04-24T23:23:24.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:23 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T23:23:25.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:24 smithi134 ceph-mon[32619]: osdmap e34: 5 total, 5 up, 5 in 2024-04-24T23:23:25.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:24 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:23:25.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:24 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:23:25.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:24 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:23:25.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:24 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:23:25.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:24 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:23:25.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:24 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:23:25.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:24 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:23:25.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:24 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:23:25.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:24 smithi148 ceph-mon[32528]: osdmap e34: 5 total, 5 up, 5 in 2024-04-24T23:23:25.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:24 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:23:25.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:24 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:23:25.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:24 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:23:25.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:24 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:23:25.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:24 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:23:25.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:24 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:23:25.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:24 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:23:25.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:24 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:23:25.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:24 smithi080 ceph-mon[29152]: osdmap e34: 5 total, 5 up, 5 in 2024-04-24T23:23:25.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:24 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:23:25.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:24 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:23:25.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:24 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:23:25.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:24 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:23:25.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:24 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:23:25.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:24 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:23:25.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:24 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:23:25.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:24 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:23:25.764 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:23:25.761+0000 7ff27e643700 1 -- 172.21.15.148:0/3991210442 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff278102390 msgr2=0x7ff2781027f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:25.764 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:23:25.761+0000 7ff27e643700 1 --2- 172.21.15.148:0/3991210442 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff278102390 0x7ff2781027f0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff26c009990 tx=0x7ff26c009ca0 comp rx=0 tx=0).stop 2024-04-24T23:23:25.764 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:23:25.761+0000 7ff27e643700 1 -- 172.21.15.148:0/3991210442 shutdown_connections 2024-04-24T23:23:25.764 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:23:25.761+0000 7ff27e643700 1 --2- 172.21.15.148:0/3991210442 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff278102d30 0x7ff2781095e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:25.764 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:23:25.761+0000 7ff27e643700 1 --2- 172.21.15.148:0/3991210442 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff278101190 0x7ff2781015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:25.764 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:23:25.761+0000 7ff27e643700 1 --2- 172.21.15.148:0/3991210442 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff278102390 0x7ff2781027f0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:25.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:23:25.761+0000 7ff27e643700 1 -- 172.21.15.148:0/3991210442 >> 172.21.15.148:0/3991210442 conn(0x7ff2780fc750 msgr2=0x7ff2780feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:25.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:23:25.761+0000 7ff27e643700 1 -- 172.21.15.148:0/3991210442 shutdown_connections 2024-04-24T23:23:25.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:23:25.761+0000 7ff27e643700 1 -- 172.21.15.148:0/3991210442 wait complete. 2024-04-24T23:23:25.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:23:25.761+0000 7ff27e643700 1 Processor -- start 2024-04-24T23:23:25.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:23:25.761+0000 7ff27e643700 1 -- start start 2024-04-24T23:23:25.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:23:25.762+0000 7ff27e643700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff278101190 0x7ff278198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:25.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:23:25.762+0000 7ff27e643700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff278102390 0x7ff278198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:25.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:23:25.762+0000 7ff27e643700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff278102d30 0x7ff27819fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:25.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:23:25.762+0000 7ff27e643700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7ff27810cdf0 con 0x7ff278102d30 2024-04-24T23:23:25.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:23:25.762+0000 7ff27e643700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff27810cc70 con 0x7ff278102390 2024-04-24T23:23:25.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:23:25.762+0000 7ff27e643700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff27810cf70 con 0x7ff278101190 2024-04-24T23:23:25.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:23:25.762+0000 7ff277fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff278101190 0x7ff278198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:25.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:23:25.762+0000 7ff277fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff278101190 0x7ff278198520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:52046/0 (socket says 172.21.15.148:52046) 2024-04-24T23:23:25.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:23:25.762+0000 7ff277fff700 1 -- 172.21.15.148:0/900950477 learned_addr learned my addr 172.21.15.148:0/900950477 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-24T23:23:25.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:23:25.762+0000 7ff2777fe700 1 --2- 172.21.15.148:0/900950477 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff278102390 0x7ff278198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:25.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:23:25.762+0000 7ff27cbe0700 1 --2- 172.21.15.148:0/900950477 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff278102d30 0x7ff27819fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:25.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:23:25.762+0000 7ff277fff700 1 -- 172.21.15.148:0/900950477 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff278102390 msgr2=0x7ff278198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:25.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:23:25.762+0000 7ff277fff700 1 --2- 172.21.15.148:0/900950477 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff278102390 0x7ff278198a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:25.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:23:25.762+0000 7ff277fff700 1 -- 172.21.15.148:0/900950477 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff278102d30 msgr2=0x7ff27819fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:25.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:23:25.762+0000 7ff277fff700 1 --2- 172.21.15.148:0/900950477 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff278102d30 0x7ff27819fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:25.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:23:25.762+0000 7ff277fff700 1 -- 172.21.15.148:0/900950477 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2781a0180 con 0x7ff278101190 2024-04-24T23:23:25.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:23:25.762+0000 7ff277fff700 1 --2- 172.21.15.148:0/900950477 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff278101190 0x7ff278198520 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7ff26800b530 tx=0x7ff26800b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:25.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:23:25.762+0000 7ff2757fa700 1 -- 172.21.15.148:0/900950477 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff268015020 con 0x7ff278101190 2024-04-24T23:23:25.767 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:23:25.763+0000 7ff2757fa700 1 -- 172.21.15.148:0/900950477 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff268011d10 con 0x7ff278101190 2024-04-24T23:23:25.767 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:23:25.763+0000 7ff27e643700 1 -- 172.21.15.148:0/900950477 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2781a0470 con 0x7ff278101190 2024-04-24T23:23:25.767 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:23:25.763+0000 7ff2757fa700 1 -- 172.21.15.148:0/900950477 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff268010d90 con 0x7ff278101190 2024-04-24T23:23:25.767 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:23:25.763+0000 7ff27e643700 1 -- 172.21.15.148:0/900950477 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2781a09e0 con 0x7ff278101190 2024-04-24T23:23:25.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:23:25.764+0000 7ff27e643700 1 -- 172.21.15.148:0/900950477 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff278068f20 con 0x7ff278101190 2024-04-24T23:23:25.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:23:25.765+0000 7ff2757fa700 1 -- 172.21.15.148:0/900950477 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7ff268018020 con 0x7ff278101190 2024-04-24T23:23:25.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:23:25.765+0000 7ff2757fa700 1 --2- 172.21.15.148:0/900950477 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7ff2600a2c20 0x7ff2600a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:25.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:23:25.765+0000 7ff2757fa700 1 -- 172.21.15.148:0/900950477 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 4101+0+0 (secure 0 0 0) 0x7ff2680cd6f0 con 0x7ff278101190 2024-04-24T23:23:25.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:23:25.766+0000 7ff2777fe700 1 --2- 172.21.15.148:0/900950477 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7ff2600a2c20 0x7ff2600a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:25.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:23:25.766+0000 7ff2777fe700 1 --2- 172.21.15.148:0/900950477 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7ff2600a2c20 0x7ff2600a50e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff26c00b5c0 tx=0x7ff26c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:25.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:23:25.770+0000 7ff2757fa700 1 -- 172.21.15.148:0/900950477 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff26801f020 con 0x7ff278101190 2024-04-24T23:23:25.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:23:25.974+0000 7ff27e643700 1 -- 172.21.15.148:0/900950477 --> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7ff2780fe8d0 con 0x7ff2600a2c20 2024-04-24T23:23:26.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:25 smithi134 ceph-mon[32619]: Detected new or changed devices on smithi134 2024-04-24T23:23:26.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:25 smithi134 ceph-mon[32619]: Adjusting osd_memory_target on smithi134 to 8544M 2024-04-24T23:23:26.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:25 smithi134 ceph-mon[32619]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:23:26.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:25 smithi148 ceph-mon[32528]: Detected new or changed devices on smithi134 2024-04-24T23:23:26.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:25 smithi148 ceph-mon[32528]: Adjusting osd_memory_target on smithi134 to 8544M 2024-04-24T23:23:26.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:25 smithi148 ceph-mon[32528]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:23:26.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:25 smithi080 ceph-mon[29152]: Detected new or changed devices on smithi134 2024-04-24T23:23:26.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:25 smithi080 ceph-mon[29152]: Adjusting osd_memory_target on smithi134 to 8544M 2024-04-24T23:23:26.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:25 smithi080 ceph-mon[29152]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:23:27.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:26 smithi134 ceph-mon[32619]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:23:27.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:26 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:23:27.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:26 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:23:27.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:26 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:23:27.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:26 smithi148 ceph-mon[32528]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:23:27.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:26 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:23:27.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:26 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:23:27.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:26 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:23:27.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:26 smithi080 ceph-mon[29152]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:23:27.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:26 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:23:27.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:26 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:23:27.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:26 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:23:28.190 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:27 smithi080 ceph-mon[29152]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:23:28.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:27 smithi134 ceph-mon[32619]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:23:28.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:27 smithi148 ceph-mon[32528]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:23:29.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:28 smithi134 ceph-mon[32619]: from='client.? 172.21.15.148:0/2561789945' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b5491337-0694-440c-b679-312f3d14cf65"}]: dispatch 2024-04-24T23:23:29.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:28 smithi134 ceph-mon[32619]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b5491337-0694-440c-b679-312f3d14cf65"}]: dispatch 2024-04-24T23:23:29.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:28 smithi134 ceph-mon[32619]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b5491337-0694-440c-b679-312f3d14cf65"}]': finished 2024-04-24T23:23:29.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:28 smithi134 ceph-mon[32619]: osdmap e35: 6 total, 5 up, 6 in 2024-04-24T23:23:29.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:28 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T23:23:29.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:28 smithi148 ceph-mon[32528]: from='client.? 172.21.15.148:0/2561789945' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b5491337-0694-440c-b679-312f3d14cf65"}]: dispatch 2024-04-24T23:23:29.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:28 smithi148 ceph-mon[32528]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b5491337-0694-440c-b679-312f3d14cf65"}]: dispatch 2024-04-24T23:23:29.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:28 smithi148 ceph-mon[32528]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b5491337-0694-440c-b679-312f3d14cf65"}]': finished 2024-04-24T23:23:29.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:28 smithi148 ceph-mon[32528]: osdmap e35: 6 total, 5 up, 6 in 2024-04-24T23:23:29.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:28 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T23:23:29.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:28 smithi080 ceph-mon[29152]: from='client.? 172.21.15.148:0/2561789945' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b5491337-0694-440c-b679-312f3d14cf65"}]: dispatch 2024-04-24T23:23:29.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:28 smithi080 ceph-mon[29152]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b5491337-0694-440c-b679-312f3d14cf65"}]: dispatch 2024-04-24T23:23:29.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:28 smithi080 ceph-mon[29152]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b5491337-0694-440c-b679-312f3d14cf65"}]': finished 2024-04-24T23:23:29.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:28 smithi080 ceph-mon[29152]: osdmap e35: 6 total, 5 up, 6 in 2024-04-24T23:23:29.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:28 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T23:23:30.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:29 smithi134 ceph-mon[32619]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:23:30.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:29 smithi134 ceph-mon[32619]: from='client.? 172.21.15.148:0/1746932138' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:23:30.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:29 smithi148 ceph-mon[32528]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:23:30.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:29 smithi148 ceph-mon[32528]: from='client.? 172.21.15.148:0/1746932138' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:23:30.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:29 smithi080 ceph-mon[29152]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:23:30.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:29 smithi080 ceph-mon[29152]: from='client.? 172.21.15.148:0/1746932138' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:23:32.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:31 smithi134 ceph-mon[32619]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:23:32.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:31 smithi148 ceph-mon[32528]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:23:32.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:31 smithi080 ceph-mon[29152]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:23:34.007 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:33 smithi148 ceph-mon[32528]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:23:34.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:33 smithi134 ceph-mon[32619]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:23:34.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:33 smithi080 ceph-mon[29152]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:23:36.006 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:35 smithi148 ceph-mon[32528]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:23:36.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:35 smithi134 ceph-mon[32619]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:23:36.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:35 smithi080 ceph-mon[29152]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:23:37.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:36 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-24T23:23:37.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:36 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:23:37.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:36 smithi134 ceph-mon[32619]: Deploying daemon osd.5 on smithi148 2024-04-24T23:23:37.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:36 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-24T23:23:37.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:36 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:23:37.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:36 smithi148 ceph-mon[32528]: Deploying daemon osd.5 on smithi148 2024-04-24T23:23:37.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:36 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-24T23:23:37.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:36 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:23:37.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:36 smithi080 ceph-mon[29152]: Deploying daemon osd.5 on smithi148 2024-04-24T23:23:38.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:37 smithi134 ceph-mon[32619]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:23:38.240 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:37 smithi148 ceph-mon[32528]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:23:38.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:37 smithi080 ceph-mon[29152]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:23:40.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:39 smithi134 ceph-mon[32619]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:23:40.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:39 smithi148 ceph-mon[32528]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:23:40.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:39 smithi080 ceph-mon[29152]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:23:42.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:41 smithi134 ceph-mon[32619]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:23:42.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:41 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:23:42.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:41 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:23:42.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:41 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:23:42.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:41 smithi148 ceph-mon[32528]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:23:42.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:41 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:23:42.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:41 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:23:42.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:41 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:23:42.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:41 smithi080 ceph-mon[29152]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:23:42.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:41 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:23:42.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:41 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:23:42.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:41 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:23:44.004 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:43 smithi148 ceph-mon[32528]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:23:44.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:43 smithi134 ceph-mon[32619]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:23:44.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:43 smithi080 ceph-mon[29152]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:23:45.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:23:45.744+0000 7ff2757fa700 1 -- 172.21.15.148:0/900950477 <== mgr.14152 v2:172.21.15.80:6800/3830596922 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff2780fe8d0 con 0x7ff2600a2c20 2024-04-24T23:23:45.746 INFO:teuthology.orchestra.run.smithi148.stdout:Created osd(s) 5 on host 'smithi148' 2024-04-24T23:23:45.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:23:45.747+0000 7ff27e643700 1 -- 172.21.15.148:0/900950477 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7ff2600a2c20 msgr2=0x7ff2600a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:45.749 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:23:45.747+0000 7ff27e643700 1 --2- 172.21.15.148:0/900950477 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7ff2600a2c20 0x7ff2600a50e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff26c00b5c0 tx=0x7ff26c005fd0 comp rx=0 tx=0).stop 2024-04-24T23:23:45.749 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:23:45.747+0000 7ff27e643700 1 -- 172.21.15.148:0/900950477 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff278101190 msgr2=0x7ff278198520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:45.749 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:23:45.747+0000 7ff27e643700 1 --2- 172.21.15.148:0/900950477 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff278101190 0x7ff278198520 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7ff26800b530 tx=0x7ff26800b840 comp rx=0 tx=0).stop 2024-04-24T23:23:45.749 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:23:45.747+0000 7ff27e643700 1 -- 172.21.15.148:0/900950477 shutdown_connections 2024-04-24T23:23:45.749 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:23:45.747+0000 7ff27e643700 1 --2- 172.21.15.148:0/900950477 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7ff2600a2c20 0x7ff2600a50e0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:45.749 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:23:45.747+0000 7ff27e643700 1 --2- 172.21.15.148:0/900950477 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff278102d30 0x7ff27819fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:45.749 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:23:45.747+0000 7ff27e643700 1 --2- 172.21.15.148:0/900950477 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff278101190 0x7ff278198520 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:45.749 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:23:45.747+0000 7ff27e643700 1 --2- 172.21.15.148:0/900950477 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff278102390 0x7ff278198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:45.749 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:23:45.747+0000 7ff27e643700 1 -- 172.21.15.148:0/900950477 >> 172.21.15.148:0/900950477 conn(0x7ff2780fc750 msgr2=0x7ff2780fe1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:45.749 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:23:45.747+0000 7ff27e643700 1 -- 172.21.15.148:0/900950477 shutdown_connections 2024-04-24T23:23:45.749 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:23:45.747+0000 7ff27e643700 1 -- 172.21.15.148:0/900950477 wait complete. 2024-04-24T23:23:45.948 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:45 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:23:45.948 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:45 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:23:45.948 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:45 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:23:45.948 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:45 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:23:45.948 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:45 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:23:45.948 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:45 smithi134 ceph-mon[32619]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:23:45.948 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:45 smithi134 ceph-mon[32619]: from='osd.5 [v2:172.21.15.148:6808/2715951281,v1:172.21.15.148:6809/2715951281]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-24T23:23:45.948 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:45 smithi134 ceph-mon[32619]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-24T23:23:46.006 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:45 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:23:46.007 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:45 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:23:46.007 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:45 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:23:46.007 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:45 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:23:46.007 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:45 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:23:46.007 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:45 smithi148 ceph-mon[32528]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:23:46.007 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:45 smithi148 ceph-mon[32528]: from='osd.5 [v2:172.21.15.148:6808/2715951281,v1:172.21.15.148:6809/2715951281]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-24T23:23:46.007 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:45 smithi148 ceph-mon[32528]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-24T23:23:46.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:45 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:23:46.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:45 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:23:46.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:45 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:23:46.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:45 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:23:46.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:45 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:23:46.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:45 smithi080 ceph-mon[29152]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:23:46.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:45 smithi080 ceph-mon[29152]: from='osd.5 [v2:172.21.15.148:6808/2715951281,v1:172.21.15.148:6809/2715951281]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-24T23:23:46.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:45 smithi080 ceph-mon[29152]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-24T23:23:46.424 DEBUG:teuthology.orchestra.run.smithi148:osd.5> sudo journalctl -f -n 0 -u ceph-ec7885aa-0290-11ef-bc93-c7b262605968@osd.5.service 2024-04-24T23:23:46.426 INFO:tasks.cephadm:Deploying osd.6 on smithi080 with /dev/vg_nvme/lv_2... 2024-04-24T23:23:46.426 DEBUG:teuthology.orchestra.run.smithi080:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ec7885aa-0290-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-24T23:23:46.676 INFO:teuthology.orchestra.run.smithi080.stderr:Inferring config /var/lib/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:23:46.788 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:46 smithi080 ceph-mon[29152]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-24T23:23:46.789 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:46 smithi080 ceph-mon[29152]: osdmap e36: 6 total, 5 up, 6 in 2024-04-24T23:23:46.789 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:46 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T23:23:46.789 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:46 smithi080 ceph-mon[29152]: from='osd.5 [v2:172.21.15.148:6808/2715951281,v1:172.21.15.148:6809/2715951281]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-04-24T23:23:46.789 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:46 smithi080 ceph-mon[29152]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-04-24T23:23:46.789 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:46 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:23:46.789 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:46 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:23:46.789 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:46 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:23:46.947 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:46 smithi134 ceph-mon[32619]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-24T23:23:46.947 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:46 smithi134 ceph-mon[32619]: osdmap e36: 6 total, 5 up, 6 in 2024-04-24T23:23:46.948 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:46 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T23:23:46.948 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:46 smithi134 ceph-mon[32619]: from='osd.5 [v2:172.21.15.148:6808/2715951281,v1:172.21.15.148:6809/2715951281]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-04-24T23:23:46.948 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:46 smithi134 ceph-mon[32619]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-04-24T23:23:46.948 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:46 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:23:46.948 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:46 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:23:46.948 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:46 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:23:46.956 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:46 smithi148 ceph-mon[32528]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-24T23:23:46.956 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:46 smithi148 ceph-mon[32528]: osdmap e36: 6 total, 5 up, 6 in 2024-04-24T23:23:46.956 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:46 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T23:23:46.956 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:46 smithi148 ceph-mon[32528]: from='osd.5 [v2:172.21.15.148:6808/2715951281,v1:172.21.15.148:6809/2715951281]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-04-24T23:23:46.956 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:46 smithi148 ceph-mon[32528]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-04-24T23:23:46.956 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:46 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:23:46.956 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:46 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:23:46.956 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:46 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:23:46.957 INFO:journalctl@ceph.osd.5.smithi148.stdout:Apr 24 23:23:46 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-5[44694]: 2024-04-24T23:23:46.683+0000 7f6fd23d7700 -1 osd.5 0 waiting for initial osdmap 2024-04-24T23:23:46.957 INFO:journalctl@ceph.osd.5.smithi148.stdout:Apr 24 23:23:46 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-5[44694]: 2024-04-24T23:23:46.689+0000 7f6fce1ca700 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T23:23:47.917 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:47 smithi148 ceph-mon[32528]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2024-04-24T23:23:47.917 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:47 smithi148 ceph-mon[32528]: osdmap e37: 6 total, 5 up, 6 in 2024-04-24T23:23:47.917 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:47 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T23:23:47.918 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:47 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T23:23:47.918 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:47 smithi148 ceph-mon[32528]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:23:47.947 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:47 smithi134 ceph-mon[32619]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2024-04-24T23:23:47.947 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:47 smithi134 ceph-mon[32619]: osdmap e37: 6 total, 5 up, 6 in 2024-04-24T23:23:47.948 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:47 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T23:23:47.948 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:47 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T23:23:47.948 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:47 smithi134 ceph-mon[32619]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:23:48.038 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:47 smithi080 ceph-mon[29152]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2024-04-24T23:23:48.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:47 smithi080 ceph-mon[29152]: osdmap e37: 6 total, 5 up, 6 in 2024-04-24T23:23:48.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:47 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T23:23:48.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:47 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T23:23:48.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:47 smithi080 ceph-mon[29152]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:23:48.823 INFO:teuthology.orchestra.run.smithi080.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-24T23:23:48.823 INFO:teuthology.orchestra.run.smithi080.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-24T23:23:48.823 INFO:teuthology.orchestra.run.smithi080.stderr: stderr: 10+0 records in 2024-04-24T23:23:48.823 INFO:teuthology.orchestra.run.smithi080.stderr:10+0 records out 2024-04-24T23:23:48.823 INFO:teuthology.orchestra.run.smithi080.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0115063 s, 911 MB/s 2024-04-24T23:23:48.823 INFO:teuthology.orchestra.run.smithi080.stderr:--> Zapping successful for: 2024-04-24T23:23:48.946 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:48 smithi148 ceph-mon[32528]: purged_snaps scrub starts 2024-04-24T23:23:48.946 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:48 smithi148 ceph-mon[32528]: purged_snaps scrub ok 2024-04-24T23:23:48.946 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:48 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T23:23:48.946 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:48 smithi148 ceph-mon[32528]: osd.5 [v2:172.21.15.148:6808/2715951281,v1:172.21.15.148:6809/2715951281] boot 2024-04-24T23:23:48.946 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:48 smithi148 ceph-mon[32528]: osdmap e38: 6 total, 6 up, 6 in 2024-04-24T23:23:48.947 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:48 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T23:23:48.947 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:48 smithi148 ceph-mon[32528]: osdmap e39: 6 total, 6 up, 6 in 2024-04-24T23:23:48.947 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:48 smithi134 ceph-mon[32619]: purged_snaps scrub starts 2024-04-24T23:23:48.947 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:48 smithi134 ceph-mon[32619]: purged_snaps scrub ok 2024-04-24T23:23:48.947 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:48 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T23:23:48.947 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:48 smithi134 ceph-mon[32619]: osd.5 [v2:172.21.15.148:6808/2715951281,v1:172.21.15.148:6809/2715951281] boot 2024-04-24T23:23:48.947 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:48 smithi134 ceph-mon[32619]: osdmap e38: 6 total, 6 up, 6 in 2024-04-24T23:23:48.947 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:48 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T23:23:48.948 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:48 smithi134 ceph-mon[32619]: osdmap e39: 6 total, 6 up, 6 in 2024-04-24T23:23:49.038 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:48 smithi080 ceph-mon[29152]: purged_snaps scrub starts 2024-04-24T23:23:49.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:48 smithi080 ceph-mon[29152]: purged_snaps scrub ok 2024-04-24T23:23:49.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:48 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T23:23:49.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:48 smithi080 ceph-mon[29152]: osd.5 [v2:172.21.15.148:6808/2715951281,v1:172.21.15.148:6809/2715951281] boot 2024-04-24T23:23:49.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:48 smithi080 ceph-mon[29152]: osdmap e38: 6 total, 6 up, 6 in 2024-04-24T23:23:49.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:48 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T23:23:49.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:48 smithi080 ceph-mon[29152]: osdmap e39: 6 total, 6 up, 6 in 2024-04-24T23:23:49.483 DEBUG:teuthology.orchestra.run.smithi080:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ec7885aa-0290-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi080:vg_nvme/lv_2 2024-04-24T23:23:49.724 INFO:teuthology.orchestra.run.smithi080.stderr:Inferring config /var/lib/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:23:50.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:50 smithi080 ceph-mon[29152]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:23:50.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:50 smithi080 ceph-mon[29152]: osdmap e40: 6 total, 6 up, 6 in 2024-04-24T23:23:50.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:50 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:23:50.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:50 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:23:50.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:50 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:23:50.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:50 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:23:50.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:50 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:23:50.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:50 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:23:50.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:50 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:23:50.290 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:50 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:23:50.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:50 smithi134 ceph-mon[32619]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:23:50.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:50 smithi134 ceph-mon[32619]: osdmap e40: 6 total, 6 up, 6 in 2024-04-24T23:23:50.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:50 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:23:50.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:50 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:23:50.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:50 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:23:50.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:50 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:23:50.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:50 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:23:50.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:50 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:23:50.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:50 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:23:50.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:50 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:23:50.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:50 smithi148 ceph-mon[32528]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:23:50.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:50 smithi148 ceph-mon[32528]: osdmap e40: 6 total, 6 up, 6 in 2024-04-24T23:23:50.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:50 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:23:50.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:50 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:23:50.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:50 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:23:50.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:50 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:23:50.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:50 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:23:50.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:50 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:23:50.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:50 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:23:50.508 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:50 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:23:51.225 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:51 smithi080 ceph-mon[29152]: Detected new or changed devices on smithi148 2024-04-24T23:23:51.225 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:51 smithi080 ceph-mon[29152]: Adjusting osd_memory_target on smithi148 to 8544M 2024-04-24T23:23:51.226 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:51 smithi080 ceph-mon[29152]: osdmap e41: 6 total, 6 up, 6 in 2024-04-24T23:23:51.342 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:23:51.327+0000 7f36c8ab9700 1 -- 172.21.15.80:0/1556477734 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f36c4100320 msgr2=0x7f36c4100780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:51.342 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:23:51.327+0000 7f36c8ab9700 1 --2- 172.21.15.80:0/1556477734 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f36c4100320 0x7f36c4100780 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f36b4009990 tx=0x7f36b4009ca0 comp rx=0 tx=0).stop 2024-04-24T23:23:51.343 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:23:51.327+0000 7f36c8ab9700 1 -- 172.21.15.80:0/1556477734 shutdown_connections 2024-04-24T23:23:51.343 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:23:51.327+0000 7f36c8ab9700 1 --2- 172.21.15.80:0/1556477734 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36c4100cc0 0x7f36c410a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:51.343 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:23:51.327+0000 7f36c8ab9700 1 --2- 172.21.15.80:0/1556477734 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f36c41038a0 0x7f36c4103cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:51.343 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:23:51.327+0000 7f36c8ab9700 1 --2- 172.21.15.80:0/1556477734 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f36c4100320 0x7f36c4100780 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:51.343 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:23:51.327+0000 7f36c8ab9700 1 -- 172.21.15.80:0/1556477734 >> 172.21.15.80:0/1556477734 conn(0x7f36c40fbee0 msgr2=0x7f36c40fe320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:51.344 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:23:51.327+0000 7f36c8ab9700 1 -- 172.21.15.80:0/1556477734 shutdown_connections 2024-04-24T23:23:51.344 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:23:51.327+0000 7f36c8ab9700 1 -- 172.21.15.80:0/1556477734 wait complete. 2024-04-24T23:23:51.344 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:23:51.328+0000 7f36c8ab9700 1 Processor -- start 2024-04-24T23:23:51.344 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:23:51.328+0000 7f36c8ab9700 1 -- start start 2024-04-24T23:23:51.344 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:23:51.328+0000 7f36c8ab9700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36c4100320 0x7f36c4102fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:51.344 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:23:51.328+0000 7f36c8ab9700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f36c4100cc0 0x7f36c41015f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:51.344 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:23:51.328+0000 7f36c8ab9700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f36c41038a0 0x7f36c4101b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:51.345 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:23:51.328+0000 7f36c8ab9700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f36c410dea0 con 0x7f36c4100cc0 2024-04-24T23:23:51.345 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:23:51.328+0000 7f36c8ab9700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f36c410dd20 con 0x7f36c41038a0 2024-04-24T23:23:51.345 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:23:51.328+0000 7f36c8ab9700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f36c410e020 con 0x7f36c4100320 2024-04-24T23:23:51.345 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:23:51.328+0000 7f36c1d9b700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f36c4100cc0 0x7f36c41015f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:51.345 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:23:51.328+0000 7f36c1d9b700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f36c4100cc0 0x7f36c41015f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:37072/0 (socket says 172.21.15.80:37072) 2024-04-24T23:23:51.345 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:23:51.328+0000 7f36c1d9b700 1 -- 172.21.15.80:0/380651319 learned_addr learned my addr 172.21.15.80:0/380651319 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:23:51.346 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:23:51.328+0000 7f36c259c700 1 --2- 172.21.15.80:0/380651319 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36c4100320 0x7f36c4102fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:51.346 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:23:51.329+0000 7f36c2d9d700 1 --2- 172.21.15.80:0/380651319 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f36c41038a0 0x7f36c4101b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:51.346 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:23:51.329+0000 7f36c1d9b700 1 -- 172.21.15.80:0/380651319 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36c4100320 msgr2=0x7f36c4102fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:51.346 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:23:51.329+0000 7f36c1d9b700 1 --2- 172.21.15.80:0/380651319 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36c4100320 0x7f36c4102fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:51.346 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:23:51.329+0000 7f36c1d9b700 1 -- 172.21.15.80:0/380651319 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f36c41038a0 msgr2=0x7f36c4101b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:51.346 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:23:51.329+0000 7f36c1d9b700 1 --2- 172.21.15.80:0/380651319 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f36c41038a0 0x7f36c4101b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:51.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:23:51.329+0000 7f36c1d9b700 1 -- 172.21.15.80:0/380651319 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36c41023e0 con 0x7f36c4100cc0 2024-04-24T23:23:51.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:23:51.329+0000 7f36c1d9b700 1 --2- 172.21.15.80:0/380651319 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f36c4100cc0 0x7f36c41015f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f36b400bff0 tx=0x7f36b4009f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:51.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:23:51.329+0000 7f36b37fe700 1 -- 172.21.15.80:0/380651319 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36b4026070 con 0x7f36c4100cc0 2024-04-24T23:23:51.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:23:51.329+0000 7f36c8ab9700 1 -- 172.21.15.80:0/380651319 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36c41a2520 con 0x7f36c4100cc0 2024-04-24T23:23:51.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:23:51.329+0000 7f36b37fe700 1 -- 172.21.15.80:0/380651319 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36b40047e0 con 0x7f36c4100cc0 2024-04-24T23:23:51.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:23:51.329+0000 7f36b37fe700 1 -- 172.21.15.80:0/380651319 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36b4005460 con 0x7f36c4100cc0 2024-04-24T23:23:51.348 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:23:51.329+0000 7f36c8ab9700 1 -- 172.21.15.80:0/380651319 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36c41a2970 con 0x7f36c4100cc0 2024-04-24T23:23:51.348 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:23:51.330+0000 7f36c8ab9700 1 -- 172.21.15.80:0/380651319 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36c4068f20 con 0x7f36c4100cc0 2024-04-24T23:23:51.348 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:23:51.331+0000 7f36b37fe700 1 -- 172.21.15.80:0/380651319 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f36b4029070 con 0x7f36c4100cc0 2024-04-24T23:23:51.348 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:23:51.331+0000 7f36b37fe700 1 --2- 172.21.15.80:0/380651319 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f36ac0a2c20 0x7f36ac0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:51.348 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:23:51.332+0000 7f36c259c700 1 --2- 172.21.15.80:0/380651319 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f36ac0a2c20 0x7f36ac0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:51.348 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:23:51.332+0000 7f36b37fe700 1 -- 172.21.15.80:0/380651319 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4533+0+0 (secure 0 0 0) 0x7f36b40cf240 con 0x7f36c4100cc0 2024-04-24T23:23:51.349 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:23:51.332+0000 7f36c259c700 1 --2- 172.21.15.80:0/380651319 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f36ac0a2c20 0x7f36ac0a50e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f36c406a6c0 tx=0x7f36b800a3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:51.349 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:23:51.335+0000 7f36b37fe700 1 -- 172.21.15.80:0/380651319 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f36b4030020 con 0x7f36c4100cc0 2024-04-24T23:23:51.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:51 smithi134 ceph-mon[32619]: Detected new or changed devices on smithi148 2024-04-24T23:23:51.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:51 smithi134 ceph-mon[32619]: Adjusting osd_memory_target on smithi148 to 8544M 2024-04-24T23:23:51.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:51 smithi134 ceph-mon[32619]: osdmap e41: 6 total, 6 up, 6 in 2024-04-24T23:23:51.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:51 smithi148 ceph-mon[32528]: Detected new or changed devices on smithi148 2024-04-24T23:23:51.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:51 smithi148 ceph-mon[32528]: Adjusting osd_memory_target on smithi148 to 8544M 2024-04-24T23:23:51.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:51 smithi148 ceph-mon[32528]: osdmap e41: 6 total, 6 up, 6 in 2024-04-24T23:23:51.544 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:23:51.542+0000 7f36c8ab9700 1 -- 172.21.15.80:0/380651319 --> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi080:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f36c4062670 con 0x7f36ac0a2c20 2024-04-24T23:23:52.429 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:52 smithi080 ceph-mon[29152]: pgmap v143: 1 pgs: 1 remapped+peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:23:52.429 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:52 smithi080 ceph-mon[29152]: osdmap e42: 6 total, 6 up, 6 in 2024-04-24T23:23:52.429 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:52 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:23:52.430 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:52 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:23:52.430 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:52 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:23:52.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:52 smithi134 ceph-mon[32619]: pgmap v143: 1 pgs: 1 remapped+peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:23:52.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:52 smithi134 ceph-mon[32619]: osdmap e42: 6 total, 6 up, 6 in 2024-04-24T23:23:52.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:52 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:23:52.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:52 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:23:52.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:52 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:23:52.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:52 smithi148 ceph-mon[32528]: pgmap v143: 1 pgs: 1 remapped+peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:23:52.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:52 smithi148 ceph-mon[32528]: osdmap e42: 6 total, 6 up, 6 in 2024-04-24T23:23:52.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:52 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:23:52.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:52 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:23:52.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:52 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:23:53.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:53 smithi134 ceph-mon[32619]: from='client.14361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi080:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:23:53.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:53 smithi148 ceph-mon[32528]: from='client.14361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi080:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:23:53.507 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:53 smithi080 ceph-mon[29152]: from='client.14361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi080:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:23:54.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:54 smithi134 ceph-mon[32619]: pgmap v145: 1 pgs: 1 remapped+peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:23:54.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:54 smithi148 ceph-mon[32528]: pgmap v145: 1 pgs: 1 remapped+peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:23:54.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:54 smithi080 ceph-mon[29152]: pgmap v145: 1 pgs: 1 remapped+peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:23:55.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:55 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/1955135313' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ba4b5df7-d86e-47cd-ab67-cf763c4e4d6c"}]: dispatch 2024-04-24T23:23:55.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:55 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/1955135313' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ba4b5df7-d86e-47cd-ab67-cf763c4e4d6c"}]': finished 2024-04-24T23:23:55.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:55 smithi134 ceph-mon[32619]: osdmap e43: 7 total, 6 up, 7 in 2024-04-24T23:23:55.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:55 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T23:23:55.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:55 smithi134 ceph-mon[32619]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-04-24T23:23:55.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:55 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/3765749459' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:23:55.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:55 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/1955135313' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ba4b5df7-d86e-47cd-ab67-cf763c4e4d6c"}]: dispatch 2024-04-24T23:23:55.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:55 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/1955135313' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ba4b5df7-d86e-47cd-ab67-cf763c4e4d6c"}]': finished 2024-04-24T23:23:55.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:55 smithi148 ceph-mon[32528]: osdmap e43: 7 total, 6 up, 7 in 2024-04-24T23:23:55.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:55 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T23:23:55.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:55 smithi148 ceph-mon[32528]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-04-24T23:23:55.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:55 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/3765749459' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:23:55.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:55 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/1955135313' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ba4b5df7-d86e-47cd-ab67-cf763c4e4d6c"}]: dispatch 2024-04-24T23:23:55.539 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:55 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/1955135313' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ba4b5df7-d86e-47cd-ab67-cf763c4e4d6c"}]': finished 2024-04-24T23:23:55.539 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:55 smithi080 ceph-mon[29152]: osdmap e43: 7 total, 6 up, 7 in 2024-04-24T23:23:55.539 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:55 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T23:23:55.539 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:55 smithi080 ceph-mon[29152]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-04-24T23:23:55.539 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:55 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/3765749459' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:23:58.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:57 smithi134 ceph-mon[32619]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 90 KiB/s, 0 objects/s recovering 2024-04-24T23:23:58.243 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:57 smithi080 ceph-mon[29152]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 90 KiB/s, 0 objects/s recovering 2024-04-24T23:23:58.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:57 smithi148 ceph-mon[32528]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 90 KiB/s, 0 objects/s recovering 2024-04-24T23:24:00.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:23:59 smithi080 ceph-mon[29152]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-24T23:24:00.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:23:59 smithi134 ceph-mon[32619]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-24T23:24:00.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:23:59 smithi148 ceph-mon[32528]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-24T23:24:02.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:01 smithi134 ceph-mon[32619]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-04-24T23:24:02.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:01 smithi148 ceph-mon[32528]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-04-24T23:24:02.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:01 smithi080 ceph-mon[29152]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-04-24T23:24:03.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:02 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-24T23:24:03.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:02 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:24:03.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:02 smithi134 ceph-mon[32619]: Deploying daemon osd.6 on smithi080 2024-04-24T23:24:03.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:02 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-24T23:24:03.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:02 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:24:03.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:02 smithi148 ceph-mon[32528]: Deploying daemon osd.6 on smithi080 2024-04-24T23:24:03.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:02 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-24T23:24:03.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:02 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:24:03.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:02 smithi080 ceph-mon[29152]: Deploying daemon osd.6 on smithi080 2024-04-24T23:24:04.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:03 smithi134 ceph-mon[32619]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-24T23:24:04.217 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:03 smithi080 ceph-mon[29152]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-24T23:24:04.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:03 smithi148 ceph-mon[32528]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-24T23:24:06.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:05 smithi134 ceph-mon[32619]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:24:06.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:05 smithi148 ceph-mon[32528]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:24:06.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:05 smithi080 ceph-mon[29152]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:24:08.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:07 smithi134 ceph-mon[32619]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:24:08.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:07 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:24:08.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:07 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:24:08.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:07 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:24:08.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:07 smithi148 ceph-mon[32528]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:24:08.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:07 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:24:08.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:07 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:24:08.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:07 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:24:08.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:07 smithi080 ceph-mon[29152]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:24:08.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:07 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:24:08.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:07 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:24:08.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:07 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:24:10.038 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:09 smithi080 ceph-mon[29152]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:24:10.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:09 smithi134 ceph-mon[32619]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:24:10.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:09 smithi148 ceph-mon[32528]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:24:11.654 INFO:teuthology.orchestra.run.smithi080.stdout:Created osd(s) 6 on host 'smithi080' 2024-04-24T23:24:11.654 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:24:11.652+0000 7f36b37fe700 1 -- 172.21.15.80:0/380651319 <== mgr.14152 v2:172.21.15.80:6800/3830596922 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f36c4062670 con 0x7f36ac0a2c20 2024-04-24T23:24:11.656 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:24:11.655+0000 7f36c8ab9700 1 -- 172.21.15.80:0/380651319 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f36ac0a2c20 msgr2=0x7f36ac0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:11.656 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:24:11.655+0000 7f36c8ab9700 1 --2- 172.21.15.80:0/380651319 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f36ac0a2c20 0x7f36ac0a50e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f36c406a6c0 tx=0x7f36b800a3b0 comp rx=0 tx=0).stop 2024-04-24T23:24:11.657 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:24:11.655+0000 7f36c8ab9700 1 -- 172.21.15.80:0/380651319 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f36c4100cc0 msgr2=0x7f36c41015f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:11.657 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:24:11.655+0000 7f36c8ab9700 1 --2- 172.21.15.80:0/380651319 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f36c4100cc0 0x7f36c41015f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f36b400bff0 tx=0x7f36b4009f50 comp rx=0 tx=0).stop 2024-04-24T23:24:11.657 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:24:11.655+0000 7f36c8ab9700 1 -- 172.21.15.80:0/380651319 shutdown_connections 2024-04-24T23:24:11.657 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:24:11.655+0000 7f36c8ab9700 1 --2- 172.21.15.80:0/380651319 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f36c41038a0 0x7f36c4101b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:11.657 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:24:11.655+0000 7f36c8ab9700 1 --2- 172.21.15.80:0/380651319 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36c4100320 0x7f36c4102fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:11.657 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:24:11.655+0000 7f36c8ab9700 1 --2- 172.21.15.80:0/380651319 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f36ac0a2c20 0x7f36ac0a50e0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:11.657 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:24:11.655+0000 7f36c8ab9700 1 --2- 172.21.15.80:0/380651319 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f36c4100cc0 0x7f36c41015f0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:11.657 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:24:11.655+0000 7f36c8ab9700 1 -- 172.21.15.80:0/380651319 >> 172.21.15.80:0/380651319 conn(0x7f36c40fbee0 msgr2=0x7f36c4106870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:11.657 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:24:11.655+0000 7f36c8ab9700 1 -- 172.21.15.80:0/380651319 shutdown_connections 2024-04-24T23:24:11.657 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:24:11.655+0000 7f36c8ab9700 1 -- 172.21.15.80:0/380651319 wait complete. 2024-04-24T23:24:11.979 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:11 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:24:11.979 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:11 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:24:11.979 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:11 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:24:11.979 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:11 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:24:11.979 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:11 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:24:11.980 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:11 smithi080 ceph-mon[29152]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:24:11.980 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:11 smithi080 ceph-mon[29152]: from='osd.6 [v2:172.21.15.80:6818/953261724,v1:172.21.15.80:6819/953261724]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-24T23:24:11.980 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:11 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:24:11.980 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:11 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:24:11.980 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:11 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:24:12.006 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:11 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:24:12.007 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:11 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:24:12.007 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:11 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:24:12.007 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:11 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:24:12.007 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:11 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:24:12.007 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:11 smithi148 ceph-mon[32528]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:24:12.007 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:11 smithi148 ceph-mon[32528]: from='osd.6 [v2:172.21.15.80:6818/953261724,v1:172.21.15.80:6819/953261724]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-24T23:24:12.007 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:11 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:24:12.007 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:11 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:24:12.007 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:11 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:24:12.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:11 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:24:12.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:11 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:24:12.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:11 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:24:12.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:11 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:24:12.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:11 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:24:12.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:11 smithi134 ceph-mon[32619]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:24:12.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:11 smithi134 ceph-mon[32619]: from='osd.6 [v2:172.21.15.80:6818/953261724,v1:172.21.15.80:6819/953261724]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-24T23:24:12.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:11 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:24:12.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:11 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:24:12.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:11 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:24:12.323 DEBUG:teuthology.orchestra.run.smithi080:osd.6> sudo journalctl -f -n 0 -u ceph-ec7885aa-0290-11ef-bc93-c7b262605968@osd.6.service 2024-04-24T23:24:12.326 INFO:tasks.cephadm:Deploying osd.7 on smithi134 with /dev/vg_nvme/lv_2... 2024-04-24T23:24:12.326 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ec7885aa-0290-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-24T23:24:12.574 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/mon.b/config 2024-04-24T23:24:12.901 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:12 smithi080 ceph-mon[29152]: from='osd.6 [v2:172.21.15.80:6818/953261724,v1:172.21.15.80:6819/953261724]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-24T23:24:12.901 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:12 smithi080 ceph-mon[29152]: osdmap e44: 7 total, 6 up, 7 in 2024-04-24T23:24:12.901 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:12 smithi080 ceph-mon[29152]: from='osd.6 [v2:172.21.15.80:6818/953261724,v1:172.21.15.80:6819/953261724]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi080", "root=default"]}]: dispatch 2024-04-24T23:24:12.901 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:12 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T23:24:12.948 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:12 smithi134 ceph-mon[32619]: from='osd.6 [v2:172.21.15.80:6818/953261724,v1:172.21.15.80:6819/953261724]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-24T23:24:12.948 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:12 smithi134 ceph-mon[32619]: osdmap e44: 7 total, 6 up, 7 in 2024-04-24T23:24:12.948 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:12 smithi134 ceph-mon[32619]: from='osd.6 [v2:172.21.15.80:6818/953261724,v1:172.21.15.80:6819/953261724]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi080", "root=default"]}]: dispatch 2024-04-24T23:24:12.948 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:12 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T23:24:13.007 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:12 smithi148 ceph-mon[32528]: from='osd.6 [v2:172.21.15.80:6818/953261724,v1:172.21.15.80:6819/953261724]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-24T23:24:13.007 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:12 smithi148 ceph-mon[32528]: osdmap e44: 7 total, 6 up, 7 in 2024-04-24T23:24:13.007 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:12 smithi148 ceph-mon[32528]: from='osd.6 [v2:172.21.15.80:6818/953261724,v1:172.21.15.80:6819/953261724]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi080", "root=default"]}]: dispatch 2024-04-24T23:24:13.007 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:12 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T23:24:13.289 INFO:journalctl@ceph.osd.6.smithi080.stdout:Apr 24 23:24:13 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-6[57095]: 2024-04-24T23:24:13.126+0000 7fee72af4700 -1 osd.6 0 waiting for initial osdmap 2024-04-24T23:24:13.289 INFO:journalctl@ceph.osd.6.smithi080.stdout:Apr 24 23:24:13 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-6[57095]: 2024-04-24T23:24:13.134+0000 7fee6a8df700 -1 osd.6 45 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T23:24:13.936 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:13 smithi080 ceph-mon[29152]: from='osd.6 [v2:172.21.15.80:6818/953261724,v1:172.21.15.80:6819/953261724]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi080", "root=default"]}]': finished 2024-04-24T23:24:13.937 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:13 smithi080 ceph-mon[29152]: osdmap e45: 7 total, 6 up, 7 in 2024-04-24T23:24:13.937 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:13 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T23:24:13.937 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:13 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T23:24:13.937 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:13 smithi080 ceph-mon[29152]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:24:13.937 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:13 smithi080 ceph-mon[29152]: from='osd.6 [v2:172.21.15.80:6818/953261724,v1:172.21.15.80:6819/953261724]' entity='osd.6' 2024-04-24T23:24:14.001 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:13 smithi134 ceph-mon[32619]: from='osd.6 [v2:172.21.15.80:6818/953261724,v1:172.21.15.80:6819/953261724]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi080", "root=default"]}]': finished 2024-04-24T23:24:14.001 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:13 smithi134 ceph-mon[32619]: osdmap e45: 7 total, 6 up, 7 in 2024-04-24T23:24:14.001 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:13 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T23:24:14.001 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:13 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T23:24:14.001 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:13 smithi134 ceph-mon[32619]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:24:14.001 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:13 smithi134 ceph-mon[32619]: from='osd.6 [v2:172.21.15.80:6818/953261724,v1:172.21.15.80:6819/953261724]' entity='osd.6' 2024-04-24T23:24:14.006 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:13 smithi148 ceph-mon[32528]: from='osd.6 [v2:172.21.15.80:6818/953261724,v1:172.21.15.80:6819/953261724]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi080", "root=default"]}]': finished 2024-04-24T23:24:14.006 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:13 smithi148 ceph-mon[32528]: osdmap e45: 7 total, 6 up, 7 in 2024-04-24T23:24:14.007 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:13 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T23:24:14.007 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:13 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T23:24:14.007 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:13 smithi148 ceph-mon[32528]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:24:14.007 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:13 smithi148 ceph-mon[32528]: from='osd.6 [v2:172.21.15.80:6818/953261724,v1:172.21.15.80:6819/953261724]' entity='osd.6' 2024-04-24T23:24:14.742 INFO:teuthology.orchestra.run.smithi134.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-24T23:24:14.742 INFO:teuthology.orchestra.run.smithi134.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-24T23:24:14.742 INFO:teuthology.orchestra.run.smithi134.stderr: stderr: 10+0 records in 2024-04-24T23:24:14.742 INFO:teuthology.orchestra.run.smithi134.stderr:10+0 records out 2024-04-24T23:24:14.742 INFO:teuthology.orchestra.run.smithi134.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0119551 s, 877 MB/s 2024-04-24T23:24:14.742 INFO:teuthology.orchestra.run.smithi134.stderr:--> Zapping successful for: 2024-04-24T23:24:14.947 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:14 smithi134 ceph-mon[32619]: purged_snaps scrub starts 2024-04-24T23:24:14.947 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:14 smithi134 ceph-mon[32619]: purged_snaps scrub ok 2024-04-24T23:24:14.947 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:14 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T23:24:14.948 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:14 smithi134 ceph-mon[32619]: osd.6 [v2:172.21.15.80:6818/953261724,v1:172.21.15.80:6819/953261724] boot 2024-04-24T23:24:14.948 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:14 smithi134 ceph-mon[32619]: osdmap e46: 7 total, 7 up, 7 in 2024-04-24T23:24:14.948 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:14 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T23:24:15.006 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:14 smithi148 ceph-mon[32528]: purged_snaps scrub starts 2024-04-24T23:24:15.006 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:14 smithi148 ceph-mon[32528]: purged_snaps scrub ok 2024-04-24T23:24:15.007 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:14 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T23:24:15.007 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:14 smithi148 ceph-mon[32528]: osd.6 [v2:172.21.15.80:6818/953261724,v1:172.21.15.80:6819/953261724] boot 2024-04-24T23:24:15.007 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:14 smithi148 ceph-mon[32528]: osdmap e46: 7 total, 7 up, 7 in 2024-04-24T23:24:15.007 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:14 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T23:24:15.038 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:14 smithi080 ceph-mon[29152]: purged_snaps scrub starts 2024-04-24T23:24:15.038 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:14 smithi080 ceph-mon[29152]: purged_snaps scrub ok 2024-04-24T23:24:15.038 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:14 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T23:24:15.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:14 smithi080 ceph-mon[29152]: osd.6 [v2:172.21.15.80:6818/953261724,v1:172.21.15.80:6819/953261724] boot 2024-04-24T23:24:15.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:14 smithi080 ceph-mon[29152]: osdmap e46: 7 total, 7 up, 7 in 2024-04-24T23:24:15.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:14 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T23:24:15.424 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ec7885aa-0290-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi134:vg_nvme/lv_2 2024-04-24T23:24:15.664 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/mon.b/config 2024-04-24T23:24:16.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:16 smithi134 ceph-mon[32619]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:24:16.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:16 smithi134 ceph-mon[32619]: osdmap e47: 7 total, 7 up, 7 in 2024-04-24T23:24:16.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:16 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:24:16.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:16 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:24:16.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:16 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:24:16.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:16 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:24:16.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:16 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:24:16.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:16 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:24:16.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:16 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:24:16.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:16 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:24:16.449 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:16 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:24:16.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:16 smithi148 ceph-mon[32528]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:24:16.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:16 smithi148 ceph-mon[32528]: osdmap e47: 7 total, 7 up, 7 in 2024-04-24T23:24:16.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:16 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:24:16.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:16 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:24:16.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:16 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:24:16.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:16 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:24:16.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:16 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:24:16.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:16 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:24:16.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:16 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:24:16.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:16 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:24:16.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:16 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:24:16.539 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:16 smithi080 ceph-mon[29152]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:24:16.539 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:16 smithi080 ceph-mon[29152]: osdmap e47: 7 total, 7 up, 7 in 2024-04-24T23:24:16.539 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:16 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:24:16.539 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:16 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:24:16.539 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:16 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:24:16.539 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:16 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:24:16.539 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:16 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:24:16.539 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:16 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:24:16.539 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:16 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:24:16.540 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:16 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:24:16.540 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:16 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:24:17.326 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:24:17.320+0000 7f37e5e82700 1 -- 172.21.15.134:0/1796519217 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f37e0100580 msgr2=0x7f37e01009a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:17.327 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:24:17.320+0000 7f37e5e82700 1 --2- 172.21.15.134:0/1796519217 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f37e0100580 0x7f37e01009a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f37d00099c0 tx=0x7f37d0009cd0 comp rx=0 tx=0).stop 2024-04-24T23:24:17.327 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:24:17.320+0000 7f37e5e82700 1 -- 172.21.15.134:0/1796519217 shutdown_connections 2024-04-24T23:24:17.327 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:24:17.320+0000 7f37e5e82700 1 --2- 172.21.15.134:0/1796519217 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f37e0103fa0 0x7f37e010a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:17.327 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:24:17.320+0000 7f37e5e82700 1 --2- 172.21.15.134:0/1796519217 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f37e0100580 0x7f37e01009a0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:17.327 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:24:17.320+0000 7f37e5e82700 1 --2- 172.21.15.134:0/1796519217 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f37e0103600 0x7f37e0103a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:17.327 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:24:17.320+0000 7f37e5e82700 1 -- 172.21.15.134:0/1796519217 >> 172.21.15.134:0/1796519217 conn(0x7f37e00fbea0 msgr2=0x7f37e00fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:17.328 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:24:17.320+0000 7f37e5e82700 1 -- 172.21.15.134:0/1796519217 shutdown_connections 2024-04-24T23:24:17.328 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:24:17.320+0000 7f37e5e82700 1 -- 172.21.15.134:0/1796519217 wait complete. 2024-04-24T23:24:17.328 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:24:17.321+0000 7f37e5e82700 1 Processor -- start 2024-04-24T23:24:17.328 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:24:17.321+0000 7f37e5e82700 1 -- start start 2024-04-24T23:24:17.328 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:24:17.321+0000 7f37e5e82700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f37e0100580 0x7f37e0198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:17.328 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:24:17.321+0000 7f37e5e82700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f37e0103600 0x7f37e01989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:17.329 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:24:17.321+0000 7f37e5e82700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f37e0103fa0 0x7f37e019fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:17.329 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:24:17.321+0000 7f37e5e82700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f37e010de90 con 0x7f37e0100580 2024-04-24T23:24:17.329 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:24:17.321+0000 7f37e5e82700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f37e010dd10 con 0x7f37e0103fa0 2024-04-24T23:24:17.329 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:24:17.321+0000 7f37e5e82700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f37e010e010 con 0x7f37e0103600 2024-04-24T23:24:17.329 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:24:17.321+0000 7f37dffff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f37e0103fa0 0x7f37e019fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:17.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:24:17.321+0000 7f37dffff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f37e0103fa0 0x7f37e019fa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:45866/0 (socket says 172.21.15.134:45866) 2024-04-24T23:24:17.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:24:17.321+0000 7f37dffff700 1 -- 172.21.15.134:0/2651714082 learned_addr learned my addr 172.21.15.134:0/2651714082 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-04-24T23:24:17.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:24:17.321+0000 7f37deffd700 1 --2- 172.21.15.134:0/2651714082 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f37e0103600 0x7f37e01989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:17.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:24:17.322+0000 7f37df7fe700 1 --2- 172.21.15.134:0/2651714082 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f37e0100580 0x7f37e0198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:17.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:24:17.322+0000 7f37dffff700 1 -- 172.21.15.134:0/2651714082 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f37e0103600 msgr2=0x7f37e01989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:17.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:24:17.322+0000 7f37dffff700 1 --2- 172.21.15.134:0/2651714082 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f37e0103600 0x7f37e01989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:17.331 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:24:17.322+0000 7f37dffff700 1 -- 172.21.15.134:0/2651714082 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f37e0100580 msgr2=0x7f37e0198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:17.331 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:24:17.322+0000 7f37dffff700 1 --2- 172.21.15.134:0/2651714082 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f37e0100580 0x7f37e0198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:17.331 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:24:17.322+0000 7f37dffff700 1 -- 172.21.15.134:0/2651714082 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37e01a00e0 con 0x7f37e0103fa0 2024-04-24T23:24:17.331 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:24:17.322+0000 7f37df7fe700 1 --2- 172.21.15.134:0/2651714082 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f37e0100580 0x7f37e0198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:24:17.331 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:24:17.322+0000 7f37dffff700 1 --2- 172.21.15.134:0/2651714082 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f37e0103fa0 0x7f37e019fa40 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f37d8007ee0 tx=0x7f37d800d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:17.331 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:24:17.322+0000 7f37dcff9700 1 -- 172.21.15.134:0/2651714082 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d800dbd0 con 0x7f37e0103fa0 2024-04-24T23:24:17.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:24:17.322+0000 7f37e5e82700 1 -- 172.21.15.134:0/2651714082 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37e01a03d0 con 0x7f37e0103fa0 2024-04-24T23:24:17.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:24:17.322+0000 7f37dcff9700 1 -- 172.21.15.134:0/2651714082 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37d800dd70 con 0x7f37e0103fa0 2024-04-24T23:24:17.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:24:17.322+0000 7f37dcff9700 1 -- 172.21.15.134:0/2651714082 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d8005a40 con 0x7f37e0103fa0 2024-04-24T23:24:17.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:24:17.322+0000 7f37e5e82700 1 -- 172.21.15.134:0/2651714082 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37e01a08c0 con 0x7f37e0103fa0 2024-04-24T23:24:17.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:24:17.323+0000 7f37e5e82700 1 -- 172.21.15.134:0/2651714082 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37e0068f20 con 0x7f37e0103fa0 2024-04-24T23:24:17.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:24:17.324+0000 7f37dcff9700 1 -- 172.21.15.134:0/2651714082 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f37d801a020 con 0x7f37e0103fa0 2024-04-24T23:24:17.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:24:17.325+0000 7f37dcff9700 1 --2- 172.21.15.134:0/2651714082 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f37c80a2c20 0x7f37c80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:17.333 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:24:17.325+0000 7f37dcff9700 1 -- 172.21.15.134:0/2651714082 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 4965+0+0 (secure 0 0 0) 0x7f37d80ce170 con 0x7f37e0103fa0 2024-04-24T23:24:17.333 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:24:17.325+0000 7f37df7fe700 1 --2- 172.21.15.134:0/2651714082 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f37c80a2c20 0x7f37c80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:17.333 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:24:17.325+0000 7f37df7fe700 1 --2- 172.21.15.134:0/2651714082 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f37c80a2c20 0x7f37c80a50e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f37d000b580 tx=0x7f37d0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:17.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:24:17.328+0000 7f37dcff9700 1 -- 172.21.15.134:0/2651714082 <== mon.2 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f37d8010070 con 0x7f37e0103fa0 2024-04-24T23:24:17.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:17 smithi134 ceph-mon[32619]: Detected new or changed devices on smithi080 2024-04-24T23:24:17.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:17 smithi134 ceph-mon[32619]: Adjusting osd_memory_target on smithi080 to 5696M 2024-04-24T23:24:17.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:17 smithi148 ceph-mon[32528]: Detected new or changed devices on smithi080 2024-04-24T23:24:17.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:17 smithi148 ceph-mon[32528]: Adjusting osd_memory_target on smithi080 to 5696M 2024-04-24T23:24:17.533 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:24:17.533+0000 7f37e5e82700 1 -- 172.21.15.134:0/2651714082 --> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f37e0062670 con 0x7f37c80a2c20 2024-04-24T23:24:17.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:17 smithi080 ceph-mon[29152]: Detected new or changed devices on smithi080 2024-04-24T23:24:17.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:17 smithi080 ceph-mon[29152]: Adjusting osd_memory_target on smithi080 to 5696M 2024-04-24T23:24:18.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:18 smithi134 ceph-mon[32619]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:24:18.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:18 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:24:18.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:18 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:24:18.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:18 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:24:18.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:18 smithi148 ceph-mon[32528]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:24:18.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:18 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:24:18.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:18 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:24:18.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:18 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:24:18.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:18 smithi080 ceph-mon[29152]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:24:18.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:18 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:24:18.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:18 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:24:18.539 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:18 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:24:19.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:19 smithi134 ceph-mon[32619]: from='client.24275 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:24:19.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:19 smithi148 ceph-mon[32528]: from='client.24275 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:24:19.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:19 smithi080 ceph-mon[29152]: from='client.24275 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:24:20.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:20 smithi134 ceph-mon[32619]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:24:20.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:20 smithi148 ceph-mon[32528]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:24:20.539 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:20 smithi080 ceph-mon[29152]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:24:21.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:21 smithi134 ceph-mon[32619]: from='client.? 172.21.15.134:0/732360478' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "464a503a-75ae-49e7-ab6e-a7f9f2a6a307"}]: dispatch 2024-04-24T23:24:21.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:21 smithi134 ceph-mon[32619]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "464a503a-75ae-49e7-ab6e-a7f9f2a6a307"}]: dispatch 2024-04-24T23:24:21.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:21 smithi134 ceph-mon[32619]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "464a503a-75ae-49e7-ab6e-a7f9f2a6a307"}]': finished 2024-04-24T23:24:21.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:21 smithi134 ceph-mon[32619]: osdmap e48: 8 total, 7 up, 8 in 2024-04-24T23:24:21.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:21 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T23:24:21.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:21 smithi134 ceph-mon[32619]: from='client.? 172.21.15.134:0/376562134' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:24:21.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:21 smithi148 ceph-mon[32528]: from='client.? 172.21.15.134:0/732360478' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "464a503a-75ae-49e7-ab6e-a7f9f2a6a307"}]: dispatch 2024-04-24T23:24:21.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:21 smithi148 ceph-mon[32528]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "464a503a-75ae-49e7-ab6e-a7f9f2a6a307"}]: dispatch 2024-04-24T23:24:21.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:21 smithi148 ceph-mon[32528]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "464a503a-75ae-49e7-ab6e-a7f9f2a6a307"}]': finished 2024-04-24T23:24:21.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:21 smithi148 ceph-mon[32528]: osdmap e48: 8 total, 7 up, 8 in 2024-04-24T23:24:21.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:21 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T23:24:21.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:21 smithi148 ceph-mon[32528]: from='client.? 172.21.15.134:0/376562134' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:24:21.539 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:21 smithi080 ceph-mon[29152]: from='client.? 172.21.15.134:0/732360478' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "464a503a-75ae-49e7-ab6e-a7f9f2a6a307"}]: dispatch 2024-04-24T23:24:21.539 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:21 smithi080 ceph-mon[29152]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "464a503a-75ae-49e7-ab6e-a7f9f2a6a307"}]: dispatch 2024-04-24T23:24:21.539 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:21 smithi080 ceph-mon[29152]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "464a503a-75ae-49e7-ab6e-a7f9f2a6a307"}]': finished 2024-04-24T23:24:21.539 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:21 smithi080 ceph-mon[29152]: osdmap e48: 8 total, 7 up, 8 in 2024-04-24T23:24:21.539 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:21 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T23:24:21.539 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:21 smithi080 ceph-mon[29152]: from='client.? 172.21.15.134:0/376562134' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:24:22.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:22 smithi134 ceph-mon[32619]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:24:22.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:22 smithi148 ceph-mon[32528]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:24:22.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:22 smithi080 ceph-mon[29152]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:24:24.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:24 smithi134 ceph-mon[32619]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:24:24.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:24 smithi148 ceph-mon[32528]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:24:24.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:24 smithi080 ceph-mon[29152]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:24:25.506 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:25 smithi134 ceph-mon[32619]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:24:25.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:25 smithi148 ceph-mon[32528]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:24:25.539 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:25 smithi080 ceph-mon[29152]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:24:28.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:27 smithi134 ceph-mon[32619]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:24:28.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:27 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-24T23:24:28.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:27 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:24:28.242 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:27 smithi080 ceph-mon[29152]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:24:28.242 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:27 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-24T23:24:28.242 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:27 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:24:28.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:27 smithi148 ceph-mon[32528]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:24:28.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:27 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-24T23:24:28.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:27 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:24:29.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:28 smithi134 ceph-mon[32619]: Deploying daemon osd.7 on smithi134 2024-04-24T23:24:29.253 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:28 smithi080 ceph-mon[29152]: Deploying daemon osd.7 on smithi134 2024-04-24T23:24:29.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:28 smithi148 ceph-mon[32528]: Deploying daemon osd.7 on smithi134 2024-04-24T23:24:30.039 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:29 smithi134 ceph-mon[32619]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:24:30.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:29 smithi148 ceph-mon[32528]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:24:30.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:29 smithi080 ceph-mon[29152]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:24:32.146 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:31 smithi134 ceph-mon[32619]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:24:32.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:31 smithi148 ceph-mon[32528]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:24:32.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:31 smithi080 ceph-mon[29152]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:24:34.132 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:33 smithi134 ceph-mon[32619]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:24:34.132 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:33 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:24:34.132 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:33 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:24:34.132 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:33 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:24:34.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:33 smithi148 ceph-mon[32528]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:24:34.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:33 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:24:34.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:33 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:24:34.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:33 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:24:34.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:33 smithi080 ceph-mon[29152]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:24:34.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:33 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:24:34.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:33 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:24:34.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:33 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:24:36.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:35 smithi134 ceph-mon[32619]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:24:36.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:35 smithi148 ceph-mon[32528]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:24:36.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:35 smithi080 ceph-mon[29152]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:24:37.551 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:24:37.549+0000 7f37dcff9700 1 -- 172.21.15.134:0/2651714082 <== mgr.14152 v2:172.21.15.80:6800/3830596922 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f37e0062670 con 0x7f37c80a2c20 2024-04-24T23:24:37.551 INFO:teuthology.orchestra.run.smithi134.stdout:Created osd(s) 7 on host 'smithi134' 2024-04-24T23:24:37.553 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:24:37.551+0000 7f37e5e82700 1 -- 172.21.15.134:0/2651714082 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f37c80a2c20 msgr2=0x7f37c80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:37.553 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:24:37.551+0000 7f37e5e82700 1 --2- 172.21.15.134:0/2651714082 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f37c80a2c20 0x7f37c80a50e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f37d000b580 tx=0x7f37d0005fb0 comp rx=0 tx=0).stop 2024-04-24T23:24:37.553 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:24:37.551+0000 7f37e5e82700 1 -- 172.21.15.134:0/2651714082 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f37e0103fa0 msgr2=0x7f37e019fa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:37.553 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:24:37.551+0000 7f37e5e82700 1 --2- 172.21.15.134:0/2651714082 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f37e0103fa0 0x7f37e019fa40 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f37d8007ee0 tx=0x7f37d800d3b0 comp rx=0 tx=0).stop 2024-04-24T23:24:37.553 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:24:37.551+0000 7f37e5e82700 1 -- 172.21.15.134:0/2651714082 shutdown_connections 2024-04-24T23:24:37.553 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:24:37.551+0000 7f37e5e82700 1 --2- 172.21.15.134:0/2651714082 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f37e0103fa0 0x7f37e019fa40 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:37.553 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:24:37.551+0000 7f37e5e82700 1 --2- 172.21.15.134:0/2651714082 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f37c80a2c20 0x7f37c80a50e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:37.553 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:24:37.551+0000 7f37e5e82700 1 --2- 172.21.15.134:0/2651714082 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f37e0100580 0x7f37e0198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:37.553 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:24:37.551+0000 7f37e5e82700 1 --2- 172.21.15.134:0/2651714082 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f37e0103600 0x7f37e01989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:37.553 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:24:37.551+0000 7f37e5e82700 1 -- 172.21.15.134:0/2651714082 >> 172.21.15.134:0/2651714082 conn(0x7f37e00fbea0 msgr2=0x7f37e0106900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:37.553 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:24:37.551+0000 7f37e5e82700 1 -- 172.21.15.134:0/2651714082 shutdown_connections 2024-04-24T23:24:37.553 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:24:37.551+0000 7f37e5e82700 1 -- 172.21.15.134:0/2651714082 wait complete. 2024-04-24T23:24:37.921 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:37 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:24:37.921 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:37 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:24:37.921 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:37 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:24:37.921 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:37 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:24:37.921 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:37 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:24:37.921 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:37 smithi134 ceph-mon[32619]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:24:37.921 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:37 smithi134 ceph-mon[32619]: from='osd.7 [v2:172.21.15.134:6816/108354844,v1:172.21.15.134:6817/108354844]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-24T23:24:37.921 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:37 smithi134 ceph-mon[32619]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-24T23:24:37.921 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:37 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:24:37.921 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:37 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:24:37.922 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:37 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:24:38.006 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:37 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:24:38.007 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:37 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:24:38.007 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:37 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:24:38.007 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:37 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:24:38.007 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:37 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:24:38.007 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:37 smithi148 ceph-mon[32528]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:24:38.007 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:37 smithi148 ceph-mon[32528]: from='osd.7 [v2:172.21.15.134:6816/108354844,v1:172.21.15.134:6817/108354844]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-24T23:24:38.007 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:37 smithi148 ceph-mon[32528]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-24T23:24:38.007 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:37 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:24:38.007 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:37 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:24:38.007 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:37 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:24:38.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:37 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:24:38.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:37 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:24:38.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:37 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:24:38.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:37 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:24:38.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:37 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:24:38.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:37 smithi080 ceph-mon[29152]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:24:38.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:37 smithi080 ceph-mon[29152]: from='osd.7 [v2:172.21.15.134:6816/108354844,v1:172.21.15.134:6817/108354844]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-24T23:24:38.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:37 smithi080 ceph-mon[29152]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-24T23:24:38.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:37 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:24:38.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:37 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:24:38.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:37 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:24:38.248 DEBUG:teuthology.orchestra.run.smithi134:osd.7> sudo journalctl -f -n 0 -u ceph-ec7885aa-0290-11ef-bc93-c7b262605968@osd.7.service 2024-04-24T23:24:38.251 INFO:tasks.cephadm:Deploying osd.8 on smithi148 with /dev/vg_nvme/lv_2... 2024-04-24T23:24:38.251 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ec7885aa-0290-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-24T23:24:38.497 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T23:24:38.825 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:38 smithi134 ceph-mon[32619]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-24T23:24:38.825 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:38 smithi134 ceph-mon[32619]: osdmap e49: 8 total, 7 up, 8 in 2024-04-24T23:24:38.825 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:38 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T23:24:38.825 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:38 smithi134 ceph-mon[32619]: from='osd.7 [v2:172.21.15.134:6816/108354844,v1:172.21.15.134:6817/108354844]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-04-24T23:24:38.825 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:38 smithi134 ceph-mon[32619]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-04-24T23:24:38.826 INFO:journalctl@ceph.osd.7.smithi134.stdout:Apr 24 23:24:38 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T23:24:38.703+0000 7f67704a3700 -1 osd.7 0 waiting for initial osdmap 2024-04-24T23:24:38.826 INFO:journalctl@ceph.osd.7.smithi134.stdout:Apr 24 23:24:38 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T23:24:38.715+0000 7f676c296700 -1 osd.7 50 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T23:24:38.891 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:38 smithi148 ceph-mon[32528]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-24T23:24:38.891 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:38 smithi148 ceph-mon[32528]: osdmap e49: 8 total, 7 up, 8 in 2024-04-24T23:24:38.891 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:38 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T23:24:38.892 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:38 smithi148 ceph-mon[32528]: from='osd.7 [v2:172.21.15.134:6816/108354844,v1:172.21.15.134:6817/108354844]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-04-24T23:24:38.892 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:38 smithi148 ceph-mon[32528]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-04-24T23:24:39.038 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:38 smithi080 ceph-mon[29152]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-24T23:24:39.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:38 smithi080 ceph-mon[29152]: osdmap e49: 8 total, 7 up, 8 in 2024-04-24T23:24:39.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:38 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T23:24:39.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:38 smithi080 ceph-mon[29152]: from='osd.7 [v2:172.21.15.134:6816/108354844,v1:172.21.15.134:6817/108354844]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-04-24T23:24:39.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:38 smithi080 ceph-mon[29152]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-04-24T23:24:39.800 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:39 smithi134 ceph-mon[32619]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-04-24T23:24:39.801 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:39 smithi134 ceph-mon[32619]: osdmap e50: 8 total, 7 up, 8 in 2024-04-24T23:24:39.801 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:39 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T23:24:39.801 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:39 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T23:24:39.801 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:39 smithi134 ceph-mon[32619]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:24:40.006 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:39 smithi148 ceph-mon[32528]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-04-24T23:24:40.007 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:39 smithi148 ceph-mon[32528]: osdmap e50: 8 total, 7 up, 8 in 2024-04-24T23:24:40.007 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:39 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T23:24:40.007 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:39 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T23:24:40.007 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:39 smithi148 ceph-mon[32528]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:24:40.038 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:39 smithi080 ceph-mon[29152]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-04-24T23:24:40.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:39 smithi080 ceph-mon[29152]: osdmap e50: 8 total, 7 up, 8 in 2024-04-24T23:24:40.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:39 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T23:24:40.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:39 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T23:24:40.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:39 smithi080 ceph-mon[29152]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:24:40.601 INFO:teuthology.orchestra.run.smithi148.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-24T23:24:40.601 INFO:teuthology.orchestra.run.smithi148.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-24T23:24:40.601 INFO:teuthology.orchestra.run.smithi148.stderr: stderr: 10+0 records in 2024-04-24T23:24:40.601 INFO:teuthology.orchestra.run.smithi148.stderr:10+0 records out 2024-04-24T23:24:40.601 INFO:teuthology.orchestra.run.smithi148.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0127666 s, 821 MB/s 2024-04-24T23:24:40.601 INFO:teuthology.orchestra.run.smithi148.stderr:--> Zapping successful for: 2024-04-24T23:24:40.789 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:40 smithi080 ceph-mon[29152]: purged_snaps scrub starts 2024-04-24T23:24:40.789 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:40 smithi080 ceph-mon[29152]: purged_snaps scrub ok 2024-04-24T23:24:40.789 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:40 smithi080 ceph-mon[29152]: osd.7 [v2:172.21.15.134:6816/108354844,v1:172.21.15.134:6817/108354844] boot 2024-04-24T23:24:40.789 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:40 smithi080 ceph-mon[29152]: osdmap e51: 8 total, 8 up, 8 in 2024-04-24T23:24:40.789 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:40 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T23:24:40.903 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:40 smithi148 ceph-mon[32528]: purged_snaps scrub starts 2024-04-24T23:24:40.903 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:40 smithi148 ceph-mon[32528]: purged_snaps scrub ok 2024-04-24T23:24:40.903 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:40 smithi148 ceph-mon[32528]: osd.7 [v2:172.21.15.134:6816/108354844,v1:172.21.15.134:6817/108354844] boot 2024-04-24T23:24:40.903 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:40 smithi148 ceph-mon[32528]: osdmap e51: 8 total, 8 up, 8 in 2024-04-24T23:24:40.903 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:40 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T23:24:40.945 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:40 smithi134 ceph-mon[32619]: purged_snaps scrub starts 2024-04-24T23:24:40.946 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:40 smithi134 ceph-mon[32619]: purged_snaps scrub ok 2024-04-24T23:24:40.946 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:40 smithi134 ceph-mon[32619]: osd.7 [v2:172.21.15.134:6816/108354844,v1:172.21.15.134:6817/108354844] boot 2024-04-24T23:24:40.946 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:40 smithi134 ceph-mon[32619]: osdmap e51: 8 total, 8 up, 8 in 2024-04-24T23:24:40.946 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:40 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T23:24:41.289 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ec7885aa-0290-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi148:vg_nvme/lv_2 2024-04-24T23:24:41.532 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T23:24:41.990 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:41 smithi148 ceph-mon[32528]: osdmap e52: 8 total, 8 up, 8 in 2024-04-24T23:24:41.990 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:41 smithi148 ceph-mon[32528]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:24:41.990 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:41 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:24:41.990 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:41 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:24:41.990 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:41 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:24:41.990 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:41 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:24:41.990 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:41 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:24:41.990 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:41 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:24:41.990 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:41 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:24:41.990 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:41 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:24:41.990 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:41 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:24:41.991 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:41 smithi148 ceph-mon[32528]: osdmap e53: 8 total, 8 up, 8 in 2024-04-24T23:24:42.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:41 smithi080 ceph-mon[29152]: osdmap e52: 8 total, 8 up, 8 in 2024-04-24T23:24:42.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:41 smithi080 ceph-mon[29152]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:24:42.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:41 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:24:42.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:41 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:24:42.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:41 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:24:42.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:41 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:24:42.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:41 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:24:42.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:41 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:24:42.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:41 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:24:42.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:41 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:24:42.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:41 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:24:42.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:41 smithi080 ceph-mon[29152]: osdmap e53: 8 total, 8 up, 8 in 2024-04-24T23:24:42.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:41 smithi134 ceph-mon[32619]: osdmap e52: 8 total, 8 up, 8 in 2024-04-24T23:24:42.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:41 smithi134 ceph-mon[32619]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:24:42.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:41 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:24:42.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:41 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:24:42.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:41 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:24:42.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:41 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:24:42.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:41 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:24:42.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:41 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:24:42.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:41 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:24:42.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:41 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:24:42.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:41 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:24:42.199 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:41 smithi134 ceph-mon[32619]: osdmap e53: 8 total, 8 up, 8 in 2024-04-24T23:24:42.971 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:42 smithi148 ceph-mon[32528]: Detected new or changed devices on smithi134 2024-04-24T23:24:42.971 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:42 smithi148 ceph-mon[32528]: Adjusting osd_memory_target on smithi134 to 5696M 2024-04-24T23:24:43.038 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:42 smithi080 ceph-mon[29152]: Detected new or changed devices on smithi134 2024-04-24T23:24:43.038 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:42 smithi080 ceph-mon[29152]: Adjusting osd_memory_target on smithi134 to 5696M 2024-04-24T23:24:43.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:42 smithi134 ceph-mon[32619]: Detected new or changed devices on smithi134 2024-04-24T23:24:43.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:42 smithi134 ceph-mon[32619]: Adjusting osd_memory_target on smithi134 to 5696M 2024-04-24T23:24:43.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:24:43.209+0000 7f7da228a700 1 -- 172.21.15.148:0/3856750515 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d9c102290 msgr2=0x7f7d9c1026b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:43.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:24:43.209+0000 7f7da228a700 1 --2- 172.21.15.148:0/3856750515 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d9c102290 0x7f7d9c1026b0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c009990 tx=0x7f7d8c009ca0 comp rx=0 tx=0).stop 2024-04-24T23:24:43.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:24:43.209+0000 7f7da228a700 1 -- 172.21.15.148:0/3856750515 shutdown_connections 2024-04-24T23:24:43.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:24:43.209+0000 7f7da228a700 1 --2- 172.21.15.148:0/3856750515 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7d9c103e30 0x7f7d9c10a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:43.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:24:43.209+0000 7f7da228a700 1 --2- 172.21.15.148:0/3856750515 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d9c102290 0x7f7d9c1026b0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:43.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:24:43.209+0000 7f7da228a700 1 --2- 172.21.15.148:0/3856750515 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d9c103490 0x7f7d9c1038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:43.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:24:43.209+0000 7f7da228a700 1 -- 172.21.15.148:0/3856750515 >> 172.21.15.148:0/3856750515 conn(0x7f7d9c0fd830 msgr2=0x7f7d9c0ffc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:43.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:24:43.209+0000 7f7da228a700 1 -- 172.21.15.148:0/3856750515 shutdown_connections 2024-04-24T23:24:43.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:24:43.209+0000 7f7da228a700 1 -- 172.21.15.148:0/3856750515 wait complete. 2024-04-24T23:24:43.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:24:43.209+0000 7f7da228a700 1 Processor -- start 2024-04-24T23:24:43.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:24:43.209+0000 7f7da228a700 1 -- start start 2024-04-24T23:24:43.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:24:43.210+0000 7f7da228a700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d9c102290 0x7f7d9c198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:43.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:24:43.210+0000 7f7da228a700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7d9c103490 0x7f7d9c1989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:43.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:24:43.210+0000 7f7da228a700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d9c103e30 0x7f7d9c19fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:43.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:24:43.210+0000 7f7da228a700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f7d9c10de70 con 0x7f7d9c103490 2024-04-24T23:24:43.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:24:43.210+0000 7f7da228a700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7d9c10dcf0 con 0x7f7d9c102290 2024-04-24T23:24:43.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:24:43.210+0000 7f7da228a700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f7d9c10dff0 con 0x7f7d9c103e30 2024-04-24T23:24:43.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:24:43.210+0000 7f7da0827700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d9c103e30 0x7f7d9c19fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:43.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:24:43.210+0000 7f7da0827700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d9c103e30 0x7f7d9c19fa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42576/0 (socket says 172.21.15.148:42576) 2024-04-24T23:24:43.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:24:43.210+0000 7f7da0827700 1 -- 172.21.15.148:0/2578648521 learned_addr learned my addr 172.21.15.148:0/2578648521 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-24T23:24:43.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:24:43.210+0000 7f7d9bfff700 1 --2- 172.21.15.148:0/2578648521 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d9c102290 0x7f7d9c198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:43.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:24:43.210+0000 7f7d9b7fe700 1 --2- 172.21.15.148:0/2578648521 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7d9c103490 0x7f7d9c1989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:43.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:24:43.210+0000 7f7da0827700 1 -- 172.21.15.148:0/2578648521 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d9c102290 msgr2=0x7f7d9c198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:43.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:24:43.210+0000 7f7da0827700 1 --2- 172.21.15.148:0/2578648521 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d9c102290 0x7f7d9c198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:43.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:24:43.210+0000 7f7da0827700 1 -- 172.21.15.148:0/2578648521 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7d9c103490 msgr2=0x7f7d9c1989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:43.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:24:43.210+0000 7f7da0827700 1 --2- 172.21.15.148:0/2578648521 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7d9c103490 0x7f7d9c1989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:43.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:24:43.210+0000 7f7da0827700 1 -- 172.21.15.148:0/2578648521 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d9c1a00e0 con 0x7f7d9c103e30 2024-04-24T23:24:43.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:24:43.210+0000 7f7d9b7fe700 1 --2- 172.21.15.148:0/2578648521 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7d9c103490 0x7f7d9c1989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:24:43.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:24:43.210+0000 7f7da0827700 1 --2- 172.21.15.148:0/2578648521 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d9c103e30 0x7f7d9c19fa40 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f7d94007d40 tx=0x7f7d9400a340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:43.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:24:43.211+0000 7f7d997fa700 1 -- 172.21.15.148:0/2578648521 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d94016020 con 0x7f7d9c103e30 2024-04-24T23:24:43.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:24:43.211+0000 7f7d997fa700 1 -- 172.21.15.148:0/2578648521 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d9400ad60 con 0x7f7d9c103e30 2024-04-24T23:24:43.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:24:43.211+0000 7f7d997fa700 1 -- 172.21.15.148:0/2578648521 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d94011600 con 0x7f7d9c103e30 2024-04-24T23:24:43.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:24:43.211+0000 7f7da228a700 1 -- 172.21.15.148:0/2578648521 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d9c1a03d0 con 0x7f7d9c103e30 2024-04-24T23:24:43.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:24:43.211+0000 7f7da228a700 1 -- 172.21.15.148:0/2578648521 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d9c1a0910 con 0x7f7d9c103e30 2024-04-24T23:24:43.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:24:43.212+0000 7f7da228a700 1 -- 172.21.15.148:0/2578648521 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d9c068f20 con 0x7f7d9c103e30 2024-04-24T23:24:43.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:24:43.213+0000 7f7d997fa700 1 -- 172.21.15.148:0/2578648521 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f7d94019020 con 0x7f7d9c103e30 2024-04-24T23:24:43.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:24:43.213+0000 7f7d997fa700 1 --2- 172.21.15.148:0/2578648521 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f7d840a2ca0 0x7f7d840a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:43.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:24:43.214+0000 7f7d997fa700 1 -- 172.21.15.148:0/2578648521 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 5397+0+0 (secure 0 0 0) 0x7f7d940ce890 con 0x7f7d9c103e30 2024-04-24T23:24:43.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:24:43.214+0000 7f7d9bfff700 1 --2- 172.21.15.148:0/2578648521 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f7d840a2ca0 0x7f7d840a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:43.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:24:43.214+0000 7f7d9bfff700 1 --2- 172.21.15.148:0/2578648521 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f7d840a2ca0 0x7f7d840a5160 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c003890 tx=0x7f7d8c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:43.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:24:43.217+0000 7f7d997fa700 1 -- 172.21.15.148:0/2578648521 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f7d94020020 con 0x7f7d9c103e30 2024-04-24T23:24:43.423 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:24:43.422+0000 7f7da228a700 1 -- 172.21.15.148:0/2578648521 --> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f7d9c062670 con 0x7f7d840a2ca0 2024-04-24T23:24:44.006 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:43 smithi148 ceph-mon[32528]: osdmap e54: 8 total, 8 up, 8 in 2024-04-24T23:24:44.006 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:43 smithi148 ceph-mon[32528]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:24:44.007 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:43 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:24:44.007 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:43 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:24:44.007 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:43 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:24:44.038 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:43 smithi080 ceph-mon[29152]: osdmap e54: 8 total, 8 up, 8 in 2024-04-24T23:24:44.038 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:43 smithi080 ceph-mon[29152]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:24:44.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:43 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:24:44.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:43 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:24:44.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:43 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:24:44.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:43 smithi134 ceph-mon[32619]: osdmap e54: 8 total, 8 up, 8 in 2024-04-24T23:24:44.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:43 smithi134 ceph-mon[32619]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:24:44.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:43 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:24:44.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:43 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:24:44.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:43 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:24:44.732 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:44 smithi148 ceph-mon[32528]: from='client.24298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:24:44.732 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:44 smithi148 ceph-mon[32528]: osdmap e55: 8 total, 8 up, 8 in 2024-04-24T23:24:45.038 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:44 smithi080 ceph-mon[29152]: from='client.24298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:24:45.038 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:44 smithi080 ceph-mon[29152]: osdmap e55: 8 total, 8 up, 8 in 2024-04-24T23:24:45.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:44 smithi134 ceph-mon[32619]: from='client.24298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:24:45.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:44 smithi134 ceph-mon[32619]: osdmap e55: 8 total, 8 up, 8 in 2024-04-24T23:24:46.006 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:45 smithi148 ceph-mon[32528]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:24:46.038 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:45 smithi080 ceph-mon[29152]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:24:46.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:45 smithi134 ceph-mon[32619]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:24:47.006 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:46 smithi148 ceph-mon[32528]: from='client.? 172.21.15.148:0/972924906' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ce544b2e-4540-47c0-9997-500c45351330"}]: dispatch 2024-04-24T23:24:47.007 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:46 smithi148 ceph-mon[32528]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ce544b2e-4540-47c0-9997-500c45351330"}]: dispatch 2024-04-24T23:24:47.007 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:46 smithi148 ceph-mon[32528]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ce544b2e-4540-47c0-9997-500c45351330"}]': finished 2024-04-24T23:24:47.007 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:46 smithi148 ceph-mon[32528]: osdmap e56: 9 total, 8 up, 9 in 2024-04-24T23:24:47.007 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:46 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T23:24:47.038 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:46 smithi080 ceph-mon[29152]: from='client.? 172.21.15.148:0/972924906' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ce544b2e-4540-47c0-9997-500c45351330"}]: dispatch 2024-04-24T23:24:47.038 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:46 smithi080 ceph-mon[29152]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ce544b2e-4540-47c0-9997-500c45351330"}]: dispatch 2024-04-24T23:24:47.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:46 smithi080 ceph-mon[29152]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ce544b2e-4540-47c0-9997-500c45351330"}]': finished 2024-04-24T23:24:47.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:46 smithi080 ceph-mon[29152]: osdmap e56: 9 total, 8 up, 9 in 2024-04-24T23:24:47.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:46 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T23:24:47.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:46 smithi134 ceph-mon[32619]: from='client.? 172.21.15.148:0/972924906' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ce544b2e-4540-47c0-9997-500c45351330"}]: dispatch 2024-04-24T23:24:47.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:46 smithi134 ceph-mon[32619]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ce544b2e-4540-47c0-9997-500c45351330"}]: dispatch 2024-04-24T23:24:47.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:46 smithi134 ceph-mon[32619]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ce544b2e-4540-47c0-9997-500c45351330"}]': finished 2024-04-24T23:24:47.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:46 smithi134 ceph-mon[32619]: osdmap e56: 9 total, 8 up, 9 in 2024-04-24T23:24:47.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:46 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T23:24:48.006 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:47 smithi148 ceph-mon[32528]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:24:48.007 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:47 smithi148 ceph-mon[32528]: from='client.? 172.21.15.148:0/2826233425' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:24:48.038 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:47 smithi080 ceph-mon[29152]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:24:48.038 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:47 smithi080 ceph-mon[29152]: from='client.? 172.21.15.148:0/2826233425' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:24:48.098 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:47 smithi134 ceph-mon[32619]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:24:48.099 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:47 smithi134 ceph-mon[32619]: from='client.? 172.21.15.148:0/2826233425' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:24:50.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:49 smithi134 ceph-mon[32619]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:24:50.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:49 smithi148 ceph-mon[32528]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:24:50.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:49 smithi080 ceph-mon[29152]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:24:52.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:51 smithi134 ceph-mon[32619]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:24:52.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:51 smithi148 ceph-mon[32528]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:24:52.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:51 smithi080 ceph-mon[29152]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:24:54.006 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:53 smithi148 ceph-mon[32528]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:24:54.007 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:53 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-24T23:24:54.007 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:53 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:24:54.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:53 smithi134 ceph-mon[32619]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:24:54.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:53 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-24T23:24:54.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:53 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:24:54.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:53 smithi080 ceph-mon[29152]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:24:54.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:53 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-24T23:24:54.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:53 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:24:55.006 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:54 smithi148 ceph-mon[32528]: Deploying daemon osd.8 on smithi148 2024-04-24T23:24:55.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:54 smithi134 ceph-mon[32619]: Deploying daemon osd.8 on smithi148 2024-04-24T23:24:55.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:54 smithi080 ceph-mon[29152]: Deploying daemon osd.8 on smithi148 2024-04-24T23:24:56.124 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:55 smithi148 ceph-mon[32528]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:24:56.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:55 smithi134 ceph-mon[32619]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:24:56.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:55 smithi080 ceph-mon[29152]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:24:58.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:57 smithi134 ceph-mon[32619]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:24:58.245 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:57 smithi148 ceph-mon[32528]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:24:58.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:57 smithi080 ceph-mon[29152]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:25:00.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:59 smithi134 ceph-mon[32619]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:25:00.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:59 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:25:00.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:59 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:25:00.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:24:59 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:25:00.234 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:59 smithi148 ceph-mon[32528]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:25:00.234 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:59 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:25:00.234 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:59 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:25:00.234 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:24:59 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:25:00.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:59 smithi080 ceph-mon[29152]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:25:00.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:59 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:25:00.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:59 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:25:00.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:24:59 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:25:02.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:01 smithi134 ceph-mon[32619]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:25:02.243 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:01 smithi148 ceph-mon[32528]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:25:02.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:01 smithi080 ceph-mon[29152]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:25:03.594 INFO:teuthology.orchestra.run.smithi148.stdout:Created osd(s) 8 on host 'smithi148' 2024-04-24T23:25:03.595 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:25:03.593+0000 7f7d997fa700 1 -- 172.21.15.148:0/2578648521 <== mgr.14152 v2:172.21.15.80:6800/3830596922 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f7d9c062670 con 0x7f7d840a2ca0 2024-04-24T23:25:03.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:25:03.595+0000 7f7da228a700 1 -- 172.21.15.148:0/2578648521 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f7d840a2ca0 msgr2=0x7f7d840a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:03.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:25:03.595+0000 7f7da228a700 1 --2- 172.21.15.148:0/2578648521 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f7d840a2ca0 0x7f7d840a5160 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c003890 tx=0x7f7d8c005fd0 comp rx=0 tx=0).stop 2024-04-24T23:25:03.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:25:03.595+0000 7f7da228a700 1 -- 172.21.15.148:0/2578648521 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d9c103e30 msgr2=0x7f7d9c19fa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:03.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:25:03.595+0000 7f7da228a700 1 --2- 172.21.15.148:0/2578648521 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d9c103e30 0x7f7d9c19fa40 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f7d94007d40 tx=0x7f7d9400a340 comp rx=0 tx=0).stop 2024-04-24T23:25:03.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:25:03.596+0000 7f7da228a700 1 -- 172.21.15.148:0/2578648521 shutdown_connections 2024-04-24T23:25:03.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:25:03.596+0000 7f7da228a700 1 --2- 172.21.15.148:0/2578648521 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d9c103e30 0x7f7d9c19fa40 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:03.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:25:03.596+0000 7f7da228a700 1 --2- 172.21.15.148:0/2578648521 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d9c102290 0x7f7d9c198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:03.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:25:03.596+0000 7f7da228a700 1 --2- 172.21.15.148:0/2578648521 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f7d840a2ca0 0x7f7d840a5160 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:03.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:25:03.596+0000 7f7da228a700 1 --2- 172.21.15.148:0/2578648521 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7d9c103490 0x7f7d9c1989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:03.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:25:03.596+0000 7f7da228a700 1 -- 172.21.15.148:0/2578648521 >> 172.21.15.148:0/2578648521 conn(0x7f7d9c0fd830 msgr2=0x7f7d9c1067e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:03.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:25:03.596+0000 7f7da228a700 1 -- 172.21.15.148:0/2578648521 shutdown_connections 2024-04-24T23:25:03.598 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:25:03.596+0000 7f7da228a700 1 -- 172.21.15.148:0/2578648521 wait complete. 2024-04-24T23:25:03.756 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:03 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:25:03.756 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:03 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:25:03.756 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:03 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:25:03.756 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:03 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:25:03.757 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:03 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:25:03.757 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:03 smithi148 ceph-mon[32528]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:25:03.757 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:03 smithi148 ceph-mon[32528]: from='osd.8 [v2:172.21.15.148:6816/3559655329,v1:172.21.15.148:6817/3559655329]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-24T23:25:03.757 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:03 smithi148 ceph-mon[32528]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-24T23:25:03.757 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:03 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:25:03.757 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:03 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:25:03.757 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:03 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:25:03.947 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:03 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:25:03.948 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:03 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:25:03.948 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:03 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:25:03.948 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:03 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:25:03.948 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:03 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:25:03.948 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:03 smithi134 ceph-mon[32619]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:25:03.948 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:03 smithi134 ceph-mon[32619]: from='osd.8 [v2:172.21.15.148:6816/3559655329,v1:172.21.15.148:6817/3559655329]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-24T23:25:03.948 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:03 smithi134 ceph-mon[32619]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-24T23:25:03.948 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:03 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:25:03.948 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:03 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:25:03.948 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:03 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:25:04.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:03 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:25:04.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:03 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:25:04.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:03 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:25:04.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:03 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:25:04.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:03 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:25:04.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:03 smithi080 ceph-mon[29152]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:25:04.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:03 smithi080 ceph-mon[29152]: from='osd.8 [v2:172.21.15.148:6816/3559655329,v1:172.21.15.148:6817/3559655329]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-24T23:25:04.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:03 smithi080 ceph-mon[29152]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-24T23:25:04.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:03 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:25:04.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:03 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:25:04.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:03 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:25:04.260 DEBUG:teuthology.orchestra.run.smithi148:osd.8> sudo journalctl -f -n 0 -u ceph-ec7885aa-0290-11ef-bc93-c7b262605968@osd.8.service 2024-04-24T23:25:04.262 INFO:tasks.cephadm:Deploying osd.9 on smithi080 with /dev/vg_nvme/lv_1... 2024-04-24T23:25:04.262 DEBUG:teuthology.orchestra.run.smithi080:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ec7885aa-0290-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-24T23:25:04.506 INFO:teuthology.orchestra.run.smithi080.stderr:Inferring config /var/lib/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:25:04.795 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:04 smithi148 ceph-mon[32528]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-24T23:25:04.796 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:04 smithi148 ceph-mon[32528]: from='osd.8 [v2:172.21.15.148:6816/3559655329,v1:172.21.15.148:6817/3559655329]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-04-24T23:25:04.796 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:04 smithi148 ceph-mon[32528]: osdmap e57: 9 total, 8 up, 9 in 2024-04-24T23:25:04.796 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:04 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T23:25:04.796 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:04 smithi148 ceph-mon[32528]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-04-24T23:25:04.796 INFO:journalctl@ceph.osd.8.smithi148.stdout:Apr 24 23:25:04 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-8[51367]: 2024-04-24T23:25:04.664+0000 7f11a116f700 -1 osd.8 0 waiting for initial osdmap 2024-04-24T23:25:04.796 INFO:journalctl@ceph.osd.8.smithi148.stdout:Apr 24 23:25:04 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-8[51367]: 2024-04-24T23:25:04.675+0000 7f119cf62700 -1 osd.8 58 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T23:25:04.879 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:04 smithi080 ceph-mon[29152]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-24T23:25:04.879 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:04 smithi080 ceph-mon[29152]: from='osd.8 [v2:172.21.15.148:6816/3559655329,v1:172.21.15.148:6817/3559655329]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-04-24T23:25:04.879 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:04 smithi080 ceph-mon[29152]: osdmap e57: 9 total, 8 up, 9 in 2024-04-24T23:25:04.880 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:04 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T23:25:04.880 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:04 smithi080 ceph-mon[29152]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-04-24T23:25:04.947 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:04 smithi134 ceph-mon[32619]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-24T23:25:04.947 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:04 smithi134 ceph-mon[32619]: from='osd.8 [v2:172.21.15.148:6816/3559655329,v1:172.21.15.148:6817/3559655329]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-04-24T23:25:04.947 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:04 smithi134 ceph-mon[32619]: osdmap e57: 9 total, 8 up, 9 in 2024-04-24T23:25:04.948 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:04 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T23:25:04.948 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:04 smithi134 ceph-mon[32619]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-04-24T23:25:05.886 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:05 smithi148 ceph-mon[32528]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2024-04-24T23:25:05.887 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:05 smithi148 ceph-mon[32528]: osdmap e58: 9 total, 8 up, 9 in 2024-04-24T23:25:05.887 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:05 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T23:25:05.887 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:05 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T23:25:05.887 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:05 smithi148 ceph-mon[32528]: pgmap v197: 1 pgs: 1 peering; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:25:05.947 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:05 smithi134 ceph-mon[32619]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2024-04-24T23:25:05.947 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:05 smithi134 ceph-mon[32619]: osdmap e58: 9 total, 8 up, 9 in 2024-04-24T23:25:05.947 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:05 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T23:25:05.947 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:05 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T23:25:05.948 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:05 smithi134 ceph-mon[32619]: pgmap v197: 1 pgs: 1 peering; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:25:06.009 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:05 smithi080 ceph-mon[29152]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2024-04-24T23:25:06.009 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:05 smithi080 ceph-mon[29152]: osdmap e58: 9 total, 8 up, 9 in 2024-04-24T23:25:06.009 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:05 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T23:25:06.009 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:05 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T23:25:06.009 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:05 smithi080 ceph-mon[29152]: pgmap v197: 1 pgs: 1 peering; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:25:06.578 INFO:teuthology.orchestra.run.smithi080.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-24T23:25:06.579 INFO:teuthology.orchestra.run.smithi080.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-24T23:25:06.579 INFO:teuthology.orchestra.run.smithi080.stderr: stderr: 10+0 records in 2024-04-24T23:25:06.579 INFO:teuthology.orchestra.run.smithi080.stderr:10+0 records out 2024-04-24T23:25:06.579 INFO:teuthology.orchestra.run.smithi080.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0112418 s, 933 MB/s 2024-04-24T23:25:06.579 INFO:teuthology.orchestra.run.smithi080.stderr:--> Zapping successful for: 2024-04-24T23:25:06.877 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:06 smithi080 ceph-mon[29152]: purged_snaps scrub starts 2024-04-24T23:25:06.877 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:06 smithi080 ceph-mon[29152]: purged_snaps scrub ok 2024-04-24T23:25:06.877 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:06 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T23:25:06.877 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:06 smithi080 ceph-mon[29152]: osd.8 [v2:172.21.15.148:6816/3559655329,v1:172.21.15.148:6817/3559655329] boot 2024-04-24T23:25:06.877 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:06 smithi080 ceph-mon[29152]: osdmap e59: 9 total, 9 up, 9 in 2024-04-24T23:25:06.877 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:06 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T23:25:06.947 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:06 smithi134 ceph-mon[32619]: purged_snaps scrub starts 2024-04-24T23:25:06.947 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:06 smithi134 ceph-mon[32619]: purged_snaps scrub ok 2024-04-24T23:25:06.947 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:06 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T23:25:06.947 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:06 smithi134 ceph-mon[32619]: osd.8 [v2:172.21.15.148:6816/3559655329,v1:172.21.15.148:6817/3559655329] boot 2024-04-24T23:25:06.947 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:06 smithi134 ceph-mon[32619]: osdmap e59: 9 total, 9 up, 9 in 2024-04-24T23:25:06.947 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:06 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T23:25:07.007 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:06 smithi148 ceph-mon[32528]: purged_snaps scrub starts 2024-04-24T23:25:07.007 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:06 smithi148 ceph-mon[32528]: purged_snaps scrub ok 2024-04-24T23:25:07.008 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:06 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T23:25:07.008 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:06 smithi148 ceph-mon[32528]: osd.8 [v2:172.21.15.148:6816/3559655329,v1:172.21.15.148:6817/3559655329] boot 2024-04-24T23:25:07.008 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:06 smithi148 ceph-mon[32528]: osdmap e59: 9 total, 9 up, 9 in 2024-04-24T23:25:07.008 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:06 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T23:25:07.294 DEBUG:teuthology.orchestra.run.smithi080:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ec7885aa-0290-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi080:vg_nvme/lv_1 2024-04-24T23:25:07.541 INFO:teuthology.orchestra.run.smithi080.stderr:Inferring config /var/lib/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:25:07.683 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:07 smithi148 ceph-mon[32528]: osdmap e60: 9 total, 9 up, 9 in 2024-04-24T23:25:07.683 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:07 smithi148 ceph-mon[32528]: pgmap v200: 1 pgs: 1 peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:25:07.683 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:07 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:25:07.683 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:07 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:25:07.683 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:07 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:25:07.683 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:07 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:25:07.683 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:07 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:25:07.683 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:07 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:25:07.683 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:07 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:25:07.683 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:07 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:25:07.683 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:07 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:25:07.947 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:07 smithi134 ceph-mon[32619]: osdmap e60: 9 total, 9 up, 9 in 2024-04-24T23:25:07.947 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:07 smithi134 ceph-mon[32619]: pgmap v200: 1 pgs: 1 peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:25:07.947 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:07 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:25:07.948 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:07 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:25:07.948 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:07 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:25:07.948 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:07 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:25:07.948 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:07 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:25:07.948 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:07 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:25:07.948 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:07 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:25:07.948 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:07 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:25:07.948 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:07 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:25:07.977 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:07 smithi080 ceph-mon[29152]: osdmap e60: 9 total, 9 up, 9 in 2024-04-24T23:25:07.977 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:07 smithi080 ceph-mon[29152]: pgmap v200: 1 pgs: 1 peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:25:07.977 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:07 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:25:07.978 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:07 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:25:07.978 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:07 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:25:07.978 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:07 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:25:07.978 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:07 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:25:07.978 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:07 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:25:07.978 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:07 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:25:07.978 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:07 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:25:07.978 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:07 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:25:08.741 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:08 smithi080 ceph-mon[29152]: Detected new or changed devices on smithi148 2024-04-24T23:25:08.741 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:08 smithi080 ceph-mon[29152]: Adjusting osd_memory_target on smithi148 to 5696M 2024-04-24T23:25:08.741 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:08 smithi080 ceph-mon[29152]: osdmap e61: 9 total, 9 up, 9 in 2024-04-24T23:25:08.741 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:08 smithi080 ceph-mon[29152]: osdmap e62: 9 total, 9 up, 9 in 2024-04-24T23:25:08.947 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:08 smithi134 ceph-mon[32619]: Detected new or changed devices on smithi148 2024-04-24T23:25:08.947 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:08 smithi134 ceph-mon[32619]: Adjusting osd_memory_target on smithi148 to 5696M 2024-04-24T23:25:08.947 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:08 smithi134 ceph-mon[32619]: osdmap e61: 9 total, 9 up, 9 in 2024-04-24T23:25:08.947 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:08 smithi134 ceph-mon[32619]: osdmap e62: 9 total, 9 up, 9 in 2024-04-24T23:25:09.006 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:08 smithi148 ceph-mon[32528]: Detected new or changed devices on smithi148 2024-04-24T23:25:09.006 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:08 smithi148 ceph-mon[32528]: Adjusting osd_memory_target on smithi148 to 5696M 2024-04-24T23:25:09.006 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:08 smithi148 ceph-mon[32528]: osdmap e61: 9 total, 9 up, 9 in 2024-04-24T23:25:09.007 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:08 smithi148 ceph-mon[32528]: osdmap e62: 9 total, 9 up, 9 in 2024-04-24T23:25:09.294 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:25:09.292+0000 7eff25536700 1 -- 172.21.15.80:0/1382677311 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7eff20102290 msgr2=0x7eff201026b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:09.294 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:25:09.292+0000 7eff25536700 1 --2- 172.21.15.80:0/1382677311 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7eff20102290 0x7eff201026b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7eff080099c0 tx=0x7eff08009cd0 comp rx=0 tx=0).stop 2024-04-24T23:25:09.294 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:25:09.292+0000 7eff25536700 1 -- 172.21.15.80:0/1382677311 shutdown_connections 2024-04-24T23:25:09.294 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:25:09.292+0000 7eff25536700 1 --2- 172.21.15.80:0/1382677311 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff20103e30 0x7eff2010a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:09.295 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:25:09.292+0000 7eff25536700 1 --2- 172.21.15.80:0/1382677311 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7eff20102290 0x7eff201026b0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:09.295 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:25:09.292+0000 7eff25536700 1 --2- 172.21.15.80:0/1382677311 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff20103490 0x7eff201038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:09.295 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:25:09.292+0000 7eff25536700 1 -- 172.21.15.80:0/1382677311 >> 172.21.15.80:0/1382677311 conn(0x7eff200fd830 msgr2=0x7eff200ffc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:09.295 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:25:09.292+0000 7eff25536700 1 -- 172.21.15.80:0/1382677311 shutdown_connections 2024-04-24T23:25:09.295 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:25:09.292+0000 7eff25536700 1 -- 172.21.15.80:0/1382677311 wait complete. 2024-04-24T23:25:09.295 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:25:09.293+0000 7eff25536700 1 Processor -- start 2024-04-24T23:25:09.295 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:25:09.293+0000 7eff25536700 1 -- start start 2024-04-24T23:25:09.297 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:25:09.293+0000 7eff25536700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7eff20102290 0x7eff201985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:09.297 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:25:09.293+0000 7eff25536700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff20103490 0x7eff20198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:09.297 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:25:09.293+0000 7eff25536700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff20103e30 0x7eff2019fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:09.297 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:25:09.293+0000 7eff25536700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7eff2010df40 con 0x7eff20102290 2024-04-24T23:25:09.297 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:25:09.293+0000 7eff25536700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7eff2010ddc0 con 0x7eff20103490 2024-04-24T23:25:09.297 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:25:09.293+0000 7eff25536700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7eff2010e0c0 con 0x7eff20103e30 2024-04-24T23:25:09.298 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:25:09.293+0000 7eff1effd700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7eff20102290 0x7eff201985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:09.298 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:25:09.293+0000 7eff1effd700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7eff20102290 0x7eff201985b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:33016/0 (socket says 172.21.15.80:33016) 2024-04-24T23:25:09.298 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:25:09.293+0000 7eff1effd700 1 -- 172.21.15.80:0/1724646694 learned_addr learned my addr 172.21.15.80:0/1724646694 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:25:09.298 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:25:09.293+0000 7eff1f7fe700 1 --2- 172.21.15.80:0/1724646694 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff20103e30 0x7eff2019fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:09.298 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:25:09.294+0000 7eff1e7fc700 1 --2- 172.21.15.80:0/1724646694 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff20103490 0x7eff20198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:09.298 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:25:09.294+0000 7eff1effd700 1 -- 172.21.15.80:0/1724646694 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff20103e30 msgr2=0x7eff2019fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:09.298 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:25:09.294+0000 7eff1effd700 1 --2- 172.21.15.80:0/1724646694 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff20103e30 0x7eff2019fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:09.298 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:25:09.294+0000 7eff1effd700 1 -- 172.21.15.80:0/1724646694 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff20103490 msgr2=0x7eff20198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:09.298 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:25:09.294+0000 7eff1effd700 1 --2- 172.21.15.80:0/1724646694 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff20103490 0x7eff20198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:09.298 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:25:09.294+0000 7eff1effd700 1 -- 172.21.15.80:0/1724646694 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff201a0210 con 0x7eff20102290 2024-04-24T23:25:09.298 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:25:09.294+0000 7eff1effd700 1 --2- 172.21.15.80:0/1724646694 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7eff20102290 0x7eff201985b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7eff0800c010 tx=0x7eff08009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:09.298 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:25:09.294+0000 7eff17fff700 1 -- 172.21.15.80:0/1724646694 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff08027070 con 0x7eff20102290 2024-04-24T23:25:09.298 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:25:09.294+0000 7eff25536700 1 -- 172.21.15.80:0/1724646694 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff201a04a0 con 0x7eff20102290 2024-04-24T23:25:09.298 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:25:09.294+0000 7eff17fff700 1 -- 172.21.15.80:0/1724646694 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff08004150 con 0x7eff20102290 2024-04-24T23:25:09.299 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:25:09.294+0000 7eff17fff700 1 -- 172.21.15.80:0/1724646694 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff08022e00 con 0x7eff20102290 2024-04-24T23:25:09.299 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:25:09.294+0000 7eff25536700 1 -- 172.21.15.80:0/1724646694 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff201a09d0 con 0x7eff20102290 2024-04-24T23:25:09.299 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:25:09.295+0000 7eff25536700 1 -- 172.21.15.80:0/1724646694 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff20068f20 con 0x7eff20102290 2024-04-24T23:25:09.299 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:25:09.296+0000 7eff17fff700 1 -- 172.21.15.80:0/1724646694 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7eff0802a070 con 0x7eff20102290 2024-04-24T23:25:09.299 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:25:09.297+0000 7eff17fff700 1 --2- 172.21.15.80:0/1724646694 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7eff0c0a2c20 0x7eff0c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:09.299 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:25:09.297+0000 7eff1e7fc700 1 --2- 172.21.15.80:0/1724646694 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7eff0c0a2c20 0x7eff0c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:09.300 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:25:09.297+0000 7eff17fff700 1 -- 172.21.15.80:0/1724646694 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5829+0+0 (secure 0 0 0) 0x7eff080cf450 con 0x7eff20102290 2024-04-24T23:25:09.300 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:25:09.297+0000 7eff1e7fc700 1 --2- 172.21.15.80:0/1724646694 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7eff0c0a2c20 0x7eff0c0a50e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7eff10004630 tx=0x7eff1000b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:09.303 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:25:09.301+0000 7eff17fff700 1 -- 172.21.15.80:0/1724646694 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7eff0805e740 con 0x7eff20102290 2024-04-24T23:25:09.508 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:25:09.506+0000 7eff25536700 1 -- 172.21.15.80:0/1724646694 --> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi080:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7eff20062670 con 0x7eff0c0a2c20 2024-04-24T23:25:10.433 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:10 smithi080 ceph-mon[29152]: pgmap v203: 1 pgs: 1 peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:25:10.433 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:10 smithi080 ceph-mon[29152]: osdmap e63: 9 total, 9 up, 9 in 2024-04-24T23:25:10.433 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:10 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:25:10.433 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:10 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:25:10.433 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:10 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:25:10.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:10 smithi134 ceph-mon[32619]: pgmap v203: 1 pgs: 1 peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:25:10.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:10 smithi134 ceph-mon[32619]: osdmap e63: 9 total, 9 up, 9 in 2024-04-24T23:25:10.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:10 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:25:10.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:10 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:25:10.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:10 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:25:10.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:10 smithi148 ceph-mon[32528]: pgmap v203: 1 pgs: 1 peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:25:10.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:10 smithi148 ceph-mon[32528]: osdmap e63: 9 total, 9 up, 9 in 2024-04-24T23:25:10.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:10 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:25:10.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:10 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:25:10.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:10 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:25:11.396 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:11 smithi080 ceph-mon[29152]: from='client.14430 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi080:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:25:11.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:11 smithi134 ceph-mon[32619]: from='client.14430 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi080:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:25:11.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:11 smithi148 ceph-mon[32528]: from='client.14430 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi080:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:25:12.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:12 smithi134 ceph-mon[32619]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 138 KiB/s, 0 objects/s recovering 2024-04-24T23:25:12.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:12 smithi148 ceph-mon[32528]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 138 KiB/s, 0 objects/s recovering 2024-04-24T23:25:12.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:12 smithi080 ceph-mon[29152]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 138 KiB/s, 0 objects/s recovering 2024-04-24T23:25:13.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:13 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/1029258911' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "06de3732-19b9-496c-a464-55322efa6759"}]: dispatch 2024-04-24T23:25:13.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:13 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/1029258911' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "06de3732-19b9-496c-a464-55322efa6759"}]': finished 2024-04-24T23:25:13.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:13 smithi134 ceph-mon[32619]: osdmap e64: 10 total, 9 up, 10 in 2024-04-24T23:25:13.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:13 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T23:25:13.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:13 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/1209034329' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:25:13.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:13 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/1029258911' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "06de3732-19b9-496c-a464-55322efa6759"}]: dispatch 2024-04-24T23:25:13.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:13 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/1029258911' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "06de3732-19b9-496c-a464-55322efa6759"}]': finished 2024-04-24T23:25:13.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:13 smithi148 ceph-mon[32528]: osdmap e64: 10 total, 9 up, 10 in 2024-04-24T23:25:13.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:13 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T23:25:13.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:13 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/1209034329' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:25:13.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:13 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/1029258911' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "06de3732-19b9-496c-a464-55322efa6759"}]: dispatch 2024-04-24T23:25:13.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:13 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/1029258911' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "06de3732-19b9-496c-a464-55322efa6759"}]': finished 2024-04-24T23:25:13.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:13 smithi080 ceph-mon[29152]: osdmap e64: 10 total, 9 up, 10 in 2024-04-24T23:25:13.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:13 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T23:25:13.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:13 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/1209034329' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:25:14.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:14 smithi134 ceph-mon[32619]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-04-24T23:25:14.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:14 smithi148 ceph-mon[32528]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-04-24T23:25:14.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:14 smithi080 ceph-mon[29152]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-04-24T23:25:16.447 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:16 smithi080 ceph-mon[29152]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-04-24T23:25:16.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:16 smithi134 ceph-mon[32619]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-04-24T23:25:16.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:16 smithi148 ceph-mon[32528]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-04-24T23:25:17.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:17 smithi134 ceph-mon[32619]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-24T23:25:17.496 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:17 smithi080 ceph-mon[29152]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-24T23:25:17.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:17 smithi148 ceph-mon[32528]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-24T23:25:20.029 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:19 smithi080 ceph-mon[29152]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-04-24T23:25:20.168 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:19 smithi134 ceph-mon[32619]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-04-24T23:25:20.183 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:19 smithi148 ceph-mon[32528]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-04-24T23:25:21.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:20 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-24T23:25:21.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:20 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:25:21.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:20 smithi080 ceph-mon[29152]: Deploying daemon osd.9 on smithi080 2024-04-24T23:25:21.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:20 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-24T23:25:21.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:20 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:25:21.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:20 smithi134 ceph-mon[32619]: Deploying daemon osd.9 on smithi080 2024-04-24T23:25:21.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:20 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-24T23:25:21.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:20 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:25:21.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:20 smithi148 ceph-mon[32528]: Deploying daemon osd.9 on smithi080 2024-04-24T23:25:22.132 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:21 smithi080 ceph-mon[29152]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:25:22.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:21 smithi134 ceph-mon[32619]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:25:22.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:21 smithi148 ceph-mon[32528]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:25:24.174 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:23 smithi080 ceph-mon[29152]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:25:24.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:23 smithi134 ceph-mon[32619]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:25:24.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:23 smithi148 ceph-mon[32528]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:25:26.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:25 smithi134 ceph-mon[32619]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:25:26.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:25 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:25:26.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:25 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:25:26.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:25 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:25:26.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:25 smithi148 ceph-mon[32528]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:25:26.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:25 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:25:26.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:25 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:25:26.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:25 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:25:26.276 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:25 smithi080 ceph-mon[29152]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:25:26.276 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:25 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:25:26.277 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:25 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:25:26.277 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:25 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:25:28.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:27 smithi134 ceph-mon[32619]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:25:28.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:27 smithi148 ceph-mon[32528]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:25:28.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:27 smithi080 ceph-mon[29152]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:25:29.941 INFO:teuthology.orchestra.run.smithi080.stdout:Created osd(s) 9 on host 'smithi080' 2024-04-24T23:25:29.941 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:25:29.939+0000 7eff17fff700 1 -- 172.21.15.80:0/1724646694 <== mgr.14152 v2:172.21.15.80:6800/3830596922 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7eff20062670 con 0x7eff0c0a2c20 2024-04-24T23:25:29.943 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:25:29.942+0000 7eff25536700 1 -- 172.21.15.80:0/1724646694 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7eff0c0a2c20 msgr2=0x7eff0c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:29.943 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:25:29.942+0000 7eff25536700 1 --2- 172.21.15.80:0/1724646694 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7eff0c0a2c20 0x7eff0c0a50e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7eff10004630 tx=0x7eff1000b3c0 comp rx=0 tx=0).stop 2024-04-24T23:25:29.944 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:25:29.942+0000 7eff25536700 1 -- 172.21.15.80:0/1724646694 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7eff20102290 msgr2=0x7eff201985b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:29.944 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:25:29.942+0000 7eff25536700 1 --2- 172.21.15.80:0/1724646694 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7eff20102290 0x7eff201985b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7eff0800c010 tx=0x7eff08009580 comp rx=0 tx=0).stop 2024-04-24T23:25:29.944 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:25:29.942+0000 7eff25536700 1 -- 172.21.15.80:0/1724646694 shutdown_connections 2024-04-24T23:25:29.944 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:25:29.942+0000 7eff25536700 1 --2- 172.21.15.80:0/1724646694 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff20103e30 0x7eff2019fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:29.944 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:25:29.942+0000 7eff25536700 1 --2- 172.21.15.80:0/1724646694 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7eff0c0a2c20 0x7eff0c0a50e0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:29.944 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:25:29.942+0000 7eff25536700 1 --2- 172.21.15.80:0/1724646694 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7eff20102290 0x7eff201985b0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:29.944 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:25:29.942+0000 7eff25536700 1 --2- 172.21.15.80:0/1724646694 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff20103490 0x7eff20198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:29.944 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:25:29.942+0000 7eff25536700 1 -- 172.21.15.80:0/1724646694 >> 172.21.15.80:0/1724646694 conn(0x7eff200fd830 msgr2=0x7eff201067b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:29.944 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:25:29.942+0000 7eff25536700 1 -- 172.21.15.80:0/1724646694 shutdown_connections 2024-04-24T23:25:29.944 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:25:29.942+0000 7eff25536700 1 -- 172.21.15.80:0/1724646694 wait complete. 2024-04-24T23:25:30.038 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:29 smithi080 ceph-mon[29152]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:25:30.038 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:29 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:25:30.038 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:29 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:25:30.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:29 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:25:30.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:29 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:25:30.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:29 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:25:30.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:29 smithi080 ceph-mon[29152]: from='osd.9 [v2:172.21.15.80:6826/4019913545,v1:172.21.15.80:6827/4019913545]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-24T23:25:30.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:29 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:25:30.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:29 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:25:30.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:29 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:25:30.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:29 smithi148 ceph-mon[32528]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:25:30.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:29 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:25:30.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:29 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:25:30.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:29 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:25:30.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:29 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:25:30.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:29 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:25:30.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:29 smithi148 ceph-mon[32528]: from='osd.9 [v2:172.21.15.80:6826/4019913545,v1:172.21.15.80:6827/4019913545]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-24T23:25:30.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:29 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:25:30.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:29 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:25:30.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:29 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:25:30.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:29 smithi134 ceph-mon[32619]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:25:30.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:29 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:25:30.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:29 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:25:30.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:29 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:25:30.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:29 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:25:30.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:29 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:25:30.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:29 smithi134 ceph-mon[32619]: from='osd.9 [v2:172.21.15.80:6826/4019913545,v1:172.21.15.80:6827/4019913545]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-24T23:25:30.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:29 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:25:30.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:29 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:25:30.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:29 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:25:30.590 DEBUG:teuthology.orchestra.run.smithi080:osd.9> sudo journalctl -f -n 0 -u ceph-ec7885aa-0290-11ef-bc93-c7b262605968@osd.9.service 2024-04-24T23:25:30.592 INFO:tasks.cephadm:Deploying osd.10 on smithi134 with /dev/vg_nvme/lv_1... 2024-04-24T23:25:30.592 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ec7885aa-0290-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-24T23:25:30.844 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/mon.b/config 2024-04-24T23:25:31.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:30 smithi148 ceph-mon[32528]: from='osd.9 [v2:172.21.15.80:6826/4019913545,v1:172.21.15.80:6827/4019913545]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-24T23:25:31.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:30 smithi148 ceph-mon[32528]: osdmap e65: 10 total, 9 up, 10 in 2024-04-24T23:25:31.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:30 smithi148 ceph-mon[32528]: from='osd.9 [v2:172.21.15.80:6826/4019913545,v1:172.21.15.80:6827/4019913545]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi080", "root=default"]}]: dispatch 2024-04-24T23:25:31.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:30 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T23:25:31.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:30 smithi080 ceph-mon[29152]: from='osd.9 [v2:172.21.15.80:6826/4019913545,v1:172.21.15.80:6827/4019913545]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-24T23:25:31.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:30 smithi080 ceph-mon[29152]: osdmap e65: 10 total, 9 up, 10 in 2024-04-24T23:25:31.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:30 smithi080 ceph-mon[29152]: from='osd.9 [v2:172.21.15.80:6826/4019913545,v1:172.21.15.80:6827/4019913545]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi080", "root=default"]}]: dispatch 2024-04-24T23:25:31.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:30 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T23:25:31.289 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:30 smithi134 ceph-mon[32619]: from='osd.9 [v2:172.21.15.80:6826/4019913545,v1:172.21.15.80:6827/4019913545]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-24T23:25:31.289 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:30 smithi134 ceph-mon[32619]: osdmap e65: 10 total, 9 up, 10 in 2024-04-24T23:25:31.289 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:30 smithi134 ceph-mon[32619]: from='osd.9 [v2:172.21.15.80:6826/4019913545,v1:172.21.15.80:6827/4019913545]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi080", "root=default"]}]: dispatch 2024-04-24T23:25:31.289 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:30 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T23:25:31.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:25:31 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:25:31.397+0000 7f8b743ca700 -1 osd.9 0 waiting for initial osdmap 2024-04-24T23:25:31.789 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:25:31 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:25:31.410+0000 7f8b6c1b5700 -1 osd.9 66 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T23:25:32.220 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:31 smithi080 ceph-mon[29152]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:25:32.221 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:31 smithi080 ceph-mon[29152]: from='osd.9 [v2:172.21.15.80:6826/4019913545,v1:172.21.15.80:6827/4019913545]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi080", "root=default"]}]': finished 2024-04-24T23:25:32.221 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:31 smithi080 ceph-mon[29152]: osdmap e66: 10 total, 9 up, 10 in 2024-04-24T23:25:32.221 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:31 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T23:25:32.221 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:31 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T23:25:32.221 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:31 smithi080 ceph-mon[29152]: from='osd.9 [v2:172.21.15.80:6826/4019913545,v1:172.21.15.80:6827/4019913545]' entity='osd.9' 2024-04-24T23:25:32.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:31 smithi148 ceph-mon[32528]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:25:32.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:31 smithi148 ceph-mon[32528]: from='osd.9 [v2:172.21.15.80:6826/4019913545,v1:172.21.15.80:6827/4019913545]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi080", "root=default"]}]': finished 2024-04-24T23:25:32.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:31 smithi148 ceph-mon[32528]: osdmap e66: 10 total, 9 up, 10 in 2024-04-24T23:25:32.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:31 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T23:25:32.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:31 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T23:25:32.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:31 smithi148 ceph-mon[32528]: from='osd.9 [v2:172.21.15.80:6826/4019913545,v1:172.21.15.80:6827/4019913545]' entity='osd.9' 2024-04-24T23:25:32.308 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:31 smithi134 ceph-mon[32619]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:25:32.308 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:31 smithi134 ceph-mon[32619]: from='osd.9 [v2:172.21.15.80:6826/4019913545,v1:172.21.15.80:6827/4019913545]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi080", "root=default"]}]': finished 2024-04-24T23:25:32.308 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:31 smithi134 ceph-mon[32619]: osdmap e66: 10 total, 9 up, 10 in 2024-04-24T23:25:32.308 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:31 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T23:25:32.308 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:31 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T23:25:32.308 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:31 smithi134 ceph-mon[32619]: from='osd.9 [v2:172.21.15.80:6826/4019913545,v1:172.21.15.80:6827/4019913545]' entity='osd.9' 2024-04-24T23:25:32.875 INFO:teuthology.orchestra.run.smithi134.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-24T23:25:32.875 INFO:teuthology.orchestra.run.smithi134.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-24T23:25:32.875 INFO:teuthology.orchestra.run.smithi134.stderr: stderr: 10+0 records in 2024-04-24T23:25:32.875 INFO:teuthology.orchestra.run.smithi134.stderr:10+0 records out 2024-04-24T23:25:32.875 INFO:teuthology.orchestra.run.smithi134.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0130639 s, 803 MB/s 2024-04-24T23:25:32.875 INFO:teuthology.orchestra.run.smithi134.stderr:--> Zapping successful for: 2024-04-24T23:25:33.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:32 smithi148 ceph-mon[32528]: purged_snaps scrub starts 2024-04-24T23:25:33.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:32 smithi148 ceph-mon[32528]: purged_snaps scrub ok 2024-04-24T23:25:33.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:32 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T23:25:33.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:32 smithi148 ceph-mon[32528]: osd.9 [v2:172.21.15.80:6826/4019913545,v1:172.21.15.80:6827/4019913545] boot 2024-04-24T23:25:33.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:32 smithi148 ceph-mon[32528]: osdmap e67: 10 total, 10 up, 10 in 2024-04-24T23:25:33.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:32 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T23:25:33.259 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:32 smithi134 ceph-mon[32619]: purged_snaps scrub starts 2024-04-24T23:25:33.259 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:32 smithi134 ceph-mon[32619]: purged_snaps scrub ok 2024-04-24T23:25:33.259 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:32 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T23:25:33.260 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:32 smithi134 ceph-mon[32619]: osd.9 [v2:172.21.15.80:6826/4019913545,v1:172.21.15.80:6827/4019913545] boot 2024-04-24T23:25:33.260 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:32 smithi134 ceph-mon[32619]: osdmap e67: 10 total, 10 up, 10 in 2024-04-24T23:25:33.260 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:32 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T23:25:33.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:32 smithi080 ceph-mon[29152]: purged_snaps scrub starts 2024-04-24T23:25:33.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:32 smithi080 ceph-mon[29152]: purged_snaps scrub ok 2024-04-24T23:25:33.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:32 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T23:25:33.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:32 smithi080 ceph-mon[29152]: osd.9 [v2:172.21.15.80:6826/4019913545,v1:172.21.15.80:6827/4019913545] boot 2024-04-24T23:25:33.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:32 smithi080 ceph-mon[29152]: osdmap e67: 10 total, 10 up, 10 in 2024-04-24T23:25:33.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:32 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T23:25:33.703 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ec7885aa-0290-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi134:vg_nvme/lv_1 2024-04-24T23:25:33.945 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/mon.b/config 2024-04-24T23:25:34.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:33 smithi148 ceph-mon[32528]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:25:34.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:33 smithi148 ceph-mon[32528]: osdmap e68: 10 total, 10 up, 10 in 2024-04-24T23:25:34.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:33 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:25:34.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:33 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:25:34.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:33 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:25:34.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:33 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:25:34.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:33 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:25:34.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:33 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:25:34.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:33 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:25:34.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:33 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:25:34.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:33 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:25:34.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:33 smithi080 ceph-mon[29152]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:25:34.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:33 smithi080 ceph-mon[29152]: osdmap e68: 10 total, 10 up, 10 in 2024-04-24T23:25:34.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:33 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:25:34.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:33 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:25:34.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:33 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:25:34.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:33 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:25:34.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:33 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:25:34.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:33 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:25:34.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:33 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:25:34.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:33 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:25:34.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:33 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:25:34.324 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:33 smithi134 ceph-mon[32619]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:25:34.324 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:33 smithi134 ceph-mon[32619]: osdmap e68: 10 total, 10 up, 10 in 2024-04-24T23:25:34.324 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:33 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:25:34.324 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:33 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:25:34.324 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:33 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:25:34.324 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:33 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:25:34.324 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:33 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:25:34.324 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:33 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:25:34.324 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:33 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:25:34.324 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:33 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:25:34.324 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:33 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:25:35.086 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:35 smithi134 ceph-mon[32619]: Detected new or changed devices on smithi080 2024-04-24T23:25:35.086 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:35 smithi134 ceph-mon[32619]: Adjusting osd_memory_target on smithi080 to 4272M 2024-04-24T23:25:35.086 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:35 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:25:35.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:35 smithi080 ceph-mon[29152]: Detected new or changed devices on smithi080 2024-04-24T23:25:35.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:35 smithi080 ceph-mon[29152]: Adjusting osd_memory_target on smithi080 to 4272M 2024-04-24T23:25:35.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:35 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:25:35.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:35 smithi148 ceph-mon[32528]: Detected new or changed devices on smithi080 2024-04-24T23:25:35.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:35 smithi148 ceph-mon[32528]: Adjusting osd_memory_target on smithi080 to 4272M 2024-04-24T23:25:35.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:35 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:25:35.620 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:25:35.617+0000 7f4d88706700 1 -- 172.21.15.134:0/1111814651 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d801023b0 msgr2=0x7f4d80102810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:35.620 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:25:35.617+0000 7f4d88706700 1 --2- 172.21.15.134:0/1111814651 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d801023b0 0x7f4d80102810 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f4d700099c0 tx=0x7f4d70009cd0 comp rx=0 tx=0).stop 2024-04-24T23:25:35.621 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:25:35.617+0000 7f4d88706700 1 -- 172.21.15.134:0/1111814651 shutdown_connections 2024-04-24T23:25:35.621 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:25:35.617+0000 7f4d88706700 1 --2- 172.21.15.134:0/1111814651 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d80102d50 0x7f4d80109600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:35.621 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:25:35.617+0000 7f4d88706700 1 --2- 172.21.15.134:0/1111814651 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4d801011b0 0x7f4d801015d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:35.621 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:25:35.617+0000 7f4d88706700 1 --2- 172.21.15.134:0/1111814651 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d801023b0 0x7f4d80102810 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:35.621 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:25:35.617+0000 7f4d88706700 1 -- 172.21.15.134:0/1111814651 >> 172.21.15.134:0/1111814651 conn(0x7f4d800fc790 msgr2=0x7f4d800febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:35.621 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:25:35.617+0000 7f4d88706700 1 -- 172.21.15.134:0/1111814651 shutdown_connections 2024-04-24T23:25:35.621 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:25:35.617+0000 7f4d88706700 1 -- 172.21.15.134:0/1111814651 wait complete. 2024-04-24T23:25:35.621 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:25:35.618+0000 7f4d88706700 1 Processor -- start 2024-04-24T23:25:35.621 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:25:35.618+0000 7f4d88706700 1 -- start start 2024-04-24T23:25:35.621 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:25:35.618+0000 7f4d88706700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4d801011b0 0x7f4d80198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:35.621 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:25:35.618+0000 7f4d88706700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d801023b0 0x7f4d801989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:35.621 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:25:35.618+0000 7f4d88706700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d80102d50 0x7f4d8019fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:35.621 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:25:35.618+0000 7f4d88706700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f4d8010ce00 con 0x7f4d801011b0 2024-04-24T23:25:35.622 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:25:35.618+0000 7f4d88706700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4d8010cc80 con 0x7f4d80102d50 2024-04-24T23:25:35.622 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:25:35.618+0000 7f4d88706700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4d8010cf80 con 0x7f4d801023b0 2024-04-24T23:25:35.622 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:25:35.618+0000 7f4d86ca3700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d80102d50 0x7f4d8019fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:35.622 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:25:35.618+0000 7f4d86ca3700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d80102d50 0x7f4d8019fa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:49206/0 (socket says 172.21.15.134:49206) 2024-04-24T23:25:35.622 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:25:35.618+0000 7f4d86ca3700 1 -- 172.21.15.134:0/2172263276 learned_addr learned my addr 172.21.15.134:0/2172263276 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-04-24T23:25:35.622 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:25:35.618+0000 7f4d85ca1700 1 --2- 172.21.15.134:0/2172263276 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d801023b0 0x7f4d801989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:35.622 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:25:35.619+0000 7f4d864a2700 1 --2- 172.21.15.134:0/2172263276 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4d801011b0 0x7f4d80198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:35.622 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:25:35.619+0000 7f4d86ca3700 1 -- 172.21.15.134:0/2172263276 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d801023b0 msgr2=0x7f4d801989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:35.622 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:25:35.619+0000 7f4d86ca3700 1 --2- 172.21.15.134:0/2172263276 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d801023b0 0x7f4d801989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:35.622 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:25:35.619+0000 7f4d86ca3700 1 -- 172.21.15.134:0/2172263276 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4d801011b0 msgr2=0x7f4d80198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:35.622 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:25:35.619+0000 7f4d86ca3700 1 --2- 172.21.15.134:0/2172263276 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4d801011b0 0x7f4d80198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:35.623 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:25:35.619+0000 7f4d86ca3700 1 -- 172.21.15.134:0/2172263276 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d801a00d0 con 0x7f4d80102d50 2024-04-24T23:25:35.623 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:25:35.619+0000 7f4d86ca3700 1 --2- 172.21.15.134:0/2172263276 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d80102d50 0x7f4d8019fa30 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4d7800b970 tx=0x7f4d7800bd30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:35.623 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:25:35.619+0000 7f4d777fe700 1 -- 172.21.15.134:0/2172263276 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d78019070 con 0x7f4d80102d50 2024-04-24T23:25:35.623 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:25:35.619+0000 7f4d88706700 1 -- 172.21.15.134:0/2172263276 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d801a03c0 con 0x7f4d80102d50 2024-04-24T23:25:35.623 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:25:35.619+0000 7f4d777fe700 1 -- 172.21.15.134:0/2172263276 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d78014440 con 0x7f4d80102d50 2024-04-24T23:25:35.623 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:25:35.619+0000 7f4d777fe700 1 -- 172.21.15.134:0/2172263276 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d780075d0 con 0x7f4d80102d50 2024-04-24T23:25:35.623 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:25:35.619+0000 7f4d88706700 1 -- 172.21.15.134:0/2172263276 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d801a09d0 con 0x7f4d80102d50 2024-04-24T23:25:35.624 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:25:35.620+0000 7f4d88706700 1 -- 172.21.15.134:0/2172263276 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d80068f20 con 0x7f4d80102d50 2024-04-24T23:25:35.624 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:25:35.621+0000 7f4d777fe700 1 -- 172.21.15.134:0/2172263276 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f4d7801c070 con 0x7f4d80102d50 2024-04-24T23:25:35.624 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:25:35.622+0000 7f4d777fe700 1 --2- 172.21.15.134:0/2172263276 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f4d6c0a2c20 0x7f4d6c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:35.624 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:25:35.622+0000 7f4d777fe700 1 -- 172.21.15.134:0/2172263276 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6261+0+0 (secure 0 0 0) 0x7f4d780ce0d0 con 0x7f4d80102d50 2024-04-24T23:25:35.624 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:25:35.622+0000 7f4d864a2700 1 --2- 172.21.15.134:0/2172263276 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f4d6c0a2c20 0x7f4d6c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:35.624 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:25:35.622+0000 7f4d864a2700 1 --2- 172.21.15.134:0/2172263276 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f4d6c0a2c20 0x7f4d6c0a50e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c005fd0 tx=0x7f4d7c005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:35.626 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:25:35.626+0000 7f4d777fe700 1 -- 172.21.15.134:0/2172263276 <== mon.2 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f4d7809b830 con 0x7f4d80102d50 2024-04-24T23:25:35.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:25:35.830+0000 7f4d88706700 1 -- 172.21.15.134:0/2172263276 --> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f4d800fe880 con 0x7f4d6c0a2c20 2024-04-24T23:25:36.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:36 smithi134 ceph-mon[32619]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:25:36.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:36 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:25:36.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:36 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:25:36.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:36 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:25:36.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:36 smithi080 ceph-mon[29152]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:25:36.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:36 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:25:36.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:36 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:25:36.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:36 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:25:36.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:36 smithi148 ceph-mon[32528]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:25:36.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:36 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:25:36.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:36 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:25:36.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:36 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:25:37.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:37 smithi134 ceph-mon[32619]: from='client.24326 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:25:37.290 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:37 smithi080 ceph-mon[29152]: from='client.24326 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:25:37.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:37 smithi148 ceph-mon[32528]: from='client.24326 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:25:38.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:38 smithi080 ceph-mon[29152]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:25:38.407 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:38 smithi148 ceph-mon[32528]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:25:38.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:38 smithi134 ceph-mon[32619]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:25:39.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:39 smithi080 ceph-mon[29152]: from='client.? 172.21.15.134:0/2397614461' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8d92f0d9-5c64-429a-b750-c33ae0e070c3"}]: dispatch 2024-04-24T23:25:39.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:39 smithi080 ceph-mon[29152]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8d92f0d9-5c64-429a-b750-c33ae0e070c3"}]: dispatch 2024-04-24T23:25:39.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:39 smithi080 ceph-mon[29152]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8d92f0d9-5c64-429a-b750-c33ae0e070c3"}]': finished 2024-04-24T23:25:39.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:39 smithi080 ceph-mon[29152]: osdmap e69: 11 total, 10 up, 11 in 2024-04-24T23:25:39.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:39 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T23:25:39.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:39 smithi134 ceph-mon[32619]: from='client.? 172.21.15.134:0/2397614461' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8d92f0d9-5c64-429a-b750-c33ae0e070c3"}]: dispatch 2024-04-24T23:25:39.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:39 smithi134 ceph-mon[32619]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8d92f0d9-5c64-429a-b750-c33ae0e070c3"}]: dispatch 2024-04-24T23:25:39.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:39 smithi134 ceph-mon[32619]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8d92f0d9-5c64-429a-b750-c33ae0e070c3"}]': finished 2024-04-24T23:25:39.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:39 smithi134 ceph-mon[32619]: osdmap e69: 11 total, 10 up, 11 in 2024-04-24T23:25:39.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:39 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T23:25:39.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:39 smithi148 ceph-mon[32528]: from='client.? 172.21.15.134:0/2397614461' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8d92f0d9-5c64-429a-b750-c33ae0e070c3"}]: dispatch 2024-04-24T23:25:39.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:39 smithi148 ceph-mon[32528]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8d92f0d9-5c64-429a-b750-c33ae0e070c3"}]: dispatch 2024-04-24T23:25:39.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:39 smithi148 ceph-mon[32528]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8d92f0d9-5c64-429a-b750-c33ae0e070c3"}]': finished 2024-04-24T23:25:39.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:39 smithi148 ceph-mon[32528]: osdmap e69: 11 total, 10 up, 11 in 2024-04-24T23:25:39.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:39 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T23:25:40.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:40 smithi080 ceph-mon[29152]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:25:40.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:40 smithi080 ceph-mon[29152]: from='client.? 172.21.15.134:0/3542071679' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:25:40.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:40 smithi134 ceph-mon[32619]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:25:40.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:40 smithi134 ceph-mon[32619]: from='client.? 172.21.15.134:0/3542071679' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:25:40.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:40 smithi148 ceph-mon[32528]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:25:40.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:40 smithi148 ceph-mon[32528]: from='client.? 172.21.15.134:0/3542071679' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:25:42.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:42 smithi080 ceph-mon[29152]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:25:42.299 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:42 smithi134 ceph-mon[32619]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:25:42.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:42 smithi148 ceph-mon[32528]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:25:44.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:44 smithi134 ceph-mon[32619]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:25:44.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:44 smithi080 ceph-mon[29152]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:25:44.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:44 smithi148 ceph-mon[32528]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:25:46.167 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:46 smithi134 ceph-mon[32619]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:25:46.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:46 smithi080 ceph-mon[29152]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:25:46.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:46 smithi148 ceph-mon[32528]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:25:47.057 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:47 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-24T23:25:47.057 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:47 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:25:47.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:47 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-24T23:25:47.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:47 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:25:47.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:47 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-24T23:25:47.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:47 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:25:48.101 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:48 smithi134 ceph-mon[32619]: Deploying daemon osd.10 on smithi134 2024-04-24T23:25:48.101 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:48 smithi134 ceph-mon[32619]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:25:48.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:48 smithi080 ceph-mon[29152]: Deploying daemon osd.10 on smithi134 2024-04-24T23:25:48.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:48 smithi080 ceph-mon[29152]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:25:48.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:48 smithi148 ceph-mon[32528]: Deploying daemon osd.10 on smithi134 2024-04-24T23:25:48.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:48 smithi148 ceph-mon[32528]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:25:50.214 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:50 smithi134 ceph-mon[32619]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:25:50.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:50 smithi148 ceph-mon[32528]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:25:50.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:50 smithi080 ceph-mon[29152]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:25:52.380 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:52 smithi134 ceph-mon[32619]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:25:52.380 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:52 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:25:52.380 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:52 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:25:52.380 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:52 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:25:52.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:52 smithi148 ceph-mon[32528]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:25:52.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:52 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:25:52.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:52 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:25:52.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:52 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:25:52.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:52 smithi080 ceph-mon[29152]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:25:52.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:52 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:25:52.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:52 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:25:52.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:52 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:25:54.297 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:54 smithi134 ceph-mon[32619]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:25:54.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:54 smithi148 ceph-mon[32528]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:25:54.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:54 smithi080 ceph-mon[29152]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:25:55.699 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:25:55.697+0000 7f4d777fe700 1 -- 172.21.15.134:0/2172263276 <== mgr.14152 v2:172.21.15.80:6800/3830596922 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f4d800fe880 con 0x7f4d6c0a2c20 2024-04-24T23:25:55.699 INFO:teuthology.orchestra.run.smithi134.stdout:Created osd(s) 10 on host 'smithi134' 2024-04-24T23:25:55.702 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:25:55.701+0000 7f4d88706700 1 -- 172.21.15.134:0/2172263276 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f4d6c0a2c20 msgr2=0x7f4d6c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:55.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:25:55.701+0000 7f4d88706700 1 --2- 172.21.15.134:0/2172263276 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f4d6c0a2c20 0x7f4d6c0a50e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c005fd0 tx=0x7f4d7c005e50 comp rx=0 tx=0).stop 2024-04-24T23:25:55.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:25:55.701+0000 7f4d88706700 1 -- 172.21.15.134:0/2172263276 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d80102d50 msgr2=0x7f4d8019fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:55.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:25:55.701+0000 7f4d88706700 1 --2- 172.21.15.134:0/2172263276 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d80102d50 0x7f4d8019fa30 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4d7800b970 tx=0x7f4d7800bd30 comp rx=0 tx=0).stop 2024-04-24T23:25:55.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:25:55.701+0000 7f4d88706700 1 -- 172.21.15.134:0/2172263276 shutdown_connections 2024-04-24T23:25:55.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:25:55.701+0000 7f4d88706700 1 --2- 172.21.15.134:0/2172263276 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d80102d50 0x7f4d8019fa30 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:55.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:25:55.701+0000 7f4d88706700 1 --2- 172.21.15.134:0/2172263276 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f4d6c0a2c20 0x7f4d6c0a50e0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:55.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:25:55.701+0000 7f4d88706700 1 --2- 172.21.15.134:0/2172263276 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4d801011b0 0x7f4d80198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:55.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:25:55.701+0000 7f4d88706700 1 --2- 172.21.15.134:0/2172263276 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d801023b0 0x7f4d801989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:55.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:25:55.701+0000 7f4d88706700 1 -- 172.21.15.134:0/2172263276 >> 172.21.15.134:0/2172263276 conn(0x7f4d800fc790 msgr2=0x7f4d800fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:55.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:25:55.701+0000 7f4d88706700 1 -- 172.21.15.134:0/2172263276 shutdown_connections 2024-04-24T23:25:55.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-24T23:25:55.701+0000 7f4d88706700 1 -- 172.21.15.134:0/2172263276 wait complete. 2024-04-24T23:25:56.366 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:56 smithi134 ceph-mon[32619]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:25:56.367 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:56 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:25:56.367 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:56 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:25:56.367 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:56 smithi134 ceph-mon[32619]: from='osd.10 [v2:172.21.15.134:6824/647343766,v1:172.21.15.134:6825/647343766]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-24T23:25:56.367 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:56 smithi134 ceph-mon[32619]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-24T23:25:56.397 DEBUG:teuthology.orchestra.run.smithi134:osd.10> sudo journalctl -f -n 0 -u ceph-ec7885aa-0290-11ef-bc93-c7b262605968@osd.10.service 2024-04-24T23:25:56.399 INFO:tasks.cephadm:Deploying osd.11 on smithi148 with /dev/vg_nvme/lv_1... 2024-04-24T23:25:56.400 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ec7885aa-0290-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-24T23:25:56.427 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:56 smithi148 ceph-mon[32528]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:25:56.427 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:56 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:25:56.427 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:56 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:25:56.427 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:56 smithi148 ceph-mon[32528]: from='osd.10 [v2:172.21.15.134:6824/647343766,v1:172.21.15.134:6825/647343766]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-24T23:25:56.427 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:56 smithi148 ceph-mon[32528]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-24T23:25:56.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:56 smithi080 ceph-mon[29152]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:25:56.539 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:56 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:25:56.539 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:56 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:25:56.539 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:56 smithi080 ceph-mon[29152]: from='osd.10 [v2:172.21.15.134:6824/647343766,v1:172.21.15.134:6825/647343766]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-24T23:25:56.539 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:56 smithi080 ceph-mon[29152]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-24T23:25:56.649 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T23:25:57.686 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:57 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:25:57.686 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:57 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:25:57.686 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:57 smithi134 ceph-mon[32619]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-24T23:25:57.686 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:57 smithi134 ceph-mon[32619]: osdmap e70: 11 total, 10 up, 11 in 2024-04-24T23:25:57.686 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:57 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T23:25:57.686 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:57 smithi134 ceph-mon[32619]: from='osd.10 [v2:172.21.15.134:6824/647343766,v1:172.21.15.134:6825/647343766]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-04-24T23:25:57.686 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:57 smithi134 ceph-mon[32619]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-04-24T23:25:57.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:57 smithi134 ceph-mon[32619]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:25:57.756 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:57 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:25:57.756 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:57 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:25:57.756 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:57 smithi148 ceph-mon[32528]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-24T23:25:57.756 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:57 smithi148 ceph-mon[32528]: osdmap e70: 11 total, 10 up, 11 in 2024-04-24T23:25:57.756 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:57 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T23:25:57.756 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:57 smithi148 ceph-mon[32528]: from='osd.10 [v2:172.21.15.134:6824/647343766,v1:172.21.15.134:6825/647343766]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-04-24T23:25:57.757 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:57 smithi148 ceph-mon[32528]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-04-24T23:25:57.757 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:57 smithi148 ceph-mon[32528]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:25:57.788 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:57 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:25:57.788 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:57 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:25:57.788 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:57 smithi080 ceph-mon[29152]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-24T23:25:57.789 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:57 smithi080 ceph-mon[29152]: osdmap e70: 11 total, 10 up, 11 in 2024-04-24T23:25:57.789 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:57 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T23:25:57.789 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:57 smithi080 ceph-mon[29152]: from='osd.10 [v2:172.21.15.134:6824/647343766,v1:172.21.15.134:6825/647343766]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-04-24T23:25:57.789 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:57 smithi080 ceph-mon[29152]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-04-24T23:25:57.789 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:57 smithi080 ceph-mon[29152]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:25:57.947 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:25:57 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:25:57.684+0000 7f02a1d08700 -1 osd.10 0 waiting for initial osdmap 2024-04-24T23:25:57.947 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:25:57 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:25:57.695+0000 7f029dafb700 -1 osd.10 71 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T23:25:58.688 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:58 smithi134 ceph-mon[32619]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-04-24T23:25:58.688 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:58 smithi134 ceph-mon[32619]: osdmap e71: 11 total, 10 up, 11 in 2024-04-24T23:25:58.689 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:58 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T23:25:58.689 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:58 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T23:25:58.689 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:58 smithi134 ceph-mon[32619]: osd.10 [v2:172.21.15.134:6824/647343766,v1:172.21.15.134:6825/647343766] boot 2024-04-24T23:25:58.689 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:58 smithi134 ceph-mon[32619]: osdmap e72: 11 total, 11 up, 11 in 2024-04-24T23:25:58.689 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:58 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T23:25:58.863 INFO:teuthology.orchestra.run.smithi148.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-24T23:25:58.864 INFO:teuthology.orchestra.run.smithi148.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-24T23:25:58.864 INFO:teuthology.orchestra.run.smithi148.stderr: stderr: 10+0 records in 2024-04-24T23:25:58.864 INFO:teuthology.orchestra.run.smithi148.stderr:10+0 records out 2024-04-24T23:25:58.864 INFO:teuthology.orchestra.run.smithi148.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0131768 s, 796 MB/s 2024-04-24T23:25:58.864 INFO:teuthology.orchestra.run.smithi148.stderr:--> Zapping successful for: 2024-04-24T23:25:59.006 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:58 smithi148 ceph-mon[32528]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-04-24T23:25:59.006 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:58 smithi148 ceph-mon[32528]: osdmap e71: 11 total, 10 up, 11 in 2024-04-24T23:25:59.007 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:58 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T23:25:59.007 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:58 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T23:25:59.007 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:58 smithi148 ceph-mon[32528]: osd.10 [v2:172.21.15.134:6824/647343766,v1:172.21.15.134:6825/647343766] boot 2024-04-24T23:25:59.007 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:58 smithi148 ceph-mon[32528]: osdmap e72: 11 total, 11 up, 11 in 2024-04-24T23:25:59.007 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:58 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T23:25:59.038 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:58 smithi080 ceph-mon[29152]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-04-24T23:25:59.038 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:58 smithi080 ceph-mon[29152]: osdmap e71: 11 total, 10 up, 11 in 2024-04-24T23:25:59.038 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:58 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T23:25:59.038 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:58 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T23:25:59.038 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:58 smithi080 ceph-mon[29152]: osd.10 [v2:172.21.15.134:6824/647343766,v1:172.21.15.134:6825/647343766] boot 2024-04-24T23:25:59.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:58 smithi080 ceph-mon[29152]: osdmap e72: 11 total, 11 up, 11 in 2024-04-24T23:25:59.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:58 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T23:25:59.646 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ec7885aa-0290-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi148:vg_nvme/lv_1 2024-04-24T23:25:59.934 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:59 smithi148 ceph-mon[32528]: purged_snaps scrub starts 2024-04-24T23:25:59.934 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:59 smithi148 ceph-mon[32528]: purged_snaps scrub ok 2024-04-24T23:25:59.934 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:59 smithi148 ceph-mon[32528]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:25:59.934 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:25:59 smithi148 ceph-mon[32528]: osdmap e73: 11 total, 11 up, 11 in 2024-04-24T23:25:59.939 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T23:25:59.947 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:59 smithi134 ceph-mon[32619]: purged_snaps scrub starts 2024-04-24T23:25:59.947 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:59 smithi134 ceph-mon[32619]: purged_snaps scrub ok 2024-04-24T23:25:59.947 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:59 smithi134 ceph-mon[32619]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:25:59.947 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:25:59 smithi134 ceph-mon[32619]: osdmap e73: 11 total, 11 up, 11 in 2024-04-24T23:25:59.963 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:59 smithi080 ceph-mon[29152]: purged_snaps scrub starts 2024-04-24T23:25:59.963 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:59 smithi080 ceph-mon[29152]: purged_snaps scrub ok 2024-04-24T23:25:59.963 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:59 smithi080 ceph-mon[29152]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:25:59.963 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:25:59 smithi080 ceph-mon[29152]: osdmap e73: 11 total, 11 up, 11 in 2024-04-24T23:26:01.169 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:01 smithi148 ceph-mon[32528]: osdmap e74: 11 total, 11 up, 11 in 2024-04-24T23:26:01.170 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:01 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:26:01.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:01 smithi080 ceph-mon[29152]: osdmap e74: 11 total, 11 up, 11 in 2024-04-24T23:26:01.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:01 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:26:01.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:01 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:26:01.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:01 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:26:01.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:01 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:26:01.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:01 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:26:01.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:01 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:26:01.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:01 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:26:01.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:01 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:26:01.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:01 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:26:01.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:01 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:26:01.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:01 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:26:01.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:01 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:26:01.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:01 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:26:01.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:01 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:26:01.431 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:01 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:26:01.431 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:01 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:26:01.431 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:01 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:26:01.431 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:01 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:26:01.431 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:01 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:26:01.431 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:01 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:26:01.431 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:01 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:26:01.431 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:01 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:26:01.431 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:01 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:26:01.432 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:01 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:26:01.432 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:01 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:26:01.432 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:01 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:26:01.432 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:01 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:26:01.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:01 smithi134 ceph-mon[32619]: osdmap e74: 11 total, 11 up, 11 in 2024-04-24T23:26:01.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:01 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:26:01.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:01 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:26:01.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:01 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:26:01.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:01 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:26:01.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:01 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:26:01.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:01 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:26:01.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:01 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:26:01.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:01 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:26:01.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:01 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:26:01.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:01 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:26:01.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:01 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:26:01.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:01 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:26:01.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:01 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:26:01.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:01 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:26:01.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:26:01.668+0000 7fe8dbfc8700 1 -- 172.21.15.148:0/3495298343 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe8d4100340 msgr2=0x7fe8d41007c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:01.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:26:01.668+0000 7fe8dbfc8700 1 --2- 172.21.15.148:0/3495298343 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe8d4100340 0x7fe8d41007c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe8c40099c0 tx=0x7fe8c4009cd0 comp rx=0 tx=0).stop 2024-04-24T23:26:01.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:26:01.668+0000 7fe8dbfc8700 1 -- 172.21.15.148:0/3495298343 shutdown_connections 2024-04-24T23:26:01.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:26:01.668+0000 7fe8dbfc8700 1 --2- 172.21.15.148:0/3495298343 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe8d4100d00 0x7fe8d410a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:01.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:26:01.668+0000 7fe8dbfc8700 1 --2- 172.21.15.148:0/3495298343 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe8d41038c0 0x7fe8d4103ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:01.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:26:01.668+0000 7fe8dbfc8700 1 --2- 172.21.15.148:0/3495298343 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe8d4100340 0x7fe8d41007c0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:01.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:26:01.668+0000 7fe8dbfc8700 1 -- 172.21.15.148:0/3495298343 >> 172.21.15.148:0/3495298343 conn(0x7fe8d40fbee0 msgr2=0x7fe8d40fe320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:01.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:26:01.668+0000 7fe8dbfc8700 1 -- 172.21.15.148:0/3495298343 shutdown_connections 2024-04-24T23:26:01.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:26:01.668+0000 7fe8dbfc8700 1 -- 172.21.15.148:0/3495298343 wait complete. 2024-04-24T23:26:01.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:26:01.669+0000 7fe8dbfc8700 1 Processor -- start 2024-04-24T23:26:01.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:26:01.669+0000 7fe8dbfc8700 1 -- start start 2024-04-24T23:26:01.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:26:01.669+0000 7fe8dbfc8700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe8d4100340 0x7fe8d41984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:01.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:26:01.669+0000 7fe8dbfc8700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe8d4100d00 0x7fe8d4198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:01.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:26:01.669+0000 7fe8dbfc8700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe8d41038c0 0x7fe8d419fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:01.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:26:01.669+0000 7fe8dbfc8700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fe8d410deb0 con 0x7fe8d4100340 2024-04-24T23:26:01.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:26:01.669+0000 7fe8dbfc8700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe8d410dd30 con 0x7fe8d41038c0 2024-04-24T23:26:01.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:26:01.669+0000 7fe8dbfc8700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe8d410e030 con 0x7fe8d4100d00 2024-04-24T23:26:01.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:26:01.669+0000 7fe8d9563700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe8d4100d00 0x7fe8d4198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:01.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:26:01.669+0000 7fe8d9563700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe8d4100d00 0x7fe8d4198a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40200/0 (socket says 172.21.15.148:40200) 2024-04-24T23:26:01.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:26:01.669+0000 7fe8d9d64700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe8d4100340 0x7fe8d41984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:01.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:26:01.669+0000 7fe8d9563700 1 -- 172.21.15.148:0/2879869002 learned_addr learned my addr 172.21.15.148:0/2879869002 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-24T23:26:01.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:26:01.669+0000 7fe8da565700 1 --2- 172.21.15.148:0/2879869002 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe8d41038c0 0x7fe8d419fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:01.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:26:01.669+0000 7fe8d9563700 1 -- 172.21.15.148:0/2879869002 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe8d41038c0 msgr2=0x7fe8d419fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:01.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:26:01.669+0000 7fe8d9563700 1 --2- 172.21.15.148:0/2879869002 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe8d41038c0 0x7fe8d419fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:01.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:26:01.669+0000 7fe8d9563700 1 -- 172.21.15.148:0/2879869002 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe8d4100340 msgr2=0x7fe8d41984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:01.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:26:01.669+0000 7fe8d9563700 1 --2- 172.21.15.148:0/2879869002 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe8d4100340 0x7fe8d41984c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:01.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:26:01.669+0000 7fe8d9563700 1 -- 172.21.15.148:0/2879869002 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8d41a0120 con 0x7fe8d4100d00 2024-04-24T23:26:01.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:26:01.669+0000 7fe8da565700 1 --2- 172.21.15.148:0/2879869002 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe8d41038c0 0x7fe8d419fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:26:01.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:26:01.669+0000 7fe8d9563700 1 --2- 172.21.15.148:0/2879869002 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe8d4100d00 0x7fe8d4198a00 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe8c400fab0 tx=0x7fe8c400fae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:01.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:26:01.670+0000 7fe8caffd700 1 -- 172.21.15.148:0/2879869002 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c4027070 con 0x7fe8d4100d00 2024-04-24T23:26:01.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:26:01.670+0000 7fe8dbfc8700 1 -- 172.21.15.148:0/2879869002 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8d41a03b0 con 0x7fe8d4100d00 2024-04-24T23:26:01.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:26:01.670+0000 7fe8caffd700 1 -- 172.21.15.148:0/2879869002 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8c400fdb0 con 0x7fe8d4100d00 2024-04-24T23:26:01.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:26:01.670+0000 7fe8caffd700 1 -- 172.21.15.148:0/2879869002 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c40173f0 con 0x7fe8d4100d00 2024-04-24T23:26:01.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:26:01.670+0000 7fe8dbfc8700 1 -- 172.21.15.148:0/2879869002 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8d41a0890 con 0x7fe8d4100d00 2024-04-24T23:26:01.674 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:26:01.671+0000 7fe8dbfc8700 1 -- 172.21.15.148:0/2879869002 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8d4068f20 con 0x7fe8d4100d00 2024-04-24T23:26:01.674 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:26:01.672+0000 7fe8caffd700 1 -- 172.21.15.148:0/2879869002 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fe8c402a070 con 0x7fe8d4100d00 2024-04-24T23:26:01.674 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:26:01.672+0000 7fe8caffd700 1 --2- 172.21.15.148:0/2879869002 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fe8c00a2c20 0x7fe8c00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:01.674 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:26:01.672+0000 7fe8caffd700 1 -- 172.21.15.148:0/2879869002 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6694+0+0 (secure 0 0 0) 0x7fe8c40cf660 con 0x7fe8d4100d00 2024-04-24T23:26:01.674 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:26:01.672+0000 7fe8d9d64700 1 --2- 172.21.15.148:0/2879869002 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fe8c00a2c20 0x7fe8c00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:01.674 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:26:01.673+0000 7fe8d9d64700 1 --2- 172.21.15.148:0/2879869002 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fe8c00a2c20 0x7fe8c00a50e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe8d406a6a0 tx=0x7fe8d000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:01.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:26:01.677+0000 7fe8caffd700 1 -- 172.21.15.148:0/2879869002 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fe8c4013070 con 0x7fe8d4100d00 2024-04-24T23:26:01.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:26:01.879+0000 7fe8dbfc8700 1 -- 172.21.15.148:0/2879869002 --> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fe8d4062670 con 0x7fe8c00a2c20 2024-04-24T23:26:02.436 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:02 smithi134 ceph-mon[32619]: Detected new or changed devices on smithi134 2024-04-24T23:26:02.436 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:02 smithi134 ceph-mon[32619]: Adjusting osd_memory_target on smithi134 to 4272M 2024-04-24T23:26:02.436 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:02 smithi134 ceph-mon[32619]: pgmap v241: 1 pgs: 1 remapped+peering; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:26:02.436 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:02 smithi134 ceph-mon[32619]: osdmap e75: 11 total, 11 up, 11 in 2024-04-24T23:26:02.436 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:02 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:26:02.436 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:02 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:26:02.436 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:02 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:26:02.494 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:02 smithi148 ceph-mon[32528]: Detected new or changed devices on smithi134 2024-04-24T23:26:02.494 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:02 smithi148 ceph-mon[32528]: Adjusting osd_memory_target on smithi134 to 4272M 2024-04-24T23:26:02.494 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:02 smithi148 ceph-mon[32528]: pgmap v241: 1 pgs: 1 remapped+peering; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:26:02.494 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:02 smithi148 ceph-mon[32528]: osdmap e75: 11 total, 11 up, 11 in 2024-04-24T23:26:02.494 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:02 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:26:02.494 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:02 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:26:02.494 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:02 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:26:02.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:02 smithi080 ceph-mon[29152]: Detected new or changed devices on smithi134 2024-04-24T23:26:02.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:02 smithi080 ceph-mon[29152]: Adjusting osd_memory_target on smithi134 to 4272M 2024-04-24T23:26:02.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:02 smithi080 ceph-mon[29152]: pgmap v241: 1 pgs: 1 remapped+peering; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:26:02.539 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:02 smithi080 ceph-mon[29152]: osdmap e75: 11 total, 11 up, 11 in 2024-04-24T23:26:02.539 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:02 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:26:02.539 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:02 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:26:02.539 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:02 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:26:03.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:03 smithi134 ceph-mon[32619]: from='client.24355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:26:03.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:03 smithi134 ceph-mon[32619]: osdmap e76: 11 total, 11 up, 11 in 2024-04-24T23:26:03.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:03 smithi148 ceph-mon[32528]: from='client.24355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:26:03.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:03 smithi148 ceph-mon[32528]: osdmap e76: 11 total, 11 up, 11 in 2024-04-24T23:26:03.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:03 smithi080 ceph-mon[29152]: from='client.24355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:26:03.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:03 smithi080 ceph-mon[29152]: osdmap e76: 11 total, 11 up, 11 in 2024-04-24T23:26:04.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:04 smithi134 ceph-mon[32619]: pgmap v244: 1 pgs: 1 remapped+peering; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:26:04.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:04 smithi148 ceph-mon[32528]: pgmap v244: 1 pgs: 1 remapped+peering; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:26:04.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:04 smithi080 ceph-mon[29152]: pgmap v244: 1 pgs: 1 remapped+peering; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:26:05.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:05 smithi134 ceph-mon[32619]: from='client.? 172.21.15.148:0/2931557296' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4c003a41-f65b-479f-bc80-b27d680cf936"}]: dispatch 2024-04-24T23:26:05.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:05 smithi134 ceph-mon[32619]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4c003a41-f65b-479f-bc80-b27d680cf936"}]: dispatch 2024-04-24T23:26:05.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:05 smithi134 ceph-mon[32619]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4c003a41-f65b-479f-bc80-b27d680cf936"}]': finished 2024-04-24T23:26:05.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:05 smithi134 ceph-mon[32619]: osdmap e77: 12 total, 11 up, 12 in 2024-04-24T23:26:05.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:05 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T23:26:05.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:05 smithi148 ceph-mon[32528]: from='client.? 172.21.15.148:0/2931557296' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4c003a41-f65b-479f-bc80-b27d680cf936"}]: dispatch 2024-04-24T23:26:05.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:05 smithi148 ceph-mon[32528]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4c003a41-f65b-479f-bc80-b27d680cf936"}]: dispatch 2024-04-24T23:26:05.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:05 smithi148 ceph-mon[32528]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4c003a41-f65b-479f-bc80-b27d680cf936"}]': finished 2024-04-24T23:26:05.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:05 smithi148 ceph-mon[32528]: osdmap e77: 12 total, 11 up, 12 in 2024-04-24T23:26:05.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:05 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T23:26:05.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:05 smithi080 ceph-mon[29152]: from='client.? 172.21.15.148:0/2931557296' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4c003a41-f65b-479f-bc80-b27d680cf936"}]: dispatch 2024-04-24T23:26:05.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:05 smithi080 ceph-mon[29152]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4c003a41-f65b-479f-bc80-b27d680cf936"}]: dispatch 2024-04-24T23:26:05.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:05 smithi080 ceph-mon[29152]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4c003a41-f65b-479f-bc80-b27d680cf936"}]': finished 2024-04-24T23:26:05.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:05 smithi080 ceph-mon[29152]: osdmap e77: 12 total, 11 up, 12 in 2024-04-24T23:26:05.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:05 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T23:26:06.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:06 smithi134 ceph-mon[32619]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 123 KiB/s, 0 objects/s recovering 2024-04-24T23:26:06.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:06 smithi134 ceph-mon[32619]: from='client.? 172.21.15.148:0/3629116616' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:26:06.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:06 smithi148 ceph-mon[32528]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 123 KiB/s, 0 objects/s recovering 2024-04-24T23:26:06.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:06 smithi148 ceph-mon[32528]: from='client.? 172.21.15.148:0/3629116616' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:26:06.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:06 smithi080 ceph-mon[29152]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 123 KiB/s, 0 objects/s recovering 2024-04-24T23:26:06.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:06 smithi080 ceph-mon[29152]: from='client.? 172.21.15.148:0/3629116616' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:26:07.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:07 smithi134 ceph-mon[32619]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-24T23:26:07.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:07 smithi148 ceph-mon[32528]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-24T23:26:07.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:07 smithi080 ceph-mon[29152]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-24T23:26:10.168 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:09 smithi148 ceph-mon[32528]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-04-24T23:26:10.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:09 smithi134 ceph-mon[32619]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-04-24T23:26:10.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:09 smithi080 ceph-mon[29152]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-04-24T23:26:12.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:11 smithi134 ceph-mon[32619]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-04-24T23:26:12.250 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:11 smithi148 ceph-mon[32528]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-04-24T23:26:12.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:11 smithi080 ceph-mon[29152]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-04-24T23:26:13.006 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:12 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-24T23:26:13.006 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:12 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:26:13.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:12 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-24T23:26:13.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:12 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:26:13.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:12 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-24T23:26:13.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:12 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:26:14.006 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:13 smithi148 ceph-mon[32528]: Deploying daemon osd.11 on smithi148 2024-04-24T23:26:14.006 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:13 smithi148 ceph-mon[32528]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-24T23:26:14.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:13 smithi134 ceph-mon[32619]: Deploying daemon osd.11 on smithi148 2024-04-24T23:26:14.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:13 smithi134 ceph-mon[32619]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-24T23:26:14.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:13 smithi080 ceph-mon[29152]: Deploying daemon osd.11 on smithi148 2024-04-24T23:26:14.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:13 smithi080 ceph-mon[29152]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-24T23:26:16.006 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:15 smithi148 ceph-mon[32528]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:26:16.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:15 smithi134 ceph-mon[32619]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:26:16.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:15 smithi080 ceph-mon[29152]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:26:18.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:17 smithi134 ceph-mon[32619]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:26:18.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:17 smithi148 ceph-mon[32528]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:26:18.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:17 smithi080 ceph-mon[29152]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:26:20.192 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:19 smithi148 ceph-mon[32528]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:26:20.192 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:19 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:26:20.192 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:19 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:26:20.192 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:19 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:26:20.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:19 smithi080 ceph-mon[29152]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:26:20.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:19 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:26:20.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:19 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:26:20.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:19 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:26:20.449 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:19 smithi134 ceph-mon[32619]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:26:20.450 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:19 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:26:20.450 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:19 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:26:20.450 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:19 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:26:22.249 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:21 smithi134 ceph-mon[32619]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:26:22.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:21 smithi148 ceph-mon[32528]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:26:22.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:21 smithi080 ceph-mon[29152]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:26:22.549 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:26:22.548+0000 7fe8caffd700 1 -- 172.21.15.148:0/2879869002 <== mgr.14152 v2:172.21.15.80:6800/3830596922 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fe8d4062670 con 0x7fe8c00a2c20 2024-04-24T23:26:22.550 INFO:teuthology.orchestra.run.smithi148.stdout:Created osd(s) 11 on host 'smithi148' 2024-04-24T23:26:22.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:26:22.551+0000 7fe8dbfc8700 1 -- 172.21.15.148:0/2879869002 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fe8c00a2c20 msgr2=0x7fe8c00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:22.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:26:22.551+0000 7fe8dbfc8700 1 --2- 172.21.15.148:0/2879869002 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fe8c00a2c20 0x7fe8c00a50e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe8d406a6a0 tx=0x7fe8d000b410 comp rx=0 tx=0).stop 2024-04-24T23:26:22.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:26:22.551+0000 7fe8dbfc8700 1 -- 172.21.15.148:0/2879869002 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe8d4100d00 msgr2=0x7fe8d4198a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:22.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:26:22.551+0000 7fe8dbfc8700 1 --2- 172.21.15.148:0/2879869002 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe8d4100d00 0x7fe8d4198a00 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe8c400fab0 tx=0x7fe8c400fae0 comp rx=0 tx=0).stop 2024-04-24T23:26:22.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:26:22.551+0000 7fe8dbfc8700 1 -- 172.21.15.148:0/2879869002 shutdown_connections 2024-04-24T23:26:22.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:26:22.551+0000 7fe8dbfc8700 1 --2- 172.21.15.148:0/2879869002 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe8d41038c0 0x7fe8d419fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:22.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:26:22.551+0000 7fe8dbfc8700 1 --2- 172.21.15.148:0/2879869002 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fe8c00a2c20 0x7fe8c00a50e0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:22.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:26:22.551+0000 7fe8dbfc8700 1 --2- 172.21.15.148:0/2879869002 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe8d4100340 0x7fe8d41984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:22.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:26:22.551+0000 7fe8dbfc8700 1 --2- 172.21.15.148:0/2879869002 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe8d4100d00 0x7fe8d4198a00 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:22.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:26:22.551+0000 7fe8dbfc8700 1 -- 172.21.15.148:0/2879869002 >> 172.21.15.148:0/2879869002 conn(0x7fe8d40fbee0 msgr2=0x7fe8d4106900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:22.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:26:22.551+0000 7fe8dbfc8700 1 -- 172.21.15.148:0/2879869002 shutdown_connections 2024-04-24T23:26:22.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:26:22.551+0000 7fe8dbfc8700 1 -- 172.21.15.148:0/2879869002 wait complete. 2024-04-24T23:26:23.510 DEBUG:teuthology.orchestra.run.smithi148:osd.11> sudo journalctl -f -n 0 -u ceph-ec7885aa-0290-11ef-bc93-c7b262605968@osd.11.service 2024-04-24T23:26:23.512 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-04-24T23:26:23.513 DEBUG:teuthology.orchestra.run.smithi080:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ec7885aa-0290-11ef-bc93-c7b262605968 -- ceph osd stat -f json 2024-04-24T23:26:23.756 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:23 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:26:23.756 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:23 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:26:23.756 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:23 smithi148 ceph-mon[32528]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:26:23.757 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:23 smithi148 ceph-mon[32528]: from='osd.11 [v2:172.21.15.148:6824/4068024116,v1:172.21.15.148:6825/4068024116]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-24T23:26:23.757 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:23 smithi148 ceph-mon[32528]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-24T23:26:23.766 INFO:teuthology.orchestra.run.smithi080.stderr:Inferring config /var/lib/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:26:23.788 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:23 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:26:23.788 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:23 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:26:23.788 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:23 smithi080 ceph-mon[29152]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:26:23.788 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:23 smithi080 ceph-mon[29152]: from='osd.11 [v2:172.21.15.148:6824/4068024116,v1:172.21.15.148:6825/4068024116]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-24T23:26:23.788 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:23 smithi080 ceph-mon[29152]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-24T23:26:23.947 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:23 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:26:23.947 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:23 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:26:23.947 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:23 smithi134 ceph-mon[32619]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:26:23.947 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:23 smithi134 ceph-mon[32619]: from='osd.11 [v2:172.21.15.148:6824/4068024116,v1:172.21.15.148:6825/4068024116]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-24T23:26:23.947 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:23 smithi134 ceph-mon[32619]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-24T23:26:24.641 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:24 smithi148 ceph-mon[32528]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-24T23:26:24.641 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:24 smithi148 ceph-mon[32528]: osdmap e78: 12 total, 11 up, 12 in 2024-04-24T23:26:24.641 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:24 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T23:26:24.641 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:24 smithi148 ceph-mon[32528]: from='osd.11 [v2:172.21.15.148:6824/4068024116,v1:172.21.15.148:6825/4068024116]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-04-24T23:26:24.642 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:24 smithi148 ceph-mon[32528]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-04-24T23:26:24.642 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:24 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:26:24.642 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:24 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:26:24.788 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:24 smithi080 ceph-mon[29152]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-24T23:26:24.788 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:24 smithi080 ceph-mon[29152]: osdmap e78: 12 total, 11 up, 12 in 2024-04-24T23:26:24.788 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:24 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T23:26:24.789 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:24 smithi080 ceph-mon[29152]: from='osd.11 [v2:172.21.15.148:6824/4068024116,v1:172.21.15.148:6825/4068024116]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-04-24T23:26:24.789 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:24 smithi080 ceph-mon[29152]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-04-24T23:26:24.789 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:24 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:26:24.789 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:24 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:26:24.947 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:24 smithi134 ceph-mon[32619]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-24T23:26:24.947 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:24 smithi134 ceph-mon[32619]: osdmap e78: 12 total, 11 up, 12 in 2024-04-24T23:26:24.947 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:24 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T23:26:24.947 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:24 smithi134 ceph-mon[32619]: from='osd.11 [v2:172.21.15.148:6824/4068024116,v1:172.21.15.148:6825/4068024116]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-04-24T23:26:24.947 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:24 smithi134 ceph-mon[32619]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-04-24T23:26:24.948 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:24 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:26:24.948 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:24 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:26:25.256 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:26:24 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:26:24.935+0000 7f3d957a0700 -1 osd.11 0 waiting for initial osdmap 2024-04-24T23:26:25.257 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:26:24 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:26:24.951+0000 7f3d8d58b700 -1 osd.11 79 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T23:26:25.541 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:25.534+0000 7fdb9c113700 1 -- 172.21.15.80:0/1213316900 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdb94101190 msgr2=0x7fdb941015b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:25.541 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:25.534+0000 7fdb9c113700 1 --2- 172.21.15.80:0/1213316900 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdb94101190 0x7fdb941015b0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c00b0d0 tx=0x7fdb8c00b3e0 comp rx=0 tx=0).stop 2024-04-24T23:26:25.542 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:25.534+0000 7fdb9c113700 1 -- 172.21.15.80:0/1213316900 shutdown_connections 2024-04-24T23:26:25.542 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:25.534+0000 7fdb9c113700 1 --2- 172.21.15.80:0/1213316900 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdb94102d30 0x7fdb941095e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:25.542 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:25.534+0000 7fdb9c113700 1 --2- 172.21.15.80:0/1213316900 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdb94101190 0x7fdb941015b0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:25.542 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:25.534+0000 7fdb9c113700 1 --2- 172.21.15.80:0/1213316900 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdb94102390 0x7fdb941027f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:25.542 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:25.534+0000 7fdb9c113700 1 -- 172.21.15.80:0/1213316900 >> 172.21.15.80:0/1213316900 conn(0x7fdb940fc730 msgr2=0x7fdb940feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:25.542 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:25.534+0000 7fdb9c113700 1 -- 172.21.15.80:0/1213316900 shutdown_connections 2024-04-24T23:26:25.542 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:25.534+0000 7fdb9c113700 1 -- 172.21.15.80:0/1213316900 wait complete. 2024-04-24T23:26:25.542 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:25.535+0000 7fdb9c113700 1 Processor -- start 2024-04-24T23:26:25.542 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:25.535+0000 7fdb9c113700 1 -- start start 2024-04-24T23:26:25.542 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:25.535+0000 7fdb9c113700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdb94101190 0x7fdb9406aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:25.542 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:25.535+0000 7fdb9c113700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdb94102390 0x7fdb9419d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:25.542 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:25.535+0000 7fdb9c113700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdb94102d30 0x7fdb9419f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:25.542 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:25.535+0000 7fdb9c113700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fdb9410ce50 con 0x7fdb94102390 2024-04-24T23:26:25.543 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:25.535+0000 7fdb9c113700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fdb9410ccd0 con 0x7fdb94101190 2024-04-24T23:26:25.543 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:25.535+0000 7fdb9c113700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fdb9410cfd0 con 0x7fdb94102d30 2024-04-24T23:26:25.543 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:25.535+0000 7fdb996ae700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdb94102390 0x7fdb9419d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:25.543 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:25.535+0000 7fdb996ae700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdb94102390 0x7fdb9419d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:32770/0 (socket says 172.21.15.80:32770) 2024-04-24T23:26:25.543 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:25.535+0000 7fdb996ae700 1 -- 172.21.15.80:0/3469360729 learned_addr learned my addr 172.21.15.80:0/3469360729 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:26:25.543 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:25.535+0000 7fdb99eaf700 1 --2- 172.21.15.80:0/3469360729 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdb94101190 0x7fdb9406aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:25.543 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:25.535+0000 7fdb9a6b0700 1 --2- 172.21.15.80:0/3469360729 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdb94102d30 0x7fdb9419f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:25.543 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:25.535+0000 7fdb996ae700 1 -- 172.21.15.80:0/3469360729 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdb94102d30 msgr2=0x7fdb9419f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:25.543 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:25.535+0000 7fdb996ae700 1 --2- 172.21.15.80:0/3469360729 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdb94102d30 0x7fdb9419f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:25.543 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:25.535+0000 7fdb996ae700 1 -- 172.21.15.80:0/3469360729 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdb94101190 msgr2=0x7fdb9406aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:25.543 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:25.535+0000 7fdb996ae700 1 --2- 172.21.15.80:0/3469360729 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdb94101190 0x7fdb9406aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:25.543 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:25.535+0000 7fdb996ae700 1 -- 172.21.15.80:0/3469360729 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb941a02b0 con 0x7fdb94102390 2024-04-24T23:26:25.543 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:25.535+0000 7fdb996ae700 1 --2- 172.21.15.80:0/3469360729 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdb94102390 0x7fdb9419d350 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fdb8400b530 tx=0x7fdb8400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:25.543 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:25.536+0000 7fdb8affd700 1 -- 172.21.15.80:0/3469360729 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb84015020 con 0x7fdb94102390 2024-04-24T23:26:25.544 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:25.536+0000 7fdb8affd700 1 -- 172.21.15.80:0/3469360729 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb84011d10 con 0x7fdb94102390 2024-04-24T23:26:25.544 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:25.536+0000 7fdb9c113700 1 -- 172.21.15.80:0/3469360729 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb941a05a0 con 0x7fdb94102390 2024-04-24T23:26:25.544 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:25.536+0000 7fdb8affd700 1 -- 172.21.15.80:0/3469360729 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb8400f2e0 con 0x7fdb94102390 2024-04-24T23:26:25.544 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:25.536+0000 7fdb9c113700 1 -- 172.21.15.80:0/3469360729 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb941a0b60 con 0x7fdb94102390 2024-04-24T23:26:25.544 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:25.537+0000 7fdb9c113700 1 -- 172.21.15.80:0/3469360729 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb94068f20 con 0x7fdb94102390 2024-04-24T23:26:25.544 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:25.538+0000 7fdb8affd700 1 -- 172.21.15.80:0/3469360729 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fdb84018020 con 0x7fdb94102390 2024-04-24T23:26:25.544 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:25.538+0000 7fdb8affd700 1 --2- 172.21.15.80:0/3469360729 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fdb800a2c20 0x7fdb800a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:25.544 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:25.538+0000 7fdb99eaf700 1 --2- 172.21.15.80:0/3469360729 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fdb800a2c20 0x7fdb800a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:25.544 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:25.538+0000 7fdb8affd700 1 -- 172.21.15.80:0/3469360729 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6847+0+0 (secure 0 0 0) 0x7fdb840ce590 con 0x7fdb94102390 2024-04-24T23:26:25.544 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:25.539+0000 7fdb99eaf700 1 --2- 172.21.15.80:0/3469360729 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fdb800a2c20 0x7fdb800a50e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c009cd0 tx=0x7fdb8c00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:25.545 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:25.542+0000 7fdb8affd700 1 -- 172.21.15.80:0/3469360729 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fdb8401f020 con 0x7fdb94102390 2024-04-24T23:26:25.736 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:25.736+0000 7fdb9c113700 1 -- 172.21.15.80:0/3469360729 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fdb9402db60 con 0x7fdb94102390 2024-04-24T23:26:25.736 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:25 smithi080 ceph-mon[29152]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2024-04-24T23:26:25.736 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:25 smithi080 ceph-mon[29152]: osdmap e79: 12 total, 11 up, 12 in 2024-04-24T23:26:25.737 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:25 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T23:26:25.737 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:25 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T23:26:25.737 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:25 smithi080 ceph-mon[29152]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:26:25.737 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:25 smithi080 ceph-mon[29152]: from='osd.11 ' entity='osd.11' 2024-04-24T23:26:25.737 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:25.736+0000 7fdb8affd700 1 -- 172.21.15.80:0/3469360729 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v79) ==== 74+0+133 (secure 0 0 0) 0x7fdb8409bab0 con 0x7fdb94102390 2024-04-24T23:26:25.738 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-24T23:26:25.738 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:25.738+0000 7fdb9c113700 1 -- 172.21.15.80:0/3469360729 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fdb800a2c20 msgr2=0x7fdb800a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:25.738 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:25.738+0000 7fdb9c113700 1 --2- 172.21.15.80:0/3469360729 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fdb800a2c20 0x7fdb800a50e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c009cd0 tx=0x7fdb8c00b040 comp rx=0 tx=0).stop 2024-04-24T23:26:25.739 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:25.738+0000 7fdb9c113700 1 -- 172.21.15.80:0/3469360729 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdb94102390 msgr2=0x7fdb9419d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:25.739 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:25.738+0000 7fdb9c113700 1 --2- 172.21.15.80:0/3469360729 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdb94102390 0x7fdb9419d350 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fdb8400b530 tx=0x7fdb8400b840 comp rx=0 tx=0).stop 2024-04-24T23:26:25.739 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:25.738+0000 7fdb9c113700 1 -- 172.21.15.80:0/3469360729 shutdown_connections 2024-04-24T23:26:25.739 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:25.738+0000 7fdb9c113700 1 --2- 172.21.15.80:0/3469360729 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdb94102d30 0x7fdb9419f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:25.739 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:25.738+0000 7fdb9c113700 1 --2- 172.21.15.80:0/3469360729 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdb94101190 0x7fdb9406aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:25.739 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:25.738+0000 7fdb9c113700 1 --2- 172.21.15.80:0/3469360729 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fdb800a2c20 0x7fdb800a50e0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:25.739 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:25.738+0000 7fdb9c113700 1 --2- 172.21.15.80:0/3469360729 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdb94102390 0x7fdb9419d350 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:25.739 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:25.738+0000 7fdb9c113700 1 -- 172.21.15.80:0/3469360729 >> 172.21.15.80:0/3469360729 conn(0x7fdb940fc730 msgr2=0x7fdb94105700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:25.739 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:25.738+0000 7fdb9c113700 1 -- 172.21.15.80:0/3469360729 shutdown_connections 2024-04-24T23:26:25.739 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:25.738+0000 7fdb9c113700 1 -- 172.21.15.80:0/3469360729 wait complete. 2024-04-24T23:26:25.746 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:25 smithi148 ceph-mon[32528]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2024-04-24T23:26:25.746 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:25 smithi148 ceph-mon[32528]: osdmap e79: 12 total, 11 up, 12 in 2024-04-24T23:26:25.746 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:25 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T23:26:25.746 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:25 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T23:26:25.746 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:25 smithi148 ceph-mon[32528]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:26:25.747 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:25 smithi148 ceph-mon[32528]: from='osd.11 ' entity='osd.11' 2024-04-24T23:26:25.947 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:25 smithi134 ceph-mon[32619]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2024-04-24T23:26:25.947 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:25 smithi134 ceph-mon[32619]: osdmap e79: 12 total, 11 up, 12 in 2024-04-24T23:26:25.947 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:25 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T23:26:25.947 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:25 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T23:26:25.947 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:25 smithi134 ceph-mon[32619]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:26:25.948 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:25 smithi134 ceph-mon[32619]: from='osd.11 ' entity='osd.11' 2024-04-24T23:26:26.462 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":79,"num_osds":12,"num_up_osds":11,"osd_up_since":1714001158,"num_in_osds":12,"osd_in_since":1714001164,"num_remapped_pgs":0} 2024-04-24T23:26:26.734 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:26 smithi080 ceph-mon[29152]: purged_snaps scrub starts 2024-04-24T23:26:26.735 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:26 smithi080 ceph-mon[29152]: purged_snaps scrub ok 2024-04-24T23:26:26.735 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:26 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T23:26:26.735 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:26 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/3469360729' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-24T23:26:26.735 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:26 smithi080 ceph-mon[29152]: osd.11 [v2:172.21.15.148:6824/4068024116,v1:172.21.15.148:6825/4068024116] boot 2024-04-24T23:26:26.735 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:26 smithi080 ceph-mon[29152]: osdmap e80: 12 total, 12 up, 12 in 2024-04-24T23:26:26.735 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:26 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T23:26:26.871 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:26 smithi148 ceph-mon[32528]: purged_snaps scrub starts 2024-04-24T23:26:26.871 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:26 smithi148 ceph-mon[32528]: purged_snaps scrub ok 2024-04-24T23:26:26.871 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:26 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T23:26:26.871 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:26 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/3469360729' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-24T23:26:26.871 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:26 smithi148 ceph-mon[32528]: osd.11 [v2:172.21.15.148:6824/4068024116,v1:172.21.15.148:6825/4068024116] boot 2024-04-24T23:26:26.871 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:26 smithi148 ceph-mon[32528]: osdmap e80: 12 total, 12 up, 12 in 2024-04-24T23:26:26.871 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:26 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T23:26:26.947 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:26 smithi134 ceph-mon[32619]: purged_snaps scrub starts 2024-04-24T23:26:26.947 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:26 smithi134 ceph-mon[32619]: purged_snaps scrub ok 2024-04-24T23:26:26.947 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:26 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T23:26:26.947 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:26 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/3469360729' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-24T23:26:26.947 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:26 smithi134 ceph-mon[32619]: osd.11 [v2:172.21.15.148:6824/4068024116,v1:172.21.15.148:6825/4068024116] boot 2024-04-24T23:26:26.947 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:26 smithi134 ceph-mon[32619]: osdmap e80: 12 total, 12 up, 12 in 2024-04-24T23:26:26.948 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:26 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T23:26:27.464 DEBUG:teuthology.orchestra.run.smithi080:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ec7885aa-0290-11ef-bc93-c7b262605968 -- ceph osd stat -f json 2024-04-24T23:26:27.707 INFO:teuthology.orchestra.run.smithi080.stderr:Inferring config /var/lib/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:26:27.788 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:27 smithi080 ceph-mon[29152]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:26:27.789 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:27 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:26:27.789 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:27 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:26:27.789 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:27 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:26:27.789 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:27 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:26:27.789 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:27 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:26:27.789 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:27 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:26:27.789 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:27 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:26:27.789 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:27 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:26:27.789 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:27 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:26:27.789 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:27 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:26:27.947 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:27 smithi134 ceph-mon[32619]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:26:27.947 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:27 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:26:27.947 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:27 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:26:27.947 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:27 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:26:27.947 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:27 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:26:27.947 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:27 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:26:27.947 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:27 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:26:27.948 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:27 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:26:27.948 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:27 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:26:27.948 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:27 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:26:27.948 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:27 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:26:28.006 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:27 smithi148 ceph-mon[32528]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:26:28.006 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:27 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:26:28.006 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:27 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:26:28.006 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:27 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:26:28.006 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:27 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:26:28.007 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:27 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:26:28.007 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:27 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:26:28.007 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:27 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:26:28.007 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:27 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:26:28.007 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:27 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:26:28.007 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:27 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:26:28.788 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:28 smithi080 ceph-mon[29152]: Detected new or changed devices on smithi148 2024-04-24T23:26:28.789 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:28 smithi080 ceph-mon[29152]: Adjusting osd_memory_target on smithi148 to 4272M 2024-04-24T23:26:28.789 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:28 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:26:28.789 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:28 smithi080 ceph-mon[29152]: osdmap e81: 12 total, 12 up, 12 in 2024-04-24T23:26:28.789 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:28 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:26:28.789 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:28 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:26:28.789 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:28 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:26:28.947 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:28 smithi134 ceph-mon[32619]: Detected new or changed devices on smithi148 2024-04-24T23:26:28.947 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:28 smithi134 ceph-mon[32619]: Adjusting osd_memory_target on smithi148 to 4272M 2024-04-24T23:26:28.947 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:28 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:26:28.947 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:28 smithi134 ceph-mon[32619]: osdmap e81: 12 total, 12 up, 12 in 2024-04-24T23:26:28.947 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:28 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:26:28.948 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:28 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:26:28.948 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:28 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:26:29.006 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:28 smithi148 ceph-mon[32528]: Detected new or changed devices on smithi148 2024-04-24T23:26:29.006 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:28 smithi148 ceph-mon[32528]: Adjusting osd_memory_target on smithi148 to 4272M 2024-04-24T23:26:29.006 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:28 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:26:29.007 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:28 smithi148 ceph-mon[32528]: osdmap e81: 12 total, 12 up, 12 in 2024-04-24T23:26:29.007 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:28 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:26:29.007 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:28 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:26:29.007 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:28 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:26:29.406 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:29.404+0000 7f485b727700 1 -- 172.21.15.80:0/760441766 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4854102d20 msgr2=0x7f48541095d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:29.406 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:29.404+0000 7f485b727700 1 --2- 172.21.15.80:0/760441766 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4854102d20 0x7f48541095d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f484c00b0d0 tx=0x7f484c00b3e0 comp rx=0 tx=0).stop 2024-04-24T23:26:29.406 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:29.405+0000 7f485b727700 1 -- 172.21.15.80:0/760441766 shutdown_connections 2024-04-24T23:26:29.406 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:29.405+0000 7f485b727700 1 --2- 172.21.15.80:0/760441766 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4854102d20 0x7f48541095d0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:29.406 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:29.405+0000 7f485b727700 1 --2- 172.21.15.80:0/760441766 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4854101180 0x7f48541015a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:29.407 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:29.405+0000 7f485b727700 1 --2- 172.21.15.80:0/760441766 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4854102380 0x7f48541027e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:29.407 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:29.405+0000 7f485b727700 1 -- 172.21.15.80:0/760441766 >> 172.21.15.80:0/760441766 conn(0x7f48540fc720 msgr2=0x7f48540feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:29.407 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:29.405+0000 7f485b727700 1 -- 172.21.15.80:0/760441766 shutdown_connections 2024-04-24T23:26:29.407 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:29.405+0000 7f485b727700 1 -- 172.21.15.80:0/760441766 wait complete. 2024-04-24T23:26:29.408 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:29.405+0000 7f485b727700 1 Processor -- start 2024-04-24T23:26:29.409 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:29.405+0000 7f485b727700 1 -- start start 2024-04-24T23:26:29.409 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:29.406+0000 7f485b727700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4854101180 0x7f485406b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:29.409 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:29.406+0000 7f485b727700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4854102380 0x7f485419d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:29.409 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:29.406+0000 7f485b727700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4854102d20 0x7f485419f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:29.409 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:29.406+0000 7f485b727700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f485410cd50 con 0x7f4854102380 2024-04-24T23:26:29.409 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:29.406+0000 7f485b727700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f485410cbd0 con 0x7f4854102d20 2024-04-24T23:26:29.409 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:29.406+0000 7f485b727700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f485410ced0 con 0x7f4854101180 2024-04-24T23:26:29.409 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:29.406+0000 7f4858cc2700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4854102380 0x7f485419d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:29.409 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:29.406+0000 7f4858cc2700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4854102380 0x7f485419d0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:32782/0 (socket says 172.21.15.80:32782) 2024-04-24T23:26:29.409 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:29.406+0000 7f4858cc2700 1 -- 172.21.15.80:0/2834147576 learned_addr learned my addr 172.21.15.80:0/2834147576 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:26:29.409 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:29.406+0000 7f48594c3700 1 --2- 172.21.15.80:0/2834147576 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4854101180 0x7f485406b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:29.409 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:29.406+0000 7f4859cc4700 1 --2- 172.21.15.80:0/2834147576 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4854102d20 0x7f485419f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:29.409 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:29.406+0000 7f4858cc2700 1 -- 172.21.15.80:0/2834147576 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4854101180 msgr2=0x7f485406b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:29.410 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:29.406+0000 7f4858cc2700 1 --2- 172.21.15.80:0/2834147576 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4854101180 0x7f485406b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:29.410 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:29.406+0000 7f4858cc2700 1 -- 172.21.15.80:0/2834147576 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4854102d20 msgr2=0x7f485419f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:29.410 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:29.406+0000 7f4858cc2700 1 --2- 172.21.15.80:0/2834147576 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4854102d20 0x7f485419f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:29.410 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:29.406+0000 7f4858cc2700 1 -- 172.21.15.80:0/2834147576 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4854196f20 con 0x7f4854102380 2024-04-24T23:26:29.410 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:29.406+0000 7f4858cc2700 1 --2- 172.21.15.80:0/2834147576 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4854102380 0x7f485419d0d0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f484400d950 tx=0x7f484400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:29.410 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:29.406+0000 7f484a7fc700 1 -- 172.21.15.80:0/2834147576 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4844016070 con 0x7f4854102380 2024-04-24T23:26:29.410 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:29.406+0000 7f484a7fc700 1 -- 172.21.15.80:0/2834147576 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4844009e20 con 0x7f4854102380 2024-04-24T23:26:29.410 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:29.406+0000 7f485b727700 1 -- 172.21.15.80:0/2834147576 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4854197210 con 0x7f4854102380 2024-04-24T23:26:29.410 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:29.406+0000 7f484a7fc700 1 -- 172.21.15.80:0/2834147576 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f484400f620 con 0x7f4854102380 2024-04-24T23:26:29.410 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:29.406+0000 7f485b727700 1 -- 172.21.15.80:0/2834147576 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4854197750 con 0x7f4854102380 2024-04-24T23:26:29.410 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:29.407+0000 7f485b727700 1 -- 172.21.15.80:0/2834147576 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4854068f20 con 0x7f4854102380 2024-04-24T23:26:29.411 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:29.409+0000 7f484a7fc700 1 -- 172.21.15.80:0/2834147576 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f4844019070 con 0x7f4854102380 2024-04-24T23:26:29.411 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:29.409+0000 7f484a7fc700 1 --2- 172.21.15.80:0/2834147576 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f48400a2c20 0x7f48400a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:29.411 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:29.409+0000 7f48594c3700 1 --2- 172.21.15.80:0/2834147576 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f48400a2c20 0x7f48400a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:29.411 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:29.410+0000 7f484a7fc700 1 -- 172.21.15.80:0/2834147576 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f48440ce870 con 0x7f4854102380 2024-04-24T23:26:29.412 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:29.410+0000 7f48594c3700 1 --2- 172.21.15.80:0/2834147576 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f48400a2c20 0x7f48400a50e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f48541021e0 tx=0x7f485000a3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:29.414 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:29.413+0000 7f484a7fc700 1 -- 172.21.15.80:0/2834147576 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f484409bbd0 con 0x7f4854102380 2024-04-24T23:26:29.616 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:29.614+0000 7f485b727700 1 -- 172.21.15.80:0/2834147576 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f4854198dc0 con 0x7f4854102380 2024-04-24T23:26:29.616 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:29.615+0000 7f484a7fc700 1 -- 172.21.15.80:0/2834147576 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v81) ==== 74+0+133 (secure 0 0 0) 0x7f484409b4d0 con 0x7f4854102380 2024-04-24T23:26:29.616 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-24T23:26:29.618 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:29.617+0000 7f485b727700 1 -- 172.21.15.80:0/2834147576 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f48400a2c20 msgr2=0x7f48400a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:29.618 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:29.617+0000 7f485b727700 1 --2- 172.21.15.80:0/2834147576 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f48400a2c20 0x7f48400a50e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f48541021e0 tx=0x7f485000a3b0 comp rx=0 tx=0).stop 2024-04-24T23:26:29.618 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:29.617+0000 7f485b727700 1 -- 172.21.15.80:0/2834147576 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4854102380 msgr2=0x7f485419d0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:29.618 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:29.617+0000 7f485b727700 1 --2- 172.21.15.80:0/2834147576 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4854102380 0x7f485419d0d0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f484400d950 tx=0x7f484400dc60 comp rx=0 tx=0).stop 2024-04-24T23:26:29.619 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:29.617+0000 7f485b727700 1 -- 172.21.15.80:0/2834147576 shutdown_connections 2024-04-24T23:26:29.619 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:29.617+0000 7f485b727700 1 --2- 172.21.15.80:0/2834147576 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4854102d20 0x7f485419f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:29.619 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:29.617+0000 7f485b727700 1 --2- 172.21.15.80:0/2834147576 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4854101180 0x7f485406b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:29.619 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:29.617+0000 7f485b727700 1 --2- 172.21.15.80:0/2834147576 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f48400a2c20 0x7f48400a50e0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:29.619 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:29.617+0000 7f485b727700 1 --2- 172.21.15.80:0/2834147576 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4854102380 0x7f485419d0d0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:29.619 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:29.617+0000 7f485b727700 1 -- 172.21.15.80:0/2834147576 >> 172.21.15.80:0/2834147576 conn(0x7f48540fc720 msgr2=0x7f48541055b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:29.619 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:29.617+0000 7f485b727700 1 -- 172.21.15.80:0/2834147576 shutdown_connections 2024-04-24T23:26:29.619 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:29.617+0000 7f485b727700 1 -- 172.21.15.80:0/2834147576 wait complete. 2024-04-24T23:26:29.788 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:29 smithi080 ceph-mon[29152]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:26:29.947 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:29 smithi134 ceph-mon[32619]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:26:30.006 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:29 smithi148 ceph-mon[32528]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:26:30.438 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":81,"num_osds":12,"num_up_osds":12,"osd_up_since":1714001185,"num_in_osds":12,"osd_in_since":1714001164,"num_remapped_pgs":0} 2024-04-24T23:26:30.439 DEBUG:teuthology.orchestra.run.smithi080:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid ec7885aa-0290-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-24T23:26:30.678 INFO:teuthology.orchestra.run.smithi080.stderr:Inferring config /var/lib/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:26:30.788 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:30 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/2834147576' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-24T23:26:30.947 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:30 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/2834147576' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-24T23:26:31.006 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:30 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/2834147576' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-24T23:26:31.788 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:31 smithi080 ceph-mon[29152]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:26:31.947 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:31 smithi134 ceph-mon[32619]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:26:32.006 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:31 smithi148 ceph-mon[32528]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:26:32.467 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:32.465+0000 7f2a1b73f700 1 -- 172.21.15.80:0/934714600 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2a1406a8e0 msgr2=0x7f2a1410f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:32.467 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:32.465+0000 7f2a1b73f700 1 --2- 172.21.15.80:0/934714600 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2a1406a8e0 0x7f2a1410f0f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c00b0a0 tx=0x7f2a0c00b3b0 comp rx=0 tx=0).stop 2024-04-24T23:26:32.467 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:32.465+0000 7f2a1b73f700 1 -- 172.21.15.80:0/934714600 shutdown_connections 2024-04-24T23:26:32.467 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:32.465+0000 7f2a1b73f700 1 --2- 172.21.15.80:0/934714600 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2a1406a8e0 0x7f2a1410f0f0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:32.467 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:32.465+0000 7f2a1b73f700 1 --2- 172.21.15.80:0/934714600 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2a141082b0 0x7f2a14108690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:32.467 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:32.465+0000 7f2a1b73f700 1 --2- 172.21.15.80:0/934714600 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2a14069f40 0x7f2a1406a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:32.467 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:32.465+0000 7f2a1b73f700 1 -- 172.21.15.80:0/934714600 >> 172.21.15.80:0/934714600 conn(0x7f2a140780e0 msgr2=0x7f2a140784f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:32.467 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:32.465+0000 7f2a1b73f700 1 -- 172.21.15.80:0/934714600 shutdown_connections 2024-04-24T23:26:32.467 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:32.465+0000 7f2a1b73f700 1 -- 172.21.15.80:0/934714600 wait complete. 2024-04-24T23:26:32.467 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:32.466+0000 7f2a1b73f700 1 Processor -- start 2024-04-24T23:26:32.467 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:32.466+0000 7f2a1b73f700 1 -- start start 2024-04-24T23:26:32.470 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:32.466+0000 7f2a1b73f700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2a14069f40 0x7f2a1419e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:32.470 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:32.466+0000 7f2a1b73f700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2a1406a8e0 0x7f2a1419ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:32.470 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:32.466+0000 7f2a1b73f700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2a141082b0 0x7f2a14198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:32.470 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:32.466+0000 7f2a1b73f700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f2a14112530 con 0x7f2a141082b0 2024-04-24T23:26:32.470 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:32.466+0000 7f2a1b73f700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2a141123b0 con 0x7f2a14069f40 2024-04-24T23:26:32.470 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:32.466+0000 7f2a1b73f700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f2a141126b0 con 0x7f2a1406a8e0 2024-04-24T23:26:32.471 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:32.466+0000 7f2a19cdc700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2a141082b0 0x7f2a14198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:32.471 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:32.466+0000 7f2a19cdc700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2a141082b0 0x7f2a14198770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:54704/0 (socket says 172.21.15.80:54704) 2024-04-24T23:26:32.471 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:32.466+0000 7f2a19cdc700 1 -- 172.21.15.80:0/2585923263 learned_addr learned my addr 172.21.15.80:0/2585923263 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:26:32.471 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:32.466+0000 7f2a18cda700 1 --2- 172.21.15.80:0/2585923263 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2a1406a8e0 0x7f2a1419ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:32.471 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:32.466+0000 7f2a194db700 1 --2- 172.21.15.80:0/2585923263 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2a14069f40 0x7f2a1419e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:32.471 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:32.467+0000 7f2a19cdc700 1 -- 172.21.15.80:0/2585923263 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2a1406a8e0 msgr2=0x7f2a1419ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:32.471 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:32.467+0000 7f2a19cdc700 1 --2- 172.21.15.80:0/2585923263 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2a1406a8e0 0x7f2a1419ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:32.471 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:32.467+0000 7f2a19cdc700 1 -- 172.21.15.80:0/2585923263 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2a14069f40 msgr2=0x7f2a1419e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:32.471 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:32.467+0000 7f2a19cdc700 1 --2- 172.21.15.80:0/2585923263 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2a14069f40 0x7f2a1419e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:32.471 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:32.467+0000 7f2a19cdc700 1 -- 172.21.15.80:0/2585923263 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a14199050 con 0x7f2a141082b0 2024-04-24T23:26:32.471 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:32.467+0000 7f2a194db700 1 --2- 172.21.15.80:0/2585923263 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2a14069f40 0x7f2a1419e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:26:32.471 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:32.467+0000 7f2a19cdc700 1 --2- 172.21.15.80:0/2585923263 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2a141082b0 0x7f2a14198770 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c0062a0 tx=0x7f2a0c004d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:32.472 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:32.467+0000 7f2a0a7fc700 1 -- 172.21.15.80:0/2585923263 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a0c021070 con 0x7f2a141082b0 2024-04-24T23:26:32.472 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:32.467+0000 7f2a0a7fc700 1 -- 172.21.15.80:0/2585923263 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a0c007c00 con 0x7f2a141082b0 2024-04-24T23:26:32.472 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:32.467+0000 7f2a1b73f700 1 -- 172.21.15.80:0/2585923263 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a141992e0 con 0x7f2a141082b0 2024-04-24T23:26:32.472 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:32.467+0000 7f2a0a7fc700 1 -- 172.21.15.80:0/2585923263 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a0c01d450 con 0x7f2a141082b0 2024-04-24T23:26:32.472 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:32.467+0000 7f2a1b73f700 1 -- 172.21.15.80:0/2585923263 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a141a5620 con 0x7f2a141082b0 2024-04-24T23:26:32.472 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:32.468+0000 7f2a1b73f700 1 -- 172.21.15.80:0/2585923263 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a14103960 con 0x7f2a141082b0 2024-04-24T23:26:32.472 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:32.469+0000 7f2a0a7fc700 1 -- 172.21.15.80:0/2585923263 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f2a0c004000 con 0x7f2a141082b0 2024-04-24T23:26:32.473 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:32.470+0000 7f2a0a7fc700 1 --2- 172.21.15.80:0/2585923263 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f2a000a2ca0 0x7f2a000a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:32.473 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:32.470+0000 7f2a194db700 1 --2- 172.21.15.80:0/2585923263 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f2a000a2ca0 0x7f2a000a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:32.473 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:32.470+0000 7f2a0a7fc700 1 -- 172.21.15.80:0/2585923263 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f2a0c00f020 con 0x7f2a141082b0 2024-04-24T23:26:32.473 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:32.470+0000 7f2a194db700 1 --2- 172.21.15.80:0/2585923263 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f2a000a2ca0 0x7f2a000a5160 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2a04005fd0 tx=0x7f2a04005db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:32.474 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:32.473+0000 7f2a0a7fc700 1 -- 172.21.15.80:0/2585923263 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f2a0c02a060 con 0x7f2a141082b0 2024-04-24T23:26:32.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:32.671+0000 7f2a1b73f700 1 -- 172.21.15.80:0/2585923263 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2a14199ca0 con 0x7f2a141082b0 2024-04-24T23:26:32.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:32.673+0000 7f2a0a7fc700 1 -- 172.21.15.80:0/2585923263 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+18452 (secure 0 0 0) 0x7f2a0c09d6e0 con 0x7f2a141082b0 2024-04-24T23:26:32.674 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-24T23:26:32.674 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":81,"fsid":"ec7885aa-0290-11ef-bc93-c7b262605968","created":"2024-04-24T23:19:05.586407+0000","modified":"2024-04-24T23:26:27.519820+0000","last_up_change":"2024-04-24T23:26:25.938316+0000","last_in_change":"2024-04-24T23:26:04.786699+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:22:33.124428+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a525741d-b46c-4eef-aa64-3f057cac1497","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":2597718112},{"type":"v1","addr":"172.21.15.80:6803","nonce":2597718112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":2597718112},{"type":"v1","addr":"172.21.15.80:6805","nonce":2597718112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2597718112},{"type":"v1","addr":"172.21.15.80:6809","nonce":2597718112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":2597718112},{"type":"v1","addr":"172.21.15.80:6807","nonce":2597718112}]},"public_addr":"172.21.15.80:6803/2597718112","cluster_addr":"172.21.15.80:6805/2597718112","heartbeat_back_addr":"172.21.15.80:6809/2597718112","heartbeat_front_addr":"172.21.15.80:6807/2597718112","state":["exists","up"]},{"osd":1,"uuid":"2d0f0d67-6db4-40ac-91cc-58604ab6795a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":2395610201},{"type":"v1","addr":"172.21.15.134:6801","nonce":2395610201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":2395610201},{"type":"v1","addr":"172.21.15.134:6803","nonce":2395610201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":2395610201},{"type":"v1","addr":"172.21.15.134:6807","nonce":2395610201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":2395610201},{"type":"v1","addr":"172.21.15.134:6805","nonce":2395610201}]},"public_addr":"172.21.15.134:6801/2395610201","cluster_addr":"172.21.15.134:6803/2395610201","heartbeat_back_addr":"172.21.15.134:6807/2395610201","heartbeat_front_addr":"172.21.15.134:6805/2395610201","state":["exists","up"]},{"osd":2,"uuid":"781fb879-fedc-4c84-82fb-59e073bc4cf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1629152187},{"type":"v1","addr":"172.21.15.148:6801","nonce":1629152187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1629152187},{"type":"v1","addr":"172.21.15.148:6803","nonce":1629152187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1629152187},{"type":"v1","addr":"172.21.15.148:6807","nonce":1629152187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1629152187},{"type":"v1","addr":"172.21.15.148:6805","nonce":1629152187}]},"public_addr":"172.21.15.148:6801/1629152187","cluster_addr":"172.21.15.148:6803/1629152187","heartbeat_back_addr":"172.21.15.148:6807/1629152187","heartbeat_front_addr":"172.21.15.148:6805/1629152187","state":["exists","up"]},{"osd":3,"uuid":"cd982757-f2f6-4296-b3e7-4cd40fbd8d9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":3128298510},{"type":"v1","addr":"172.21.15.80:6811","nonce":3128298510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":3128298510},{"type":"v1","addr":"172.21.15.80:6813","nonce":3128298510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":3128298510},{"type":"v1","addr":"172.21.15.80:6817","nonce":3128298510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":3128298510},{"type":"v1","addr":"172.21.15.80:6815","nonce":3128298510}]},"public_addr":"172.21.15.80:6811/3128298510","cluster_addr":"172.21.15.80:6813/3128298510","heartbeat_back_addr":"172.21.15.80:6817/3128298510","heartbeat_front_addr":"172.21.15.80:6815/3128298510","state":["exists","up"]},{"osd":4,"uuid":"dfc55a79-a7e8-417a-b898-ec44171a54af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1917556060},{"type":"v1","addr":"172.21.15.134:6809","nonce":1917556060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1917556060},{"type":"v1","addr":"172.21.15.134:6811","nonce":1917556060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1917556060},{"type":"v1","addr":"172.21.15.134:6815","nonce":1917556060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1917556060},{"type":"v1","addr":"172.21.15.134:6813","nonce":1917556060}]},"public_addr":"172.21.15.134:6809/1917556060","cluster_addr":"172.21.15.134:6811/1917556060","heartbeat_back_addr":"172.21.15.134:6815/1917556060","heartbeat_front_addr":"172.21.15.134:6813/1917556060","state":["exists","up"]},{"osd":5,"uuid":"b5491337-0694-440c-b679-312f3d14cf65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":2715951281},{"type":"v1","addr":"172.21.15.148:6809","nonce":2715951281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":2715951281},{"type":"v1","addr":"172.21.15.148:6811","nonce":2715951281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":2715951281},{"type":"v1","addr":"172.21.15.148:6815","nonce":2715951281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":2715951281},{"type":"v1","addr":"172.21.15.148:6813","nonce":2715951281}]},"public_addr":"172.21.15.148:6809/2715951281","cluster_addr":"172.21.15.148:6811/2715951281","heartbeat_back_addr":"172.21.15.148:6815/2715951281","heartbeat_front_addr":"172.21.15.148:6813/2715951281","state":["exists","up"]},{"osd":6,"uuid":"ba4b5df7-d86e-47cd-ab67-cf763c4e4d6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":953261724},{"type":"v1","addr":"172.21.15.80:6819","nonce":953261724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":953261724},{"type":"v1","addr":"172.21.15.80:6821","nonce":953261724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":953261724},{"type":"v1","addr":"172.21.15.80:6825","nonce":953261724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":953261724},{"type":"v1","addr":"172.21.15.80:6823","nonce":953261724}]},"public_addr":"172.21.15.80:6819/953261724","cluster_addr":"172.21.15.80:6821/953261724","heartbeat_back_addr":"172.21.15.80:6825/953261724","heartbeat_front_addr":"172.21.15.80:6823/953261724","state":["exists","up"]},{"osd":7,"uuid":"464a503a-75ae-49e7-ab6e-a7f9f2a6a307","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":108354844},{"type":"v1","addr":"172.21.15.134:6817","nonce":108354844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":108354844},{"type":"v1","addr":"172.21.15.134:6819","nonce":108354844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":108354844},{"type":"v1","addr":"172.21.15.134:6823","nonce":108354844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":108354844},{"type":"v1","addr":"172.21.15.134:6821","nonce":108354844}]},"public_addr":"172.21.15.134:6817/108354844","cluster_addr":"172.21.15.134:6819/108354844","heartbeat_back_addr":"172.21.15.134:6823/108354844","heartbeat_front_addr":"172.21.15.134:6821/108354844","state":["exists","up"]},{"osd":8,"uuid":"ce544b2e-4540-47c0-9997-500c45351330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3559655329},{"type":"v1","addr":"172.21.15.148:6817","nonce":3559655329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":3559655329},{"type":"v1","addr":"172.21.15.148:6819","nonce":3559655329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":3559655329},{"type":"v1","addr":"172.21.15.148:6823","nonce":3559655329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3559655329},{"type":"v1","addr":"172.21.15.148:6821","nonce":3559655329}]},"public_addr":"172.21.15.148:6817/3559655329","cluster_addr":"172.21.15.148:6819/3559655329","heartbeat_back_addr":"172.21.15.148:6823/3559655329","heartbeat_front_addr":"172.21.15.148:6821/3559655329","state":["exists","up"]},{"osd":9,"uuid":"06de3732-19b9-496c-a464-55322efa6759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":4019913545},{"type":"v1","addr":"172.21.15.80:6827","nonce":4019913545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":4019913545},{"type":"v1","addr":"172.21.15.80:6829","nonce":4019913545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":4019913545},{"type":"v1","addr":"172.21.15.80:6833","nonce":4019913545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":4019913545},{"type":"v1","addr":"172.21.15.80:6831","nonce":4019913545}]},"public_addr":"172.21.15.80:6827/4019913545","cluster_addr":"172.21.15.80:6829/4019913545","heartbeat_back_addr":"172.21.15.80:6833/4019913545","heartbeat_front_addr":"172.21.15.80:6831/4019913545","state":["exists","up"]},{"osd":10,"uuid":"8d92f0d9-5c64-429a-b750-c33ae0e070c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":647343766},{"type":"v1","addr":"172.21.15.134:6825","nonce":647343766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":647343766},{"type":"v1","addr":"172.21.15.134:6827","nonce":647343766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":647343766},{"type":"v1","addr":"172.21.15.134:6831","nonce":647343766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":647343766},{"type":"v1","addr":"172.21.15.134:6829","nonce":647343766}]},"public_addr":"172.21.15.134:6825/647343766","cluster_addr":"172.21.15.134:6827/647343766","heartbeat_back_addr":"172.21.15.134:6831/647343766","heartbeat_front_addr":"172.21.15.134:6829/647343766","state":["exists","up"]},{"osd":11,"uuid":"4c003a41-f65b-479f-bc80-b27d680cf936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":4068024116},{"type":"v1","addr":"172.21.15.148:6825","nonce":4068024116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":4068024116},{"type":"v1","addr":"172.21.15.148:6827","nonce":4068024116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":4068024116},{"type":"v1","addr":"172.21.15.148:6831","nonce":4068024116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":4068024116},{"type":"v1","addr":"172.21.15.148:6829","nonce":4068024116}]},"public_addr":"172.21.15.148:6825/4068024116","cluster_addr":"172.21.15.148:6827/4068024116","heartbeat_back_addr":"172.21.15.148:6831/4068024116","heartbeat_front_addr":"172.21.15.148:6829/4068024116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:21:42.433631+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:22:06.145933+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:22:30.726143+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:22:55.729961+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:23:21.280913+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:23:46.609983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:24:12.585247+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:24:38.359287+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:25:04.491232+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:25:30.788089+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:25:56.891078+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:26:23.899113+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.80:0/2828378749":"2024-04-25T23:20:06.793609+0000","172.21.15.80:6801/233364790":"2024-04-25T23:20:06.793609+0000","172.21.15.80:6800/1614691410":"2024-04-25T23:19:40.490692+0000","172.21.15.80:0/1406187145":"2024-04-25T23:20:06.793609+0000","172.21.15.80:0/92953396":"2024-04-25T23:19:40.490692+0000","172.21.15.80:0/598129432":"2024-04-25T23:20:06.793609+0000","172.21.15.80:0/232193118":"2024-04-25T23:19:40.490692+0000","172.21.15.80:6800/233364790":"2024-04-25T23:20:06.793609+0000","172.21.15.80:6801/1614691410":"2024-04-25T23:19:40.490692+0000","172.21.15.80:0/2081562529":"2024-04-25T23:19:40.490692+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:26:32.676 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:32.674+0000 7f2a1b73f700 1 -- 172.21.15.80:0/2585923263 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f2a000a2ca0 msgr2=0x7f2a000a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:32.676 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:32.674+0000 7f2a1b73f700 1 --2- 172.21.15.80:0/2585923263 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f2a000a2ca0 0x7f2a000a5160 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2a04005fd0 tx=0x7f2a04005db0 comp rx=0 tx=0).stop 2024-04-24T23:26:32.676 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:32.674+0000 7f2a1b73f700 1 -- 172.21.15.80:0/2585923263 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2a141082b0 msgr2=0x7f2a14198770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:32.676 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:32.675+0000 7f2a1b73f700 1 --2- 172.21.15.80:0/2585923263 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2a141082b0 0x7f2a14198770 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c0062a0 tx=0x7f2a0c004d70 comp rx=0 tx=0).stop 2024-04-24T23:26:32.676 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:32.675+0000 7f2a1b73f700 1 -- 172.21.15.80:0/2585923263 shutdown_connections 2024-04-24T23:26:32.676 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:32.675+0000 7f2a1b73f700 1 --2- 172.21.15.80:0/2585923263 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f2a000a2ca0 0x7f2a000a5160 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:32.676 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:32.675+0000 7f2a1b73f700 1 --2- 172.21.15.80:0/2585923263 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2a141082b0 0x7f2a14198770 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:32.676 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:32.675+0000 7f2a1b73f700 1 --2- 172.21.15.80:0/2585923263 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2a14069f40 0x7f2a1419e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:32.677 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:32.675+0000 7f2a1b73f700 1 --2- 172.21.15.80:0/2585923263 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2a1406a8e0 0x7f2a1419ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:32.677 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:32.675+0000 7f2a1b73f700 1 -- 172.21.15.80:0/2585923263 >> 172.21.15.80:0/2585923263 conn(0x7f2a140780e0 msgr2=0x7f2a140ffc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:32.677 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:32.675+0000 7f2a1b73f700 1 -- 172.21.15.80:0/2585923263 shutdown_connections 2024-04-24T23:26:32.677 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:32.675+0000 7f2a1b73f700 1 -- 172.21.15.80:0/2585923263 wait complete. 2024-04-24T23:26:33.006 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:32 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/2585923263' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:26:33.038 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:32 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/2585923263' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:26:33.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:32 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/2585923263' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:26:33.475 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:22:33.124428+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:26:33.476 DEBUG:teuthology.orchestra.run.smithi080:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid ec7885aa-0290-11ef-bc93-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-04-24T23:26:33.713 INFO:teuthology.orchestra.run.smithi080.stderr:Inferring config /var/lib/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:26:33.949 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:33 smithi134 ceph-mon[32619]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:26:33.991 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:33 smithi080 ceph-mon[29152]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:26:34.006 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:33 smithi148 ceph-mon[32528]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:26:35.504 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:35.502+0000 7f08d9076700 1 -- 172.21.15.80:0/2183262901 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f08d4102510 msgr2=0x7f08d41028f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:35.504 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:35.502+0000 7f08d9076700 1 --2- 172.21.15.80:0/2183262901 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f08d4102510 0x7f08d41028f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f08c4009990 tx=0x7f08c4009ca0 comp rx=0 tx=0).stop 2024-04-24T23:26:35.504 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:35.502+0000 7f08d9076700 1 -- 172.21.15.80:0/2183262901 shutdown_connections 2024-04-24T23:26:35.504 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:35.502+0000 7f08d9076700 1 --2- 172.21.15.80:0/2183262901 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f08d4107590 0x7f08d410f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:35.504 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:35.502+0000 7f08d9076700 1 --2- 172.21.15.80:0/2183262901 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f08d4102510 0x7f08d41028f0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:35.504 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:35.502+0000 7f08d9076700 1 --2- 172.21.15.80:0/2183262901 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f08d4102ec0 0x7f08d4106eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:35.504 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:35.502+0000 7f08d9076700 1 -- 172.21.15.80:0/2183262901 >> 172.21.15.80:0/2183262901 conn(0x7f08d40793a0 msgr2=0x7f08d40797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:35.505 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:35.502+0000 7f08d9076700 1 -- 172.21.15.80:0/2183262901 shutdown_connections 2024-04-24T23:26:35.505 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:35.502+0000 7f08d9076700 1 -- 172.21.15.80:0/2183262901 wait complete. 2024-04-24T23:26:35.506 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:35.503+0000 7f08d9076700 1 Processor -- start 2024-04-24T23:26:35.506 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:35.503+0000 7f08d9076700 1 -- start start 2024-04-24T23:26:35.507 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:35.503+0000 7f08d9076700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f08d4102510 0x7f08d419e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:35.507 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:35.503+0000 7f08d9076700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f08d4102ec0 0x7f08d419ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:35.507 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:35.503+0000 7f08d9076700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f08d4107590 0x7f08d4198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:35.507 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:35.503+0000 7f08d9076700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f08d4112650 con 0x7f08d4107590 2024-04-24T23:26:35.507 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:35.503+0000 7f08d9076700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f08d41124d0 con 0x7f08d4102510 2024-04-24T23:26:35.507 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:35.503+0000 7f08d9076700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f08d41127d0 con 0x7f08d4102ec0 2024-04-24T23:26:35.507 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:35.504+0000 7f08d359e700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f08d4107590 0x7f08d4198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:35.507 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:35.504+0000 7f08d359e700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f08d4107590 0x7f08d4198970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:54726/0 (socket says 172.21.15.80:54726) 2024-04-24T23:26:35.507 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:35.504+0000 7f08d359e700 1 -- 172.21.15.80:0/2604760224 learned_addr learned my addr 172.21.15.80:0/2604760224 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:26:35.507 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:35.504+0000 7f08d2d9d700 1 --2- 172.21.15.80:0/2604760224 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f08d4102510 0x7f08d419e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:35.507 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:35.504+0000 7f08d259c700 1 --2- 172.21.15.80:0/2604760224 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f08d4102ec0 0x7f08d419ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:35.507 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:35.504+0000 7f08d359e700 1 -- 172.21.15.80:0/2604760224 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f08d4102ec0 msgr2=0x7f08d419ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:35.508 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:35.504+0000 7f08d359e700 1 --2- 172.21.15.80:0/2604760224 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f08d4102ec0 0x7f08d419ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:35.508 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:35.504+0000 7f08d359e700 1 -- 172.21.15.80:0/2604760224 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f08d4102510 msgr2=0x7f08d419e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:35.508 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:35.504+0000 7f08d359e700 1 --2- 172.21.15.80:0/2604760224 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f08d4102510 0x7f08d419e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:35.508 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:35.504+0000 7f08d359e700 1 -- 172.21.15.80:0/2604760224 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08d41990d0 con 0x7f08d4107590 2024-04-24T23:26:35.508 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:35.504+0000 7f08d359e700 1 --2- 172.21.15.80:0/2604760224 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f08d4107590 0x7f08d4198970 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f08cc00b970 tx=0x7f08cc00bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:35.508 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:35.504+0000 7f08bbfff700 1 -- 172.21.15.80:0/2604760224 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08cc00d860 con 0x7f08d4107590 2024-04-24T23:26:35.508 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:35.504+0000 7f08bbfff700 1 -- 172.21.15.80:0/2604760224 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08cc00e430 con 0x7f08d4107590 2024-04-24T23:26:35.508 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:35.504+0000 7f08d9076700 1 -- 172.21.15.80:0/2604760224 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08d41993c0 con 0x7f08d4107590 2024-04-24T23:26:35.508 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:35.504+0000 7f08bbfff700 1 -- 172.21.15.80:0/2604760224 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08cc017660 con 0x7f08d4107590 2024-04-24T23:26:35.508 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:35.504+0000 7f08d9076700 1 -- 172.21.15.80:0/2604760224 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08d41a5680 con 0x7f08d4107590 2024-04-24T23:26:35.508 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:35.505+0000 7f08d9076700 1 -- 172.21.15.80:0/2604760224 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08d4068f20 con 0x7f08d4107590 2024-04-24T23:26:35.509 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:35.506+0000 7f08bbfff700 1 -- 172.21.15.80:0/2604760224 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f08cc00e5e0 con 0x7f08d4107590 2024-04-24T23:26:35.509 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:35.507+0000 7f08bbfff700 1 --2- 172.21.15.80:0/2604760224 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f08bc0a2c20 0x7f08bc0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:35.509 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:35.507+0000 7f08d2d9d700 1 --2- 172.21.15.80:0/2604760224 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f08bc0a2c20 0x7f08bc0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:35.509 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:35.507+0000 7f08bbfff700 1 -- 172.21.15.80:0/2604760224 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f08cc0ce970 con 0x7f08d4107590 2024-04-24T23:26:35.509 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:35.507+0000 7f08d2d9d700 1 --2- 172.21.15.80:0/2604760224 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f08bc0a2c20 0x7f08bc0a50e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f08c400b5c0 tx=0x7f08c4005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:35.511 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:35.511+0000 7f08bbfff700 1 -- 172.21.15.80:0/2604760224 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f08cc013030 con 0x7f08d4107590 2024-04-24T23:26:35.705 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:35.703+0000 7f08d9076700 1 -- 172.21.15.80:0/2604760224 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f08d4199ea0 con 0x7f08d4107590 2024-04-24T23:26:35.705 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:35.704+0000 7f08bbfff700 1 -- 172.21.15.80:0/2604760224 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7f08cc09bd70 con 0x7f08d4107590 2024-04-24T23:26:35.705 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 1 2024-04-24T23:26:35.707 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:35.706+0000 7f08d9076700 1 -- 172.21.15.80:0/2604760224 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f08bc0a2c20 msgr2=0x7f08bc0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:35.707 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:35.706+0000 7f08d9076700 1 --2- 172.21.15.80:0/2604760224 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f08bc0a2c20 0x7f08bc0a50e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f08c400b5c0 tx=0x7f08c4005960 comp rx=0 tx=0).stop 2024-04-24T23:26:35.707 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:35.706+0000 7f08d9076700 1 -- 172.21.15.80:0/2604760224 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f08d4107590 msgr2=0x7f08d4198970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:35.707 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:35.706+0000 7f08d9076700 1 --2- 172.21.15.80:0/2604760224 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f08d4107590 0x7f08d4198970 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f08cc00b970 tx=0x7f08cc00bc80 comp rx=0 tx=0).stop 2024-04-24T23:26:35.707 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:35.706+0000 7f08d9076700 1 -- 172.21.15.80:0/2604760224 shutdown_connections 2024-04-24T23:26:35.708 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:35.706+0000 7f08d9076700 1 --2- 172.21.15.80:0/2604760224 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f08bc0a2c20 0x7f08bc0a50e0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:35.708 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:35.706+0000 7f08d9076700 1 --2- 172.21.15.80:0/2604760224 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f08d4107590 0x7f08d4198970 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:35.708 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:35.706+0000 7f08d9076700 1 --2- 172.21.15.80:0/2604760224 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f08d4102510 0x7f08d419e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:35.708 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:35.706+0000 7f08d9076700 1 --2- 172.21.15.80:0/2604760224 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f08d4102ec0 0x7f08d419ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:35.708 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:35.706+0000 7f08d9076700 1 -- 172.21.15.80:0/2604760224 >> 172.21.15.80:0/2604760224 conn(0x7f08d40793a0 msgr2=0x7f08d4108710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:35.708 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:35.706+0000 7f08d9076700 1 -- 172.21.15.80:0/2604760224 shutdown_connections 2024-04-24T23:26:35.708 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:35.706+0000 7f08d9076700 1 -- 172.21.15.80:0/2604760224 wait complete. 2024-04-24T23:26:36.038 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:35 smithi080 ceph-mon[29152]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:26:36.038 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:35 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/2604760224' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:26:36.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:35 smithi134 ceph-mon[32619]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:26:36.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:35 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/2604760224' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:26:36.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:35 smithi148 ceph-mon[32528]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:26:36.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:35 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/2604760224' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:26:36.513 INFO:tasks.cephadm:Adding mds.a on smithi080 2024-04-24T23:26:36.513 INFO:tasks.cephadm:Adding mds.d on smithi080 2024-04-24T23:26:36.513 INFO:tasks.cephadm:Adding mds.g on smithi080 2024-04-24T23:26:36.513 INFO:tasks.cephadm:Adding mds.j on smithi080 2024-04-24T23:26:36.513 INFO:tasks.cephadm:Adding mds.b on smithi134 2024-04-24T23:26:36.513 INFO:tasks.cephadm:Adding mds.e on smithi134 2024-04-24T23:26:36.513 INFO:tasks.cephadm:Adding mds.h on smithi134 2024-04-24T23:26:36.513 INFO:tasks.cephadm:Adding mds.k on smithi134 2024-04-24T23:26:36.514 INFO:tasks.cephadm:Adding mds.c on smithi148 2024-04-24T23:26:36.514 INFO:tasks.cephadm:Adding mds.f on smithi148 2024-04-24T23:26:36.514 INFO:tasks.cephadm:Adding mds.i on smithi148 2024-04-24T23:26:36.514 INFO:tasks.cephadm:Adding mds.l on smithi148 2024-04-24T23:26:36.514 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ec7885aa-0290-11ef-bc93-c7b262605968 -- ceph orch apply mds all '12;smithi080=a;smithi080=d;smithi080=g;smithi080=j;smithi134=b;smithi134=e;smithi134=h;smithi134=k;smithi148=c;smithi148=f;smithi148=i;smithi148=l' 2024-04-24T23:26:36.764 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T23:26:38.038 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:37 smithi080 ceph-mon[29152]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:26:38.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:37 smithi134 ceph-mon[32619]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:26:38.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:37 smithi148 ceph-mon[32528]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:26:38.506 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:26:38.503+0000 7fe40e86e700 1 -- 172.21.15.148:0/717321510 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4081023b0 msgr2=0x7fe408102810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:38.506 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:26:38.503+0000 7fe40e86e700 1 --2- 172.21.15.148:0/717321510 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4081023b0 0x7fe408102810 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe3f8009990 tx=0x7fe3f8009ca0 comp rx=0 tx=0).stop 2024-04-24T23:26:38.506 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:26:38.503+0000 7fe40e86e700 1 -- 172.21.15.148:0/717321510 shutdown_connections 2024-04-24T23:26:38.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:26:38.503+0000 7fe40e86e700 1 --2- 172.21.15.148:0/717321510 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe408102d50 0x7fe408109600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:38.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:26:38.503+0000 7fe40e86e700 1 --2- 172.21.15.148:0/717321510 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe4081011b0 0x7fe4081015d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:38.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:26:38.503+0000 7fe40e86e700 1 --2- 172.21.15.148:0/717321510 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4081023b0 0x7fe408102810 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:38.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:26:38.503+0000 7fe40e86e700 1 -- 172.21.15.148:0/717321510 >> 172.21.15.148:0/717321510 conn(0x7fe4080fc790 msgr2=0x7fe4080febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:38.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:26:38.503+0000 7fe40e86e700 1 -- 172.21.15.148:0/717321510 shutdown_connections 2024-04-24T23:26:38.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:26:38.503+0000 7fe40e86e700 1 -- 172.21.15.148:0/717321510 wait complete. 2024-04-24T23:26:38.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:26:38.504+0000 7fe40e86e700 1 Processor -- start 2024-04-24T23:26:38.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:26:38.504+0000 7fe40e86e700 1 -- start start 2024-04-24T23:26:38.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:26:38.504+0000 7fe40e86e700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe4081011b0 0x7fe40806b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:38.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:26:38.504+0000 7fe40e86e700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe4081023b0 0x7fe40819d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:38.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:26:38.504+0000 7fe40e86e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe408102d50 0x7fe40819f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:38.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:26:38.504+0000 7fe40e86e700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fe40810ce00 con 0x7fe4081011b0 2024-04-24T23:26:38.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:26:38.504+0000 7fe40e86e700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe40810cc80 con 0x7fe4081023b0 2024-04-24T23:26:38.508 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:26:38.504+0000 7fe40e86e700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe40810cf80 con 0x7fe408102d50 2024-04-24T23:26:38.508 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:26:38.505+0000 7fe40ce0b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe408102d50 0x7fe40819f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:38.508 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:26:38.505+0000 7fe40ce0b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe408102d50 0x7fe40819f890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:37646/0 (socket says 172.21.15.148:37646) 2024-04-24T23:26:38.508 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:26:38.505+0000 7fe40ce0b700 1 -- 172.21.15.148:0/2367607679 learned_addr learned my addr 172.21.15.148:0/2367607679 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-24T23:26:38.508 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:26:38.505+0000 7fe407fff700 1 --2- 172.21.15.148:0/2367607679 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe4081011b0 0x7fe40806b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:38.508 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:26:38.505+0000 7fe4077fe700 1 --2- 172.21.15.148:0/2367607679 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe4081023b0 0x7fe40819d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:38.508 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:26:38.505+0000 7fe40ce0b700 1 -- 172.21.15.148:0/2367607679 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe4081023b0 msgr2=0x7fe40819d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:38.508 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:26:38.505+0000 7fe40ce0b700 1 --2- 172.21.15.148:0/2367607679 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe4081023b0 0x7fe40819d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:38.508 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:26:38.505+0000 7fe40ce0b700 1 -- 172.21.15.148:0/2367607679 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe4081011b0 msgr2=0x7fe40806b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:38.508 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:26:38.505+0000 7fe40ce0b700 1 --2- 172.21.15.148:0/2367607679 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe4081011b0 0x7fe40806b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:38.508 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:26:38.505+0000 7fe40ce0b700 1 -- 172.21.15.148:0/2367607679 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4081a0160 con 0x7fe408102d50 2024-04-24T23:26:38.508 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:26:38.505+0000 7fe40ce0b700 1 --2- 172.21.15.148:0/2367607679 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe408102d50 0x7fe40819f890 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe40000b970 tx=0x7fe40000bc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:38.508 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:26:38.505+0000 7fe4057fa700 1 -- 172.21.15.148:0/2367607679 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe400019070 con 0x7fe408102d50 2024-04-24T23:26:38.508 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:26:38.505+0000 7fe4057fa700 1 -- 172.21.15.148:0/2367607679 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe40000ce10 con 0x7fe408102d50 2024-04-24T23:26:38.508 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:26:38.505+0000 7fe4057fa700 1 -- 172.21.15.148:0/2367607679 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe400007370 con 0x7fe408102d50 2024-04-24T23:26:38.509 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:26:38.505+0000 7fe40e86e700 1 -- 172.21.15.148:0/2367607679 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4081a0450 con 0x7fe408102d50 2024-04-24T23:26:38.509 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:26:38.505+0000 7fe40e86e700 1 -- 172.21.15.148:0/2367607679 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4081a0940 con 0x7fe408102d50 2024-04-24T23:26:38.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:26:38.506+0000 7fe40e86e700 1 -- 172.21.15.148:0/2367607679 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe408068f20 con 0x7fe408102d50 2024-04-24T23:26:38.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:26:38.508+0000 7fe4057fa700 1 -- 172.21.15.148:0/2367607679 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fe40001c070 con 0x7fe408102d50 2024-04-24T23:26:38.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:26:38.509+0000 7fe4057fa700 1 --2- 172.21.15.148:0/2367607679 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fe3f40a2c20 0x7fe3f40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:38.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:26:38.509+0000 7fe4057fa700 1 -- 172.21.15.148:0/2367607679 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fe4000cdf70 con 0x7fe408102d50 2024-04-24T23:26:38.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:26:38.509+0000 7fe407fff700 1 --2- 172.21.15.148:0/2367607679 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fe3f40a2c20 0x7fe3f40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:38.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:26:38.510+0000 7fe407fff700 1 --2- 172.21.15.148:0/2367607679 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fe3f40a2c20 0x7fe3f40a50e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe408102210 tx=0x7fe3f0007400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:38.515 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:26:38.515+0000 7fe4057fa700 1 -- 172.21.15.148:0/2367607679 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fe40009b370 con 0x7fe408102d50 2024-04-24T23:26:38.721 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:26:38.720+0000 7fe40e86e700 1 -- 172.21.15.148:0/2367607679 --> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi080=a;smithi080=d;smithi080=g;smithi080=j;smithi134=b;smithi134=e;smithi134=h;smithi134=k;smithi148=c;smithi148=f;smithi148=i;smithi148=l", "target": ["mon-mgr", ""]}) -- 0x7fe408062870 con 0x7fe3f40a2c20 2024-04-24T23:26:38.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:26:38.728+0000 7fe4057fa700 1 -- 172.21.15.148:0/2367607679 <== mgr.14152 v2:172.21.15.80:6800/3830596922 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7fe408062870 con 0x7fe3f40a2c20 2024-04-24T23:26:38.729 INFO:teuthology.orchestra.run.smithi148.stdout:Scheduled mds.all update... 2024-04-24T23:26:38.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:26:38.730+0000 7fe40e86e700 1 -- 172.21.15.148:0/2367607679 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fe3f40a2c20 msgr2=0x7fe3f40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:38.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:26:38.730+0000 7fe40e86e700 1 --2- 172.21.15.148:0/2367607679 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fe3f40a2c20 0x7fe3f40a50e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe408102210 tx=0x7fe3f0007400 comp rx=0 tx=0).stop 2024-04-24T23:26:38.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:26:38.730+0000 7fe40e86e700 1 -- 172.21.15.148:0/2367607679 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe408102d50 msgr2=0x7fe40819f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:38.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:26:38.730+0000 7fe40e86e700 1 --2- 172.21.15.148:0/2367607679 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe408102d50 0x7fe40819f890 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe40000b970 tx=0x7fe40000bc80 comp rx=0 tx=0).stop 2024-04-24T23:26:38.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:26:38.730+0000 7fe40e86e700 1 -- 172.21.15.148:0/2367607679 shutdown_connections 2024-04-24T23:26:38.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:26:38.730+0000 7fe40e86e700 1 --2- 172.21.15.148:0/2367607679 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe408102d50 0x7fe40819f890 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:38.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:26:38.730+0000 7fe40e86e700 1 --2- 172.21.15.148:0/2367607679 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fe3f40a2c20 0x7fe3f40a50e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:38.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:26:38.730+0000 7fe40e86e700 1 --2- 172.21.15.148:0/2367607679 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe4081011b0 0x7fe40806b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:38.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:26:38.730+0000 7fe40e86e700 1 --2- 172.21.15.148:0/2367607679 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe4081023b0 0x7fe40819d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:38.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:26:38.730+0000 7fe40e86e700 1 -- 172.21.15.148:0/2367607679 >> 172.21.15.148:0/2367607679 conn(0x7fe4080fc790 msgr2=0x7fe408105730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:38.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:26:38.730+0000 7fe40e86e700 1 -- 172.21.15.148:0/2367607679 shutdown_connections 2024-04-24T23:26:38.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-24T23:26:38.730+0000 7fe40e86e700 1 -- 172.21.15.148:0/2367607679 wait complete. 2024-04-24T23:26:39.465 DEBUG:teuthology.orchestra.run.smithi080:mds.a> sudo journalctl -f -n 0 -u ceph-ec7885aa-0290-11ef-bc93-c7b262605968@mds.a.service 2024-04-24T23:26:39.468 DEBUG:teuthology.orchestra.run.smithi080:mds.d> sudo journalctl -f -n 0 -u ceph-ec7885aa-0290-11ef-bc93-c7b262605968@mds.d.service 2024-04-24T23:26:39.512 DEBUG:teuthology.orchestra.run.smithi080:mds.g> sudo journalctl -f -n 0 -u ceph-ec7885aa-0290-11ef-bc93-c7b262605968@mds.g.service 2024-04-24T23:26:39.556 DEBUG:teuthology.orchestra.run.smithi080:mds.j> sudo journalctl -f -n 0 -u ceph-ec7885aa-0290-11ef-bc93-c7b262605968@mds.j.service 2024-04-24T23:26:39.599 DEBUG:teuthology.orchestra.run.smithi134:mds.b> sudo journalctl -f -n 0 -u ceph-ec7885aa-0290-11ef-bc93-c7b262605968@mds.b.service 2024-04-24T23:26:39.602 DEBUG:teuthology.orchestra.run.smithi134:mds.e> sudo journalctl -f -n 0 -u ceph-ec7885aa-0290-11ef-bc93-c7b262605968@mds.e.service 2024-04-24T23:26:39.645 DEBUG:teuthology.orchestra.run.smithi134:mds.h> sudo journalctl -f -n 0 -u ceph-ec7885aa-0290-11ef-bc93-c7b262605968@mds.h.service 2024-04-24T23:26:39.690 DEBUG:teuthology.orchestra.run.smithi134:mds.k> sudo journalctl -f -n 0 -u ceph-ec7885aa-0290-11ef-bc93-c7b262605968@mds.k.service 2024-04-24T23:26:39.734 DEBUG:teuthology.orchestra.run.smithi148:mds.c> sudo journalctl -f -n 0 -u ceph-ec7885aa-0290-11ef-bc93-c7b262605968@mds.c.service 2024-04-24T23:26:39.736 DEBUG:teuthology.orchestra.run.smithi148:mds.f> sudo journalctl -f -n 0 -u ceph-ec7885aa-0290-11ef-bc93-c7b262605968@mds.f.service 2024-04-24T23:26:39.780 DEBUG:teuthology.orchestra.run.smithi148:mds.i> sudo journalctl -f -n 0 -u ceph-ec7885aa-0290-11ef-bc93-c7b262605968@mds.i.service 2024-04-24T23:26:39.788 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:39 smithi080 ceph-mon[29152]: from='client.24397 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi080=a;smithi080=d;smithi080=g;smithi080=j;smithi134=b;smithi134=e;smithi134=h;smithi134=k;smithi148=c;smithi148=f;smithi148=i;smithi148=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:26:39.788 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:39 smithi080 ceph-mon[29152]: Saving service mds.all spec with placement smithi080=a;smithi080=d;smithi080=g;smithi080=j;smithi134=b;smithi134=e;smithi134=h;smithi134=k;smithi148=c;smithi148=f;smithi148=i;smithi148=l;count:12 2024-04-24T23:26:39.788 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:39 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:26:39.788 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:39 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:26:39.788 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:39 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:26:39.788 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:39 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:26:39.789 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:39 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:26:39.789 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:39 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:26:39.789 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:39 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:26:39.789 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:39 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:26:39.789 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:39 smithi080 ceph-mon[29152]: Deploying daemon mds.l on smithi148 2024-04-24T23:26:39.789 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:39 smithi080 ceph-mon[29152]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:26:39.824 DEBUG:teuthology.orchestra.run.smithi148:mds.l> sudo journalctl -f -n 0 -u ceph-ec7885aa-0290-11ef-bc93-c7b262605968@mds.l.service 2024-04-24T23:26:39.866 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-04-24T23:26:39.867 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:26:39.947 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:39 smithi134 ceph-mon[32619]: from='client.24397 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi080=a;smithi080=d;smithi080=g;smithi080=j;smithi134=b;smithi134=e;smithi134=h;smithi134=k;smithi148=c;smithi148=f;smithi148=i;smithi148=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:26:39.947 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:39 smithi134 ceph-mon[32619]: Saving service mds.all spec with placement smithi080=a;smithi080=d;smithi080=g;smithi080=j;smithi134=b;smithi134=e;smithi134=h;smithi134=k;smithi148=c;smithi148=f;smithi148=i;smithi148=l;count:12 2024-04-24T23:26:39.948 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:39 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:26:39.948 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:39 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:26:39.948 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:39 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:26:39.948 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:39 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:26:39.948 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:39 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:26:39.948 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:39 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:26:39.948 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:39 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:26:39.948 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:39 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:26:39.948 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:39 smithi134 ceph-mon[32619]: Deploying daemon mds.l on smithi148 2024-04-24T23:26:39.948 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:39 smithi134 ceph-mon[32619]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:26:39.991 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:39.989+0000 7ff41cf9e640 1 -- 172.21.15.80:0/1596611676 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff4181049c0 msgr2=0x7ff418106db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:39.991 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:39.989+0000 7ff41cf9e640 1 --2- 172.21.15.80:0/1596611676 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff4181049c0 0x7ff418106db0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff40c009900 tx=0x7ff40c02a410 comp rx=0 tx=0).stop 2024-04-24T23:26:39.991 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:39.989+0000 7ff41cf9e640 1 -- 172.21.15.80:0/1596611676 shutdown_connections 2024-04-24T23:26:39.991 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:39.989+0000 7ff41cf9e640 1 --2- 172.21.15.80:0/1596611676 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff4181072f0 0x7ff4181096e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:39.991 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:39.989+0000 7ff41cf9e640 1 --2- 172.21.15.80:0/1596611676 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff4181049c0 0x7ff418106db0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:39.991 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:39.989+0000 7ff41cf9e640 1 --2- 172.21.15.80:0/1596611676 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff418102090 0x7ff418104480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:39.991 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:39.989+0000 7ff41cf9e640 1 -- 172.21.15.80:0/1596611676 >> 172.21.15.80:0/1596611676 conn(0x7ff4180fba60 msgr2=0x7ff4180fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:39.991 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:39.989+0000 7ff41cf9e640 1 -- 172.21.15.80:0/1596611676 shutdown_connections 2024-04-24T23:26:39.991 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:39.989+0000 7ff41cf9e640 1 -- 172.21.15.80:0/1596611676 wait complete. 2024-04-24T23:26:39.993 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:39.990+0000 7ff41cf9e640 1 Processor -- start 2024-04-24T23:26:39.994 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:39.990+0000 7ff41cf9e640 1 -- start start 2024-04-24T23:26:39.994 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:39.991+0000 7ff41cf9e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff418102090 0x7ff418197850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:39.994 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:39.991+0000 7ff41cf9e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff4181049c0 0x7ff418197d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:39.994 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:39.991+0000 7ff41cf9e640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff4181072f0 0x7ff41819ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:39.994 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:39.991+0000 7ff41cf9e640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7ff41810d260 con 0x7ff4181072f0 2024-04-24T23:26:39.994 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:39.991+0000 7ff41cf9e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff41810d0e0 con 0x7ff4181049c0 2024-04-24T23:26:39.994 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:39.991+0000 7ff41cf9e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff41810d3e0 con 0x7ff418102090 2024-04-24T23:26:39.994 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:39.991+0000 7ff417fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff4181072f0 0x7ff41819ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:39.994 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:39.991+0000 7ff417fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff4181072f0 0x7ff41819ee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:54736/0 (socket says 172.21.15.80:54736) 2024-04-24T23:26:39.994 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:39.991+0000 7ff417fff640 1 -- 172.21.15.80:0/2236524362 learned_addr learned my addr 172.21.15.80:0/2236524362 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:26:39.994 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:39.991+0000 7ff4177fe640 1 --2- 172.21.15.80:0/2236524362 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff418102090 0x7ff418197850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:39.994 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:39.991+0000 7ff416ffd640 1 --2- 172.21.15.80:0/2236524362 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff4181049c0 0x7ff418197d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:39.994 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:39.991+0000 7ff417fff640 1 -- 172.21.15.80:0/2236524362 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff418102090 msgr2=0x7ff418197850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:39.995 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:39.991+0000 7ff417fff640 1 --2- 172.21.15.80:0/2236524362 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff418102090 0x7ff418197850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:39.995 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:39.991+0000 7ff417fff640 1 -- 172.21.15.80:0/2236524362 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff4181049c0 msgr2=0x7ff418197d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:39.995 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:39.991+0000 7ff417fff640 1 --2- 172.21.15.80:0/2236524362 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff4181049c0 0x7ff418197d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:39.995 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:39.991+0000 7ff417fff640 1 -- 172.21.15.80:0/2236524362 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff41819f470 con 0x7ff4181072f0 2024-04-24T23:26:39.995 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:39.991+0000 7ff4177fe640 1 --2- 172.21.15.80:0/2236524362 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff418102090 0x7ff418197850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:26:39.995 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:39.991+0000 7ff416ffd640 1 --2- 172.21.15.80:0/2236524362 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff4181049c0 0x7ff418197d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:26:39.995 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:39.991+0000 7ff417fff640 1 --2- 172.21.15.80:0/2236524362 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff4181072f0 0x7ff41819ee00 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff410009fd0 tx=0x7ff41000def0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:39.995 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:39.991+0000 7ff414ff9640 1 -- 172.21.15.80:0/2236524362 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff410017070 con 0x7ff4181072f0 2024-04-24T23:26:39.995 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:39.991+0000 7ff414ff9640 1 -- 172.21.15.80:0/2236524362 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff410003e50 con 0x7ff4181072f0 2024-04-24T23:26:39.995 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:39.991+0000 7ff41cf9e640 1 -- 172.21.15.80:0/2236524362 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff41819f760 con 0x7ff4181072f0 2024-04-24T23:26:39.995 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:39.991+0000 7ff414ff9640 1 -- 172.21.15.80:0/2236524362 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff410010460 con 0x7ff4181072f0 2024-04-24T23:26:39.995 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:39.991+0000 7ff41cf9e640 1 -- 172.21.15.80:0/2236524362 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff41819fc30 con 0x7ff4181072f0 2024-04-24T23:26:39.995 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:39.993+0000 7ff41cf9e640 1 -- 172.21.15.80:0/2236524362 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4180fd550 con 0x7ff4181072f0 2024-04-24T23:26:39.996 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:39.994+0000 7ff414ff9640 1 -- 172.21.15.80:0/2236524362 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7ff410003970 con 0x7ff4181072f0 2024-04-24T23:26:39.996 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:39.994+0000 7ff414ff9640 1 --2- 172.21.15.80:0/2236524362 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7ff4000a2b90 0x7ff4000a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:39.996 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:39.994+0000 7ff4177fe640 1 --2- 172.21.15.80:0/2236524362 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7ff4000a2b90 0x7ff4000a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:39.996 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:39.994+0000 7ff414ff9640 1 -- 172.21.15.80:0/2236524362 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7ff4100ce870 con 0x7ff4181072f0 2024-04-24T23:26:39.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:39.995+0000 7ff4177fe640 1 --2- 172.21.15.80:0/2236524362 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7ff4000a2b90 0x7ff4000a5050 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff40800afb0 tx=0x7ff40800a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:39.999 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:39.998+0000 7ff414ff9640 1 -- 172.21.15.80:0/2236524362 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff41009bc70 con 0x7ff4181072f0 2024-04-24T23:26:40.006 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:39 smithi148 ceph-mon[32528]: from='client.24397 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi080=a;smithi080=d;smithi080=g;smithi080=j;smithi134=b;smithi134=e;smithi134=h;smithi134=k;smithi148=c;smithi148=f;smithi148=i;smithi148=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:26:40.006 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:39 smithi148 ceph-mon[32528]: Saving service mds.all spec with placement smithi080=a;smithi080=d;smithi080=g;smithi080=j;smithi134=b;smithi134=e;smithi134=h;smithi134=k;smithi148=c;smithi148=f;smithi148=i;smithi148=l;count:12 2024-04-24T23:26:40.006 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:39 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:26:40.006 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:39 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:26:40.007 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:39 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:26:40.007 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:39 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:26:40.007 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:39 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:26:40.007 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:39 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:26:40.007 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:39 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:26:40.007 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:39 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:26:40.007 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:39 smithi148 ceph-mon[32528]: Deploying daemon mds.l on smithi148 2024-04-24T23:26:40.007 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:39 smithi148 ceph-mon[32528]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:26:40.179 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.178+0000 7ff41cf9e640 1 -- 172.21.15.80:0/2236524362 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff418062340 con 0x7ff4181072f0 2024-04-24T23:26:40.180 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.179+0000 7ff414ff9640 1 -- 172.21.15.80:0/2236524362 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+18452 (secure 0 0 0) 0x7ff4100bb070 con 0x7ff4181072f0 2024-04-24T23:26:40.180 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-24T23:26:40.180 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":81,"fsid":"ec7885aa-0290-11ef-bc93-c7b262605968","created":"2024-04-24T23:19:05.586407+0000","modified":"2024-04-24T23:26:27.519820+0000","last_up_change":"2024-04-24T23:26:25.938316+0000","last_in_change":"2024-04-24T23:26:04.786699+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:22:33.124428+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a525741d-b46c-4eef-aa64-3f057cac1497","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":2597718112},{"type":"v1","addr":"172.21.15.80:6803","nonce":2597718112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":2597718112},{"type":"v1","addr":"172.21.15.80:6805","nonce":2597718112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2597718112},{"type":"v1","addr":"172.21.15.80:6809","nonce":2597718112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":2597718112},{"type":"v1","addr":"172.21.15.80:6807","nonce":2597718112}]},"public_addr":"172.21.15.80:6803/2597718112","cluster_addr":"172.21.15.80:6805/2597718112","heartbeat_back_addr":"172.21.15.80:6809/2597718112","heartbeat_front_addr":"172.21.15.80:6807/2597718112","state":["exists","up"]},{"osd":1,"uuid":"2d0f0d67-6db4-40ac-91cc-58604ab6795a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":2395610201},{"type":"v1","addr":"172.21.15.134:6801","nonce":2395610201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":2395610201},{"type":"v1","addr":"172.21.15.134:6803","nonce":2395610201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":2395610201},{"type":"v1","addr":"172.21.15.134:6807","nonce":2395610201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":2395610201},{"type":"v1","addr":"172.21.15.134:6805","nonce":2395610201}]},"public_addr":"172.21.15.134:6801/2395610201","cluster_addr":"172.21.15.134:6803/2395610201","heartbeat_back_addr":"172.21.15.134:6807/2395610201","heartbeat_front_addr":"172.21.15.134:6805/2395610201","state":["exists","up"]},{"osd":2,"uuid":"781fb879-fedc-4c84-82fb-59e073bc4cf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1629152187},{"type":"v1","addr":"172.21.15.148:6801","nonce":1629152187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1629152187},{"type":"v1","addr":"172.21.15.148:6803","nonce":1629152187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1629152187},{"type":"v1","addr":"172.21.15.148:6807","nonce":1629152187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1629152187},{"type":"v1","addr":"172.21.15.148:6805","nonce":1629152187}]},"public_addr":"172.21.15.148:6801/1629152187","cluster_addr":"172.21.15.148:6803/1629152187","heartbeat_back_addr":"172.21.15.148:6807/1629152187","heartbeat_front_addr":"172.21.15.148:6805/1629152187","state":["exists","up"]},{"osd":3,"uuid":"cd982757-f2f6-4296-b3e7-4cd40fbd8d9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":3128298510},{"type":"v1","addr":"172.21.15.80:6811","nonce":3128298510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":3128298510},{"type":"v1","addr":"172.21.15.80:6813","nonce":3128298510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":3128298510},{"type":"v1","addr":"172.21.15.80:6817","nonce":3128298510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":3128298510},{"type":"v1","addr":"172.21.15.80:6815","nonce":3128298510}]},"public_addr":"172.21.15.80:6811/3128298510","cluster_addr":"172.21.15.80:6813/3128298510","heartbeat_back_addr":"172.21.15.80:6817/3128298510","heartbeat_front_addr":"172.21.15.80:6815/3128298510","state":["exists","up"]},{"osd":4,"uuid":"dfc55a79-a7e8-417a-b898-ec44171a54af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1917556060},{"type":"v1","addr":"172.21.15.134:6809","nonce":1917556060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1917556060},{"type":"v1","addr":"172.21.15.134:6811","nonce":1917556060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1917556060},{"type":"v1","addr":"172.21.15.134:6815","nonce":1917556060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1917556060},{"type":"v1","addr":"172.21.15.134:6813","nonce":1917556060}]},"public_addr":"172.21.15.134:6809/1917556060","cluster_addr":"172.21.15.134:6811/1917556060","heartbeat_back_addr":"172.21.15.134:6815/1917556060","heartbeat_front_addr":"172.21.15.134:6813/1917556060","state":["exists","up"]},{"osd":5,"uuid":"b5491337-0694-440c-b679-312f3d14cf65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":2715951281},{"type":"v1","addr":"172.21.15.148:6809","nonce":2715951281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":2715951281},{"type":"v1","addr":"172.21.15.148:6811","nonce":2715951281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":2715951281},{"type":"v1","addr":"172.21.15.148:6815","nonce":2715951281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":2715951281},{"type":"v1","addr":"172.21.15.148:6813","nonce":2715951281}]},"public_addr":"172.21.15.148:6809/2715951281","cluster_addr":"172.21.15.148:6811/2715951281","heartbeat_back_addr":"172.21.15.148:6815/2715951281","heartbeat_front_addr":"172.21.15.148:6813/2715951281","state":["exists","up"]},{"osd":6,"uuid":"ba4b5df7-d86e-47cd-ab67-cf763c4e4d6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":953261724},{"type":"v1","addr":"172.21.15.80:6819","nonce":953261724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":953261724},{"type":"v1","addr":"172.21.15.80:6821","nonce":953261724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":953261724},{"type":"v1","addr":"172.21.15.80:6825","nonce":953261724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":953261724},{"type":"v1","addr":"172.21.15.80:6823","nonce":953261724}]},"public_addr":"172.21.15.80:6819/953261724","cluster_addr":"172.21.15.80:6821/953261724","heartbeat_back_addr":"172.21.15.80:6825/953261724","heartbeat_front_addr":"172.21.15.80:6823/953261724","state":["exists","up"]},{"osd":7,"uuid":"464a503a-75ae-49e7-ab6e-a7f9f2a6a307","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":108354844},{"type":"v1","addr":"172.21.15.134:6817","nonce":108354844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":108354844},{"type":"v1","addr":"172.21.15.134:6819","nonce":108354844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":108354844},{"type":"v1","addr":"172.21.15.134:6823","nonce":108354844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":108354844},{"type":"v1","addr":"172.21.15.134:6821","nonce":108354844}]},"public_addr":"172.21.15.134:6817/108354844","cluster_addr":"172.21.15.134:6819/108354844","heartbeat_back_addr":"172.21.15.134:6823/108354844","heartbeat_front_addr":"172.21.15.134:6821/108354844","state":["exists","up"]},{"osd":8,"uuid":"ce544b2e-4540-47c0-9997-500c45351330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3559655329},{"type":"v1","addr":"172.21.15.148:6817","nonce":3559655329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":3559655329},{"type":"v1","addr":"172.21.15.148:6819","nonce":3559655329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":3559655329},{"type":"v1","addr":"172.21.15.148:6823","nonce":3559655329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3559655329},{"type":"v1","addr":"172.21.15.148:6821","nonce":3559655329}]},"public_addr":"172.21.15.148:6817/3559655329","cluster_addr":"172.21.15.148:6819/3559655329","heartbeat_back_addr":"172.21.15.148:6823/3559655329","heartbeat_front_addr":"172.21.15.148:6821/3559655329","state":["exists","up"]},{"osd":9,"uuid":"06de3732-19b9-496c-a464-55322efa6759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":4019913545},{"type":"v1","addr":"172.21.15.80:6827","nonce":4019913545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":4019913545},{"type":"v1","addr":"172.21.15.80:6829","nonce":4019913545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":4019913545},{"type":"v1","addr":"172.21.15.80:6833","nonce":4019913545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":4019913545},{"type":"v1","addr":"172.21.15.80:6831","nonce":4019913545}]},"public_addr":"172.21.15.80:6827/4019913545","cluster_addr":"172.21.15.80:6829/4019913545","heartbeat_back_addr":"172.21.15.80:6833/4019913545","heartbeat_front_addr":"172.21.15.80:6831/4019913545","state":["exists","up"]},{"osd":10,"uuid":"8d92f0d9-5c64-429a-b750-c33ae0e070c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":647343766},{"type":"v1","addr":"172.21.15.134:6825","nonce":647343766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":647343766},{"type":"v1","addr":"172.21.15.134:6827","nonce":647343766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":647343766},{"type":"v1","addr":"172.21.15.134:6831","nonce":647343766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":647343766},{"type":"v1","addr":"172.21.15.134:6829","nonce":647343766}]},"public_addr":"172.21.15.134:6825/647343766","cluster_addr":"172.21.15.134:6827/647343766","heartbeat_back_addr":"172.21.15.134:6831/647343766","heartbeat_front_addr":"172.21.15.134:6829/647343766","state":["exists","up"]},{"osd":11,"uuid":"4c003a41-f65b-479f-bc80-b27d680cf936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":4068024116},{"type":"v1","addr":"172.21.15.148:6825","nonce":4068024116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":4068024116},{"type":"v1","addr":"172.21.15.148:6827","nonce":4068024116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":4068024116},{"type":"v1","addr":"172.21.15.148:6831","nonce":4068024116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":4068024116},{"type":"v1","addr":"172.21.15.148:6829","nonce":4068024116}]},"public_addr":"172.21.15.148:6825/4068024116","cluster_addr":"172.21.15.148:6827/4068024116","heartbeat_back_addr":"172.21.15.148:6831/4068024116","heartbeat_front_addr":"172.21.15.148:6829/4068024116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:21:42.433631+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:22:06.145933+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:22:30.726143+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:22:55.729961+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:23:21.280913+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:23:46.609983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:24:12.585247+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:24:38.359287+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:25:04.491232+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:25:30.788089+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:25:56.891078+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:26:23.899113+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.80:0/2828378749":"2024-04-25T23:20:06.793609+0000","172.21.15.80:6801/233364790":"2024-04-25T23:20:06.793609+0000","172.21.15.80:6800/1614691410":"2024-04-25T23:19:40.490692+0000","172.21.15.80:0/1406187145":"2024-04-25T23:20:06.793609+0000","172.21.15.80:0/92953396":"2024-04-25T23:19:40.490692+0000","172.21.15.80:0/598129432":"2024-04-25T23:20:06.793609+0000","172.21.15.80:0/232193118":"2024-04-25T23:19:40.490692+0000","172.21.15.80:6800/233364790":"2024-04-25T23:20:06.793609+0000","172.21.15.80:6801/1614691410":"2024-04-25T23:19:40.490692+0000","172.21.15.80:0/2081562529":"2024-04-25T23:19:40.490692+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:26:40.182 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.180+0000 7ff41cf9e640 1 -- 172.21.15.80:0/2236524362 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7ff4000a2b90 msgr2=0x7ff4000a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:40.182 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.180+0000 7ff41cf9e640 1 --2- 172.21.15.80:0/2236524362 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7ff4000a2b90 0x7ff4000a5050 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff40800afb0 tx=0x7ff40800a2d0 comp rx=0 tx=0).stop 2024-04-24T23:26:40.182 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.180+0000 7ff41cf9e640 1 -- 172.21.15.80:0/2236524362 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff4181072f0 msgr2=0x7ff41819ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:40.182 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.180+0000 7ff41cf9e640 1 --2- 172.21.15.80:0/2236524362 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff4181072f0 0x7ff41819ee00 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff410009fd0 tx=0x7ff41000def0 comp rx=0 tx=0).stop 2024-04-24T23:26:40.182 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.181+0000 7ff41cf9e640 1 -- 172.21.15.80:0/2236524362 shutdown_connections 2024-04-24T23:26:40.182 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.181+0000 7ff41cf9e640 1 --2- 172.21.15.80:0/2236524362 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7ff4000a2b90 0x7ff4000a5050 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:40.182 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.181+0000 7ff41cf9e640 1 --2- 172.21.15.80:0/2236524362 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff4181072f0 0x7ff41819ee00 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:40.182 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.181+0000 7ff41cf9e640 1 --2- 172.21.15.80:0/2236524362 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff4181049c0 0x7ff418197d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:40.183 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.181+0000 7ff41cf9e640 1 --2- 172.21.15.80:0/2236524362 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff418102090 0x7ff418197850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:40.183 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.181+0000 7ff41cf9e640 1 -- 172.21.15.80:0/2236524362 >> 172.21.15.80:0/2236524362 conn(0x7ff4180fba60 msgr2=0x7ff4181029d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:40.183 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.181+0000 7ff41cf9e640 1 -- 172.21.15.80:0/2236524362 shutdown_connections 2024-04-24T23:26:40.183 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.181+0000 7ff41cf9e640 1 -- 172.21.15.80:0/2236524362 wait complete. 2024-04-24T23:26:40.193 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:22:33.124428+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:26:40.193 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:26:40.297 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.293+0000 7f46c07e7640 1 -- 172.21.15.80:0/2114884725 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f46b8104b30 msgr2=0x7f46b8106f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:40.297 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.293+0000 7f46c07e7640 1 --2- 172.21.15.80:0/2114884725 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f46b8104b30 0x7f46b8106f20 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f46a8009900 tx=0x7f46a802a410 comp rx=0 tx=0).stop 2024-04-24T23:26:40.297 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.293+0000 7f46c07e7640 1 -- 172.21.15.80:0/2114884725 shutdown_connections 2024-04-24T23:26:40.297 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.293+0000 7f46c07e7640 1 --2- 172.21.15.80:0/2114884725 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46b8107460 0x7f46b81098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:40.297 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.293+0000 7f46c07e7640 1 --2- 172.21.15.80:0/2114884725 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f46b8104b30 0x7f46b8106f20 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:40.297 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.293+0000 7f46c07e7640 1 --2- 172.21.15.80:0/2114884725 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f46b8102200 0x7f46b81045f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:40.297 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.293+0000 7f46c07e7640 1 -- 172.21.15.80:0/2114884725 >> 172.21.15.80:0/2114884725 conn(0x7f46b80fbbf0 msgr2=0x7f46b80fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:40.297 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.293+0000 7f46c07e7640 1 -- 172.21.15.80:0/2114884725 shutdown_connections 2024-04-24T23:26:40.297 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.293+0000 7f46c07e7640 1 -- 172.21.15.80:0/2114884725 wait complete. 2024-04-24T23:26:40.297 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.294+0000 7f46c07e7640 1 Processor -- start 2024-04-24T23:26:40.298 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.294+0000 7f46c07e7640 1 -- start start 2024-04-24T23:26:40.298 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.294+0000 7f46c07e7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f46b8102200 0x7f46b8199d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:40.298 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.294+0000 7f46c07e7640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f46b8104b30 0x7f46b819a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:40.298 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.294+0000 7f46c07e7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46b8107460 0x7f46b81a1270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:40.298 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.294+0000 7f46c07e7640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f46b810d3f0 con 0x7f46b8104b30 2024-04-24T23:26:40.298 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.294+0000 7f46c07e7640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f46b810d270 con 0x7f46b8102200 2024-04-24T23:26:40.298 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.294+0000 7f46c07e7640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f46b810d570 con 0x7f46b8107460 2024-04-24T23:26:40.298 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.294+0000 7f46bdd82640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f46b8104b30 0x7f46b819a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:40.298 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.294+0000 7f46bdd82640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f46b8104b30 0x7f46b819a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:54756/0 (socket says 172.21.15.80:54756) 2024-04-24T23:26:40.298 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.294+0000 7f46bdd82640 1 -- 172.21.15.80:0/1985786485 learned_addr learned my addr 172.21.15.80:0/1985786485 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:26:40.298 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.294+0000 7f46bed84640 1 --2- 172.21.15.80:0/1985786485 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46b8107460 0x7f46b81a1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:40.298 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.294+0000 7f46be583640 1 --2- 172.21.15.80:0/1985786485 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f46b8102200 0x7f46b8199d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:40.298 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.295+0000 7f46bdd82640 1 -- 172.21.15.80:0/1985786485 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46b8107460 msgr2=0x7f46b81a1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:40.298 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.295+0000 7f46bdd82640 1 --2- 172.21.15.80:0/1985786485 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46b8107460 0x7f46b81a1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:40.299 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.295+0000 7f46bdd82640 1 -- 172.21.15.80:0/1985786485 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f46b8102200 msgr2=0x7f46b8199d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:40.299 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.295+0000 7f46bdd82640 1 --2- 172.21.15.80:0/1985786485 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f46b8102200 0x7f46b8199d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:40.299 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.295+0000 7f46bdd82640 1 -- 172.21.15.80:0/1985786485 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46b81a1910 con 0x7f46b8104b30 2024-04-24T23:26:40.299 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.295+0000 7f46be583640 1 --2- 172.21.15.80:0/1985786485 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f46b8102200 0x7f46b8199d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:26:40.299 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.295+0000 7f46bdd82640 1 --2- 172.21.15.80:0/1985786485 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f46b8104b30 0x7f46b819a240 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f46a8031610 tx=0x7f46a80043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:40.299 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.295+0000 7f46af7fe640 1 -- 172.21.15.80:0/1985786485 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a803f030 con 0x7f46b8104b30 2024-04-24T23:26:40.299 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.295+0000 7f46c07e7640 1 -- 172.21.15.80:0/1985786485 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46b81a1ba0 con 0x7f46b8104b30 2024-04-24T23:26:40.299 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.295+0000 7f46af7fe640 1 -- 172.21.15.80:0/1985786485 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46a8003ce0 con 0x7f46b8104b30 2024-04-24T23:26:40.299 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.295+0000 7f46af7fe640 1 -- 172.21.15.80:0/1985786485 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a8035070 con 0x7f46b8104b30 2024-04-24T23:26:40.299 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.295+0000 7f46c07e7640 1 -- 172.21.15.80:0/1985786485 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46b81a20e0 con 0x7f46b8104b30 2024-04-24T23:26:40.299 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.296+0000 7f46c07e7640 1 -- 172.21.15.80:0/1985786485 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46b8108fb0 con 0x7f46b8104b30 2024-04-24T23:26:40.300 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.297+0000 7f46af7fe640 1 -- 172.21.15.80:0/1985786485 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f46a8042030 con 0x7f46b8104b30 2024-04-24T23:26:40.300 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.298+0000 7f46af7fe640 1 --2- 172.21.15.80:0/1985786485 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f46a40a2b90 0x7f46a40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:40.300 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.298+0000 7f46be583640 1 --2- 172.21.15.80:0/1985786485 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f46a40a2b90 0x7f46a40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:40.300 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.298+0000 7f46af7fe640 1 -- 172.21.15.80:0/1985786485 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f46a80ef410 con 0x7f46b8104b30 2024-04-24T23:26:40.300 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.298+0000 7f46be583640 1 --2- 172.21.15.80:0/1985786485 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f46a40a2b90 0x7f46a40a5050 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f46b819b1d0 tx=0x7f46b4009400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:40.302 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.301+0000 7f46af7fe640 1 -- 172.21.15.80:0/1985786485 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f46a80bc810 con 0x7f46b8104b30 2024-04-24T23:26:40.477 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.475+0000 7f46c07e7640 1 -- 172.21.15.80:0/1985786485 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f46b802d340 con 0x7f46b8104b30 2024-04-24T23:26:40.477 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.476+0000 7f46af7fe640 1 -- 172.21.15.80:0/1985786485 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7f46a80bc110 con 0x7f46b8104b30 2024-04-24T23:26:40.477 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 1 2024-04-24T23:26:40.479 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.478+0000 7f46c07e7640 1 -- 172.21.15.80:0/1985786485 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f46a40a2b90 msgr2=0x7f46a40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:40.479 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.478+0000 7f46c07e7640 1 --2- 172.21.15.80:0/1985786485 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f46a40a2b90 0x7f46a40a5050 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f46b819b1d0 tx=0x7f46b4009400 comp rx=0 tx=0).stop 2024-04-24T23:26:40.479 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.478+0000 7f46c07e7640 1 -- 172.21.15.80:0/1985786485 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f46b8104b30 msgr2=0x7f46b819a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:40.479 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.478+0000 7f46c07e7640 1 --2- 172.21.15.80:0/1985786485 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f46b8104b30 0x7f46b819a240 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f46a8031610 tx=0x7f46a80043b0 comp rx=0 tx=0).stop 2024-04-24T23:26:40.479 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.478+0000 7f46c07e7640 1 -- 172.21.15.80:0/1985786485 shutdown_connections 2024-04-24T23:26:40.479 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.478+0000 7f46c07e7640 1 --2- 172.21.15.80:0/1985786485 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f46a40a2b90 0x7f46a40a5050 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:40.480 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.478+0000 7f46c07e7640 1 --2- 172.21.15.80:0/1985786485 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46b8107460 0x7f46b81a1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:40.480 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.478+0000 7f46c07e7640 1 --2- 172.21.15.80:0/1985786485 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f46b8104b30 0x7f46b819a240 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:40.480 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.478+0000 7f46c07e7640 1 --2- 172.21.15.80:0/1985786485 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f46b8102200 0x7f46b8199d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:40.480 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.478+0000 7f46c07e7640 1 -- 172.21.15.80:0/1985786485 >> 172.21.15.80:0/1985786485 conn(0x7f46b80fbbf0 msgr2=0x7f46b81018b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:40.480 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.478+0000 7f46c07e7640 1 -- 172.21.15.80:0/1985786485 shutdown_connections 2024-04-24T23:26:40.480 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.478+0000 7f46c07e7640 1 -- 172.21.15.80:0/1985786485 wait complete. 2024-04-24T23:26:40.490 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:26:40.591 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.587+0000 7f0888d1b640 1 -- 172.21.15.80:0/3878696024 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0884101c30 msgr2=0x7f0884102050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:40.591 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.587+0000 7f0888d1b640 1 --2- 172.21.15.80:0/3878696024 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0884101c30 0x7f0884102050 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0874009900 tx=0x7f087402a410 comp rx=0 tx=0).stop 2024-04-24T23:26:40.591 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.587+0000 7f0888d1b640 1 -- 172.21.15.80:0/3878696024 shutdown_connections 2024-04-24T23:26:40.591 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.587+0000 7f0888d1b640 1 --2- 172.21.15.80:0/3878696024 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0884101320 0x7f0884108bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:40.591 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.587+0000 7f0888d1b640 1 --2- 172.21.15.80:0/3878696024 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0884100960 0x7f0884100de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:40.591 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.587+0000 7f0888d1b640 1 --2- 172.21.15.80:0/3878696024 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0884101c30 0x7f0884102050 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:40.591 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.587+0000 7f0888d1b640 1 -- 172.21.15.80:0/3878696024 >> 172.21.15.80:0/3878696024 conn(0x7f08840fb3b0 msgr2=0x7f08840fd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:40.591 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.587+0000 7f0888d1b640 1 -- 172.21.15.80:0/3878696024 shutdown_connections 2024-04-24T23:26:40.592 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.587+0000 7f0888d1b640 1 -- 172.21.15.80:0/3878696024 wait complete. 2024-04-24T23:26:40.592 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.588+0000 7f0888d1b640 1 Processor -- start 2024-04-24T23:26:40.592 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.588+0000 7f0888d1b640 1 -- start start 2024-04-24T23:26:40.592 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.588+0000 7f0888d1b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0884100960 0x7f08841003e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:40.592 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.588+0000 7f0888d1b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0884101320 0x7f088419c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:40.592 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.588+0000 7f0888d1b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0884101c30 0x7f088419ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:40.592 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.588+0000 7f0888d1b640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f088410c330 con 0x7f0884101c30 2024-04-24T23:26:40.592 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.588+0000 7f0888d1b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f088410c1b0 con 0x7f0884101320 2024-04-24T23:26:40.592 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.588+0000 7f0888d1b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f088410c4b0 con 0x7f0884100960 2024-04-24T23:26:40.592 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.588+0000 7f0882d9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0884101c30 0x7f088419ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:40.592 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.588+0000 7f0882d9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0884101c30 0x7f088419ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:54768/0 (socket says 172.21.15.80:54768) 2024-04-24T23:26:40.592 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.588+0000 7f088259c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0884100960 0x7f08841003e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:40.592 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.588+0000 7f0882d9d640 1 -- 172.21.15.80:0/3386655802 learned_addr learned my addr 172.21.15.80:0/3386655802 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:26:40.592 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.589+0000 7f0881d9b640 1 --2- 172.21.15.80:0/3386655802 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0884101320 0x7f088419c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:40.592 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.589+0000 7f0882d9d640 1 -- 172.21.15.80:0/3386655802 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0884100960 msgr2=0x7f08841003e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:40.593 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.589+0000 7f0882d9d640 1 --2- 172.21.15.80:0/3386655802 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0884100960 0x7f08841003e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:40.593 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.589+0000 7f0882d9d640 1 -- 172.21.15.80:0/3386655802 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0884101320 msgr2=0x7f088419c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:40.593 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.589+0000 7f0882d9d640 1 --2- 172.21.15.80:0/3386655802 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0884101320 0x7f088419c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:40.593 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.589+0000 7f0882d9d640 1 -- 172.21.15.80:0/3386655802 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f088419f6a0 con 0x7f0884101c30 2024-04-24T23:26:40.593 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.589+0000 7f088259c640 1 --2- 172.21.15.80:0/3386655802 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0884100960 0x7f08841003e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:26:40.593 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.589+0000 7f0881d9b640 1 --2- 172.21.15.80:0/3386655802 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0884101320 0x7f088419c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:26:40.593 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.589+0000 7f0882d9d640 1 --2- 172.21.15.80:0/3386655802 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0884101c30 0x7f088419ede0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f087c00eb10 tx=0x7f087c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:40.593 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.589+0000 7f08737fe640 1 -- 172.21.15.80:0/3386655802 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f087c004b20 con 0x7f0884101c30 2024-04-24T23:26:40.593 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.589+0000 7f0888d1b640 1 -- 172.21.15.80:0/3386655802 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f088419f990 con 0x7f0884101c30 2024-04-24T23:26:40.593 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.589+0000 7f08737fe640 1 -- 172.21.15.80:0/3386655802 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f087c004cc0 con 0x7f0884101c30 2024-04-24T23:26:40.593 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.589+0000 7f08737fe640 1 -- 172.21.15.80:0/3386655802 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f087c010760 con 0x7f0884101c30 2024-04-24T23:26:40.593 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.589+0000 7f0888d1b640 1 -- 172.21.15.80:0/3386655802 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f088419fee0 con 0x7f0884101c30 2024-04-24T23:26:40.593 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.590+0000 7f0888d1b640 1 -- 172.21.15.80:0/3386655802 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0884107a40 con 0x7f0884101c30 2024-04-24T23:26:40.594 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.591+0000 7f08737fe640 1 -- 172.21.15.80:0/3386655802 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f087c005020 con 0x7f0884101c30 2024-04-24T23:26:40.594 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.592+0000 7f08737fe640 1 --2- 172.21.15.80:0/3386655802 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f086c0a2b90 0x7f086c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:40.594 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.592+0000 7f088259c640 1 --2- 172.21.15.80:0/3386655802 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f086c0a2b90 0x7f086c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:40.594 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.592+0000 7f08737fe640 1 -- 172.21.15.80:0/3386655802 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f087c0cf2f0 con 0x7f0884101c30 2024-04-24T23:26:40.594 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.592+0000 7f088259c640 1 --2- 172.21.15.80:0/3386655802 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f086c0a2b90 0x7f086c0a5050 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0874005ab0 tx=0x7f0874000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:40.598 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.596+0000 7f08737fe640 1 -- 172.21.15.80:0/3386655802 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f087c09c6f0 con 0x7f0884101c30 2024-04-24T23:26:40.809 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.808+0000 7f0888d1b640 1 -- 172.21.15.80:0/3386655802 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f088402d170 con 0x7f0884101c30 2024-04-24T23:26:40.809 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.808+0000 7f08737fe640 1 -- 172.21.15.80:0/3386655802 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+569 (secure 0 0 0) 0x7f087c09bff0 con 0x7f0884101c30 2024-04-24T23:26:40.810 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-24T23:26:40.810 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-04-24T23:26:40.810 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 1 2024-04-24T23:26:40.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.810+0000 7f0888d1b640 1 -- 172.21.15.80:0/3386655802 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f086c0a2b90 msgr2=0x7f086c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:40.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.810+0000 7f0888d1b640 1 --2- 172.21.15.80:0/3386655802 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f086c0a2b90 0x7f086c0a5050 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0874005ab0 tx=0x7f0874000bc0 comp rx=0 tx=0).stop 2024-04-24T23:26:40.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.810+0000 7f0888d1b640 1 -- 172.21.15.80:0/3386655802 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0884101c30 msgr2=0x7f088419ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:40.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.810+0000 7f0888d1b640 1 --2- 172.21.15.80:0/3386655802 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0884101c30 0x7f088419ede0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f087c00eb10 tx=0x7f087c00c2d0 comp rx=0 tx=0).stop 2024-04-24T23:26:40.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.810+0000 7f0888d1b640 1 -- 172.21.15.80:0/3386655802 shutdown_connections 2024-04-24T23:26:40.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.810+0000 7f0888d1b640 1 --2- 172.21.15.80:0/3386655802 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f086c0a2b90 0x7f086c0a5050 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:40.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.810+0000 7f0888d1b640 1 --2- 172.21.15.80:0/3386655802 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0884101c30 0x7f088419ede0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:40.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.810+0000 7f0888d1b640 1 --2- 172.21.15.80:0/3386655802 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0884101320 0x7f088419c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:40.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.810+0000 7f0888d1b640 1 --2- 172.21.15.80:0/3386655802 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0884100960 0x7f08841003e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:40.813 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.810+0000 7f0888d1b640 1 -- 172.21.15.80:0/3386655802 >> 172.21.15.80:0/3386655802 conn(0x7f08840fb3b0 msgr2=0x7f0884104c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:40.813 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.810+0000 7f0888d1b640 1 -- 172.21.15.80:0/3386655802 shutdown_connections 2024-04-24T23:26:40.813 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:40.811+0000 7f0888d1b640 1 -- 172.21.15.80:0/3386655802 wait complete. 2024-04-24T23:26:41.006 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:40 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/2236524362' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:26:41.006 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:40 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/1985786485' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:26:41.038 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:40 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/2236524362' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:26:41.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:40 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/1985786485' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:26:41.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:40 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/2236524362' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:26:41.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:40 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/1985786485' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:26:41.438 INFO:journalctl@ceph.mds.l.smithi148.stdout:Apr 24 23:26:41 smithi148 systemd[1]: Starting Ceph mds.l for ec7885aa-0290-11ef-bc93-c7b262605968... 2024-04-24T23:26:41.701 INFO:journalctl@ceph.mds.l.smithi148.stdout:Apr 24 23:26:41 smithi148 podman[62764]: 2024-04-24 23:26:41.340108663 +0000 UTC m=+0.018212126 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:26:41.702 INFO:journalctl@ceph.mds.l.smithi148.stdout:Apr 24 23:26:41 smithi148 podman[62764]: 2024-04-24 23:26:41.482087772 +0000 UTC m=+0.160191207 container create 8b2dcc3e6dc8b84363cd18d1443305f1a00700f0361d77019c08d73d1e8eb4f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-l, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS) 2024-04-24T23:26:41.702 INFO:journalctl@ceph.mds.l.smithi148.stdout:Apr 24 23:26:41 smithi148 podman[62764]: 2024-04-24 23:26:41.701782026 +0000 UTC m=+0.379885463 container init 8b2dcc3e6dc8b84363cd18d1443305f1a00700f0361d77019c08d73d1e8eb4f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-l, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, ceph=True) 2024-04-24T23:26:42.006 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:41 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/3386655802' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:26:42.006 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:41 smithi148 ceph-mon[32528]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:26:42.006 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:41 smithi148 ceph-mon[32528]: mds.? [v2:172.21.15.148:6832/124777462,v1:172.21.15.148:6833/124777462] up:boot 2024-04-24T23:26:42.007 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:41 smithi148 ceph-mon[32528]: fsmap 1 up:standby 2024-04-24T23:26:42.007 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:41 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-24T23:26:42.007 INFO:journalctl@ceph.mds.l.smithi148.stdout:Apr 24 23:26:41 smithi148 podman[62764]: 2024-04-24 23:26:41.704530068 +0000 UTC m=+0.382633501 container start 8b2dcc3e6dc8b84363cd18d1443305f1a00700f0361d77019c08d73d1e8eb4f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-l, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T23:26:42.007 INFO:journalctl@ceph.mds.l.smithi148.stdout:Apr 24 23:26:41 smithi148 ceph-mds[62783]: -- 172.21.15.148:0/3299267545 <== mon.1 v2:172.21.15.148:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e2fe421a40 con 0x55e2fe3f4800 2024-04-24T23:26:42.007 INFO:journalctl@ceph.mds.l.smithi148.stdout:Apr 24 23:26:41 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-l[62779]: starting mds.l at 2024-04-24T23:26:42.007 INFO:journalctl@ceph.mds.l.smithi148.stdout:Apr 24 23:26:41 smithi148 bash[62764]: 8b2dcc3e6dc8b84363cd18d1443305f1a00700f0361d77019c08d73d1e8eb4f8 2024-04-24T23:26:42.007 INFO:journalctl@ceph.mds.l.smithi148.stdout:Apr 24 23:26:41 smithi148 systemd[1]: Started Ceph mds.l for ec7885aa-0290-11ef-bc93-c7b262605968. 2024-04-24T23:26:42.038 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:41 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/3386655802' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:26:42.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:41 smithi080 ceph-mon[29152]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:26:42.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:41 smithi080 ceph-mon[29152]: mds.? [v2:172.21.15.148:6832/124777462,v1:172.21.15.148:6833/124777462] up:boot 2024-04-24T23:26:42.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:41 smithi080 ceph-mon[29152]: fsmap 1 up:standby 2024-04-24T23:26:42.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:41 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-24T23:26:42.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:41 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/3386655802' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:26:42.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:41 smithi134 ceph-mon[32619]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:26:42.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:41 smithi134 ceph-mon[32619]: mds.? [v2:172.21.15.148:6832/124777462,v1:172.21.15.148:6833/124777462] up:boot 2024-04-24T23:26:42.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:41 smithi134 ceph-mon[32619]: fsmap 1 up:standby 2024-04-24T23:26:42.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:41 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-24T23:26:42.824 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:26:42.942 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:42.938+0000 7fbd172bd640 1 -- 172.21.15.80:0/3435364953 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbd100741a0 msgr2=0x7fbd10074620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:42.942 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:42.938+0000 7fbd172bd640 1 --2- 172.21.15.80:0/3435364953 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbd100741a0 0x7fbd10074620 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fbd00009900 tx=0x7fbd0002a410 comp rx=0 tx=0).stop 2024-04-24T23:26:42.942 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:42.939+0000 7fbd172bd640 1 -- 172.21.15.80:0/3435364953 shutdown_connections 2024-04-24T23:26:42.943 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:42.939+0000 7fbd172bd640 1 --2- 172.21.15.80:0/3435364953 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbd10109650 0x7fbd1010bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:42.943 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:42.939+0000 7fbd172bd640 1 --2- 172.21.15.80:0/3435364953 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbd100741a0 0x7fbd10074620 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:42.943 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:42.939+0000 7fbd172bd640 1 --2- 172.21.15.80:0/3435364953 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbd10075780 0x7fbd10073bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:42.943 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:42.939+0000 7fbd172bd640 1 -- 172.21.15.80:0/3435364953 >> 172.21.15.80:0/3435364953 conn(0x7fbd100fcd20 msgr2=0x7fbd100ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:42.943 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:42.939+0000 7fbd172bd640 1 -- 172.21.15.80:0/3435364953 shutdown_connections 2024-04-24T23:26:42.943 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:42.939+0000 7fbd172bd640 1 -- 172.21.15.80:0/3435364953 wait complete. 2024-04-24T23:26:42.943 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:42.939+0000 7fbd172bd640 1 Processor -- start 2024-04-24T23:26:42.943 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:42.939+0000 7fbd172bd640 1 -- start start 2024-04-24T23:26:42.943 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:42.939+0000 7fbd172bd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbd100741a0 0x7fbd10199cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:42.943 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:42.939+0000 7fbd172bd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbd10075780 0x7fbd1019a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:42.943 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:42.939+0000 7fbd172bd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbd10109650 0x7fbd101a1260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:42.943 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:42.939+0000 7fbd172bd640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fbd1010f5d0 con 0x7fbd10109650 2024-04-24T23:26:42.943 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:42.939+0000 7fbd172bd640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbd1010f450 con 0x7fbd10075780 2024-04-24T23:26:42.944 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:42.939+0000 7fbd172bd640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fbd1010f750 con 0x7fbd100741a0 2024-04-24T23:26:42.944 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:42.940+0000 7fbd1585a640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbd10109650 0x7fbd101a1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:42.944 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:42.940+0000 7fbd1585a640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbd10109650 0x7fbd101a1260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:33028/0 (socket says 172.21.15.80:33028) 2024-04-24T23:26:42.944 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:42.940+0000 7fbd1585a640 1 -- 172.21.15.80:0/3512119854 learned_addr learned my addr 172.21.15.80:0/3512119854 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:26:42.944 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:42.940+0000 7fbd14858640 1 --2- 172.21.15.80:0/3512119854 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbd10075780 0x7fbd1019a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:42.944 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:42.940+0000 7fbd15059640 1 --2- 172.21.15.80:0/3512119854 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbd100741a0 0x7fbd10199cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:42.944 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:42.940+0000 7fbd1585a640 1 -- 172.21.15.80:0/3512119854 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbd100741a0 msgr2=0x7fbd10199cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:42.944 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:42.940+0000 7fbd1585a640 1 --2- 172.21.15.80:0/3512119854 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbd100741a0 0x7fbd10199cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:42.944 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:42.940+0000 7fbd1585a640 1 -- 172.21.15.80:0/3512119854 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbd10075780 msgr2=0x7fbd1019a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:42.944 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:42.940+0000 7fbd1585a640 1 --2- 172.21.15.80:0/3512119854 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbd10075780 0x7fbd1019a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:42.944 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:42.940+0000 7fbd1585a640 1 -- 172.21.15.80:0/3512119854 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd101a1900 con 0x7fbd10109650 2024-04-24T23:26:42.944 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:42.940+0000 7fbd15059640 1 --2- 172.21.15.80:0/3512119854 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbd100741a0 0x7fbd10199cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:26:42.944 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:42.940+0000 7fbd1585a640 1 --2- 172.21.15.80:0/3512119854 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbd10109650 0x7fbd101a1260 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fbd0800eaa0 tx=0x7fbd0800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:42.945 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:42.940+0000 7fbd067fc640 1 -- 172.21.15.80:0/3512119854 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd0800cdb0 con 0x7fbd10109650 2024-04-24T23:26:42.945 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:42.940+0000 7fbd172bd640 1 -- 172.21.15.80:0/3512119854 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd101a1bf0 con 0x7fbd10109650 2024-04-24T23:26:42.945 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:42.940+0000 7fbd067fc640 1 -- 172.21.15.80:0/3512119854 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd08005020 con 0x7fbd10109650 2024-04-24T23:26:42.945 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:42.940+0000 7fbd067fc640 1 -- 172.21.15.80:0/3512119854 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd080116b0 con 0x7fbd10109650 2024-04-24T23:26:42.945 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:42.940+0000 7fbd172bd640 1 -- 172.21.15.80:0/3512119854 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd101a20c0 con 0x7fbd10109650 2024-04-24T23:26:42.945 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:42.941+0000 7fbd172bd640 1 -- 172.21.15.80:0/3512119854 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd10108380 con 0x7fbd10109650 2024-04-24T23:26:42.945 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:42.942+0000 7fbd067fc640 1 -- 172.21.15.80:0/3512119854 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fbd08005560 con 0x7fbd10109650 2024-04-24T23:26:42.946 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:42.943+0000 7fbd067fc640 1 --2- 172.21.15.80:0/3512119854 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fbcfc0a2b90 0x7fbcfc0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:42.946 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:42.943+0000 7fbd067fc640 1 -- 172.21.15.80:0/3512119854 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fbd080cedb0 con 0x7fbd10109650 2024-04-24T23:26:42.946 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:42.943+0000 7fbd15059640 1 --2- 172.21.15.80:0/3512119854 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fbcfc0a2b90 0x7fbcfc0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:42.946 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:42.943+0000 7fbd15059640 1 --2- 172.21.15.80:0/3512119854 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fbcfc0a2b90 0x7fbcfc0a5050 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fbd0c005d00 tx=0x7fbd0c005c90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:42.948 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:42.947+0000 7fbd067fc640 1 -- 172.21.15.80:0/3512119854 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fbd08016030 con 0x7fbd10109650 2024-04-24T23:26:43.162 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:43.161+0000 7fbd172bd640 1 -- 172.21.15.80:0/3512119854 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbd1002d320 con 0x7fbd10109650 2024-04-24T23:26:43.163 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:43.162+0000 7fbd067fc640 1 -- 172.21.15.80:0/3512119854 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1393 (secure 0 0 0) 0x7fbd0809c1b0 con 0x7fbd10109650 2024-04-24T23:26:43.163 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-24T23:26:43.163 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/124777462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":124777462},{"type":"v1","addr":"172.21.15.148:6833","nonce":124777462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-24T23:26:43.163 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 2 2024-04-24T23:26:43.165 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:43.164+0000 7fbd172bd640 1 -- 172.21.15.80:0/3512119854 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fbcfc0a2b90 msgr2=0x7fbcfc0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:43.165 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:43.164+0000 7fbd172bd640 1 --2- 172.21.15.80:0/3512119854 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fbcfc0a2b90 0x7fbcfc0a5050 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fbd0c005d00 tx=0x7fbd0c005c90 comp rx=0 tx=0).stop 2024-04-24T23:26:43.165 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:43.164+0000 7fbd172bd640 1 -- 172.21.15.80:0/3512119854 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbd10109650 msgr2=0x7fbd101a1260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:43.165 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:43.164+0000 7fbd172bd640 1 --2- 172.21.15.80:0/3512119854 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbd10109650 0x7fbd101a1260 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fbd0800eaa0 tx=0x7fbd0800ef60 comp rx=0 tx=0).stop 2024-04-24T23:26:43.165 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:43.164+0000 7fbd172bd640 1 -- 172.21.15.80:0/3512119854 shutdown_connections 2024-04-24T23:26:43.165 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:43.164+0000 7fbd172bd640 1 --2- 172.21.15.80:0/3512119854 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fbcfc0a2b90 0x7fbcfc0a5050 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:43.165 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:43.164+0000 7fbd172bd640 1 --2- 172.21.15.80:0/3512119854 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbd10109650 0x7fbd101a1260 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:43.166 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:43.164+0000 7fbd172bd640 1 --2- 172.21.15.80:0/3512119854 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbd10075780 0x7fbd1019a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:43.166 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:43.164+0000 7fbd172bd640 1 --2- 172.21.15.80:0/3512119854 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbd100741a0 0x7fbd10199cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:43.166 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:43.164+0000 7fbd172bd640 1 -- 172.21.15.80:0/3512119854 >> 172.21.15.80:0/3512119854 conn(0x7fbd100fcd20 msgr2=0x7fbd10105550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:43.166 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:43.164+0000 7fbd172bd640 1 -- 172.21.15.80:0/3512119854 shutdown_connections 2024-04-24T23:26:43.166 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:43.164+0000 7fbd172bd640 1 -- 172.21.15.80:0/3512119854 wait complete. 2024-04-24T23:26:43.175 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:42 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:26:43.175 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:42 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:26:43.175 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:42 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:26:43.176 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:42 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:26:43.176 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:42 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:26:43.176 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:42 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:26:43.176 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:42 smithi080 ceph-mon[29152]: Deploying daemon mds.k on smithi134 2024-04-24T23:26:43.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:42 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:26:43.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:42 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:26:43.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:42 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:26:43.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:42 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:26:43.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:42 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:26:43.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:42 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:26:43.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:42 smithi134 ceph-mon[32619]: Deploying daemon mds.k on smithi134 2024-04-24T23:26:43.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:42 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:26:43.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:42 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:26:43.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:42 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:26:43.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:42 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:26:43.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:42 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:26:43.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:42 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:26:43.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:42 smithi148 ceph-mon[32528]: Deploying daemon mds.k on smithi134 2024-04-24T23:26:43.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:43 smithi134 ceph-mon[32619]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:26:43.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:43 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/3512119854' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:26:44.197 INFO:journalctl@ceph.mds.k.smithi134.stdout:Apr 24 23:26:44 smithi134 systemd[1]: Starting Ceph mds.k for ec7885aa-0290-11ef-bc93-c7b262605968... 2024-04-24T23:26:44.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:43 smithi148 ceph-mon[32528]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:26:44.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:43 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/3512119854' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:26:44.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:43 smithi080 ceph-mon[29152]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:26:44.319 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:43 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/3512119854' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:26:44.610 INFO:journalctl@ceph.mds.k.smithi134.stdout:Apr 24 23:26:44 smithi134 podman[62449]: 2024-04-24 23:26:44.236105585 +0000 UTC m=+0.017703419 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:26:44.610 INFO:journalctl@ceph.mds.k.smithi134.stdout:Apr 24 23:26:44 smithi134 podman[62449]: 2024-04-24 23:26:44.356866984 +0000 UTC m=+0.138464796 container create def966a79765fee2677b3704200bdb30c8850c6cbdeca97a4f7b692d6609b915 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-k, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True) 2024-04-24T23:26:44.610 INFO:journalctl@ceph.mds.k.smithi134.stdout:Apr 24 23:26:44 smithi134 podman[62449]: 2024-04-24 23:26:44.60973963 +0000 UTC m=+0.391337445 container init def966a79765fee2677b3704200bdb30c8850c6cbdeca97a4f7b692d6609b915 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-k, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424) 2024-04-24T23:26:44.947 INFO:journalctl@ceph.mds.k.smithi134.stdout:Apr 24 23:26:44 smithi134 podman[62449]: 2024-04-24 23:26:44.612755675 +0000 UTC m=+0.394353496 container start def966a79765fee2677b3704200bdb30c8850c6cbdeca97a4f7b692d6609b915 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-k, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, ceph=True) 2024-04-24T23:26:44.947 INFO:journalctl@ceph.mds.k.smithi134.stdout:Apr 24 23:26:44 smithi134 ceph-mds[62468]: -- 172.21.15.134:0/939120231 <== mon.2 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556d72cf5a40 con 0x556d72cc8800 2024-04-24T23:26:44.948 INFO:journalctl@ceph.mds.k.smithi134.stdout:Apr 24 23:26:44 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-k[62463]: starting mds.k at 2024-04-24T23:26:44.948 INFO:journalctl@ceph.mds.k.smithi134.stdout:Apr 24 23:26:44 smithi134 bash[62449]: def966a79765fee2677b3704200bdb30c8850c6cbdeca97a4f7b692d6609b915 2024-04-24T23:26:44.948 INFO:journalctl@ceph.mds.k.smithi134.stdout:Apr 24 23:26:44 smithi134 systemd[1]: Started Ceph mds.k for ec7885aa-0290-11ef-bc93-c7b262605968. 2024-04-24T23:26:45.178 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:26:45.283 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:45.282+0000 7fd24c44c640 1 -- 172.21.15.80:0/2075321034 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd244101870 msgr2=0x7fd244101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:45.283 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:45.282+0000 7fd24c44c640 1 --2- 172.21.15.80:0/2075321034 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd244101870 0x7fd244101cf0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd234009900 tx=0x7fd23402a410 comp rx=0 tx=0).stop 2024-04-24T23:26:45.283 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:45.282+0000 7fd24c44c640 1 -- 172.21.15.80:0/2075321034 shutdown_connections 2024-04-24T23:26:45.284 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:45.282+0000 7fd24c44c640 1 --2- 172.21.15.80:0/2075321034 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd244102230 0x7fd244108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:45.284 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:45.282+0000 7fd24c44c640 1 --2- 172.21.15.80:0/2075321034 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd244101870 0x7fd244101cf0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:45.284 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:45.282+0000 7fd24c44c640 1 --2- 172.21.15.80:0/2075321034 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd244100680 0x7fd244100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:45.284 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:45.282+0000 7fd24c44c640 1 -- 172.21.15.80:0/2075321034 >> 172.21.15.80:0/2075321034 conn(0x7fd2440fbbf0 msgr2=0x7fd2440fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:45.284 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:45.282+0000 7fd24c44c640 1 -- 172.21.15.80:0/2075321034 shutdown_connections 2024-04-24T23:26:45.284 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:45.282+0000 7fd24c44c640 1 -- 172.21.15.80:0/2075321034 wait complete. 2024-04-24T23:26:45.284 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:45.282+0000 7fd24c44c640 1 Processor -- start 2024-04-24T23:26:45.284 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:45.283+0000 7fd24c44c640 1 -- start start 2024-04-24T23:26:45.286 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:45.283+0000 7fd24c44c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd244100680 0x7fd244196960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:45.286 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:45.283+0000 7fd24c44c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd244101870 0x7fd244196ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:45.286 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:45.283+0000 7fd24c44c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd244102230 0x7fd24419df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:45.287 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:45.283+0000 7fd24c44c640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fd24410c350 con 0x7fd244100680 2024-04-24T23:26:45.287 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:45.283+0000 7fd24c44c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd24410c1d0 con 0x7fd244102230 2024-04-24T23:26:45.287 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:45.283+0000 7fd24c44c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd24410c4d0 con 0x7fd244101870 2024-04-24T23:26:45.287 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:45.283+0000 7fd24a1e8640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd244100680 0x7fd244196960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:45.287 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:45.283+0000 7fd24a1e8640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd244100680 0x7fd244196960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:33046/0 (socket says 172.21.15.80:33046) 2024-04-24T23:26:45.287 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:45.283+0000 7fd24a1e8640 1 -- 172.21.15.80:0/3368983488 learned_addr learned my addr 172.21.15.80:0/3368983488 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:26:45.287 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:45.283+0000 7fd2499e7640 1 --2- 172.21.15.80:0/3368983488 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd244101870 0x7fd244196ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:45.287 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:45.283+0000 7fd24a1e8640 1 -- 172.21.15.80:0/3368983488 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd244101870 msgr2=0x7fd244196ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:45.287 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:45.283+0000 7fd24a1e8640 1 --2- 172.21.15.80:0/3368983488 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd244101870 0x7fd244196ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:45.287 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:45.283+0000 7fd24a1e8640 1 -- 172.21.15.80:0/3368983488 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd244102230 msgr2=0x7fd24419df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:45.287 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:45.283+0000 7fd24a9e9640 1 --2- 172.21.15.80:0/3368983488 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd244102230 0x7fd24419df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:45.287 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:45.283+0000 7fd24a1e8640 1 --2- 172.21.15.80:0/3368983488 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd244102230 0x7fd24419df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:45.288 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:45.283+0000 7fd24a1e8640 1 -- 172.21.15.80:0/3368983488 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd24419e580 con 0x7fd244100680 2024-04-24T23:26:45.288 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:45.283+0000 7fd2499e7640 1 --2- 172.21.15.80:0/3368983488 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd244101870 0x7fd244196ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:26:45.288 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:45.283+0000 7fd24a1e8640 1 --2- 172.21.15.80:0/3368983488 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd244100680 0x7fd244196960 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd24000c970 tx=0x7fd24000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:45.288 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:45.283+0000 7fd23b7fe640 1 -- 172.21.15.80:0/3368983488 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd240004220 con 0x7fd244100680 2024-04-24T23:26:45.288 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:45.284+0000 7fd24c44c640 1 -- 172.21.15.80:0/3368983488 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd24419e870 con 0x7fd244100680 2024-04-24T23:26:45.288 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:45.284+0000 7fd23b7fe640 1 -- 172.21.15.80:0/3368983488 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd2400043c0 con 0x7fd244100680 2024-04-24T23:26:45.288 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:45.284+0000 7fd23b7fe640 1 -- 172.21.15.80:0/3368983488 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd240004a00 con 0x7fd244100680 2024-04-24T23:26:45.288 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:45.284+0000 7fd24c44c640 1 -- 172.21.15.80:0/3368983488 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd24419edc0 con 0x7fd244100680 2024-04-24T23:26:45.288 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:45.285+0000 7fd24c44c640 1 -- 172.21.15.80:0/3368983488 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2441079a0 con 0x7fd244100680 2024-04-24T23:26:45.288 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:45.286+0000 7fd23b7fe640 1 -- 172.21.15.80:0/3368983488 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fd2400079b0 con 0x7fd244100680 2024-04-24T23:26:45.289 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:45.286+0000 7fd23b7fe640 1 --2- 172.21.15.80:0/3368983488 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fd2300a2b90 0x7fd2300a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:45.289 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:45.287+0000 7fd2499e7640 1 --2- 172.21.15.80:0/3368983488 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fd2300a2b90 0x7fd2300a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:45.289 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:45.287+0000 7fd23b7fe640 1 -- 172.21.15.80:0/3368983488 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fd2400ce9f0 con 0x7fd244100680 2024-04-24T23:26:45.289 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:45.287+0000 7fd2499e7640 1 --2- 172.21.15.80:0/3368983488 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fd2300a2b90 0x7fd2300a5050 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd23402a930 tx=0x7fd234000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:45.292 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:45.291+0000 7fd23b7fe640 1 -- 172.21.15.80:0/3368983488 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fd240014030 con 0x7fd244100680 2024-04-24T23:26:45.522 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:45.521+0000 7fd24c44c640 1 -- 172.21.15.80:0/3368983488 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd24402d170 con 0x7fd244100680 2024-04-24T23:26:45.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:45.522+0000 7fd23b7fe640 1 -- 172.21.15.80:0/3368983488 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2221 (secure 0 0 0) 0x7fd24402d170 con 0x7fd244100680 2024-04-24T23:26:45.523 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-24T23:26:45.523 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24395,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/3736784385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3736784385},{"type":"v1","addr":"172.21.15.134:6833","nonce":3736784385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/124777462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":124777462},{"type":"v1","addr":"172.21.15.148:6833","nonce":124777462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-24T23:26:45.523 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 3 2024-04-24T23:26:45.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:45.524+0000 7fd24c44c640 1 -- 172.21.15.80:0/3368983488 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fd2300a2b90 msgr2=0x7fd2300a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:45.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:45.524+0000 7fd24c44c640 1 --2- 172.21.15.80:0/3368983488 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fd2300a2b90 0x7fd2300a5050 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd23402a930 tx=0x7fd234000bc0 comp rx=0 tx=0).stop 2024-04-24T23:26:45.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:45.524+0000 7fd24c44c640 1 -- 172.21.15.80:0/3368983488 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd244100680 msgr2=0x7fd244196960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:45.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:45.524+0000 7fd24c44c640 1 --2- 172.21.15.80:0/3368983488 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd244100680 0x7fd244196960 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd24000c970 tx=0x7fd24000ce30 comp rx=0 tx=0).stop 2024-04-24T23:26:45.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:45.524+0000 7fd24c44c640 1 -- 172.21.15.80:0/3368983488 shutdown_connections 2024-04-24T23:26:45.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:45.524+0000 7fd24c44c640 1 --2- 172.21.15.80:0/3368983488 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fd2300a2b90 0x7fd2300a5050 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:45.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:45.524+0000 7fd24c44c640 1 --2- 172.21.15.80:0/3368983488 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd244102230 0x7fd24419df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:45.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:45.524+0000 7fd24c44c640 1 --2- 172.21.15.80:0/3368983488 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd244101870 0x7fd244196ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:45.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:45.524+0000 7fd24c44c640 1 --2- 172.21.15.80:0/3368983488 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd244100680 0x7fd244196960 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:45.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:45.524+0000 7fd24c44c640 1 -- 172.21.15.80:0/3368983488 >> 172.21.15.80:0/3368983488 conn(0x7fd2440fbbf0 msgr2=0x7fd244104b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:45.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:45.524+0000 7fd24c44c640 1 -- 172.21.15.80:0/3368983488 shutdown_connections 2024-04-24T23:26:45.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:45.524+0000 7fd24c44c640 1 -- 172.21.15.80:0/3368983488 wait complete. 2024-04-24T23:26:46.038 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:45 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:26:46.038 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:45 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:26:46.038 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:45 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:26:46.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:45 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:26:46.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:45 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:26:46.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:45 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:26:46.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:45 smithi080 ceph-mon[29152]: Deploying daemon mds.j on smithi080 2024-04-24T23:26:46.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:45 smithi080 ceph-mon[29152]: mds.? [v2:172.21.15.134:6832/3736784385,v1:172.21.15.134:6833/3736784385] up:boot 2024-04-24T23:26:46.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:45 smithi080 ceph-mon[29152]: fsmap 2 up:standby 2024-04-24T23:26:46.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:45 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-24T23:26:46.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:45 smithi080 ceph-mon[29152]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:26:46.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:45 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/3368983488' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:26:46.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:45 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:26:46.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:45 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:26:46.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:45 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:26:46.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:45 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:26:46.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:45 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:26:46.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:45 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:26:46.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:45 smithi134 ceph-mon[32619]: Deploying daemon mds.j on smithi080 2024-04-24T23:26:46.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:45 smithi134 ceph-mon[32619]: mds.? [v2:172.21.15.134:6832/3736784385,v1:172.21.15.134:6833/3736784385] up:boot 2024-04-24T23:26:46.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:45 smithi134 ceph-mon[32619]: fsmap 2 up:standby 2024-04-24T23:26:46.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:45 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-24T23:26:46.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:45 smithi134 ceph-mon[32619]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:26:46.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:45 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/3368983488' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:26:46.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:45 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:26:46.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:45 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:26:46.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:45 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:26:46.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:45 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:26:46.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:45 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:26:46.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:45 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:26:46.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:45 smithi148 ceph-mon[32528]: Deploying daemon mds.j on smithi080 2024-04-24T23:26:46.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:45 smithi148 ceph-mon[32528]: mds.? [v2:172.21.15.134:6832/3736784385,v1:172.21.15.134:6833/3736784385] up:boot 2024-04-24T23:26:46.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:45 smithi148 ceph-mon[32528]: fsmap 2 up:standby 2024-04-24T23:26:46.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:45 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-24T23:26:46.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:45 smithi148 ceph-mon[32528]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:26:46.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:45 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/3368983488' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:26:47.038 INFO:journalctl@ceph.mds.j.smithi080.stdout:Apr 24 23:26:46 smithi080 systemd[1]: Starting Ceph mds.j for ec7885aa-0290-11ef-bc93-c7b262605968... 2024-04-24T23:26:47.419 INFO:journalctl@ceph.mds.j.smithi080.stdout:Apr 24 23:26:47 smithi080 podman[69015]: 2024-04-24 23:26:47.058934187 +0000 UTC m=+0.018234411 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:26:47.419 INFO:journalctl@ceph.mds.j.smithi080.stdout:Apr 24 23:26:47 smithi080 podman[69015]: 2024-04-24 23:26:47.184082401 +0000 UTC m=+0.143382666 container create 9bb75b8333b673a5689732ca07f816a933ca12c3a4ceb02eb77e2f665ed4725d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-j, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-04-24T23:26:47.419 INFO:journalctl@ceph.mds.j.smithi080.stdout:Apr 24 23:26:47 smithi080 podman[69015]: 2024-04-24 23:26:47.419182135 +0000 UTC m=+0.378482341 container init 9bb75b8333b673a5689732ca07f816a933ca12c3a4ceb02eb77e2f665ed4725d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-j, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-d2c45d9, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424) 2024-04-24T23:26:47.537 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:26:47.645 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:47.643+0000 7f185902a640 1 -- 172.21.15.80:0/10212840 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f18541009b0 msgr2=0x7f1854100e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:47.645 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:47.643+0000 7f185902a640 1 --2- 172.21.15.80:0/10212840 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f18541009b0 0x7f1854100e10 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f1848009900 tx=0x7f184802a410 comp rx=0 tx=0).stop 2024-04-24T23:26:47.645 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:47.643+0000 7f185902a640 1 -- 172.21.15.80:0/10212840 shutdown_connections 2024-04-24T23:26:47.645 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:47.643+0000 7f185902a640 1 --2- 172.21.15.80:0/10212840 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1854101350 0x7f18541017d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:47.645 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:47.643+0000 7f185902a640 1 --2- 172.21.15.80:0/10212840 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f18541009b0 0x7f1854100e10 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:47.646 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:47.643+0000 7f185902a640 1 --2- 172.21.15.80:0/10212840 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1854100050 0x7f1854100470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:47.646 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:47.644+0000 7f185902a640 1 -- 172.21.15.80:0/10212840 >> 172.21.15.80:0/10212840 conn(0x7f18540fbbf0 msgr2=0x7f18540fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:47.646 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:47.644+0000 7f185902a640 1 -- 172.21.15.80:0/10212840 shutdown_connections 2024-04-24T23:26:47.646 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:47.644+0000 7f185902a640 1 -- 172.21.15.80:0/10212840 wait complete. 2024-04-24T23:26:47.646 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:47.644+0000 7f185902a640 1 Processor -- start 2024-04-24T23:26:47.648 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:47.644+0000 7f185902a640 1 -- start start 2024-04-24T23:26:47.648 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:47.645+0000 7f185902a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1854100050 0x7f1854199cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:47.648 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:47.645+0000 7f185902a640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f18541009b0 0x7f185419a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:47.648 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:47.645+0000 7f185902a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1854101350 0x7f18541a1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:47.648 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:47.645+0000 7f185902a640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f185410f640 con 0x7f18541009b0 2024-04-24T23:26:47.648 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:47.645+0000 7f185902a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f185410f4c0 con 0x7f1854101350 2024-04-24T23:26:47.648 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:47.645+0000 7f185902a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f185410f7c0 con 0x7f1854100050 2024-04-24T23:26:47.648 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:47.645+0000 7f185259c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f18541009b0 0x7f185419a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:47.648 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:47.645+0000 7f185259c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f18541009b0 0x7f185419a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:33074/0 (socket says 172.21.15.80:33074) 2024-04-24T23:26:47.649 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:47.645+0000 7f185259c640 1 -- 172.21.15.80:0/2343367097 learned_addr learned my addr 172.21.15.80:0/2343367097 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:26:47.649 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:47.645+0000 7f185359e640 1 --2- 172.21.15.80:0/2343367097 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1854101350 0x7f18541a1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:47.649 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:47.645+0000 7f1852d9d640 1 --2- 172.21.15.80:0/2343367097 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1854100050 0x7f1854199cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:47.649 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:47.645+0000 7f185259c640 1 -- 172.21.15.80:0/2343367097 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1854100050 msgr2=0x7f1854199cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:47.649 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:47.645+0000 7f185259c640 1 --2- 172.21.15.80:0/2343367097 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1854100050 0x7f1854199cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:47.649 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:47.645+0000 7f185259c640 1 -- 172.21.15.80:0/2343367097 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1854101350 msgr2=0x7f18541a1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:47.649 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:47.645+0000 7f185259c640 1 --2- 172.21.15.80:0/2343367097 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1854101350 0x7f18541a1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:47.649 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:47.645+0000 7f185259c640 1 -- 172.21.15.80:0/2343367097 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18541a18f0 con 0x7f18541009b0 2024-04-24T23:26:47.649 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:47.645+0000 7f185259c640 1 --2- 172.21.15.80:0/2343367097 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f18541009b0 0x7f185419a1f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1848031440 tx=0x7f1848032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:47.650 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:47.645+0000 7f183bfff640 1 -- 172.21.15.80:0/2343367097 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f184803f030 con 0x7f18541009b0 2024-04-24T23:26:47.650 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:47.646+0000 7f185902a640 1 -- 172.21.15.80:0/2343367097 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18541a1b80 con 0x7f18541009b0 2024-04-24T23:26:47.650 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:47.646+0000 7f183bfff640 1 -- 172.21.15.80:0/2343367097 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1848003ce0 con 0x7f18541009b0 2024-04-24T23:26:47.650 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:47.646+0000 7f183bfff640 1 -- 172.21.15.80:0/2343367097 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1848035070 con 0x7f18541009b0 2024-04-24T23:26:47.650 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:47.646+0000 7f185902a640 1 -- 172.21.15.80:0/2343367097 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18541a20c0 con 0x7f18541009b0 2024-04-24T23:26:47.650 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:47.647+0000 7f185902a640 1 -- 172.21.15.80:0/2343367097 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f185410af40 con 0x7f18541009b0 2024-04-24T23:26:47.651 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:47.648+0000 7f183bfff640 1 -- 172.21.15.80:0/2343367097 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f1848042030 con 0x7f18541009b0 2024-04-24T23:26:47.651 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:47.648+0000 7f183bfff640 1 --2- 172.21.15.80:0/2343367097 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f183c0a2b90 0x7f183c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:47.651 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:47.648+0000 7f1852d9d640 1 --2- 172.21.15.80:0/2343367097 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f183c0a2b90 0x7f183c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:47.651 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:47.648+0000 7f183bfff640 1 -- 172.21.15.80:0/2343367097 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f18480ef660 con 0x7f18541009b0 2024-04-24T23:26:47.651 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:47.649+0000 7f1852d9d640 1 --2- 172.21.15.80:0/2343367097 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f183c0a2b90 0x7f183c0a5050 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f185419ab80 tx=0x7f1844005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:47.653 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:47.652+0000 7f183bfff640 1 -- 172.21.15.80:0/2343367097 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f18480bc9c0 con 0x7f18541009b0 2024-04-24T23:26:47.788 INFO:journalctl@ceph.mds.j.smithi080.stdout:Apr 24 23:26:47 smithi080 podman[69015]: 2024-04-24 23:26:47.421803814 +0000 UTC m=+0.381104024 container start 9bb75b8333b673a5689732ca07f816a933ca12c3a4ceb02eb77e2f665ed4725d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-j, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, ceph=True) 2024-04-24T23:26:47.788 INFO:journalctl@ceph.mds.j.smithi080.stdout:Apr 24 23:26:47 smithi080 ceph-mds[69034]: -- 172.21.15.80:0/1005277935 <== mon.0 v2:172.21.15.80:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562d0016da40 con 0x562d00141000 2024-04-24T23:26:47.788 INFO:journalctl@ceph.mds.j.smithi080.stdout:Apr 24 23:26:47 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-j[69030]: starting mds.j at 2024-04-24T23:26:47.788 INFO:journalctl@ceph.mds.j.smithi080.stdout:Apr 24 23:26:47 smithi080 bash[69015]: 9bb75b8333b673a5689732ca07f816a933ca12c3a4ceb02eb77e2f665ed4725d 2024-04-24T23:26:47.789 INFO:journalctl@ceph.mds.j.smithi080.stdout:Apr 24 23:26:47 smithi080 systemd[1]: Started Ceph mds.j for ec7885aa-0290-11ef-bc93-c7b262605968. 2024-04-24T23:26:47.875 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:47.874+0000 7f185902a640 1 -- 172.21.15.80:0/2343367097 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f185402d330 con 0x7f18541009b0 2024-04-24T23:26:47.876 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:47.875+0000 7f183bfff640 1 -- 172.21.15.80:0/2343367097 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2221 (secure 0 0 0) 0x7f18480bc2c0 con 0x7f18541009b0 2024-04-24T23:26:47.876 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-24T23:26:47.876 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24395,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/3736784385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3736784385},{"type":"v1","addr":"172.21.15.134:6833","nonce":3736784385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/124777462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":124777462},{"type":"v1","addr":"172.21.15.148:6833","nonce":124777462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-24T23:26:47.876 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 3 2024-04-24T23:26:47.878 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:47.877+0000 7f185902a640 1 -- 172.21.15.80:0/2343367097 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f183c0a2b90 msgr2=0x7f183c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:47.878 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:47.877+0000 7f185902a640 1 --2- 172.21.15.80:0/2343367097 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f183c0a2b90 0x7f183c0a5050 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f185419ab80 tx=0x7f1844005f90 comp rx=0 tx=0).stop 2024-04-24T23:26:47.878 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:47.877+0000 7f185902a640 1 -- 172.21.15.80:0/2343367097 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f18541009b0 msgr2=0x7f185419a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:47.878 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:47.877+0000 7f185902a640 1 --2- 172.21.15.80:0/2343367097 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f18541009b0 0x7f185419a1f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1848031440 tx=0x7f1848032ce0 comp rx=0 tx=0).stop 2024-04-24T23:26:47.879 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:47.877+0000 7f185902a640 1 -- 172.21.15.80:0/2343367097 shutdown_connections 2024-04-24T23:26:47.879 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:47.877+0000 7f185902a640 1 --2- 172.21.15.80:0/2343367097 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f183c0a2b90 0x7f183c0a5050 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:47.879 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:47.877+0000 7f185902a640 1 --2- 172.21.15.80:0/2343367097 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1854101350 0x7f18541a1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:47.879 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:47.877+0000 7f185902a640 1 --2- 172.21.15.80:0/2343367097 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f18541009b0 0x7f185419a1f0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:47.879 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:47.877+0000 7f185902a640 1 --2- 172.21.15.80:0/2343367097 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1854100050 0x7f1854199cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:47.879 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:47.877+0000 7f185902a640 1 -- 172.21.15.80:0/2343367097 >> 172.21.15.80:0/2343367097 conn(0x7f18540fbbf0 msgr2=0x7f18540fd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:47.879 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:47.877+0000 7f185902a640 1 -- 172.21.15.80:0/2343367097 shutdown_connections 2024-04-24T23:26:47.879 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:47.877+0000 7f185902a640 1 -- 172.21.15.80:0/2343367097 wait complete. 2024-04-24T23:26:48.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:47 smithi134 ceph-mon[32619]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:26:48.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:47 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:26:48.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:47 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:26:48.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:47 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:26:48.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:47 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:26:48.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:47 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:26:48.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:47 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:26:48.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:47 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/2343367097' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:26:48.236 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:47 smithi080 ceph-mon[29152]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:26:48.237 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:47 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:26:48.237 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:47 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:26:48.237 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:47 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:26:48.237 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:47 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:26:48.237 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:47 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:26:48.238 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:47 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:26:48.238 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:47 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/2343367097' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:26:48.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:47 smithi148 ceph-mon[32528]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:26:48.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:47 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:26:48.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:47 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:26:48.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:47 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:26:48.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:47 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:26:48.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:47 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:26:48.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:47 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:26:48.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:47 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/2343367097' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:26:49.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:48 smithi134 ceph-mon[32619]: Deploying daemon mds.d on smithi080 2024-04-24T23:26:49.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:48 smithi134 ceph-mon[32619]: mds.? [v2:172.21.15.80:6834/891591085,v1:172.21.15.80:6835/891591085] up:boot 2024-04-24T23:26:49.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:48 smithi134 ceph-mon[32619]: fsmap 3 up:standby 2024-04-24T23:26:49.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:48 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-24T23:26:49.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:48 smithi148 ceph-mon[32528]: Deploying daemon mds.d on smithi080 2024-04-24T23:26:49.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:48 smithi148 ceph-mon[32528]: mds.? [v2:172.21.15.80:6834/891591085,v1:172.21.15.80:6835/891591085] up:boot 2024-04-24T23:26:49.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:48 smithi148 ceph-mon[32528]: fsmap 3 up:standby 2024-04-24T23:26:49.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:48 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-24T23:26:49.285 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:48 smithi080 ceph-mon[29152]: Deploying daemon mds.d on smithi080 2024-04-24T23:26:49.285 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:48 smithi080 ceph-mon[29152]: mds.? [v2:172.21.15.80:6834/891591085,v1:172.21.15.80:6835/891591085] up:boot 2024-04-24T23:26:49.285 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:48 smithi080 ceph-mon[29152]: fsmap 3 up:standby 2024-04-24T23:26:49.286 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:48 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-24T23:26:49.891 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:26:49.914 INFO:journalctl@ceph.mds.d.smithi080.stdout:Apr 24 23:26:49 smithi080 systemd[1]: Starting Ceph mds.d for ec7885aa-0290-11ef-bc93-c7b262605968... 2024-04-24T23:26:49.996 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:49.993+0000 7f51949f9640 1 -- 172.21.15.80:0/1365778452 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5190101a00 msgr2=0x7f5190108320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:49.996 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:49.993+0000 7f51949f9640 1 --2- 172.21.15.80:0/1365778452 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5190101a00 0x7f5190108320 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5188009ca0 tx=0x7f518802a8c0 comp rx=0 tx=0).stop 2024-04-24T23:26:49.996 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:49.993+0000 7f51949f9640 1 -- 172.21.15.80:0/1365778452 shutdown_connections 2024-04-24T23:26:49.996 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:49.993+0000 7f51949f9640 1 --2- 172.21.15.80:0/1365778452 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5190101a00 0x7f5190108320 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:49.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:49.993+0000 7f51949f9640 1 --2- 172.21.15.80:0/1365778452 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5190101040 0x7f51901014c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:49.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:49.993+0000 7f51949f9640 1 --2- 172.21.15.80:0/1365778452 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f51900ffe60 0x7f5190100280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:49.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:49.993+0000 7f51949f9640 1 -- 172.21.15.80:0/1365778452 >> 172.21.15.80:0/1365778452 conn(0x7f51900fb420 msgr2=0x7f51900fd860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:49.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:49.993+0000 7f51949f9640 1 -- 172.21.15.80:0/1365778452 shutdown_connections 2024-04-24T23:26:49.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:49.993+0000 7f51949f9640 1 -- 172.21.15.80:0/1365778452 wait complete. 2024-04-24T23:26:49.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:49.994+0000 7f51949f9640 1 Processor -- start 2024-04-24T23:26:49.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:49.994+0000 7f51949f9640 1 -- start start 2024-04-24T23:26:49.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:49.994+0000 7f51949f9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51900ffe60 0x7f51900fd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:49.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:49.994+0000 7f51949f9640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5190101040 0x7f519019c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:49.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:49.994+0000 7f51949f9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5190101a00 0x7f519019ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:49.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:49.994+0000 7f51949f9640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f519010bbb0 con 0x7f5190101040 2024-04-24T23:26:49.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:49.994+0000 7f51949f9640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f519010ba30 con 0x7f5190101a00 2024-04-24T23:26:49.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:49.994+0000 7f51949f9640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f519010bd30 con 0x7f51900ffe60 2024-04-24T23:26:49.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:49.994+0000 7f518dd9b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5190101040 0x7f519019c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:49.998 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:49.994+0000 7f518dd9b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5190101040 0x7f519019c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:33092/0 (socket says 172.21.15.80:33092) 2024-04-24T23:26:49.998 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:49.994+0000 7f518dd9b640 1 -- 172.21.15.80:0/1582398653 learned_addr learned my addr 172.21.15.80:0/1582398653 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:26:49.998 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:49.994+0000 7f518e59c640 1 --2- 172.21.15.80:0/1582398653 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51900ffe60 0x7f51900fd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:49.998 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:49.994+0000 7f518ed9d640 1 --2- 172.21.15.80:0/1582398653 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5190101a00 0x7f519019ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:49.998 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:49.994+0000 7f518dd9b640 1 -- 172.21.15.80:0/1582398653 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51900ffe60 msgr2=0x7f51900fd4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:49.998 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:49.994+0000 7f518dd9b640 1 --2- 172.21.15.80:0/1582398653 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51900ffe60 0x7f51900fd4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:49.998 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:49.994+0000 7f518dd9b640 1 -- 172.21.15.80:0/1582398653 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5190101a00 msgr2=0x7f519019ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:49.998 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:49.994+0000 7f518dd9b640 1 --2- 172.21.15.80:0/1582398653 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5190101a00 0x7f519019ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:49.998 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:49.994+0000 7f518dd9b640 1 -- 172.21.15.80:0/1582398653 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f519019f780 con 0x7f5190101040 2024-04-24T23:26:49.998 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:49.995+0000 7f518dd9b640 1 --2- 172.21.15.80:0/1582398653 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5190101040 0x7f519019c850 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f518400b4a0 tx=0x7f518400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:49.998 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:49.995+0000 7f517f7fe640 1 -- 172.21.15.80:0/1582398653 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5184014070 con 0x7f5190101040 2024-04-24T23:26:49.998 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:49.995+0000 7f517f7fe640 1 -- 172.21.15.80:0/1582398653 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5184010af0 con 0x7f5190101040 2024-04-24T23:26:49.998 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:49.995+0000 7f51949f9640 1 -- 172.21.15.80:0/1582398653 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f519019fa10 con 0x7f5190101040 2024-04-24T23:26:49.999 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:49.995+0000 7f517f7fe640 1 -- 172.21.15.80:0/1582398653 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5184017d10 con 0x7f5190101040 2024-04-24T23:26:49.999 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:49.995+0000 7f51949f9640 1 -- 172.21.15.80:0/1582398653 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f519019ff10 con 0x7f5190101040 2024-04-24T23:26:49.999 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:49.996+0000 7f51949f9640 1 -- 172.21.15.80:0/1582398653 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51901a01a0 con 0x7f5190101040 2024-04-24T23:26:50.000 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:49.997+0000 7f517f7fe640 1 -- 172.21.15.80:0/1582398653 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f5184017070 con 0x7f5190101040 2024-04-24T23:26:50.000 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:49.997+0000 7f517f7fe640 1 --2- 172.21.15.80:0/1582398653 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f51780a2b90 0x7f51780a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:50.000 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:49.998+0000 7f518e59c640 1 --2- 172.21.15.80:0/1582398653 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f51780a2b90 0x7f51780a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:50.000 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:49.998+0000 7f517f7fe640 1 -- 172.21.15.80:0/1582398653 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f51840ce260 con 0x7f5190101040 2024-04-24T23:26:50.000 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:49.998+0000 7f518e59c640 1 --2- 172.21.15.80:0/1582398653 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f51780a2b90 0x7f51780a5050 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f5190100bb0 tx=0x7f518000a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:50.002 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:50.001+0000 7f517f7fe640 1 -- 172.21.15.80:0/1582398653 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f518409b010 con 0x7f5190101040 2024-04-24T23:26:50.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:49 smithi134 ceph-mon[32619]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:26:50.222 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:50.221+0000 7f51949f9640 1 -- 172.21.15.80:0/1582398653 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5190107510 con 0x7f5190101040 2024-04-24T23:26:50.223 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:50.221+0000 7f517f7fe640 1 -- 172.21.15.80:0/1582398653 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3043 (secure 0 0 0) 0x7f518409f310 con 0x7f5190101040 2024-04-24T23:26:50.223 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-24T23:26:50.223 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/891591085","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":891591085},{"type":"v1","addr":"172.21.15.80:6835","nonce":891591085}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24395,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/3736784385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3736784385},{"type":"v1","addr":"172.21.15.134:6833","nonce":3736784385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/124777462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":124777462},{"type":"v1","addr":"172.21.15.148:6833","nonce":124777462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-24T23:26:50.223 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 4 2024-04-24T23:26:50.224 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:49 smithi080 ceph-mon[29152]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:26:50.224 INFO:journalctl@ceph.mds.d.smithi080.stdout:Apr 24 23:26:49 smithi080 podman[69363]: 2024-04-24 23:26:49.861270709 +0000 UTC m=+0.017990808 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:26:50.225 INFO:journalctl@ceph.mds.d.smithi080.stdout:Apr 24 23:26:49 smithi080 podman[69363]: 2024-04-24 23:26:49.980003949 +0000 UTC m=+0.136724036 container create bc0ce64524d4e912a17ee98fde0712324f610f6b53b487c3ea7d8c7500a0c465 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-d, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9) 2024-04-24T23:26:50.225 INFO:journalctl@ceph.mds.d.smithi080.stdout:Apr 24 23:26:50 smithi080 podman[69363]: 2024-04-24 23:26:50.224727133 +0000 UTC m=+0.381447220 container init bc0ce64524d4e912a17ee98fde0712324f610f6b53b487c3ea7d8c7500a0c465 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-d, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T23:26:50.225 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:50.224+0000 7f51949f9640 1 -- 172.21.15.80:0/1582398653 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f51780a2b90 msgr2=0x7f51780a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:50.226 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:50.224+0000 7f51949f9640 1 --2- 172.21.15.80:0/1582398653 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f51780a2b90 0x7f51780a5050 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f5190100bb0 tx=0x7f518000a2d0 comp rx=0 tx=0).stop 2024-04-24T23:26:50.226 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:50.224+0000 7f51949f9640 1 -- 172.21.15.80:0/1582398653 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5190101040 msgr2=0x7f519019c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:50.226 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:50.224+0000 7f51949f9640 1 --2- 172.21.15.80:0/1582398653 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5190101040 0x7f519019c850 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f518400b4a0 tx=0x7f518400b960 comp rx=0 tx=0).stop 2024-04-24T23:26:50.226 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:50.224+0000 7f51949f9640 1 -- 172.21.15.80:0/1582398653 shutdown_connections 2024-04-24T23:26:50.226 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:50.224+0000 7f51949f9640 1 --2- 172.21.15.80:0/1582398653 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f51780a2b90 0x7f51780a5050 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:50.226 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:50.224+0000 7f51949f9640 1 --2- 172.21.15.80:0/1582398653 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5190101a00 0x7f519019ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:50.227 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:50.224+0000 7f51949f9640 1 --2- 172.21.15.80:0/1582398653 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5190101040 0x7f519019c850 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:50.227 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:50.224+0000 7f51949f9640 1 --2- 172.21.15.80:0/1582398653 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51900ffe60 0x7f51900fd4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:50.227 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:50.224+0000 7f51949f9640 1 -- 172.21.15.80:0/1582398653 >> 172.21.15.80:0/1582398653 conn(0x7f51900fb420 msgr2=0x7f51901046c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:50.227 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:50.224+0000 7f51949f9640 1 -- 172.21.15.80:0/1582398653 shutdown_connections 2024-04-24T23:26:50.227 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:50.224+0000 7f51949f9640 1 -- 172.21.15.80:0/1582398653 wait complete. 2024-04-24T23:26:50.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:49 smithi148 ceph-mon[32528]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:26:50.538 INFO:journalctl@ceph.mds.d.smithi080.stdout:Apr 24 23:26:50 smithi080 podman[69363]: 2024-04-24 23:26:50.227752415 +0000 UTC m=+0.384472504 container start bc0ce64524d4e912a17ee98fde0712324f610f6b53b487c3ea7d8c7500a0c465 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-d, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424) 2024-04-24T23:26:50.538 INFO:journalctl@ceph.mds.d.smithi080.stdout:Apr 24 23:26:50 smithi080 ceph-mds[69423]: -- 172.21.15.80:0/3289458570 <== mon.0 v2:172.21.15.80:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563890b41a40 con 0x563890b15000 2024-04-24T23:26:50.539 INFO:journalctl@ceph.mds.d.smithi080.stdout:Apr 24 23:26:50 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-d[69417]: starting mds.d at 2024-04-24T23:26:50.539 INFO:journalctl@ceph.mds.d.smithi080.stdout:Apr 24 23:26:50 smithi080 bash[69363]: bc0ce64524d4e912a17ee98fde0712324f610f6b53b487c3ea7d8c7500a0c465 2024-04-24T23:26:50.539 INFO:journalctl@ceph.mds.d.smithi080.stdout:Apr 24 23:26:50 smithi080 systemd[1]: Started Ceph mds.d for ec7885aa-0290-11ef-bc93-c7b262605968. 2024-04-24T23:26:51.006 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:50 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/1582398653' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:26:51.006 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:50 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:26:51.007 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:50 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:26:51.007 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:50 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:26:51.007 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:50 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:26:51.007 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:50 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:26:51.007 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:50 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:26:51.213 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:50 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/1582398653' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:26:51.213 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:50 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:26:51.213 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:50 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:26:51.213 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:50 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:26:51.213 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:50 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:26:51.213 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:50 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:26:51.213 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:50 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:26:51.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:50 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/1582398653' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:26:51.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:50 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:26:51.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:50 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:26:51.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:50 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:26:51.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:50 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:26:51.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:50 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:26:51.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:50 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:26:52.185 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:51 smithi134 ceph-mon[32619]: Deploying daemon mds.e on smithi134 2024-04-24T23:26:52.185 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:51 smithi134 ceph-mon[32619]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:26:52.185 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:51 smithi134 ceph-mon[32619]: mds.? [v2:172.21.15.80:6836/259529026,v1:172.21.15.80:6837/259529026] up:boot 2024-04-24T23:26:52.185 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:51 smithi134 ceph-mon[32619]: fsmap 4 up:standby 2024-04-24T23:26:52.186 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:51 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-24T23:26:52.238 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:26:52.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:51 smithi148 ceph-mon[32528]: Deploying daemon mds.e on smithi134 2024-04-24T23:26:52.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:51 smithi148 ceph-mon[32528]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:26:52.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:51 smithi148 ceph-mon[32528]: mds.? [v2:172.21.15.80:6836/259529026,v1:172.21.15.80:6837/259529026] up:boot 2024-04-24T23:26:52.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:51 smithi148 ceph-mon[32528]: fsmap 4 up:standby 2024-04-24T23:26:52.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:51 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-24T23:26:52.263 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:51 smithi080 ceph-mon[29152]: Deploying daemon mds.e on smithi134 2024-04-24T23:26:52.263 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:51 smithi080 ceph-mon[29152]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:26:52.263 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:51 smithi080 ceph-mon[29152]: mds.? [v2:172.21.15.80:6836/259529026,v1:172.21.15.80:6837/259529026] up:boot 2024-04-24T23:26:52.264 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:51 smithi080 ceph-mon[29152]: fsmap 4 up:standby 2024-04-24T23:26:52.264 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:51 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-24T23:26:52.350 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:52.347+0000 7f3944e72640 1 -- 172.21.15.80:0/2966322361 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f39401006a0 msgr2=0x7f3940100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:52.350 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:52.347+0000 7f3944e72640 1 --2- 172.21.15.80:0/2966322361 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f39401006a0 0x7f3940100ac0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3934009900 tx=0x7f393402a410 comp rx=0 tx=0).stop 2024-04-24T23:26:52.351 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:52.347+0000 7f3944e72640 1 -- 172.21.15.80:0/2966322361 shutdown_connections 2024-04-24T23:26:52.351 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:52.347+0000 7f3944e72640 1 --2- 172.21.15.80:0/2966322361 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3940102250 0x7f3940108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:52.351 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:52.347+0000 7f3944e72640 1 --2- 172.21.15.80:0/2966322361 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3940101890 0x7f3940101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:52.351 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:52.347+0000 7f3944e72640 1 --2- 172.21.15.80:0/2966322361 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f39401006a0 0x7f3940100ac0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:52.351 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:52.347+0000 7f3944e72640 1 -- 172.21.15.80:0/2966322361 >> 172.21.15.80:0/2966322361 conn(0x7f39400fbc30 msgr2=0x7f39400fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:52.351 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:52.347+0000 7f3944e72640 1 -- 172.21.15.80:0/2966322361 shutdown_connections 2024-04-24T23:26:52.351 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:52.347+0000 7f3944e72640 1 -- 172.21.15.80:0/2966322361 wait complete. 2024-04-24T23:26:52.351 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:52.348+0000 7f3944e72640 1 Processor -- start 2024-04-24T23:26:52.351 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:52.348+0000 7f3944e72640 1 -- start start 2024-04-24T23:26:52.351 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:52.348+0000 7f3944e72640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f39401006a0 0x7f3940197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:52.351 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:52.348+0000 7f3944e72640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3940101890 0x7f3940198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:52.351 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:52.348+0000 7f3944e72640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3940102250 0x7f394019f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:52.351 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:52.348+0000 7f3944e72640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f394010d440 con 0x7f3940102250 2024-04-24T23:26:52.351 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:52.348+0000 7f3944e72640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f394010d2c0 con 0x7f39401006a0 2024-04-24T23:26:52.352 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:52.348+0000 7f3944e72640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f394010d5c0 con 0x7f3940101890 2024-04-24T23:26:52.352 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:52.348+0000 7f393ed9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3940102250 0x7f394019f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:52.352 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:52.348+0000 7f393ed9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3940102250 0x7f394019f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:33140/0 (socket says 172.21.15.80:33140) 2024-04-24T23:26:52.352 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:52.348+0000 7f393ed9d640 1 -- 172.21.15.80:0/3831335743 learned_addr learned my addr 172.21.15.80:0/3831335743 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:26:52.352 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:52.348+0000 7f393e59c640 1 --2- 172.21.15.80:0/3831335743 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f39401006a0 0x7f3940197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:52.352 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:52.348+0000 7f393dd9b640 1 --2- 172.21.15.80:0/3831335743 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3940101890 0x7f3940198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:52.352 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:52.348+0000 7f393ed9d640 1 -- 172.21.15.80:0/3831335743 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3940101890 msgr2=0x7f3940198030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:52.352 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:52.348+0000 7f393ed9d640 1 --2- 172.21.15.80:0/3831335743 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3940101890 0x7f3940198030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:52.352 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:52.348+0000 7f393ed9d640 1 -- 172.21.15.80:0/3831335743 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f39401006a0 msgr2=0x7f3940197af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:52.352 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:52.348+0000 7f393ed9d640 1 --2- 172.21.15.80:0/3831335743 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f39401006a0 0x7f3940197af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:52.352 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:52.348+0000 7f393ed9d640 1 -- 172.21.15.80:0/3831335743 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f394019f710 con 0x7f3940102250 2024-04-24T23:26:52.352 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:52.349+0000 7f393ed9d640 1 --2- 172.21.15.80:0/3831335743 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3940102250 0x7f394019f0a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f3938009fd0 tx=0x7f393800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:52.352 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:52.349+0000 7f392f7fe640 1 -- 172.21.15.80:0/3831335743 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3938017070 con 0x7f3940102250 2024-04-24T23:26:52.352 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:52.349+0000 7f3944e72640 1 -- 172.21.15.80:0/3831335743 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f394019fa00 con 0x7f3940102250 2024-04-24T23:26:52.353 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:52.349+0000 7f392f7fe640 1 -- 172.21.15.80:0/3831335743 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3938003e50 con 0x7f3940102250 2024-04-24T23:26:52.353 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:52.349+0000 7f392f7fe640 1 -- 172.21.15.80:0/3831335743 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39380106c0 con 0x7f3940102250 2024-04-24T23:26:52.353 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:52.349+0000 7f3944e72640 1 -- 172.21.15.80:0/3831335743 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f394019ff50 con 0x7f3940102250 2024-04-24T23:26:52.353 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:52.350+0000 7f3944e72640 1 -- 172.21.15.80:0/3831335743 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3940107a60 con 0x7f3940102250 2024-04-24T23:26:52.354 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:52.351+0000 7f392f7fe640 1 -- 172.21.15.80:0/3831335743 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f3938003970 con 0x7f3940102250 2024-04-24T23:26:52.354 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:52.351+0000 7f392f7fe640 1 --2- 172.21.15.80:0/3831335743 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f39280a2b90 0x7f39280a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:52.354 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:52.351+0000 7f393e59c640 1 --2- 172.21.15.80:0/3831335743 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f39280a2b90 0x7f39280a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:52.354 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:52.352+0000 7f392f7fe640 1 -- 172.21.15.80:0/3831335743 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f39380ce8b0 con 0x7f3940102250 2024-04-24T23:26:52.354 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:52.352+0000 7f393e59c640 1 --2- 172.21.15.80:0/3831335743 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f39280a2b90 0x7f39280a5050 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3934005ab0 tx=0x7f3934000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:52.357 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:52.356+0000 7f392f7fe640 1 -- 172.21.15.80:0/3831335743 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f393809b660 con 0x7f3940102250 2024-04-24T23:26:52.573 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:52.572+0000 7f3944e72640 1 -- 172.21.15.80:0/3831335743 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f394002d170 con 0x7f3940102250 2024-04-24T23:26:52.574 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:52.572+0000 7f392f7fe640 1 -- 172.21.15.80:0/3831335743 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3865 (secure 0 0 0) 0x7f393809f960 con 0x7f3940102250 2024-04-24T23:26:52.574 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-24T23:26:52.574 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/891591085","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":891591085},{"type":"v1","addr":"172.21.15.80:6835","nonce":891591085}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6837/259529026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6836","nonce":259529026},{"type":"v1","addr":"172.21.15.80:6837","nonce":259529026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24395,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/3736784385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3736784385},{"type":"v1","addr":"172.21.15.134:6833","nonce":3736784385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/124777462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":124777462},{"type":"v1","addr":"172.21.15.148:6833","nonce":124777462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-24T23:26:52.574 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 5 2024-04-24T23:26:52.576 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:52.574+0000 7f3944e72640 1 -- 172.21.15.80:0/3831335743 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f39280a2b90 msgr2=0x7f39280a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:52.576 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:52.574+0000 7f3944e72640 1 --2- 172.21.15.80:0/3831335743 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f39280a2b90 0x7f39280a5050 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3934005ab0 tx=0x7f3934000bc0 comp rx=0 tx=0).stop 2024-04-24T23:26:52.576 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:52.574+0000 7f3944e72640 1 -- 172.21.15.80:0/3831335743 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3940102250 msgr2=0x7f394019f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:52.576 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:52.574+0000 7f3944e72640 1 --2- 172.21.15.80:0/3831335743 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3940102250 0x7f394019f0a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f3938009fd0 tx=0x7f393800b260 comp rx=0 tx=0).stop 2024-04-24T23:26:52.576 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:52.574+0000 7f3944e72640 1 -- 172.21.15.80:0/3831335743 shutdown_connections 2024-04-24T23:26:52.576 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:52.574+0000 7f3944e72640 1 --2- 172.21.15.80:0/3831335743 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f39280a2b90 0x7f39280a5050 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:52.576 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:52.574+0000 7f3944e72640 1 --2- 172.21.15.80:0/3831335743 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3940102250 0x7f394019f0a0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:52.576 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:52.574+0000 7f3944e72640 1 --2- 172.21.15.80:0/3831335743 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3940101890 0x7f3940198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:52.576 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:52.575+0000 7f3944e72640 1 --2- 172.21.15.80:0/3831335743 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f39401006a0 0x7f3940197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:52.576 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:52.575+0000 7f3944e72640 1 -- 172.21.15.80:0/3831335743 >> 172.21.15.80:0/3831335743 conn(0x7f39400fbc30 msgr2=0x7f3940104c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:52.576 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:52.575+0000 7f3944e72640 1 -- 172.21.15.80:0/3831335743 shutdown_connections 2024-04-24T23:26:52.577 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:52.575+0000 7f3944e72640 1 -- 172.21.15.80:0/3831335743 wait complete. 2024-04-24T23:26:52.869 INFO:journalctl@ceph.mds.e.smithi134.stdout:Apr 24 23:26:52 smithi134 systemd[1]: Starting Ceph mds.e for ec7885aa-0290-11ef-bc93-c7b262605968... 2024-04-24T23:26:53.140 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:52 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/3831335743' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:26:53.141 INFO:journalctl@ceph.mds.e.smithi134.stdout:Apr 24 23:26:52 smithi134 podman[62769]: 2024-04-24 23:26:52.771717671 +0000 UTC m=+0.017905908 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:26:53.141 INFO:journalctl@ceph.mds.e.smithi134.stdout:Apr 24 23:26:52 smithi134 podman[62769]: 2024-04-24 23:26:52.887936044 +0000 UTC m=+0.134124262 container create 366560a918ca496bbf848611a0f5b9abc47ad79b25d39a2f4bba7db063753e2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-e, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-04-24T23:26:53.141 INFO:journalctl@ceph.mds.e.smithi134.stdout:Apr 24 23:26:53 smithi134 podman[62769]: 2024-04-24 23:26:53.140934498 +0000 UTC m=+0.387122716 container init 366560a918ca496bbf848611a0f5b9abc47ad79b25d39a2f4bba7db063753e2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-e, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-04-24T23:26:53.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:52 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/3831335743' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:26:53.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:52 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/3831335743' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:26:53.447 INFO:journalctl@ceph.mds.e.smithi134.stdout:Apr 24 23:26:53 smithi134 podman[62769]: 2024-04-24 23:26:53.143953032 +0000 UTC m=+0.390141250 container start 366560a918ca496bbf848611a0f5b9abc47ad79b25d39a2f4bba7db063753e2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-e, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T23:26:53.447 INFO:journalctl@ceph.mds.e.smithi134.stdout:Apr 24 23:26:53 smithi134 ceph-mds[62789]: -- 172.21.15.134:0/1250951051 <== mon.2 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555a3fa9da40 con 0x555a3fa70800 2024-04-24T23:26:53.448 INFO:journalctl@ceph.mds.e.smithi134.stdout:Apr 24 23:26:53 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-e[62785]: starting mds.e at 2024-04-24T23:26:53.448 INFO:journalctl@ceph.mds.e.smithi134.stdout:Apr 24 23:26:53 smithi134 bash[62769]: 366560a918ca496bbf848611a0f5b9abc47ad79b25d39a2f4bba7db063753e2a 2024-04-24T23:26:53.448 INFO:journalctl@ceph.mds.e.smithi134.stdout:Apr 24 23:26:53 smithi134 systemd[1]: Started Ceph mds.e for ec7885aa-0290-11ef-bc93-c7b262605968. 2024-04-24T23:26:54.255 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:53 smithi148 ceph-mon[32528]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:26:54.255 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:53 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:26:54.255 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:53 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:26:54.255 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:53 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:26:54.255 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:53 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:26:54.255 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:53 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:26:54.255 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:53 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:26:54.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:53 smithi080 ceph-mon[29152]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:26:54.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:53 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:26:54.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:53 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:26:54.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:53 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:26:54.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:53 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:26:54.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:53 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:26:54.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:53 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:26:54.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:53 smithi134 ceph-mon[32619]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:26:54.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:53 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:26:54.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:53 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:26:54.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:53 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:26:54.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:53 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:26:54.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:53 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:26:54.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:53 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:26:54.588 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:26:54.701 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:54.697+0000 7f1206ed4640 1 -- 172.21.15.80:0/3047226890 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f12001006a0 msgr2=0x7f1200100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:54.701 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:54.697+0000 7f1206ed4640 1 --2- 172.21.15.80:0/3047226890 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f12001006a0 0x7f1200100ac0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f11f4009900 tx=0x7f11f402a410 comp rx=0 tx=0).stop 2024-04-24T23:26:54.701 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:54.697+0000 7f1206ed4640 1 -- 172.21.15.80:0/3047226890 shutdown_connections 2024-04-24T23:26:54.702 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:54.697+0000 7f1206ed4640 1 --2- 172.21.15.80:0/3047226890 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1200102250 0x7f1200108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:54.702 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:54.697+0000 7f1206ed4640 1 --2- 172.21.15.80:0/3047226890 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1200101890 0x7f1200101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:54.702 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:54.697+0000 7f1206ed4640 1 --2- 172.21.15.80:0/3047226890 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f12001006a0 0x7f1200100ac0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:54.702 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:54.697+0000 7f1206ed4640 1 -- 172.21.15.80:0/3047226890 >> 172.21.15.80:0/3047226890 conn(0x7f12000fbc30 msgr2=0x7f12000fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:54.702 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:54.697+0000 7f1206ed4640 1 -- 172.21.15.80:0/3047226890 shutdown_connections 2024-04-24T23:26:54.702 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:54.697+0000 7f1206ed4640 1 -- 172.21.15.80:0/3047226890 wait complete. 2024-04-24T23:26:54.702 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:54.698+0000 7f1206ed4640 1 Processor -- start 2024-04-24T23:26:54.702 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:54.698+0000 7f1206ed4640 1 -- start start 2024-04-24T23:26:54.702 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:54.698+0000 7f1206ed4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12001006a0 0x7f120006a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:54.702 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:54.698+0000 7f1206ed4640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1200101890 0x7f120019c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:54.703 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:54.698+0000 7f1206ed4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1200102250 0x7f120019ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:54.703 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:54.698+0000 7f1206ed4640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f120010c2f0 con 0x7f1200101890 2024-04-24T23:26:54.703 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:54.698+0000 7f1206ed4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f120010c170 con 0x7f1200102250 2024-04-24T23:26:54.703 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:54.698+0000 7f1206ed4640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f120010c470 con 0x7f12001006a0 2024-04-24T23:26:54.703 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:54.698+0000 7f11fffff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1200101890 0x7f120019c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:54.703 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:54.698+0000 7f11fffff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1200101890 0x7f120019c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:39494/0 (socket says 172.21.15.80:39494) 2024-04-24T23:26:54.703 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:54.698+0000 7f11fffff640 1 -- 172.21.15.80:0/1044078937 learned_addr learned my addr 172.21.15.80:0/1044078937 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:26:54.703 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:54.698+0000 7f1204c70640 1 --2- 172.21.15.80:0/1044078937 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12001006a0 0x7f120006a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:54.703 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:54.698+0000 7f1205471640 1 --2- 172.21.15.80:0/1044078937 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1200102250 0x7f120019ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:54.703 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:54.698+0000 7f11fffff640 1 -- 172.21.15.80:0/1044078937 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12001006a0 msgr2=0x7f120006a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:54.704 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:54.698+0000 7f11fffff640 1 --2- 172.21.15.80:0/1044078937 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12001006a0 0x7f120006a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:54.704 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:54.698+0000 7f11fffff640 1 -- 172.21.15.80:0/1044078937 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1200102250 msgr2=0x7f120019ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:54.704 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:54.698+0000 7f11fffff640 1 --2- 172.21.15.80:0/1044078937 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1200102250 0x7f120019ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:54.704 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:54.698+0000 7f11fffff640 1 -- 172.21.15.80:0/1044078937 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f120019f720 con 0x7f1200101890 2024-04-24T23:26:54.704 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:54.698+0000 7f11fffff640 1 --2- 172.21.15.80:0/1044078937 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1200101890 0x7f120019c820 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f11ec00d950 tx=0x7f11ec00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:54.704 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:54.699+0000 7f11fdffb640 1 -- 172.21.15.80:0/1044078937 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11ec017030 con 0x7f1200101890 2024-04-24T23:26:54.704 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:54.699+0000 7f11fdffb640 1 -- 172.21.15.80:0/1044078937 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11ec004020 con 0x7f1200101890 2024-04-24T23:26:54.704 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:54.699+0000 7f1206ed4640 1 -- 172.21.15.80:0/1044078937 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f120019fa10 con 0x7f1200101890 2024-04-24T23:26:54.704 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:54.699+0000 7f11fdffb640 1 -- 172.21.15.80:0/1044078937 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11ec0047a0 con 0x7f1200101890 2024-04-24T23:26:54.704 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:54.699+0000 7f1206ed4640 1 -- 172.21.15.80:0/1044078937 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f120019ff60 con 0x7f1200101890 2024-04-24T23:26:54.704 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:54.700+0000 7f1206ed4640 1 -- 172.21.15.80:0/1044078937 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1200107a40 con 0x7f1200101890 2024-04-24T23:26:54.705 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:54.701+0000 7f11fdffb640 1 -- 172.21.15.80:0/1044078937 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f11ec0041d0 con 0x7f1200101890 2024-04-24T23:26:54.705 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:54.701+0000 7f11fdffb640 1 --2- 172.21.15.80:0/1044078937 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f11f00a2b90 0x7f11f00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:54.705 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:54.701+0000 7f1204c70640 1 --2- 172.21.15.80:0/1044078937 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f11f00a2b90 0x7f11f00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:54.706 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:54.702+0000 7f11fdffb640 1 -- 172.21.15.80:0/1044078937 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f11ec0ce560 con 0x7f1200101890 2024-04-24T23:26:54.706 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:54.702+0000 7f1204c70640 1 --2- 172.21.15.80:0/1044078937 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f11f00a2b90 0x7f11f00a5050 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f11f4005ab0 tx=0x7f11f4000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:54.707 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:54.706+0000 7f11fdffb640 1 -- 172.21.15.80:0/1044078937 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f11ec09b960 con 0x7f1200101890 2024-04-24T23:26:54.920 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:54.919+0000 7f1206ed4640 1 -- 172.21.15.80:0/1044078937 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f120002d170 con 0x7f1200101890 2024-04-24T23:26:54.921 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:54.920+0000 7f11fdffb640 1 -- 172.21.15.80:0/1044078937 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4693 (secure 0 0 0) 0x7f11ec09b260 con 0x7f1200101890 2024-04-24T23:26:54.921 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-24T23:26:54.921 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/891591085","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":891591085},{"type":"v1","addr":"172.21.15.80:6835","nonce":891591085}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6837/259529026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6836","nonce":259529026},{"type":"v1","addr":"172.21.15.80:6837","nonce":259529026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24395,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/3736784385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3736784385},{"type":"v1","addr":"172.21.15.134:6833","nonce":3736784385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/124777462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":124777462},{"type":"v1","addr":"172.21.15.148:6833","nonce":124777462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/1399801120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1399801120},{"type":"v1","addr":"172.21.15.134:6835","nonce":1399801120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-24T23:26:54.922 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 6 2024-04-24T23:26:54.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:54.922+0000 7f1206ed4640 1 -- 172.21.15.80:0/1044078937 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f11f00a2b90 msgr2=0x7f11f00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:54.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:54.922+0000 7f1206ed4640 1 --2- 172.21.15.80:0/1044078937 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f11f00a2b90 0x7f11f00a5050 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f11f4005ab0 tx=0x7f11f4000bc0 comp rx=0 tx=0).stop 2024-04-24T23:26:54.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:54.922+0000 7f1206ed4640 1 -- 172.21.15.80:0/1044078937 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1200101890 msgr2=0x7f120019c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:54.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:54.922+0000 7f1206ed4640 1 --2- 172.21.15.80:0/1044078937 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1200101890 0x7f120019c820 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f11ec00d950 tx=0x7f11ec00de10 comp rx=0 tx=0).stop 2024-04-24T23:26:54.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:54.922+0000 7f1206ed4640 1 -- 172.21.15.80:0/1044078937 shutdown_connections 2024-04-24T23:26:54.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:54.922+0000 7f1206ed4640 1 --2- 172.21.15.80:0/1044078937 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f11f00a2b90 0x7f11f00a5050 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:54.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:54.922+0000 7f1206ed4640 1 --2- 172.21.15.80:0/1044078937 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1200102250 0x7f120019ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:54.924 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:54.922+0000 7f1206ed4640 1 --2- 172.21.15.80:0/1044078937 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1200101890 0x7f120019c820 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:54.924 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:54.922+0000 7f1206ed4640 1 --2- 172.21.15.80:0/1044078937 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12001006a0 0x7f120006a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:54.924 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:54.922+0000 7f1206ed4640 1 -- 172.21.15.80:0/1044078937 >> 172.21.15.80:0/1044078937 conn(0x7f12000fbc30 msgr2=0x7f1200104c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:54.924 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:54.922+0000 7f1206ed4640 1 -- 172.21.15.80:0/1044078937 shutdown_connections 2024-04-24T23:26:54.924 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:54.922+0000 7f1206ed4640 1 -- 172.21.15.80:0/1044078937 wait complete. 2024-04-24T23:26:55.006 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:54 smithi148 ceph-mon[32528]: Deploying daemon mds.f on smithi148 2024-04-24T23:26:55.006 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:54 smithi148 ceph-mon[32528]: mds.? [v2:172.21.15.134:6834/1399801120,v1:172.21.15.134:6835/1399801120] up:boot 2024-04-24T23:26:55.007 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:54 smithi148 ceph-mon[32528]: fsmap 5 up:standby 2024-04-24T23:26:55.007 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:54 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-24T23:26:55.007 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:54 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/1044078937' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:26:55.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:54 smithi080 ceph-mon[29152]: Deploying daemon mds.f on smithi148 2024-04-24T23:26:55.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:54 smithi080 ceph-mon[29152]: mds.? [v2:172.21.15.134:6834/1399801120,v1:172.21.15.134:6835/1399801120] up:boot 2024-04-24T23:26:55.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:54 smithi080 ceph-mon[29152]: fsmap 5 up:standby 2024-04-24T23:26:55.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:54 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-24T23:26:55.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:54 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/1044078937' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:26:55.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:54 smithi134 ceph-mon[32619]: Deploying daemon mds.f on smithi148 2024-04-24T23:26:55.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:54 smithi134 ceph-mon[32619]: mds.? [v2:172.21.15.134:6834/1399801120,v1:172.21.15.134:6835/1399801120] up:boot 2024-04-24T23:26:55.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:54 smithi134 ceph-mon[32619]: fsmap 5 up:standby 2024-04-24T23:26:55.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:54 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-24T23:26:55.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:54 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/1044078937' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:26:55.748 INFO:journalctl@ceph.mds.f.smithi148.stdout:Apr 24 23:26:55 smithi148 systemd[1]: Starting Ceph mds.f for ec7885aa-0290-11ef-bc93-c7b262605968... 2024-04-24T23:26:56.006 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:55 smithi148 ceph-mon[32528]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:26:56.006 INFO:journalctl@ceph.mds.f.smithi148.stdout:Apr 24 23:26:55 smithi148 podman[63072]: 2024-04-24 23:26:55.650294672 +0000 UTC m=+0.018320260 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:26:56.007 INFO:journalctl@ceph.mds.f.smithi148.stdout:Apr 24 23:26:55 smithi148 podman[63072]: 2024-04-24 23:26:55.771283382 +0000 UTC m=+0.139308957 container create 9f0a8e178d1a527dbaf09fc8d5a3c430f989596aa16b17c0e7ba06f158555eaa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-f, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9) 2024-04-24T23:26:56.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:55 smithi080 ceph-mon[29152]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:26:56.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:55 smithi134 ceph-mon[32619]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:26:56.506 INFO:journalctl@ceph.mds.f.smithi148.stdout:Apr 24 23:26:56 smithi148 podman[63072]: 2024-04-24 23:26:56.040515473 +0000 UTC m=+0.408541048 container init 9f0a8e178d1a527dbaf09fc8d5a3c430f989596aa16b17c0e7ba06f158555eaa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-f, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True) 2024-04-24T23:26:56.507 INFO:journalctl@ceph.mds.f.smithi148.stdout:Apr 24 23:26:56 smithi148 podman[63072]: 2024-04-24 23:26:56.043283058 +0000 UTC m=+0.411308627 container start 9f0a8e178d1a527dbaf09fc8d5a3c430f989596aa16b17c0e7ba06f158555eaa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9) 2024-04-24T23:26:56.507 INFO:journalctl@ceph.mds.f.smithi148.stdout:Apr 24 23:26:56 smithi148 ceph-mds[63091]: -- 172.21.15.148:0/2969598825 <== mon.1 v2:172.21.15.148:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558d5e4bfa40 con 0x558d5e493000 2024-04-24T23:26:56.507 INFO:journalctl@ceph.mds.f.smithi148.stdout:Apr 24 23:26:56 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-f[63087]: starting mds.f at 2024-04-24T23:26:56.507 INFO:journalctl@ceph.mds.f.smithi148.stdout:Apr 24 23:26:56 smithi148 bash[63072]: 9f0a8e178d1a527dbaf09fc8d5a3c430f989596aa16b17c0e7ba06f158555eaa 2024-04-24T23:26:56.507 INFO:journalctl@ceph.mds.f.smithi148.stdout:Apr 24 23:26:56 smithi148 systemd[1]: Started Ceph mds.f for ec7885aa-0290-11ef-bc93-c7b262605968. 2024-04-24T23:26:56.935 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:26:57.050 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:57.047+0000 7fedcf51d640 1 -- 172.21.15.80:0/3507830834 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fedc81006a0 msgr2=0x7fedc8100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:57.050 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:57.047+0000 7fedcf51d640 1 --2- 172.21.15.80:0/3507830834 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fedc81006a0 0x7fedc8100ac0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fedc0009900 tx=0x7fedc002a410 comp rx=0 tx=0).stop 2024-04-24T23:26:57.050 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:57.047+0000 7fedcf51d640 1 -- 172.21.15.80:0/3507830834 shutdown_connections 2024-04-24T23:26:57.051 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:57.047+0000 7fedcf51d640 1 --2- 172.21.15.80:0/3507830834 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fedc8102250 0x7fedc8108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:57.051 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:57.047+0000 7fedcf51d640 1 --2- 172.21.15.80:0/3507830834 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fedc8101890 0x7fedc8101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:57.051 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:57.047+0000 7fedcf51d640 1 --2- 172.21.15.80:0/3507830834 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fedc81006a0 0x7fedc8100ac0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:57.051 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:57.047+0000 7fedcf51d640 1 -- 172.21.15.80:0/3507830834 >> 172.21.15.80:0/3507830834 conn(0x7fedc80fbc30 msgr2=0x7fedc80fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:57.051 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:57.047+0000 7fedcf51d640 1 -- 172.21.15.80:0/3507830834 shutdown_connections 2024-04-24T23:26:57.051 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:57.047+0000 7fedcf51d640 1 -- 172.21.15.80:0/3507830834 wait complete. 2024-04-24T23:26:57.051 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:57.048+0000 7fedcf51d640 1 Processor -- start 2024-04-24T23:26:57.051 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:57.048+0000 7fedcf51d640 1 -- start start 2024-04-24T23:26:57.051 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:57.048+0000 7fedcf51d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fedc81006a0 0x7fedc8197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:57.051 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:57.048+0000 7fedcf51d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fedc8101890 0x7fedc8198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:57.051 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:57.048+0000 7fedcf51d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fedc8102250 0x7fedc819f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:57.051 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:57.048+0000 7fedcf51d640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fedc810c370 con 0x7fedc8101890 2024-04-24T23:26:57.051 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:57.048+0000 7fedcf51d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fedc810c1f0 con 0x7fedc8102250 2024-04-24T23:26:57.052 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:57.048+0000 7fedcf51d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fedc810c4f0 con 0x7fedc81006a0 2024-04-24T23:26:57.052 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:57.048+0000 7fedccab8640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fedc8101890 0x7fedc8198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:57.052 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:57.048+0000 7fedccab8640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fedc8101890 0x7fedc8198060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:39522/0 (socket says 172.21.15.80:39522) 2024-04-24T23:26:57.052 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:57.048+0000 7fedccab8640 1 -- 172.21.15.80:0/946088870 learned_addr learned my addr 172.21.15.80:0/946088870 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:26:57.052 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:57.048+0000 7fedcd2b9640 1 --2- 172.21.15.80:0/946088870 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fedc81006a0 0x7fedc8197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:57.052 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:57.048+0000 7fedcdaba640 1 --2- 172.21.15.80:0/946088870 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fedc8102250 0x7fedc819f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:57.052 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:57.048+0000 7fedccab8640 1 -- 172.21.15.80:0/946088870 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fedc81006a0 msgr2=0x7fedc8197b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:57.052 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:57.048+0000 7fedccab8640 1 --2- 172.21.15.80:0/946088870 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fedc81006a0 0x7fedc8197b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:57.052 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:57.048+0000 7fedccab8640 1 -- 172.21.15.80:0/946088870 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fedc8102250 msgr2=0x7fedc819f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:57.052 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:57.048+0000 7fedccab8640 1 --2- 172.21.15.80:0/946088870 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fedc8102250 0x7fedc819f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:57.052 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:57.048+0000 7fedccab8640 1 -- 172.21.15.80:0/946088870 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedc819f740 con 0x7fedc8101890 2024-04-24T23:26:57.052 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:57.048+0000 7fedccab8640 1 --2- 172.21.15.80:0/946088870 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fedc8101890 0x7fedc8198060 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fedb800d950 tx=0x7fedb800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:57.052 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:57.049+0000 7fedbe7fc640 1 -- 172.21.15.80:0/946088870 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedb8017070 con 0x7fedc8101890 2024-04-24T23:26:57.052 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:57.049+0000 7fedbe7fc640 1 -- 172.21.15.80:0/946088870 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fedb8009c30 con 0x7fedc8101890 2024-04-24T23:26:57.053 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:57.049+0000 7fedcf51d640 1 -- 172.21.15.80:0/946088870 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedc819fa30 con 0x7fedc8101890 2024-04-24T23:26:57.053 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:57.049+0000 7fedbe7fc640 1 -- 172.21.15.80:0/946088870 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedb800f6b0 con 0x7fedc8101890 2024-04-24T23:26:57.053 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:57.049+0000 7fedcf51d640 1 -- 172.21.15.80:0/946088870 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedc81a0000 con 0x7fedc8101890 2024-04-24T23:26:57.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:57.050+0000 7fedcf51d640 1 -- 172.21.15.80:0/946088870 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fedc81a0230 con 0x7fedc8101890 2024-04-24T23:26:57.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:57.051+0000 7fedbe7fc640 1 -- 172.21.15.80:0/946088870 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fedb80109d0 con 0x7fedc8101890 2024-04-24T23:26:57.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:57.051+0000 7fedbe7fc640 1 --2- 172.21.15.80:0/946088870 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fedb40a2b90 0x7fedb40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:57.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:57.051+0000 7fedcd2b9640 1 --2- 172.21.15.80:0/946088870 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fedb40a2b90 0x7fedb40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:57.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:57.051+0000 7fedbe7fc640 1 -- 172.21.15.80:0/946088870 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fedb80ce700 con 0x7fedc8101890 2024-04-24T23:26:57.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:57.052+0000 7fedcd2b9640 1 --2- 172.21.15.80:0/946088870 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fedb40a2b90 0x7fedb40a5050 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fedc0005ab0 tx=0x7fedc0000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:57.055 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:57.055+0000 7fedbe7fc640 1 -- 172.21.15.80:0/946088870 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fedb8009de0 con 0x7fedc8101890 2024-04-24T23:26:57.274 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:57.272+0000 7fedcf51d640 1 -- 172.21.15.80:0/946088870 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fedc81a0400 con 0x7fedc8101890 2024-04-24T23:26:57.274 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:57.273+0000 7fedbe7fc640 1 -- 172.21.15.80:0/946088870 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5521 (secure 0 0 0) 0x7fedb809b4b0 con 0x7fedc8101890 2024-04-24T23:26:57.274 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-24T23:26:57.274 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/891591085","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":891591085},{"type":"v1","addr":"172.21.15.80:6835","nonce":891591085}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6837/259529026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6836","nonce":259529026},{"type":"v1","addr":"172.21.15.80:6837","nonce":259529026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24395,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/3736784385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3736784385},{"type":"v1","addr":"172.21.15.134:6833","nonce":3736784385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/124777462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":124777462},{"type":"v1","addr":"172.21.15.148:6833","nonce":124777462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/1399801120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1399801120},{"type":"v1","addr":"172.21.15.134:6835","nonce":1399801120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24460,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/3002319310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3002319310},{"type":"v1","addr":"172.21.15.148:6835","nonce":3002319310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-04-24T23:26:57.275 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 7 2024-04-24T23:26:57.276 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:57.275+0000 7fedcf51d640 1 -- 172.21.15.80:0/946088870 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fedb40a2b90 msgr2=0x7fedb40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:57.276 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:57.275+0000 7fedcf51d640 1 --2- 172.21.15.80:0/946088870 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fedb40a2b90 0x7fedb40a5050 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fedc0005ab0 tx=0x7fedc0000bc0 comp rx=0 tx=0).stop 2024-04-24T23:26:57.276 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:57.275+0000 7fedcf51d640 1 -- 172.21.15.80:0/946088870 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fedc8101890 msgr2=0x7fedc8198060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:57.277 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:57.275+0000 7fedcf51d640 1 --2- 172.21.15.80:0/946088870 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fedc8101890 0x7fedc8198060 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fedb800d950 tx=0x7fedb800de10 comp rx=0 tx=0).stop 2024-04-24T23:26:57.277 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:57.275+0000 7fedcf51d640 1 -- 172.21.15.80:0/946088870 shutdown_connections 2024-04-24T23:26:57.277 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:57.275+0000 7fedcf51d640 1 --2- 172.21.15.80:0/946088870 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fedb40a2b90 0x7fedb40a5050 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:57.277 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:57.275+0000 7fedcf51d640 1 --2- 172.21.15.80:0/946088870 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fedc8102250 0x7fedc819f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:57.277 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:57.275+0000 7fedcf51d640 1 --2- 172.21.15.80:0/946088870 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fedc8101890 0x7fedc8198060 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:57.277 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:57.275+0000 7fedcf51d640 1 --2- 172.21.15.80:0/946088870 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fedc81006a0 0x7fedc8197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:57.277 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:57.275+0000 7fedcf51d640 1 -- 172.21.15.80:0/946088870 >> 172.21.15.80:0/946088870 conn(0x7fedc80fbc30 msgr2=0x7fedc80fd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:57.277 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:57.275+0000 7fedcf51d640 1 -- 172.21.15.80:0/946088870 shutdown_connections 2024-04-24T23:26:57.277 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:57.275+0000 7fedcf51d640 1 -- 172.21.15.80:0/946088870 wait complete. 2024-04-24T23:26:57.286 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:57 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:26:57.286 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:57 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:26:57.286 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:57 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:26:57.286 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:57 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:26:57.286 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:57 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:26:57.286 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:57 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:26:57.286 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:57 smithi080 ceph-mon[29152]: Deploying daemon mds.c on smithi148 2024-04-24T23:26:57.286 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:57 smithi080 ceph-mon[29152]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:26:57.287 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:57 smithi080 ceph-mon[29152]: mds.? [v2:172.21.15.148:6834/3002319310,v1:172.21.15.148:6835/3002319310] up:boot 2024-04-24T23:26:57.287 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:57 smithi080 ceph-mon[29152]: fsmap 6 up:standby 2024-04-24T23:26:57.287 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:57 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-24T23:26:57.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:57 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:26:57.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:57 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:26:57.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:57 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:26:57.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:57 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:26:57.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:57 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:26:57.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:57 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:26:57.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:57 smithi134 ceph-mon[32619]: Deploying daemon mds.c on smithi148 2024-04-24T23:26:57.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:57 smithi134 ceph-mon[32619]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:26:57.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:57 smithi134 ceph-mon[32619]: mds.? [v2:172.21.15.148:6834/3002319310,v1:172.21.15.148:6835/3002319310] up:boot 2024-04-24T23:26:57.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:57 smithi134 ceph-mon[32619]: fsmap 6 up:standby 2024-04-24T23:26:57.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:57 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-24T23:26:57.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:57 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:26:57.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:57 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:26:57.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:57 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:26:57.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:57 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:26:57.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:57 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:26:57.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:57 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:26:57.508 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:57 smithi148 ceph-mon[32528]: Deploying daemon mds.c on smithi148 2024-04-24T23:26:57.508 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:57 smithi148 ceph-mon[32528]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:26:57.508 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:57 smithi148 ceph-mon[32528]: mds.? [v2:172.21.15.148:6834/3002319310,v1:172.21.15.148:6835/3002319310] up:boot 2024-04-24T23:26:57.508 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:57 smithi148 ceph-mon[32528]: fsmap 6 up:standby 2024-04-24T23:26:57.508 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:57 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-24T23:26:58.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:58 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/946088870' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:26:58.507 INFO:journalctl@ceph.mds.c.smithi148.stdout:Apr 24 23:26:58 smithi148 systemd[1]: Starting Ceph mds.c for ec7885aa-0290-11ef-bc93-c7b262605968... 2024-04-24T23:26:58.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:58 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/946088870' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:26:58.697 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:58 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/946088870' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:26:58.821 INFO:journalctl@ceph.mds.c.smithi148.stdout:Apr 24 23:26:58 smithi148 podman[63381]: 2024-04-24 23:26:58.420596755 +0000 UTC m=+0.018221677 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:26:58.821 INFO:journalctl@ceph.mds.c.smithi148.stdout:Apr 24 23:26:58 smithi148 podman[63381]: 2024-04-24 23:26:58.534545529 +0000 UTC m=+0.132170449 container create 85fdccac975f816f9193eff752196fbe07ddf2176ba81b212dafcdcff2062b7d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-c, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424) 2024-04-24T23:26:58.821 INFO:journalctl@ceph.mds.c.smithi148.stdout:Apr 24 23:26:58 smithi148 podman[63381]: 2024-04-24 23:26:58.737247948 +0000 UTC m=+0.334872867 container init 85fdccac975f816f9193eff752196fbe07ddf2176ba81b212dafcdcff2062b7d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-c, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9) 2024-04-24T23:26:58.821 INFO:journalctl@ceph.mds.c.smithi148.stdout:Apr 24 23:26:58 smithi148 podman[63381]: 2024-04-24 23:26:58.739804653 +0000 UTC m=+0.337429572 container start 85fdccac975f816f9193eff752196fbe07ddf2176ba81b212dafcdcff2062b7d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-c, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-04-24T23:26:58.821 INFO:journalctl@ceph.mds.c.smithi148.stdout:Apr 24 23:26:58 smithi148 ceph-mds[63400]: -- 172.21.15.148:0/4260828397 <== mon.1 v2:172.21.15.148:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558a4d5d1a40 con 0x558a4d5a5000 2024-04-24T23:26:58.821 INFO:journalctl@ceph.mds.c.smithi148.stdout:Apr 24 23:26:58 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-c[63396]: starting mds.c at 2024-04-24T23:26:58.821 INFO:journalctl@ceph.mds.c.smithi148.stdout:Apr 24 23:26:58 smithi148 bash[63381]: 85fdccac975f816f9193eff752196fbe07ddf2176ba81b212dafcdcff2062b7d 2024-04-24T23:26:59.256 INFO:journalctl@ceph.mds.c.smithi148.stdout:Apr 24 23:26:58 smithi148 systemd[1]: Started Ceph mds.c for ec7885aa-0290-11ef-bc93-c7b262605968. 2024-04-24T23:26:59.289 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:26:59.392 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:59.388+0000 7f810419e640 1 -- 172.21.15.80:0/3255583362 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f80fc101870 msgr2=0x7f80fc101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:59.392 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:59.388+0000 7f810419e640 1 --2- 172.21.15.80:0/3255583362 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f80fc101870 0x7f80fc101cf0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f80f8009900 tx=0x7f80f802a410 comp rx=0 tx=0).stop 2024-04-24T23:26:59.392 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:59.388+0000 7f810419e640 1 -- 172.21.15.80:0/3255583362 shutdown_connections 2024-04-24T23:26:59.392 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:59.388+0000 7f810419e640 1 --2- 172.21.15.80:0/3255583362 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f80fc102230 0x7f80fc108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:59.392 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:59.388+0000 7f810419e640 1 --2- 172.21.15.80:0/3255583362 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f80fc101870 0x7f80fc101cf0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:59.392 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:59.388+0000 7f810419e640 1 --2- 172.21.15.80:0/3255583362 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f80fc100680 0x7f80fc100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:59.392 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:59.388+0000 7f810419e640 1 -- 172.21.15.80:0/3255583362 >> 172.21.15.80:0/3255583362 conn(0x7f80fc0fbbf0 msgr2=0x7f80fc0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:59.392 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:59.388+0000 7f810419e640 1 -- 172.21.15.80:0/3255583362 shutdown_connections 2024-04-24T23:26:59.392 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:59.388+0000 7f810419e640 1 -- 172.21.15.80:0/3255583362 wait complete. 2024-04-24T23:26:59.392 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:59.388+0000 7f810419e640 1 Processor -- start 2024-04-24T23:26:59.393 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:59.388+0000 7f810419e640 1 -- start start 2024-04-24T23:26:59.393 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:59.389+0000 7f810419e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f80fc100680 0x7f80fc069440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:59.393 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:59.389+0000 7f810419e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f80fc101870 0x7f80fc19c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:59.393 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:59.389+0000 7f810419e640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f80fc102230 0x7f80fc19ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:59.393 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:59.389+0000 7f810419e640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f80fc10c300 con 0x7f80fc102230 2024-04-24T23:26:59.393 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:59.389+0000 7f810419e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f80fc10c180 con 0x7f80fc101870 2024-04-24T23:26:59.393 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:59.389+0000 7f810419e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f80fc10c480 con 0x7f80fc100680 2024-04-24T23:26:59.393 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:59.389+0000 7f810273b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f80fc102230 0x7f80fc19ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:59.393 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:59.389+0000 7f810273b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f80fc102230 0x7f80fc19ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:39536/0 (socket says 172.21.15.80:39536) 2024-04-24T23:26:59.393 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:59.389+0000 7f810273b640 1 -- 172.21.15.80:0/331524676 learned_addr learned my addr 172.21.15.80:0/331524676 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:26:59.393 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:59.389+0000 7f8101739640 1 --2- 172.21.15.80:0/331524676 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f80fc101870 0x7f80fc19c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:59.393 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:59.389+0000 7f8101f3a640 1 --2- 172.21.15.80:0/331524676 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f80fc100680 0x7f80fc069440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:59.393 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:59.389+0000 7f810273b640 1 -- 172.21.15.80:0/331524676 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f80fc100680 msgr2=0x7f80fc069440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:59.393 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:59.389+0000 7f810273b640 1 --2- 172.21.15.80:0/331524676 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f80fc100680 0x7f80fc069440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:59.394 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:59.389+0000 7f810273b640 1 -- 172.21.15.80:0/331524676 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f80fc101870 msgr2=0x7f80fc19c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:59.394 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:59.389+0000 7f810273b640 1 --2- 172.21.15.80:0/331524676 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f80fc101870 0x7f80fc19c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:59.394 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:59.389+0000 7f810273b640 1 -- 172.21.15.80:0/331524676 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80fc19f720 con 0x7f80fc102230 2024-04-24T23:26:59.394 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:59.389+0000 7f810273b640 1 --2- 172.21.15.80:0/331524676 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f80fc102230 0x7f80fc19ee10 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f80f400eaa0 tx=0x7f80f400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:59.394 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:59.389+0000 7f80f2ffd640 1 -- 172.21.15.80:0/331524676 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80f400cdb0 con 0x7f80fc102230 2024-04-24T23:26:59.394 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:59.389+0000 7f80f2ffd640 1 -- 172.21.15.80:0/331524676 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80f4005020 con 0x7f80fc102230 2024-04-24T23:26:59.394 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:59.389+0000 7f80f2ffd640 1 -- 172.21.15.80:0/331524676 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80f40116b0 con 0x7f80fc102230 2024-04-24T23:26:59.394 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:59.389+0000 7f810419e640 1 -- 172.21.15.80:0/331524676 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80fc19f9b0 con 0x7f80fc102230 2024-04-24T23:26:59.395 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:59.389+0000 7f810419e640 1 -- 172.21.15.80:0/331524676 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80fc19fed0 con 0x7f80fc102230 2024-04-24T23:26:59.395 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:59.391+0000 7f810419e640 1 -- 172.21.15.80:0/331524676 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80fc1079a0 con 0x7f80fc102230 2024-04-24T23:26:59.396 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:59.391+0000 7f80f2ffd640 1 -- 172.21.15.80:0/331524676 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f80f4005560 con 0x7f80fc102230 2024-04-24T23:26:59.396 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:59.392+0000 7f80f2ffd640 1 --2- 172.21.15.80:0/331524676 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f80e80a2b90 0x7f80e80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:59.396 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:59.392+0000 7f8101f3a640 1 --2- 172.21.15.80:0/331524676 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f80e80a2b90 0x7f80e80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:59.396 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:59.392+0000 7f80f2ffd640 1 -- 172.21.15.80:0/331524676 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f80f40cedd0 con 0x7f80fc102230 2024-04-24T23:26:59.396 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:59.392+0000 7f8101f3a640 1 --2- 172.21.15.80:0/331524676 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f80e80a2b90 0x7f80e80a5050 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f80fc1016d0 tx=0x7f80ec005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:59.397 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:59.396+0000 7f80f2ffd640 1 -- 172.21.15.80:0/331524676 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f80f4016030 con 0x7f80fc102230 2024-04-24T23:26:59.617 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:59.616+0000 7f810419e640 1 -- 172.21.15.80:0/331524676 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f80fc062340 con 0x7f80fc102230 2024-04-24T23:26:59.618 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:59.617+0000 7f80f2ffd640 1 -- 172.21.15.80:0/331524676 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6346 (secure 0 0 0) 0x7f80f409c1d0 con 0x7f80fc102230 2024-04-24T23:26:59.618 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-24T23:26:59.618 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/891591085","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":891591085},{"type":"v1","addr":"172.21.15.80:6835","nonce":891591085}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6837/259529026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6836","nonce":259529026},{"type":"v1","addr":"172.21.15.80:6837","nonce":259529026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24395,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/3736784385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3736784385},{"type":"v1","addr":"172.21.15.134:6833","nonce":3736784385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/124777462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":124777462},{"type":"v1","addr":"172.21.15.148:6833","nonce":124777462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/1399801120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1399801120},{"type":"v1","addr":"172.21.15.134:6835","nonce":1399801120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24460,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/3002319310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3002319310},{"type":"v1","addr":"172.21.15.148:6835","nonce":3002319310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24472,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/856528403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":856528403},{"type":"v1","addr":"172.21.15.148:6837","nonce":856528403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-04-24T23:26:59.618 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 8 2024-04-24T23:26:59.620 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:59.619+0000 7f810419e640 1 -- 172.21.15.80:0/331524676 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f80e80a2b90 msgr2=0x7f80e80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:59.620 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:59.619+0000 7f810419e640 1 --2- 172.21.15.80:0/331524676 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f80e80a2b90 0x7f80e80a5050 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f80fc1016d0 tx=0x7f80ec005f90 comp rx=0 tx=0).stop 2024-04-24T23:26:59.620 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:59.619+0000 7f810419e640 1 -- 172.21.15.80:0/331524676 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f80fc102230 msgr2=0x7f80fc19ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:59.620 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:59.619+0000 7f810419e640 1 --2- 172.21.15.80:0/331524676 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f80fc102230 0x7f80fc19ee10 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f80f400eaa0 tx=0x7f80f400ef60 comp rx=0 tx=0).stop 2024-04-24T23:26:59.620 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:59.619+0000 7f810419e640 1 -- 172.21.15.80:0/331524676 shutdown_connections 2024-04-24T23:26:59.620 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:59.619+0000 7f810419e640 1 --2- 172.21.15.80:0/331524676 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f80e80a2b90 0x7f80e80a5050 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:59.620 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:59.619+0000 7f810419e640 1 --2- 172.21.15.80:0/331524676 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f80fc102230 0x7f80fc19ee10 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:59.621 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:59.619+0000 7f810419e640 1 --2- 172.21.15.80:0/331524676 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f80fc101870 0x7f80fc19c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:59.621 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:59.619+0000 7f810419e640 1 --2- 172.21.15.80:0/331524676 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f80fc100680 0x7f80fc069440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:59.621 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:59.619+0000 7f810419e640 1 -- 172.21.15.80:0/331524676 >> 172.21.15.80:0/331524676 conn(0x7f80fc0fbbf0 msgr2=0x7f80fc104b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:59.621 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:59.619+0000 7f810419e640 1 -- 172.21.15.80:0/331524676 shutdown_connections 2024-04-24T23:26:59.621 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:26:59.619+0000 7f810419e640 1 -- 172.21.15.80:0/331524676 wait complete. 2024-04-24T23:27:00.038 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:59 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:27:00.038 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:59 smithi080 ceph-mon[29152]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:27:00.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:59 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:27:00.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:59 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:27:00.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:59 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:27:00.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:59 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:27:00.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:59 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:27:00.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:59 smithi080 ceph-mon[29152]: Deploying daemon mds.h on smithi134 2024-04-24T23:27:00.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:59 smithi080 ceph-mon[29152]: mds.? [v2:172.21.15.148:6836/856528403,v1:172.21.15.148:6837/856528403] up:boot 2024-04-24T23:27:00.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:59 smithi080 ceph-mon[29152]: fsmap 7 up:standby 2024-04-24T23:27:00.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:59 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-24T23:27:00.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:26:59 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/331524676' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:27:00.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:59 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:27:00.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:59 smithi134 ceph-mon[32619]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:27:00.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:59 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:27:00.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:59 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:27:00.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:59 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:27:00.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:59 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:27:00.199 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:59 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:27:00.199 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:59 smithi134 ceph-mon[32619]: Deploying daemon mds.h on smithi134 2024-04-24T23:27:00.199 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:59 smithi134 ceph-mon[32619]: mds.? [v2:172.21.15.148:6836/856528403,v1:172.21.15.148:6837/856528403] up:boot 2024-04-24T23:27:00.199 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:59 smithi134 ceph-mon[32619]: fsmap 7 up:standby 2024-04-24T23:27:00.199 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:59 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-24T23:27:00.199 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:26:59 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/331524676' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:27:00.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:59 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:27:00.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:59 smithi148 ceph-mon[32528]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:27:00.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:59 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:27:00.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:59 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:27:00.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:59 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:27:00.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:59 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:27:00.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:59 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:27:00.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:59 smithi148 ceph-mon[32528]: Deploying daemon mds.h on smithi134 2024-04-24T23:27:00.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:59 smithi148 ceph-mon[32528]: mds.? [v2:172.21.15.148:6836/856528403,v1:172.21.15.148:6837/856528403] up:boot 2024-04-24T23:27:00.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:59 smithi148 ceph-mon[32528]: fsmap 7 up:standby 2024-04-24T23:27:00.258 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:59 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-24T23:27:00.258 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:26:59 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/331524676' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:27:01.383 INFO:journalctl@ceph.mds.h.smithi134.stdout:Apr 24 23:27:01 smithi134 systemd[1]: Starting Ceph mds.h for ec7885aa-0290-11ef-bc93-c7b262605968... 2024-04-24T23:27:01.632 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:27:01.671 INFO:journalctl@ceph.mds.h.smithi134.stdout:Apr 24 23:27:01 smithi134 podman[63082]: 2024-04-24 23:27:01.284916378 +0000 UTC m=+0.018647155 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:27:01.671 INFO:journalctl@ceph.mds.h.smithi134.stdout:Apr 24 23:27:01 smithi134 podman[63082]: 2024-04-24 23:27:01.40220931 +0000 UTC m=+0.135940077 container create d8e5842d437bad8b290cb0569bde221c5226d5c7e83b263ee2813388e3a36316 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-h, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-24T23:27:01.671 INFO:journalctl@ceph.mds.h.smithi134.stdout:Apr 24 23:27:01 smithi134 podman[63082]: 2024-04-24 23:27:01.671503212 +0000 UTC m=+0.405233978 container init d8e5842d437bad8b290cb0569bde221c5226d5c7e83b263ee2813388e3a36316 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-h, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True) 2024-04-24T23:27:01.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:01.740+0000 7efca0015640 1 -- 172.21.15.80:0/2670639326 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efc98101870 msgr2=0x7efc98101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:01.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:01.740+0000 7efca0015640 1 --2- 172.21.15.80:0/2670639326 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efc98101870 0x7efc98101cf0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7efc88009900 tx=0x7efc8802a410 comp rx=0 tx=0).stop 2024-04-24T23:27:01.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:01.740+0000 7efca0015640 1 -- 172.21.15.80:0/2670639326 shutdown_connections 2024-04-24T23:27:01.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:01.740+0000 7efca0015640 1 --2- 172.21.15.80:0/2670639326 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efc98102230 0x7efc98108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:01.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:01.740+0000 7efca0015640 1 --2- 172.21.15.80:0/2670639326 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efc98101870 0x7efc98101cf0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:01.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:01.740+0000 7efca0015640 1 --2- 172.21.15.80:0/2670639326 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc98100680 0x7efc98100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:01.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:01.740+0000 7efca0015640 1 -- 172.21.15.80:0/2670639326 >> 172.21.15.80:0/2670639326 conn(0x7efc980fbbf0 msgr2=0x7efc980fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:01.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:01.740+0000 7efca0015640 1 -- 172.21.15.80:0/2670639326 shutdown_connections 2024-04-24T23:27:01.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:01.740+0000 7efca0015640 1 -- 172.21.15.80:0/2670639326 wait complete. 2024-04-24T23:27:01.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:01.741+0000 7efca0015640 1 Processor -- start 2024-04-24T23:27:01.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:01.741+0000 7efca0015640 1 -- start start 2024-04-24T23:27:01.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:01.741+0000 7efca0015640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efc98100680 0x7efc98196980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:01.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:01.741+0000 7efca0015640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efc98101870 0x7efc98196ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:01.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:01.741+0000 7efca0015640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc98102230 0x7efc9819df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:01.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:01.741+0000 7efca0015640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7efc9810c300 con 0x7efc98100680 2024-04-24T23:27:01.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:01.741+0000 7efca0015640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7efc9810c180 con 0x7efc98101870 2024-04-24T23:27:01.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:01.741+0000 7efca0015640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7efc9810c480 con 0x7efc98102230 2024-04-24T23:27:01.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:01.741+0000 7efc9ddb1640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efc98100680 0x7efc98196980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:01.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:01.741+0000 7efc9ddb1640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efc98100680 0x7efc98196980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:39560/0 (socket says 172.21.15.80:39560) 2024-04-24T23:27:01.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:01.741+0000 7efc9ddb1640 1 -- 172.21.15.80:0/3482129976 learned_addr learned my addr 172.21.15.80:0/3482129976 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:27:01.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:01.741+0000 7efc9d5b0640 1 --2- 172.21.15.80:0/3482129976 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efc98101870 0x7efc98196ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:01.746 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:01.741+0000 7efc9e5b2640 1 --2- 172.21.15.80:0/3482129976 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc98102230 0x7efc9819df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:01.746 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:01.741+0000 7efc9ddb1640 1 -- 172.21.15.80:0/3482129976 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc98102230 msgr2=0x7efc9819df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:01.746 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:01.741+0000 7efc9ddb1640 1 --2- 172.21.15.80:0/3482129976 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc98102230 0x7efc9819df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:01.746 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:01.741+0000 7efc9ddb1640 1 -- 172.21.15.80:0/3482129976 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efc98101870 msgr2=0x7efc98196ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:01.746 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:01.741+0000 7efc9ddb1640 1 --2- 172.21.15.80:0/3482129976 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efc98101870 0x7efc98196ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:01.746 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:01.741+0000 7efc9ddb1640 1 -- 172.21.15.80:0/3482129976 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc9819e5a0 con 0x7efc98100680 2024-04-24T23:27:01.746 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:01.741+0000 7efc9e5b2640 1 --2- 172.21.15.80:0/3482129976 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc98102230 0x7efc9819df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:27:01.746 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:01.742+0000 7efc9ddb1640 1 --2- 172.21.15.80:0/3482129976 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efc98100680 0x7efc98196980 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7efc9400d950 tx=0x7efc9400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:01.746 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:01.742+0000 7efc8effd640 1 -- 172.21.15.80:0/3482129976 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc94017070 con 0x7efc98100680 2024-04-24T23:27:01.746 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:01.742+0000 7efc8effd640 1 -- 172.21.15.80:0/3482129976 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc94009c30 con 0x7efc98100680 2024-04-24T23:27:01.746 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:01.742+0000 7efca0015640 1 -- 172.21.15.80:0/3482129976 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc9819e890 con 0x7efc98100680 2024-04-24T23:27:01.746 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:01.742+0000 7efc8effd640 1 -- 172.21.15.80:0/3482129976 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc9400f600 con 0x7efc98100680 2024-04-24T23:27:01.746 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:01.742+0000 7efca0015640 1 -- 172.21.15.80:0/3482129976 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc9819ed60 con 0x7efc98100680 2024-04-24T23:27:01.746 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:01.743+0000 7efca0015640 1 -- 172.21.15.80:0/3482129976 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc98107a20 con 0x7efc98100680 2024-04-24T23:27:01.747 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:01.744+0000 7efc8effd640 1 -- 172.21.15.80:0/3482129976 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7efc9401a070 con 0x7efc98100680 2024-04-24T23:27:01.747 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:01.744+0000 7efc8effd640 1 --2- 172.21.15.80:0/3482129976 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7efc840a2b90 0x7efc840a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:01.747 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:01.745+0000 7efc8effd640 1 -- 172.21.15.80:0/3482129976 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7efc940ce6d0 con 0x7efc98100680 2024-04-24T23:27:01.747 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:01.745+0000 7efc9d5b0640 1 --2- 172.21.15.80:0/3482129976 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7efc840a2b90 0x7efc840a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:01.747 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:01.745+0000 7efc9d5b0640 1 --2- 172.21.15.80:0/3482129976 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7efc840a2b90 0x7efc840a5050 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7efc88000c00 tx=0x7efc880315d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:01.749 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:01.748+0000 7efc8effd640 1 -- 172.21.15.80:0/3482129976 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7efc94009de0 con 0x7efc98100680 2024-04-24T23:27:01.947 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:01 smithi134 ceph-mon[32619]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:27:01.947 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:01 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:27:01.947 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:01 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:27:01.947 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:01 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:27:01.947 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:01 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:27:01.948 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:01 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:27:01.948 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:01 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:27:01.948 INFO:journalctl@ceph.mds.h.smithi134.stdout:Apr 24 23:27:01 smithi134 podman[63082]: 2024-04-24 23:27:01.674108041 +0000 UTC m=+0.407838815 container start d8e5842d437bad8b290cb0569bde221c5226d5c7e83b263ee2813388e3a36316 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-h, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.vendor=CentOS) 2024-04-24T23:27:01.948 INFO:journalctl@ceph.mds.h.smithi134.stdout:Apr 24 23:27:01 smithi134 ceph-mds[63101]: -- 172.21.15.134:0/3337764283 <== mon.2 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557b320a5a40 con 0x557b32078c00 2024-04-24T23:27:01.948 INFO:journalctl@ceph.mds.h.smithi134.stdout:Apr 24 23:27:01 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-h[63097]: starting mds.h at 2024-04-24T23:27:01.948 INFO:journalctl@ceph.mds.h.smithi134.stdout:Apr 24 23:27:01 smithi134 bash[63082]: d8e5842d437bad8b290cb0569bde221c5226d5c7e83b263ee2813388e3a36316 2024-04-24T23:27:01.948 INFO:journalctl@ceph.mds.h.smithi134.stdout:Apr 24 23:27:01 smithi134 systemd[1]: Started Ceph mds.h for ec7885aa-0290-11ef-bc93-c7b262605968. 2024-04-24T23:27:01.969 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:01.968+0000 7efca0015640 1 -- 172.21.15.80:0/3482129976 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efc9802d170 con 0x7efc98100680 2024-04-24T23:27:01.970 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:01.969+0000 7efc8effd640 1 -- 172.21.15.80:0/3482129976 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7171 (secure 0 0 0) 0x7efc9409bad0 con 0x7efc98100680 2024-04-24T23:27:01.970 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-24T23:27:01.970 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/891591085","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":891591085},{"type":"v1","addr":"172.21.15.80:6835","nonce":891591085}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6837/259529026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6836","nonce":259529026},{"type":"v1","addr":"172.21.15.80:6837","nonce":259529026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24395,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/3736784385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3736784385},{"type":"v1","addr":"172.21.15.134:6833","nonce":3736784385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/124777462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":124777462},{"type":"v1","addr":"172.21.15.148:6833","nonce":124777462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/1399801120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1399801120},{"type":"v1","addr":"172.21.15.134:6835","nonce":1399801120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24458,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/430512704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":430512704},{"type":"v1","addr":"172.21.15.134:6837","nonce":430512704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24460,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/3002319310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3002319310},{"type":"v1","addr":"172.21.15.148:6835","nonce":3002319310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24472,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/856528403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":856528403},{"type":"v1","addr":"172.21.15.148:6837","nonce":856528403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-04-24T23:27:01.970 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 9 2024-04-24T23:27:01.972 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:01.971+0000 7efca0015640 1 -- 172.21.15.80:0/3482129976 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7efc840a2b90 msgr2=0x7efc840a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:01.972 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:01.971+0000 7efca0015640 1 --2- 172.21.15.80:0/3482129976 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7efc840a2b90 0x7efc840a5050 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7efc88000c00 tx=0x7efc880315d0 comp rx=0 tx=0).stop 2024-04-24T23:27:01.972 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:01.971+0000 7efca0015640 1 -- 172.21.15.80:0/3482129976 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efc98100680 msgr2=0x7efc98196980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:01.972 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:01.971+0000 7efca0015640 1 --2- 172.21.15.80:0/3482129976 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efc98100680 0x7efc98196980 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7efc9400d950 tx=0x7efc9400de10 comp rx=0 tx=0).stop 2024-04-24T23:27:01.972 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:01.971+0000 7efca0015640 1 -- 172.21.15.80:0/3482129976 shutdown_connections 2024-04-24T23:27:01.973 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:01.971+0000 7efca0015640 1 --2- 172.21.15.80:0/3482129976 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7efc840a2b90 0x7efc840a5050 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:01.973 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:01.971+0000 7efca0015640 1 --2- 172.21.15.80:0/3482129976 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc98102230 0x7efc9819df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:01.973 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:01.971+0000 7efca0015640 1 --2- 172.21.15.80:0/3482129976 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efc98101870 0x7efc98196ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:01.973 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:01.971+0000 7efca0015640 1 --2- 172.21.15.80:0/3482129976 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efc98100680 0x7efc98196980 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:01.973 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:01.971+0000 7efca0015640 1 -- 172.21.15.80:0/3482129976 >> 172.21.15.80:0/3482129976 conn(0x7efc980fbbf0 msgr2=0x7efc98104bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:01.973 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:01.971+0000 7efca0015640 1 -- 172.21.15.80:0/3482129976 shutdown_connections 2024-04-24T23:27:01.973 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:01.971+0000 7efca0015640 1 -- 172.21.15.80:0/3482129976 wait complete. 2024-04-24T23:27:01.981 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:01 smithi080 ceph-mon[29152]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:27:01.982 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:01 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:27:01.982 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:01 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:27:01.982 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:01 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:27:01.982 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:01 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:27:01.982 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:01 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:27:01.982 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:01 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:27:02.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:01 smithi148 ceph-mon[32528]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:27:02.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:01 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:27:02.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:01 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:27:02.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:01 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:27:02.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:01 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:27:02.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:01 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:27:02.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:01 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:27:03.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:02 smithi134 ceph-mon[32619]: Deploying daemon mds.i on smithi148 2024-04-24T23:27:03.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:02 smithi134 ceph-mon[32619]: mds.? [v2:172.21.15.134:6836/430512704,v1:172.21.15.134:6837/430512704] up:boot 2024-04-24T23:27:03.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:02 smithi134 ceph-mon[32619]: fsmap 8 up:standby 2024-04-24T23:27:03.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:02 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-24T23:27:03.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:02 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/3482129976' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:27:03.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:02 smithi148 ceph-mon[32528]: Deploying daemon mds.i on smithi148 2024-04-24T23:27:03.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:02 smithi148 ceph-mon[32528]: mds.? [v2:172.21.15.134:6836/430512704,v1:172.21.15.134:6837/430512704] up:boot 2024-04-24T23:27:03.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:02 smithi148 ceph-mon[32528]: fsmap 8 up:standby 2024-04-24T23:27:03.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:02 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-24T23:27:03.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:02 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/3482129976' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:27:03.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:02 smithi080 ceph-mon[29152]: Deploying daemon mds.i on smithi148 2024-04-24T23:27:03.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:02 smithi080 ceph-mon[29152]: mds.? [v2:172.21.15.134:6836/430512704,v1:172.21.15.134:6837/430512704] up:boot 2024-04-24T23:27:03.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:02 smithi080 ceph-mon[29152]: fsmap 8 up:standby 2024-04-24T23:27:03.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:02 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-24T23:27:03.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:02 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/3482129976' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:27:03.984 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:27:04.100 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:04.098+0000 7fee67ad4640 1 -- 172.21.15.80:0/3296673123 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fee60105320 msgr2=0x7fee601057a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:04.100 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:04.098+0000 7fee67ad4640 1 --2- 172.21.15.80:0/3296673123 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fee60105320 0x7fee601057a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fee58009ca0 tx=0x7fee5802a8c0 comp rx=0 tx=0).stop 2024-04-24T23:27:04.100 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:04.098+0000 7fee67ad4640 1 -- 172.21.15.80:0/3296673123 shutdown_connections 2024-04-24T23:27:04.100 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:04.098+0000 7fee67ad4640 1 --2- 172.21.15.80:0/3296673123 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fee60105320 0x7fee601057a0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:04.100 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:04.098+0000 7fee67ad4640 1 --2- 172.21.15.80:0/3296673123 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee60102970 0x7fee60104de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:04.100 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:04.098+0000 7fee67ad4640 1 --2- 172.21.15.80:0/3296673123 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fee60100000 0x7fee60102430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:04.100 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:04.098+0000 7fee67ad4640 1 -- 172.21.15.80:0/3296673123 >> 172.21.15.80:0/3296673123 conn(0x7fee600fbc20 msgr2=0x7fee600fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:04.100 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:04.098+0000 7fee67ad4640 1 -- 172.21.15.80:0/3296673123 shutdown_connections 2024-04-24T23:27:04.100 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:04.098+0000 7fee67ad4640 1 -- 172.21.15.80:0/3296673123 wait complete. 2024-04-24T23:27:04.101 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:04.098+0000 7fee67ad4640 1 Processor -- start 2024-04-24T23:27:04.101 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:04.098+0000 7fee67ad4640 1 -- start start 2024-04-24T23:27:04.101 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:04.099+0000 7fee67ad4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee60100000 0x7fee6019df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:04.101 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:04.099+0000 7fee67ad4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fee60102970 0x7fee6019e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:04.101 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:04.099+0000 7fee67ad4640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fee60105320 0x7fee601a5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:04.101 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:04.099+0000 7fee67ad4640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fee60113a50 con 0x7fee60105320 2024-04-24T23:27:04.101 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:04.099+0000 7fee67ad4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fee601138d0 con 0x7fee60102970 2024-04-24T23:27:04.101 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:04.099+0000 7fee67ad4640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fee60113bd0 con 0x7fee60100000 2024-04-24T23:27:04.101 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:04.099+0000 7fee66071640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fee60105320 0x7fee601a5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:04.101 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:04.099+0000 7fee66071640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fee60105320 0x7fee601a5530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:51714/0 (socket says 172.21.15.80:51714) 2024-04-24T23:27:04.101 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:04.099+0000 7fee66071640 1 -- 172.21.15.80:0/4218651883 learned_addr learned my addr 172.21.15.80:0/4218651883 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:27:04.101 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:04.099+0000 7fee65870640 1 --2- 172.21.15.80:0/4218651883 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee60100000 0x7fee6019df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:04.101 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:04.099+0000 7fee6506f640 1 --2- 172.21.15.80:0/4218651883 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fee60102970 0x7fee6019e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:04.101 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:04.099+0000 7fee66071640 1 -- 172.21.15.80:0/4218651883 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee60100000 msgr2=0x7fee6019df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:04.102 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:04.099+0000 7fee66071640 1 --2- 172.21.15.80:0/4218651883 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee60100000 0x7fee6019df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:04.102 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:04.099+0000 7fee66071640 1 -- 172.21.15.80:0/4218651883 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fee60102970 msgr2=0x7fee6019e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:04.102 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:04.099+0000 7fee66071640 1 --2- 172.21.15.80:0/4218651883 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fee60102970 0x7fee6019e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:04.102 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:04.099+0000 7fee66071640 1 -- 172.21.15.80:0/4218651883 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee601a5ba0 con 0x7fee60105320 2024-04-24T23:27:04.102 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:04.099+0000 7fee66071640 1 --2- 172.21.15.80:0/4218651883 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fee60105320 0x7fee601a5530 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fee580319f0 tx=0x7fee5802f450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:04.102 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:04.099+0000 7fee56ffd640 1 -- 172.21.15.80:0/4218651883 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee58003c70 con 0x7fee60105320 2024-04-24T23:27:04.102 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:04.099+0000 7fee56ffd640 1 -- 172.21.15.80:0/4218651883 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee58003e10 con 0x7fee60105320 2024-04-24T23:27:04.102 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:04.099+0000 7fee67ad4640 1 -- 172.21.15.80:0/4218651883 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee601a5e30 con 0x7fee60105320 2024-04-24T23:27:04.102 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:04.099+0000 7fee56ffd640 1 -- 172.21.15.80:0/4218651883 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee5802fcc0 con 0x7fee60105320 2024-04-24T23:27:04.102 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:04.100+0000 7fee67ad4640 1 -- 172.21.15.80:0/4218651883 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee601a62f0 con 0x7fee60105320 2024-04-24T23:27:04.103 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:04.101+0000 7fee67ad4640 1 -- 172.21.15.80:0/4218651883 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee600fd710 con 0x7fee60105320 2024-04-24T23:27:04.103 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:04.102+0000 7fee56ffd640 1 -- 172.21.15.80:0/4218651883 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fee58004b40 con 0x7fee60105320 2024-04-24T23:27:04.103 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:04.102+0000 7fee56ffd640 1 --2- 172.21.15.80:0/4218651883 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fee4c0a2b90 0x7fee4c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:04.103 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:04.102+0000 7fee56ffd640 1 -- 172.21.15.80:0/4218651883 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fee580f01b0 con 0x7fee60105320 2024-04-24T23:27:04.103 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:04.102+0000 7fee65870640 1 --2- 172.21.15.80:0/4218651883 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fee4c0a2b90 0x7fee4c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:04.103 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:04.103+0000 7fee65870640 1 --2- 172.21.15.80:0/4218651883 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fee4c0a2b90 0x7fee4c0a5050 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fee5c00abe0 tx=0x7fee5c00a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:04.106 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:04.106+0000 7fee56ffd640 1 -- 172.21.15.80:0/4218651883 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fee580bd5b0 con 0x7fee60105320 2024-04-24T23:27:04.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:03 smithi134 ceph-mon[32619]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:27:04.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:03 smithi148 ceph-mon[32528]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:27:04.256 INFO:journalctl@ceph.mds.i.smithi148.stdout:Apr 24 23:27:04 smithi148 systemd[1]: Starting Ceph mds.i for ec7885aa-0290-11ef-bc93-c7b262605968... 2024-04-24T23:27:04.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:03 smithi080 ceph-mon[29152]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:27:04.320 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:04.320+0000 7fee67ad4640 1 -- 172.21.15.80:0/4218651883 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fee6002d170 con 0x7fee60105320 2024-04-24T23:27:04.320 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:04.320+0000 7fee56ffd640 1 -- 172.21.15.80:0/4218651883 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7171 (secure 0 0 0) 0x7fee580bceb0 con 0x7fee60105320 2024-04-24T23:27:04.320 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-24T23:27:04.321 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/891591085","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":891591085},{"type":"v1","addr":"172.21.15.80:6835","nonce":891591085}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6837/259529026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6836","nonce":259529026},{"type":"v1","addr":"172.21.15.80:6837","nonce":259529026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24395,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/3736784385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3736784385},{"type":"v1","addr":"172.21.15.134:6833","nonce":3736784385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/124777462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":124777462},{"type":"v1","addr":"172.21.15.148:6833","nonce":124777462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/1399801120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1399801120},{"type":"v1","addr":"172.21.15.134:6835","nonce":1399801120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24458,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/430512704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":430512704},{"type":"v1","addr":"172.21.15.134:6837","nonce":430512704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24460,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/3002319310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3002319310},{"type":"v1","addr":"172.21.15.148:6835","nonce":3002319310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24472,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/856528403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":856528403},{"type":"v1","addr":"172.21.15.148:6837","nonce":856528403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-04-24T23:27:04.321 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 9 2024-04-24T23:27:04.323 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:04.322+0000 7fee67ad4640 1 -- 172.21.15.80:0/4218651883 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fee4c0a2b90 msgr2=0x7fee4c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:04.323 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:04.322+0000 7fee67ad4640 1 --2- 172.21.15.80:0/4218651883 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fee4c0a2b90 0x7fee4c0a5050 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fee5c00abe0 tx=0x7fee5c00a2d0 comp rx=0 tx=0).stop 2024-04-24T23:27:04.323 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:04.322+0000 7fee67ad4640 1 -- 172.21.15.80:0/4218651883 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fee60105320 msgr2=0x7fee601a5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:04.323 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:04.322+0000 7fee67ad4640 1 --2- 172.21.15.80:0/4218651883 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fee60105320 0x7fee601a5530 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fee580319f0 tx=0x7fee5802f450 comp rx=0 tx=0).stop 2024-04-24T23:27:04.323 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:04.322+0000 7fee67ad4640 1 -- 172.21.15.80:0/4218651883 shutdown_connections 2024-04-24T23:27:04.323 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:04.322+0000 7fee67ad4640 1 --2- 172.21.15.80:0/4218651883 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fee4c0a2b90 0x7fee4c0a5050 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:04.323 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:04.322+0000 7fee67ad4640 1 --2- 172.21.15.80:0/4218651883 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fee60105320 0x7fee601a5530 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:04.323 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:04.322+0000 7fee67ad4640 1 --2- 172.21.15.80:0/4218651883 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fee60102970 0x7fee6019e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:04.323 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:04.322+0000 7fee67ad4640 1 --2- 172.21.15.80:0/4218651883 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee60100000 0x7fee6019df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:04.323 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:04.322+0000 7fee67ad4640 1 -- 172.21.15.80:0/4218651883 >> 172.21.15.80:0/4218651883 conn(0x7fee600fbc20 msgr2=0x7fee60100980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:04.323 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:04.322+0000 7fee67ad4640 1 -- 172.21.15.80:0/4218651883 shutdown_connections 2024-04-24T23:27:04.323 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:04.322+0000 7fee67ad4640 1 -- 172.21.15.80:0/4218651883 wait complete. 2024-04-24T23:27:04.566 INFO:journalctl@ceph.mds.i.smithi148.stdout:Apr 24 23:27:04 smithi148 podman[63689]: 2024-04-24 23:27:04.187984126 +0000 UTC m=+0.018347299 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:27:04.566 INFO:journalctl@ceph.mds.i.smithi148.stdout:Apr 24 23:27:04 smithi148 podman[63689]: 2024-04-24 23:27:04.321494547 +0000 UTC m=+0.151857692 container create b42a579c659da5e4e47221006d17636bc576710c114efe4561f342011b020f33 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-i, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS) 2024-04-24T23:27:04.905 INFO:journalctl@ceph.mds.i.smithi148.stdout:Apr 24 23:27:04 smithi148 podman[63689]: 2024-04-24 23:27:04.566286268 +0000 UTC m=+0.396649423 container init b42a579c659da5e4e47221006d17636bc576710c114efe4561f342011b020f33 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-i, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0) 2024-04-24T23:27:04.905 INFO:journalctl@ceph.mds.i.smithi148.stdout:Apr 24 23:27:04 smithi148 podman[63689]: 2024-04-24 23:27:04.569264141 +0000 UTC m=+0.399627296 container start b42a579c659da5e4e47221006d17636bc576710c114efe4561f342011b020f33 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-i, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9) 2024-04-24T23:27:04.905 INFO:journalctl@ceph.mds.i.smithi148.stdout:Apr 24 23:27:04 smithi148 ceph-mds[63708]: -- 172.21.15.148:0/2126410579 <== mon.1 v2:172.21.15.148:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55bae4613a40 con 0x55bae45e6800 2024-04-24T23:27:04.905 INFO:journalctl@ceph.mds.i.smithi148.stdout:Apr 24 23:27:04 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-i[63704]: starting mds.i at 2024-04-24T23:27:04.905 INFO:journalctl@ceph.mds.i.smithi148.stdout:Apr 24 23:27:04 smithi148 bash[63689]: b42a579c659da5e4e47221006d17636bc576710c114efe4561f342011b020f33 2024-04-24T23:27:04.905 INFO:journalctl@ceph.mds.i.smithi148.stdout:Apr 24 23:27:04 smithi148 systemd[1]: Started Ceph mds.i for ec7885aa-0290-11ef-bc93-c7b262605968. 2024-04-24T23:27:05.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:04 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/4218651883' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:27:05.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:04 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:27:05.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:04 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:27:05.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:04 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:27:05.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:04 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:27:05.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:04 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:27:05.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:04 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:27:05.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:04 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/4218651883' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:27:05.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:04 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:27:05.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:04 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:27:05.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:04 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:27:05.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:04 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:27:05.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:04 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:27:05.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:04 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:27:05.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:04 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/4218651883' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:27:05.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:04 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:27:05.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:04 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:27:05.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:04 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:27:05.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:04 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:27:05.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:04 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:27:05.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:04 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:27:06.038 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:05 smithi080 ceph-mon[29152]: Deploying daemon mds.g on smithi080 2024-04-24T23:27:06.038 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:05 smithi080 ceph-mon[29152]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:27:06.038 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:05 smithi080 ceph-mon[29152]: mds.? [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] up:boot 2024-04-24T23:27:06.038 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:05 smithi080 ceph-mon[29152]: fsmap 9 up:standby 2024-04-24T23:27:06.038 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:05 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-24T23:27:06.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:05 smithi134 ceph-mon[32619]: Deploying daemon mds.g on smithi080 2024-04-24T23:27:06.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:05 smithi134 ceph-mon[32619]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:27:06.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:05 smithi134 ceph-mon[32619]: mds.? [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] up:boot 2024-04-24T23:27:06.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:05 smithi134 ceph-mon[32619]: fsmap 9 up:standby 2024-04-24T23:27:06.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:05 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-24T23:27:06.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:05 smithi148 ceph-mon[32528]: Deploying daemon mds.g on smithi080 2024-04-24T23:27:06.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:05 smithi148 ceph-mon[32528]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:27:06.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:05 smithi148 ceph-mon[32528]: mds.? [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] up:boot 2024-04-24T23:27:06.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:05 smithi148 ceph-mon[32528]: fsmap 9 up:standby 2024-04-24T23:27:06.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:05 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-24T23:27:06.334 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:27:06.437 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:06.436+0000 7fd039a5d640 1 -- 172.21.15.80:0/765761688 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd034103360 msgr2=0x7fd034109bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:06.437 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:06.436+0000 7fd039a5d640 1 --2- 172.21.15.80:0/765761688 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd034103360 0x7fd034109bf0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd02c009ca0 tx=0x7fd02c02a8c0 comp rx=0 tx=0).stop 2024-04-24T23:27:06.437 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:06.436+0000 7fd039a5d640 1 -- 172.21.15.80:0/765761688 shutdown_connections 2024-04-24T23:27:06.437 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:06.436+0000 7fd039a5d640 1 --2- 172.21.15.80:0/765761688 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd034103360 0x7fd034109bf0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:06.437 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:06.436+0000 7fd039a5d640 1 --2- 172.21.15.80:0/765761688 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd0341029a0 0x7fd034102e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:06.437 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:06.436+0000 7fd039a5d640 1 --2- 172.21.15.80:0/765761688 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0341017b0 0x7fd034101bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:06.437 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:06.436+0000 7fd039a5d640 1 -- 172.21.15.80:0/765761688 >> 172.21.15.80:0/765761688 conn(0x7fd0340fcd20 msgr2=0x7fd0340ff160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:06.437 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:06.436+0000 7fd039a5d640 1 -- 172.21.15.80:0/765761688 shutdown_connections 2024-04-24T23:27:06.438 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:06.436+0000 7fd039a5d640 1 -- 172.21.15.80:0/765761688 wait complete. 2024-04-24T23:27:06.438 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:06.437+0000 7fd039a5d640 1 Processor -- start 2024-04-24T23:27:06.440 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:06.437+0000 7fd039a5d640 1 -- start start 2024-04-24T23:27:06.440 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:06.437+0000 7fd039a5d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd0341017b0 0x7fd034197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:06.440 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:06.438+0000 7fd039a5d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0341029a0 0x7fd034197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:06.440 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:06.438+0000 7fd039a5d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd034103360 0x7fd03419f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:06.440 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:06.438+0000 7fd039a5d640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fd03410d490 con 0x7fd034103360 2024-04-24T23:27:06.440 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:06.438+0000 7fd039a5d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd03410d310 con 0x7fd0341029a0 2024-04-24T23:27:06.440 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:06.438+0000 7fd039a5d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd03410d610 con 0x7fd0341017b0 2024-04-24T23:27:06.440 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:06.438+0000 7fd0337fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd034103360 0x7fd03419f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:06.440 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:06.438+0000 7fd0337fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd034103360 0x7fd03419f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:51722/0 (socket says 172.21.15.80:51722) 2024-04-24T23:27:06.440 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:06.438+0000 7fd0337fe640 1 -- 172.21.15.80:0/139784949 learned_addr learned my addr 172.21.15.80:0/139784949 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:27:06.440 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:06.438+0000 7fd032ffd640 1 --2- 172.21.15.80:0/139784949 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd0341017b0 0x7fd034197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:06.441 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:06.438+0000 7fd0327fc640 1 --2- 172.21.15.80:0/139784949 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0341029a0 0x7fd034197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:06.441 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:06.438+0000 7fd0337fe640 1 -- 172.21.15.80:0/139784949 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd0341017b0 msgr2=0x7fd034197a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:06.441 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:06.438+0000 7fd0337fe640 1 --2- 172.21.15.80:0/139784949 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd0341017b0 0x7fd034197a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:06.441 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:06.438+0000 7fd0337fe640 1 -- 172.21.15.80:0/139784949 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0341029a0 msgr2=0x7fd034197fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:06.441 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:06.438+0000 7fd0337fe640 1 --2- 172.21.15.80:0/139784949 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0341029a0 0x7fd034197fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:06.441 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:06.438+0000 7fd0337fe640 1 -- 172.21.15.80:0/139784949 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd03419f6a0 con 0x7fd034103360 2024-04-24T23:27:06.441 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:06.438+0000 7fd0337fe640 1 --2- 172.21.15.80:0/139784949 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd034103360 0x7fd03419f030 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd02c005e30 tx=0x7fd02c031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:06.441 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:06.438+0000 7fd038a5b640 1 -- 172.21.15.80:0/139784949 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd02c003c30 con 0x7fd034103360 2024-04-24T23:27:06.441 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:06.438+0000 7fd039a5d640 1 -- 172.21.15.80:0/139784949 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd03419f930 con 0x7fd034103360 2024-04-24T23:27:06.441 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:06.438+0000 7fd038a5b640 1 -- 172.21.15.80:0/139784949 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd02c003dd0 con 0x7fd034103360 2024-04-24T23:27:06.441 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:06.438+0000 7fd038a5b640 1 -- 172.21.15.80:0/139784949 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd02c02fde0 con 0x7fd034103360 2024-04-24T23:27:06.441 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:06.438+0000 7fd039a5d640 1 -- 172.21.15.80:0/139784949 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd03419ff50 con 0x7fd034103360 2024-04-24T23:27:06.441 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:06.439+0000 7fd039a5d640 1 -- 172.21.15.80:0/139784949 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd034108a70 con 0x7fd034103360 2024-04-24T23:27:06.442 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:06.440+0000 7fd038a5b640 1 -- 172.21.15.80:0/139784949 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fd02c004050 con 0x7fd034103360 2024-04-24T23:27:06.442 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:06.441+0000 7fd038a5b640 1 --2- 172.21.15.80:0/139784949 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fd01c0a2b90 0x7fd01c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:06.442 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:06.441+0000 7fd032ffd640 1 --2- 172.21.15.80:0/139784949 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fd01c0a2b90 0x7fd01c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:06.443 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:06.441+0000 7fd038a5b640 1 -- 172.21.15.80:0/139784949 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fd02c0f01e0 con 0x7fd034103360 2024-04-24T23:27:06.443 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:06.441+0000 7fd032ffd640 1 --2- 172.21.15.80:0/139784949 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fd01c0a2b90 0x7fd01c0a5050 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd024005fd0 tx=0x7fd024005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:06.445 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:06.445+0000 7fd038a5b640 1 -- 172.21.15.80:0/139784949 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fd02c0bd5e0 con 0x7fd034103360 2024-04-24T23:27:06.670 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:06.670+0000 7fd039a5d640 1 -- 172.21.15.80:0/139784949 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd03402d170 con 0x7fd034103360 2024-04-24T23:27:06.670 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:06.670+0000 7fd038a5b640 1 -- 172.21.15.80:0/139784949 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8001 (secure 0 0 0) 0x7fd02c0bcee0 con 0x7fd034103360 2024-04-24T23:27:06.671 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-24T23:27:06.671 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/891591085","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":891591085},{"type":"v1","addr":"172.21.15.80:6835","nonce":891591085}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6837/259529026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6836","nonce":259529026},{"type":"v1","addr":"172.21.15.80:6837","nonce":259529026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24395,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/3736784385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3736784385},{"type":"v1","addr":"172.21.15.134:6833","nonce":3736784385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/124777462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":124777462},{"type":"v1","addr":"172.21.15.148:6833","nonce":124777462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/1399801120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1399801120},{"type":"v1","addr":"172.21.15.134:6835","nonce":1399801120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24458,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/430512704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":430512704},{"type":"v1","addr":"172.21.15.134:6837","nonce":430512704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24460,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/3002319310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3002319310},{"type":"v1","addr":"172.21.15.148:6835","nonce":3002319310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24472,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/856528403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":856528403},{"type":"v1","addr":"172.21.15.148:6837","nonce":856528403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24496,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6839/2952757594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":2952757594},{"type":"v1","addr":"172.21.15.148:6839","nonce":2952757594}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-24T23:27:06.671 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 10 2024-04-24T23:27:06.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:06.672+0000 7fd039a5d640 1 -- 172.21.15.80:0/139784949 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fd01c0a2b90 msgr2=0x7fd01c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:06.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:06.672+0000 7fd039a5d640 1 --2- 172.21.15.80:0/139784949 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fd01c0a2b90 0x7fd01c0a5050 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd024005fd0 tx=0x7fd024005850 comp rx=0 tx=0).stop 2024-04-24T23:27:06.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:06.672+0000 7fd039a5d640 1 -- 172.21.15.80:0/139784949 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd034103360 msgr2=0x7fd03419f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:06.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:06.672+0000 7fd039a5d640 1 --2- 172.21.15.80:0/139784949 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd034103360 0x7fd03419f030 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd02c005e30 tx=0x7fd02c031ee0 comp rx=0 tx=0).stop 2024-04-24T23:27:06.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:06.672+0000 7fd039a5d640 1 -- 172.21.15.80:0/139784949 shutdown_connections 2024-04-24T23:27:06.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:06.672+0000 7fd039a5d640 1 --2- 172.21.15.80:0/139784949 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fd01c0a2b90 0x7fd01c0a5050 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:06.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:06.672+0000 7fd039a5d640 1 --2- 172.21.15.80:0/139784949 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd034103360 0x7fd03419f030 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:06.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:06.672+0000 7fd039a5d640 1 --2- 172.21.15.80:0/139784949 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0341029a0 0x7fd034197fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:06.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:06.672+0000 7fd039a5d640 1 --2- 172.21.15.80:0/139784949 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd0341017b0 0x7fd034197a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:06.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:06.672+0000 7fd039a5d640 1 -- 172.21.15.80:0/139784949 >> 172.21.15.80:0/139784949 conn(0x7fd0340fcd20 msgr2=0x7fd034105c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:06.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:06.673+0000 7fd039a5d640 1 -- 172.21.15.80:0/139784949 shutdown_connections 2024-04-24T23:27:06.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:06.673+0000 7fd039a5d640 1 -- 172.21.15.80:0/139784949 wait complete. 2024-04-24T23:27:07.038 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:06 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/139784949' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:27:07.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:06 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/139784949' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:27:07.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:06 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/139784949' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:27:07.788 INFO:journalctl@ceph.mds.g.smithi080.stdout:Apr 24 23:27:07 smithi080 systemd[1]: Started Ceph mds.g for ec7885aa-0290-11ef-bc93-c7b262605968. 2024-04-24T23:27:08.172 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:07 smithi148 ceph-mon[32528]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:27:08.172 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:07 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:27:08.172 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:07 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:27:08.172 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:07 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:27:08.172 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:07 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:27:08.172 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:07 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:27:08.172 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:07 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:27:08.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:07 smithi134 ceph-mon[32619]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:27:08.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:07 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:27:08.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:07 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:27:08.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:07 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:27:08.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:07 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:27:08.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:07 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:27:08.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:07 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:27:08.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:07 smithi080 ceph-mon[29152]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:27:08.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:07 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:27:08.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:07 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:27:08.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:07 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:27:08.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:07 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:27:08.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:07 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:27:08.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:07 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:27:08.685 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:27:08.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:08.787+0000 7f21799b0640 1 -- 172.21.15.80:0/1625521513 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2174103350 msgr2=0x7f2174109be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:08.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:08.787+0000 7f21799b0640 1 --2- 172.21.15.80:0/1625521513 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2174103350 0x7f2174109be0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f216c009ca0 tx=0x7f216c02a8c0 comp rx=0 tx=0).stop 2024-04-24T23:27:08.790 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:08.787+0000 7f21799b0640 1 -- 172.21.15.80:0/1625521513 shutdown_connections 2024-04-24T23:27:08.790 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:08.787+0000 7f21799b0640 1 --2- 172.21.15.80:0/1625521513 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2174103350 0x7f2174109be0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:08.790 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:08.787+0000 7f21799b0640 1 --2- 172.21.15.80:0/1625521513 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2174102990 0x7f2174102e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:08.790 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:08.787+0000 7f21799b0640 1 --2- 172.21.15.80:0/1625521513 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f21741017a0 0x7f2174101bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:08.790 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:08.787+0000 7f21799b0640 1 -- 172.21.15.80:0/1625521513 >> 172.21.15.80:0/1625521513 conn(0x7f21740fccf0 msgr2=0x7f21740ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:08.790 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:08.787+0000 7f21799b0640 1 -- 172.21.15.80:0/1625521513 shutdown_connections 2024-04-24T23:27:08.790 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:08.787+0000 7f21799b0640 1 -- 172.21.15.80:0/1625521513 wait complete. 2024-04-24T23:27:08.790 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:08.788+0000 7f21799b0640 1 Processor -- start 2024-04-24T23:27:08.790 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:08.788+0000 7f21799b0640 1 -- start start 2024-04-24T23:27:08.790 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:08.788+0000 7f21799b0640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f21741017a0 0x7f2174197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:08.790 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:08.788+0000 7f21799b0640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2174102990 0x7f2174197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:08.790 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:08.788+0000 7f21799b0640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2174103350 0x7f217419f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:08.790 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:08.788+0000 7f21799b0640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f217410d420 con 0x7f21741017a0 2024-04-24T23:27:08.791 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:08.788+0000 7f21799b0640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f217410d2a0 con 0x7f2174103350 2024-04-24T23:27:08.791 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:08.788+0000 7f21799b0640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f217410d5a0 con 0x7f2174102990 2024-04-24T23:27:08.791 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:08.788+0000 7f2172ffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f21741017a0 0x7f2174197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:08.791 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:08.788+0000 7f2172ffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f21741017a0 0x7f2174197a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:51752/0 (socket says 172.21.15.80:51752) 2024-04-24T23:27:08.791 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:08.788+0000 7f2172ffd640 1 -- 172.21.15.80:0/496590691 learned_addr learned my addr 172.21.15.80:0/496590691 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:27:08.791 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:08.788+0000 7f216bfff640 1 --2- 172.21.15.80:0/496590691 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2174102990 0x7f2174197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:08.791 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:08.788+0000 7f2172ffd640 1 -- 172.21.15.80:0/496590691 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2174102990 msgr2=0x7f2174197fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:08.791 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:08.788+0000 7f2172ffd640 1 --2- 172.21.15.80:0/496590691 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2174102990 0x7f2174197fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:08.791 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:08.788+0000 7f2172ffd640 1 -- 172.21.15.80:0/496590691 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2174103350 msgr2=0x7f217419f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:08.791 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:08.788+0000 7f2172ffd640 1 --2- 172.21.15.80:0/496590691 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2174103350 0x7f217419f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:08.791 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:08.788+0000 7f2172ffd640 1 -- 172.21.15.80:0/496590691 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f217419f680 con 0x7f21741017a0 2024-04-24T23:27:08.791 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:08.788+0000 7f2172ffd640 1 --2- 172.21.15.80:0/496590691 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f21741017a0 0x7f2174197a60 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f216400c970 tx=0x7f216400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:08.791 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:08.788+0000 7f21789ae640 1 -- 172.21.15.80:0/496590691 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2164004220 con 0x7f21741017a0 2024-04-24T23:27:08.791 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:08.789+0000 7f21789ae640 1 -- 172.21.15.80:0/496590691 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21640043c0 con 0x7f21741017a0 2024-04-24T23:27:08.792 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:08.789+0000 7f21799b0640 1 -- 172.21.15.80:0/496590691 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f217419f970 con 0x7f21741017a0 2024-04-24T23:27:08.792 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:08.789+0000 7f21789ae640 1 -- 172.21.15.80:0/496590691 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f216400f450 con 0x7f21741017a0 2024-04-24T23:27:08.792 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:08.789+0000 7f21799b0640 1 -- 172.21.15.80:0/496590691 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f217419fec0 con 0x7f21741017a0 2024-04-24T23:27:08.792 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:08.789+0000 7f21799b0640 1 -- 172.21.15.80:0/496590691 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2174068560 con 0x7f21741017a0 2024-04-24T23:27:08.793 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:08.790+0000 7f21789ae640 1 -- 172.21.15.80:0/496590691 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f21640079b0 con 0x7f21741017a0 2024-04-24T23:27:08.793 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:08.791+0000 7f21789ae640 1 --2- 172.21.15.80:0/496590691 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f21600a2b90 0x7f21600a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:08.793 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:08.791+0000 7f216bfff640 1 --2- 172.21.15.80:0/496590691 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f21600a2b90 0x7f21600a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:08.793 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:08.791+0000 7f21789ae640 1 -- 172.21.15.80:0/496590691 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f21640ce930 con 0x7f21741017a0 2024-04-24T23:27:08.793 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:08.792+0000 7f216bfff640 1 --2- 172.21.15.80:0/496590691 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f21600a2b90 0x7f21600a5050 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2174198f70 tx=0x7f215c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:08.796 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:08.796+0000 7f21789ae640 1 -- 172.21.15.80:0/496590691 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f2164014030 con 0x7f21741017a0 2024-04-24T23:27:09.015 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:09.015+0000 7f21799b0640 1 -- 172.21.15.80:0/496590691 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f217402d240 con 0x7f21741017a0 2024-04-24T23:27:09.015 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:09.015+0000 7f21789ae640 1 -- 172.21.15.80:0/496590691 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8827 (secure 0 0 0) 0x7f216409bd30 con 0x7f21741017a0 2024-04-24T23:27:09.015 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-24T23:27:09.016 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/891591085","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":891591085},{"type":"v1","addr":"172.21.15.80:6835","nonce":891591085}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6837/259529026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6836","nonce":259529026},{"type":"v1","addr":"172.21.15.80:6837","nonce":259529026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14646,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6839/4194284854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6838","nonce":4194284854},{"type":"v1","addr":"172.21.15.80:6839","nonce":4194284854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24395,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/3736784385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3736784385},{"type":"v1","addr":"172.21.15.134:6833","nonce":3736784385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/124777462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":124777462},{"type":"v1","addr":"172.21.15.148:6833","nonce":124777462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/1399801120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1399801120},{"type":"v1","addr":"172.21.15.134:6835","nonce":1399801120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24458,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/430512704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":430512704},{"type":"v1","addr":"172.21.15.134:6837","nonce":430512704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24460,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/3002319310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3002319310},{"type":"v1","addr":"172.21.15.148:6835","nonce":3002319310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24472,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/856528403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":856528403},{"type":"v1","addr":"172.21.15.148:6837","nonce":856528403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24496,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6839/2952757594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":2952757594},{"type":"v1","addr":"172.21.15.148:6839","nonce":2952757594}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-24T23:27:09.016 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 11 2024-04-24T23:27:09.017 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:09.017+0000 7f21799b0640 1 -- 172.21.15.80:0/496590691 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f21600a2b90 msgr2=0x7f21600a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:09.018 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:09.017+0000 7f21799b0640 1 --2- 172.21.15.80:0/496590691 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f21600a2b90 0x7f21600a5050 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2174198f70 tx=0x7f215c005f90 comp rx=0 tx=0).stop 2024-04-24T23:27:09.018 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:09.017+0000 7f21799b0640 1 -- 172.21.15.80:0/496590691 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f21741017a0 msgr2=0x7f2174197a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:09.018 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:09.017+0000 7f21799b0640 1 --2- 172.21.15.80:0/496590691 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f21741017a0 0x7f2174197a60 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f216400c970 tx=0x7f216400ce30 comp rx=0 tx=0).stop 2024-04-24T23:27:09.018 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:09.017+0000 7f21799b0640 1 -- 172.21.15.80:0/496590691 shutdown_connections 2024-04-24T23:27:09.018 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:09.017+0000 7f21799b0640 1 --2- 172.21.15.80:0/496590691 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f21600a2b90 0x7f21600a5050 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:09.018 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:09.017+0000 7f21799b0640 1 --2- 172.21.15.80:0/496590691 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2174103350 0x7f217419f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:09.018 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:09.017+0000 7f21799b0640 1 --2- 172.21.15.80:0/496590691 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2174102990 0x7f2174197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:09.018 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:09.017+0000 7f21799b0640 1 --2- 172.21.15.80:0/496590691 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f21741017a0 0x7f2174197a60 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:09.018 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:09.017+0000 7f21799b0640 1 -- 172.21.15.80:0/496590691 >> 172.21.15.80:0/496590691 conn(0x7f21740fccf0 msgr2=0x7f2174105c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:09.018 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:09.017+0000 7f21799b0640 1 -- 172.21.15.80:0/496590691 shutdown_connections 2024-04-24T23:27:09.018 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:09.017+0000 7f21799b0640 1 -- 172.21.15.80:0/496590691 wait complete. 2024-04-24T23:27:09.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:08 smithi134 ceph-mon[32619]: Deploying daemon mds.a on smithi080 2024-04-24T23:27:09.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:08 smithi134 ceph-mon[32619]: mds.? [v2:172.21.15.80:6838/4194284854,v1:172.21.15.80:6839/4194284854] up:boot 2024-04-24T23:27:09.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:08 smithi134 ceph-mon[32619]: fsmap 10 up:standby 2024-04-24T23:27:09.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:08 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-24T23:27:09.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:08 smithi148 ceph-mon[32528]: Deploying daemon mds.a on smithi080 2024-04-24T23:27:09.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:08 smithi148 ceph-mon[32528]: mds.? [v2:172.21.15.80:6838/4194284854,v1:172.21.15.80:6839/4194284854] up:boot 2024-04-24T23:27:09.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:08 smithi148 ceph-mon[32528]: fsmap 10 up:standby 2024-04-24T23:27:09.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:08 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-24T23:27:09.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:08 smithi080 ceph-mon[29152]: Deploying daemon mds.a on smithi080 2024-04-24T23:27:09.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:08 smithi080 ceph-mon[29152]: mds.? [v2:172.21.15.80:6838/4194284854,v1:172.21.15.80:6839/4194284854] up:boot 2024-04-24T23:27:09.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:08 smithi080 ceph-mon[29152]: fsmap 10 up:standby 2024-04-24T23:27:09.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:08 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-24T23:27:09.927 INFO:journalctl@ceph.mds.a.smithi080.stdout:Apr 24 23:27:09 smithi080 systemd[1]: Starting Ceph mds.a for ec7885aa-0290-11ef-bc93-c7b262605968... 2024-04-24T23:27:10.189 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:09 smithi080 ceph-mon[29152]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:27:10.189 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:09 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/496590691' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:27:10.189 INFO:journalctl@ceph.mds.a.smithi080.stdout:Apr 24 23:27:10 smithi080 podman[70364]: 2024-04-24 23:27:09.931293438 +0000 UTC m=+0.019094050 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:27:10.189 INFO:journalctl@ceph.mds.a.smithi080.stdout:Apr 24 23:27:10 smithi080 podman[70364]: 2024-04-24 23:27:10.049842143 +0000 UTC m=+0.137642742 container create 0e7f74f1b192bdd92c318ee9dab63d52d291c133a313e64233ac99c6ea9a503f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-a, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T23:27:10.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:09 smithi134 ceph-mon[32619]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:27:10.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:09 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/496590691' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:27:10.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:09 smithi148 ceph-mon[32528]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:27:10.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:09 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/496590691' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:27:10.788 INFO:journalctl@ceph.mds.a.smithi080.stdout:Apr 24 23:27:10 smithi080 podman[70364]: 2024-04-24 23:27:10.294586366 +0000 UTC m=+0.382386963 container init 0e7f74f1b192bdd92c318ee9dab63d52d291c133a313e64233ac99c6ea9a503f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-a, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T23:27:10.789 INFO:journalctl@ceph.mds.a.smithi080.stdout:Apr 24 23:27:10 smithi080 podman[70364]: 2024-04-24 23:27:10.297172235 +0000 UTC m=+0.384972835 container start 0e7f74f1b192bdd92c318ee9dab63d52d291c133a313e64233ac99c6ea9a503f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-a, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T23:27:10.789 INFO:journalctl@ceph.mds.a.smithi080.stdout:Apr 24 23:27:10 smithi080 ceph-mds[70383]: -- 172.21.15.80:0/324652082 <== mon.0 v2:172.21.15.80:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564b5e4b9a40 con 0x564b5e48d000 2024-04-24T23:27:10.789 INFO:journalctl@ceph.mds.a.smithi080.stdout:Apr 24 23:27:10 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-a[70379]: starting mds.a at 2024-04-24T23:27:10.789 INFO:journalctl@ceph.mds.a.smithi080.stdout:Apr 24 23:27:10 smithi080 bash[70364]: 0e7f74f1b192bdd92c318ee9dab63d52d291c133a313e64233ac99c6ea9a503f 2024-04-24T23:27:10.789 INFO:journalctl@ceph.mds.a.smithi080.stdout:Apr 24 23:27:10 smithi080 systemd[1]: Started Ceph mds.a for ec7885aa-0290-11ef-bc93-c7b262605968. 2024-04-24T23:27:11.029 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:27:11.136 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:11.134+0000 7f3352f09640 1 -- 172.21.15.80:0/2194406679 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f334c1029a0 msgr2=0x7f334c102e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:11.136 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:11.134+0000 7f3352f09640 1 --2- 172.21.15.80:0/2194406679 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f334c1029a0 0x7f334c102e20 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f3338009900 tx=0x7f333802a410 comp rx=0 tx=0).stop 2024-04-24T23:27:11.136 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:11.134+0000 7f3352f09640 1 -- 172.21.15.80:0/2194406679 shutdown_connections 2024-04-24T23:27:11.137 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:11.134+0000 7f3352f09640 1 --2- 172.21.15.80:0/2194406679 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f334c103360 0x7f334c109bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:11.137 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:11.134+0000 7f3352f09640 1 --2- 172.21.15.80:0/2194406679 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f334c1029a0 0x7f334c102e20 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:11.137 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:11.134+0000 7f3352f09640 1 --2- 172.21.15.80:0/2194406679 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f334c1017b0 0x7f334c101bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:11.137 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:11.134+0000 7f3352f09640 1 -- 172.21.15.80:0/2194406679 >> 172.21.15.80:0/2194406679 conn(0x7f334c0fcd40 msgr2=0x7f334c0ff160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:11.137 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:11.134+0000 7f3352f09640 1 -- 172.21.15.80:0/2194406679 shutdown_connections 2024-04-24T23:27:11.137 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:11.134+0000 7f3352f09640 1 -- 172.21.15.80:0/2194406679 wait complete. 2024-04-24T23:27:11.137 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:11.134+0000 7f3352f09640 1 Processor -- start 2024-04-24T23:27:11.137 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:11.134+0000 7f3352f09640 1 -- start start 2024-04-24T23:27:11.137 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:11.135+0000 7f3352f09640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f334c1017b0 0x7f334c197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:11.137 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:11.135+0000 7f3352f09640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f334c1029a0 0x7f334c198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:11.137 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:11.135+0000 7f3352f09640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f334c103360 0x7f334c19f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:11.137 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:11.135+0000 7f3352f09640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f334c10d430 con 0x7f334c103360 2024-04-24T23:27:11.137 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:11.135+0000 7f3352f09640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f334c10d2b0 con 0x7f334c1029a0 2024-04-24T23:27:11.137 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:11.135+0000 7f3352f09640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f334c10d5b0 con 0x7f334c1017b0 2024-04-24T23:27:11.138 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:11.135+0000 7f33514a6640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f334c103360 0x7f334c19f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:11.138 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:11.135+0000 7f33514a6640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f334c103360 0x7f334c19f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:51782/0 (socket says 172.21.15.80:51782) 2024-04-24T23:27:11.138 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:11.135+0000 7f33514a6640 1 -- 172.21.15.80:0/909439142 learned_addr learned my addr 172.21.15.80:0/909439142 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:27:11.138 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:11.135+0000 7f334bfff640 1 --2- 172.21.15.80:0/909439142 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f334c1029a0 0x7f334c198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:11.138 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:11.135+0000 7f3350ca5640 1 --2- 172.21.15.80:0/909439142 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f334c1017b0 0x7f334c197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:11.138 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:11.135+0000 7f33514a6640 1 -- 172.21.15.80:0/909439142 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f334c1017b0 msgr2=0x7f334c197ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:11.138 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:11.135+0000 7f33514a6640 1 --2- 172.21.15.80:0/909439142 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f334c1017b0 0x7f334c197ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:11.138 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:11.135+0000 7f33514a6640 1 -- 172.21.15.80:0/909439142 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f334c1029a0 msgr2=0x7f334c198010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:11.138 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:11.135+0000 7f33514a6640 1 --2- 172.21.15.80:0/909439142 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f334c1029a0 0x7f334c198010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:11.138 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:11.135+0000 7f33514a6640 1 -- 172.21.15.80:0/909439142 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f334c19f6f0 con 0x7f334c103360 2024-04-24T23:27:11.138 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:11.135+0000 7f33514a6640 1 --2- 172.21.15.80:0/909439142 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f334c103360 0x7f334c19f080 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f334400eaa0 tx=0x7f334400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:11.138 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:11.135+0000 7f3349ffb640 1 -- 172.21.15.80:0/909439142 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f334400cdb0 con 0x7f334c103360 2024-04-24T23:27:11.138 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:11.135+0000 7f3349ffb640 1 -- 172.21.15.80:0/909439142 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3344005020 con 0x7f334c103360 2024-04-24T23:27:11.138 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:11.135+0000 7f3352f09640 1 -- 172.21.15.80:0/909439142 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f334c19f9e0 con 0x7f334c103360 2024-04-24T23:27:11.138 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:11.135+0000 7f3349ffb640 1 -- 172.21.15.80:0/909439142 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33440116b0 con 0x7f334c103360 2024-04-24T23:27:11.139 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:11.136+0000 7f3352f09640 1 -- 172.21.15.80:0/909439142 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f334c19feb0 con 0x7f334c103360 2024-04-24T23:27:11.139 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:11.137+0000 7f3352f09640 1 -- 172.21.15.80:0/909439142 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f334c108ad0 con 0x7f334c103360 2024-04-24T23:27:11.139 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:11.137+0000 7f3349ffb640 1 -- 172.21.15.80:0/909439142 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f3344005560 con 0x7f334c103360 2024-04-24T23:27:11.139 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:11.138+0000 7f3349ffb640 1 --2- 172.21.15.80:0/909439142 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f333c0a2b90 0x7f333c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:11.140 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:11.138+0000 7f3350ca5640 1 --2- 172.21.15.80:0/909439142 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f333c0a2b90 0x7f333c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:11.140 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:11.138+0000 7f3349ffb640 1 -- 172.21.15.80:0/909439142 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f33440ced60 con 0x7f334c103360 2024-04-24T23:27:11.140 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:11.138+0000 7f3350ca5640 1 --2- 172.21.15.80:0/909439142 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f333c0a2b90 0x7f333c0a5050 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f3340005fd0 tx=0x7f334000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:11.142 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:11.142+0000 7f3349ffb640 1 -- 172.21.15.80:0/909439142 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f3344016030 con 0x7f334c103360 2024-04-24T23:27:11.352 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:11.353+0000 7f3352f09640 1 -- 172.21.15.80:0/909439142 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f334c02d170 con 0x7f334c103360 2024-04-24T23:27:11.353 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:11.353+0000 7f3349ffb640 1 -- 172.21.15.80:0/909439142 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9653 (secure 0 0 0) 0x7f334409baf0 con 0x7f334c103360 2024-04-24T23:27:11.353 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-24T23:27:11.354 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/891591085","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":891591085},{"type":"v1","addr":"172.21.15.80:6835","nonce":891591085}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6837/259529026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6836","nonce":259529026},{"type":"v1","addr":"172.21.15.80:6837","nonce":259529026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14646,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6839/4194284854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6838","nonce":4194284854},{"type":"v1","addr":"172.21.15.80:6839","nonce":4194284854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14658,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6841/3145730958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6840","nonce":3145730958},{"type":"v1","addr":"172.21.15.80:6841","nonce":3145730958}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24395,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/3736784385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3736784385},{"type":"v1","addr":"172.21.15.134:6833","nonce":3736784385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/124777462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":124777462},{"type":"v1","addr":"172.21.15.148:6833","nonce":124777462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/1399801120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1399801120},{"type":"v1","addr":"172.21.15.134:6835","nonce":1399801120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24458,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/430512704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":430512704},{"type":"v1","addr":"172.21.15.134:6837","nonce":430512704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24460,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/3002319310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3002319310},{"type":"v1","addr":"172.21.15.148:6835","nonce":3002319310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24472,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/856528403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":856528403},{"type":"v1","addr":"172.21.15.148:6837","nonce":856528403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24496,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6839/2952757594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":2952757594},{"type":"v1","addr":"172.21.15.148:6839","nonce":2952757594}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-24T23:27:11.354 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 12 2024-04-24T23:27:11.356 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:11.355+0000 7f3352f09640 1 -- 172.21.15.80:0/909439142 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f333c0a2b90 msgr2=0x7f333c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:11.356 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:11.355+0000 7f3352f09640 1 --2- 172.21.15.80:0/909439142 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f333c0a2b90 0x7f333c0a5050 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f3340005fd0 tx=0x7f334000b390 comp rx=0 tx=0).stop 2024-04-24T23:27:11.356 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:11.355+0000 7f3352f09640 1 -- 172.21.15.80:0/909439142 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f334c103360 msgr2=0x7f334c19f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:11.356 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:11.355+0000 7f3352f09640 1 --2- 172.21.15.80:0/909439142 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f334c103360 0x7f334c19f080 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f334400eaa0 tx=0x7f334400ef60 comp rx=0 tx=0).stop 2024-04-24T23:27:11.356 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:11.356+0000 7f3352f09640 1 -- 172.21.15.80:0/909439142 shutdown_connections 2024-04-24T23:27:11.356 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:11.356+0000 7f3352f09640 1 --2- 172.21.15.80:0/909439142 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f333c0a2b90 0x7f333c0a5050 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:11.356 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:11.356+0000 7f3352f09640 1 --2- 172.21.15.80:0/909439142 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f334c103360 0x7f334c19f080 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:11.356 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:11.356+0000 7f3352f09640 1 --2- 172.21.15.80:0/909439142 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f334c1029a0 0x7f334c198010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:11.356 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:11.356+0000 7f3352f09640 1 --2- 172.21.15.80:0/909439142 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f334c1017b0 0x7f334c197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:11.356 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:11.356+0000 7f3352f09640 1 -- 172.21.15.80:0/909439142 >> 172.21.15.80:0/909439142 conn(0x7f334c0fcd40 msgr2=0x7f334c105ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:11.356 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:11.356+0000 7f3352f09640 1 -- 172.21.15.80:0/909439142 shutdown_connections 2024-04-24T23:27:11.357 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:11.356+0000 7f3352f09640 1 -- 172.21.15.80:0/909439142 wait complete. 2024-04-24T23:27:11.697 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:11 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:27:11.697 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:11 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:27:11.697 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:11 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:27:11.697 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:11 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:27:11.697 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:11 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:27:11.698 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:11 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:27:11.698 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:11 smithi134 ceph-mon[32619]: Deploying daemon mds.b on smithi134 2024-04-24T23:27:11.698 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:11 smithi134 ceph-mon[32619]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:27:11.698 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:11 smithi134 ceph-mon[32619]: mds.? [v2:172.21.15.80:6840/3145730958,v1:172.21.15.80:6841/3145730958] up:boot 2024-04-24T23:27:11.698 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:11 smithi134 ceph-mon[32619]: fsmap 11 up:standby 2024-04-24T23:27:11.698 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:11 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-24T23:27:11.698 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:11 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/909439142' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:27:11.756 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:11 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:27:11.756 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:11 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:27:11.756 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:11 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:27:11.757 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:11 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:27:11.757 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:11 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:27:11.757 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:11 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:27:11.757 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:11 smithi148 ceph-mon[32528]: Deploying daemon mds.b on smithi134 2024-04-24T23:27:11.757 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:11 smithi148 ceph-mon[32528]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:27:11.757 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:11 smithi148 ceph-mon[32528]: mds.? [v2:172.21.15.80:6840/3145730958,v1:172.21.15.80:6841/3145730958] up:boot 2024-04-24T23:27:11.757 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:11 smithi148 ceph-mon[32528]: fsmap 11 up:standby 2024-04-24T23:27:11.757 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:11 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-24T23:27:11.757 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:11 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/909439142' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:27:11.788 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:11 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:27:11.788 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:11 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:27:11.788 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:11 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:27:11.788 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:11 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:27:11.788 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:11 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:27:11.789 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:11 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:27:11.789 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:11 smithi080 ceph-mon[29152]: Deploying daemon mds.b on smithi134 2024-04-24T23:27:11.789 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:11 smithi080 ceph-mon[29152]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:27:11.789 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:11 smithi080 ceph-mon[29152]: mds.? [v2:172.21.15.80:6840/3145730958,v1:172.21.15.80:6841/3145730958] up:boot 2024-04-24T23:27:11.789 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:11 smithi080 ceph-mon[29152]: fsmap 11 up:standby 2024-04-24T23:27:11.789 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:11 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-24T23:27:11.789 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:11 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/909439142' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:27:12.915 INFO:journalctl@ceph.mds.b.smithi134.stdout:Apr 24 23:27:12 smithi134 systemd[1]: Starting Ceph mds.b for ec7885aa-0290-11ef-bc93-c7b262605968... 2024-04-24T23:27:13.176 INFO:journalctl@ceph.mds.b.smithi134.stdout:Apr 24 23:27:12 smithi134 podman[63390]: 2024-04-24 23:27:12.817930034 +0000 UTC m=+0.017860331 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:27:13.177 INFO:journalctl@ceph.mds.b.smithi134.stdout:Apr 24 23:27:12 smithi134 podman[63390]: 2024-04-24 23:27:12.940615778 +0000 UTC m=+0.140546061 container create 0e52ed6fdb94e1ebc4d6c475e166a124b9ac698b2b862fa91998711c1957cda7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-b, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T23:27:13.177 INFO:journalctl@ceph.mds.b.smithi134.stdout:Apr 24 23:27:13 smithi134 podman[63390]: 2024-04-24 23:27:13.176713062 +0000 UTC m=+0.376643346 container init 0e52ed6fdb94e1ebc4d6c475e166a124b9ac698b2b862fa91998711c1957cda7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-b, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T23:27:13.367 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:27:13.447 INFO:journalctl@ceph.mds.b.smithi134.stdout:Apr 24 23:27:13 smithi134 podman[63390]: 2024-04-24 23:27:13.179473823 +0000 UTC m=+0.379404110 container start 0e52ed6fdb94e1ebc4d6c475e166a124b9ac698b2b862fa91998711c1957cda7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-b, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240424) 2024-04-24T23:27:13.447 INFO:journalctl@ceph.mds.b.smithi134.stdout:Apr 24 23:27:13 smithi134 ceph-mds[63409]: -- 172.21.15.134:0/663358478 <== mon.2 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d450227a40 con 0x55d4501fac00 2024-04-24T23:27:13.447 INFO:journalctl@ceph.mds.b.smithi134.stdout:Apr 24 23:27:13 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-b[63405]: starting mds.b at 2024-04-24T23:27:13.447 INFO:journalctl@ceph.mds.b.smithi134.stdout:Apr 24 23:27:13 smithi134 bash[63390]: 0e52ed6fdb94e1ebc4d6c475e166a124b9ac698b2b862fa91998711c1957cda7 2024-04-24T23:27:13.447 INFO:journalctl@ceph.mds.b.smithi134.stdout:Apr 24 23:27:13 smithi134 systemd[1]: Started Ceph mds.b for ec7885aa-0290-11ef-bc93-c7b262605968. 2024-04-24T23:27:13.482 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:13.480+0000 7f0cdfbc4640 1 -- 172.21.15.80:0/694452609 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0cd8101790 msgr2=0x7f0cd8101bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:13.482 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:13.480+0000 7f0cdfbc4640 1 --2- 172.21.15.80:0/694452609 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0cd8101790 0x7f0cd8101bb0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f0cd4009900 tx=0x7f0cd402a410 comp rx=0 tx=0).stop 2024-04-24T23:27:13.482 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:13.480+0000 7f0cdfbc4640 1 -- 172.21.15.80:0/694452609 shutdown_connections 2024-04-24T23:27:13.482 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:13.480+0000 7f0cdfbc4640 1 --2- 172.21.15.80:0/694452609 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0cd8103340 0x7f0cd8109bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:13.483 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:13.480+0000 7f0cdfbc4640 1 --2- 172.21.15.80:0/694452609 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0cd8102980 0x7f0cd8102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:13.483 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:13.480+0000 7f0cdfbc4640 1 --2- 172.21.15.80:0/694452609 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0cd8101790 0x7f0cd8101bb0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:13.483 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:13.480+0000 7f0cdfbc4640 1 -- 172.21.15.80:0/694452609 >> 172.21.15.80:0/694452609 conn(0x7f0cd80fcd20 msgr2=0x7f0cd80ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:13.483 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:13.480+0000 7f0cdfbc4640 1 -- 172.21.15.80:0/694452609 shutdown_connections 2024-04-24T23:27:13.483 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:13.480+0000 7f0cdfbc4640 1 -- 172.21.15.80:0/694452609 wait complete. 2024-04-24T23:27:13.483 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:13.480+0000 7f0cdfbc4640 1 Processor -- start 2024-04-24T23:27:13.483 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:13.481+0000 7f0cdfbc4640 1 -- start start 2024-04-24T23:27:13.483 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:13.481+0000 7f0cdfbc4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0cd8101790 0x7f0cd80754f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:13.483 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:13.481+0000 7f0cdfbc4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0cd8102980 0x7f0cd8073b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:13.483 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:13.481+0000 7f0cdfbc4640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0cd8103340 0x7f0cd8074070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:13.483 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:13.481+0000 7f0cdfbc4640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f0cd810d530 con 0x7f0cd8103340 2024-04-24T23:27:13.483 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:13.481+0000 7f0cdfbc4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0cd810d3b0 con 0x7f0cd8101790 2024-04-24T23:27:13.484 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:13.481+0000 7f0cdfbc4640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f0cd810d6b0 con 0x7f0cd8102980 2024-04-24T23:27:13.484 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:13.481+0000 7f0cde161640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0cd8103340 0x7f0cd8074070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:13.484 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:13.481+0000 7f0cde161640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0cd8103340 0x7f0cd8074070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:39080/0 (socket says 172.21.15.80:39080) 2024-04-24T23:27:13.484 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:13.481+0000 7f0cde161640 1 -- 172.21.15.80:0/3328078351 learned_addr learned my addr 172.21.15.80:0/3328078351 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:27:13.484 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:13.481+0000 7f0cdd960640 1 --2- 172.21.15.80:0/3328078351 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0cd8101790 0x7f0cd80754f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:13.484 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:13.481+0000 7f0cdd15f640 1 --2- 172.21.15.80:0/3328078351 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0cd8102980 0x7f0cd8073b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:13.484 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:13.481+0000 7f0cde161640 1 -- 172.21.15.80:0/3328078351 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0cd8102980 msgr2=0x7f0cd8073b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:13.484 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:13.481+0000 7f0cde161640 1 --2- 172.21.15.80:0/3328078351 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0cd8102980 0x7f0cd8073b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:13.484 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:13.481+0000 7f0cde161640 1 -- 172.21.15.80:0/3328078351 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0cd8101790 msgr2=0x7f0cd80754f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:13.484 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:13.481+0000 7f0cde161640 1 --2- 172.21.15.80:0/3328078351 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0cd8101790 0x7f0cd80754f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:13.484 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:13.481+0000 7f0cde161640 1 -- 172.21.15.80:0/3328078351 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cd8074900 con 0x7f0cd8103340 2024-04-24T23:27:13.484 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:13.481+0000 7f0cde161640 1 --2- 172.21.15.80:0/3328078351 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0cd8103340 0x7f0cd8074070 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0cd000b970 tx=0x7f0cd000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:13.484 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:13.481+0000 7f0cceffd640 1 -- 172.21.15.80:0/3328078351 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cd0019070 con 0x7f0cd8103340 2024-04-24T23:27:13.484 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:13.481+0000 7f0cdfbc4640 1 -- 172.21.15.80:0/3328078351 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cd81a1d30 con 0x7f0cd8103340 2024-04-24T23:27:13.484 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:13.481+0000 7f0cceffd640 1 -- 172.21.15.80:0/3328078351 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0cd000cb40 con 0x7f0cd8103340 2024-04-24T23:27:13.485 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:13.481+0000 7f0cceffd640 1 -- 172.21.15.80:0/3328078351 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cd0007650 con 0x7f0cd8103340 2024-04-24T23:27:13.485 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:13.482+0000 7f0cdfbc4640 1 -- 172.21.15.80:0/3328078351 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cd81a21d0 con 0x7f0cd8103340 2024-04-24T23:27:13.485 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:13.482+0000 7f0cdfbc4640 1 -- 172.21.15.80:0/3328078351 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cd8108b30 con 0x7f0cd8103340 2024-04-24T23:27:13.485 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:13.483+0000 7f0cceffd640 1 -- 172.21.15.80:0/3328078351 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f0cd001c070 con 0x7f0cd8103340 2024-04-24T23:27:13.486 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:13.484+0000 7f0cceffd640 1 --2- 172.21.15.80:0/3328078351 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f0cc40a2b90 0x7f0cc40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:13.486 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:13.484+0000 7f0cdd960640 1 --2- 172.21.15.80:0/3328078351 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f0cc40a2b90 0x7f0cc40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:13.486 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:13.484+0000 7f0cceffd640 1 -- 172.21.15.80:0/3328078351 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f0cd00ce1d0 con 0x7f0cd8103340 2024-04-24T23:27:13.486 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:13.485+0000 7f0cdd960640 1 --2- 172.21.15.80:0/3328078351 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f0cc40a2b90 0x7f0cc40a5050 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0cd402aa00 tx=0x7f0cd40315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:13.489 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:13.489+0000 7f0cceffd640 1 -- 172.21.15.80:0/3328078351 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f0cd009af80 con 0x7f0cd8103340 2024-04-24T23:27:13.716 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:13.715+0000 7f0cdfbc4640 1 -- 172.21.15.80:0/3328078351 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0cd802d370 con 0x7f0cd8103340 2024-04-24T23:27:13.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:13.716+0000 7f0cceffd640 1 -- 172.21.15.80:0/3328078351 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9653 (secure 0 0 0) 0x7f0cd009f280 con 0x7f0cd8103340 2024-04-24T23:27:13.717 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-24T23:27:13.717 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/891591085","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":891591085},{"type":"v1","addr":"172.21.15.80:6835","nonce":891591085}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6837/259529026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6836","nonce":259529026},{"type":"v1","addr":"172.21.15.80:6837","nonce":259529026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14646,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6839/4194284854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6838","nonce":4194284854},{"type":"v1","addr":"172.21.15.80:6839","nonce":4194284854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14658,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6841/3145730958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6840","nonce":3145730958},{"type":"v1","addr":"172.21.15.80:6841","nonce":3145730958}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24395,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/3736784385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3736784385},{"type":"v1","addr":"172.21.15.134:6833","nonce":3736784385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/124777462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":124777462},{"type":"v1","addr":"172.21.15.148:6833","nonce":124777462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/1399801120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1399801120},{"type":"v1","addr":"172.21.15.134:6835","nonce":1399801120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24458,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/430512704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":430512704},{"type":"v1","addr":"172.21.15.134:6837","nonce":430512704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24460,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/3002319310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3002319310},{"type":"v1","addr":"172.21.15.148:6835","nonce":3002319310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24472,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/856528403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":856528403},{"type":"v1","addr":"172.21.15.148:6837","nonce":856528403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24496,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6839/2952757594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":2952757594},{"type":"v1","addr":"172.21.15.148:6839","nonce":2952757594}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-24T23:27:13.717 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 12 2024-04-24T23:27:13.719 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:13.718+0000 7f0cdfbc4640 1 -- 172.21.15.80:0/3328078351 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f0cc40a2b90 msgr2=0x7f0cc40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:13.719 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:13.718+0000 7f0cdfbc4640 1 --2- 172.21.15.80:0/3328078351 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f0cc40a2b90 0x7f0cc40a5050 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0cd402aa00 tx=0x7f0cd40315b0 comp rx=0 tx=0).stop 2024-04-24T23:27:13.719 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:13.718+0000 7f0cdfbc4640 1 -- 172.21.15.80:0/3328078351 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0cd8103340 msgr2=0x7f0cd8074070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:13.719 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:13.718+0000 7f0cdfbc4640 1 --2- 172.21.15.80:0/3328078351 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0cd8103340 0x7f0cd8074070 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0cd000b970 tx=0x7f0cd000be30 comp rx=0 tx=0).stop 2024-04-24T23:27:13.719 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:13.718+0000 7f0cdfbc4640 1 -- 172.21.15.80:0/3328078351 shutdown_connections 2024-04-24T23:27:13.719 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:13.718+0000 7f0cdfbc4640 1 --2- 172.21.15.80:0/3328078351 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f0cc40a2b90 0x7f0cc40a5050 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:13.720 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:13.718+0000 7f0cdfbc4640 1 --2- 172.21.15.80:0/3328078351 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0cd8103340 0x7f0cd8074070 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:13.720 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:13.718+0000 7f0cdfbc4640 1 --2- 172.21.15.80:0/3328078351 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0cd8102980 0x7f0cd8073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:13.720 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:13.718+0000 7f0cdfbc4640 1 --2- 172.21.15.80:0/3328078351 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0cd8101790 0x7f0cd80754f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:13.720 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:13.718+0000 7f0cdfbc4640 1 -- 172.21.15.80:0/3328078351 >> 172.21.15.80:0/3328078351 conn(0x7f0cd80fcd20 msgr2=0x7f0cd8105d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:13.720 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:13.718+0000 7f0cdfbc4640 1 -- 172.21.15.80:0/3328078351 shutdown_connections 2024-04-24T23:27:13.720 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:13.718+0000 7f0cdfbc4640 1 -- 172.21.15.80:0/3328078351 wait complete. 2024-04-24T23:27:14.038 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:13 smithi080 ceph-mon[29152]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:27:14.038 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:13 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:27:14.038 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:13 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:27:14.038 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:13 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:27:14.038 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:13 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:27:14.038 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:13 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:27:14.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:13 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:27:14.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:13 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/3328078351' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:27:14.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:13 smithi134 ceph-mon[32619]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:27:14.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:13 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:27:14.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:13 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:27:14.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:13 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:27:14.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:13 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:27:14.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:13 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:27:14.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:13 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:27:14.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:13 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/3328078351' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:27:14.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:13 smithi148 ceph-mon[32528]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:27:14.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:13 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:27:14.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:13 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:27:14.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:13 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:27:14.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:13 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:27:14.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:13 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:27:14.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:13 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:27:14.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:13 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/3328078351' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:27:15.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:14 smithi134 ceph-mon[32619]: mds.? [v2:172.21.15.134:6838/1743815021,v1:172.21.15.134:6839/1743815021] up:boot 2024-04-24T23:27:15.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:14 smithi134 ceph-mon[32619]: fsmap 12 up:standby 2024-04-24T23:27:15.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:14 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-24T23:27:15.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:14 smithi148 ceph-mon[32528]: mds.? [v2:172.21.15.134:6838/1743815021,v1:172.21.15.134:6839/1743815021] up:boot 2024-04-24T23:27:15.293 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:14 smithi148 ceph-mon[32528]: fsmap 12 up:standby 2024-04-24T23:27:15.293 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:14 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-24T23:27:15.293 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:14 smithi080 ceph-mon[29152]: mds.? [v2:172.21.15.134:6838/1743815021,v1:172.21.15.134:6839/1743815021] up:boot 2024-04-24T23:27:15.293 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:14 smithi080 ceph-mon[29152]: fsmap 12 up:standby 2024-04-24T23:27:15.294 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:14 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-24T23:27:15.731 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:27:15.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:15.832+0000 7fedd7f89640 1 -- 172.21.15.80:0/3516506662 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fedd01029a0 msgr2=0x7fedd0102e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:15.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:15.832+0000 7fedd7f89640 1 --2- 172.21.15.80:0/3516506662 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fedd01029a0 0x7fedd0102e20 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fedc0009900 tx=0x7fedc002a410 comp rx=0 tx=0).stop 2024-04-24T23:27:15.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:15.833+0000 7fedd7f89640 1 -- 172.21.15.80:0/3516506662 shutdown_connections 2024-04-24T23:27:15.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:15.833+0000 7fedd7f89640 1 --2- 172.21.15.80:0/3516506662 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fedd0103360 0x7fedd0109bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:15.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:15.833+0000 7fedd7f89640 1 --2- 172.21.15.80:0/3516506662 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fedd01029a0 0x7fedd0102e20 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:15.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:15.833+0000 7fedd7f89640 1 --2- 172.21.15.80:0/3516506662 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fedd01017b0 0x7fedd0101bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:15.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:15.833+0000 7fedd7f89640 1 -- 172.21.15.80:0/3516506662 >> 172.21.15.80:0/3516506662 conn(0x7fedd00fcd40 msgr2=0x7fedd00ff160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:15.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:15.833+0000 7fedd7f89640 1 -- 172.21.15.80:0/3516506662 shutdown_connections 2024-04-24T23:27:15.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:15.833+0000 7fedd7f89640 1 -- 172.21.15.80:0/3516506662 wait complete. 2024-04-24T23:27:15.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:15.833+0000 7fedd7f89640 1 Processor -- start 2024-04-24T23:27:15.837 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:15.833+0000 7fedd7f89640 1 -- start start 2024-04-24T23:27:15.837 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:15.833+0000 7fedd7f89640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fedd01017b0 0x7fedd0197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:15.837 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:15.833+0000 7fedd7f89640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fedd01029a0 0x7fedd0198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:15.837 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:15.833+0000 7fedd7f89640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fedd0103360 0x7fedd019f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:15.837 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:15.833+0000 7fedd7f89640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fedd010d430 con 0x7fedd01017b0 2024-04-24T23:27:15.837 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:15.833+0000 7fedd7f89640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fedd010d2b0 con 0x7fedd01029a0 2024-04-24T23:27:15.837 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:15.833+0000 7fedd7f89640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fedd010d5b0 con 0x7fedd0103360 2024-04-24T23:27:15.837 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:15.834+0000 7fedd5d25640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fedd01017b0 0x7fedd0197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:15.837 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:15.834+0000 7fedd5d25640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fedd01017b0 0x7fedd0197af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:39090/0 (socket says 172.21.15.80:39090) 2024-04-24T23:27:15.837 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:15.834+0000 7fedd5d25640 1 -- 172.21.15.80:0/2832680235 learned_addr learned my addr 172.21.15.80:0/2832680235 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:27:15.837 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:15.834+0000 7fedd5524640 1 --2- 172.21.15.80:0/2832680235 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fedd01029a0 0x7fedd0198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:15.837 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:15.834+0000 7fedd6526640 1 --2- 172.21.15.80:0/2832680235 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fedd0103360 0x7fedd019f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:15.837 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:15.834+0000 7fedd5d25640 1 -- 172.21.15.80:0/2832680235 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fedd0103360 msgr2=0x7fedd019f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:15.837 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:15.834+0000 7fedd5d25640 1 --2- 172.21.15.80:0/2832680235 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fedd0103360 0x7fedd019f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:15.838 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:15.834+0000 7fedd5d25640 1 -- 172.21.15.80:0/2832680235 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fedd01029a0 msgr2=0x7fedd0198030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:15.838 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:15.834+0000 7fedd5d25640 1 --2- 172.21.15.80:0/2832680235 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fedd01029a0 0x7fedd0198030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:15.838 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:15.834+0000 7fedd5d25640 1 -- 172.21.15.80:0/2832680235 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedd019f710 con 0x7fedd01017b0 2024-04-24T23:27:15.838 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:15.834+0000 7fedd5d25640 1 --2- 172.21.15.80:0/2832680235 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fedd01017b0 0x7fedd0197af0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fedcc00c970 tx=0x7fedcc00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:15.838 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:15.834+0000 7fedc6ffd640 1 -- 172.21.15.80:0/2832680235 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedcc004220 con 0x7fedd01017b0 2024-04-24T23:27:15.838 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:15.834+0000 7fedd7f89640 1 -- 172.21.15.80:0/2832680235 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedd019fa00 con 0x7fedd01017b0 2024-04-24T23:27:15.838 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:15.834+0000 7fedc6ffd640 1 -- 172.21.15.80:0/2832680235 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fedcc0043c0 con 0x7fedd01017b0 2024-04-24T23:27:15.838 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:15.834+0000 7fedc6ffd640 1 -- 172.21.15.80:0/2832680235 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedcc004a00 con 0x7fedd01017b0 2024-04-24T23:27:15.838 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:15.834+0000 7fedd7f89640 1 -- 172.21.15.80:0/2832680235 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedd019ff20 con 0x7fedd01017b0 2024-04-24T23:27:15.839 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:15.835+0000 7fedd7f89640 1 -- 172.21.15.80:0/2832680235 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fedd0067cd0 con 0x7fedd01017b0 2024-04-24T23:27:15.839 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:15.837+0000 7fedc6ffd640 1 -- 172.21.15.80:0/2832680235 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fedcc0079b0 con 0x7fedd01017b0 2024-04-24T23:27:15.839 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:15.837+0000 7fedc6ffd640 1 --2- 172.21.15.80:0/2832680235 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fedbc0a2b90 0x7fedbc0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:15.839 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:15.837+0000 7fedc6ffd640 1 -- 172.21.15.80:0/2832680235 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fedcc0ced20 con 0x7fedd01017b0 2024-04-24T23:27:15.839 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:15.837+0000 7fedd5524640 1 --2- 172.21.15.80:0/2832680235 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fedbc0a2b90 0x7fedbc0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:15.840 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:15.838+0000 7fedd5524640 1 --2- 172.21.15.80:0/2832680235 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fedbc0a2b90 0x7fedbc0a5050 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fedc00314e0 tx=0x7fedc0000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:15.842 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:15.841+0000 7fedc6ffd640 1 -- 172.21.15.80:0/2832680235 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fedcc014030 con 0x7fedd01017b0 2024-04-24T23:27:16.038 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:15 smithi080 ceph-mon[29152]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:27:16.038 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:15 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:27:16.038 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:15 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:27:16.038 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:15 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:27:16.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:15 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:27:16.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:15 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:27:16.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:15 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:27:16.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:15 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:27:16.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:15 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:27:16.039 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:15 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:27:16.066 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.065+0000 7fedd7f89640 1 -- 172.21.15.80:0/2832680235 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fedd0108ad0 con 0x7fedd01017b0 2024-04-24T23:27:16.067 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.066+0000 7fedc6ffd640 1 -- 172.21.15.80:0/2832680235 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10482 (secure 0 0 0) 0x7fedcc09c120 con 0x7fedd01017b0 2024-04-24T23:27:16.067 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-24T23:27:16.067 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/891591085","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":891591085},{"type":"v1","addr":"172.21.15.80:6835","nonce":891591085}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6837/259529026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6836","nonce":259529026},{"type":"v1","addr":"172.21.15.80:6837","nonce":259529026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14646,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6839/4194284854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6838","nonce":4194284854},{"type":"v1","addr":"172.21.15.80:6839","nonce":4194284854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14658,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6841/3145730958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6840","nonce":3145730958},{"type":"v1","addr":"172.21.15.80:6841","nonce":3145730958}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24395,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/3736784385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3736784385},{"type":"v1","addr":"172.21.15.134:6833","nonce":3736784385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/124777462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":124777462},{"type":"v1","addr":"172.21.15.148:6833","nonce":124777462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/1399801120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1399801120},{"type":"v1","addr":"172.21.15.134:6835","nonce":1399801120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24458,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/430512704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":430512704},{"type":"v1","addr":"172.21.15.134:6837","nonce":430512704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24460,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/3002319310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3002319310},{"type":"v1","addr":"172.21.15.148:6835","nonce":3002319310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24472,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/856528403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":856528403},{"type":"v1","addr":"172.21.15.148:6837","nonce":856528403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24488,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6839/1743815021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":1743815021},{"type":"v1","addr":"172.21.15.134:6839","nonce":1743815021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":24496,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6839/2952757594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":2952757594},{"type":"v1","addr":"172.21.15.148:6839","nonce":2952757594}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-24T23:27:16.068 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 13 2024-04-24T23:27:16.069 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.068+0000 7fedd7f89640 1 -- 172.21.15.80:0/2832680235 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fedbc0a2b90 msgr2=0x7fedbc0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:16.069 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.068+0000 7fedd7f89640 1 --2- 172.21.15.80:0/2832680235 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fedbc0a2b90 0x7fedbc0a5050 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fedc00314e0 tx=0x7fedc0000bc0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.069 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.068+0000 7fedd7f89640 1 -- 172.21.15.80:0/2832680235 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fedd01017b0 msgr2=0x7fedd0197af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:16.069 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.068+0000 7fedd7f89640 1 --2- 172.21.15.80:0/2832680235 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fedd01017b0 0x7fedd0197af0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fedcc00c970 tx=0x7fedcc00ce30 comp rx=0 tx=0).stop 2024-04-24T23:27:16.070 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.068+0000 7fedd7f89640 1 -- 172.21.15.80:0/2832680235 shutdown_connections 2024-04-24T23:27:16.070 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.068+0000 7fedd7f89640 1 --2- 172.21.15.80:0/2832680235 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fedbc0a2b90 0x7fedbc0a5050 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.070 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.068+0000 7fedd7f89640 1 --2- 172.21.15.80:0/2832680235 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fedd0103360 0x7fedd019f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.070 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.068+0000 7fedd7f89640 1 --2- 172.21.15.80:0/2832680235 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fedd01029a0 0x7fedd0198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.070 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.068+0000 7fedd7f89640 1 --2- 172.21.15.80:0/2832680235 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fedd01017b0 0x7fedd0197af0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.070 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.068+0000 7fedd7f89640 1 -- 172.21.15.80:0/2832680235 >> 172.21.15.80:0/2832680235 conn(0x7fedd00fcd40 msgr2=0x7fedd0105ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:16.070 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.068+0000 7fedd7f89640 1 -- 172.21.15.80:0/2832680235 shutdown_connections 2024-04-24T23:27:16.070 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.068+0000 7fedd7f89640 1 -- 172.21.15.80:0/2832680235 wait complete. 2024-04-24T23:27:16.082 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:27:16.186 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.182+0000 7fe7d12bc640 1 -- 172.21.15.80:0/2338631304 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe7cc101870 msgr2=0x7fe7cc101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:16.186 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.182+0000 7fe7d12bc640 1 --2- 172.21.15.80:0/2338631304 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe7cc101870 0x7fe7cc101cf0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe7c0009900 tx=0x7fe7c002a410 comp rx=0 tx=0).stop 2024-04-24T23:27:16.186 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.182+0000 7fe7d12bc640 1 -- 172.21.15.80:0/2338631304 shutdown_connections 2024-04-24T23:27:16.186 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.182+0000 7fe7d12bc640 1 --2- 172.21.15.80:0/2338631304 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe7cc102230 0x7fe7cc108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.186 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.182+0000 7fe7d12bc640 1 --2- 172.21.15.80:0/2338631304 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe7cc101870 0x7fe7cc101cf0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.186 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.182+0000 7fe7d12bc640 1 --2- 172.21.15.80:0/2338631304 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe7cc100680 0x7fe7cc100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.186 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.182+0000 7fe7d12bc640 1 -- 172.21.15.80:0/2338631304 >> 172.21.15.80:0/2338631304 conn(0x7fe7cc0fbbf0 msgr2=0x7fe7cc0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:16.186 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.183+0000 7fe7d12bc640 1 -- 172.21.15.80:0/2338631304 shutdown_connections 2024-04-24T23:27:16.187 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.183+0000 7fe7d12bc640 1 -- 172.21.15.80:0/2338631304 wait complete. 2024-04-24T23:27:16.187 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.183+0000 7fe7d12bc640 1 Processor -- start 2024-04-24T23:27:16.187 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.183+0000 7fe7d12bc640 1 -- start start 2024-04-24T23:27:16.187 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.183+0000 7fe7d12bc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe7cc100680 0x7fe7cc197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:16.187 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.183+0000 7fe7d12bc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe7cc101870 0x7fe7cc198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:16.187 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.183+0000 7fe7d12bc640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe7cc102230 0x7fe7cc19f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:16.187 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.183+0000 7fe7d12bc640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fe7cc10d420 con 0x7fe7cc102230 2024-04-24T23:27:16.187 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.183+0000 7fe7d12bc640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe7cc10d2a0 con 0x7fe7cc101870 2024-04-24T23:27:16.187 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.183+0000 7fe7d12bc640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe7cc10d5a0 con 0x7fe7cc100680 2024-04-24T23:27:16.187 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.184+0000 7fe7cb7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe7cc102230 0x7fe7cc19f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:16.187 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.184+0000 7fe7cb7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe7cc102230 0x7fe7cc19f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:39112/0 (socket says 172.21.15.80:39112) 2024-04-24T23:27:16.187 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.184+0000 7fe7ca7fc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe7cc101870 0x7fe7cc198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:16.187 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.184+0000 7fe7cb7fe640 1 -- 172.21.15.80:0/1380085007 learned_addr learned my addr 172.21.15.80:0/1380085007 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:27:16.187 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.184+0000 7fe7caffd640 1 --2- 172.21.15.80:0/1380085007 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe7cc100680 0x7fe7cc197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:16.188 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.184+0000 7fe7cb7fe640 1 -- 172.21.15.80:0/1380085007 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe7cc100680 msgr2=0x7fe7cc197ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:16.188 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.184+0000 7fe7cb7fe640 1 --2- 172.21.15.80:0/1380085007 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe7cc100680 0x7fe7cc197ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.188 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.184+0000 7fe7cb7fe640 1 -- 172.21.15.80:0/1380085007 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe7cc101870 msgr2=0x7fe7cc198000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:16.188 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.184+0000 7fe7cb7fe640 1 --2- 172.21.15.80:0/1380085007 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe7cc101870 0x7fe7cc198000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.188 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.184+0000 7fe7cb7fe640 1 -- 172.21.15.80:0/1380085007 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7cc19f6e0 con 0x7fe7cc102230 2024-04-24T23:27:16.188 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.184+0000 7fe7cb7fe640 1 --2- 172.21.15.80:0/1380085007 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe7cc102230 0x7fe7cc19f070 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe7c4009fd0 tx=0x7fe7c400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:16.188 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.184+0000 7fe7b3fff640 1 -- 172.21.15.80:0/1380085007 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7c4017070 con 0x7fe7cc102230 2024-04-24T23:27:16.188 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.184+0000 7fe7d12bc640 1 -- 172.21.15.80:0/1380085007 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7cc19f9d0 con 0x7fe7cc102230 2024-04-24T23:27:16.188 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.184+0000 7fe7b3fff640 1 -- 172.21.15.80:0/1380085007 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7c4003e50 con 0x7fe7cc102230 2024-04-24T23:27:16.188 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.184+0000 7fe7b3fff640 1 -- 172.21.15.80:0/1380085007 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7c40106c0 con 0x7fe7cc102230 2024-04-24T23:27:16.188 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.184+0000 7fe7d12bc640 1 -- 172.21.15.80:0/1380085007 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7cc19fea0 con 0x7fe7cc102230 2024-04-24T23:27:16.189 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.185+0000 7fe7d12bc640 1 -- 172.21.15.80:0/1380085007 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7cc107a70 con 0x7fe7cc102230 2024-04-24T23:27:16.189 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.186+0000 7fe7b3fff640 1 -- 172.21.15.80:0/1380085007 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fe7c4003970 con 0x7fe7cc102230 2024-04-24T23:27:16.189 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.187+0000 7fe7b3fff640 1 --2- 172.21.15.80:0/1380085007 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fe7b40a2b90 0x7fe7b40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:16.189 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.187+0000 7fe7caffd640 1 --2- 172.21.15.80:0/1380085007 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fe7b40a2b90 0x7fe7b40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:16.189 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.187+0000 7fe7b3fff640 1 -- 172.21.15.80:0/1380085007 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fe7c40ce890 con 0x7fe7cc102230 2024-04-24T23:27:16.190 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.187+0000 7fe7caffd640 1 --2- 172.21.15.80:0/1380085007 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fe7b40a2b90 0x7fe7b40a5050 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe7bc005fd0 tx=0x7fe7bc00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:16.196 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.195+0000 7fe7b3fff640 1 -- 172.21.15.80:0/1380085007 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fe7c409b640 con 0x7fe7cc102230 2024-04-24T23:27:16.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:15 smithi134 ceph-mon[32619]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:27:16.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:15 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:27:16.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:15 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:27:16.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:15 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:27:16.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:15 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:27:16.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:15 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:27:16.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:15 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:27:16.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:15 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:27:16.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:15 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:27:16.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:15 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:27:16.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:15 smithi148 ceph-mon[32528]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:27:16.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:15 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:27:16.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:15 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:27:16.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:15 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:27:16.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:15 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:27:16.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:15 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:27:16.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:15 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:27:16.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:15 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:27:16.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:15 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:27:16.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:15 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:27:16.377 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.376+0000 7fe7d12bc640 1 -- 172.21.15.80:0/1380085007 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe7cc062340 con 0x7fe7cc102230 2024-04-24T23:27:16.382 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.378+0000 7fe7b3fff640 1 -- 172.21.15.80:0/1380085007 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+18452 (secure 0 0 0) 0x7fe7c4097030 con 0x7fe7cc102230 2024-04-24T23:27:16.384 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-24T23:27:16.384 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":81,"fsid":"ec7885aa-0290-11ef-bc93-c7b262605968","created":"2024-04-24T23:19:05.586407+0000","modified":"2024-04-24T23:26:27.519820+0000","last_up_change":"2024-04-24T23:26:25.938316+0000","last_in_change":"2024-04-24T23:26:04.786699+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:22:33.124428+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a525741d-b46c-4eef-aa64-3f057cac1497","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":2597718112},{"type":"v1","addr":"172.21.15.80:6803","nonce":2597718112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":2597718112},{"type":"v1","addr":"172.21.15.80:6805","nonce":2597718112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2597718112},{"type":"v1","addr":"172.21.15.80:6809","nonce":2597718112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":2597718112},{"type":"v1","addr":"172.21.15.80:6807","nonce":2597718112}]},"public_addr":"172.21.15.80:6803/2597718112","cluster_addr":"172.21.15.80:6805/2597718112","heartbeat_back_addr":"172.21.15.80:6809/2597718112","heartbeat_front_addr":"172.21.15.80:6807/2597718112","state":["exists","up"]},{"osd":1,"uuid":"2d0f0d67-6db4-40ac-91cc-58604ab6795a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":2395610201},{"type":"v1","addr":"172.21.15.134:6801","nonce":2395610201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":2395610201},{"type":"v1","addr":"172.21.15.134:6803","nonce":2395610201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":2395610201},{"type":"v1","addr":"172.21.15.134:6807","nonce":2395610201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":2395610201},{"type":"v1","addr":"172.21.15.134:6805","nonce":2395610201}]},"public_addr":"172.21.15.134:6801/2395610201","cluster_addr":"172.21.15.134:6803/2395610201","heartbeat_back_addr":"172.21.15.134:6807/2395610201","heartbeat_front_addr":"172.21.15.134:6805/2395610201","state":["exists","up"]},{"osd":2,"uuid":"781fb879-fedc-4c84-82fb-59e073bc4cf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1629152187},{"type":"v1","addr":"172.21.15.148:6801","nonce":1629152187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1629152187},{"type":"v1","addr":"172.21.15.148:6803","nonce":1629152187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1629152187},{"type":"v1","addr":"172.21.15.148:6807","nonce":1629152187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1629152187},{"type":"v1","addr":"172.21.15.148:6805","nonce":1629152187}]},"public_addr":"172.21.15.148:6801/1629152187","cluster_addr":"172.21.15.148:6803/1629152187","heartbeat_back_addr":"172.21.15.148:6807/1629152187","heartbeat_front_addr":"172.21.15.148:6805/1629152187","state":["exists","up"]},{"osd":3,"uuid":"cd982757-f2f6-4296-b3e7-4cd40fbd8d9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":3128298510},{"type":"v1","addr":"172.21.15.80:6811","nonce":3128298510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":3128298510},{"type":"v1","addr":"172.21.15.80:6813","nonce":3128298510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":3128298510},{"type":"v1","addr":"172.21.15.80:6817","nonce":3128298510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":3128298510},{"type":"v1","addr":"172.21.15.80:6815","nonce":3128298510}]},"public_addr":"172.21.15.80:6811/3128298510","cluster_addr":"172.21.15.80:6813/3128298510","heartbeat_back_addr":"172.21.15.80:6817/3128298510","heartbeat_front_addr":"172.21.15.80:6815/3128298510","state":["exists","up"]},{"osd":4,"uuid":"dfc55a79-a7e8-417a-b898-ec44171a54af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1917556060},{"type":"v1","addr":"172.21.15.134:6809","nonce":1917556060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1917556060},{"type":"v1","addr":"172.21.15.134:6811","nonce":1917556060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1917556060},{"type":"v1","addr":"172.21.15.134:6815","nonce":1917556060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1917556060},{"type":"v1","addr":"172.21.15.134:6813","nonce":1917556060}]},"public_addr":"172.21.15.134:6809/1917556060","cluster_addr":"172.21.15.134:6811/1917556060","heartbeat_back_addr":"172.21.15.134:6815/1917556060","heartbeat_front_addr":"172.21.15.134:6813/1917556060","state":["exists","up"]},{"osd":5,"uuid":"b5491337-0694-440c-b679-312f3d14cf65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":2715951281},{"type":"v1","addr":"172.21.15.148:6809","nonce":2715951281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":2715951281},{"type":"v1","addr":"172.21.15.148:6811","nonce":2715951281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":2715951281},{"type":"v1","addr":"172.21.15.148:6815","nonce":2715951281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":2715951281},{"type":"v1","addr":"172.21.15.148:6813","nonce":2715951281}]},"public_addr":"172.21.15.148:6809/2715951281","cluster_addr":"172.21.15.148:6811/2715951281","heartbeat_back_addr":"172.21.15.148:6815/2715951281","heartbeat_front_addr":"172.21.15.148:6813/2715951281","state":["exists","up"]},{"osd":6,"uuid":"ba4b5df7-d86e-47cd-ab67-cf763c4e4d6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":953261724},{"type":"v1","addr":"172.21.15.80:6819","nonce":953261724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":953261724},{"type":"v1","addr":"172.21.15.80:6821","nonce":953261724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":953261724},{"type":"v1","addr":"172.21.15.80:6825","nonce":953261724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":953261724},{"type":"v1","addr":"172.21.15.80:6823","nonce":953261724}]},"public_addr":"172.21.15.80:6819/953261724","cluster_addr":"172.21.15.80:6821/953261724","heartbeat_back_addr":"172.21.15.80:6825/953261724","heartbeat_front_addr":"172.21.15.80:6823/953261724","state":["exists","up"]},{"osd":7,"uuid":"464a503a-75ae-49e7-ab6e-a7f9f2a6a307","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":108354844},{"type":"v1","addr":"172.21.15.134:6817","nonce":108354844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":108354844},{"type":"v1","addr":"172.21.15.134:6819","nonce":108354844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":108354844},{"type":"v1","addr":"172.21.15.134:6823","nonce":108354844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":108354844},{"type":"v1","addr":"172.21.15.134:6821","nonce":108354844}]},"public_addr":"172.21.15.134:6817/108354844","cluster_addr":"172.21.15.134:6819/108354844","heartbeat_back_addr":"172.21.15.134:6823/108354844","heartbeat_front_addr":"172.21.15.134:6821/108354844","state":["exists","up"]},{"osd":8,"uuid":"ce544b2e-4540-47c0-9997-500c45351330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3559655329},{"type":"v1","addr":"172.21.15.148:6817","nonce":3559655329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":3559655329},{"type":"v1","addr":"172.21.15.148:6819","nonce":3559655329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":3559655329},{"type":"v1","addr":"172.21.15.148:6823","nonce":3559655329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3559655329},{"type":"v1","addr":"172.21.15.148:6821","nonce":3559655329}]},"public_addr":"172.21.15.148:6817/3559655329","cluster_addr":"172.21.15.148:6819/3559655329","heartbeat_back_addr":"172.21.15.148:6823/3559655329","heartbeat_front_addr":"172.21.15.148:6821/3559655329","state":["exists","up"]},{"osd":9,"uuid":"06de3732-19b9-496c-a464-55322efa6759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":4019913545},{"type":"v1","addr":"172.21.15.80:6827","nonce":4019913545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":4019913545},{"type":"v1","addr":"172.21.15.80:6829","nonce":4019913545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":4019913545},{"type":"v1","addr":"172.21.15.80:6833","nonce":4019913545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":4019913545},{"type":"v1","addr":"172.21.15.80:6831","nonce":4019913545}]},"public_addr":"172.21.15.80:6827/4019913545","cluster_addr":"172.21.15.80:6829/4019913545","heartbeat_back_addr":"172.21.15.80:6833/4019913545","heartbeat_front_addr":"172.21.15.80:6831/4019913545","state":["exists","up"]},{"osd":10,"uuid":"8d92f0d9-5c64-429a-b750-c33ae0e070c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":647343766},{"type":"v1","addr":"172.21.15.134:6825","nonce":647343766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":647343766},{"type":"v1","addr":"172.21.15.134:6827","nonce":647343766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":647343766},{"type":"v1","addr":"172.21.15.134:6831","nonce":647343766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":647343766},{"type":"v1","addr":"172.21.15.134:6829","nonce":647343766}]},"public_addr":"172.21.15.134:6825/647343766","cluster_addr":"172.21.15.134:6827/647343766","heartbeat_back_addr":"172.21.15.134:6831/647343766","heartbeat_front_addr":"172.21.15.134:6829/647343766","state":["exists","up"]},{"osd":11,"uuid":"4c003a41-f65b-479f-bc80-b27d680cf936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":4068024116},{"type":"v1","addr":"172.21.15.148:6825","nonce":4068024116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":4068024116},{"type":"v1","addr":"172.21.15.148:6827","nonce":4068024116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":4068024116},{"type":"v1","addr":"172.21.15.148:6831","nonce":4068024116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":4068024116},{"type":"v1","addr":"172.21.15.148:6829","nonce":4068024116}]},"public_addr":"172.21.15.148:6825/4068024116","cluster_addr":"172.21.15.148:6827/4068024116","heartbeat_back_addr":"172.21.15.148:6831/4068024116","heartbeat_front_addr":"172.21.15.148:6829/4068024116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:21:42.433631+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:22:06.145933+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:22:30.726143+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:22:55.729961+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:23:21.280913+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:23:46.609983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:24:12.585247+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:24:38.359287+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:25:04.491232+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:25:30.788089+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:25:56.891078+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:26:23.899113+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.80:0/2828378749":"2024-04-25T23:20:06.793609+0000","172.21.15.80:6801/233364790":"2024-04-25T23:20:06.793609+0000","172.21.15.80:6800/1614691410":"2024-04-25T23:19:40.490692+0000","172.21.15.80:0/1406187145":"2024-04-25T23:20:06.793609+0000","172.21.15.80:0/92953396":"2024-04-25T23:19:40.490692+0000","172.21.15.80:0/598129432":"2024-04-25T23:20:06.793609+0000","172.21.15.80:0/232193118":"2024-04-25T23:19:40.490692+0000","172.21.15.80:6800/233364790":"2024-04-25T23:20:06.793609+0000","172.21.15.80:6801/1614691410":"2024-04-25T23:19:40.490692+0000","172.21.15.80:0/2081562529":"2024-04-25T23:19:40.490692+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:27:16.385 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.380+0000 7fe7d12bc640 1 -- 172.21.15.80:0/1380085007 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fe7b40a2b90 msgr2=0x7fe7b40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:16.386 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.380+0000 7fe7d12bc640 1 --2- 172.21.15.80:0/1380085007 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fe7b40a2b90 0x7fe7b40a5050 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe7bc005fd0 tx=0x7fe7bc00b390 comp rx=0 tx=0).stop 2024-04-24T23:27:16.386 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.380+0000 7fe7d12bc640 1 -- 172.21.15.80:0/1380085007 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe7cc102230 msgr2=0x7fe7cc19f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:16.386 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.380+0000 7fe7d12bc640 1 --2- 172.21.15.80:0/1380085007 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe7cc102230 0x7fe7cc19f070 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe7c4009fd0 tx=0x7fe7c400b260 comp rx=0 tx=0).stop 2024-04-24T23:27:16.386 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.380+0000 7fe7d12bc640 1 -- 172.21.15.80:0/1380085007 shutdown_connections 2024-04-24T23:27:16.386 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.380+0000 7fe7d12bc640 1 --2- 172.21.15.80:0/1380085007 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fe7b40a2b90 0x7fe7b40a5050 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.386 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.380+0000 7fe7d12bc640 1 --2- 172.21.15.80:0/1380085007 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe7cc102230 0x7fe7cc19f070 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.386 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.380+0000 7fe7d12bc640 1 --2- 172.21.15.80:0/1380085007 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe7cc101870 0x7fe7cc198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.386 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.380+0000 7fe7d12bc640 1 --2- 172.21.15.80:0/1380085007 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe7cc100680 0x7fe7cc197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.386 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.380+0000 7fe7d12bc640 1 -- 172.21.15.80:0/1380085007 >> 172.21.15.80:0/1380085007 conn(0x7fe7cc0fbbf0 msgr2=0x7fe7cc104c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:16.386 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.380+0000 7fe7d12bc640 1 -- 172.21.15.80:0/1380085007 shutdown_connections 2024-04-24T23:27:16.386 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.380+0000 7fe7d12bc640 1 -- 172.21.15.80:0/1380085007 wait complete. 2024-04-24T23:27:16.393 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:22:33.124428+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:27:16.393 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:27:16.503 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.497+0000 7f923b431640 1 -- 172.21.15.80:0/2884390993 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9234103340 msgr2=0x7f9234109bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:16.504 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.497+0000 7f923b431640 1 --2- 172.21.15.80:0/2884390993 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9234103340 0x7f9234109bd0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f922c009ca0 tx=0x7f922c02a8c0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.504 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.497+0000 7f923b431640 1 -- 172.21.15.80:0/2884390993 shutdown_connections 2024-04-24T23:27:16.504 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.497+0000 7f923b431640 1 --2- 172.21.15.80:0/2884390993 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9234103340 0x7f9234109bd0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.504 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.497+0000 7f923b431640 1 --2- 172.21.15.80:0/2884390993 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9234102980 0x7f9234102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.504 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.497+0000 7f923b431640 1 --2- 172.21.15.80:0/2884390993 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9234101790 0x7f9234101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.504 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.497+0000 7f923b431640 1 -- 172.21.15.80:0/2884390993 >> 172.21.15.80:0/2884390993 conn(0x7f92340fcd20 msgr2=0x7f92340ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:16.505 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.497+0000 7f923b431640 1 -- 172.21.15.80:0/2884390993 shutdown_connections 2024-04-24T23:27:16.505 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.498+0000 7f923b431640 1 -- 172.21.15.80:0/2884390993 wait complete. 2024-04-24T23:27:16.505 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.498+0000 7f923b431640 1 Processor -- start 2024-04-24T23:27:16.505 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.498+0000 7f923b431640 1 -- start start 2024-04-24T23:27:16.505 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.498+0000 7f923b431640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9234101790 0x7f9234197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:16.505 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.498+0000 7f923b431640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9234102980 0x7f9234197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:16.505 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.498+0000 7f923b431640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9234103340 0x7f923419f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:16.506 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.498+0000 7f923b431640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f923410d440 con 0x7f9234103340 2024-04-24T23:27:16.506 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.498+0000 7f923b431640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f923410d2c0 con 0x7f9234102980 2024-04-24T23:27:16.506 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.498+0000 7f923b431640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f923410d5c0 con 0x7f9234101790 2024-04-24T23:27:16.506 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.498+0000 7f92399ce640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9234103340 0x7f923419f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:16.506 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.498+0000 7f92391cd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9234101790 0x7f9234197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:16.506 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.498+0000 7f92399ce640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9234103340 0x7f923419f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:39132/0 (socket says 172.21.15.80:39132) 2024-04-24T23:27:16.507 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.498+0000 7f92399ce640 1 -- 172.21.15.80:0/3296286395 learned_addr learned my addr 172.21.15.80:0/3296286395 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:27:16.507 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.498+0000 7f92389cc640 1 --2- 172.21.15.80:0/3296286395 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9234102980 0x7f9234197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:16.507 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.499+0000 7f92399ce640 1 -- 172.21.15.80:0/3296286395 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9234101790 msgr2=0x7f9234197a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:16.507 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.499+0000 7f92399ce640 1 --2- 172.21.15.80:0/3296286395 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9234101790 0x7f9234197a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.507 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.499+0000 7f92399ce640 1 -- 172.21.15.80:0/3296286395 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9234102980 msgr2=0x7f9234197f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:16.507 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.499+0000 7f92399ce640 1 --2- 172.21.15.80:0/3296286395 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9234102980 0x7f9234197f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.507 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.499+0000 7f92399ce640 1 -- 172.21.15.80:0/3296286395 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f923419f670 con 0x7f9234103340 2024-04-24T23:27:16.507 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.499+0000 7f92391cd640 1 --2- 172.21.15.80:0/3296286395 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9234101790 0x7f9234197a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:27:16.508 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.499+0000 7f92399ce640 1 --2- 172.21.15.80:0/3296286395 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9234103340 0x7f923419f000 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f922c02f540 tx=0x7f922c031750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:16.508 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.499+0000 7f922a7fc640 1 -- 172.21.15.80:0/3296286395 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f922c003b70 con 0x7f9234103340 2024-04-24T23:27:16.508 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.499+0000 7f922a7fc640 1 -- 172.21.15.80:0/3296286395 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f922c003d10 con 0x7f9234103340 2024-04-24T23:27:16.508 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.499+0000 7f923b431640 1 -- 172.21.15.80:0/3296286395 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f923419f900 con 0x7f9234103340 2024-04-24T23:27:16.508 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.499+0000 7f922a7fc640 1 -- 172.21.15.80:0/3296286395 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f922c02fb50 con 0x7f9234103340 2024-04-24T23:27:16.508 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.499+0000 7f923b431640 1 -- 172.21.15.80:0/3296286395 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f923419fe50 con 0x7f9234103340 2024-04-24T23:27:16.508 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.500+0000 7f923b431640 1 -- 172.21.15.80:0/3296286395 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9234108a50 con 0x7f9234103340 2024-04-24T23:27:16.509 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.502+0000 7f922a7fc640 1 -- 172.21.15.80:0/3296286395 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f922c02fcf0 con 0x7f9234103340 2024-04-24T23:27:16.509 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.502+0000 7f922a7fc640 1 --2- 172.21.15.80:0/3296286395 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f92200a2b90 0x7f92200a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:16.509 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.502+0000 7f922a7fc640 1 -- 172.21.15.80:0/3296286395 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f922c0effb0 con 0x7f9234103340 2024-04-24T23:27:16.509 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.502+0000 7f92391cd640 1 --2- 172.21.15.80:0/3296286395 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f92200a2b90 0x7f92200a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:16.510 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.503+0000 7f92391cd640 1 --2- 172.21.15.80:0/3296286395 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f92200a2b90 0x7f92200a5050 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f92341027e0 tx=0x7f9230005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:16.510 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.507+0000 7f922a7fc640 1 -- 172.21.15.80:0/3296286395 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f922c0bd3b0 con 0x7f9234103340 2024-04-24T23:27:16.711 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.709+0000 7f923b431640 1 -- 172.21.15.80:0/3296286395 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9234062340 con 0x7f9234103340 2024-04-24T23:27:16.711 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.710+0000 7f922a7fc640 1 -- 172.21.15.80:0/3296286395 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7f922c0bccb0 con 0x7f9234103340 2024-04-24T23:27:16.711 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 1 2024-04-24T23:27:16.713 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.711+0000 7f923b431640 1 -- 172.21.15.80:0/3296286395 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f92200a2b90 msgr2=0x7f92200a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:16.713 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.711+0000 7f923b431640 1 --2- 172.21.15.80:0/3296286395 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f92200a2b90 0x7f92200a5050 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f92341027e0 tx=0x7f9230005f90 comp rx=0 tx=0).stop 2024-04-24T23:27:16.713 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.711+0000 7f923b431640 1 -- 172.21.15.80:0/3296286395 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9234103340 msgr2=0x7f923419f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:16.713 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.711+0000 7f923b431640 1 --2- 172.21.15.80:0/3296286395 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9234103340 0x7f923419f000 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f922c02f540 tx=0x7f922c031750 comp rx=0 tx=0).stop 2024-04-24T23:27:16.713 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.711+0000 7f923b431640 1 -- 172.21.15.80:0/3296286395 shutdown_connections 2024-04-24T23:27:16.713 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.711+0000 7f923b431640 1 --2- 172.21.15.80:0/3296286395 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f92200a2b90 0x7f92200a5050 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.713 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.711+0000 7f923b431640 1 --2- 172.21.15.80:0/3296286395 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9234103340 0x7f923419f000 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.713 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.712+0000 7f923b431640 1 --2- 172.21.15.80:0/3296286395 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9234102980 0x7f9234197f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.713 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.712+0000 7f923b431640 1 --2- 172.21.15.80:0/3296286395 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9234101790 0x7f9234197a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.713 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.712+0000 7f923b431640 1 -- 172.21.15.80:0/3296286395 >> 172.21.15.80:0/3296286395 conn(0x7f92340fcd20 msgr2=0x7f9234105c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:16.713 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.712+0000 7f923b431640 1 -- 172.21.15.80:0/3296286395 shutdown_connections 2024-04-24T23:27:16.713 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.712+0000 7f923b431640 1 -- 172.21.15.80:0/3296286395 wait complete. 2024-04-24T23:27:16.724 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-24T23:27:16.823 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.820+0000 7f2c9fa1c640 1 -- 172.21.15.80:0/3117927234 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2c981001d0 msgr2=0x7f2c98109d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:16.823 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.820+0000 7f2c9fa1c640 1 --2- 172.21.15.80:0/3117927234 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2c981001d0 0x7f2c98109d80 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f2c88009900 tx=0x7f2c8802a410 comp rx=0 tx=0).stop 2024-04-24T23:27:16.823 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.820+0000 7f2c9fa1c640 1 -- 172.21.15.80:0/3117927234 shutdown_connections 2024-04-24T23:27:16.824 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.820+0000 7f2c9fa1c640 1 --2- 172.21.15.80:0/3117927234 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2c981001d0 0x7f2c98109d80 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.824 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.820+0000 7f2c9fa1c640 1 --2- 172.21.15.80:0/3117927234 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2c980ff810 0x7f2c980ffc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.824 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.820+0000 7f2c9fa1c640 1 --2- 172.21.15.80:0/3117927234 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2c98102da0 0x7f2c981031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.824 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.820+0000 7f2c9fa1c640 1 -- 172.21.15.80:0/3117927234 >> 172.21.15.80:0/3117927234 conn(0x7f2c980fb3b0 msgr2=0x7f2c980fd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:16.824 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.820+0000 7f2c9fa1c640 1 -- 172.21.15.80:0/3117927234 shutdown_connections 2024-04-24T23:27:16.824 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.820+0000 7f2c9fa1c640 1 -- 172.21.15.80:0/3117927234 wait complete. 2024-04-24T23:27:16.824 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.820+0000 7f2c9fa1c640 1 Processor -- start 2024-04-24T23:27:16.824 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.820+0000 7f2c9fa1c640 1 -- start start 2024-04-24T23:27:16.824 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.820+0000 7f2c9fa1c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2c980ff810 0x7f2c981024a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:16.824 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.821+0000 7f2c9fa1c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2c981001d0 0x7f2c98100ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:16.824 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.821+0000 7f2c9fa1c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2c98102da0 0x7f2c98101020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:16.824 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.821+0000 7f2c9fa1c640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f2c9810d480 con 0x7f2c980ff810 2024-04-24T23:27:16.824 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.821+0000 7f2c9fa1c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2c9810d300 con 0x7f2c981001d0 2024-04-24T23:27:16.824 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.821+0000 7f2c9fa1c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f2c9810d600 con 0x7f2c98102da0 2024-04-24T23:27:16.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.821+0000 7f2c9d7b8640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2c980ff810 0x7f2c981024a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:16.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.821+0000 7f2c9d7b8640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2c980ff810 0x7f2c981024a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:39154/0 (socket says 172.21.15.80:39154) 2024-04-24T23:27:16.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.821+0000 7f2c9d7b8640 1 -- 172.21.15.80:0/439842181 learned_addr learned my addr 172.21.15.80:0/439842181 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:27:16.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.821+0000 7f2c9dfb9640 1 --2- 172.21.15.80:0/439842181 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2c98102da0 0x7f2c98101020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:16.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.821+0000 7f2c9cfb7640 1 --2- 172.21.15.80:0/439842181 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2c981001d0 0x7f2c98100ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:16.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.821+0000 7f2c9d7b8640 1 -- 172.21.15.80:0/439842181 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2c98102da0 msgr2=0x7f2c98101020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:16.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.821+0000 7f2c9d7b8640 1 --2- 172.21.15.80:0/439842181 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2c98102da0 0x7f2c98101020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.821+0000 7f2c9d7b8640 1 -- 172.21.15.80:0/439842181 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2c981001d0 msgr2=0x7f2c98100ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:16.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.821+0000 7f2c9d7b8640 1 --2- 172.21.15.80:0/439842181 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2c981001d0 0x7f2c98100ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:16.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.821+0000 7f2c9d7b8640 1 -- 172.21.15.80:0/439842181 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c981018b0 con 0x7f2c980ff810 2024-04-24T23:27:16.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.821+0000 7f2c9dfb9640 1 --2- 172.21.15.80:0/439842181 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2c98102da0 0x7f2c98101020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:27:16.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.821+0000 7f2c9cfb7640 1 --2- 172.21.15.80:0/439842181 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2c981001d0 0x7f2c98100ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:27:16.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.821+0000 7f2c9d7b8640 1 --2- 172.21.15.80:0/439842181 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2c980ff810 0x7f2c981024a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f2c9000ccb0 tx=0x7f2c9000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:16.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.821+0000 7f2c967fc640 1 -- 172.21.15.80:0/439842181 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c90003990 con 0x7f2c980ff810 2024-04-24T23:27:16.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.821+0000 7f2c967fc640 1 -- 172.21.15.80:0/439842181 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c90003b30 con 0x7f2c980ff810 2024-04-24T23:27:16.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.821+0000 7f2c9fa1c640 1 -- 172.21.15.80:0/439842181 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c981a1b80 con 0x7f2c980ff810 2024-04-24T23:27:16.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.821+0000 7f2c967fc640 1 -- 172.21.15.80:0/439842181 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c9000f7b0 con 0x7f2c980ff810 2024-04-24T23:27:16.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.821+0000 7f2c9fa1c640 1 -- 172.21.15.80:0/439842181 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c981a20d0 con 0x7f2c980ff810 2024-04-24T23:27:16.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.822+0000 7f2c9fa1c640 1 -- 172.21.15.80:0/439842181 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c98108c00 con 0x7f2c980ff810 2024-04-24T23:27:16.827 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.823+0000 7f2c967fc640 1 -- 172.21.15.80:0/439842181 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f2c9000f950 con 0x7f2c980ff810 2024-04-24T23:27:16.827 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.824+0000 7f2c967fc640 1 --2- 172.21.15.80:0/439842181 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f2c800a2b90 0x7f2c800a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:16.827 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.824+0000 7f2c9cfb7640 1 --2- 172.21.15.80:0/439842181 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f2c800a2b90 0x7f2c800a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:16.827 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.824+0000 7f2c967fc640 1 -- 172.21.15.80:0/439842181 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f2c900cee00 con 0x7f2c980ff810 2024-04-24T23:27:16.827 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.824+0000 7f2c9cfb7640 1 --2- 172.21.15.80:0/439842181 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f2c800a2b90 0x7f2c800a5050 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c006fd0 tx=0x7f2c8c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:16.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:16.828+0000 7f2c967fc640 1 -- 172.21.15.80:0/439842181 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f2c90014030 con 0x7f2c980ff810 2024-04-24T23:27:17.022 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:17.021+0000 7f2c9fa1c640 1 -- 172.21.15.80:0/439842181 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f2c9802d3e0 con 0x7f2c980ff810 2024-04-24T23:27:17.022 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:17.021+0000 7f2c967fc640 1 -- 172.21.15.80:0/439842181 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v81) ==== 84+0+65 (secure 0 0 0) 0x7f2c9009c200 con 0x7f2c980ff810 2024-04-24T23:27:17.023 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-24T23:27:17.023 INFO:teuthology.orchestra.run.smithi080.stdout:[ 2024-04-24T23:27:17.023 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-24T23:27:17.023 INFO:teuthology.orchestra.run.smithi080.stdout: "poolnum": 1, 2024-04-24T23:27:17.023 INFO:teuthology.orchestra.run.smithi080.stdout: "poolname": ".mgr" 2024-04-24T23:27:17.023 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:27:17.023 INFO:teuthology.orchestra.run.smithi080.stdout:] 2024-04-24T23:27:17.025 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:17.023+0000 7f2c9fa1c640 1 -- 172.21.15.80:0/439842181 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f2c800a2b90 msgr2=0x7f2c800a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:17.025 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:17.023+0000 7f2c9fa1c640 1 --2- 172.21.15.80:0/439842181 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f2c800a2b90 0x7f2c800a5050 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c006fd0 tx=0x7f2c8c008040 comp rx=0 tx=0).stop 2024-04-24T23:27:17.025 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:17.023+0000 7f2c9fa1c640 1 -- 172.21.15.80:0/439842181 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2c980ff810 msgr2=0x7f2c981024a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:17.025 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:17.023+0000 7f2c9fa1c640 1 --2- 172.21.15.80:0/439842181 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2c980ff810 0x7f2c981024a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f2c9000ccb0 tx=0x7f2c9000a620 comp rx=0 tx=0).stop 2024-04-24T23:27:17.025 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:17.024+0000 7f2c9fa1c640 1 -- 172.21.15.80:0/439842181 shutdown_connections 2024-04-24T23:27:17.025 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:17.024+0000 7f2c9fa1c640 1 --2- 172.21.15.80:0/439842181 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f2c800a2b90 0x7f2c800a5050 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:17.025 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:17.024+0000 7f2c9fa1c640 1 --2- 172.21.15.80:0/439842181 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2c98102da0 0x7f2c98101020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:17.025 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:17.024+0000 7f2c9fa1c640 1 --2- 172.21.15.80:0/439842181 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2c981001d0 0x7f2c98100ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:17.025 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:17.024+0000 7f2c9fa1c640 1 --2- 172.21.15.80:0/439842181 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2c980ff810 0x7f2c981024a0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:17.026 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:17.024+0000 7f2c9fa1c640 1 -- 172.21.15.80:0/439842181 >> 172.21.15.80:0/439842181 conn(0x7f2c980fb3b0 msgr2=0x7f2c98105dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:17.026 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:17.024+0000 7f2c9fa1c640 1 -- 172.21.15.80:0/439842181 shutdown_connections 2024-04-24T23:27:17.026 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:17.024+0000 7f2c9fa1c640 1 -- 172.21.15.80:0/439842181 wait complete. 2024-04-24T23:27:17.026 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:16 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:27:17.027 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:16 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/2832680235' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:27:17.027 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:16 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/1380085007' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:27:17.027 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:16 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/3296286395' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:27:17.027 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:16 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:27:17.036 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-24T23:27:17.036 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-24T23:27:17.101 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:16 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:27:17.101 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:16 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/2832680235' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:27:17.101 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:16 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/1380085007' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:27:17.101 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:16 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/3296286395' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:27:17.101 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:16 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:27:17.141 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:17.138+0000 7fb55e709640 1 -- 172.21.15.80:0/3810570767 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb558101870 msgr2=0x7fb558101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:17.141 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:17.138+0000 7fb55e709640 1 --2- 172.21.15.80:0/3810570767 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb558101870 0x7fb558101cf0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb54c009900 tx=0x7fb54c02a410 comp rx=0 tx=0).stop 2024-04-24T23:27:17.141 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:17.138+0000 7fb55e709640 1 -- 172.21.15.80:0/3810570767 shutdown_connections 2024-04-24T23:27:17.141 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:17.138+0000 7fb55e709640 1 --2- 172.21.15.80:0/3810570767 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb558102230 0x7fb558108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:17.141 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:17.138+0000 7fb55e709640 1 --2- 172.21.15.80:0/3810570767 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb558101870 0x7fb558101cf0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:17.141 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:17.138+0000 7fb55e709640 1 --2- 172.21.15.80:0/3810570767 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb558100680 0x7fb558100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:17.141 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:17.138+0000 7fb55e709640 1 -- 172.21.15.80:0/3810570767 >> 172.21.15.80:0/3810570767 conn(0x7fb5580fbbf0 msgr2=0x7fb5580fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:17.141 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:17.138+0000 7fb55e709640 1 -- 172.21.15.80:0/3810570767 shutdown_connections 2024-04-24T23:27:17.141 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:17.138+0000 7fb55e709640 1 -- 172.21.15.80:0/3810570767 wait complete. 2024-04-24T23:27:17.141 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:17.138+0000 7fb55e709640 1 Processor -- start 2024-04-24T23:27:17.142 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:17.138+0000 7fb55e709640 1 -- start start 2024-04-24T23:27:17.142 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:17.139+0000 7fb55e709640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb558100680 0x7fb5580694a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:17.142 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:17.139+0000 7fb55e709640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb558101870 0x7fb55819c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:17.142 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:17.139+0000 7fb55e709640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb558102230 0x7fb55819edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:17.142 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:17.139+0000 7fb55e709640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fb55810c300 con 0x7fb558101870 2024-04-24T23:27:17.142 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:17.139+0000 7fb55e709640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb55810c180 con 0x7fb558100680 2024-04-24T23:27:17.142 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:17.139+0000 7fb55e709640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb55810c480 con 0x7fb558102230 2024-04-24T23:27:17.142 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:17.139+0000 7fb5577fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb558101870 0x7fb55819c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:17.142 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:17.139+0000 7fb5577fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb558101870 0x7fb55819c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:39162/0 (socket says 172.21.15.80:39162) 2024-04-24T23:27:17.142 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:17.139+0000 7fb5577fe640 1 -- 172.21.15.80:0/1839905228 learned_addr learned my addr 172.21.15.80:0/1839905228 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:27:17.142 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:17.139+0000 7fb55cca6640 1 --2- 172.21.15.80:0/1839905228 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb558102230 0x7fb55819edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:17.142 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:17.139+0000 7fb557fff640 1 --2- 172.21.15.80:0/1839905228 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb558100680 0x7fb5580694a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:17.142 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:17.139+0000 7fb5577fe640 1 -- 172.21.15.80:0/1839905228 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb558102230 msgr2=0x7fb55819edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:17.143 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:17.139+0000 7fb5577fe640 1 --2- 172.21.15.80:0/1839905228 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb558102230 0x7fb55819edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:17.143 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:17.139+0000 7fb5577fe640 1 -- 172.21.15.80:0/1839905228 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb558100680 msgr2=0x7fb5580694a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:17.143 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:17.139+0000 7fb5577fe640 1 --2- 172.21.15.80:0/1839905228 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb558100680 0x7fb5580694a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:17.143 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:17.139+0000 7fb5577fe640 1 -- 172.21.15.80:0/1839905228 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb55819f690 con 0x7fb558101870 2024-04-24T23:27:17.143 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:17.139+0000 7fb5577fe640 1 --2- 172.21.15.80:0/1839905228 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb558101870 0x7fb55819c760 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb54c009230 tx=0x7fb54c004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:17.143 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:17.139+0000 7fb5557fa640 1 -- 172.21.15.80:0/1839905228 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb54c03f030 con 0x7fb558101870 2024-04-24T23:27:17.143 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:17.139+0000 7fb5557fa640 1 -- 172.21.15.80:0/1839905228 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb54c003ce0 con 0x7fb558101870 2024-04-24T23:27:17.143 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:17.139+0000 7fb55e709640 1 -- 172.21.15.80:0/1839905228 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb55819f920 con 0x7fb558101870 2024-04-24T23:27:17.143 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:17.139+0000 7fb5557fa640 1 -- 172.21.15.80:0/1839905228 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb54c035070 con 0x7fb558101870 2024-04-24T23:27:17.143 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:17.139+0000 7fb55e709640 1 -- 172.21.15.80:0/1839905228 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb55819fe10 con 0x7fb558101870 2024-04-24T23:27:17.144 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:17.140+0000 7fb55e709640 1 -- 172.21.15.80:0/1839905228 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5581079f0 con 0x7fb558101870 2024-04-24T23:27:17.144 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:17.141+0000 7fb5557fa640 1 -- 172.21.15.80:0/1839905228 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fb54c042030 con 0x7fb558101870 2024-04-24T23:27:17.144 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:17.142+0000 7fb5557fa640 1 --2- 172.21.15.80:0/1839905228 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fb5400a2b90 0x7fb5400a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:17.144 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:17.142+0000 7fb557fff640 1 --2- 172.21.15.80:0/1839905228 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fb5400a2b90 0x7fb5400a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:17.144 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:17.142+0000 7fb5557fa640 1 -- 172.21.15.80:0/1839905228 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fb54c0ef6e0 con 0x7fb558101870 2024-04-24T23:27:17.144 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:17.142+0000 7fb557fff640 1 --2- 172.21.15.80:0/1839905228 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fb5400a2b90 0x7fb5400a5050 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb548005fd0 tx=0x7fb548005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:17.147 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:17.146+0000 7fb5557fa640 1 -- 172.21.15.80:0/1839905228 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fb54c0bca40 con 0x7fb558101870 2024-04-24T23:27:17.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:16 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:27:17.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:16 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/2832680235' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:27:17.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:16 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/1380085007' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:27:17.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:16 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/3296286395' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:27:17.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:16 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:27:17.326 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:17.325+0000 7fb55e709640 1 -- 172.21.15.80:0/1839905228 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fb558062340 con 0x7fb558101870 2024-04-24T23:27:17.947 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:17.946+0000 7fb5557fa640 1 -- 172.21.15.80:0/1839905228 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v82) ==== 138+0+0 (secure 0 0 0) 0x7fb54c0bc340 con 0x7fb558101870 2024-04-24T23:27:17.947 INFO:teuthology.orchestra.run.smithi080.stderr:pool 'cephfs_metadata' created 2024-04-24T23:27:17.949 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:17.948+0000 7fb55e709640 1 -- 172.21.15.80:0/1839905228 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fb5400a2b90 msgr2=0x7fb5400a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:17.949 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:17.948+0000 7fb55e709640 1 --2- 172.21.15.80:0/1839905228 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fb5400a2b90 0x7fb5400a5050 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb548005fd0 tx=0x7fb548005850 comp rx=0 tx=0).stop 2024-04-24T23:27:17.949 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:17.948+0000 7fb55e709640 1 -- 172.21.15.80:0/1839905228 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb558101870 msgr2=0x7fb55819c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:17.949 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:17.948+0000 7fb55e709640 1 --2- 172.21.15.80:0/1839905228 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb558101870 0x7fb55819c760 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb54c009230 tx=0x7fb54c004360 comp rx=0 tx=0).stop 2024-04-24T23:27:17.950 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:17.948+0000 7fb55e709640 1 -- 172.21.15.80:0/1839905228 shutdown_connections 2024-04-24T23:27:17.950 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:17.948+0000 7fb55e709640 1 --2- 172.21.15.80:0/1839905228 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fb5400a2b90 0x7fb5400a5050 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:17.950 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:17.948+0000 7fb55e709640 1 --2- 172.21.15.80:0/1839905228 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb558102230 0x7fb55819edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:17.950 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:17.948+0000 7fb55e709640 1 --2- 172.21.15.80:0/1839905228 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb558101870 0x7fb55819c760 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:17.950 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:17.948+0000 7fb55e709640 1 --2- 172.21.15.80:0/1839905228 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb558100680 0x7fb5580694a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:17.950 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:17.948+0000 7fb55e709640 1 -- 172.21.15.80:0/1839905228 >> 172.21.15.80:0/1839905228 conn(0x7fb5580fbbf0 msgr2=0x7fb558104bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:17.950 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:17.948+0000 7fb55e709640 1 -- 172.21.15.80:0/1839905228 shutdown_connections 2024-04-24T23:27:17.950 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:17.948+0000 7fb55e709640 1 -- 172.21.15.80:0/1839905228 wait complete. 2024-04-24T23:27:17.963 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-04-24T23:27:18.070 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:18.067+0000 7f86358ba640 1 -- 172.21.15.80:0/589524474 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f86301022e0 msgr2=0x7f86301046d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:18.070 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:18.067+0000 7f86358ba640 1 --2- 172.21.15.80:0/589524474 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f86301022e0 0x7f86301046d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8620009900 tx=0x7f862002a410 comp rx=0 tx=0).stop 2024-04-24T23:27:18.070 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:18.067+0000 7f86358ba640 1 -- 172.21.15.80:0/589524474 shutdown_connections 2024-04-24T23:27:18.070 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:18.067+0000 7f86358ba640 1 --2- 172.21.15.80:0/589524474 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8630107540 0x7f8630109930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:18.071 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:18.067+0000 7f86358ba640 1 --2- 172.21.15.80:0/589524474 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8630104c10 0x7f8630107000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:18.071 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:18.067+0000 7f86358ba640 1 --2- 172.21.15.80:0/589524474 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f86301022e0 0x7f86301046d0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:18.071 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:18.067+0000 7f86358ba640 1 -- 172.21.15.80:0/589524474 >> 172.21.15.80:0/589524474 conn(0x7f86300fbc10 msgr2=0x7f86300fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:18.071 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:18.067+0000 7f86358ba640 1 -- 172.21.15.80:0/589524474 shutdown_connections 2024-04-24T23:27:18.071 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:18.067+0000 7f86358ba640 1 -- 172.21.15.80:0/589524474 wait complete. 2024-04-24T23:27:18.071 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:18.068+0000 7f86358ba640 1 Processor -- start 2024-04-24T23:27:18.071 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:18.068+0000 7f86358ba640 1 -- start start 2024-04-24T23:27:18.071 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:18.068+0000 7f86358ba640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86301022e0 0x7f8630197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:18.071 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:18.068+0000 7f86358ba640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8630104c10 0x7f8630198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:18.071 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:18.068+0000 7f86358ba640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8630107540 0x7f863019f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:18.071 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:18.068+0000 7f86358ba640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f863010d450 con 0x7f8630104c10 2024-04-24T23:27:18.071 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:18.068+0000 7f86358ba640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f863010d2d0 con 0x7f86301022e0 2024-04-24T23:27:18.071 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:18.068+0000 7f86358ba640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f863010d5d0 con 0x7f8630107540 2024-04-24T23:27:18.071 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:18.068+0000 7f862e7fc640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8630104c10 0x7f8630198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:18.071 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:18.068+0000 7f862e7fc640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8630104c10 0x7f8630198070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:39190/0 (socket says 172.21.15.80:39190) 2024-04-24T23:27:18.072 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:18.068+0000 7f862e7fc640 1 -- 172.21.15.80:0/2830769121 learned_addr learned my addr 172.21.15.80:0/2830769121 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:27:18.072 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:18.068+0000 7f862f7fe640 1 --2- 172.21.15.80:0/2830769121 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8630107540 0x7f863019f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:18.072 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:18.068+0000 7f862e7fc640 1 -- 172.21.15.80:0/2830769121 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8630107540 msgr2=0x7f863019f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:18.072 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:18.068+0000 7f862effd640 1 --2- 172.21.15.80:0/2830769121 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86301022e0 0x7f8630197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:18.072 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:18.068+0000 7f862e7fc640 1 --2- 172.21.15.80:0/2830769121 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8630107540 0x7f863019f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:18.072 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:18.068+0000 7f862e7fc640 1 -- 172.21.15.80:0/2830769121 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86301022e0 msgr2=0x7f8630197b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:18.072 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:18.068+0000 7f862e7fc640 1 --2- 172.21.15.80:0/2830769121 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86301022e0 0x7f8630197b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:18.072 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:18.068+0000 7f862e7fc640 1 -- 172.21.15.80:0/2830769121 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f863019f750 con 0x7f8630104c10 2024-04-24T23:27:18.072 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:18.068+0000 7f862f7fe640 1 --2- 172.21.15.80:0/2830769121 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8630107540 0x7f863019f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:27:18.072 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:18.069+0000 7f862e7fc640 1 --2- 172.21.15.80:0/2830769121 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8630104c10 0x7f8630198070 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f862400c970 tx=0x7f862400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:18.072 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:18.069+0000 7f86348b8640 1 -- 172.21.15.80:0/2830769121 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8624007bb0 con 0x7f8630104c10 2024-04-24T23:27:18.072 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:18.069+0000 7f86348b8640 1 -- 172.21.15.80:0/2830769121 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8624007d50 con 0x7f8630104c10 2024-04-24T23:27:18.072 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:18.069+0000 7f86358ba640 1 -- 172.21.15.80:0/2830769121 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f863019fa40 con 0x7f8630104c10 2024-04-24T23:27:18.072 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:18.069+0000 7f86348b8640 1 -- 172.21.15.80:0/2830769121 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f862400f450 con 0x7f8630104c10 2024-04-24T23:27:18.072 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:18.069+0000 7f86358ba640 1 -- 172.21.15.80:0/2830769121 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f863019ff90 con 0x7f8630104c10 2024-04-24T23:27:18.073 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:18.070+0000 7f86358ba640 1 -- 172.21.15.80:0/2830769121 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86300fd700 con 0x7f8630104c10 2024-04-24T23:27:18.073 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:18.071+0000 7f86348b8640 1 -- 172.21.15.80:0/2830769121 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f8624004f70 con 0x7f8630104c10 2024-04-24T23:27:18.074 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:18.071+0000 7f86348b8640 1 --2- 172.21.15.80:0/2830769121 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f86180a2b90 0x7f86180a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:18.074 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:18.072+0000 7f862effd640 1 --2- 172.21.15.80:0/2830769121 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f86180a2b90 0x7f86180a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:18.074 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:18.072+0000 7f86348b8640 1 -- 172.21.15.80:0/2830769121 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7497+0+0 (secure 0 0 0) 0x7f86240cef10 con 0x7f8630104c10 2024-04-24T23:27:18.074 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:18.072+0000 7f862effd640 1 --2- 172.21.15.80:0/2830769121 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f86180a2b90 0x7f86180a5050 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8620031610 tx=0x7f8620000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:18.076 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:18.075+0000 7f86348b8640 1 -- 172.21.15.80:0/2830769121 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f8624014030 con 0x7f8630104c10 2024-04-24T23:27:18.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:17 smithi134 ceph-mon[32619]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:27:18.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:17 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/439842181' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-24T23:27:18.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:17 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/1839905228' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-24T23:27:18.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:17 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:27:18.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:17 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:27:18.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:17 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:27:18.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:17 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:27:18.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:17 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:27:18.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:17 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:27:18.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:17 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:27:18.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:17 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:27:18.198 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:17 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:27:18.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:17 smithi148 ceph-mon[32528]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:27:18.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:17 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/439842181' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-24T23:27:18.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:17 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/1839905228' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-24T23:27:18.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:17 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:27:18.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:17 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:27:18.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:17 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:27:18.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:17 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:27:18.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:17 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:27:18.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:17 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:27:18.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:17 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:27:18.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:17 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:27:18.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:17 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:27:18.259 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:18.259+0000 7f86358ba640 1 -- 172.21.15.80:0/2830769121 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7f863002d170 con 0x7f8630104c10 2024-04-24T23:27:18.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:17 smithi080 ceph-mon[29152]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:27:18.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:17 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/439842181' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-24T23:27:18.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:17 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/1839905228' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-24T23:27:18.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:17 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:27:18.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:17 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:27:18.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:17 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:27:18.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:17 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:27:18.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:17 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:27:18.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:17 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:27:18.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:17 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:27:18.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:17 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:27:18.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:17 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:27:18.949 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:18.948+0000 7f86348b8640 1 -- 172.21.15.80:0/2830769121 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v83) ==== 171+0+0 (secure 0 0 0) 0x7f862409c1a0 con 0x7f8630104c10 2024-04-24T23:27:18.949 INFO:teuthology.orchestra.run.smithi080.stderr:pool 'cephfs_data' created 2024-04-24T23:27:18.952 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:18.951+0000 7f86358ba640 1 -- 172.21.15.80:0/2830769121 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f86180a2b90 msgr2=0x7f86180a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:18.952 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:18.951+0000 7f86358ba640 1 --2- 172.21.15.80:0/2830769121 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f86180a2b90 0x7f86180a5050 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8620031610 tx=0x7f8620000bc0 comp rx=0 tx=0).stop 2024-04-24T23:27:18.952 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:18.951+0000 7f86358ba640 1 -- 172.21.15.80:0/2830769121 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8630104c10 msgr2=0x7f8630198070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:18.952 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:18.951+0000 7f86358ba640 1 --2- 172.21.15.80:0/2830769121 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8630104c10 0x7f8630198070 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f862400c970 tx=0x7f862400ce30 comp rx=0 tx=0).stop 2024-04-24T23:27:18.952 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:18.951+0000 7f86358ba640 1 -- 172.21.15.80:0/2830769121 shutdown_connections 2024-04-24T23:27:18.952 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:18.951+0000 7f86358ba640 1 --2- 172.21.15.80:0/2830769121 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f86180a2b90 0x7f86180a5050 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:18.953 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:18.951+0000 7f86358ba640 1 --2- 172.21.15.80:0/2830769121 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8630107540 0x7f863019f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:18.953 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:18.951+0000 7f86358ba640 1 --2- 172.21.15.80:0/2830769121 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8630104c10 0x7f8630198070 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:18.953 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:18.951+0000 7f86358ba640 1 --2- 172.21.15.80:0/2830769121 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86301022e0 0x7f8630197b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:18.953 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:18.951+0000 7f86358ba640 1 -- 172.21.15.80:0/2830769121 >> 172.21.15.80:0/2830769121 conn(0x7f86300fbc10 msgr2=0x7f8630102c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:18.953 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:18.951+0000 7f86358ba640 1 -- 172.21.15.80:0/2830769121 shutdown_connections 2024-04-24T23:27:18.953 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:18.952+0000 7f86358ba640 1 -- 172.21.15.80:0/2830769121 wait complete. 2024-04-24T23:27:18.973 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-24T23:27:19.078 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:19.075+0000 7fcbb5b28640 1 -- 172.21.15.80:0/3140157737 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcbb01033e0 msgr2=0x7fcbb0109c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:19.078 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:19.075+0000 7fcbb5b28640 1 --2- 172.21.15.80:0/3140157737 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcbb01033e0 0x7fcbb0109c70 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fcba8009cd0 tx=0x7fcba802a8e0 comp rx=0 tx=0).stop 2024-04-24T23:27:19.078 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:19.075+0000 7fcbb5b28640 1 -- 172.21.15.80:0/3140157737 shutdown_connections 2024-04-24T23:27:19.079 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:19.075+0000 7fcbb5b28640 1 --2- 172.21.15.80:0/3140157737 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcbb01033e0 0x7fcbb0109c70 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:19.079 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:19.075+0000 7fcbb5b28640 1 --2- 172.21.15.80:0/3140157737 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcbb0102a20 0x7fcbb0102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:19.079 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:19.075+0000 7fcbb5b28640 1 --2- 172.21.15.80:0/3140157737 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcbb00ffa50 0x7fcbb00ffe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:19.079 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:19.075+0000 7fcbb5b28640 1 -- 172.21.15.80:0/3140157737 >> 172.21.15.80:0/3140157737 conn(0x7fcbb00fb380 msgr2=0x7fcbb00fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:19.079 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:19.075+0000 7fcbb5b28640 1 -- 172.21.15.80:0/3140157737 shutdown_connections 2024-04-24T23:27:19.079 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:19.075+0000 7fcbb5b28640 1 -- 172.21.15.80:0/3140157737 wait complete. 2024-04-24T23:27:19.079 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:19.075+0000 7fcbb5b28640 1 Processor -- start 2024-04-24T23:27:19.079 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:19.076+0000 7fcbb5b28640 1 -- start start 2024-04-24T23:27:19.079 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:19.076+0000 7fcbb5b28640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcbb00ffa50 0x7fcbb0197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:19.079 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:19.076+0000 7fcbb5b28640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcbb0102a20 0x7fcbb0198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:19.079 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:19.076+0000 7fcbb5b28640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcbb01033e0 0x7fcbb019f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:19.079 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:19.076+0000 7fcbb5b28640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fcbb010d4a0 con 0x7fcbb01033e0 2024-04-24T23:27:19.079 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:19.076+0000 7fcbb5b28640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcbb010d320 con 0x7fcbb0102a20 2024-04-24T23:27:19.079 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:19.076+0000 7fcbb5b28640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fcbb010d620 con 0x7fcbb00ffa50 2024-04-24T23:27:19.079 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:19.076+0000 7fcbaffff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcbb01033e0 0x7fcbb019f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:19.080 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:19.076+0000 7fcbaffff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcbb01033e0 0x7fcbb019f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:39202/0 (socket says 172.21.15.80:39202) 2024-04-24T23:27:19.080 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:19.076+0000 7fcbaffff640 1 -- 172.21.15.80:0/1259907447 learned_addr learned my addr 172.21.15.80:0/1259907447 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:27:19.080 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:19.076+0000 7fcbaeffd640 1 --2- 172.21.15.80:0/1259907447 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcbb0102a20 0x7fcbb0198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:19.080 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:19.076+0000 7fcbaf7fe640 1 --2- 172.21.15.80:0/1259907447 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcbb00ffa50 0x7fcbb0197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:19.080 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:19.076+0000 7fcbaffff640 1 -- 172.21.15.80:0/1259907447 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcbb00ffa50 msgr2=0x7fcbb0197ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:19.080 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:19.076+0000 7fcbaffff640 1 --2- 172.21.15.80:0/1259907447 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcbb00ffa50 0x7fcbb0197ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:19.080 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:19.076+0000 7fcbaffff640 1 -- 172.21.15.80:0/1259907447 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcbb0102a20 msgr2=0x7fcbb0198000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:19.080 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:19.076+0000 7fcbaffff640 1 --2- 172.21.15.80:0/1259907447 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcbb0102a20 0x7fcbb0198000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:19.080 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:19.076+0000 7fcbaffff640 1 -- 172.21.15.80:0/1259907447 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbb019f6e0 con 0x7fcbb01033e0 2024-04-24T23:27:19.080 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:19.076+0000 7fcbaf7fe640 1 --2- 172.21.15.80:0/1259907447 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcbb00ffa50 0x7fcbb0197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:27:19.080 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:19.076+0000 7fcbaeffd640 1 --2- 172.21.15.80:0/1259907447 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcbb0102a20 0x7fcbb0198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:27:19.080 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:19.076+0000 7fcbaffff640 1 --2- 172.21.15.80:0/1259907447 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcbb01033e0 0x7fcbb019f070 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fcba8004af0 tx=0x7fcba8003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:19.080 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:19.076+0000 7fcbacff9640 1 -- 172.21.15.80:0/1259907447 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba8004450 con 0x7fcbb01033e0 2024-04-24T23:27:19.081 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:19.076+0000 7fcbacff9640 1 -- 172.21.15.80:0/1259907447 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcba80045f0 con 0x7fcbb01033e0 2024-04-24T23:27:19.081 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:19.076+0000 7fcbb5b28640 1 -- 172.21.15.80:0/1259907447 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbb019f970 con 0x7fcbb01033e0 2024-04-24T23:27:19.081 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:19.076+0000 7fcbacff9640 1 -- 172.21.15.80:0/1259907447 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba80466c0 con 0x7fcbb01033e0 2024-04-24T23:27:19.081 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:19.076+0000 7fcbb5b28640 1 -- 172.21.15.80:0/1259907447 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbb019ff40 con 0x7fcbb01033e0 2024-04-24T23:27:19.081 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:19.077+0000 7fcbb5b28640 1 -- 172.21.15.80:0/1259907447 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbb0108b50 con 0x7fcbb01033e0 2024-04-24T23:27:19.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:19.078+0000 7fcbacff9640 1 -- 172.21.15.80:0/1259907447 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fcba8046860 con 0x7fcbb01033e0 2024-04-24T23:27:19.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:19.079+0000 7fcbacff9640 1 --2- 172.21.15.80:0/1259907447 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fcb980a2b90 0x7fcb980a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:19.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:19.079+0000 7fcbaf7fe640 1 --2- 172.21.15.80:0/1259907447 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fcb980a2b90 0x7fcb980a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:19.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:19.079+0000 7fcbacff9640 1 -- 172.21.15.80:0/1259907447 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7879+0+0 (secure 0 0 0) 0x7fcba802e030 con 0x7fcbb01033e0 2024-04-24T23:27:19.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:19.080+0000 7fcbaf7fe640 1 --2- 172.21.15.80:0/1259907447 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fcb980a2b90 0x7fcb980a5050 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fcbb0069bc0 tx=0x7fcba0005e70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:19.084 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:19.083+0000 7fcbacff9640 1 -- 172.21.15.80:0/1259907447 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fcba804b060 con 0x7fcbb01033e0 2024-04-24T23:27:19.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:18 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/1839905228' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-24T23:27:19.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:18 smithi148 ceph-mon[32528]: osdmap e82: 12 total, 12 up, 12 in 2024-04-24T23:27:19.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:18 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/2830769121' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-04-24T23:27:19.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:18 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/1839905228' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-24T23:27:19.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:18 smithi080 ceph-mon[29152]: osdmap e82: 12 total, 12 up, 12 in 2024-04-24T23:27:19.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:18 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/2830769121' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-04-24T23:27:19.299 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:19.298+0000 7fcbb5b28640 1 -- 172.21.15.80:0/1259907447 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fcbb002d170 con 0x7fcbb01033e0 2024-04-24T23:27:19.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:18 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/1839905228' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-24T23:27:19.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:18 smithi134 ceph-mon[32619]: osdmap e82: 12 total, 12 up, 12 in 2024-04-24T23:27:19.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:18 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/2830769121' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-04-24T23:27:20.186 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:20.185+0000 7fcbacff9640 1 -- 172.21.15.80:0/1259907447 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7fcba80bdab0 con 0x7fcbb01033e0 2024-04-24T23:27:20.186 INFO:teuthology.orchestra.run.smithi080.stderr:new fs with metadata pool 2 and data pool 3 2024-04-24T23:27:20.189 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:20.188+0000 7fcbb5b28640 1 -- 172.21.15.80:0/1259907447 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fcb980a2b90 msgr2=0x7fcb980a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:20.189 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:20.188+0000 7fcbb5b28640 1 --2- 172.21.15.80:0/1259907447 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fcb980a2b90 0x7fcb980a5050 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fcbb0069bc0 tx=0x7fcba0005e70 comp rx=0 tx=0).stop 2024-04-24T23:27:20.189 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:20.188+0000 7fcbb5b28640 1 -- 172.21.15.80:0/1259907447 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcbb01033e0 msgr2=0x7fcbb019f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:20.189 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:20.188+0000 7fcbb5b28640 1 --2- 172.21.15.80:0/1259907447 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcbb01033e0 0x7fcbb019f070 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fcba8004af0 tx=0x7fcba8003c10 comp rx=0 tx=0).stop 2024-04-24T23:27:20.189 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:20.188+0000 7fcbb5b28640 1 -- 172.21.15.80:0/1259907447 shutdown_connections 2024-04-24T23:27:20.190 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:20.188+0000 7fcbb5b28640 1 --2- 172.21.15.80:0/1259907447 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fcb980a2b90 0x7fcb980a5050 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:20.190 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:20.188+0000 7fcbb5b28640 1 --2- 172.21.15.80:0/1259907447 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcbb01033e0 0x7fcbb019f070 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:20.190 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:20.188+0000 7fcbb5b28640 1 --2- 172.21.15.80:0/1259907447 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcbb0102a20 0x7fcbb0198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:20.190 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:20.188+0000 7fcbb5b28640 1 --2- 172.21.15.80:0/1259907447 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcbb00ffa50 0x7fcbb0197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:20.190 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:20.188+0000 7fcbb5b28640 1 -- 172.21.15.80:0/1259907447 >> 172.21.15.80:0/1259907447 conn(0x7fcbb00fb380 msgr2=0x7fcbb0105d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:20.190 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:20.188+0000 7fcbb5b28640 1 -- 172.21.15.80:0/1259907447 shutdown_connections 2024-04-24T23:27:20.190 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:20.188+0000 7fcbb5b28640 1 -- 172.21.15.80:0/1259907447 wait complete. 2024-04-24T23:27:20.206 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-04-24T23:27:20.206 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-04-24T23:27:20.228 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:19 smithi080 ceph-mon[29152]: pgmap v288: 33 pgs: 16 creating+peering, 1 active+clean, 16 unknown; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:27:20.228 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:19 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/2830769121' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-04-24T23:27:20.228 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:19 smithi080 ceph-mon[29152]: osdmap e83: 12 total, 12 up, 12 in 2024-04-24T23:27:20.228 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:19 smithi080 ceph-mon[29152]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-24T23:27:20.228 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:19 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/1259907447' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-24T23:27:20.228 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:19 smithi080 ceph-mon[29152]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-04-24T23:27:20.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:19 smithi148 ceph-mon[32528]: pgmap v288: 33 pgs: 16 creating+peering, 1 active+clean, 16 unknown; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:27:20.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:19 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/2830769121' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-04-24T23:27:20.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:19 smithi148 ceph-mon[32528]: osdmap e83: 12 total, 12 up, 12 in 2024-04-24T23:27:20.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:19 smithi148 ceph-mon[32528]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-24T23:27:20.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:19 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/1259907447' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-24T23:27:20.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:19 smithi148 ceph-mon[32528]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-04-24T23:27:20.316 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:20.313+0000 7f2a8659e640 1 -- 172.21.15.80:0/2396987106 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2a80103340 msgr2=0x7f2a80109bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:20.316 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:20.313+0000 7f2a8659e640 1 --2- 172.21.15.80:0/2396987106 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2a80103340 0x7f2a80109bd0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2a78009ca0 tx=0x7f2a7802a8c0 comp rx=0 tx=0).stop 2024-04-24T23:27:20.316 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:20.313+0000 7f2a8659e640 1 -- 172.21.15.80:0/2396987106 shutdown_connections 2024-04-24T23:27:20.316 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:20.313+0000 7f2a8659e640 1 --2- 172.21.15.80:0/2396987106 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2a80103340 0x7f2a80109bd0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:20.316 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:20.313+0000 7f2a8659e640 1 --2- 172.21.15.80:0/2396987106 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2a80102980 0x7f2a80102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:20.317 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:20.313+0000 7f2a8659e640 1 --2- 172.21.15.80:0/2396987106 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2a80101790 0x7f2a80101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:20.317 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:20.313+0000 7f2a8659e640 1 -- 172.21.15.80:0/2396987106 >> 172.21.15.80:0/2396987106 conn(0x7f2a800fcd20 msgr2=0x7f2a800ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:20.317 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:20.313+0000 7f2a8659e640 1 -- 172.21.15.80:0/2396987106 shutdown_connections 2024-04-24T23:27:20.317 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:20.313+0000 7f2a8659e640 1 -- 172.21.15.80:0/2396987106 wait complete. 2024-04-24T23:27:20.317 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:20.313+0000 7f2a8659e640 1 Processor -- start 2024-04-24T23:27:20.317 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:20.313+0000 7f2a8659e640 1 -- start start 2024-04-24T23:27:20.317 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:20.314+0000 7f2a8659e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2a80101790 0x7f2a80197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:20.317 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:20.314+0000 7f2a8659e640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2a80102980 0x7f2a80198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:20.317 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:20.314+0000 7f2a8659e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2a80103340 0x7f2a8019f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:20.317 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:20.314+0000 7f2a8659e640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f2a8010d470 con 0x7f2a80102980 2024-04-24T23:27:20.317 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:20.314+0000 7f2a8659e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2a8010d2f0 con 0x7f2a80101790 2024-04-24T23:27:20.317 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:20.314+0000 7f2a8659e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f2a8010d5f0 con 0x7f2a80103340 2024-04-24T23:27:20.317 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:20.314+0000 7f2a7f7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2a80102980 0x7f2a80198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:20.317 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:20.314+0000 7f2a7f7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2a80102980 0x7f2a80198020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:39216/0 (socket says 172.21.15.80:39216) 2024-04-24T23:27:20.318 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:20.314+0000 7f2a7f7fe640 1 -- 172.21.15.80:0/461304663 learned_addr learned my addr 172.21.15.80:0/461304663 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:27:20.318 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:20.314+0000 7f2a7ffff640 1 --2- 172.21.15.80:0/461304663 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2a80101790 0x7f2a80197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:20.318 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:20.314+0000 7f2a84b3b640 1 --2- 172.21.15.80:0/461304663 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2a80103340 0x7f2a8019f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:20.318 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:20.314+0000 7f2a7f7fe640 1 -- 172.21.15.80:0/461304663 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2a80103340 msgr2=0x7f2a8019f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:20.318 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:20.314+0000 7f2a7f7fe640 1 --2- 172.21.15.80:0/461304663 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2a80103340 0x7f2a8019f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:20.318 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:20.314+0000 7f2a7f7fe640 1 -- 172.21.15.80:0/461304663 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2a80101790 msgr2=0x7f2a80197ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:20.318 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:20.314+0000 7f2a7f7fe640 1 --2- 172.21.15.80:0/461304663 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2a80101790 0x7f2a80197ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:20.318 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:20.314+0000 7f2a7f7fe640 1 -- 172.21.15.80:0/461304663 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a8019f700 con 0x7f2a80102980 2024-04-24T23:27:20.318 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:20.314+0000 7f2a7f7fe640 1 --2- 172.21.15.80:0/461304663 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2a80102980 0x7f2a80198020 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f2a7400b4a0 tx=0x7f2a7400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:20.318 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:20.314+0000 7f2a7d7fa640 1 -- 172.21.15.80:0/461304663 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a74014070 con 0x7f2a80102980 2024-04-24T23:27:20.318 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:20.314+0000 7f2a8659e640 1 -- 172.21.15.80:0/461304663 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a8019f9f0 con 0x7f2a80102980 2024-04-24T23:27:20.318 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:20.314+0000 7f2a7d7fa640 1 -- 172.21.15.80:0/461304663 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a74010af0 con 0x7f2a80102980 2024-04-24T23:27:20.318 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:20.314+0000 7f2a7d7fa640 1 -- 172.21.15.80:0/461304663 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a74017d10 con 0x7f2a80102980 2024-04-24T23:27:20.318 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:20.315+0000 7f2a8659e640 1 -- 172.21.15.80:0/461304663 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a8019ff40 con 0x7f2a80102980 2024-04-24T23:27:20.319 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:20.315+0000 7f2a8659e640 1 -- 172.21.15.80:0/461304663 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a801089e0 con 0x7f2a80102980 2024-04-24T23:27:20.319 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:20.316+0000 7f2a7d7fa640 1 -- 172.21.15.80:0/461304663 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f2a74017070 con 0x7f2a80102980 2024-04-24T23:27:20.319 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:20.317+0000 7f2a7d7fa640 1 --2- 172.21.15.80:0/461304663 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f2a680a2b90 0x7f2a680a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:20.319 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:20.317+0000 7f2a7ffff640 1 --2- 172.21.15.80:0/461304663 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f2a680a2b90 0x7f2a680a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:20.320 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:20.317+0000 7f2a7d7fa640 1 -- 172.21.15.80:0/461304663 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7f2a740ce2c0 con 0x7f2a80102980 2024-04-24T23:27:20.320 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:20.317+0000 7f2a7ffff640 1 --2- 172.21.15.80:0/461304663 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f2a680a2b90 0x7f2a680a5050 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2a801027e0 tx=0x7f2a70005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:20.322 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:20.322+0000 7f2a7d7fa640 1 -- 172.21.15.80:0/461304663 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f2a7409b370 con 0x7f2a80102980 2024-04-24T23:27:20.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:19 smithi134 ceph-mon[32619]: pgmap v288: 33 pgs: 16 creating+peering, 1 active+clean, 16 unknown; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:27:20.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:19 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/2830769121' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-04-24T23:27:20.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:19 smithi134 ceph-mon[32619]: osdmap e83: 12 total, 12 up, 12 in 2024-04-24T23:27:20.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:19 smithi134 ceph-mon[32619]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-24T23:27:20.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:19 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/1259907447' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-24T23:27:20.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:19 smithi134 ceph-mon[32619]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-04-24T23:27:20.507 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:20.506+0000 7f2a8659e640 1 -- 172.21.15.80:0/461304663 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7f2a8002d340 con 0x7f2a80102980 2024-04-24T23:27:20.966 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:20.965+0000 7f2a7d7fa640 1 -- 172.21.15.80:0/461304663 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v85) ==== 157+0+0 (secure 0 0 0) 0x7f2a7409ac70 con 0x7f2a80102980 2024-04-24T23:27:20.969 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:20.968+0000 7f2a8659e640 1 -- 172.21.15.80:0/461304663 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f2a680a2b90 msgr2=0x7f2a680a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:20.969 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:20.968+0000 7f2a8659e640 1 --2- 172.21.15.80:0/461304663 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f2a680a2b90 0x7f2a680a5050 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2a801027e0 tx=0x7f2a70005f90 comp rx=0 tx=0).stop 2024-04-24T23:27:20.969 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:20.968+0000 7f2a8659e640 1 -- 172.21.15.80:0/461304663 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2a80102980 msgr2=0x7f2a80198020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:20.969 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:20.968+0000 7f2a8659e640 1 --2- 172.21.15.80:0/461304663 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2a80102980 0x7f2a80198020 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f2a7400b4a0 tx=0x7f2a7400b960 comp rx=0 tx=0).stop 2024-04-24T23:27:20.969 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:20.968+0000 7f2a8659e640 1 -- 172.21.15.80:0/461304663 shutdown_connections 2024-04-24T23:27:20.970 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:20.968+0000 7f2a8659e640 1 --2- 172.21.15.80:0/461304663 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f2a680a2b90 0x7f2a680a5050 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:20.970 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:20.968+0000 7f2a8659e640 1 --2- 172.21.15.80:0/461304663 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2a80103340 0x7f2a8019f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:20.970 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:20.968+0000 7f2a8659e640 1 --2- 172.21.15.80:0/461304663 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2a80102980 0x7f2a80198020 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:20.970 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:20.968+0000 7f2a8659e640 1 --2- 172.21.15.80:0/461304663 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2a80101790 0x7f2a80197ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:20.970 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:20.968+0000 7f2a8659e640 1 -- 172.21.15.80:0/461304663 >> 172.21.15.80:0/461304663 conn(0x7f2a800fcd20 msgr2=0x7f2a80105bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:20.970 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:20.968+0000 7f2a8659e640 1 -- 172.21.15.80:0/461304663 shutdown_connections 2024-04-24T23:27:20.970 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:20.968+0000 7f2a8659e640 1 -- 172.21.15.80:0/461304663 wait complete. 2024-04-24T23:27:20.980 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-04-24T23:27:21.129 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:21.126+0000 7f183b4ee640 1 -- 172.21.15.80:0/3263748211 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f18341006a0 msgr2=0x7f1834100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:21.129 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:21.126+0000 7f183b4ee640 1 --2- 172.21.15.80:0/3263748211 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f18341006a0 0x7f1834100ac0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f1824009900 tx=0x7f182402a410 comp rx=0 tx=0).stop 2024-04-24T23:27:21.130 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:21.126+0000 7f183b4ee640 1 -- 172.21.15.80:0/3263748211 shutdown_connections 2024-04-24T23:27:21.130 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:21.126+0000 7f183b4ee640 1 --2- 172.21.15.80:0/3263748211 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1834102250 0x7f1834108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:21.130 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:21.126+0000 7f183b4ee640 1 --2- 172.21.15.80:0/3263748211 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1834101890 0x7f1834101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:21.130 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:21.126+0000 7f183b4ee640 1 --2- 172.21.15.80:0/3263748211 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f18341006a0 0x7f1834100ac0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:21.130 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:21.126+0000 7f183b4ee640 1 -- 172.21.15.80:0/3263748211 >> 172.21.15.80:0/3263748211 conn(0x7f18340fbc30 msgr2=0x7f18340fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:21.130 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:21.126+0000 7f183b4ee640 1 -- 172.21.15.80:0/3263748211 shutdown_connections 2024-04-24T23:27:21.130 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:21.126+0000 7f183b4ee640 1 -- 172.21.15.80:0/3263748211 wait complete. 2024-04-24T23:27:21.130 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:21.127+0000 7f183b4ee640 1 Processor -- start 2024-04-24T23:27:21.130 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:21.127+0000 7f183b4ee640 1 -- start start 2024-04-24T23:27:21.130 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:21.127+0000 7f183b4ee640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f18341006a0 0x7f183406a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:21.130 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:21.127+0000 7f183b4ee640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1834101890 0x7f183419c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:21.130 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:21.127+0000 7f183b4ee640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1834102250 0x7f183419eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:21.131 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:21.127+0000 7f183b4ee640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f183410c390 con 0x7f1834102250 2024-04-24T23:27:21.131 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:21.127+0000 7f183b4ee640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f183410c210 con 0x7f18341006a0 2024-04-24T23:27:21.131 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:21.127+0000 7f183b4ee640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f183410c510 con 0x7f1834101890 2024-04-24T23:27:21.131 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:21.127+0000 7f1839a8b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1834102250 0x7f183419eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:21.131 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:21.127+0000 7f1839a8b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1834102250 0x7f183419eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:39236/0 (socket says 172.21.15.80:39236) 2024-04-24T23:27:21.131 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:21.127+0000 7f1839a8b640 1 -- 172.21.15.80:0/3154340866 learned_addr learned my addr 172.21.15.80:0/3154340866 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:27:21.131 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:21.127+0000 7f183928a640 1 --2- 172.21.15.80:0/3154340866 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f18341006a0 0x7f183406a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:21.131 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:21.127+0000 7f1839a8b640 1 -- 172.21.15.80:0/3154340866 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1834101890 msgr2=0x7f183419c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:21.131 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:21.127+0000 7f1838a89640 1 --2- 172.21.15.80:0/3154340866 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1834101890 0x7f183419c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:21.131 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:21.127+0000 7f1839a8b640 1 --2- 172.21.15.80:0/3154340866 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1834101890 0x7f183419c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:21.131 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:21.127+0000 7f1839a8b640 1 -- 172.21.15.80:0/3154340866 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f18341006a0 msgr2=0x7f183406a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:21.131 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:21.127+0000 7f1839a8b640 1 --2- 172.21.15.80:0/3154340866 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f18341006a0 0x7f183406a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:21.131 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:21.127+0000 7f1839a8b640 1 -- 172.21.15.80:0/3154340866 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f183419f7e0 con 0x7f1834102250 2024-04-24T23:27:21.131 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:21.127+0000 7f1839a8b640 1 --2- 172.21.15.80:0/3154340866 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1834102250 0x7f183419eed0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f182c00b970 tx=0x7f182c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:21.132 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:21.127+0000 7f182a7fc640 1 -- 172.21.15.80:0/3154340866 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f182c019070 con 0x7f1834102250 2024-04-24T23:27:21.132 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:21.127+0000 7f182a7fc640 1 -- 172.21.15.80:0/3154340866 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f182c00cb40 con 0x7f1834102250 2024-04-24T23:27:21.132 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:21.128+0000 7f183b4ee640 1 -- 172.21.15.80:0/3154340866 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f183419fa70 con 0x7f1834102250 2024-04-24T23:27:21.132 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:21.128+0000 7f182a7fc640 1 -- 172.21.15.80:0/3154340866 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f182c007650 con 0x7f1834102250 2024-04-24T23:27:21.132 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:21.128+0000 7f183b4ee640 1 -- 172.21.15.80:0/3154340866 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f183419fff0 con 0x7f1834102250 2024-04-24T23:27:21.132 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:21.129+0000 7f183b4ee640 1 -- 172.21.15.80:0/3154340866 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1834107a40 con 0x7f1834102250 2024-04-24T23:27:21.133 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:21.130+0000 7f182a7fc640 1 -- 172.21.15.80:0/3154340866 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f182c01c070 con 0x7f1834102250 2024-04-24T23:27:21.133 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:21.130+0000 7f182a7fc640 1 --2- 172.21.15.80:0/3154340866 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f18200a2b90 0x7f18200a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:21.133 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:21.130+0000 7f183928a640 1 --2- 172.21.15.80:0/3154340866 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f18200a2b90 0x7f18200a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:21.133 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:21.130+0000 7f182a7fc640 1 -- 172.21.15.80:0/3154340866 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8277+0+0 (secure 0 0 0) 0x7f182c0ce730 con 0x7f1834102250 2024-04-24T23:27:21.133 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:21.131+0000 7f183928a640 1 --2- 172.21.15.80:0/3154340866 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f18200a2b90 0x7f18200a5050 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1824006010 tx=0x7f1824000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:21.136 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:21.135+0000 7f182a7fc640 1 -- 172.21.15.80:0/3154340866 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f182c09b660 con 0x7f1834102250 2024-04-24T23:27:21.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:20 smithi148 ceph-mon[32528]: osdmap e84: 12 total, 12 up, 12 in 2024-04-24T23:27:21.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:20 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/1259907447' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-24T23:27:21.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:20 smithi148 ceph-mon[32528]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-04-24T23:27:21.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:20 smithi148 ceph-mon[32528]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-04-24T23:27:21.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:20 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/461304663' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-04-24T23:27:21.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:20 smithi080 ceph-mon[29152]: osdmap e84: 12 total, 12 up, 12 in 2024-04-24T23:27:21.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:20 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/1259907447' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-24T23:27:21.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:20 smithi080 ceph-mon[29152]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-04-24T23:27:21.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:20 smithi080 ceph-mon[29152]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-04-24T23:27:21.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:20 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/461304663' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-04-24T23:27:21.315 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:21.314+0000 7f183b4ee640 1 -- 172.21.15.80:0/3154340866 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f1834062340 con 0x7f1834102250 2024-04-24T23:27:21.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:20 smithi134 ceph-mon[32619]: osdmap e84: 12 total, 12 up, 12 in 2024-04-24T23:27:21.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:20 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/1259907447' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-24T23:27:21.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:20 smithi134 ceph-mon[32619]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-04-24T23:27:21.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:20 smithi134 ceph-mon[32619]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-04-24T23:27:21.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:20 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/461304663' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-04-24T23:27:22.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:21 smithi148 ceph-mon[32528]: pgmap v291: 97 pgs: 8 active+clean, 47 creating+peering, 42 unknown; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:27:22.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:21 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/461304663' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-04-24T23:27:22.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:21 smithi148 ceph-mon[32528]: osdmap e85: 12 total, 12 up, 12 in 2024-04-24T23:27:22.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:21 smithi148 ceph-mon[32528]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-24T23:27:22.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:21 smithi148 ceph-mon[32528]: Cluster is now healthy 2024-04-24T23:27:22.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:21 smithi148 ceph-mon[32528]: mds.? [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] up:active 2024-04-24T23:27:22.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:21 smithi148 ceph-mon[32528]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-04-24T23:27:22.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:21 smithi148 ceph-mon[32528]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-04-24T23:27:22.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:21 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/3154340866' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-24T23:27:22.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:21 smithi080 ceph-mon[29152]: pgmap v291: 97 pgs: 8 active+clean, 47 creating+peering, 42 unknown; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:27:22.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:21 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/461304663' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-04-24T23:27:22.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:21 smithi080 ceph-mon[29152]: osdmap e85: 12 total, 12 up, 12 in 2024-04-24T23:27:22.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:21 smithi080 ceph-mon[29152]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-24T23:27:22.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:21 smithi080 ceph-mon[29152]: Cluster is now healthy 2024-04-24T23:27:22.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:21 smithi080 ceph-mon[29152]: mds.? [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] up:active 2024-04-24T23:27:22.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:21 smithi080 ceph-mon[29152]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-04-24T23:27:22.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:21 smithi080 ceph-mon[29152]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-04-24T23:27:22.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:21 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/3154340866' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-24T23:27:22.348 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:21 smithi134 ceph-mon[32619]: pgmap v291: 97 pgs: 8 active+clean, 47 creating+peering, 42 unknown; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:27:22.349 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:21 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/461304663' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-04-24T23:27:22.349 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:21 smithi134 ceph-mon[32619]: osdmap e85: 12 total, 12 up, 12 in 2024-04-24T23:27:22.349 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:21 smithi134 ceph-mon[32619]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-24T23:27:22.349 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:21 smithi134 ceph-mon[32619]: Cluster is now healthy 2024-04-24T23:27:22.349 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:21 smithi134 ceph-mon[32619]: mds.? [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] up:active 2024-04-24T23:27:22.349 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:21 smithi134 ceph-mon[32619]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-04-24T23:27:22.349 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:21 smithi134 ceph-mon[32619]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-04-24T23:27:22.349 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:21 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/3154340866' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-24T23:27:23.202 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:23.201+0000 7f182a7fc640 1 -- 172.21.15.80:0/3154340866 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v87) ==== 228+0+0 (secure 0 0 0) 0x7f182c09af60 con 0x7f1834102250 2024-04-24T23:27:23.203 INFO:teuthology.orchestra.run.smithi080.stderr:pool 'cephfs_data_ec' created 2024-04-24T23:27:23.205 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:23.204+0000 7f183b4ee640 1 -- 172.21.15.80:0/3154340866 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f18200a2b90 msgr2=0x7f18200a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:23.205 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:23.204+0000 7f183b4ee640 1 --2- 172.21.15.80:0/3154340866 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f18200a2b90 0x7f18200a5050 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1824006010 tx=0x7f1824000bc0 comp rx=0 tx=0).stop 2024-04-24T23:27:23.206 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:23.204+0000 7f183b4ee640 1 -- 172.21.15.80:0/3154340866 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1834102250 msgr2=0x7f183419eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:23.206 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:23.204+0000 7f183b4ee640 1 --2- 172.21.15.80:0/3154340866 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1834102250 0x7f183419eed0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f182c00b970 tx=0x7f182c00be30 comp rx=0 tx=0).stop 2024-04-24T23:27:23.206 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:23.204+0000 7f183b4ee640 1 -- 172.21.15.80:0/3154340866 shutdown_connections 2024-04-24T23:27:23.206 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:23.204+0000 7f183b4ee640 1 --2- 172.21.15.80:0/3154340866 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f18200a2b90 0x7f18200a5050 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:23.206 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:23.204+0000 7f183b4ee640 1 --2- 172.21.15.80:0/3154340866 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1834102250 0x7f183419eed0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:23.206 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:23.204+0000 7f183b4ee640 1 --2- 172.21.15.80:0/3154340866 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1834101890 0x7f183419c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:23.206 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:23.204+0000 7f183b4ee640 1 --2- 172.21.15.80:0/3154340866 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f18341006a0 0x7f183406a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:23.206 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:23.204+0000 7f183b4ee640 1 -- 172.21.15.80:0/3154340866 >> 172.21.15.80:0/3154340866 conn(0x7f18340fbc30 msgr2=0x7f1834104c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:23.206 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:23.204+0000 7f183b4ee640 1 -- 172.21.15.80:0/3154340866 shutdown_connections 2024-04-24T23:27:23.206 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:23.204+0000 7f183b4ee640 1 -- 172.21.15.80:0/3154340866 wait complete. 2024-04-24T23:27:23.223 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-04-24T23:27:23.329 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:23.326+0000 7fd4d2189640 1 -- 172.21.15.80:0/725346437 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd4cc100680 msgr2=0x7fd4cc100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:23.329 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:23.326+0000 7fd4d2189640 1 --2- 172.21.15.80:0/725346437 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd4cc100680 0x7fd4cc100aa0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd4c0009900 tx=0x7fd4c002a410 comp rx=0 tx=0).stop 2024-04-24T23:27:23.330 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:23.326+0000 7fd4d2189640 1 -- 172.21.15.80:0/725346437 shutdown_connections 2024-04-24T23:27:23.330 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:23.326+0000 7fd4d2189640 1 --2- 172.21.15.80:0/725346437 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4cc102230 0x7fd4cc108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:23.330 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:23.326+0000 7fd4d2189640 1 --2- 172.21.15.80:0/725346437 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd4cc101870 0x7fd4cc101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:23.330 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:23.326+0000 7fd4d2189640 1 --2- 172.21.15.80:0/725346437 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd4cc100680 0x7fd4cc100aa0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:23.330 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:23.326+0000 7fd4d2189640 1 -- 172.21.15.80:0/725346437 >> 172.21.15.80:0/725346437 conn(0x7fd4cc0fbbf0 msgr2=0x7fd4cc0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:23.330 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:23.326+0000 7fd4d2189640 1 -- 172.21.15.80:0/725346437 shutdown_connections 2024-04-24T23:27:23.330 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:23.326+0000 7fd4d2189640 1 -- 172.21.15.80:0/725346437 wait complete. 2024-04-24T23:27:23.330 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:23.327+0000 7fd4d2189640 1 Processor -- start 2024-04-24T23:27:23.330 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:23.327+0000 7fd4d2189640 1 -- start start 2024-04-24T23:27:23.330 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:23.327+0000 7fd4d2189640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd4cc100680 0x7fd4cc069440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:23.330 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:23.327+0000 7fd4d2189640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd4cc101870 0x7fd4cc19ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:23.330 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:23.327+0000 7fd4d2189640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4cc102230 0x7fd4cc1a11d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:23.330 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:23.327+0000 7fd4d2189640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fd4cc10c300 con 0x7fd4cc101870 2024-04-24T23:27:23.331 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:23.327+0000 7fd4d2189640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd4cc10c180 con 0x7fd4cc102230 2024-04-24T23:27:23.331 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:23.327+0000 7fd4d2189640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd4cc10c480 con 0x7fd4cc100680 2024-04-24T23:27:23.331 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:23.327+0000 7fd4caffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd4cc101870 0x7fd4cc19ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:23.331 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:23.327+0000 7fd4caffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd4cc101870 0x7fd4cc19ec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:35570/0 (socket says 172.21.15.80:35570) 2024-04-24T23:27:23.331 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:23.327+0000 7fd4caffd640 1 -- 172.21.15.80:0/3171401818 learned_addr learned my addr 172.21.15.80:0/3171401818 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:27:23.333 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:23.327+0000 7fd4cbfff640 1 --2- 172.21.15.80:0/3171401818 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4cc102230 0x7fd4cc1a11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:23.333 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:23.327+0000 7fd4cb7fe640 1 --2- 172.21.15.80:0/3171401818 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd4cc100680 0x7fd4cc069440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:23.333 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:23.327+0000 7fd4caffd640 1 -- 172.21.15.80:0/3171401818 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd4cc100680 msgr2=0x7fd4cc069440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:23.333 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:23.327+0000 7fd4caffd640 1 --2- 172.21.15.80:0/3171401818 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd4cc100680 0x7fd4cc069440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:23.333 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:23.327+0000 7fd4caffd640 1 -- 172.21.15.80:0/3171401818 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4cc102230 msgr2=0x7fd4cc1a11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:23.333 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:23.327+0000 7fd4caffd640 1 --2- 172.21.15.80:0/3171401818 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4cc102230 0x7fd4cc1a11d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:23.333 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:23.327+0000 7fd4caffd640 1 -- 172.21.15.80:0/3171401818 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4cc1a19c0 con 0x7fd4cc101870 2024-04-24T23:27:23.333 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:23.328+0000 7fd4caffd640 1 --2- 172.21.15.80:0/3171401818 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd4cc101870 0x7fd4cc19ec80 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc00c970 tx=0x7fd4bc00cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:23.334 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:23.328+0000 7fd4c8ff9640 1 -- 172.21.15.80:0/3171401818 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4bc007bb0 con 0x7fd4cc101870 2024-04-24T23:27:23.334 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:23.328+0000 7fd4c8ff9640 1 -- 172.21.15.80:0/3171401818 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4bc007d50 con 0x7fd4cc101870 2024-04-24T23:27:23.334 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:23.328+0000 7fd4d2189640 1 -- 172.21.15.80:0/3171401818 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4cc1a1c50 con 0x7fd4cc101870 2024-04-24T23:27:23.334 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:23.328+0000 7fd4c8ff9640 1 -- 172.21.15.80:0/3171401818 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4bc00f450 con 0x7fd4cc101870 2024-04-24T23:27:23.334 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:23.328+0000 7fd4d2189640 1 -- 172.21.15.80:0/3171401818 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4cc1a2150 con 0x7fd4cc101870 2024-04-24T23:27:23.335 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:23.329+0000 7fd4d2189640 1 -- 172.21.15.80:0/3171401818 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4cc107a10 con 0x7fd4cc101870 2024-04-24T23:27:23.335 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:23.330+0000 7fd4c8ff9640 1 -- 172.21.15.80:0/3171401818 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fd4bc004f00 con 0x7fd4cc101870 2024-04-24T23:27:23.335 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:23.331+0000 7fd4c8ff9640 1 --2- 172.21.15.80:0/3171401818 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fd4b40a2bc0 0x7fd4b40a5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:23.335 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:23.331+0000 7fd4cb7fe640 1 --2- 172.21.15.80:0/3171401818 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fd4b40a2bc0 0x7fd4b40a5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:23.335 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:23.331+0000 7fd4c8ff9640 1 -- 172.21.15.80:0/3171401818 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8770+0+0 (secure 0 0 0) 0x7fd4bc0cefb0 con 0x7fd4cc101870 2024-04-24T23:27:23.335 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:23.331+0000 7fd4cb7fe640 1 --2- 172.21.15.80:0/3171401818 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fd4b40a2bc0 0x7fd4b40a5080 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd4c0006010 tx=0x7fd4c0000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:23.336 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:23.335+0000 7fd4c8ff9640 1 -- 172.21.15.80:0/3171401818 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fd4bc014030 con 0x7fd4cc101870 2024-04-24T23:27:23.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:23 smithi134 ceph-mon[32619]: osdmap e86: 12 total, 12 up, 12 in 2024-04-24T23:27:23.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:23 smithi134 ceph-mon[32619]: pgmap v294: 97 pgs: 25 active+clean, 47 creating+peering, 25 unknown; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 4 op/s 2024-04-24T23:27:23.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:23 smithi148 ceph-mon[32528]: osdmap e86: 12 total, 12 up, 12 in 2024-04-24T23:27:23.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:23 smithi148 ceph-mon[32528]: pgmap v294: 97 pgs: 25 active+clean, 47 creating+peering, 25 unknown; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 4 op/s 2024-04-24T23:27:23.513 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:23.512+0000 7fd4d2189640 1 -- 172.21.15.80:0/3171401818 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7fd4cc0623b0 con 0x7fd4cc101870 2024-04-24T23:27:23.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:23 smithi080 ceph-mon[29152]: osdmap e86: 12 total, 12 up, 12 in 2024-04-24T23:27:23.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:23 smithi080 ceph-mon[29152]: pgmap v294: 97 pgs: 25 active+clean, 47 creating+peering, 25 unknown; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 4 op/s 2024-04-24T23:27:24.210 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:24.209+0000 7fd4c8ff9640 1 -- 172.21.15.80:0/3171401818 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v88) ==== 169+0+0 (secure 0 0 0) 0x7fd4bc09bcf0 con 0x7fd4cc101870 2024-04-24T23:27:24.211 INFO:teuthology.orchestra.run.smithi080.stderr:set pool 4 allow_ec_overwrites to true 2024-04-24T23:27:24.214 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:24.213+0000 7fd4d2189640 1 -- 172.21.15.80:0/3171401818 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fd4b40a2bc0 msgr2=0x7fd4b40a5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:24.214 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:24.213+0000 7fd4d2189640 1 --2- 172.21.15.80:0/3171401818 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fd4b40a2bc0 0x7fd4b40a5080 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd4c0006010 tx=0x7fd4c0000bc0 comp rx=0 tx=0).stop 2024-04-24T23:27:24.214 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:24.213+0000 7fd4d2189640 1 -- 172.21.15.80:0/3171401818 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd4cc101870 msgr2=0x7fd4cc19ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:24.214 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:24.213+0000 7fd4d2189640 1 --2- 172.21.15.80:0/3171401818 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd4cc101870 0x7fd4cc19ec80 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc00c970 tx=0x7fd4bc00cd70 comp rx=0 tx=0).stop 2024-04-24T23:27:24.214 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:24.213+0000 7fd4d2189640 1 -- 172.21.15.80:0/3171401818 shutdown_connections 2024-04-24T23:27:24.215 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:24.213+0000 7fd4d2189640 1 --2- 172.21.15.80:0/3171401818 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fd4b40a2bc0 0x7fd4b40a5080 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:24.215 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:24.213+0000 7fd4d2189640 1 --2- 172.21.15.80:0/3171401818 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4cc102230 0x7fd4cc1a11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:24.215 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:24.213+0000 7fd4d2189640 1 --2- 172.21.15.80:0/3171401818 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd4cc101870 0x7fd4cc19ec80 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:24.215 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:24.213+0000 7fd4d2189640 1 --2- 172.21.15.80:0/3171401818 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd4cc100680 0x7fd4cc069440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:24.215 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:24.213+0000 7fd4d2189640 1 -- 172.21.15.80:0/3171401818 >> 172.21.15.80:0/3171401818 conn(0x7fd4cc0fbbf0 msgr2=0x7fd4cc104be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:24.215 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:24.213+0000 7fd4d2189640 1 -- 172.21.15.80:0/3171401818 shutdown_connections 2024-04-24T23:27:24.215 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:24.213+0000 7fd4d2189640 1 -- 172.21.15.80:0/3171401818 wait complete. 2024-04-24T23:27:24.237 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-04-24T23:27:24.342 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:24.338+0000 7ff48cba5640 1 -- 172.21.15.80:0/2484122233 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff488100680 msgr2=0x7ff488100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:24.342 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:24.338+0000 7ff48cba5640 1 --2- 172.21.15.80:0/2484122233 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff488100680 0x7ff488100aa0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ff478009900 tx=0x7ff47802a410 comp rx=0 tx=0).stop 2024-04-24T23:27:24.342 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:24.338+0000 7ff48cba5640 1 -- 172.21.15.80:0/2484122233 shutdown_connections 2024-04-24T23:27:24.342 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:24.338+0000 7ff48cba5640 1 --2- 172.21.15.80:0/2484122233 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff488102230 0x7ff488108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:24.343 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:24.338+0000 7ff48cba5640 1 --2- 172.21.15.80:0/2484122233 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff488101870 0x7ff488101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:24.343 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:24.338+0000 7ff48cba5640 1 --2- 172.21.15.80:0/2484122233 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff488100680 0x7ff488100aa0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:24.343 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:24.338+0000 7ff48cba5640 1 -- 172.21.15.80:0/2484122233 >> 172.21.15.80:0/2484122233 conn(0x7ff4880fbbf0 msgr2=0x7ff4880fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:24.343 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:24.338+0000 7ff48cba5640 1 -- 172.21.15.80:0/2484122233 shutdown_connections 2024-04-24T23:27:24.343 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:24.339+0000 7ff48cba5640 1 -- 172.21.15.80:0/2484122233 wait complete. 2024-04-24T23:27:24.343 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:24.339+0000 7ff48cba5640 1 Processor -- start 2024-04-24T23:27:24.343 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:24.339+0000 7ff48cba5640 1 -- start start 2024-04-24T23:27:24.343 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:24.339+0000 7ff48cba5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff488100680 0x7ff488197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:24.343 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:24.339+0000 7ff48cba5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff488101870 0x7ff488198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:24.343 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:24.339+0000 7ff48cba5640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff488102230 0x7ff48819f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:24.343 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:24.339+0000 7ff48cba5640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7ff48810c370 con 0x7ff488102230 2024-04-24T23:27:24.343 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:24.339+0000 7ff48cba5640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff48810c1f0 con 0x7ff488101870 2024-04-24T23:27:24.344 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:24.339+0000 7ff48cba5640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff48810c4f0 con 0x7ff488100680 2024-04-24T23:27:24.344 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:24.339+0000 7ff486d9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff488102230 0x7ff48819f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:24.344 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:24.339+0000 7ff486d9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff488102230 0x7ff48819f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:35586/0 (socket says 172.21.15.80:35586) 2024-04-24T23:27:24.344 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:24.339+0000 7ff486d9d640 1 -- 172.21.15.80:0/2120142007 learned_addr learned my addr 172.21.15.80:0/2120142007 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:27:24.344 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:24.340+0000 7ff47ffff640 1 --2- 172.21.15.80:0/2120142007 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff488101870 0x7ff488198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:24.344 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:24.340+0000 7ff48659c640 1 --2- 172.21.15.80:0/2120142007 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff488100680 0x7ff488197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:24.344 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:24.340+0000 7ff486d9d640 1 -- 172.21.15.80:0/2120142007 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff488100680 msgr2=0x7ff488197b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:24.344 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:24.340+0000 7ff486d9d640 1 --2- 172.21.15.80:0/2120142007 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff488100680 0x7ff488197b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:24.344 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:24.340+0000 7ff486d9d640 1 -- 172.21.15.80:0/2120142007 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff488101870 msgr2=0x7ff488198080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:24.344 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:24.340+0000 7ff486d9d640 1 --2- 172.21.15.80:0/2120142007 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff488101870 0x7ff488198080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:24.344 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:24.340+0000 7ff486d9d640 1 -- 172.21.15.80:0/2120142007 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff48819f760 con 0x7ff488102230 2024-04-24T23:27:24.344 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:24.340+0000 7ff48659c640 1 --2- 172.21.15.80:0/2120142007 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff488100680 0x7ff488197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:27:24.344 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:24.340+0000 7ff486d9d640 1 --2- 172.21.15.80:0/2120142007 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff488102230 0x7ff48819f0f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff48000eaa0 tx=0x7ff48000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:24.344 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:24.340+0000 7ff47f7fe640 1 -- 172.21.15.80:0/2120142007 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff480004d20 con 0x7ff488102230 2024-04-24T23:27:24.345 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:24.340+0000 7ff47f7fe640 1 -- 172.21.15.80:0/2120142007 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff480005020 con 0x7ff488102230 2024-04-24T23:27:24.345 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:24.340+0000 7ff48cba5640 1 -- 172.21.15.80:0/2120142007 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff48819fa50 con 0x7ff488102230 2024-04-24T23:27:24.345 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:24.340+0000 7ff47f7fe640 1 -- 172.21.15.80:0/2120142007 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4800116d0 con 0x7ff488102230 2024-04-24T23:27:24.345 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:24.340+0000 7ff48cba5640 1 -- 172.21.15.80:0/2120142007 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff48819ffa0 con 0x7ff488102230 2024-04-24T23:27:24.345 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:24.341+0000 7ff48cba5640 1 -- 172.21.15.80:0/2120142007 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4881a0290 con 0x7ff488102230 2024-04-24T23:27:24.346 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:24.342+0000 7ff47f7fe640 1 -- 172.21.15.80:0/2120142007 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7ff4800051d0 con 0x7ff488102230 2024-04-24T23:27:24.346 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:24.343+0000 7ff47f7fe640 1 --2- 172.21.15.80:0/2120142007 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7ff4740a2b90 0x7ff4740a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:24.346 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:24.343+0000 7ff48659c640 1 --2- 172.21.15.80:0/2120142007 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7ff4740a2b90 0x7ff4740a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:24.346 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:24.343+0000 7ff47f7fe640 1 -- 172.21.15.80:0/2120142007 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8770+0+0 (secure 0 0 0) 0x7ff4800cf4a0 con 0x7ff488102230 2024-04-24T23:27:24.346 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:24.344+0000 7ff48659c640 1 --2- 172.21.15.80:0/2120142007 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7ff4740a2b90 0x7ff4740a5050 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff478006010 tx=0x7ff478032dd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:24.349 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:24.348+0000 7ff47f7fe640 1 -- 172.21.15.80:0/2120142007 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff480016030 con 0x7ff488102230 2024-04-24T23:27:24.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:24 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/3154340866' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-24T23:27:24.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:24 smithi148 ceph-mon[32528]: osdmap e87: 12 total, 12 up, 12 in 2024-04-24T23:27:24.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:24 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/3171401818' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-04-24T23:27:24.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:24 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/3154340866' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-24T23:27:24.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:24 smithi080 ceph-mon[29152]: osdmap e87: 12 total, 12 up, 12 in 2024-04-24T23:27:24.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:24 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/3171401818' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-04-24T23:27:24.566 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:24.563+0000 7ff48cba5640 1 -- 172.21.15.80:0/2120142007 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7ff4881a0460 con 0x7ff488102230 2024-04-24T23:27:24.697 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:24 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/3154340866' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-24T23:27:24.697 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:24 smithi134 ceph-mon[32619]: osdmap e87: 12 total, 12 up, 12 in 2024-04-24T23:27:24.697 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:24 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/3171401818' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-04-24T23:27:25.221 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.220+0000 7ff47f7fe640 1 -- 172.21.15.80:0/2120142007 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) ==== 137+0+0 (secure 0 0 0) 0x7ff48009c1e0 con 0x7ff488102230 2024-04-24T23:27:25.221 INFO:teuthology.orchestra.run.smithi080.stderr:added data pool 4 to fsmap 2024-04-24T23:27:25.224 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.222+0000 7ff48cba5640 1 -- 172.21.15.80:0/2120142007 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7ff4740a2b90 msgr2=0x7ff4740a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:25.224 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.222+0000 7ff48cba5640 1 --2- 172.21.15.80:0/2120142007 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7ff4740a2b90 0x7ff4740a5050 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff478006010 tx=0x7ff478032dd0 comp rx=0 tx=0).stop 2024-04-24T23:27:25.224 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.222+0000 7ff48cba5640 1 -- 172.21.15.80:0/2120142007 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff488102230 msgr2=0x7ff48819f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:25.224 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.222+0000 7ff48cba5640 1 --2- 172.21.15.80:0/2120142007 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff488102230 0x7ff48819f0f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff48000eaa0 tx=0x7ff48000eea0 comp rx=0 tx=0).stop 2024-04-24T23:27:25.224 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.222+0000 7ff48cba5640 1 -- 172.21.15.80:0/2120142007 shutdown_connections 2024-04-24T23:27:25.224 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.222+0000 7ff48cba5640 1 --2- 172.21.15.80:0/2120142007 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7ff4740a2b90 0x7ff4740a5050 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:25.224 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.222+0000 7ff48cba5640 1 --2- 172.21.15.80:0/2120142007 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff488102230 0x7ff48819f0f0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:25.224 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.222+0000 7ff48cba5640 1 --2- 172.21.15.80:0/2120142007 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff488101870 0x7ff488198080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:25.224 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.223+0000 7ff48cba5640 1 --2- 172.21.15.80:0/2120142007 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff488100680 0x7ff488197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:25.224 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.223+0000 7ff48cba5640 1 -- 172.21.15.80:0/2120142007 >> 172.21.15.80:0/2120142007 conn(0x7ff4880fbbf0 msgr2=0x7ff4880fc260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:25.224 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.223+0000 7ff48cba5640 1 -- 172.21.15.80:0/2120142007 shutdown_connections 2024-04-24T23:27:25.224 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.223+0000 7ff48cba5640 1 -- 172.21.15.80:0/2120142007 wait complete. 2024-04-24T23:27:25.225 INFO:journalctl@ceph.mds.i.smithi148.stdout:Apr 24 23:27:24 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-i[63704]: 2024-04-24T23:27:24.904+0000 7f8b8132e700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-24T23:27:25.236 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:27:25.342 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.335+0000 7f8429853640 1 -- 172.21.15.80:0/2208092435 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8424107500 msgr2=0x7f84241098f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:25.342 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.335+0000 7f8429853640 1 --2- 172.21.15.80:0/2208092435 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8424107500 0x7f84241098f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f841c009ca0 tx=0x7f841c02a8c0 comp rx=0 tx=0).stop 2024-04-24T23:27:25.342 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.335+0000 7f8429853640 1 -- 172.21.15.80:0/2208092435 shutdown_connections 2024-04-24T23:27:25.342 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.335+0000 7f8429853640 1 --2- 172.21.15.80:0/2208092435 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8424107500 0x7f84241098f0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:25.342 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.335+0000 7f8429853640 1 --2- 172.21.15.80:0/2208092435 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8424104bd0 0x7f8424106fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:25.343 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.335+0000 7f8429853640 1 --2- 172.21.15.80:0/2208092435 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84241022a0 0x7f8424104690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:25.343 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.335+0000 7f8429853640 1 -- 172.21.15.80:0/2208092435 >> 172.21.15.80:0/2208092435 conn(0x7f84240fbbf0 msgr2=0x7f84240fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:25.343 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.335+0000 7f8429853640 1 -- 172.21.15.80:0/2208092435 shutdown_connections 2024-04-24T23:27:25.343 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.335+0000 7f8429853640 1 -- 172.21.15.80:0/2208092435 wait complete. 2024-04-24T23:27:25.343 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.336+0000 7f8429853640 1 Processor -- start 2024-04-24T23:27:25.343 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.336+0000 7f8429853640 1 -- start start 2024-04-24T23:27:25.344 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.336+0000 7f8429853640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f84241022a0 0x7f8424197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:25.344 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.336+0000 7f8429853640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8424104bd0 0x7f8424198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:25.344 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.336+0000 7f8429853640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8424107500 0x7f842419f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:25.344 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.336+0000 7f8429853640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f842410d4a0 con 0x7f84241022a0 2024-04-24T23:27:25.344 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.336+0000 7f8429853640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f842410d320 con 0x7f8424104bd0 2024-04-24T23:27:25.345 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.336+0000 7f8429853640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f842410d620 con 0x7f8424107500 2024-04-24T23:27:25.345 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.336+0000 7f8422ffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f84241022a0 0x7f8424197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:25.345 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.336+0000 7f8422ffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f84241022a0 0x7f8424197ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:35606/0 (socket says 172.21.15.80:35606) 2024-04-24T23:27:25.345 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.336+0000 7f8422ffd640 1 -- 172.21.15.80:0/967464213 learned_addr learned my addr 172.21.15.80:0/967464213 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:27:25.345 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.336+0000 7f84237fe640 1 --2- 172.21.15.80:0/967464213 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8424107500 0x7f842419f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:25.345 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.336+0000 7f8422ffd640 1 -- 172.21.15.80:0/967464213 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8424107500 msgr2=0x7f842419f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:25.346 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.336+0000 7f8422ffd640 1 --2- 172.21.15.80:0/967464213 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8424107500 0x7f842419f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:25.346 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.336+0000 7f8422ffd640 1 -- 172.21.15.80:0/967464213 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8424104bd0 msgr2=0x7f8424198000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:25.346 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.336+0000 7f8422ffd640 1 --2- 172.21.15.80:0/967464213 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8424104bd0 0x7f8424198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:25.346 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.336+0000 7f8422ffd640 1 -- 172.21.15.80:0/967464213 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f842419f6e0 con 0x7f84241022a0 2024-04-24T23:27:25.346 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.336+0000 7f8422ffd640 1 --2- 172.21.15.80:0/967464213 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f84241022a0 0x7f8424197ac0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f841400b4a0 tx=0x7f841400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:25.346 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.337+0000 7f8428851640 1 -- 172.21.15.80:0/967464213 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8414014070 con 0x7f84241022a0 2024-04-24T23:27:25.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.337+0000 7f8428851640 1 -- 172.21.15.80:0/967464213 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8414010af0 con 0x7f84241022a0 2024-04-24T23:27:25.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.337+0000 7f8429853640 1 -- 172.21.15.80:0/967464213 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f842419f9d0 con 0x7f84241022a0 2024-04-24T23:27:25.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.337+0000 7f8428851640 1 -- 172.21.15.80:0/967464213 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8414017d10 con 0x7f84241022a0 2024-04-24T23:27:25.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.337+0000 7f8429853640 1 -- 172.21.15.80:0/967464213 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f842419ffa0 con 0x7f84241022a0 2024-04-24T23:27:25.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.338+0000 7f8429853640 1 -- 172.21.15.80:0/967464213 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84240fd6e0 con 0x7f84241022a0 2024-04-24T23:27:25.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.339+0000 7f8428851640 1 -- 172.21.15.80:0/967464213 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f8414017070 con 0x7f84241022a0 2024-04-24T23:27:25.348 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.339+0000 7f8428851640 1 --2- 172.21.15.80:0/967464213 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f84100a2b90 0x7f84100a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:25.348 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.339+0000 7f841bfff640 1 --2- 172.21.15.80:0/967464213 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f84100a2b90 0x7f84100a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:25.348 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.340+0000 7f8428851640 1 -- 172.21.15.80:0/967464213 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f84140ce700 con 0x7f84241022a0 2024-04-24T23:27:25.348 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.340+0000 7f841bfff640 1 --2- 172.21.15.80:0/967464213 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f84100a2b90 0x7f84100a5050 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f840c005fd0 tx=0x7f840c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:25.349 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.344+0000 7f8428851640 1 -- 172.21.15.80:0/967464213 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f841409b420 con 0x7f84241022a0 2024-04-24T23:27:25.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:25 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/3171401818' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-04-24T23:27:25.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:25 smithi148 ceph-mon[32528]: osdmap e88: 12 total, 12 up, 12 in 2024-04-24T23:27:25.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:25 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/2120142007' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-04-24T23:27:25.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:25 smithi148 ceph-mon[32528]: pgmap v297: 129 pgs: 101 active+clean, 2 creating+activating, 18 creating+peering, 8 unknown; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s wr, 11 op/s 2024-04-24T23:27:25.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:25 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/3171401818' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-04-24T23:27:25.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:25 smithi080 ceph-mon[29152]: osdmap e88: 12 total, 12 up, 12 in 2024-04-24T23:27:25.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:25 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/2120142007' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-04-24T23:27:25.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:25 smithi080 ceph-mon[29152]: pgmap v297: 129 pgs: 101 active+clean, 2 creating+activating, 18 creating+peering, 8 unknown; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s wr, 11 op/s 2024-04-24T23:27:25.559 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.558+0000 7f8429853640 1 -- 172.21.15.80:0/967464213 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f842402d170 con 0x7f84241022a0 2024-04-24T23:27:25.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.559+0000 7f8428851640 1 -- 172.21.15.80:0/967464213 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11800 (secure 0 0 0) 0x7f841409ad20 con 0x7f84241022a0 2024-04-24T23:27:25.560 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-24T23:27:25.560 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/891591085","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":891591085},{"type":"v1","addr":"172.21.15.80:6835","nonce":891591085}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6837/259529026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6836","nonce":259529026},{"type":"v1","addr":"172.21.15.80:6837","nonce":259529026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14646,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6839/4194284854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6838","nonce":4194284854},{"type":"v1","addr":"172.21.15.80:6839","nonce":4194284854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14658,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6841/3145730958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6840","nonce":3145730958},{"type":"v1","addr":"172.21.15.80:6841","nonce":3145730958}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24395,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/3736784385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3736784385},{"type":"v1","addr":"172.21.15.134:6833","nonce":3736784385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/124777462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":124777462},{"type":"v1","addr":"172.21.15.148:6833","nonce":124777462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/1399801120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1399801120},{"type":"v1","addr":"172.21.15.134:6835","nonce":1399801120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24458,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/430512704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":430512704},{"type":"v1","addr":"172.21.15.134:6837","nonce":430512704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24460,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/3002319310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3002319310},{"type":"v1","addr":"172.21.15.148:6835","nonce":3002319310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24472,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/856528403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":856528403},{"type":"v1","addr":"172.21.15.148:6837","nonce":856528403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24488,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6839/1743815021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":1743815021},{"type":"v1","addr":"172.21.15.134:6839","nonce":1743815021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:27:19.300566+0000","modified":"2024-04-24T23:27:24.564912+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.148:6839/2952757594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":2952757594},{"type":"v1","addr":"172.21.15.148:6839","nonce":2952757594}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24496,"qdb_cluster":[24496]},"id":1}]} 2024-04-24T23:27:25.561 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 17 2024-04-24T23:27:25.562 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.561+0000 7f8429853640 1 -- 172.21.15.80:0/967464213 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f84100a2b90 msgr2=0x7f84100a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:25.562 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.561+0000 7f8429853640 1 --2- 172.21.15.80:0/967464213 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f84100a2b90 0x7f84100a5050 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f840c005fd0 tx=0x7f840c005850 comp rx=0 tx=0).stop 2024-04-24T23:27:25.562 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.561+0000 7f8429853640 1 -- 172.21.15.80:0/967464213 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f84241022a0 msgr2=0x7f8424197ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:25.562 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.561+0000 7f8429853640 1 --2- 172.21.15.80:0/967464213 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f84241022a0 0x7f8424197ac0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f841400b4a0 tx=0x7f841400b960 comp rx=0 tx=0).stop 2024-04-24T23:27:25.562 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.561+0000 7f8429853640 1 -- 172.21.15.80:0/967464213 shutdown_connections 2024-04-24T23:27:25.563 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.561+0000 7f8429853640 1 --2- 172.21.15.80:0/967464213 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f84100a2b90 0x7f84100a5050 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:25.563 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.561+0000 7f8429853640 1 --2- 172.21.15.80:0/967464213 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8424107500 0x7f842419f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:25.563 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.561+0000 7f8429853640 1 --2- 172.21.15.80:0/967464213 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8424104bd0 0x7f8424198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:25.563 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.561+0000 7f8429853640 1 --2- 172.21.15.80:0/967464213 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f84241022a0 0x7f8424197ac0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:25.563 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.561+0000 7f8429853640 1 -- 172.21.15.80:0/967464213 >> 172.21.15.80:0/967464213 conn(0x7f84240fbbf0 msgr2=0x7f8424102be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:25.563 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.561+0000 7f8429853640 1 -- 172.21.15.80:0/967464213 shutdown_connections 2024-04-24T23:27:25.563 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.561+0000 7f8429853640 1 -- 172.21.15.80:0/967464213 wait complete. 2024-04-24T23:27:25.573 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:27:25.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.670+0000 7fae8cc3f640 1 -- 172.21.15.80:0/3882112140 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fae88101040 msgr2=0x7fae881014c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:25.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.670+0000 7fae8cc3f640 1 --2- 172.21.15.80:0/3882112140 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fae88101040 0x7fae881014c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fae7c009900 tx=0x7fae7c02a410 comp rx=0 tx=0).stop 2024-04-24T23:27:25.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.670+0000 7fae8cc3f640 1 -- 172.21.15.80:0/3882112140 shutdown_connections 2024-04-24T23:27:25.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.670+0000 7fae8cc3f640 1 --2- 172.21.15.80:0/3882112140 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae88101a00 0x7fae88108320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:25.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.670+0000 7fae8cc3f640 1 --2- 172.21.15.80:0/3882112140 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fae88101040 0x7fae881014c0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:25.674 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.670+0000 7fae8cc3f640 1 --2- 172.21.15.80:0/3882112140 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fae880ffe60 0x7fae88100280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:25.674 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.670+0000 7fae8cc3f640 1 -- 172.21.15.80:0/3882112140 >> 172.21.15.80:0/3882112140 conn(0x7fae880fb420 msgr2=0x7fae880fd860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:25.674 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.670+0000 7fae8cc3f640 1 -- 172.21.15.80:0/3882112140 shutdown_connections 2024-04-24T23:27:25.674 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.670+0000 7fae8cc3f640 1 -- 172.21.15.80:0/3882112140 wait complete. 2024-04-24T23:27:25.674 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.671+0000 7fae8cc3f640 1 Processor -- start 2024-04-24T23:27:25.674 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.671+0000 7fae8cc3f640 1 -- start start 2024-04-24T23:27:25.674 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.671+0000 7fae8cc3f640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fae880ffe60 0x7fae88197c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:25.674 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.671+0000 7fae8cc3f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fae88101040 0x7fae88198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:25.674 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.671+0000 7fae8cc3f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae88101a00 0x7fae8819f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:25.674 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.671+0000 7fae8cc3f640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fae8810bc10 con 0x7fae880ffe60 2024-04-24T23:27:25.674 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.671+0000 7fae8cc3f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fae8810ba90 con 0x7fae88101040 2024-04-24T23:27:25.674 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.671+0000 7fae8cc3f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fae8810bd90 con 0x7fae88101a00 2024-04-24T23:27:25.674 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.671+0000 7fae8659c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fae880ffe60 0x7fae88197c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:25.674 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.671+0000 7fae8659c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fae880ffe60 0x7fae88197c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:35624/0 (socket says 172.21.15.80:35624) 2024-04-24T23:27:25.675 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.671+0000 7fae8659c640 1 -- 172.21.15.80:0/2164384578 learned_addr learned my addr 172.21.15.80:0/2164384578 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:27:25.675 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.671+0000 7fae86d9d640 1 --2- 172.21.15.80:0/2164384578 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae88101a00 0x7fae8819f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:25.675 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.671+0000 7fae8659c640 1 -- 172.21.15.80:0/2164384578 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae88101a00 msgr2=0x7fae8819f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:25.675 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.671+0000 7fae8659c640 1 --2- 172.21.15.80:0/2164384578 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae88101a00 0x7fae8819f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:25.675 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.671+0000 7fae8659c640 1 -- 172.21.15.80:0/2164384578 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fae88101040 msgr2=0x7fae88198180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:25.675 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.671+0000 7fae8659c640 1 --2- 172.21.15.80:0/2164384578 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fae88101040 0x7fae88198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:25.675 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.671+0000 7fae8659c640 1 -- 172.21.15.80:0/2164384578 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae8819f810 con 0x7fae880ffe60 2024-04-24T23:27:25.675 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.672+0000 7fae8659c640 1 --2- 172.21.15.80:0/2164384578 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fae880ffe60 0x7fae88197c40 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fae7800b450 tx=0x7fae7800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:25.675 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.672+0000 7fae777fe640 1 -- 172.21.15.80:0/2164384578 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae78015020 con 0x7fae880ffe60 2024-04-24T23:27:25.675 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.672+0000 7fae777fe640 1 -- 172.21.15.80:0/2164384578 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae78011d70 con 0x7fae880ffe60 2024-04-24T23:27:25.675 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.672+0000 7fae8cc3f640 1 -- 172.21.15.80:0/2164384578 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae8819fb00 con 0x7fae880ffe60 2024-04-24T23:27:25.675 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.672+0000 7fae777fe640 1 -- 172.21.15.80:0/2164384578 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae7800f340 con 0x7fae880ffe60 2024-04-24T23:27:25.675 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.672+0000 7fae8cc3f640 1 -- 172.21.15.80:0/2164384578 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae881a0050 con 0x7fae880ffe60 2024-04-24T23:27:25.676 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.673+0000 7fae8cc3f640 1 -- 172.21.15.80:0/2164384578 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae88107c20 con 0x7fae880ffe60 2024-04-24T23:27:25.676 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.674+0000 7fae777fe640 1 -- 172.21.15.80:0/2164384578 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fae78018020 con 0x7fae880ffe60 2024-04-24T23:27:25.676 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.674+0000 7fae777fe640 1 --2- 172.21.15.80:0/2164384578 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fae700a2b90 0x7fae700a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:25.676 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.675+0000 7fae85d9b640 1 --2- 172.21.15.80:0/2164384578 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fae700a2b90 0x7fae700a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:25.677 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.675+0000 7fae777fe640 1 -- 172.21.15.80:0/2164384578 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fae780ced90 con 0x7fae880ffe60 2024-04-24T23:27:25.677 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.675+0000 7fae85d9b640 1 --2- 172.21.15.80:0/2164384578 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fae700a2b90 0x7fae700a5050 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fae7c0057c0 tx=0x7fae7c031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:25.680 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.679+0000 7fae777fe640 1 -- 172.21.15.80:0/2164384578 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fae7801f020 con 0x7fae880ffe60 2024-04-24T23:27:25.697 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:25 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/3171401818' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-04-24T23:27:25.697 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:25 smithi134 ceph-mon[32619]: osdmap e88: 12 total, 12 up, 12 in 2024-04-24T23:27:25.697 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:25 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/2120142007' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-04-24T23:27:25.697 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:25 smithi134 ceph-mon[32619]: pgmap v297: 129 pgs: 101 active+clean, 2 creating+activating, 18 creating+peering, 8 unknown; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s wr, 11 op/s 2024-04-24T23:27:25.856 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.855+0000 7fae8cc3f640 1 -- 172.21.15.80:0/2164384578 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fae88062340 con 0x7fae880ffe60 2024-04-24T23:27:25.858 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.857+0000 7fae777fe640 1 -- 172.21.15.80:0/2164384578 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24137 (secure 0 0 0) 0x7fae7800d040 con 0x7fae880ffe60 2024-04-24T23:27:25.858 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-24T23:27:25.858 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":89,"fsid":"ec7885aa-0290-11ef-bc93-c7b262605968","created":"2024-04-24T23:19:05.586407+0000","modified":"2024-04-24T23:27:24.564722+0000","last_up_change":"2024-04-24T23:26:25.938316+0000","last_in_change":"2024-04-24T23:26:04.786699+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:22:33.124428+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:27:17.575788+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:27:18.492762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T23:27:22.438503+0000","flags":32773,"flags_names":"hashpspool,ec_overwrites,creating","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a525741d-b46c-4eef-aa64-3f057cac1497","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":2597718112},{"type":"v1","addr":"172.21.15.80:6803","nonce":2597718112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":2597718112},{"type":"v1","addr":"172.21.15.80:6805","nonce":2597718112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2597718112},{"type":"v1","addr":"172.21.15.80:6809","nonce":2597718112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":2597718112},{"type":"v1","addr":"172.21.15.80:6807","nonce":2597718112}]},"public_addr":"172.21.15.80:6803/2597718112","cluster_addr":"172.21.15.80:6805/2597718112","heartbeat_back_addr":"172.21.15.80:6809/2597718112","heartbeat_front_addr":"172.21.15.80:6807/2597718112","state":["exists","up"]},{"osd":1,"uuid":"2d0f0d67-6db4-40ac-91cc-58604ab6795a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":2395610201},{"type":"v1","addr":"172.21.15.134:6801","nonce":2395610201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":2395610201},{"type":"v1","addr":"172.21.15.134:6803","nonce":2395610201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":2395610201},{"type":"v1","addr":"172.21.15.134:6807","nonce":2395610201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":2395610201},{"type":"v1","addr":"172.21.15.134:6805","nonce":2395610201}]},"public_addr":"172.21.15.134:6801/2395610201","cluster_addr":"172.21.15.134:6803/2395610201","heartbeat_back_addr":"172.21.15.134:6807/2395610201","heartbeat_front_addr":"172.21.15.134:6805/2395610201","state":["exists","up"]},{"osd":2,"uuid":"781fb879-fedc-4c84-82fb-59e073bc4cf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1629152187},{"type":"v1","addr":"172.21.15.148:6801","nonce":1629152187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1629152187},{"type":"v1","addr":"172.21.15.148:6803","nonce":1629152187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1629152187},{"type":"v1","addr":"172.21.15.148:6807","nonce":1629152187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1629152187},{"type":"v1","addr":"172.21.15.148:6805","nonce":1629152187}]},"public_addr":"172.21.15.148:6801/1629152187","cluster_addr":"172.21.15.148:6803/1629152187","heartbeat_back_addr":"172.21.15.148:6807/1629152187","heartbeat_front_addr":"172.21.15.148:6805/1629152187","state":["exists","up"]},{"osd":3,"uuid":"cd982757-f2f6-4296-b3e7-4cd40fbd8d9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":3128298510},{"type":"v1","addr":"172.21.15.80:6811","nonce":3128298510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":3128298510},{"type":"v1","addr":"172.21.15.80:6813","nonce":3128298510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":3128298510},{"type":"v1","addr":"172.21.15.80:6817","nonce":3128298510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":3128298510},{"type":"v1","addr":"172.21.15.80:6815","nonce":3128298510}]},"public_addr":"172.21.15.80:6811/3128298510","cluster_addr":"172.21.15.80:6813/3128298510","heartbeat_back_addr":"172.21.15.80:6817/3128298510","heartbeat_front_addr":"172.21.15.80:6815/3128298510","state":["exists","up"]},{"osd":4,"uuid":"dfc55a79-a7e8-417a-b898-ec44171a54af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1917556060},{"type":"v1","addr":"172.21.15.134:6809","nonce":1917556060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1917556060},{"type":"v1","addr":"172.21.15.134:6811","nonce":1917556060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1917556060},{"type":"v1","addr":"172.21.15.134:6815","nonce":1917556060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1917556060},{"type":"v1","addr":"172.21.15.134:6813","nonce":1917556060}]},"public_addr":"172.21.15.134:6809/1917556060","cluster_addr":"172.21.15.134:6811/1917556060","heartbeat_back_addr":"172.21.15.134:6815/1917556060","heartbeat_front_addr":"172.21.15.134:6813/1917556060","state":["exists","up"]},{"osd":5,"uuid":"b5491337-0694-440c-b679-312f3d14cf65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":2715951281},{"type":"v1","addr":"172.21.15.148:6809","nonce":2715951281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":2715951281},{"type":"v1","addr":"172.21.15.148:6811","nonce":2715951281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":2715951281},{"type":"v1","addr":"172.21.15.148:6815","nonce":2715951281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":2715951281},{"type":"v1","addr":"172.21.15.148:6813","nonce":2715951281}]},"public_addr":"172.21.15.148:6809/2715951281","cluster_addr":"172.21.15.148:6811/2715951281","heartbeat_back_addr":"172.21.15.148:6815/2715951281","heartbeat_front_addr":"172.21.15.148:6813/2715951281","state":["exists","up"]},{"osd":6,"uuid":"ba4b5df7-d86e-47cd-ab67-cf763c4e4d6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":953261724},{"type":"v1","addr":"172.21.15.80:6819","nonce":953261724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":953261724},{"type":"v1","addr":"172.21.15.80:6821","nonce":953261724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":953261724},{"type":"v1","addr":"172.21.15.80:6825","nonce":953261724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":953261724},{"type":"v1","addr":"172.21.15.80:6823","nonce":953261724}]},"public_addr":"172.21.15.80:6819/953261724","cluster_addr":"172.21.15.80:6821/953261724","heartbeat_back_addr":"172.21.15.80:6825/953261724","heartbeat_front_addr":"172.21.15.80:6823/953261724","state":["exists","up"]},{"osd":7,"uuid":"464a503a-75ae-49e7-ab6e-a7f9f2a6a307","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":108354844},{"type":"v1","addr":"172.21.15.134:6817","nonce":108354844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":108354844},{"type":"v1","addr":"172.21.15.134:6819","nonce":108354844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":108354844},{"type":"v1","addr":"172.21.15.134:6823","nonce":108354844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":108354844},{"type":"v1","addr":"172.21.15.134:6821","nonce":108354844}]},"public_addr":"172.21.15.134:6817/108354844","cluster_addr":"172.21.15.134:6819/108354844","heartbeat_back_addr":"172.21.15.134:6823/108354844","heartbeat_front_addr":"172.21.15.134:6821/108354844","state":["exists","up"]},{"osd":8,"uuid":"ce544b2e-4540-47c0-9997-500c45351330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3559655329},{"type":"v1","addr":"172.21.15.148:6817","nonce":3559655329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":3559655329},{"type":"v1","addr":"172.21.15.148:6819","nonce":3559655329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":3559655329},{"type":"v1","addr":"172.21.15.148:6823","nonce":3559655329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3559655329},{"type":"v1","addr":"172.21.15.148:6821","nonce":3559655329}]},"public_addr":"172.21.15.148:6817/3559655329","cluster_addr":"172.21.15.148:6819/3559655329","heartbeat_back_addr":"172.21.15.148:6823/3559655329","heartbeat_front_addr":"172.21.15.148:6821/3559655329","state":["exists","up"]},{"osd":9,"uuid":"06de3732-19b9-496c-a464-55322efa6759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":4019913545},{"type":"v1","addr":"172.21.15.80:6827","nonce":4019913545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":4019913545},{"type":"v1","addr":"172.21.15.80:6829","nonce":4019913545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":4019913545},{"type":"v1","addr":"172.21.15.80:6833","nonce":4019913545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":4019913545},{"type":"v1","addr":"172.21.15.80:6831","nonce":4019913545}]},"public_addr":"172.21.15.80:6827/4019913545","cluster_addr":"172.21.15.80:6829/4019913545","heartbeat_back_addr":"172.21.15.80:6833/4019913545","heartbeat_front_addr":"172.21.15.80:6831/4019913545","state":["exists","up"]},{"osd":10,"uuid":"8d92f0d9-5c64-429a-b750-c33ae0e070c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":647343766},{"type":"v1","addr":"172.21.15.134:6825","nonce":647343766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":647343766},{"type":"v1","addr":"172.21.15.134:6827","nonce":647343766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":647343766},{"type":"v1","addr":"172.21.15.134:6831","nonce":647343766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":647343766},{"type":"v1","addr":"172.21.15.134:6829","nonce":647343766}]},"public_addr":"172.21.15.134:6825/647343766","cluster_addr":"172.21.15.134:6827/647343766","heartbeat_back_addr":"172.21.15.134:6831/647343766","heartbeat_front_addr":"172.21.15.134:6829/647343766","state":["exists","up"]},{"osd":11,"uuid":"4c003a41-f65b-479f-bc80-b27d680cf936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":4068024116},{"type":"v1","addr":"172.21.15.148:6825","nonce":4068024116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":4068024116},{"type":"v1","addr":"172.21.15.148:6827","nonce":4068024116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":4068024116},{"type":"v1","addr":"172.21.15.148:6831","nonce":4068024116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":4068024116},{"type":"v1","addr":"172.21.15.148:6829","nonce":4068024116}]},"public_addr":"172.21.15.148:6825/4068024116","cluster_addr":"172.21.15.148:6827/4068024116","heartbeat_back_addr":"172.21.15.148:6831/4068024116","heartbeat_front_addr":"172.21.15.148:6829/4068024116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:21:42.433631+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:22:06.145933+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:22:30.726143+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:22:55.729961+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:23:21.280913+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:23:46.609983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:24:12.585247+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:24:38.359287+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:25:04.491232+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:25:30.788089+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:25:56.891078+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:26:23.899113+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.80:0/2828378749":"2024-04-25T23:20:06.793609+0000","172.21.15.80:6801/233364790":"2024-04-25T23:20:06.793609+0000","172.21.15.80:6800/1614691410":"2024-04-25T23:19:40.490692+0000","172.21.15.80:0/1406187145":"2024-04-25T23:20:06.793609+0000","172.21.15.80:0/92953396":"2024-04-25T23:19:40.490692+0000","172.21.15.80:0/598129432":"2024-04-25T23:20:06.793609+0000","172.21.15.80:0/232193118":"2024-04-25T23:19:40.490692+0000","172.21.15.80:6800/233364790":"2024-04-25T23:20:06.793609+0000","172.21.15.80:6801/1614691410":"2024-04-25T23:19:40.490692+0000","172.21.15.80:0/2081562529":"2024-04-25T23:19:40.490692+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:27:25.860 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.859+0000 7fae8cc3f640 1 -- 172.21.15.80:0/2164384578 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fae700a2b90 msgr2=0x7fae700a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:25.860 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.859+0000 7fae8cc3f640 1 --2- 172.21.15.80:0/2164384578 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fae700a2b90 0x7fae700a5050 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fae7c0057c0 tx=0x7fae7c031400 comp rx=0 tx=0).stop 2024-04-24T23:27:25.861 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.859+0000 7fae8cc3f640 1 -- 172.21.15.80:0/2164384578 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fae880ffe60 msgr2=0x7fae88197c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:25.861 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.859+0000 7fae8cc3f640 1 --2- 172.21.15.80:0/2164384578 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fae880ffe60 0x7fae88197c40 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fae7800b450 tx=0x7fae7800b910 comp rx=0 tx=0).stop 2024-04-24T23:27:25.861 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.859+0000 7fae8cc3f640 1 -- 172.21.15.80:0/2164384578 shutdown_connections 2024-04-24T23:27:25.861 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.859+0000 7fae8cc3f640 1 --2- 172.21.15.80:0/2164384578 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fae700a2b90 0x7fae700a5050 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:25.861 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.859+0000 7fae8cc3f640 1 --2- 172.21.15.80:0/2164384578 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae88101a00 0x7fae8819f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:25.861 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.859+0000 7fae8cc3f640 1 --2- 172.21.15.80:0/2164384578 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fae88101040 0x7fae88198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:25.861 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.859+0000 7fae8cc3f640 1 --2- 172.21.15.80:0/2164384578 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fae880ffe60 0x7fae88197c40 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:25.861 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.859+0000 7fae8cc3f640 1 -- 172.21.15.80:0/2164384578 >> 172.21.15.80:0/2164384578 conn(0x7fae880fb420 msgr2=0x7fae880fc600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:25.861 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.859+0000 7fae8cc3f640 1 -- 172.21.15.80:0/2164384578 shutdown_connections 2024-04-24T23:27:25.861 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.859+0000 7fae8cc3f640 1 -- 172.21.15.80:0/2164384578 wait complete. 2024-04-24T23:27:25.872 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:27:25.972 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.968+0000 7f5a3e390640 1 -- 172.21.15.80:0/3144276010 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5a38102260 msgr2=0x7f5a38104650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:25.972 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.968+0000 7f5a3e390640 1 --2- 172.21.15.80:0/3144276010 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5a38102260 0x7f5a38104650 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c009900 tx=0x7f5a2c02a410 comp rx=0 tx=0).stop 2024-04-24T23:27:25.972 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.968+0000 7f5a3e390640 1 -- 172.21.15.80:0/3144276010 shutdown_connections 2024-04-24T23:27:25.972 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.968+0000 7f5a3e390640 1 --2- 172.21.15.80:0/3144276010 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a381074c0 0x7f5a381098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:25.972 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.968+0000 7f5a3e390640 1 --2- 172.21.15.80:0/3144276010 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a38104b90 0x7f5a38106f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:25.972 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.968+0000 7f5a3e390640 1 --2- 172.21.15.80:0/3144276010 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5a38102260 0x7f5a38104650 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:25.972 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.968+0000 7f5a3e390640 1 -- 172.21.15.80:0/3144276010 >> 172.21.15.80:0/3144276010 conn(0x7f5a380fbbf0 msgr2=0x7f5a380fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:25.972 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.968+0000 7f5a3e390640 1 -- 172.21.15.80:0/3144276010 shutdown_connections 2024-04-24T23:27:25.972 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.968+0000 7f5a3e390640 1 -- 172.21.15.80:0/3144276010 wait complete. 2024-04-24T23:27:25.972 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.969+0000 7f5a3e390640 1 Processor -- start 2024-04-24T23:27:25.973 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.969+0000 7f5a3e390640 1 -- start start 2024-04-24T23:27:25.973 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.969+0000 7f5a3e390640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a38102260 0x7f5a38197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:25.973 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.969+0000 7f5a3e390640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5a38104b90 0x7f5a38198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:25.973 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.969+0000 7f5a3e390640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a381074c0 0x7f5a3819f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:25.973 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.969+0000 7f5a3e390640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f5a3810d3f0 con 0x7f5a38104b90 2024-04-24T23:27:25.973 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.969+0000 7f5a3e390640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5a3810d270 con 0x7f5a38102260 2024-04-24T23:27:25.973 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.969+0000 7f5a3e390640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5a3810d570 con 0x7f5a381074c0 2024-04-24T23:27:25.973 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.969+0000 7f5a377fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5a38104b90 0x7f5a38198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:25.973 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.969+0000 7f5a377fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5a38104b90 0x7f5a38198000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:35636/0 (socket says 172.21.15.80:35636) 2024-04-24T23:27:25.973 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.969+0000 7f5a377fe640 1 -- 172.21.15.80:0/2779928268 learned_addr learned my addr 172.21.15.80:0/2779928268 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:27:25.973 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.969+0000 7f5a37fff640 1 --2- 172.21.15.80:0/2779928268 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a38102260 0x7f5a38197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:25.973 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.970+0000 7f5a3c92d640 1 --2- 172.21.15.80:0/2779928268 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a381074c0 0x7f5a3819f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:25.973 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.970+0000 7f5a377fe640 1 -- 172.21.15.80:0/2779928268 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a381074c0 msgr2=0x7f5a3819f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:25.973 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.970+0000 7f5a377fe640 1 --2- 172.21.15.80:0/2779928268 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a381074c0 0x7f5a3819f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:25.974 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.970+0000 7f5a377fe640 1 -- 172.21.15.80:0/2779928268 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a38102260 msgr2=0x7f5a38197ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:25.974 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.970+0000 7f5a377fe640 1 --2- 172.21.15.80:0/2779928268 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a38102260 0x7f5a38197ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:25.974 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.970+0000 7f5a377fe640 1 -- 172.21.15.80:0/2779928268 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a3819f6e0 con 0x7f5a38104b90 2024-04-24T23:27:25.974 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.970+0000 7f5a377fe640 1 --2- 172.21.15.80:0/2779928268 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5a38104b90 0x7f5a38198000 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5a2800d770 tx=0x7f5a2800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:25.974 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.970+0000 7f5a357fa640 1 -- 172.21.15.80:0/2779928268 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a28017070 con 0x7f5a38104b90 2024-04-24T23:27:25.974 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.970+0000 7f5a3e390640 1 -- 172.21.15.80:0/2779928268 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a3819f9d0 con 0x7f5a38104b90 2024-04-24T23:27:25.974 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.970+0000 7f5a357fa640 1 -- 172.21.15.80:0/2779928268 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a28009c30 con 0x7f5a38104b90 2024-04-24T23:27:25.974 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.970+0000 7f5a357fa640 1 -- 172.21.15.80:0/2779928268 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a2800f620 con 0x7f5a38104b90 2024-04-24T23:27:25.974 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.970+0000 7f5a3e390640 1 -- 172.21.15.80:0/2779928268 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a3819ff20 con 0x7f5a38104b90 2024-04-24T23:27:25.974 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.971+0000 7f5a3e390640 1 -- 172.21.15.80:0/2779928268 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a380fd6e0 con 0x7f5a38104b90 2024-04-24T23:27:25.975 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.972+0000 7f5a357fa640 1 -- 172.21.15.80:0/2779928268 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f5a2801a070 con 0x7f5a38104b90 2024-04-24T23:27:25.975 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.972+0000 7f5a357fa640 1 --2- 172.21.15.80:0/2779928268 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f5a200a2b90 0x7f5a200a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:25.975 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.973+0000 7f5a37fff640 1 --2- 172.21.15.80:0/2779928268 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f5a200a2b90 0x7f5a200a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:25.975 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.973+0000 7f5a357fa640 1 -- 172.21.15.80:0/2779928268 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f5a280ced90 con 0x7f5a38104b90 2024-04-24T23:27:25.975 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.973+0000 7f5a37fff640 1 --2- 172.21.15.80:0/2779928268 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f5a200a2b90 0x7f5a200a5050 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c0314e0 tx=0x7f5a2c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:25.978 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:25.977+0000 7f5a357fa640 1 -- 172.21.15.80:0/2779928268 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f5a28009de0 con 0x7f5a38104b90 2024-04-24T23:27:26.153 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.152+0000 7f5a3e390640 1 -- 172.21.15.80:0/2779928268 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5a380623e0 con 0x7f5a38104b90 2024-04-24T23:27:26.155 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.154+0000 7f5a357fa640 1 -- 172.21.15.80:0/2779928268 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24137 (secure 0 0 0) 0x7f5a2809bab0 con 0x7f5a38104b90 2024-04-24T23:27:26.155 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-24T23:27:26.155 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":89,"fsid":"ec7885aa-0290-11ef-bc93-c7b262605968","created":"2024-04-24T23:19:05.586407+0000","modified":"2024-04-24T23:27:24.564722+0000","last_up_change":"2024-04-24T23:26:25.938316+0000","last_in_change":"2024-04-24T23:26:04.786699+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:22:33.124428+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:27:17.575788+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:27:18.492762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T23:27:22.438503+0000","flags":32773,"flags_names":"hashpspool,ec_overwrites,creating","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a525741d-b46c-4eef-aa64-3f057cac1497","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":2597718112},{"type":"v1","addr":"172.21.15.80:6803","nonce":2597718112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":2597718112},{"type":"v1","addr":"172.21.15.80:6805","nonce":2597718112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2597718112},{"type":"v1","addr":"172.21.15.80:6809","nonce":2597718112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":2597718112},{"type":"v1","addr":"172.21.15.80:6807","nonce":2597718112}]},"public_addr":"172.21.15.80:6803/2597718112","cluster_addr":"172.21.15.80:6805/2597718112","heartbeat_back_addr":"172.21.15.80:6809/2597718112","heartbeat_front_addr":"172.21.15.80:6807/2597718112","state":["exists","up"]},{"osd":1,"uuid":"2d0f0d67-6db4-40ac-91cc-58604ab6795a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":2395610201},{"type":"v1","addr":"172.21.15.134:6801","nonce":2395610201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":2395610201},{"type":"v1","addr":"172.21.15.134:6803","nonce":2395610201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":2395610201},{"type":"v1","addr":"172.21.15.134:6807","nonce":2395610201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":2395610201},{"type":"v1","addr":"172.21.15.134:6805","nonce":2395610201}]},"public_addr":"172.21.15.134:6801/2395610201","cluster_addr":"172.21.15.134:6803/2395610201","heartbeat_back_addr":"172.21.15.134:6807/2395610201","heartbeat_front_addr":"172.21.15.134:6805/2395610201","state":["exists","up"]},{"osd":2,"uuid":"781fb879-fedc-4c84-82fb-59e073bc4cf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1629152187},{"type":"v1","addr":"172.21.15.148:6801","nonce":1629152187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1629152187},{"type":"v1","addr":"172.21.15.148:6803","nonce":1629152187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1629152187},{"type":"v1","addr":"172.21.15.148:6807","nonce":1629152187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1629152187},{"type":"v1","addr":"172.21.15.148:6805","nonce":1629152187}]},"public_addr":"172.21.15.148:6801/1629152187","cluster_addr":"172.21.15.148:6803/1629152187","heartbeat_back_addr":"172.21.15.148:6807/1629152187","heartbeat_front_addr":"172.21.15.148:6805/1629152187","state":["exists","up"]},{"osd":3,"uuid":"cd982757-f2f6-4296-b3e7-4cd40fbd8d9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":3128298510},{"type":"v1","addr":"172.21.15.80:6811","nonce":3128298510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":3128298510},{"type":"v1","addr":"172.21.15.80:6813","nonce":3128298510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":3128298510},{"type":"v1","addr":"172.21.15.80:6817","nonce":3128298510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":3128298510},{"type":"v1","addr":"172.21.15.80:6815","nonce":3128298510}]},"public_addr":"172.21.15.80:6811/3128298510","cluster_addr":"172.21.15.80:6813/3128298510","heartbeat_back_addr":"172.21.15.80:6817/3128298510","heartbeat_front_addr":"172.21.15.80:6815/3128298510","state":["exists","up"]},{"osd":4,"uuid":"dfc55a79-a7e8-417a-b898-ec44171a54af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1917556060},{"type":"v1","addr":"172.21.15.134:6809","nonce":1917556060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1917556060},{"type":"v1","addr":"172.21.15.134:6811","nonce":1917556060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1917556060},{"type":"v1","addr":"172.21.15.134:6815","nonce":1917556060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1917556060},{"type":"v1","addr":"172.21.15.134:6813","nonce":1917556060}]},"public_addr":"172.21.15.134:6809/1917556060","cluster_addr":"172.21.15.134:6811/1917556060","heartbeat_back_addr":"172.21.15.134:6815/1917556060","heartbeat_front_addr":"172.21.15.134:6813/1917556060","state":["exists","up"]},{"osd":5,"uuid":"b5491337-0694-440c-b679-312f3d14cf65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":2715951281},{"type":"v1","addr":"172.21.15.148:6809","nonce":2715951281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":2715951281},{"type":"v1","addr":"172.21.15.148:6811","nonce":2715951281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":2715951281},{"type":"v1","addr":"172.21.15.148:6815","nonce":2715951281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":2715951281},{"type":"v1","addr":"172.21.15.148:6813","nonce":2715951281}]},"public_addr":"172.21.15.148:6809/2715951281","cluster_addr":"172.21.15.148:6811/2715951281","heartbeat_back_addr":"172.21.15.148:6815/2715951281","heartbeat_front_addr":"172.21.15.148:6813/2715951281","state":["exists","up"]},{"osd":6,"uuid":"ba4b5df7-d86e-47cd-ab67-cf763c4e4d6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":953261724},{"type":"v1","addr":"172.21.15.80:6819","nonce":953261724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":953261724},{"type":"v1","addr":"172.21.15.80:6821","nonce":953261724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":953261724},{"type":"v1","addr":"172.21.15.80:6825","nonce":953261724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":953261724},{"type":"v1","addr":"172.21.15.80:6823","nonce":953261724}]},"public_addr":"172.21.15.80:6819/953261724","cluster_addr":"172.21.15.80:6821/953261724","heartbeat_back_addr":"172.21.15.80:6825/953261724","heartbeat_front_addr":"172.21.15.80:6823/953261724","state":["exists","up"]},{"osd":7,"uuid":"464a503a-75ae-49e7-ab6e-a7f9f2a6a307","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":108354844},{"type":"v1","addr":"172.21.15.134:6817","nonce":108354844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":108354844},{"type":"v1","addr":"172.21.15.134:6819","nonce":108354844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":108354844},{"type":"v1","addr":"172.21.15.134:6823","nonce":108354844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":108354844},{"type":"v1","addr":"172.21.15.134:6821","nonce":108354844}]},"public_addr":"172.21.15.134:6817/108354844","cluster_addr":"172.21.15.134:6819/108354844","heartbeat_back_addr":"172.21.15.134:6823/108354844","heartbeat_front_addr":"172.21.15.134:6821/108354844","state":["exists","up"]},{"osd":8,"uuid":"ce544b2e-4540-47c0-9997-500c45351330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3559655329},{"type":"v1","addr":"172.21.15.148:6817","nonce":3559655329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":3559655329},{"type":"v1","addr":"172.21.15.148:6819","nonce":3559655329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":3559655329},{"type":"v1","addr":"172.21.15.148:6823","nonce":3559655329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3559655329},{"type":"v1","addr":"172.21.15.148:6821","nonce":3559655329}]},"public_addr":"172.21.15.148:6817/3559655329","cluster_addr":"172.21.15.148:6819/3559655329","heartbeat_back_addr":"172.21.15.148:6823/3559655329","heartbeat_front_addr":"172.21.15.148:6821/3559655329","state":["exists","up"]},{"osd":9,"uuid":"06de3732-19b9-496c-a464-55322efa6759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":4019913545},{"type":"v1","addr":"172.21.15.80:6827","nonce":4019913545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":4019913545},{"type":"v1","addr":"172.21.15.80:6829","nonce":4019913545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":4019913545},{"type":"v1","addr":"172.21.15.80:6833","nonce":4019913545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":4019913545},{"type":"v1","addr":"172.21.15.80:6831","nonce":4019913545}]},"public_addr":"172.21.15.80:6827/4019913545","cluster_addr":"172.21.15.80:6829/4019913545","heartbeat_back_addr":"172.21.15.80:6833/4019913545","heartbeat_front_addr":"172.21.15.80:6831/4019913545","state":["exists","up"]},{"osd":10,"uuid":"8d92f0d9-5c64-429a-b750-c33ae0e070c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":647343766},{"type":"v1","addr":"172.21.15.134:6825","nonce":647343766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":647343766},{"type":"v1","addr":"172.21.15.134:6827","nonce":647343766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":647343766},{"type":"v1","addr":"172.21.15.134:6831","nonce":647343766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":647343766},{"type":"v1","addr":"172.21.15.134:6829","nonce":647343766}]},"public_addr":"172.21.15.134:6825/647343766","cluster_addr":"172.21.15.134:6827/647343766","heartbeat_back_addr":"172.21.15.134:6831/647343766","heartbeat_front_addr":"172.21.15.134:6829/647343766","state":["exists","up"]},{"osd":11,"uuid":"4c003a41-f65b-479f-bc80-b27d680cf936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":4068024116},{"type":"v1","addr":"172.21.15.148:6825","nonce":4068024116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":4068024116},{"type":"v1","addr":"172.21.15.148:6827","nonce":4068024116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":4068024116},{"type":"v1","addr":"172.21.15.148:6831","nonce":4068024116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":4068024116},{"type":"v1","addr":"172.21.15.148:6829","nonce":4068024116}]},"public_addr":"172.21.15.148:6825/4068024116","cluster_addr":"172.21.15.148:6827/4068024116","heartbeat_back_addr":"172.21.15.148:6831/4068024116","heartbeat_front_addr":"172.21.15.148:6829/4068024116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:21:42.433631+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:22:06.145933+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:22:30.726143+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:22:55.729961+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:23:21.280913+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:23:46.609983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:24:12.585247+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:24:38.359287+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:25:04.491232+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:25:30.788089+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:25:56.891078+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:26:23.899113+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.80:0/2828378749":"2024-04-25T23:20:06.793609+0000","172.21.15.80:6801/233364790":"2024-04-25T23:20:06.793609+0000","172.21.15.80:6800/1614691410":"2024-04-25T23:19:40.490692+0000","172.21.15.80:0/1406187145":"2024-04-25T23:20:06.793609+0000","172.21.15.80:0/92953396":"2024-04-25T23:19:40.490692+0000","172.21.15.80:0/598129432":"2024-04-25T23:20:06.793609+0000","172.21.15.80:0/232193118":"2024-04-25T23:19:40.490692+0000","172.21.15.80:6800/233364790":"2024-04-25T23:20:06.793609+0000","172.21.15.80:6801/1614691410":"2024-04-25T23:19:40.490692+0000","172.21.15.80:0/2081562529":"2024-04-25T23:19:40.490692+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:27:26.157 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.156+0000 7f5a3e390640 1 -- 172.21.15.80:0/2779928268 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f5a200a2b90 msgr2=0x7f5a200a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:26.157 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.156+0000 7f5a3e390640 1 --2- 172.21.15.80:0/2779928268 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f5a200a2b90 0x7f5a200a5050 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c0314e0 tx=0x7f5a2c000bc0 comp rx=0 tx=0).stop 2024-04-24T23:27:26.157 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.156+0000 7f5a3e390640 1 -- 172.21.15.80:0/2779928268 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5a38104b90 msgr2=0x7f5a38198000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:26.157 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.156+0000 7f5a3e390640 1 --2- 172.21.15.80:0/2779928268 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5a38104b90 0x7f5a38198000 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5a2800d770 tx=0x7f5a2800dc30 comp rx=0 tx=0).stop 2024-04-24T23:27:26.157 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.156+0000 7f5a3e390640 1 -- 172.21.15.80:0/2779928268 shutdown_connections 2024-04-24T23:27:26.157 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.156+0000 7f5a3e390640 1 --2- 172.21.15.80:0/2779928268 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f5a200a2b90 0x7f5a200a5050 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:26.157 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.156+0000 7f5a3e390640 1 --2- 172.21.15.80:0/2779928268 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a381074c0 0x7f5a3819f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:26.157 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.156+0000 7f5a3e390640 1 --2- 172.21.15.80:0/2779928268 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5a38104b90 0x7f5a38198000 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:26.158 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.156+0000 7f5a3e390640 1 --2- 172.21.15.80:0/2779928268 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a38102260 0x7f5a38197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:26.158 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.156+0000 7f5a3e390640 1 -- 172.21.15.80:0/2779928268 >> 172.21.15.80:0/2779928268 conn(0x7f5a380fbbf0 msgr2=0x7f5a38102ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:26.158 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.156+0000 7f5a3e390640 1 -- 172.21.15.80:0/2779928268 shutdown_connections 2024-04-24T23:27:26.158 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.156+0000 7f5a3e390640 1 -- 172.21.15.80:0/2779928268 wait complete. 2024-04-24T23:27:26.183 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:27:26.287 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.284+0000 7f900a589640 1 -- 172.21.15.80:0/3849151622 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9004100090 msgr2=0x7f90041004b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:26.287 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.284+0000 7f900a589640 1 --2- 172.21.15.80:0/3849151622 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9004100090 0x7f90041004b0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f8ff4009900 tx=0x7f8ff402a480 comp rx=0 tx=0).stop 2024-04-24T23:27:26.287 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.284+0000 7f900a589640 1 -- 172.21.15.80:0/3849151622 shutdown_connections 2024-04-24T23:27:26.287 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.284+0000 7f900a589640 1 --2- 172.21.15.80:0/3849151622 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9004101390 0x7f9004101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:26.287 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.284+0000 7f900a589640 1 --2- 172.21.15.80:0/3849151622 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90041009f0 0x7f9004100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:26.287 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.284+0000 7f900a589640 1 --2- 172.21.15.80:0/3849151622 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9004100090 0x7f90041004b0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:26.288 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.284+0000 7f900a589640 1 -- 172.21.15.80:0/3849151622 >> 172.21.15.80:0/3849151622 conn(0x7f90040fbc30 msgr2=0x7f90040fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:26.288 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.284+0000 7f900a589640 1 -- 172.21.15.80:0/3849151622 shutdown_connections 2024-04-24T23:27:26.288 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.284+0000 7f900a589640 1 -- 172.21.15.80:0/3849151622 wait complete. 2024-04-24T23:27:26.288 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.284+0000 7f900a589640 1 Processor -- start 2024-04-24T23:27:26.288 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.284+0000 7f900a589640 1 -- start start 2024-04-24T23:27:26.288 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.285+0000 7f900a589640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9004100090 0x7f9004199c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:26.288 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.285+0000 7f900a589640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90041009f0 0x7f900419a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:26.288 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.285+0000 7f900a589640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9004101390 0x7f90041a1230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:26.288 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.285+0000 7f900a589640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f900410f610 con 0x7f9004101390 2024-04-24T23:27:26.288 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.285+0000 7f900a589640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f900410f490 con 0x7f90041009f0 2024-04-24T23:27:26.288 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.285+0000 7f900a589640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f900410f790 con 0x7f9004100090 2024-04-24T23:27:26.288 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.285+0000 7f9008b26640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9004101390 0x7f90041a1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:26.288 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.285+0000 7f9008b26640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9004101390 0x7f90041a1230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:35668/0 (socket says 172.21.15.80:35668) 2024-04-24T23:27:26.288 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.285+0000 7f9008b26640 1 -- 172.21.15.80:0/396779641 learned_addr learned my addr 172.21.15.80:0/396779641 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:27:26.288 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.285+0000 7f9003fff640 1 --2- 172.21.15.80:0/396779641 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9004100090 0x7f9004199c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:26.289 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.285+0000 7f90037fe640 1 --2- 172.21.15.80:0/396779641 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90041009f0 0x7f900419a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:26.289 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.285+0000 7f9008b26640 1 -- 172.21.15.80:0/396779641 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9004100090 msgr2=0x7f9004199c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:26.289 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.285+0000 7f9008b26640 1 --2- 172.21.15.80:0/396779641 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9004100090 0x7f9004199c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:26.289 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.285+0000 7f9008b26640 1 -- 172.21.15.80:0/396779641 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90041009f0 msgr2=0x7f900419a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:26.289 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.285+0000 7f9008b26640 1 --2- 172.21.15.80:0/396779641 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90041009f0 0x7f900419a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:26.289 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.285+0000 7f9008b26640 1 -- 172.21.15.80:0/396779641 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90041a18d0 con 0x7f9004101390 2024-04-24T23:27:26.289 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.285+0000 7f90037fe640 1 --2- 172.21.15.80:0/396779641 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90041009f0 0x7f900419a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:27:26.289 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.285+0000 7f9008b26640 1 --2- 172.21.15.80:0/396779641 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9004101390 0x7f90041a1230 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc00eb10 tx=0x7f8ffc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:26.289 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.285+0000 7f90017fa640 1 -- 172.21.15.80:0/396779641 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ffc004b20 con 0x7f9004101390 2024-04-24T23:27:26.289 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.285+0000 7f900a589640 1 -- 172.21.15.80:0/396779641 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90041a1bc0 con 0x7f9004101390 2024-04-24T23:27:26.289 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.285+0000 7f90017fa640 1 -- 172.21.15.80:0/396779641 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ffc004cc0 con 0x7f9004101390 2024-04-24T23:27:26.289 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.285+0000 7f90017fa640 1 -- 172.21.15.80:0/396779641 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ffc010760 con 0x7f9004101390 2024-04-24T23:27:26.289 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.285+0000 7f900a589640 1 -- 172.21.15.80:0/396779641 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90041a2110 con 0x7f9004101390 2024-04-24T23:27:26.289 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.286+0000 7f900a589640 1 -- 172.21.15.80:0/396779641 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f900410ade0 con 0x7f9004101390 2024-04-24T23:27:26.290 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.287+0000 7f90017fa640 1 -- 172.21.15.80:0/396779641 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f8ffc005020 con 0x7f9004101390 2024-04-24T23:27:26.290 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.288+0000 7f90017fa640 1 --2- 172.21.15.80:0/396779641 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f8fec0a2b90 0x7f8fec0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:26.290 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.288+0000 7f9003fff640 1 --2- 172.21.15.80:0/396779641 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f8fec0a2b90 0x7f8fec0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:26.290 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.288+0000 7f90017fa640 1 -- 172.21.15.80:0/396779641 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f8ffc0cf9d0 con 0x7f9004101390 2024-04-24T23:27:26.291 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.289+0000 7f9003fff640 1 --2- 172.21.15.80:0/396779641 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f8fec0a2b90 0x7f8fec0a5050 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f900419ab60 tx=0x7f8ff4000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:26.293 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.293+0000 7f90017fa640 1 -- 172.21.15.80:0/396779641 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f8ffc09c6f0 con 0x7f9004101390 2024-04-24T23:27:26.505 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.504+0000 7f900a589640 1 -- 172.21.15.80:0/396779641 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f900402d170 con 0x7f9004101390 2024-04-24T23:27:26.506 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.505+0000 7f90017fa640 1 -- 172.21.15.80:0/396779641 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11800 (secure 0 0 0) 0x7f8ffc09bff0 con 0x7f9004101390 2024-04-24T23:27:26.506 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-24T23:27:26.506 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/891591085","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":891591085},{"type":"v1","addr":"172.21.15.80:6835","nonce":891591085}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6837/259529026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6836","nonce":259529026},{"type":"v1","addr":"172.21.15.80:6837","nonce":259529026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14646,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6839/4194284854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6838","nonce":4194284854},{"type":"v1","addr":"172.21.15.80:6839","nonce":4194284854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14658,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6841/3145730958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6840","nonce":3145730958},{"type":"v1","addr":"172.21.15.80:6841","nonce":3145730958}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24395,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/3736784385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3736784385},{"type":"v1","addr":"172.21.15.134:6833","nonce":3736784385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/124777462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":124777462},{"type":"v1","addr":"172.21.15.148:6833","nonce":124777462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/1399801120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1399801120},{"type":"v1","addr":"172.21.15.134:6835","nonce":1399801120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24458,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/430512704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":430512704},{"type":"v1","addr":"172.21.15.134:6837","nonce":430512704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24460,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/3002319310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3002319310},{"type":"v1","addr":"172.21.15.148:6835","nonce":3002319310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24472,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/856528403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":856528403},{"type":"v1","addr":"172.21.15.148:6837","nonce":856528403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24488,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6839/1743815021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":1743815021},{"type":"v1","addr":"172.21.15.134:6839","nonce":1743815021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:27:19.300566+0000","modified":"2024-04-24T23:27:24.564912+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.148:6839/2952757594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":2952757594},{"type":"v1","addr":"172.21.15.148:6839","nonce":2952757594}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24496,"qdb_cluster":[24496]},"id":1}]} 2024-04-24T23:27:26.507 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 17 2024-04-24T23:27:26.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:26 smithi148 ceph-mon[32528]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-24T23:27:26.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:26 smithi148 ceph-mon[32528]: osdmap e89: 12 total, 12 up, 12 in 2024-04-24T23:27:26.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:26 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/2120142007' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-04-24T23:27:26.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:26 smithi148 ceph-mon[32528]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-04-24T23:27:26.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:26 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/967464213' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:27:26.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:26 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/2164384578' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:27:26.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:26 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/2779928268' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:27:26.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:26 smithi148 ceph-mon[32528]: osdmap e90: 12 total, 12 up, 12 in 2024-04-24T23:27:26.508 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.507+0000 7f900a589640 1 -- 172.21.15.80:0/396779641 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f8fec0a2b90 msgr2=0x7f8fec0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:26.508 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.507+0000 7f900a589640 1 --2- 172.21.15.80:0/396779641 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f8fec0a2b90 0x7f8fec0a5050 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f900419ab60 tx=0x7f8ff4000bc0 comp rx=0 tx=0).stop 2024-04-24T23:27:26.508 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.507+0000 7f900a589640 1 -- 172.21.15.80:0/396779641 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9004101390 msgr2=0x7f90041a1230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:26.509 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.507+0000 7f900a589640 1 --2- 172.21.15.80:0/396779641 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9004101390 0x7f90041a1230 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc00eb10 tx=0x7f8ffc00c2d0 comp rx=0 tx=0).stop 2024-04-24T23:27:26.509 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.507+0000 7f900a589640 1 -- 172.21.15.80:0/396779641 shutdown_connections 2024-04-24T23:27:26.509 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.507+0000 7f900a589640 1 --2- 172.21.15.80:0/396779641 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f8fec0a2b90 0x7f8fec0a5050 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:26.509 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.507+0000 7f900a589640 1 --2- 172.21.15.80:0/396779641 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9004101390 0x7f90041a1230 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:26.509 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.507+0000 7f900a589640 1 --2- 172.21.15.80:0/396779641 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90041009f0 0x7f900419a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:26.509 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.507+0000 7f900a589640 1 --2- 172.21.15.80:0/396779641 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9004100090 0x7f9004199c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:26.509 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.507+0000 7f900a589640 1 -- 172.21.15.80:0/396779641 >> 172.21.15.80:0/396779641 conn(0x7f90040fbc30 msgr2=0x7f90040fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:26.509 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.507+0000 7f900a589640 1 -- 172.21.15.80:0/396779641 shutdown_connections 2024-04-24T23:27:26.509 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.507+0000 7f900a589640 1 -- 172.21.15.80:0/396779641 wait complete. 2024-04-24T23:27:26.518 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:26 smithi080 ceph-mon[29152]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-24T23:27:26.518 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:26 smithi080 ceph-mon[29152]: osdmap e89: 12 total, 12 up, 12 in 2024-04-24T23:27:26.518 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:26 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/2120142007' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-04-24T23:27:26.518 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:26 smithi080 ceph-mon[29152]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-04-24T23:27:26.518 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:26 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/967464213' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:27:26.519 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:26 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/2164384578' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:27:26.519 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:26 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/2779928268' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:27:26.519 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:26 smithi080 ceph-mon[29152]: osdmap e90: 12 total, 12 up, 12 in 2024-04-24T23:27:26.520 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:27:26.620 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.616+0000 7f579b095640 1 -- 172.21.15.80:0/1726118420 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5794107460 msgr2=0x7f57941098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:26.620 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.616+0000 7f579b095640 1 --2- 172.21.15.80:0/1726118420 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5794107460 0x7f57941098b0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f578c009ca0 tx=0x7f578c02a8c0 comp rx=0 tx=0).stop 2024-04-24T23:27:26.620 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.616+0000 7f579b095640 1 -- 172.21.15.80:0/1726118420 shutdown_connections 2024-04-24T23:27:26.620 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.616+0000 7f579b095640 1 --2- 172.21.15.80:0/1726118420 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5794107460 0x7f57941098b0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:26.620 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.616+0000 7f579b095640 1 --2- 172.21.15.80:0/1726118420 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5794104b30 0x7f5794106f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:26.620 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.616+0000 7f579b095640 1 --2- 172.21.15.80:0/1726118420 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5794102200 0x7f57941045f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:26.620 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.616+0000 7f579b095640 1 -- 172.21.15.80:0/1726118420 >> 172.21.15.80:0/1726118420 conn(0x7f57940fbbf0 msgr2=0x7f57940fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:26.620 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.616+0000 7f579b095640 1 -- 172.21.15.80:0/1726118420 shutdown_connections 2024-04-24T23:27:26.620 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.616+0000 7f579b095640 1 -- 172.21.15.80:0/1726118420 wait complete. 2024-04-24T23:27:26.621 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.617+0000 7f579b095640 1 Processor -- start 2024-04-24T23:27:26.621 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.617+0000 7f579b095640 1 -- start start 2024-04-24T23:27:26.621 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.617+0000 7f579b095640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5794102200 0x7f5794101b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:26.621 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.617+0000 7f579b095640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5794104b30 0x7f57941001b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:26.621 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.617+0000 7f579b095640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5794107460 0x7f57941006f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:26.621 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.617+0000 7f579b095640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f579410d420 con 0x7f5794107460 2024-04-24T23:27:26.621 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.617+0000 7f579b095640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f579410d2a0 con 0x7f5794102200 2024-04-24T23:27:26.621 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.617+0000 7f579b095640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f579410d5a0 con 0x7f5794104b30 2024-04-24T23:27:26.621 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.617+0000 7f5799632640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5794107460 0x7f57941006f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:26.621 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.617+0000 7f5799632640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5794107460 0x7f57941006f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:35680/0 (socket says 172.21.15.80:35680) 2024-04-24T23:27:26.621 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.617+0000 7f5799632640 1 -- 172.21.15.80:0/1340170933 learned_addr learned my addr 172.21.15.80:0/1340170933 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:27:26.621 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.617+0000 7f5793fff640 1 --2- 172.21.15.80:0/1340170933 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5794104b30 0x7f57941001b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:26.621 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.617+0000 7f5798e31640 1 --2- 172.21.15.80:0/1340170933 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5794102200 0x7f5794101b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:26.621 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.617+0000 7f5799632640 1 -- 172.21.15.80:0/1340170933 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5794104b30 msgr2=0x7f57941001b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:26.622 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.617+0000 7f5799632640 1 --2- 172.21.15.80:0/1340170933 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5794104b30 0x7f57941001b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:26.622 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.617+0000 7f5799632640 1 -- 172.21.15.80:0/1340170933 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5794102200 msgr2=0x7f5794101b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:26.622 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.617+0000 7f5799632640 1 --2- 172.21.15.80:0/1340170933 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5794102200 0x7f5794101b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:26.622 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.617+0000 7f5799632640 1 -- 172.21.15.80:0/1340170933 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5794100f80 con 0x7f5794107460 2024-04-24T23:27:26.622 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.618+0000 7f5799632640 1 --2- 172.21.15.80:0/1340170933 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5794107460 0x7f57941006f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f578c005e30 tx=0x7f578c031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:26.622 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.618+0000 7f5791ffb640 1 -- 172.21.15.80:0/1340170933 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f578c003c30 con 0x7f5794107460 2024-04-24T23:27:26.622 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.618+0000 7f5791ffb640 1 -- 172.21.15.80:0/1340170933 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f578c003dd0 con 0x7f5794107460 2024-04-24T23:27:26.622 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.618+0000 7f579b095640 1 -- 172.21.15.80:0/1340170933 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57941a1b20 con 0x7f5794107460 2024-04-24T23:27:26.622 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.618+0000 7f5791ffb640 1 -- 172.21.15.80:0/1340170933 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f578c02fde0 con 0x7f5794107460 2024-04-24T23:27:26.622 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.618+0000 7f579b095640 1 -- 172.21.15.80:0/1340170933 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57941a2070 con 0x7f5794107460 2024-04-24T23:27:26.622 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.619+0000 7f579b095640 1 -- 172.21.15.80:0/1340170933 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57940fd6e0 con 0x7f5794107460 2024-04-24T23:27:26.623 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.620+0000 7f5791ffb640 1 -- 172.21.15.80:0/1340170933 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f578c004050 con 0x7f5794107460 2024-04-24T23:27:26.623 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.621+0000 7f5791ffb640 1 --2- 172.21.15.80:0/1340170933 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f57840a2b90 0x7f57840a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:26.623 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.621+0000 7f5798e31640 1 --2- 172.21.15.80:0/1340170933 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f57840a2b90 0x7f57840a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:26.623 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.621+0000 7f5791ffb640 1 -- 172.21.15.80:0/1340170933 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f578c0f0850 con 0x7f5794107460 2024-04-24T23:27:26.623 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.621+0000 7f5798e31640 1 --2- 172.21.15.80:0/1340170933 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f57840a2b90 0x7f57840a5050 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f5788005fd0 tx=0x7f5788005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:26.626 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.625+0000 7f5791ffb640 1 -- 172.21.15.80:0/1340170933 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f578c0bd570 con 0x7f5794107460 2024-04-24T23:27:26.697 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:26 smithi134 ceph-mon[32619]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-24T23:27:26.697 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:26 smithi134 ceph-mon[32619]: osdmap e89: 12 total, 12 up, 12 in 2024-04-24T23:27:26.697 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:26 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/2120142007' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-04-24T23:27:26.697 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:26 smithi134 ceph-mon[32619]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-04-24T23:27:26.698 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:26 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/967464213' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:27:26.698 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:26 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/2164384578' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:27:26.698 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:26 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/2779928268' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:27:26.698 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:26 smithi134 ceph-mon[32619]: osdmap e90: 12 total, 12 up, 12 in 2024-04-24T23:27:26.806 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.805+0000 7f579b095640 1 -- 172.21.15.80:0/1340170933 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5794062340 con 0x7f5794107460 2024-04-24T23:27:26.807 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.806+0000 7f5791ffb640 1 -- 172.21.15.80:0/1340170933 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24124 (secure 0 0 0) 0x7f578c0bce70 con 0x7f5794107460 2024-04-24T23:27:26.807 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-24T23:27:26.808 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":90,"fsid":"ec7885aa-0290-11ef-bc93-c7b262605968","created":"2024-04-24T23:19:05.586407+0000","modified":"2024-04-24T23:27:26.212213+0000","last_up_change":"2024-04-24T23:26:25.938316+0000","last_in_change":"2024-04-24T23:26:04.786699+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:22:33.124428+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:27:17.575788+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:27:18.492762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T23:27:22.438503+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a525741d-b46c-4eef-aa64-3f057cac1497","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":2597718112},{"type":"v1","addr":"172.21.15.80:6803","nonce":2597718112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":2597718112},{"type":"v1","addr":"172.21.15.80:6805","nonce":2597718112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2597718112},{"type":"v1","addr":"172.21.15.80:6809","nonce":2597718112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":2597718112},{"type":"v1","addr":"172.21.15.80:6807","nonce":2597718112}]},"public_addr":"172.21.15.80:6803/2597718112","cluster_addr":"172.21.15.80:6805/2597718112","heartbeat_back_addr":"172.21.15.80:6809/2597718112","heartbeat_front_addr":"172.21.15.80:6807/2597718112","state":["exists","up"]},{"osd":1,"uuid":"2d0f0d67-6db4-40ac-91cc-58604ab6795a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":2395610201},{"type":"v1","addr":"172.21.15.134:6801","nonce":2395610201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":2395610201},{"type":"v1","addr":"172.21.15.134:6803","nonce":2395610201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":2395610201},{"type":"v1","addr":"172.21.15.134:6807","nonce":2395610201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":2395610201},{"type":"v1","addr":"172.21.15.134:6805","nonce":2395610201}]},"public_addr":"172.21.15.134:6801/2395610201","cluster_addr":"172.21.15.134:6803/2395610201","heartbeat_back_addr":"172.21.15.134:6807/2395610201","heartbeat_front_addr":"172.21.15.134:6805/2395610201","state":["exists","up"]},{"osd":2,"uuid":"781fb879-fedc-4c84-82fb-59e073bc4cf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1629152187},{"type":"v1","addr":"172.21.15.148:6801","nonce":1629152187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1629152187},{"type":"v1","addr":"172.21.15.148:6803","nonce":1629152187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1629152187},{"type":"v1","addr":"172.21.15.148:6807","nonce":1629152187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1629152187},{"type":"v1","addr":"172.21.15.148:6805","nonce":1629152187}]},"public_addr":"172.21.15.148:6801/1629152187","cluster_addr":"172.21.15.148:6803/1629152187","heartbeat_back_addr":"172.21.15.148:6807/1629152187","heartbeat_front_addr":"172.21.15.148:6805/1629152187","state":["exists","up"]},{"osd":3,"uuid":"cd982757-f2f6-4296-b3e7-4cd40fbd8d9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":3128298510},{"type":"v1","addr":"172.21.15.80:6811","nonce":3128298510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":3128298510},{"type":"v1","addr":"172.21.15.80:6813","nonce":3128298510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":3128298510},{"type":"v1","addr":"172.21.15.80:6817","nonce":3128298510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":3128298510},{"type":"v1","addr":"172.21.15.80:6815","nonce":3128298510}]},"public_addr":"172.21.15.80:6811/3128298510","cluster_addr":"172.21.15.80:6813/3128298510","heartbeat_back_addr":"172.21.15.80:6817/3128298510","heartbeat_front_addr":"172.21.15.80:6815/3128298510","state":["exists","up"]},{"osd":4,"uuid":"dfc55a79-a7e8-417a-b898-ec44171a54af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1917556060},{"type":"v1","addr":"172.21.15.134:6809","nonce":1917556060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1917556060},{"type":"v1","addr":"172.21.15.134:6811","nonce":1917556060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1917556060},{"type":"v1","addr":"172.21.15.134:6815","nonce":1917556060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1917556060},{"type":"v1","addr":"172.21.15.134:6813","nonce":1917556060}]},"public_addr":"172.21.15.134:6809/1917556060","cluster_addr":"172.21.15.134:6811/1917556060","heartbeat_back_addr":"172.21.15.134:6815/1917556060","heartbeat_front_addr":"172.21.15.134:6813/1917556060","state":["exists","up"]},{"osd":5,"uuid":"b5491337-0694-440c-b679-312f3d14cf65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":2715951281},{"type":"v1","addr":"172.21.15.148:6809","nonce":2715951281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":2715951281},{"type":"v1","addr":"172.21.15.148:6811","nonce":2715951281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":2715951281},{"type":"v1","addr":"172.21.15.148:6815","nonce":2715951281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":2715951281},{"type":"v1","addr":"172.21.15.148:6813","nonce":2715951281}]},"public_addr":"172.21.15.148:6809/2715951281","cluster_addr":"172.21.15.148:6811/2715951281","heartbeat_back_addr":"172.21.15.148:6815/2715951281","heartbeat_front_addr":"172.21.15.148:6813/2715951281","state":["exists","up"]},{"osd":6,"uuid":"ba4b5df7-d86e-47cd-ab67-cf763c4e4d6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":953261724},{"type":"v1","addr":"172.21.15.80:6819","nonce":953261724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":953261724},{"type":"v1","addr":"172.21.15.80:6821","nonce":953261724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":953261724},{"type":"v1","addr":"172.21.15.80:6825","nonce":953261724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":953261724},{"type":"v1","addr":"172.21.15.80:6823","nonce":953261724}]},"public_addr":"172.21.15.80:6819/953261724","cluster_addr":"172.21.15.80:6821/953261724","heartbeat_back_addr":"172.21.15.80:6825/953261724","heartbeat_front_addr":"172.21.15.80:6823/953261724","state":["exists","up"]},{"osd":7,"uuid":"464a503a-75ae-49e7-ab6e-a7f9f2a6a307","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":108354844},{"type":"v1","addr":"172.21.15.134:6817","nonce":108354844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":108354844},{"type":"v1","addr":"172.21.15.134:6819","nonce":108354844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":108354844},{"type":"v1","addr":"172.21.15.134:6823","nonce":108354844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":108354844},{"type":"v1","addr":"172.21.15.134:6821","nonce":108354844}]},"public_addr":"172.21.15.134:6817/108354844","cluster_addr":"172.21.15.134:6819/108354844","heartbeat_back_addr":"172.21.15.134:6823/108354844","heartbeat_front_addr":"172.21.15.134:6821/108354844","state":["exists","up"]},{"osd":8,"uuid":"ce544b2e-4540-47c0-9997-500c45351330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3559655329},{"type":"v1","addr":"172.21.15.148:6817","nonce":3559655329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":3559655329},{"type":"v1","addr":"172.21.15.148:6819","nonce":3559655329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":3559655329},{"type":"v1","addr":"172.21.15.148:6823","nonce":3559655329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3559655329},{"type":"v1","addr":"172.21.15.148:6821","nonce":3559655329}]},"public_addr":"172.21.15.148:6817/3559655329","cluster_addr":"172.21.15.148:6819/3559655329","heartbeat_back_addr":"172.21.15.148:6823/3559655329","heartbeat_front_addr":"172.21.15.148:6821/3559655329","state":["exists","up"]},{"osd":9,"uuid":"06de3732-19b9-496c-a464-55322efa6759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":4019913545},{"type":"v1","addr":"172.21.15.80:6827","nonce":4019913545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":4019913545},{"type":"v1","addr":"172.21.15.80:6829","nonce":4019913545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":4019913545},{"type":"v1","addr":"172.21.15.80:6833","nonce":4019913545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":4019913545},{"type":"v1","addr":"172.21.15.80:6831","nonce":4019913545}]},"public_addr":"172.21.15.80:6827/4019913545","cluster_addr":"172.21.15.80:6829/4019913545","heartbeat_back_addr":"172.21.15.80:6833/4019913545","heartbeat_front_addr":"172.21.15.80:6831/4019913545","state":["exists","up"]},{"osd":10,"uuid":"8d92f0d9-5c64-429a-b750-c33ae0e070c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":647343766},{"type":"v1","addr":"172.21.15.134:6825","nonce":647343766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":647343766},{"type":"v1","addr":"172.21.15.134:6827","nonce":647343766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":647343766},{"type":"v1","addr":"172.21.15.134:6831","nonce":647343766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":647343766},{"type":"v1","addr":"172.21.15.134:6829","nonce":647343766}]},"public_addr":"172.21.15.134:6825/647343766","cluster_addr":"172.21.15.134:6827/647343766","heartbeat_back_addr":"172.21.15.134:6831/647343766","heartbeat_front_addr":"172.21.15.134:6829/647343766","state":["exists","up"]},{"osd":11,"uuid":"4c003a41-f65b-479f-bc80-b27d680cf936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":4068024116},{"type":"v1","addr":"172.21.15.148:6825","nonce":4068024116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":4068024116},{"type":"v1","addr":"172.21.15.148:6827","nonce":4068024116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":4068024116},{"type":"v1","addr":"172.21.15.148:6831","nonce":4068024116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":4068024116},{"type":"v1","addr":"172.21.15.148:6829","nonce":4068024116}]},"public_addr":"172.21.15.148:6825/4068024116","cluster_addr":"172.21.15.148:6827/4068024116","heartbeat_back_addr":"172.21.15.148:6831/4068024116","heartbeat_front_addr":"172.21.15.148:6829/4068024116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:21:42.433631+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:22:06.145933+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:22:30.726143+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:22:55.729961+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:23:21.280913+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:23:46.609983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:24:12.585247+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:24:38.359287+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:25:04.491232+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:25:30.788089+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:25:56.891078+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:26:23.899113+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.80:0/2828378749":"2024-04-25T23:20:06.793609+0000","172.21.15.80:6801/233364790":"2024-04-25T23:20:06.793609+0000","172.21.15.80:6800/1614691410":"2024-04-25T23:19:40.490692+0000","172.21.15.80:0/1406187145":"2024-04-25T23:20:06.793609+0000","172.21.15.80:0/92953396":"2024-04-25T23:19:40.490692+0000","172.21.15.80:0/598129432":"2024-04-25T23:20:06.793609+0000","172.21.15.80:0/232193118":"2024-04-25T23:19:40.490692+0000","172.21.15.80:6800/233364790":"2024-04-25T23:20:06.793609+0000","172.21.15.80:6801/1614691410":"2024-04-25T23:19:40.490692+0000","172.21.15.80:0/2081562529":"2024-04-25T23:19:40.490692+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:27:26.810 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.808+0000 7f579b095640 1 -- 172.21.15.80:0/1340170933 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f57840a2b90 msgr2=0x7f57840a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:26.810 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.808+0000 7f579b095640 1 --2- 172.21.15.80:0/1340170933 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f57840a2b90 0x7f57840a5050 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f5788005fd0 tx=0x7f5788005850 comp rx=0 tx=0).stop 2024-04-24T23:27:26.810 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.808+0000 7f579b095640 1 -- 172.21.15.80:0/1340170933 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5794107460 msgr2=0x7f57941006f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:26.810 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.808+0000 7f579b095640 1 --2- 172.21.15.80:0/1340170933 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5794107460 0x7f57941006f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f578c005e30 tx=0x7f578c031ee0 comp rx=0 tx=0).stop 2024-04-24T23:27:26.810 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.808+0000 7f579b095640 1 -- 172.21.15.80:0/1340170933 shutdown_connections 2024-04-24T23:27:26.810 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.808+0000 7f579b095640 1 --2- 172.21.15.80:0/1340170933 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f57840a2b90 0x7f57840a5050 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:26.810 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.808+0000 7f579b095640 1 --2- 172.21.15.80:0/1340170933 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5794107460 0x7f57941006f0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:26.810 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.808+0000 7f579b095640 1 --2- 172.21.15.80:0/1340170933 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5794104b30 0x7f57941001b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:26.810 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.808+0000 7f579b095640 1 --2- 172.21.15.80:0/1340170933 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5794102200 0x7f5794101b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:26.810 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.808+0000 7f579b095640 1 -- 172.21.15.80:0/1340170933 >> 172.21.15.80:0/1340170933 conn(0x7f57940fbbf0 msgr2=0x7f5794102b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:26.810 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.808+0000 7f579b095640 1 -- 172.21.15.80:0/1340170933 shutdown_connections 2024-04-24T23:27:26.810 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.808+0000 7f579b095640 1 -- 172.21.15.80:0/1340170933 wait complete. 2024-04-24T23:27:26.821 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-24T23:27:19.300566+0000', 'modified': '2024-04-24T23:27:24.564912+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24496}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24496': {'gid': 24496, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.148:6839/2952757594', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6838', 'nonce': 2952757594}, {'type': 'v1', 'addr': '172.21.15.148:6839', 'nonce': 2952757594}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 24496, 'qdb_cluster': [24496]} 2024-04-24T23:27:26.821 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-24T23:27:26.822 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-24T23:27:26.924 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.921+0000 7f493e2cd640 1 -- 172.21.15.80:0/815569889 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f49381022c0 msgr2=0x7f49381046b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:26.924 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.921+0000 7f493e2cd640 1 --2- 172.21.15.80:0/815569889 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f49381022c0 0x7f49381046b0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f4928009900 tx=0x7f492802a410 comp rx=0 tx=0).stop 2024-04-24T23:27:26.924 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.921+0000 7f493e2cd640 1 -- 172.21.15.80:0/815569889 shutdown_connections 2024-04-24T23:27:26.925 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.921+0000 7f493e2cd640 1 --2- 172.21.15.80:0/815569889 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4938107520 0x7f4938109910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:26.925 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.921+0000 7f493e2cd640 1 --2- 172.21.15.80:0/815569889 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4938104bf0 0x7f4938106fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:26.925 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.921+0000 7f493e2cd640 1 --2- 172.21.15.80:0/815569889 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f49381022c0 0x7f49381046b0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:26.925 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.921+0000 7f493e2cd640 1 -- 172.21.15.80:0/815569889 >> 172.21.15.80:0/815569889 conn(0x7f49380fbc30 msgr2=0x7f49380fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:26.925 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.921+0000 7f493e2cd640 1 -- 172.21.15.80:0/815569889 shutdown_connections 2024-04-24T23:27:26.925 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.921+0000 7f493e2cd640 1 -- 172.21.15.80:0/815569889 wait complete. 2024-04-24T23:27:26.925 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.922+0000 7f493e2cd640 1 Processor -- start 2024-04-24T23:27:26.925 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.922+0000 7f493e2cd640 1 -- start start 2024-04-24T23:27:26.925 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.922+0000 7f493e2cd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49381022c0 0x7f4938197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:26.925 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.922+0000 7f493e2cd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4938104bf0 0x7f4938198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:26.925 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.922+0000 7f493e2cd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4938107520 0x7f493819f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:26.925 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.922+0000 7f493e2cd640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f493810d490 con 0x7f4938104bf0 2024-04-24T23:27:26.925 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.922+0000 7f493e2cd640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f493810d310 con 0x7f4938107520 2024-04-24T23:27:26.925 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.922+0000 7f493e2cd640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f493810d610 con 0x7f49381022c0 2024-04-24T23:27:26.926 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.922+0000 7f49377fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4938104bf0 0x7f4938198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:26.926 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.922+0000 7f49377fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4938104bf0 0x7f4938198010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:35696/0 (socket says 172.21.15.80:35696) 2024-04-24T23:27:26.926 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.922+0000 7f49377fe640 1 -- 172.21.15.80:0/2564866994 learned_addr learned my addr 172.21.15.80:0/2564866994 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:27:26.926 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.922+0000 7f4937fff640 1 --2- 172.21.15.80:0/2564866994 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49381022c0 0x7f4938197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:26.926 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.922+0000 7f49377fe640 1 -- 172.21.15.80:0/2564866994 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49381022c0 msgr2=0x7f4938197ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:26.926 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.922+0000 7f49377fe640 1 --2- 172.21.15.80:0/2564866994 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49381022c0 0x7f4938197ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:26.926 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.922+0000 7f49377fe640 1 -- 172.21.15.80:0/2564866994 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4938107520 msgr2=0x7f493819f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:26.926 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.922+0000 7f49377fe640 1 --2- 172.21.15.80:0/2564866994 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4938107520 0x7f493819f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:26.926 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.922+0000 7f49377fe640 1 -- 172.21.15.80:0/2564866994 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f493819f6f0 con 0x7f4938104bf0 2024-04-24T23:27:26.926 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.923+0000 7f49377fe640 1 --2- 172.21.15.80:0/2564866994 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4938104bf0 0x7f4938198010 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f492c00c970 tx=0x7f492c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:26.926 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.923+0000 7f49357fa640 1 -- 172.21.15.80:0/2564866994 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f492c014070 con 0x7f4938104bf0 2024-04-24T23:27:26.926 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.923+0000 7f493e2cd640 1 -- 172.21.15.80:0/2564866994 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f493819f9e0 con 0x7f4938104bf0 2024-04-24T23:27:26.926 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.923+0000 7f49357fa640 1 -- 172.21.15.80:0/2564866994 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f492c004440 con 0x7f4938104bf0 2024-04-24T23:27:26.927 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.923+0000 7f49357fa640 1 -- 172.21.15.80:0/2564866994 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f492c00f450 con 0x7f4938104bf0 2024-04-24T23:27:26.927 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.923+0000 7f493e2cd640 1 -- 172.21.15.80:0/2564866994 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f493819ffb0 con 0x7f4938104bf0 2024-04-24T23:27:26.933 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.925+0000 7f49357fa640 1 -- 172.21.15.80:0/2564866994 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f492c017070 con 0x7f4938104bf0 2024-04-24T23:27:26.933 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.926+0000 7f4940fdc740 1 Processor -- start 2024-04-24T23:27:26.933 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.926+0000 7f4940fdc740 1 -- start start 2024-04-24T23:27:26.933 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.926+0000 7f4940fdc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb3e4a0440 0x55cb3e4a0860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:26.933 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.926+0000 7f4940fdc740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cb3e4a0e30 0x55cb3e4a5930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:26.933 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.926+0000 7f4940fdc740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cb3e4a5fa0 0x55cb3e4a8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:26.934 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.926+0000 7f4940fdc740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x55cb3e403f00 con 0x55cb3e4a0e30 2024-04-24T23:27:26.934 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.926+0000 7f4940fdc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55cb3e3ac750 con 0x55cb3e4a0440 2024-04-24T23:27:26.934 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.926+0000 7f4940fdc740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55cb3e3ffec0 con 0x55cb3e4a5fa0 2024-04-24T23:27:26.934 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.926+0000 7f4937fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cb3e4a0e30 0x55cb3e4a5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:26.934 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.926+0000 7f4937fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cb3e4a0e30 0x55cb3e4a5930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:35702/0 (socket says 172.21.15.80:35702) 2024-04-24T23:27:26.934 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.926+0000 7f4937fff640 1 -- 172.21.15.80:0/4252146798 learned_addr learned my addr 172.21.15.80:0/4252146798 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:27:26.934 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.926+0000 7f49357fa640 1 --2- 172.21.15.80:0/2564866994 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f49200a2b90 0x7f49200a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:26.934 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.926+0000 7f493c86a640 1 --2- 172.21.15.80:0/2564866994 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f49200a2b90 0x7f49200a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:26.934 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.926+0000 7f493c86a640 1 --2- 172.21.15.80:0/4252146798 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb3e4a0440 0x55cb3e4a0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:26.934 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.926+0000 7f4937fff640 1 -- 172.21.15.80:0/4252146798 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cb3e4a5fa0 msgr2=0x55cb3e4a8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:26.934 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.926+0000 7f49357fa640 1 -- 172.21.15.80:0/2564866994 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f492c0cf0c0 con 0x7f4938104bf0 2024-04-24T23:27:26.934 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.926+0000 7f49377fe640 1 --2- 172.21.15.80:0/4252146798 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cb3e4a5fa0 0x55cb3e4a8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:26.934 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.926+0000 7f4937fff640 1 --2- 172.21.15.80:0/4252146798 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cb3e4a5fa0 0x55cb3e4a8430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:26.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.926+0000 7f4937fff640 1 -- 172.21.15.80:0/4252146798 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb3e4a0440 msgr2=0x55cb3e4a0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:26.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.926+0000 7f4937fff640 1 --2- 172.21.15.80:0/4252146798 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb3e4a0440 0x55cb3e4a0860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:26.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.926+0000 7f4937fff640 1 -- 172.21.15.80:0/4252146798 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb3e1182d0 con 0x55cb3e4a0e30 2024-04-24T23:27:26.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.926+0000 7f49377fe640 1 --2- 172.21.15.80:0/4252146798 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cb3e4a5fa0 0x55cb3e4a8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:27:26.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.927+0000 7f4937fff640 1 --2- 172.21.15.80:0/4252146798 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cb3e4a0e30 0x55cb3e4a5930 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f4928031610 tx=0x7f4928005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50bf63c1476033dc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:26.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.927+0000 7f493c86a640 1 --2- 172.21.15.80:0/2564866994 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f49200a2b90 0x7f49200a5050 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x55cb3e4a1690 tx=0x7f493000b250 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:26.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.927+0000 7f491e7fc640 1 -- 172.21.15.80:0/4252146798 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4928040070 con 0x55cb3e4a0e30 2024-04-24T23:27:26.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.927+0000 7f491e7fc640 1 -- 172.21.15.80:0/4252146798 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4928005500 con 0x55cb3e4a0e30 2024-04-24T23:27:26.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.927+0000 7f491e7fc640 1 -- 172.21.15.80:0/4252146798 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4928035040 con 0x55cb3e4a0e30 2024-04-24T23:27:26.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.927+0000 7f4940fdc740 1 -- 172.21.15.80:0/4252146798 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cb3e4a0e30 msgr2=0x55cb3e4a5930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:26.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.927+0000 7f4940fdc740 1 --2- 172.21.15.80:0/4252146798 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cb3e4a0e30 0x55cb3e4a5930 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f4928031610 tx=0x7f4928005c00 comp rx=0 tx=0).stop 2024-04-24T23:27:26.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.927+0000 7f4940fdc740 1 -- 172.21.15.80:0/4252146798 shutdown_connections 2024-04-24T23:27:26.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.927+0000 7f4940fdc740 1 --2- 172.21.15.80:0/4252146798 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cb3e4a5fa0 0x55cb3e4a8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:26.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.927+0000 7f4940fdc740 1 --2- 172.21.15.80:0/4252146798 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cb3e4a0e30 0x55cb3e4a5930 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:26.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.927+0000 7f4940fdc740 1 --2- 172.21.15.80:0/4252146798 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb3e4a0440 0x55cb3e4a0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:26.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.927+0000 7f4940fdc740 1 -- 172.21.15.80:0/4252146798 >> 172.21.15.80:0/4252146798 conn(0x55cb3e49bab0 msgr2=0x55cb3e49beb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:26.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.927+0000 7f4940fdc740 1 -- 172.21.15.80:0/4252146798 shutdown_connections 2024-04-24T23:27:26.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.927+0000 7f4940fdc740 1 -- 172.21.15.80:0/4252146798 wait complete. 2024-04-24T23:27:26.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.928+0000 7f4940fdc740 1 -- start start 2024-04-24T23:27:26.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.928+0000 7f4940fdc740 1 Processor -- start 2024-04-24T23:27:26.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.928+0000 7f4940fdc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb3e4a0440 0x55cb3e4aa7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:26.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.928+0000 7f4940fdc740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cb3e4a0e30 0x55cb3e4aad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:26.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.928+0000 7f4940fdc740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cb3e4a5fa0 0x55cb3e4b1d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:26.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.928+0000 7f4940fdc740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x55cb3e4a8ef0 con 0x55cb3e4a5fa0 2024-04-24T23:27:26.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.928+0000 7f4940fdc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55cb3e3bfcf0 con 0x55cb3e4a0440 2024-04-24T23:27:26.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.928+0000 7f4940fdc740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55cb3e4a9070 con 0x55cb3e4a0e30 2024-04-24T23:27:26.937 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.928+0000 7f4940fdc740 10 client.0 _ll_register_callbacks cb 0x55cb3e3bf6e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:27:26.937 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.928+0000 7f493c86a640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cb3e4a5fa0 0x55cb3e4b1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:26.937 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.928+0000 7f493c86a640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cb3e4a5fa0 0x55cb3e4b1d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:35706/0 (socket says 172.21.15.80:35706) 2024-04-24T23:27:26.937 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.928+0000 7f493c86a640 1 -- 172.21.15.80:0/2220780739 learned_addr learned my addr 172.21.15.80:0/2220780739 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:27:26.937 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.928+0000 7f493c86a640 1 -- 172.21.15.80:0/2220780739 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cb3e4a0e30 msgr2=0x55cb3e4aad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:26.937 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.928+0000 7f4937fff640 1 --2- 172.21.15.80:0/2220780739 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb3e4a0440 0x55cb3e4aa7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:26.937 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.928+0000 7f49377fe640 1 --2- 172.21.15.80:0/2220780739 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cb3e4a0e30 0x55cb3e4aad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:26.937 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.928+0000 7f493c86a640 1 --2- 172.21.15.80:0/2220780739 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cb3e4a0e30 0x55cb3e4aad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:26.937 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.928+0000 7f493c86a640 1 -- 172.21.15.80:0/2220780739 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb3e4a0440 msgr2=0x55cb3e4aa7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:26.937 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.928+0000 7f493c86a640 1 --2- 172.21.15.80:0/2220780739 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb3e4a0440 0x55cb3e4aa7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:26.937 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.929+0000 7f493c86a640 1 -- 172.21.15.80:0/2220780739 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb3e1e26b0 con 0x55cb3e4a5fa0 2024-04-24T23:27:26.937 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.929+0000 7f493c86a640 1 --2- 172.21.15.80:0/2220780739 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cb3e4a5fa0 0x55cb3e4b1d50 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f4930006040 tx=0x7f4930004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3e593237aaa787b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:26.937 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.929+0000 7f49177fe640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-04-24T23:27:26.937 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.929+0000 7f49177fe640 1 -- 172.21.15.80:0/2220780739 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f493000eb90 con 0x55cb3e4a5fa0 2024-04-24T23:27:26.937 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.929+0000 7f49177fe640 1 -- 172.21.15.80:0/2220780739 <== mon.0 v2:172.21.15.80:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f493000ed30 con 0x55cb3e4a5fa0 2024-04-24T23:27:26.938 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.929+0000 7f4940fdc740 1 -- 172.21.15.80:0/2220780739 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4928005500 con 0x55cb3e4a5fa0 2024-04-24T23:27:26.938 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.929+0000 7f49177fe640 1 -- 172.21.15.80:0/2220780739 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f4930029430 con 0x55cb3e4a5fa0 2024-04-24T23:27:26.938 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.929+0000 7f4940fdc740 10 client.14781 fetch_fsmap learned FSMap version 17 2024-04-24T23:27:26.938 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.929+0000 7f4940fdc740 1 -- 172.21.15.80:0/2220780739 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x55cb3e4b4d10 con 0x55cb3e4a5fa0 2024-04-24T23:27:26.938 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.929+0000 7f49177fe640 1 -- 172.21.15.80:0/2220780739 <== mon.0 v2:172.21.15.80:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4928005500 con 0x55cb3e4a5fa0 2024-04-24T23:27:26.938 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.929+0000 7f49177fe640 1 -- 172.21.15.80:0/2220780739 <== mon.0 v2:172.21.15.80:3300/0 5 ==== fsmap(e 17) ==== 8342+0+0 (secure 0 0 0) 0x7f4930022570 con 0x55cb3e4a5fa0 2024-04-24T23:27:26.938 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.929+0000 7f4940fdc740 10 client.14781 fetch_fsmap finished waiting for FSMap version 17 2024-04-24T23:27:26.938 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.929+0000 7f4940fdc740 10 client.14781 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-24T23:27:26.938 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.929+0000 7f4940fdc740 20 client.14781 populate_metadata read hostname 'smithi080' 2024-04-24T23:27:26.938 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.929+0000 7f4940fdc740 1 --2- 172.21.15.80:0/2220780739 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x55cb3e4b7490 0x55cb3e4b8900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:26.938 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.929+0000 7f4940fdc740 4 client.14781 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:27:26.938 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.929+0000 7f4940fdc740 1 -- 172.21.15.80:0/2220780739 --> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cb3e411440 con 0x55cb3e4b7490 2024-04-24T23:27:26.938 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.930+0000 7f4937fff640 1 --2- 172.21.15.80:0/2220780739 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x55cb3e4b7490 0x55cb3e4b8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:26.939 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.930+0000 7f4937fff640 1 --2- 172.21.15.80:0/2220780739 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x55cb3e4b7490 0x55cb3e4b8900 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2706d43446909e5c server_cookie=cf24899b92fd7cd in_seq=0 out_seq=0 2024-04-24T23:27:26.939 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.930+0000 7f49177fe640 10 client.14781 ms_handle_connect on v2:172.21.15.148:6838/2952757594 2024-04-24T23:27:26.939 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.931+0000 7f49177fe640 1 -- 172.21.15.80:0/2220780739 <== mds.0 v2:172.21.15.148:6838/2952757594 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cb3e411440 con 0x55cb3e4b7490 2024-04-24T23:27:26.939 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.931+0000 7f49177fe640 10 client.14781 handle_command_reply: tid=0 2024-04-24T23:27:26.939 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.931+0000 7f4940fdc740 1 client.14781 shutdown 2024-04-24T23:27:26.939 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.931+0000 7f4940fdc740 1 -- 172.21.15.80:0/2220780739 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cb3e4a5fa0 msgr2=0x55cb3e4b1d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:26.939 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.931+0000 7f4940fdc740 1 --2- 172.21.15.80:0/2220780739 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cb3e4a5fa0 0x55cb3e4b1d50 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f4930006040 tx=0x7f4930004b50 comp rx=0 tx=0).stop 2024-04-24T23:27:26.939 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.931+0000 7f4940fdc740 1 -- 172.21.15.80:0/2220780739 shutdown_connections 2024-04-24T23:27:26.939 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.931+0000 7f4940fdc740 1 --2- 172.21.15.80:0/2220780739 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x55cb3e4b7490 0x55cb3e4b8900 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:26.939 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.931+0000 7f4940fdc740 1 --2- 172.21.15.80:0/2220780739 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cb3e4a5fa0 0x55cb3e4b1d50 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:26.939 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.931+0000 7f4940fdc740 1 --2- 172.21.15.80:0/2220780739 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cb3e4a0e30 0x55cb3e4aad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:26.939 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.931+0000 7f49177fe640 0 client.14781 ms_handle_reset on v2:172.21.15.148:6838/2952757594 2024-04-24T23:27:26.939 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.931+0000 7f4940fdc740 1 --2- 172.21.15.80:0/2220780739 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb3e4a0440 0x55cb3e4aa7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:26.939 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.931+0000 7f4940fdc740 1 -- 172.21.15.80:0/2220780739 >> 172.21.15.80:0/2220780739 conn(0x55cb3e49bab0 msgr2=0x55cb3e49aae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:26.940 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.931+0000 7f4940fdc740 1 -- 172.21.15.80:0/2220780739 shutdown_connections 2024-04-24T23:27:26.940 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.931+0000 7f4940fdc740 1 -- 172.21.15.80:0/2220780739 wait complete. 2024-04-24T23:27:26.940 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.931+0000 7f4940fdc740 20 client.14781 trim_cache size 0 max 16384 2024-04-24T23:27:26.955 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.949+0000 7f4940fdc740 1 Processor -- start 2024-04-24T23:27:26.955 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.949+0000 7f4940fdc740 1 -- start start 2024-04-24T23:27:26.955 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.949+0000 7f4940fdc740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cb3e4a0440 0x55cb3e4aa7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:26.955 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.949+0000 7f4940fdc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb3e4a0e30 0x55cb3e4aad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:26.955 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.949+0000 7f4940fdc740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cb3e4a5fa0 0x55cb3e4b1c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:26.956 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.949+0000 7f4940fdc740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x55cb3e4b6c40 con 0x55cb3e4a0440 2024-04-24T23:27:26.956 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.949+0000 7f4940fdc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55cb3e4b7310 con 0x55cb3e4a0e30 2024-04-24T23:27:26.956 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.949+0000 7f4940fdc740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55cb3e4b79e0 con 0x55cb3e4a5fa0 2024-04-24T23:27:26.956 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.949+0000 7f4937fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cb3e4a0440 0x55cb3e4aa7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:26.956 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.949+0000 7f4937fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cb3e4a0440 0x55cb3e4aa7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:35708/0 (socket says 172.21.15.80:35708) 2024-04-24T23:27:26.956 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.949+0000 7f4937fff640 1 -- 172.21.15.80:0/3231877959 learned_addr learned my addr 172.21.15.80:0/3231877959 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:27:26.956 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.949+0000 7f4937fff640 1 -- 172.21.15.80:0/3231877959 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cb3e4a5fa0 msgr2=0x55cb3e4b1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:26.956 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.949+0000 7f4937fff640 1 --2- 172.21.15.80:0/3231877959 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cb3e4a5fa0 0x55cb3e4b1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:26.956 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.949+0000 7f4937fff640 1 -- 172.21.15.80:0/3231877959 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb3e4a0e30 msgr2=0x55cb3e4aad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:26.956 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.949+0000 7f49377fe640 1 --2- 172.21.15.80:0/3231877959 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb3e4a0e30 0x55cb3e4aad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:26.956 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.949+0000 7f4937fff640 1 --2- 172.21.15.80:0/3231877959 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb3e4a0e30 0x55cb3e4aad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:26.956 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.949+0000 7f4937fff640 1 -- 172.21.15.80:0/3231877959 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb3e4a9230 con 0x55cb3e4a0440 2024-04-24T23:27:26.956 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.949+0000 7f49377fe640 1 --2- 172.21.15.80:0/3231877959 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb3e4a0e30 0x55cb3e4aad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:27:26.957 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.950+0000 7f4937fff640 1 --2- 172.21.15.80:0/3231877959 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cb3e4a0440 0x55cb3e4aa7d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f49280314e0 tx=0x7f492802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b223d567d492abfb server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:26.957 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.950+0000 7f491dffb640 1 -- 172.21.15.80:0/3231877959 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4928040040 con 0x55cb3e4a0440 2024-04-24T23:27:26.957 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.950+0000 7f491dffb640 1 -- 172.21.15.80:0/3231877959 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4928004240 con 0x55cb3e4a0440 2024-04-24T23:27:26.957 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.950+0000 7f491dffb640 1 -- 172.21.15.80:0/3231877959 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4928035040 con 0x55cb3e4a0440 2024-04-24T23:27:26.957 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.950+0000 7f4940fdc740 1 -- 172.21.15.80:0/3231877959 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cb3e4a0440 msgr2=0x55cb3e4aa7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:26.957 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.950+0000 7f4940fdc740 1 --2- 172.21.15.80:0/3231877959 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cb3e4a0440 0x55cb3e4aa7d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f49280314e0 tx=0x7f492802a410 comp rx=0 tx=0).stop 2024-04-24T23:27:26.957 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.950+0000 7f4940fdc740 1 -- 172.21.15.80:0/3231877959 shutdown_connections 2024-04-24T23:27:26.957 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.950+0000 7f4940fdc740 1 --2- 172.21.15.80:0/3231877959 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cb3e4a5fa0 0x55cb3e4b1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:26.957 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.950+0000 7f4940fdc740 1 --2- 172.21.15.80:0/3231877959 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb3e4a0e30 0x55cb3e4aad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:26.957 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.950+0000 7f4940fdc740 1 --2- 172.21.15.80:0/3231877959 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cb3e4a0440 0x55cb3e4aa7d0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:26.957 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.950+0000 7f4940fdc740 1 -- 172.21.15.80:0/3231877959 >> 172.21.15.80:0/3231877959 conn(0x55cb3e49bab0 msgr2=0x55cb3e496690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:26.957 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.950+0000 7f4940fdc740 1 -- 172.21.15.80:0/3231877959 shutdown_connections 2024-04-24T23:27:26.957 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.950+0000 7f4940fdc740 1 -- 172.21.15.80:0/3231877959 wait complete. 2024-04-24T23:27:26.957 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.951+0000 7f4940fdc740 1 -- start start 2024-04-24T23:27:26.957 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.951+0000 7f4940fdc740 1 Processor -- start 2024-04-24T23:27:26.958 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.951+0000 7f4940fdc740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cb3e4a0440 0x55cb3e4ad2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:26.958 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.951+0000 7f4940fdc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb3e4a0e30 0x55cb3e4ad830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:26.958 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.951+0000 7f4940fdc740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cb3e4a5fa0 0x55cb3e4b07f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:26.958 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.951+0000 7f4940fdc740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x55cb3e4b8780 con 0x55cb3e4a0440 2024-04-24T23:27:26.958 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.951+0000 7f4940fdc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55cb3e4b80b0 con 0x55cb3e4a0e30 2024-04-24T23:27:26.958 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.951+0000 7f4940fdc740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55cb3e4b8e50 con 0x55cb3e4a5fa0 2024-04-24T23:27:26.958 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.951+0000 7f4940fdc740 10 client.0 _ll_register_callbacks cb 0x7f4930006750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:27:26.958 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.951+0000 7f4937fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cb3e4a0440 0x55cb3e4ad2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:26.958 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.951+0000 7f4937fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cb3e4a0440 0x55cb3e4ad2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:35724/0 (socket says 172.21.15.80:35724) 2024-04-24T23:27:26.958 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.951+0000 7f4937fff640 1 -- 172.21.15.80:0/2611306042 learned_addr learned my addr 172.21.15.80:0/2611306042 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:27:26.958 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.951+0000 7f49377fe640 1 --2- 172.21.15.80:0/2611306042 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb3e4a0e30 0x55cb3e4ad830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:26.958 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.951+0000 7f493c86a640 1 --2- 172.21.15.80:0/2611306042 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cb3e4a5fa0 0x55cb3e4b07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:26.958 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.951+0000 7f4937fff640 1 -- 172.21.15.80:0/2611306042 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cb3e4a5fa0 msgr2=0x55cb3e4b07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:26.959 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.951+0000 7f4937fff640 1 --2- 172.21.15.80:0/2611306042 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cb3e4a5fa0 0x55cb3e4b07f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:26.959 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.951+0000 7f4937fff640 1 -- 172.21.15.80:0/2611306042 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb3e4a0e30 msgr2=0x55cb3e4ad830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:27:26.959 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.951+0000 7f4937fff640 1 --2- 172.21.15.80:0/2611306042 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb3e4a0e30 0x55cb3e4ad830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:26.959 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.951+0000 7f4937fff640 1 -- 172.21.15.80:0/2611306042 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb3e4b0de0 con 0x55cb3e4a0440 2024-04-24T23:27:26.959 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.951+0000 7f49377fe640 1 --2- 172.21.15.80:0/2611306042 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb3e4a0e30 0x55cb3e4ad830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:27:26.959 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.951+0000 7f493c86a640 1 --2- 172.21.15.80:0/2611306042 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cb3e4a5fa0 0x55cb3e4b07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:27:26.959 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.952+0000 7f4937fff640 1 --2- 172.21.15.80:0/2611306042 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cb3e4a0440 0x55cb3e4ad2f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4928031bf0 tx=0x7f4928003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec996289e1c6deb8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:26.959 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.952+0000 7f49177fe640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-04-24T23:27:26.959 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.952+0000 7f49177fe640 1 -- 172.21.15.80:0/2611306042 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f492805d060 con 0x55cb3e4a0440 2024-04-24T23:27:26.959 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.952+0000 7f49177fe640 1 -- 172.21.15.80:0/2611306042 <== mon.0 v2:172.21.15.80:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f49280037f0 con 0x55cb3e4a0440 2024-04-24T23:27:26.959 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.952+0000 7f4940fdc740 1 -- 172.21.15.80:0/2611306042 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4928004240 con 0x55cb3e4a0440 2024-04-24T23:27:26.959 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.952+0000 7f49177fe640 1 -- 172.21.15.80:0/2611306042 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f4928035050 con 0x55cb3e4a0440 2024-04-24T23:27:26.959 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.952+0000 7f4940fdc740 10 client.14787 fetch_fsmap learned FSMap version 17 2024-04-24T23:27:26.960 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.952+0000 7f4940fdc740 1 -- 172.21.15.80:0/2611306042 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x55cb3e4b2050 con 0x55cb3e4a0440 2024-04-24T23:27:26.960 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.952+0000 7f49177fe640 1 -- 172.21.15.80:0/2611306042 <== mon.0 v2:172.21.15.80:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4928004240 con 0x55cb3e4a0440 2024-04-24T23:27:26.960 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.952+0000 7f49177fe640 1 -- 172.21.15.80:0/2611306042 <== mon.0 v2:172.21.15.80:3300/0 5 ==== fsmap(e 17) ==== 8342+0+0 (secure 0 0 0) 0x7f49280584e0 con 0x55cb3e4a0440 2024-04-24T23:27:26.960 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.952+0000 7f4940fdc740 10 client.14787 fetch_fsmap finished waiting for FSMap version 17 2024-04-24T23:27:26.960 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.952+0000 7f4940fdc740 10 client.14787 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-24T23:27:26.960 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.952+0000 7f4940fdc740 20 client.14787 populate_metadata read hostname 'smithi080' 2024-04-24T23:27:26.960 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.952+0000 7f4940fdc740 1 --2- 172.21.15.80:0/2611306042 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x55cb3e521750 0x55cb3e520460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:26.960 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.952+0000 7f4940fdc740 4 client.14787 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T23:27:26.960 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.952+0000 7f4940fdc740 1 -- 172.21.15.80:0/2611306042 --> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] -- command(tid 0: {"prefix": "status"}) -- 0x55cb3e40b790 con 0x55cb3e521750 2024-04-24T23:27:26.960 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.952+0000 7f49377fe640 1 --2- 172.21.15.80:0/2611306042 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x55cb3e521750 0x55cb3e520460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:26.960 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.953+0000 7f49177fe640 10 client.14787 ms_handle_connect on v2:172.21.15.148:6838/2952757594 2024-04-24T23:27:26.960 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.953+0000 7f49377fe640 1 --2- 172.21.15.80:0/2611306042 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x55cb3e521750 0x55cb3e520460 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec7aede1c346ec63 server_cookie=f2054cd5abdd409a in_seq=0 out_seq=0 2024-04-24T23:27:26.960 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.953+0000 7f49177fe640 1 -- 172.21.15.80:0/2611306042 <== mds.0 v2:172.21.15.148:6838/2952757594 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55cb3e40b790 con 0x55cb3e521750 2024-04-24T23:27:26.961 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.953+0000 7f49177fe640 10 client.14787 handle_command_reply: tid=0 2024-04-24T23:27:26.961 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.953+0000 7f4940fdc740 1 client.14787 shutdown 2024-04-24T23:27:26.961 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.953+0000 7f4940fdc740 1 -- 172.21.15.80:0/2611306042 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cb3e4a0440 msgr2=0x55cb3e4ad2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:26.961 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.953+0000 7f4940fdc740 1 --2- 172.21.15.80:0/2611306042 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cb3e4a0440 0x55cb3e4ad2f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4928031bf0 tx=0x7f4928003f10 comp rx=0 tx=0).stop 2024-04-24T23:27:26.961 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.953+0000 7f4940fdc740 1 -- 172.21.15.80:0/2611306042 shutdown_connections 2024-04-24T23:27:26.961 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.953+0000 7f4940fdc740 1 --2- 172.21.15.80:0/2611306042 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x55cb3e521750 0x55cb3e520460 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:26.961 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.953+0000 7f4940fdc740 1 --2- 172.21.15.80:0/2611306042 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cb3e4a5fa0 0x55cb3e4b07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:26.961 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.953+0000 7f4940fdc740 1 --2- 172.21.15.80:0/2611306042 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb3e4a0e30 0x55cb3e4ad830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:26.961 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.953+0000 7f4940fdc740 1 --2- 172.21.15.80:0/2611306042 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cb3e4a0440 0x55cb3e4ad2f0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:26.961 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.953+0000 7f49177fe640 0 client.14787 ms_handle_reset on v2:172.21.15.148:6838/2952757594 2024-04-24T23:27:26.961 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.953+0000 7f4940fdc740 1 -- 172.21.15.80:0/2611306042 >> 172.21.15.80:0/2611306042 conn(0x55cb3e49bab0 msgr2=0x55cb3e4a2150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:26.961 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.953+0000 7f4940fdc740 1 -- 172.21.15.80:0/2611306042 shutdown_connections 2024-04-24T23:27:26.961 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.953+0000 7f4940fdc740 1 -- 172.21.15.80:0/2611306042 wait complete. 2024-04-24T23:27:26.962 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.953+0000 7f4940fdc740 20 client.14787 trim_cache size 0 max 16384 2024-04-24T23:27:26.962 INFO:teuthology.orchestra.run.smithi080.stdout:{ 2024-04-24T23:27:26.962 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_fsid": "ec7885aa-0290-11ef-bc93-c7b262605968", 2024-04-24T23:27:26.962 INFO:teuthology.orchestra.run.smithi080.stdout: "whoami": 0, 2024-04-24T23:27:26.962 INFO:teuthology.orchestra.run.smithi080.stdout: "id": 24496, 2024-04-24T23:27:26.962 INFO:teuthology.orchestra.run.smithi080.stdout: "want_state": "up:active", 2024-04-24T23:27:26.962 INFO:teuthology.orchestra.run.smithi080.stdout: "state": "up:active", 2024-04-24T23:27:26.962 INFO:teuthology.orchestra.run.smithi080.stdout: "fs_name": "cephfs", 2024-04-24T23:27:26.962 INFO:teuthology.orchestra.run.smithi080.stdout: "rank_uptime": 6.7666150680000001, 2024-04-24T23:27:26.962 INFO:teuthology.orchestra.run.smithi080.stdout: "mdsmap_epoch": 17, 2024-04-24T23:27:26.963 INFO:teuthology.orchestra.run.smithi080.stdout: "osdmap_epoch": 90, 2024-04-24T23:27:26.963 INFO:teuthology.orchestra.run.smithi080.stdout: "osdmap_epoch_barrier": 89, 2024-04-24T23:27:26.963 INFO:teuthology.orchestra.run.smithi080.stdout: "uptime": 22.359479224000001 2024-04-24T23:27:26.963 INFO:teuthology.orchestra.run.smithi080.stdout:} 2024-04-24T23:27:26.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.954+0000 7f493e2cd640 1 -- 172.21.15.80:0/2564866994 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f49200a2b90 msgr2=0x7f49200a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:26.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.954+0000 7f493e2cd640 1 --2- 172.21.15.80:0/2564866994 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f49200a2b90 0x7f49200a5050 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x55cb3e4a1690 tx=0x7f493000b250 comp rx=0 tx=0).stop 2024-04-24T23:27:26.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.954+0000 7f493e2cd640 1 -- 172.21.15.80:0/2564866994 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4938104bf0 msgr2=0x7f4938198010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:26.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.954+0000 7f493e2cd640 1 --2- 172.21.15.80:0/2564866994 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4938104bf0 0x7f4938198010 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f492c00c970 tx=0x7f492c00ce30 comp rx=0 tx=0).stop 2024-04-24T23:27:26.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.954+0000 7f493e2cd640 1 -- 172.21.15.80:0/2564866994 shutdown_connections 2024-04-24T23:27:26.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.954+0000 7f493e2cd640 1 --2- 172.21.15.80:0/2564866994 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f49200a2b90 0x7f49200a5050 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:26.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.954+0000 7f493e2cd640 1 --2- 172.21.15.80:0/2564866994 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4938107520 0x7f493819f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:26.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.954+0000 7f493e2cd640 1 --2- 172.21.15.80:0/2564866994 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4938104bf0 0x7f4938198010 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:26.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.954+0000 7f493e2cd640 1 --2- 172.21.15.80:0/2564866994 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49381022c0 0x7f4938197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:26.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.954+0000 7f493e2cd640 1 -- 172.21.15.80:0/2564866994 >> 172.21.15.80:0/2564866994 conn(0x7f49380fbc30 msgr2=0x7f4938102c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:26.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.954+0000 7f493e2cd640 1 -- 172.21.15.80:0/2564866994 shutdown_connections 2024-04-24T23:27:26.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:26.954+0000 7f493e2cd640 1 -- 172.21.15.80:0/2564866994 wait complete. 2024-04-24T23:27:26.966 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-24T23:27:26.966 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-24T23:27:26.966 DEBUG:teuthology.orchestra.run.smithi080:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid ec7885aa-0290-11ef-bc93-c7b262605968 -- ceph fs ls 2024-04-24T23:27:27.200 INFO:teuthology.orchestra.run.smithi080.stderr:Inferring config /var/lib/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:27:27.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:27 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/396779641' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:27:27.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:27 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/1340170933' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:27:27.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:27 smithi148 ceph-mon[32528]: pgmap v300: 129 pgs: 105 active+clean, 2 creating+activating, 18 creating+peering, 4 unknown; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 6 op/s 2024-04-24T23:27:27.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:27 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/396779641' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:27:27.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:27 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/1340170933' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:27:27.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:27 smithi080 ceph-mon[29152]: pgmap v300: 129 pgs: 105 active+clean, 2 creating+activating, 18 creating+peering, 4 unknown; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 6 op/s 2024-04-24T23:27:27.697 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:27 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/396779641' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:27:27.697 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:27 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/1340170933' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:27:27.697 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:27 smithi134 ceph-mon[32619]: pgmap v300: 129 pgs: 105 active+clean, 2 creating+activating, 18 creating+peering, 4 unknown; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 6 op/s 2024-04-24T23:27:28.470 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:28 smithi080 ceph-mon[29152]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-24T23:27:28.471 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:28 smithi080 ceph-mon[29152]: Cluster is now healthy 2024-04-24T23:27:28.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:28 smithi148 ceph-mon[32528]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-24T23:27:28.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:28 smithi148 ceph-mon[32528]: Cluster is now healthy 2024-04-24T23:27:28.697 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:28 smithi134 ceph-mon[32619]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-24T23:27:28.697 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:28 smithi134 ceph-mon[32619]: Cluster is now healthy 2024-04-24T23:27:29.021 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:29.016+0000 7f5285f43700 1 -- 172.21.15.80:0/2049970837 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f528006a8e0 msgr2=0x7f528010f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:29.021 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:29.016+0000 7f5285f43700 1 --2- 172.21.15.80:0/2049970837 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f528006a8e0 0x7f528010f0f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f527800b0a0 tx=0x7f527800b3b0 comp rx=0 tx=0).stop 2024-04-24T23:27:29.021 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:29.016+0000 7f5285f43700 1 -- 172.21.15.80:0/2049970837 shutdown_connections 2024-04-24T23:27:29.021 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:29.016+0000 7f5285f43700 1 --2- 172.21.15.80:0/2049970837 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f528006a8e0 0x7f528010f0f0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:29.021 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:29.016+0000 7f5285f43700 1 --2- 172.21.15.80:0/2049970837 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52801082b0 0x7f5280108690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:29.021 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:29.016+0000 7f5285f43700 1 --2- 172.21.15.80:0/2049970837 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5280069f40 0x7f528006a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:29.021 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:29.016+0000 7f5285f43700 1 -- 172.21.15.80:0/2049970837 >> 172.21.15.80:0/2049970837 conn(0x7f52800780e0 msgr2=0x7f52800784f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:29.022 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:29.016+0000 7f5285f43700 1 -- 172.21.15.80:0/2049970837 shutdown_connections 2024-04-24T23:27:29.022 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:29.016+0000 7f5285f43700 1 -- 172.21.15.80:0/2049970837 wait complete. 2024-04-24T23:27:29.022 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:29.017+0000 7f5285f43700 1 Processor -- start 2024-04-24T23:27:29.022 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:29.017+0000 7f5285f43700 1 -- start start 2024-04-24T23:27:29.022 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:29.017+0000 7f5285f43700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5280069f40 0x7f528019e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:29.022 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:29.017+0000 7f5285f43700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f528006a8e0 0x7f528019ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:29.022 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:29.017+0000 7f5285f43700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52801082b0 0x7f5280198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:29.022 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:29.017+0000 7f5285f43700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f52801125b0 con 0x7f528006a8e0 2024-04-24T23:27:29.022 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:29.017+0000 7f5285f43700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5280112430 con 0x7f5280069f40 2024-04-24T23:27:29.022 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:29.017+0000 7f5285f43700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5280112730 con 0x7f52801082b0 2024-04-24T23:27:29.022 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:29.017+0000 7f5276ffd700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f528006a8e0 0x7f528019ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:29.022 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:29.017+0000 7f5276ffd700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f528006a8e0 0x7f528019ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:35746/0 (socket says 172.21.15.80:35746) 2024-04-24T23:27:29.022 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:29.017+0000 7f5276ffd700 1 -- 172.21.15.80:0/3157119658 learned_addr learned my addr 172.21.15.80:0/3157119658 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:27:29.023 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:29.017+0000 7f527ffff700 1 --2- 172.21.15.80:0/3157119658 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52801082b0 0x7f5280198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:29.023 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:29.017+0000 7f527f7fe700 1 --2- 172.21.15.80:0/3157119658 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5280069f40 0x7f528019e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:29.023 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:29.018+0000 7f5276ffd700 1 -- 172.21.15.80:0/3157119658 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52801082b0 msgr2=0x7f5280198870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:29.023 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:29.018+0000 7f5276ffd700 1 --2- 172.21.15.80:0/3157119658 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52801082b0 0x7f5280198870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:29.023 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:29.018+0000 7f5276ffd700 1 -- 172.21.15.80:0/3157119658 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5280069f40 msgr2=0x7f528019e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:29.023 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:29.018+0000 7f5276ffd700 1 --2- 172.21.15.80:0/3157119658 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5280069f40 0x7f528019e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:29.023 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:29.018+0000 7f5276ffd700 1 -- 172.21.15.80:0/3157119658 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5280198fd0 con 0x7f528006a8e0 2024-04-24T23:27:29.023 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:29.018+0000 7f5276ffd700 1 --2- 172.21.15.80:0/3157119658 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f528006a8e0 0x7f528019ebf0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f526800b530 tx=0x7f526800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:29.023 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:29.018+0000 7f527d7fa700 1 -- 172.21.15.80:0/3157119658 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5268015020 con 0x7f528006a8e0 2024-04-24T23:27:29.023 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:29.018+0000 7f5285f43700 1 -- 172.21.15.80:0/3157119658 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52801992c0 con 0x7f528006a8e0 2024-04-24T23:27:29.023 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:29.018+0000 7f527d7fa700 1 -- 172.21.15.80:0/3157119658 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5268011d10 con 0x7f528006a8e0 2024-04-24T23:27:29.023 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:29.018+0000 7f527d7fa700 1 -- 172.21.15.80:0/3157119658 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f526800f2e0 con 0x7f528006a8e0 2024-04-24T23:27:29.024 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:29.018+0000 7f5285f43700 1 -- 172.21.15.80:0/3157119658 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52801a5580 con 0x7f528006a8e0 2024-04-24T23:27:29.024 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:29.019+0000 7f5285f43700 1 -- 172.21.15.80:0/3157119658 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5280068f20 con 0x7f528006a8e0 2024-04-24T23:27:29.024 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:29.020+0000 7f527d7fa700 1 -- 172.21.15.80:0/3157119658 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f5268018020 con 0x7f528006a8e0 2024-04-24T23:27:29.025 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:29.021+0000 7f527d7fa700 1 --2- 172.21.15.80:0/3157119658 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f526c0a2c20 0x7f526c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:29.025 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:29.021+0000 7f527f7fe700 1 --2- 172.21.15.80:0/3157119658 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f526c0a2c20 0x7f526c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:29.025 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:29.021+0000 7f527d7fa700 1 -- 172.21.15.80:0/3157119658 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f52680cef40 con 0x7f528006a8e0 2024-04-24T23:27:29.025 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:29.022+0000 7f527f7fe700 1 --2- 172.21.15.80:0/3157119658 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f526c0a2c20 0x7f526c0a50e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f5270005950 tx=0x7f527000a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:29.026 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:29.025+0000 7f527d7fa700 1 -- 172.21.15.80:0/3157119658 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f526801f020 con 0x7f528006a8e0 2024-04-24T23:27:29.268 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:29.267+0000 7f5285f43700 1 -- 172.21.15.80:0/3157119658 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f5280199da0 con 0x7f528006a8e0 2024-04-24T23:27:29.268 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:29.267+0000 7f527d7fa700 1 -- 172.21.15.80:0/3157119658 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+88 (secure 0 0 0) 0x7f526809bc00 con 0x7f528006a8e0 2024-04-24T23:27:29.268 INFO:teuthology.orchestra.run.smithi080.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-24T23:27:29.270 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:29.269+0000 7f5285f43700 1 -- 172.21.15.80:0/3157119658 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f526c0a2c20 msgr2=0x7f526c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:29.273 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:29.269+0000 7f5285f43700 1 --2- 172.21.15.80:0/3157119658 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f526c0a2c20 0x7f526c0a50e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f5270005950 tx=0x7f527000a300 comp rx=0 tx=0).stop 2024-04-24T23:27:29.273 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:29.269+0000 7f5285f43700 1 -- 172.21.15.80:0/3157119658 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f528006a8e0 msgr2=0x7f528019ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:29.273 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:29.269+0000 7f5285f43700 1 --2- 172.21.15.80:0/3157119658 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f528006a8e0 0x7f528019ebf0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f526800b530 tx=0x7f526800b840 comp rx=0 tx=0).stop 2024-04-24T23:27:29.273 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:29.269+0000 7f5285f43700 1 -- 172.21.15.80:0/3157119658 shutdown_connections 2024-04-24T23:27:29.274 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:29.269+0000 7f5285f43700 1 --2- 172.21.15.80:0/3157119658 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52801082b0 0x7f5280198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:29.274 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:29.269+0000 7f5285f43700 1 --2- 172.21.15.80:0/3157119658 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5280069f40 0x7f528019e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:29.274 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:29.269+0000 7f5285f43700 1 --2- 172.21.15.80:0/3157119658 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f526c0a2c20 0x7f526c0a50e0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:29.274 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:29.269+0000 7f5285f43700 1 --2- 172.21.15.80:0/3157119658 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f528006a8e0 0x7f528019ebf0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:29.274 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:29.269+0000 7f5285f43700 1 -- 172.21.15.80:0/3157119658 >> 172.21.15.80:0/3157119658 conn(0x7f52800780e0 msgr2=0x7f52800ffc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:29.274 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:29.269+0000 7f5285f43700 1 -- 172.21.15.80:0/3157119658 shutdown_connections 2024-04-24T23:27:29.274 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:29.269+0000 7f5285f43700 1 -- 172.21.15.80:0/3157119658 wait complete. 2024-04-24T23:27:29.281 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:29 smithi080 ceph-mon[29152]: pgmap v301: 129 pgs: 119 active+clean, 2 creating+activating, 8 creating+peering; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 3 op/s 2024-04-24T23:27:29.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:29 smithi148 ceph-mon[32528]: pgmap v301: 129 pgs: 119 active+clean, 2 creating+activating, 8 creating+peering; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 3 op/s 2024-04-24T23:27:29.697 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:29 smithi134 ceph-mon[32619]: pgmap v301: 129 pgs: 119 active+clean, 2 creating+activating, 8 creating+peering; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 3 op/s 2024-04-24T23:27:30.066 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-24T23:27:30.066 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-24T23:27:30.066 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi080.front.sepia.ceph.com 2024-04-24T23:27:30.066 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-04-24T23:27:30.066 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-24T23:27:30.066 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-24T23:27:30.066 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-24T23:27:30.066 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-24T23:27:30.066 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-04-24T23:27:30.066 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:27:30.067 DEBUG:teuthology.orchestra.run.smithi080:> ip addr 2024-04-24T23:27:30.082 INFO:teuthology.orchestra.run.smithi080.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-24T23:27:30.082 INFO:teuthology.orchestra.run.smithi080.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-24T23:27:30.082 INFO:teuthology.orchestra.run.smithi080.stdout: inet 127.0.0.1/8 scope host lo 2024-04-24T23:27:30.082 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:27:30.083 INFO:teuthology.orchestra.run.smithi080.stdout: inet6 ::1/128 scope host 2024-04-24T23:27:30.083 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:27:30.083 INFO:teuthology.orchestra.run.smithi080.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:27:30.083 INFO:teuthology.orchestra.run.smithi080.stdout: link/ether 0c:c4:7a:d8:d9:b2 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:27:30.083 INFO:teuthology.orchestra.run.smithi080.stdout: altname enp7s0f0 2024-04-24T23:27:30.083 INFO:teuthology.orchestra.run.smithi080.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:27:30.083 INFO:teuthology.orchestra.run.smithi080.stdout: link/ether 0c:c4:7a:d8:d9:b3 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:27:30.083 INFO:teuthology.orchestra.run.smithi080.stdout: altname enp7s0f1 2024-04-24T23:27:30.083 INFO:teuthology.orchestra.run.smithi080.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:27:30.083 INFO:teuthology.orchestra.run.smithi080.stdout: link/ether 0c:c4:7a:8f:cb:a8 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:27:30.083 INFO:teuthology.orchestra.run.smithi080.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-24T23:27:30.083 INFO:teuthology.orchestra.run.smithi080.stdout: link/ether 0c:c4:7a:8f:cb:a9 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:27:30.083 INFO:teuthology.orchestra.run.smithi080.stdout: inet 172.21.15.80/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-24T23:27:30.083 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft 42242sec preferred_lft 42242sec 2024-04-24T23:27:30.084 INFO:teuthology.orchestra.run.smithi080.stdout: inet6 fe80::477b:93ee:6d5c:3d53/64 scope link noprefixroute 2024-04-24T23:27:30.084 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:27:30.084 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-24T23:27:30.084 DEBUG:teuthology.orchestra.run.smithi080:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T23:27:30.084 DEBUG:teuthology.orchestra.run.smithi080:> set -e 2024-04-24T23:27:30.084 DEBUG:teuthology.orchestra.run.smithi080:> sudo ip link add name ceph-brx type bridge 2024-04-24T23:27:30.085 DEBUG:teuthology.orchestra.run.smithi080:> sudo ip addr flush dev ceph-brx 2024-04-24T23:27:30.085 DEBUG:teuthology.orchestra.run.smithi080:> sudo ip link set ceph-brx up 2024-04-24T23:27:30.085 DEBUG:teuthology.orchestra.run.smithi080:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-24T23:27:30.085 DEBUG:teuthology.orchestra.run.smithi080:> ') 2024-04-24T23:27:30.171 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:30 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:27:30.257 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:27:30.264 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:27:30.264 DEBUG:teuthology.orchestra.run.smithi080:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-24T23:27:30.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:30 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/3157119658' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-24T23:27:30.289 INFO:teuthology.orchestra.run.smithi080.stdout:1 2024-04-24T23:27:30.291 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:27:30.291 DEBUG:teuthology.orchestra.run.smithi080:> route 2024-04-24T23:27:30.358 INFO:teuthology.orchestra.run.smithi080.stdout:Kernel IP routing table 2024-04-24T23:27:30.358 INFO:teuthology.orchestra.run.smithi080.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-24T23:27:30.358 INFO:teuthology.orchestra.run.smithi080.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-24T23:27:30.358 INFO:teuthology.orchestra.run.smithi080.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-24T23:27:30.358 INFO:teuthology.orchestra.run.smithi080.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-24T23:27:30.359 DEBUG:teuthology.orchestra.run.smithi080:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T23:27:30.359 DEBUG:teuthology.orchestra.run.smithi080:> set -e 2024-04-24T23:27:30.359 DEBUG:teuthology.orchestra.run.smithi080:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-24T23:27:30.359 DEBUG:teuthology.orchestra.run.smithi080:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-24T23:27:30.359 DEBUG:teuthology.orchestra.run.smithi080:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-24T23:27:30.359 DEBUG:teuthology.orchestra.run.smithi080:> ') 2024-04-24T23:27:30.446 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:30 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:27:30.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:30 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/3157119658' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-24T23:27:30.642 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:27:30.648 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:27:30.648 DEBUG:teuthology.orchestra.run.smithi080:> ip netns list 2024-04-24T23:27:30.663 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:27:30.663 DEBUG:teuthology.orchestra.run.smithi080:> ip netns list-id 2024-04-24T23:27:30.697 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:30 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/3157119658' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-24T23:27:30.720 DEBUG:teuthology.orchestra.run.smithi080:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T23:27:30.720 DEBUG:teuthology.orchestra.run.smithi080:> set -e 2024-04-24T23:27:30.720 DEBUG:teuthology.orchestra.run.smithi080:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-24T23:27:30.720 DEBUG:teuthology.orchestra.run.smithi080:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-04-24T23:27:30.720 DEBUG:teuthology.orchestra.run.smithi080:> ') 2024-04-24T23:27:30.807 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:30 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:27:30.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:27:30.835 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-04-24T23:27:30.836 DEBUG:teuthology.orchestra.run.smithi080:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T23:27:30.836 DEBUG:teuthology.orchestra.run.smithi080:> set -e 2024-04-24T23:27:30.836 DEBUG:teuthology.orchestra.run.smithi080:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-04-24T23:27:30.836 DEBUG:teuthology.orchestra.run.smithi080:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-24T23:27:30.836 DEBUG:teuthology.orchestra.run.smithi080:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-04-24T23:27:30.836 DEBUG:teuthology.orchestra.run.smithi080:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-04-24T23:27:30.836 DEBUG:teuthology.orchestra.run.smithi080:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-04-24T23:27:30.836 DEBUG:teuthology.orchestra.run.smithi080:> ') 2024-04-24T23:27:30.884 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:30 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:27:30.972 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:27:30.978 DEBUG:teuthology.orchestra.run.smithi080:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T23:27:30.978 DEBUG:teuthology.orchestra.run.smithi080:> set -e 2024-04-24T23:27:30.978 DEBUG:teuthology.orchestra.run.smithi080:> sudo ip link set brx.0 up 2024-04-24T23:27:30.978 DEBUG:teuthology.orchestra.run.smithi080:> sudo ip link set dev brx.0 master ceph-brx 2024-04-24T23:27:30.978 DEBUG:teuthology.orchestra.run.smithi080:> ') 2024-04-24T23:27:31.026 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:31 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:27:31.047 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:27:31.054 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-04-24T23:27:31.055 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T23:27:31.055 DEBUG:teuthology.orchestra.run.smithi080:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-04-24T23:27:31.070 INFO:teuthology.orchestra.run.smithi080.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-04-24T23:27:31.070 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T23:27:31.070 DEBUG:teuthology.orchestra.run.smithi080:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-04-24T23:27:31.125 DEBUG:teuthology.orchestra.run.smithi080:> sudo modprobe fuse 2024-04-24T23:27:31.189 DEBUG:teuthology.orchestra.run.smithi080:> cat /proc/self/mounts | awk '{print $2}' 2024-04-24T23:27:31.245 INFO:teuthology.orchestra.run.smithi080.stdout:/proc 2024-04-24T23:27:31.246 INFO:teuthology.orchestra.run.smithi080.stdout:/sys 2024-04-24T23:27:31.246 INFO:teuthology.orchestra.run.smithi080.stdout:/dev 2024-04-24T23:27:31.246 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/security 2024-04-24T23:27:31.246 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/shm 2024-04-24T23:27:31.246 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/pts 2024-04-24T23:27:31.246 INFO:teuthology.orchestra.run.smithi080.stdout:/run 2024-04-24T23:27:31.246 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/cgroup 2024-04-24T23:27:31.246 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/pstore 2024-04-24T23:27:31.246 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/bpf 2024-04-24T23:27:31.246 INFO:teuthology.orchestra.run.smithi080.stdout:/ 2024-04-24T23:27:31.246 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/selinux 2024-04-24T23:27:31.246 INFO:teuthology.orchestra.run.smithi080.stdout:/proc/sys/fs/binfmt_misc 2024-04-24T23:27:31.246 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/hugepages 2024-04-24T23:27:31.246 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/mqueue 2024-04-24T23:27:31.247 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/debug 2024-04-24T23:27:31.247 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/tracing 2024-04-24T23:27:31.247 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/config 2024-04-24T23:27:31.247 INFO:teuthology.orchestra.run.smithi080.stdout:/run/credentials/systemd-sysusers.service 2024-04-24T23:27:31.247 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/fuse/connections 2024-04-24T23:27:31.247 INFO:teuthology.orchestra.run.smithi080.stdout:/run/credentials/systemd-sysctl.service 2024-04-24T23:27:31.247 INFO:teuthology.orchestra.run.smithi080.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-24T23:27:31.247 INFO:teuthology.orchestra.run.smithi080.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-24T23:27:31.247 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/nfs/rpc_pipefs 2024-04-24T23:27:31.247 INFO:teuthology.orchestra.run.smithi080.stdout:/run/user/1000 2024-04-24T23:27:31.247 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph 2024-04-24T23:27:31.247 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/containers/storage/overlay 2024-04-24T23:27:31.247 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/containers/storage/overlay/b8852a80c56907c162a63d4a5645ad5b4106f418501273c8fa3ea2a9c7a6ac88/merged 2024-04-24T23:27:31.247 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/containers/storage/overlay/dd4eda2c7c6db41a84dbf5e2d19353167301d5bfa92e6b534fdd2f2e2f666ce4/merged 2024-04-24T23:27:31.248 INFO:teuthology.orchestra.run.smithi080.stdout:/run/user/0 2024-04-24T23:27:31.248 INFO:teuthology.orchestra.run.smithi080.stdout:/proc/sys/fs/binfmt_misc 2024-04-24T23:27:31.248 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/containers/storage/overlay/894526bd16964ef79a0c4024e7b668f8336c335769291c19e55db237256f4345/merged 2024-04-24T23:27:31.248 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/containers/storage/overlay/6904dbca4c6e4fd6054c6a0403dc7ac8f80ba15bc47a4013a5d2934832474656/merged 2024-04-24T23:27:31.248 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/containers/storage/overlay/93267a0dc6528793d70d105975104b48c0fce1cbcafead056fa2dd8d880cd865/merged 2024-04-24T23:27:31.248 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/containers/storage/overlay/508c33caac5e1322de18fe6179fa10abe818095778493edebd0351c648a5f0bb/merged 2024-04-24T23:27:31.248 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/containers/storage/overlay/2ba3e9e6f7a5c286cff2be92d118ba0d7cd6e2d2e7e0f62369ffe1631dc64643/merged 2024-04-24T23:27:31.248 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/containers/storage/overlay/399bae25bdb50ced963486a91942a5e511a0013ce2cc0fecbc22c664b71a6fd0/merged 2024-04-24T23:27:31.248 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/containers/storage/overlay/891c0411686ee4ce4980b862d75fd34491f750f7708e194e23bef1b59434b6b3/merged 2024-04-24T23:27:31.248 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/containers/storage/overlay/5a3ce38ede23c16cee1ba51f54949a6d56768959a21936d507f42db56f663e0d/merged 2024-04-24T23:27:31.248 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns 2024-04-24T23:27:31.249 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-24T23:27:31.249 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-24T23:27:31.249 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:27:31.249 DEBUG:teuthology.orchestra.run.smithi080:> ls /sys/fs/fuse/connections 2024-04-24T23:27:31.301 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-24T23:27:31.301 DEBUG:teuthology.orchestra.run.smithi080:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-04-24T23:27:31.343 DEBUG:teuthology.orchestra.run.smithi080:> sudo modprobe fuse 2024-04-24T23:27:31.366 DEBUG:teuthology.orchestra.run.smithi080:> cat /proc/self/mounts | awk '{print $2}' 2024-04-24T23:27:31.422 INFO:teuthology.orchestra.run.smithi080.stdout:/proc 2024-04-24T23:27:31.422 INFO:teuthology.orchestra.run.smithi080.stdout:/sys 2024-04-24T23:27:31.422 INFO:teuthology.orchestra.run.smithi080.stdout:/dev 2024-04-24T23:27:31.422 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/security 2024-04-24T23:27:31.422 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/shm 2024-04-24T23:27:31.422 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/pts 2024-04-24T23:27:31.422 INFO:teuthology.orchestra.run.smithi080.stdout:/run 2024-04-24T23:27:31.422 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/cgroup 2024-04-24T23:27:31.423 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/pstore 2024-04-24T23:27:31.423 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/bpf 2024-04-24T23:27:31.423 INFO:teuthology.orchestra.run.smithi080.stdout:/ 2024-04-24T23:27:31.423 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/selinux 2024-04-24T23:27:31.423 INFO:teuthology.orchestra.run.smithi080.stdout:/proc/sys/fs/binfmt_misc 2024-04-24T23:27:31.423 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/hugepages 2024-04-24T23:27:31.423 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/mqueue 2024-04-24T23:27:31.423 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/debug 2024-04-24T23:27:31.423 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/tracing 2024-04-24T23:27:31.423 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/config 2024-04-24T23:27:31.423 INFO:teuthology.orchestra.run.smithi080.stdout:/run/credentials/systemd-sysusers.service 2024-04-24T23:27:31.423 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/fuse/connections 2024-04-24T23:27:31.423 INFO:teuthology.orchestra.run.smithi080.stdout:/run/credentials/systemd-sysctl.service 2024-04-24T23:27:31.423 INFO:teuthology.orchestra.run.smithi080.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-24T23:27:31.424 INFO:teuthology.orchestra.run.smithi080.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-24T23:27:31.424 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/nfs/rpc_pipefs 2024-04-24T23:27:31.424 INFO:teuthology.orchestra.run.smithi080.stdout:/run/user/1000 2024-04-24T23:27:31.424 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph 2024-04-24T23:27:31.424 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/containers/storage/overlay 2024-04-24T23:27:31.424 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/containers/storage/overlay/b8852a80c56907c162a63d4a5645ad5b4106f418501273c8fa3ea2a9c7a6ac88/merged 2024-04-24T23:27:31.424 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/containers/storage/overlay/dd4eda2c7c6db41a84dbf5e2d19353167301d5bfa92e6b534fdd2f2e2f666ce4/merged 2024-04-24T23:27:31.424 INFO:teuthology.orchestra.run.smithi080.stdout:/run/user/0 2024-04-24T23:27:31.424 INFO:teuthology.orchestra.run.smithi080.stdout:/proc/sys/fs/binfmt_misc 2024-04-24T23:27:31.424 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/containers/storage/overlay/894526bd16964ef79a0c4024e7b668f8336c335769291c19e55db237256f4345/merged 2024-04-24T23:27:31.424 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/containers/storage/overlay/6904dbca4c6e4fd6054c6a0403dc7ac8f80ba15bc47a4013a5d2934832474656/merged 2024-04-24T23:27:31.424 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/containers/storage/overlay/93267a0dc6528793d70d105975104b48c0fce1cbcafead056fa2dd8d880cd865/merged 2024-04-24T23:27:31.424 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/containers/storage/overlay/508c33caac5e1322de18fe6179fa10abe818095778493edebd0351c648a5f0bb/merged 2024-04-24T23:27:31.425 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/containers/storage/overlay/2ba3e9e6f7a5c286cff2be92d118ba0d7cd6e2d2e7e0f62369ffe1631dc64643/merged 2024-04-24T23:27:31.425 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/containers/storage/overlay/399bae25bdb50ced963486a91942a5e511a0013ce2cc0fecbc22c664b71a6fd0/merged 2024-04-24T23:27:31.425 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/containers/storage/overlay/891c0411686ee4ce4980b862d75fd34491f750f7708e194e23bef1b59434b6b3/merged 2024-04-24T23:27:31.425 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/containers/storage/overlay/5a3ce38ede23c16cee1ba51f54949a6d56768959a21936d507f42db56f663e0d/merged 2024-04-24T23:27:31.425 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns 2024-04-24T23:27:31.425 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-24T23:27:31.425 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-24T23:27:31.425 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:27:31.426 DEBUG:teuthology.orchestra.run.smithi080:> ls /sys/fs/fuse/connections 2024-04-24T23:27:31.440 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi080.stderr:2024-04-24T23:27:31.439+0000 7f5810fda480 -1 init, newargv = 0x5579912ff130 newargc=15 2024-04-24T23:27:31.441 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi080.stderr:ceph-fuse[72649]: starting ceph client 2024-04-24T23:27:31.447 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi080.stderr:ceph-fuse[72649]: starting fuse 2024-04-24T23:27:31.455 INFO:teuthology.orchestra.run.smithi080.stdout:115 2024-04-24T23:27:31.455 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [115] 2024-04-24T23:27:31.455 DEBUG:teuthology.orchestra.run.smithi080:> sudo stdin-killer -- python3 -c ' 2024-04-24T23:27:31.455 DEBUG:teuthology.orchestra.run.smithi080:> import glob 2024-04-24T23:27:31.455 DEBUG:teuthology.orchestra.run.smithi080:> import re 2024-04-24T23:27:31.456 DEBUG:teuthology.orchestra.run.smithi080:> import os 2024-04-24T23:27:31.456 DEBUG:teuthology.orchestra.run.smithi080:> import subprocess 2024-04-24T23:27:31.456 DEBUG:teuthology.orchestra.run.smithi080:> 2024-04-24T23:27:31.456 DEBUG:teuthology.orchestra.run.smithi080:> def _find_admin_socket(client_name): 2024-04-24T23:27:31.456 DEBUG:teuthology.orchestra.run.smithi080:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-04-24T23:27:31.456 DEBUG:teuthology.orchestra.run.smithi080:> files = glob.glob(asok_path) 2024-04-24T23:27:31.456 DEBUG:teuthology.orchestra.run.smithi080:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-04-24T23:27:31.456 DEBUG:teuthology.orchestra.run.smithi080:> 2024-04-24T23:27:31.456 DEBUG:teuthology.orchestra.run.smithi080:> # Given a non-glob path, it better be there 2024-04-24T23:27:31.456 DEBUG:teuthology.orchestra.run.smithi080:> if "*" not in asok_path: 2024-04-24T23:27:31.456 DEBUG:teuthology.orchestra.run.smithi080:> assert(len(files) == 1) 2024-04-24T23:27:31.456 DEBUG:teuthology.orchestra.run.smithi080:> return files[0] 2024-04-24T23:27:31.456 DEBUG:teuthology.orchestra.run.smithi080:> 2024-04-24T23:27:31.456 DEBUG:teuthology.orchestra.run.smithi080:> for f in files: 2024-04-24T23:27:31.456 DEBUG:teuthology.orchestra.run.smithi080:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-24T23:27:31.456 DEBUG:teuthology.orchestra.run.smithi080:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-24T23:27:31.457 DEBUG:teuthology.orchestra.run.smithi080:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-24T23:27:31.457 DEBUG:teuthology.orchestra.run.smithi080:> contents = proc_f.read() 2024-04-24T23:27:31.457 DEBUG:teuthology.orchestra.run.smithi080:> if mountpoint in contents: 2024-04-24T23:27:31.457 DEBUG:teuthology.orchestra.run.smithi080:> return f 2024-04-24T23:27:31.457 DEBUG:teuthology.orchestra.run.smithi080:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-24T23:27:31.457 DEBUG:teuthology.orchestra.run.smithi080:> 2024-04-24T23:27:31.457 DEBUG:teuthology.orchestra.run.smithi080:> print(_find_admin_socket("client.admin")) 2024-04-24T23:27:31.457 DEBUG:teuthology.orchestra.run.smithi080:> ' 2024-04-24T23:27:31.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:31 smithi148 ceph-mon[32528]: pgmap v302: 129 pgs: 129 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-04-24T23:27:31.516 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:31 smithi080 ceph-mon[29152]: pgmap v302: 129 pgs: 129 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-04-24T23:27:31.572 INFO:teuthology.orchestra.run.smithi080.stdout:/var/run/ceph/ceph-client.admin.72649.asok 2024-04-24T23:27:31.575 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:27:31.582 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.72649.asok 2024-04-24T23:27:31.582 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:27:31.583 DEBUG:teuthology.orchestra.run.smithi080:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.72649.asok status 2024-04-24T23:27:31.697 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:31 smithi134 ceph-mon[32619]: pgmap v302: 129 pgs: 129 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-04-24T23:27:31.715 INFO:teuthology.orchestra.run.smithi080.stdout:{ 2024-04-24T23:27:31.715 INFO:teuthology.orchestra.run.smithi080.stdout: "metadata": { 2024-04-24T23:27:31.715 INFO:teuthology.orchestra.run.smithi080.stdout: "ceph_sha1": "d2c45d977142e024a59b68a8f1dce8f0b9840551", 2024-04-24T23:27:31.715 INFO:teuthology.orchestra.run.smithi080.stdout: "ceph_version": "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)", 2024-04-24T23:27:31.715 INFO:teuthology.orchestra.run.smithi080.stdout: "entity_id": "admin", 2024-04-24T23:27:31.715 INFO:teuthology.orchestra.run.smithi080.stdout: "hostname": "smithi080", 2024-04-24T23:27:31.715 INFO:teuthology.orchestra.run.smithi080.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-04-24T23:27:31.715 INFO:teuthology.orchestra.run.smithi080.stdout: "pid": "72649", 2024-04-24T23:27:31.715 INFO:teuthology.orchestra.run.smithi080.stdout: "root": "/" 2024-04-24T23:27:31.715 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:27:31.716 INFO:teuthology.orchestra.run.smithi080.stdout: "dentry_count": 0, 2024-04-24T23:27:31.716 INFO:teuthology.orchestra.run.smithi080.stdout: "dentry_pinned_count": 0, 2024-04-24T23:27:31.716 INFO:teuthology.orchestra.run.smithi080.stdout: "id": 14799, 2024-04-24T23:27:31.716 INFO:teuthology.orchestra.run.smithi080.stdout: "inst": { 2024-04-24T23:27:31.716 INFO:teuthology.orchestra.run.smithi080.stdout: "name": { 2024-04-24T23:27:31.716 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "client", 2024-04-24T23:27:31.716 INFO:teuthology.orchestra.run.smithi080.stdout: "num": 14799 2024-04-24T23:27:31.716 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:27:31.716 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": { 2024-04-24T23:27:31.716 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-24T23:27:31.716 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "192.168.0.1:0", 2024-04-24T23:27:31.716 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 147094855 2024-04-24T23:27:31.716 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:27:31.716 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:27:31.716 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": { 2024-04-24T23:27:31.717 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-24T23:27:31.717 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "192.168.0.1:0", 2024-04-24T23:27:31.717 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 147094855 2024-04-24T23:27:31.717 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:27:31.717 INFO:teuthology.orchestra.run.smithi080.stdout: "inst_str": "client.14799 192.168.0.1:0/147094855", 2024-04-24T23:27:31.717 INFO:teuthology.orchestra.run.smithi080.stdout: "addr_str": "192.168.0.1:0/147094855", 2024-04-24T23:27:31.717 INFO:teuthology.orchestra.run.smithi080.stdout: "inode_count": 1, 2024-04-24T23:27:31.717 INFO:teuthology.orchestra.run.smithi080.stdout: "mds_epoch": 17, 2024-04-24T23:27:31.717 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_epoch": 90, 2024-04-24T23:27:31.717 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_epoch_barrier": 0, 2024-04-24T23:27:31.717 INFO:teuthology.orchestra.run.smithi080.stdout: "blocklisted": false, 2024-04-24T23:27:31.717 INFO:teuthology.orchestra.run.smithi080.stdout: "fs_name": "cephfs" 2024-04-24T23:27:31.717 INFO:teuthology.orchestra.run.smithi080.stdout:} 2024-04-24T23:27:31.723 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:27:31.723 DEBUG:teuthology.orchestra.run.smithi080:> stat --file-system '--printf=%T 2024-04-24T23:27:31.724 DEBUG:teuthology.orchestra.run.smithi080:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-04-24T23:27:31.739 INFO:teuthology.orchestra.run.smithi080.stdout:fuseblk 2024-04-24T23:27:31.739 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-04-24T23:27:31.739 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:27:31.740 DEBUG:teuthology.orchestra.run.smithi080:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-04-24T23:27:31.804 DEBUG:teuthology.orchestra.run.smithi080:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-04-24T23:27:31.894 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:31 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-24T23:27:31.915 INFO:teuthology.orchestra.run.smithi080.stdout:# file: . 2024-04-24T23:27:31.915 INFO:teuthology.orchestra.run.smithi080.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-04-24T23:27:31.915 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-24T23:27:31.915 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:27:31.921 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-24T23:27:31.921 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:27:31.940 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:27:32.080 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.076+0000 7f7d56f6a640 1 -- 172.21.15.80:0/3019363252 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7d50104b30 msgr2=0x7f7d50106f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:32.081 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.076+0000 7f7d56f6a640 1 --2- 172.21.15.80:0/3019363252 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7d50104b30 0x7f7d50106f20 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f7d40009900 tx=0x7f7d4002a410 comp rx=0 tx=0).stop 2024-04-24T23:27:32.081 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.076+0000 7f7d56f6a640 1 -- 172.21.15.80:0/3019363252 shutdown_connections 2024-04-24T23:27:32.081 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.076+0000 7f7d56f6a640 1 --2- 172.21.15.80:0/3019363252 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d50107460 0x7f7d501098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:32.081 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.076+0000 7f7d56f6a640 1 --2- 172.21.15.80:0/3019363252 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7d50104b30 0x7f7d50106f20 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:32.081 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.076+0000 7f7d56f6a640 1 --2- 172.21.15.80:0/3019363252 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d50102200 0x7f7d501045f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:32.081 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.076+0000 7f7d56f6a640 1 -- 172.21.15.80:0/3019363252 >> 172.21.15.80:0/3019363252 conn(0x7f7d500fbbf0 msgr2=0x7f7d500fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:32.081 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.076+0000 7f7d56f6a640 1 -- 172.21.15.80:0/3019363252 shutdown_connections 2024-04-24T23:27:32.081 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.077+0000 7f7d56f6a640 1 -- 172.21.15.80:0/3019363252 wait complete. 2024-04-24T23:27:32.081 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.077+0000 7f7d56f6a640 1 Processor -- start 2024-04-24T23:27:32.081 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.077+0000 7f7d56f6a640 1 -- start start 2024-04-24T23:27:32.081 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.077+0000 7f7d56f6a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d50102200 0x7f7d50197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:32.081 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.077+0000 7f7d56f6a640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7d50104b30 0x7f7d50198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:32.081 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.077+0000 7f7d56f6a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d50107460 0x7f7d5019f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:32.081 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.077+0000 7f7d56f6a640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f7d5010d450 con 0x7f7d50104b30 2024-04-24T23:27:32.081 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.077+0000 7f7d56f6a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7d5010d2d0 con 0x7f7d50102200 2024-04-24T23:27:32.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.077+0000 7f7d56f6a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f7d5010d5d0 con 0x7f7d50107460 2024-04-24T23:27:32.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.077+0000 7f7d4ffff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7d50104b30 0x7f7d50198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:32.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.077+0000 7f7d4ffff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7d50104b30 0x7f7d50198010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:35754/0 (socket says 172.21.15.80:35754) 2024-04-24T23:27:32.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.077+0000 7f7d4ffff640 1 -- 172.21.15.80:0/3831195524 learned_addr learned my addr 172.21.15.80:0/3831195524 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:27:32.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.077+0000 7f7d55507640 1 --2- 172.21.15.80:0/3831195524 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d50107460 0x7f7d5019f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:32.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.078+0000 7f7d54d06640 1 --2- 172.21.15.80:0/3831195524 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d50102200 0x7f7d50197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:32.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.078+0000 7f7d4ffff640 1 -- 172.21.15.80:0/3831195524 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d50107460 msgr2=0x7f7d5019f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:32.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.078+0000 7f7d4ffff640 1 --2- 172.21.15.80:0/3831195524 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d50107460 0x7f7d5019f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:32.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.078+0000 7f7d4ffff640 1 -- 172.21.15.80:0/3831195524 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d50102200 msgr2=0x7f7d50197ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:32.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.078+0000 7f7d4ffff640 1 --2- 172.21.15.80:0/3831195524 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d50102200 0x7f7d50197ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:32.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.078+0000 7f7d4ffff640 1 -- 172.21.15.80:0/3831195524 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d5019f6a0 con 0x7f7d50104b30 2024-04-24T23:27:32.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.078+0000 7f7d55507640 1 --2- 172.21.15.80:0/3831195524 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d50107460 0x7f7d5019f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:27:32.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.078+0000 7f7d54d06640 1 --2- 172.21.15.80:0/3831195524 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d50102200 0x7f7d50197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:27:32.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.078+0000 7f7d4ffff640 1 --2- 172.21.15.80:0/3831195524 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7d50104b30 0x7f7d50198010 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f7d40031610 tx=0x7f7d400043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:32.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.078+0000 7f7d4dffb640 1 -- 172.21.15.80:0/3831195524 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d4003f030 con 0x7f7d50104b30 2024-04-24T23:27:32.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.078+0000 7f7d4dffb640 1 -- 172.21.15.80:0/3831195524 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d40003ce0 con 0x7f7d50104b30 2024-04-24T23:27:32.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.078+0000 7f7d56f6a640 1 -- 172.21.15.80:0/3831195524 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d5019f930 con 0x7f7d50104b30 2024-04-24T23:27:32.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.078+0000 7f7d4dffb640 1 -- 172.21.15.80:0/3831195524 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d40035070 con 0x7f7d50104b30 2024-04-24T23:27:32.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.078+0000 7f7d56f6a640 1 -- 172.21.15.80:0/3831195524 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d5019fea0 con 0x7f7d50104b30 2024-04-24T23:27:32.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.079+0000 7f7d56f6a640 1 -- 172.21.15.80:0/3831195524 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d500fd6e0 con 0x7f7d50104b30 2024-04-24T23:27:32.084 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.081+0000 7f7d4dffb640 1 -- 172.21.15.80:0/3831195524 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f7d40042030 con 0x7f7d50104b30 2024-04-24T23:27:32.084 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.081+0000 7f7d4dffb640 1 --2- 172.21.15.80:0/3831195524 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f7d3c0a2b90 0x7f7d3c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:32.084 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.081+0000 7f7d54d06640 1 --2- 172.21.15.80:0/3831195524 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f7d3c0a2b90 0x7f7d3c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:32.084 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.081+0000 7f7d4dffb640 1 -- 172.21.15.80:0/3831195524 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f7d400efc00 con 0x7f7d50104b30 2024-04-24T23:27:32.084 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.082+0000 7f7d54d06640 1 --2- 172.21.15.80:0/3831195524 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f7d3c0a2b90 0x7f7d3c0a5050 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f7d48008e00 tx=0x7f7d48008350 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:32.086 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.085+0000 7f7d4dffb640 1 -- 172.21.15.80:0/3831195524 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f7d400bc920 con 0x7f7d50104b30 2024-04-24T23:27:32.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.265+0000 7f7d56f6a640 1 -- 172.21.15.80:0/3831195524 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7d5002d370 con 0x7f7d50104b30 2024-04-24T23:27:32.267 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.267+0000 7f7d4dffb640 1 -- 172.21.15.80:0/3831195524 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24124 (secure 0 0 0) 0x7f7d400bc220 con 0x7f7d50104b30 2024-04-24T23:27:32.268 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-24T23:27:32.268 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":90,"fsid":"ec7885aa-0290-11ef-bc93-c7b262605968","created":"2024-04-24T23:19:05.586407+0000","modified":"2024-04-24T23:27:26.212213+0000","last_up_change":"2024-04-24T23:26:25.938316+0000","last_in_change":"2024-04-24T23:26:04.786699+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:22:33.124428+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:27:17.575788+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:27:18.492762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T23:27:22.438503+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a525741d-b46c-4eef-aa64-3f057cac1497","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":2597718112},{"type":"v1","addr":"172.21.15.80:6803","nonce":2597718112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":2597718112},{"type":"v1","addr":"172.21.15.80:6805","nonce":2597718112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2597718112},{"type":"v1","addr":"172.21.15.80:6809","nonce":2597718112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":2597718112},{"type":"v1","addr":"172.21.15.80:6807","nonce":2597718112}]},"public_addr":"172.21.15.80:6803/2597718112","cluster_addr":"172.21.15.80:6805/2597718112","heartbeat_back_addr":"172.21.15.80:6809/2597718112","heartbeat_front_addr":"172.21.15.80:6807/2597718112","state":["exists","up"]},{"osd":1,"uuid":"2d0f0d67-6db4-40ac-91cc-58604ab6795a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":2395610201},{"type":"v1","addr":"172.21.15.134:6801","nonce":2395610201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":2395610201},{"type":"v1","addr":"172.21.15.134:6803","nonce":2395610201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":2395610201},{"type":"v1","addr":"172.21.15.134:6807","nonce":2395610201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":2395610201},{"type":"v1","addr":"172.21.15.134:6805","nonce":2395610201}]},"public_addr":"172.21.15.134:6801/2395610201","cluster_addr":"172.21.15.134:6803/2395610201","heartbeat_back_addr":"172.21.15.134:6807/2395610201","heartbeat_front_addr":"172.21.15.134:6805/2395610201","state":["exists","up"]},{"osd":2,"uuid":"781fb879-fedc-4c84-82fb-59e073bc4cf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1629152187},{"type":"v1","addr":"172.21.15.148:6801","nonce":1629152187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1629152187},{"type":"v1","addr":"172.21.15.148:6803","nonce":1629152187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1629152187},{"type":"v1","addr":"172.21.15.148:6807","nonce":1629152187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1629152187},{"type":"v1","addr":"172.21.15.148:6805","nonce":1629152187}]},"public_addr":"172.21.15.148:6801/1629152187","cluster_addr":"172.21.15.148:6803/1629152187","heartbeat_back_addr":"172.21.15.148:6807/1629152187","heartbeat_front_addr":"172.21.15.148:6805/1629152187","state":["exists","up"]},{"osd":3,"uuid":"cd982757-f2f6-4296-b3e7-4cd40fbd8d9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":3128298510},{"type":"v1","addr":"172.21.15.80:6811","nonce":3128298510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":3128298510},{"type":"v1","addr":"172.21.15.80:6813","nonce":3128298510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":3128298510},{"type":"v1","addr":"172.21.15.80:6817","nonce":3128298510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":3128298510},{"type":"v1","addr":"172.21.15.80:6815","nonce":3128298510}]},"public_addr":"172.21.15.80:6811/3128298510","cluster_addr":"172.21.15.80:6813/3128298510","heartbeat_back_addr":"172.21.15.80:6817/3128298510","heartbeat_front_addr":"172.21.15.80:6815/3128298510","state":["exists","up"]},{"osd":4,"uuid":"dfc55a79-a7e8-417a-b898-ec44171a54af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1917556060},{"type":"v1","addr":"172.21.15.134:6809","nonce":1917556060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1917556060},{"type":"v1","addr":"172.21.15.134:6811","nonce":1917556060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1917556060},{"type":"v1","addr":"172.21.15.134:6815","nonce":1917556060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1917556060},{"type":"v1","addr":"172.21.15.134:6813","nonce":1917556060}]},"public_addr":"172.21.15.134:6809/1917556060","cluster_addr":"172.21.15.134:6811/1917556060","heartbeat_back_addr":"172.21.15.134:6815/1917556060","heartbeat_front_addr":"172.21.15.134:6813/1917556060","state":["exists","up"]},{"osd":5,"uuid":"b5491337-0694-440c-b679-312f3d14cf65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":2715951281},{"type":"v1","addr":"172.21.15.148:6809","nonce":2715951281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":2715951281},{"type":"v1","addr":"172.21.15.148:6811","nonce":2715951281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":2715951281},{"type":"v1","addr":"172.21.15.148:6815","nonce":2715951281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":2715951281},{"type":"v1","addr":"172.21.15.148:6813","nonce":2715951281}]},"public_addr":"172.21.15.148:6809/2715951281","cluster_addr":"172.21.15.148:6811/2715951281","heartbeat_back_addr":"172.21.15.148:6815/2715951281","heartbeat_front_addr":"172.21.15.148:6813/2715951281","state":["exists","up"]},{"osd":6,"uuid":"ba4b5df7-d86e-47cd-ab67-cf763c4e4d6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":953261724},{"type":"v1","addr":"172.21.15.80:6819","nonce":953261724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":953261724},{"type":"v1","addr":"172.21.15.80:6821","nonce":953261724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":953261724},{"type":"v1","addr":"172.21.15.80:6825","nonce":953261724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":953261724},{"type":"v1","addr":"172.21.15.80:6823","nonce":953261724}]},"public_addr":"172.21.15.80:6819/953261724","cluster_addr":"172.21.15.80:6821/953261724","heartbeat_back_addr":"172.21.15.80:6825/953261724","heartbeat_front_addr":"172.21.15.80:6823/953261724","state":["exists","up"]},{"osd":7,"uuid":"464a503a-75ae-49e7-ab6e-a7f9f2a6a307","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":108354844},{"type":"v1","addr":"172.21.15.134:6817","nonce":108354844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":108354844},{"type":"v1","addr":"172.21.15.134:6819","nonce":108354844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":108354844},{"type":"v1","addr":"172.21.15.134:6823","nonce":108354844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":108354844},{"type":"v1","addr":"172.21.15.134:6821","nonce":108354844}]},"public_addr":"172.21.15.134:6817/108354844","cluster_addr":"172.21.15.134:6819/108354844","heartbeat_back_addr":"172.21.15.134:6823/108354844","heartbeat_front_addr":"172.21.15.134:6821/108354844","state":["exists","up"]},{"osd":8,"uuid":"ce544b2e-4540-47c0-9997-500c45351330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3559655329},{"type":"v1","addr":"172.21.15.148:6817","nonce":3559655329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":3559655329},{"type":"v1","addr":"172.21.15.148:6819","nonce":3559655329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":3559655329},{"type":"v1","addr":"172.21.15.148:6823","nonce":3559655329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3559655329},{"type":"v1","addr":"172.21.15.148:6821","nonce":3559655329}]},"public_addr":"172.21.15.148:6817/3559655329","cluster_addr":"172.21.15.148:6819/3559655329","heartbeat_back_addr":"172.21.15.148:6823/3559655329","heartbeat_front_addr":"172.21.15.148:6821/3559655329","state":["exists","up"]},{"osd":9,"uuid":"06de3732-19b9-496c-a464-55322efa6759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":4019913545},{"type":"v1","addr":"172.21.15.80:6827","nonce":4019913545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":4019913545},{"type":"v1","addr":"172.21.15.80:6829","nonce":4019913545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":4019913545},{"type":"v1","addr":"172.21.15.80:6833","nonce":4019913545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":4019913545},{"type":"v1","addr":"172.21.15.80:6831","nonce":4019913545}]},"public_addr":"172.21.15.80:6827/4019913545","cluster_addr":"172.21.15.80:6829/4019913545","heartbeat_back_addr":"172.21.15.80:6833/4019913545","heartbeat_front_addr":"172.21.15.80:6831/4019913545","state":["exists","up"]},{"osd":10,"uuid":"8d92f0d9-5c64-429a-b750-c33ae0e070c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":647343766},{"type":"v1","addr":"172.21.15.134:6825","nonce":647343766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":647343766},{"type":"v1","addr":"172.21.15.134:6827","nonce":647343766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":647343766},{"type":"v1","addr":"172.21.15.134:6831","nonce":647343766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":647343766},{"type":"v1","addr":"172.21.15.134:6829","nonce":647343766}]},"public_addr":"172.21.15.134:6825/647343766","cluster_addr":"172.21.15.134:6827/647343766","heartbeat_back_addr":"172.21.15.134:6831/647343766","heartbeat_front_addr":"172.21.15.134:6829/647343766","state":["exists","up"]},{"osd":11,"uuid":"4c003a41-f65b-479f-bc80-b27d680cf936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":4068024116},{"type":"v1","addr":"172.21.15.148:6825","nonce":4068024116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":4068024116},{"type":"v1","addr":"172.21.15.148:6827","nonce":4068024116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":4068024116},{"type":"v1","addr":"172.21.15.148:6831","nonce":4068024116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":4068024116},{"type":"v1","addr":"172.21.15.148:6829","nonce":4068024116}]},"public_addr":"172.21.15.148:6825/4068024116","cluster_addr":"172.21.15.148:6827/4068024116","heartbeat_back_addr":"172.21.15.148:6831/4068024116","heartbeat_front_addr":"172.21.15.148:6829/4068024116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:21:42.433631+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:22:06.145933+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:22:30.726143+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:22:55.729961+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:23:21.280913+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:23:46.609983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:24:12.585247+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:24:38.359287+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:25:04.491232+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:25:30.788089+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:25:56.891078+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:26:23.899113+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.80:0/2828378749":"2024-04-25T23:20:06.793609+0000","172.21.15.80:6801/233364790":"2024-04-25T23:20:06.793609+0000","172.21.15.80:6800/1614691410":"2024-04-25T23:19:40.490692+0000","172.21.15.80:0/1406187145":"2024-04-25T23:20:06.793609+0000","172.21.15.80:0/92953396":"2024-04-25T23:19:40.490692+0000","172.21.15.80:0/598129432":"2024-04-25T23:20:06.793609+0000","172.21.15.80:0/232193118":"2024-04-25T23:19:40.490692+0000","172.21.15.80:6800/233364790":"2024-04-25T23:20:06.793609+0000","172.21.15.80:6801/1614691410":"2024-04-25T23:19:40.490692+0000","172.21.15.80:0/2081562529":"2024-04-25T23:19:40.490692+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:27:32.270 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.269+0000 7f7d56f6a640 1 -- 172.21.15.80:0/3831195524 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f7d3c0a2b90 msgr2=0x7f7d3c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:32.270 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.269+0000 7f7d56f6a640 1 --2- 172.21.15.80:0/3831195524 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f7d3c0a2b90 0x7f7d3c0a5050 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f7d48008e00 tx=0x7f7d48008350 comp rx=0 tx=0).stop 2024-04-24T23:27:32.270 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.269+0000 7f7d56f6a640 1 -- 172.21.15.80:0/3831195524 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7d50104b30 msgr2=0x7f7d50198010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:32.270 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.269+0000 7f7d56f6a640 1 --2- 172.21.15.80:0/3831195524 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7d50104b30 0x7f7d50198010 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f7d40031610 tx=0x7f7d400043b0 comp rx=0 tx=0).stop 2024-04-24T23:27:32.270 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.269+0000 7f7d56f6a640 1 -- 172.21.15.80:0/3831195524 shutdown_connections 2024-04-24T23:27:32.270 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.269+0000 7f7d56f6a640 1 --2- 172.21.15.80:0/3831195524 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f7d3c0a2b90 0x7f7d3c0a5050 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:32.271 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.269+0000 7f7d56f6a640 1 --2- 172.21.15.80:0/3831195524 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d50107460 0x7f7d5019f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:32.271 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.269+0000 7f7d56f6a640 1 --2- 172.21.15.80:0/3831195524 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7d50104b30 0x7f7d50198010 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:32.271 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.269+0000 7f7d56f6a640 1 --2- 172.21.15.80:0/3831195524 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d50102200 0x7f7d50197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:32.271 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.269+0000 7f7d56f6a640 1 -- 172.21.15.80:0/3831195524 >> 172.21.15.80:0/3831195524 conn(0x7f7d500fbbf0 msgr2=0x7f7d50102b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:32.271 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.269+0000 7f7d56f6a640 1 -- 172.21.15.80:0/3831195524 shutdown_connections 2024-04-24T23:27:32.271 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.269+0000 7f7d56f6a640 1 -- 172.21.15.80:0/3831195524 wait complete. 2024-04-24T23:27:32.281 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:22:33.124428+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T23:27:17.575788+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T23:27:18.492762+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-24T23:27:22.438503+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-24T23:27:32.281 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:27:32.385 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.382+0000 7f3e620da640 1 -- 172.21.15.80:0/826776861 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3e5c101890 msgr2=0x7f3e5c101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:32.385 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.382+0000 7f3e620da640 1 --2- 172.21.15.80:0/826776861 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3e5c101890 0x7f3e5c101d10 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f3e50009900 tx=0x7f3e5002a410 comp rx=0 tx=0).stop 2024-04-24T23:27:32.385 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.382+0000 7f3e620da640 1 -- 172.21.15.80:0/826776861 shutdown_connections 2024-04-24T23:27:32.385 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.382+0000 7f3e620da640 1 --2- 172.21.15.80:0/826776861 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3e5c102250 0x7f3e5c108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:32.385 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.382+0000 7f3e620da640 1 --2- 172.21.15.80:0/826776861 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3e5c101890 0x7f3e5c101d10 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:32.385 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.382+0000 7f3e620da640 1 --2- 172.21.15.80:0/826776861 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3e5c1006a0 0x7f3e5c100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:32.385 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.382+0000 7f3e620da640 1 -- 172.21.15.80:0/826776861 >> 172.21.15.80:0/826776861 conn(0x7f3e5c0fbc30 msgr2=0x7f3e5c0fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:32.385 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.382+0000 7f3e620da640 1 -- 172.21.15.80:0/826776861 shutdown_connections 2024-04-24T23:27:32.385 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.382+0000 7f3e620da640 1 -- 172.21.15.80:0/826776861 wait complete. 2024-04-24T23:27:32.386 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.382+0000 7f3e620da640 1 Processor -- start 2024-04-24T23:27:32.386 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.383+0000 7f3e620da640 1 -- start start 2024-04-24T23:27:32.386 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.383+0000 7f3e620da640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3e5c1006a0 0x7f3e5c197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:32.386 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.383+0000 7f3e620da640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3e5c101890 0x7f3e5c198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:32.386 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.383+0000 7f3e620da640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3e5c102250 0x7f3e5c19f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:32.386 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.383+0000 7f3e620da640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f3e5c10d480 con 0x7f3e5c102250 2024-04-24T23:27:32.386 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.383+0000 7f3e620da640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3e5c10d300 con 0x7f3e5c1006a0 2024-04-24T23:27:32.386 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.383+0000 7f3e620da640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3e5c10d600 con 0x7f3e5c101890 2024-04-24T23:27:32.386 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.383+0000 7f3e5bfff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3e5c102250 0x7f3e5c19f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:32.386 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.383+0000 7f3e5bfff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3e5c102250 0x7f3e5c19f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:35768/0 (socket says 172.21.15.80:35768) 2024-04-24T23:27:32.386 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.383+0000 7f3e5bfff640 1 -- 172.21.15.80:0/3519414971 learned_addr learned my addr 172.21.15.80:0/3519414971 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:27:32.386 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.383+0000 7f3e5b7fe640 1 --2- 172.21.15.80:0/3519414971 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3e5c1006a0 0x7f3e5c197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:32.386 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.383+0000 7f3e5affd640 1 --2- 172.21.15.80:0/3519414971 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3e5c101890 0x7f3e5c198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:32.387 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.383+0000 7f3e5bfff640 1 -- 172.21.15.80:0/3519414971 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3e5c101890 msgr2=0x7f3e5c198090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:32.387 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.383+0000 7f3e5bfff640 1 --2- 172.21.15.80:0/3519414971 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3e5c101890 0x7f3e5c198090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:32.387 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.383+0000 7f3e5bfff640 1 -- 172.21.15.80:0/3519414971 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3e5c1006a0 msgr2=0x7f3e5c197b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:32.387 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.383+0000 7f3e5bfff640 1 --2- 172.21.15.80:0/3519414971 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3e5c1006a0 0x7f3e5c197b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:32.387 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.383+0000 7f3e5bfff640 1 -- 172.21.15.80:0/3519414971 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e5c19f770 con 0x7f3e5c102250 2024-04-24T23:27:32.387 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.383+0000 7f3e5bfff640 1 --2- 172.21.15.80:0/3519414971 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3e5c102250 0x7f3e5c19f100 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f3e5400b970 tx=0x7f3e5400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:32.387 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.383+0000 7f3e58ff9640 1 -- 172.21.15.80:0/3519414971 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e54019070 con 0x7f3e5c102250 2024-04-24T23:27:32.387 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.383+0000 7f3e58ff9640 1 -- 172.21.15.80:0/3519414971 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e5400cb40 con 0x7f3e5c102250 2024-04-24T23:27:32.387 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.383+0000 7f3e620da640 1 -- 172.21.15.80:0/3519414971 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e5c19fa60 con 0x7f3e5c102250 2024-04-24T23:27:32.387 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.383+0000 7f3e58ff9640 1 -- 172.21.15.80:0/3519414971 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e54007610 con 0x7f3e5c102250 2024-04-24T23:27:32.387 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.383+0000 7f3e620da640 1 -- 172.21.15.80:0/3519414971 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e5c19ff80 con 0x7f3e5c102250 2024-04-24T23:27:32.388 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.384+0000 7f3e620da640 1 -- 172.21.15.80:0/3519414971 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e5c1079e0 con 0x7f3e5c102250 2024-04-24T23:27:32.388 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.385+0000 7f3e58ff9640 1 -- 172.21.15.80:0/3519414971 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f3e5401c070 con 0x7f3e5c102250 2024-04-24T23:27:32.388 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.386+0000 7f3e58ff9640 1 --2- 172.21.15.80:0/3519414971 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f3e440a2b90 0x7f3e440a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:32.388 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.386+0000 7f3e5b7fe640 1 --2- 172.21.15.80:0/3519414971 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f3e440a2b90 0x7f3e440a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:32.388 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.386+0000 7f3e58ff9640 1 -- 172.21.15.80:0/3519414971 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f3e540ce7f0 con 0x7f3e5c102250 2024-04-24T23:27:32.389 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.386+0000 7f3e5b7fe640 1 --2- 172.21.15.80:0/3519414971 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f3e440a2b90 0x7f3e440a5050 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f3e4c005fd0 tx=0x7f3e4c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:32.391 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.391+0000 7f3e58ff9640 1 -- 172.21.15.80:0/3519414971 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f3e5409aec0 con 0x7f3e5c102250 2024-04-24T23:27:32.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:32 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/3831195524' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:27:32.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.568+0000 7f3e620da640 1 -- 172.21.15.80:0/3519414971 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3e5c0623e0 con 0x7f3e5c102250 2024-04-24T23:27:32.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.568+0000 7f3e58ff9640 1 -- 172.21.15.80:0/3519414971 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) ==== 93+0+10 (secure 0 0 0) 0x7f3e5409f1c0 con 0x7f3e5c102250 2024-04-24T23:27:32.569 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 1 2024-04-24T23:27:32.571 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.570+0000 7f3e620da640 1 -- 172.21.15.80:0/3519414971 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f3e440a2b90 msgr2=0x7f3e440a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:32.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.570+0000 7f3e620da640 1 --2- 172.21.15.80:0/3519414971 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f3e440a2b90 0x7f3e440a5050 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f3e4c005fd0 tx=0x7f3e4c00b410 comp rx=0 tx=0).stop 2024-04-24T23:27:32.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.570+0000 7f3e620da640 1 -- 172.21.15.80:0/3519414971 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3e5c102250 msgr2=0x7f3e5c19f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:32.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.570+0000 7f3e620da640 1 --2- 172.21.15.80:0/3519414971 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3e5c102250 0x7f3e5c19f100 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f3e5400b970 tx=0x7f3e5400be30 comp rx=0 tx=0).stop 2024-04-24T23:27:32.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.570+0000 7f3e620da640 1 -- 172.21.15.80:0/3519414971 shutdown_connections 2024-04-24T23:27:32.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.570+0000 7f3e620da640 1 --2- 172.21.15.80:0/3519414971 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f3e440a2b90 0x7f3e440a5050 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:32.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.570+0000 7f3e620da640 1 --2- 172.21.15.80:0/3519414971 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3e5c102250 0x7f3e5c19f100 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:32.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.570+0000 7f3e620da640 1 --2- 172.21.15.80:0/3519414971 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3e5c101890 0x7f3e5c198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:32.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.570+0000 7f3e620da640 1 --2- 172.21.15.80:0/3519414971 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3e5c1006a0 0x7f3e5c197b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:32.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.570+0000 7f3e620da640 1 -- 172.21.15.80:0/3519414971 >> 172.21.15.80:0/3519414971 conn(0x7f3e5c0fbc30 msgr2=0x7f3e5c104bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:32.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.570+0000 7f3e620da640 1 -- 172.21.15.80:0/3519414971 shutdown_connections 2024-04-24T23:27:32.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.570+0000 7f3e620da640 1 -- 172.21.15.80:0/3519414971 wait complete. 2024-04-24T23:27:32.582 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T23:27:32.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.681+0000 7feea54a5640 1 -- 172.21.15.80:0/2017876038 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feea0101870 msgr2=0x7feea0101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:32.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.681+0000 7feea54a5640 1 --2- 172.21.15.80:0/2017876038 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feea0101870 0x7feea0101cf0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fee94009900 tx=0x7fee9402a410 comp rx=0 tx=0).stop 2024-04-24T23:27:32.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.681+0000 7feea54a5640 1 -- 172.21.15.80:0/2017876038 shutdown_connections 2024-04-24T23:27:32.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.681+0000 7feea54a5640 1 --2- 172.21.15.80:0/2017876038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feea0102230 0x7feea0108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:32.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.681+0000 7feea54a5640 1 --2- 172.21.15.80:0/2017876038 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feea0101870 0x7feea0101cf0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:32.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.681+0000 7feea54a5640 1 --2- 172.21.15.80:0/2017876038 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feea0100680 0x7feea0100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:32.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.681+0000 7feea54a5640 1 -- 172.21.15.80:0/2017876038 >> 172.21.15.80:0/2017876038 conn(0x7feea00fbbf0 msgr2=0x7feea00fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:32.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.681+0000 7feea54a5640 1 -- 172.21.15.80:0/2017876038 shutdown_connections 2024-04-24T23:27:32.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.681+0000 7feea54a5640 1 -- 172.21.15.80:0/2017876038 wait complete. 2024-04-24T23:27:32.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.682+0000 7feea54a5640 1 Processor -- start 2024-04-24T23:27:32.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.682+0000 7feea54a5640 1 -- start start 2024-04-24T23:27:32.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.682+0000 7feea54a5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feea0100680 0x7feea0069440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:32.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.682+0000 7feea54a5640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feea0101870 0x7feea019c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:32.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.682+0000 7feea54a5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feea0102230 0x7feea019ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:32.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.682+0000 7feea54a5640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7feea010c2e0 con 0x7feea0101870 2024-04-24T23:27:32.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.682+0000 7feea54a5640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7feea010c160 con 0x7feea0102230 2024-04-24T23:27:32.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.682+0000 7feea54a5640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7feea010c460 con 0x7feea0100680 2024-04-24T23:27:32.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.682+0000 7fee9e7fc640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feea0101870 0x7feea019c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:32.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.682+0000 7fee9e7fc640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feea0101870 0x7feea019c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:50460/0 (socket says 172.21.15.80:50460) 2024-04-24T23:27:32.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.682+0000 7fee9e7fc640 1 -- 172.21.15.80:0/3384028671 learned_addr learned my addr 172.21.15.80:0/3384028671 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:27:32.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.682+0000 7fee9f7fe640 1 --2- 172.21.15.80:0/3384028671 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feea0102230 0x7feea019ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:32.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.682+0000 7fee9effd640 1 --2- 172.21.15.80:0/3384028671 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feea0100680 0x7feea0069440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:32.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.683+0000 7fee9e7fc640 1 -- 172.21.15.80:0/3384028671 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feea0100680 msgr2=0x7feea0069440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:32.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.683+0000 7fee9e7fc640 1 --2- 172.21.15.80:0/3384028671 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feea0100680 0x7feea0069440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:32.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.683+0000 7fee9e7fc640 1 -- 172.21.15.80:0/3384028671 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feea0102230 msgr2=0x7feea019ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:32.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.683+0000 7fee9e7fc640 1 --2- 172.21.15.80:0/3384028671 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feea0102230 0x7feea019ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:32.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.683+0000 7fee9e7fc640 1 -- 172.21.15.80:0/3384028671 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feea019f750 con 0x7feea0101870 2024-04-24T23:27:32.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.683+0000 7fee9effd640 1 --2- 172.21.15.80:0/3384028671 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feea0100680 0x7feea0069440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:27:32.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.683+0000 7fee9e7fc640 1 --2- 172.21.15.80:0/3384028671 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feea0101870 0x7feea019c820 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fee94009230 tx=0x7fee94031b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:32.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.683+0000 7fee87fff640 1 -- 172.21.15.80:0/3384028671 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee9403f030 con 0x7feea0101870 2024-04-24T23:27:32.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.683+0000 7fee87fff640 1 -- 172.21.15.80:0/3384028671 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee94003ad0 con 0x7feea0101870 2024-04-24T23:27:32.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.683+0000 7feea54a5640 1 -- 172.21.15.80:0/3384028671 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feea019f980 con 0x7feea0101870 2024-04-24T23:27:32.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.683+0000 7fee87fff640 1 -- 172.21.15.80:0/3384028671 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee94035070 con 0x7feea0101870 2024-04-24T23:27:32.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.683+0000 7feea54a5640 1 -- 172.21.15.80:0/3384028671 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feea019fe70 con 0x7feea0101870 2024-04-24T23:27:32.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.684+0000 7feea54a5640 1 -- 172.21.15.80:0/3384028671 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feea0107a90 con 0x7feea0101870 2024-04-24T23:27:32.688 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.685+0000 7fee87fff640 1 -- 172.21.15.80:0/3384028671 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fee94042030 con 0x7feea0101870 2024-04-24T23:27:32.688 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.685+0000 7fee87fff640 1 --2- 172.21.15.80:0/3384028671 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fee880a2b90 0x7fee880a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:32.689 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.686+0000 7fee9effd640 1 --2- 172.21.15.80:0/3384028671 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fee880a2b90 0x7fee880a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:32.689 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.686+0000 7fee87fff640 1 -- 172.21.15.80:0/3384028671 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fee940ef810 con 0x7feea0101870 2024-04-24T23:27:32.689 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.686+0000 7fee9effd640 1 --2- 172.21.15.80:0/3384028671 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fee880a2b90 0x7fee880a5050 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7feea01016d0 tx=0x7fee9000a800 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:32.691 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.690+0000 7fee87fff640 1 -- 172.21.15.80:0/3384028671 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fee940bc530 con 0x7feea0101870 2024-04-24T23:27:32.697 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:32 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/3831195524' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:27:32.756 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:32 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/3831195524' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:27:32.867 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.866+0000 7feea54a5640 1 -- 172.21.15.80:0/3384028671 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7feea0062340 con 0x7feea0101870 2024-04-24T23:27:32.867 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.866+0000 7fee87fff640 1 -- 172.21.15.80:0/3384028671 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) ==== 104+0+11 (secure 0 0 0) 0x7fee940bbe30 con 0x7feea0101870 2024-04-24T23:27:32.867 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 32 2024-04-24T23:27:32.869 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.868+0000 7feea54a5640 1 -- 172.21.15.80:0/3384028671 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fee880a2b90 msgr2=0x7fee880a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:32.869 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.868+0000 7feea54a5640 1 --2- 172.21.15.80:0/3384028671 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fee880a2b90 0x7fee880a5050 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7feea01016d0 tx=0x7fee9000a800 comp rx=0 tx=0).stop 2024-04-24T23:27:32.870 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.868+0000 7feea54a5640 1 -- 172.21.15.80:0/3384028671 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feea0101870 msgr2=0x7feea019c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:32.870 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.868+0000 7feea54a5640 1 --2- 172.21.15.80:0/3384028671 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feea0101870 0x7feea019c820 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fee94009230 tx=0x7fee94031b60 comp rx=0 tx=0).stop 2024-04-24T23:27:32.870 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.868+0000 7feea54a5640 1 -- 172.21.15.80:0/3384028671 shutdown_connections 2024-04-24T23:27:32.870 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.868+0000 7feea54a5640 1 --2- 172.21.15.80:0/3384028671 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fee880a2b90 0x7fee880a5050 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:32.870 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.868+0000 7feea54a5640 1 --2- 172.21.15.80:0/3384028671 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feea0102230 0x7feea019ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:32.870 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.868+0000 7feea54a5640 1 --2- 172.21.15.80:0/3384028671 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feea0101870 0x7feea019c820 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:32.870 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.868+0000 7feea54a5640 1 --2- 172.21.15.80:0/3384028671 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feea0100680 0x7feea0069440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:32.870 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.868+0000 7feea54a5640 1 -- 172.21.15.80:0/3384028671 >> 172.21.15.80:0/3384028671 conn(0x7feea00fbbf0 msgr2=0x7feea0104c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:32.870 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.868+0000 7feea54a5640 1 -- 172.21.15.80:0/3384028671 shutdown_connections 2024-04-24T23:27:32.870 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.868+0000 7feea54a5640 1 -- 172.21.15.80:0/3384028671 wait complete. 2024-04-24T23:27:32.880 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T23:27:32.983 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.980+0000 7f33f2509640 1 -- 172.21.15.80:0/2282544549 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f33ec101890 msgr2=0x7f33ec101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:32.983 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.980+0000 7f33f2509640 1 --2- 172.21.15.80:0/2282544549 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f33ec101890 0x7f33ec101d10 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f33e0009900 tx=0x7f33e002a410 comp rx=0 tx=0).stop 2024-04-24T23:27:32.983 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.980+0000 7f33f2509640 1 -- 172.21.15.80:0/2282544549 shutdown_connections 2024-04-24T23:27:32.983 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.980+0000 7f33f2509640 1 --2- 172.21.15.80:0/2282544549 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f33ec102250 0x7f33ec108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:32.983 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.980+0000 7f33f2509640 1 --2- 172.21.15.80:0/2282544549 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f33ec101890 0x7f33ec101d10 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:32.983 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.980+0000 7f33f2509640 1 --2- 172.21.15.80:0/2282544549 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f33ec1006a0 0x7f33ec100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:32.983 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.980+0000 7f33f2509640 1 -- 172.21.15.80:0/2282544549 >> 172.21.15.80:0/2282544549 conn(0x7f33ec0fbc30 msgr2=0x7f33ec0fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:32.984 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.980+0000 7f33f2509640 1 -- 172.21.15.80:0/2282544549 shutdown_connections 2024-04-24T23:27:32.984 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.980+0000 7f33f2509640 1 -- 172.21.15.80:0/2282544549 wait complete. 2024-04-24T23:27:32.984 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.980+0000 7f33f2509640 1 Processor -- start 2024-04-24T23:27:32.984 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.980+0000 7f33f2509640 1 -- start start 2024-04-24T23:27:32.984 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.981+0000 7f33f2509640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f33ec1006a0 0x7f33ec197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:32.984 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.981+0000 7f33f2509640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f33ec101890 0x7f33ec197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:32.984 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.981+0000 7f33f2509640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f33ec102250 0x7f33ec19f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:32.984 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.981+0000 7f33f2509640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f33ec10d440 con 0x7f33ec1006a0 2024-04-24T23:27:32.984 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.981+0000 7f33f2509640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f33ec10d2c0 con 0x7f33ec102250 2024-04-24T23:27:32.984 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.981+0000 7f33f2509640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f33ec10d5c0 con 0x7f33ec101890 2024-04-24T23:27:32.984 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.981+0000 7f33ebfff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f33ec1006a0 0x7f33ec197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:32.984 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.981+0000 7f33ebfff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f33ec1006a0 0x7f33ec197a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:50478/0 (socket says 172.21.15.80:50478) 2024-04-24T23:27:32.984 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.981+0000 7f33ebfff640 1 -- 172.21.15.80:0/177005380 learned_addr learned my addr 172.21.15.80:0/177005380 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:27:32.984 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.981+0000 7f33eb7fe640 1 --2- 172.21.15.80:0/177005380 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f33ec101890 0x7f33ec197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:32.985 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.981+0000 7f33ebfff640 1 -- 172.21.15.80:0/177005380 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f33ec101890 msgr2=0x7f33ec197fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:32.985 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.981+0000 7f33ebfff640 1 --2- 172.21.15.80:0/177005380 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f33ec101890 0x7f33ec197fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:32.985 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.981+0000 7f33ebfff640 1 -- 172.21.15.80:0/177005380 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f33ec102250 msgr2=0x7f33ec19f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:32.985 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.981+0000 7f33ebfff640 1 --2- 172.21.15.80:0/177005380 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f33ec102250 0x7f33ec19f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:32.985 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.981+0000 7f33ebfff640 1 -- 172.21.15.80:0/177005380 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33ec19f6b0 con 0x7f33ec1006a0 2024-04-24T23:27:32.985 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.981+0000 7f33ebfff640 1 --2- 172.21.15.80:0/177005380 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f33ec1006a0 0x7f33ec197a90 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f33dc00d950 tx=0x7f33dc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:32.985 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.981+0000 7f33e97fa640 1 -- 172.21.15.80:0/177005380 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33dc017070 con 0x7f33ec1006a0 2024-04-24T23:27:32.985 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.981+0000 7f33e97fa640 1 -- 172.21.15.80:0/177005380 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33dc009c30 con 0x7f33ec1006a0 2024-04-24T23:27:32.985 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.981+0000 7f33f2509640 1 -- 172.21.15.80:0/177005380 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33ec19f9a0 con 0x7f33ec1006a0 2024-04-24T23:27:32.985 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.981+0000 7f33e97fa640 1 -- 172.21.15.80:0/177005380 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33dc00f600 con 0x7f33ec1006a0 2024-04-24T23:27:32.985 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.981+0000 7f33f2509640 1 -- 172.21.15.80:0/177005380 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33ec19fe70 con 0x7f33ec1006a0 2024-04-24T23:27:32.986 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.983+0000 7f33f2509640 1 -- 172.21.15.80:0/177005380 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33ec0689b0 con 0x7f33ec1006a0 2024-04-24T23:27:32.986 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.984+0000 7f33e97fa640 1 -- 172.21.15.80:0/177005380 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f33dc01a070 con 0x7f33ec1006a0 2024-04-24T23:27:32.986 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.984+0000 7f33e97fa640 1 --2- 172.21.15.80:0/177005380 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f33d40a2b90 0x7f33d40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:32.986 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.984+0000 7f33eb7fe640 1 --2- 172.21.15.80:0/177005380 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f33d40a2b90 0x7f33d40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:32.986 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.984+0000 7f33e97fa640 1 -- 172.21.15.80:0/177005380 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f33dc0ced10 con 0x7f33ec1006a0 2024-04-24T23:27:32.987 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.985+0000 7f33eb7fe640 1 --2- 172.21.15.80:0/177005380 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f33d40a2b90 0x7f33d40a5050 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f33e00314e0 tx=0x7f33e0000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:32.989 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:32.989+0000 7f33e97fa640 1 -- 172.21.15.80:0/177005380 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f33dc009de0 con 0x7f33ec1006a0 2024-04-24T23:27:33.170 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.166+0000 7f33f2509640 1 -- 172.21.15.80:0/177005380 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f33ec0623e0 con 0x7f33ec1006a0 2024-04-24T23:27:33.170 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.167+0000 7f33e97fa640 1 -- 172.21.15.80:0/177005380 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) ==== 100+0+11 (secure 0 0 0) 0x7f33dc09ba30 con 0x7f33ec1006a0 2024-04-24T23:27:33.171 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 64 2024-04-24T23:27:33.172 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.168+0000 7f33f2509640 1 -- 172.21.15.80:0/177005380 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f33d40a2b90 msgr2=0x7f33d40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:33.172 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.168+0000 7f33f2509640 1 --2- 172.21.15.80:0/177005380 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f33d40a2b90 0x7f33d40a5050 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f33e00314e0 tx=0x7f33e0000bc0 comp rx=0 tx=0).stop 2024-04-24T23:27:33.172 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.169+0000 7f33f2509640 1 -- 172.21.15.80:0/177005380 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f33ec1006a0 msgr2=0x7f33ec197a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:33.172 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.169+0000 7f33f2509640 1 --2- 172.21.15.80:0/177005380 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f33ec1006a0 0x7f33ec197a90 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f33dc00d950 tx=0x7f33dc00de10 comp rx=0 tx=0).stop 2024-04-24T23:27:33.172 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.169+0000 7f33f2509640 1 -- 172.21.15.80:0/177005380 shutdown_connections 2024-04-24T23:27:33.172 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.169+0000 7f33f2509640 1 --2- 172.21.15.80:0/177005380 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f33d40a2b90 0x7f33d40a5050 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:33.172 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.169+0000 7f33f2509640 1 --2- 172.21.15.80:0/177005380 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f33ec102250 0x7f33ec19f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:33.172 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.169+0000 7f33f2509640 1 --2- 172.21.15.80:0/177005380 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f33ec101890 0x7f33ec197fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:33.172 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.169+0000 7f33f2509640 1 --2- 172.21.15.80:0/177005380 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f33ec1006a0 0x7f33ec197a90 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:33.172 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.169+0000 7f33f2509640 1 -- 172.21.15.80:0/177005380 >> 172.21.15.80:0/177005380 conn(0x7f33ec0fbc30 msgr2=0x7f33ec104c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:33.172 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.169+0000 7f33f2509640 1 -- 172.21.15.80:0/177005380 shutdown_connections 2024-04-24T23:27:33.172 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.169+0000 7f33f2509640 1 -- 172.21.15.80:0/177005380 wait complete. 2024-04-24T23:27:33.180 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-24T23:27:33.283 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.280+0000 7f450d437640 1 -- 172.21.15.80:0/3081867815 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4508101870 msgr2=0x7f4508101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:33.284 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.280+0000 7f450d437640 1 --2- 172.21.15.80:0/3081867815 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4508101870 0x7f4508101cf0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f44fc009900 tx=0x7f44fc02a410 comp rx=0 tx=0).stop 2024-04-24T23:27:33.284 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.280+0000 7f450d437640 1 -- 172.21.15.80:0/3081867815 shutdown_connections 2024-04-24T23:27:33.284 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.280+0000 7f450d437640 1 --2- 172.21.15.80:0/3081867815 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4508102230 0x7f4508108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:33.284 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.280+0000 7f450d437640 1 --2- 172.21.15.80:0/3081867815 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4508101870 0x7f4508101cf0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:33.284 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.280+0000 7f450d437640 1 --2- 172.21.15.80:0/3081867815 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4508100680 0x7f4508100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:33.284 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.280+0000 7f450d437640 1 -- 172.21.15.80:0/3081867815 >> 172.21.15.80:0/3081867815 conn(0x7f45080fbbf0 msgr2=0x7f45080fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:33.284 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.280+0000 7f450d437640 1 -- 172.21.15.80:0/3081867815 shutdown_connections 2024-04-24T23:27:33.284 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.280+0000 7f450d437640 1 -- 172.21.15.80:0/3081867815 wait complete. 2024-04-24T23:27:33.284 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.281+0000 7f450d437640 1 Processor -- start 2024-04-24T23:27:33.284 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.281+0000 7f450d437640 1 -- start start 2024-04-24T23:27:33.284 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.281+0000 7f450d437640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4508100680 0x7f4508197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:33.284 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.281+0000 7f450d437640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4508101870 0x7f4508198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:33.284 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.281+0000 7f450d437640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4508102230 0x7f450819f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:33.285 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.281+0000 7f450d437640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f450810c2e0 con 0x7f4508102230 2024-04-24T23:27:33.285 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.281+0000 7f450d437640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f450810c160 con 0x7f4508101870 2024-04-24T23:27:33.285 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.281+0000 7f450d437640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f450810c460 con 0x7f4508100680 2024-04-24T23:27:33.285 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.281+0000 7f45077fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4508102230 0x7f450819f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:33.285 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.281+0000 7f45077fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4508102230 0x7f450819f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:50500/0 (socket says 172.21.15.80:50500) 2024-04-24T23:27:33.285 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.281+0000 7f45077fe640 1 -- 172.21.15.80:0/3874849520 learned_addr learned my addr 172.21.15.80:0/3874849520 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:27:33.285 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.281+0000 7f45067fc640 1 --2- 172.21.15.80:0/3874849520 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4508101870 0x7f4508198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:33.285 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.282+0000 7f4506ffd640 1 --2- 172.21.15.80:0/3874849520 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4508100680 0x7f4508197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:33.285 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.282+0000 7f45077fe640 1 -- 172.21.15.80:0/3874849520 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4508100680 msgr2=0x7f4508197ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:33.285 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.282+0000 7f45077fe640 1 --2- 172.21.15.80:0/3874849520 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4508100680 0x7f4508197ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:33.285 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.282+0000 7f45077fe640 1 -- 172.21.15.80:0/3874849520 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4508101870 msgr2=0x7f4508198010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:33.285 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.282+0000 7f45077fe640 1 --2- 172.21.15.80:0/3874849520 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4508101870 0x7f4508198010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:33.285 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.282+0000 7f45077fe640 1 -- 172.21.15.80:0/3874849520 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f450819f6f0 con 0x7f4508102230 2024-04-24T23:27:33.285 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.282+0000 7f45077fe640 1 --2- 172.21.15.80:0/3874849520 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4508102230 0x7f450819f080 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f450000d750 tx=0x7f450000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:33.286 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.282+0000 7f44effff640 1 -- 172.21.15.80:0/3874849520 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4500017070 con 0x7f4508102230 2024-04-24T23:27:33.286 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.282+0000 7f44effff640 1 -- 172.21.15.80:0/3874849520 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4500005500 con 0x7f4508102230 2024-04-24T23:27:33.286 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.282+0000 7f450d437640 1 -- 172.21.15.80:0/3874849520 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f450819f9e0 con 0x7f4508102230 2024-04-24T23:27:33.286 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.282+0000 7f44effff640 1 -- 172.21.15.80:0/3874849520 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4500004d90 con 0x7f4508102230 2024-04-24T23:27:33.286 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.282+0000 7f450d437640 1 -- 172.21.15.80:0/3874849520 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f450819feb0 con 0x7f4508102230 2024-04-24T23:27:33.287 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.283+0000 7f450d437640 1 -- 172.21.15.80:0/3874849520 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45081a0140 con 0x7f4508102230 2024-04-24T23:27:33.287 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.284+0000 7f44effff640 1 -- 172.21.15.80:0/3874849520 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f4500004f30 con 0x7f4508102230 2024-04-24T23:27:33.287 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.284+0000 7f44effff640 1 --2- 172.21.15.80:0/3874849520 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f44f00a2b90 0x7f44f00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:33.287 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.285+0000 7f4506ffd640 1 --2- 172.21.15.80:0/3874849520 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f44f00a2b90 0x7f44f00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:33.287 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.285+0000 7f44effff640 1 -- 172.21.15.80:0/3874849520 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f45000cec10 con 0x7f4508102230 2024-04-24T23:27:33.287 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.285+0000 7f4506ffd640 1 --2- 172.21.15.80:0/3874849520 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f44f00a2b90 0x7f44f00a5050 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f45081016d0 tx=0x7f44f8009670 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:33.290 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.289+0000 7f44effff640 1 -- 172.21.15.80:0/3874849520 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f450009b2e0 con 0x7f4508102230 2024-04-24T23:27:33.468 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.467+0000 7f450d437640 1 -- 172.21.15.80:0/3874849520 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f45081a0310 con 0x7f4508102230 2024-04-24T23:27:33.469 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.468+0000 7f44effff640 1 -- 172.21.15.80:0/3874849520 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v90) ==== 103+0+11 (secure 0 0 0) 0x7f450009f5e0 con 0x7f4508102230 2024-04-24T23:27:33.469 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 32 2024-04-24T23:27:33.471 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.470+0000 7f450d437640 1 -- 172.21.15.80:0/3874849520 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f44f00a2b90 msgr2=0x7f44f00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:33.471 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.470+0000 7f450d437640 1 --2- 172.21.15.80:0/3874849520 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f44f00a2b90 0x7f44f00a5050 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f45081016d0 tx=0x7f44f8009670 comp rx=0 tx=0).stop 2024-04-24T23:27:33.471 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.470+0000 7f450d437640 1 -- 172.21.15.80:0/3874849520 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4508102230 msgr2=0x7f450819f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:33.471 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.470+0000 7f450d437640 1 --2- 172.21.15.80:0/3874849520 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4508102230 0x7f450819f080 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f450000d750 tx=0x7f450000dc10 comp rx=0 tx=0).stop 2024-04-24T23:27:33.471 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.470+0000 7f450d437640 1 -- 172.21.15.80:0/3874849520 shutdown_connections 2024-04-24T23:27:33.471 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.470+0000 7f450d437640 1 --2- 172.21.15.80:0/3874849520 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f44f00a2b90 0x7f44f00a5050 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:33.471 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.470+0000 7f450d437640 1 --2- 172.21.15.80:0/3874849520 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4508102230 0x7f450819f080 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:33.471 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.470+0000 7f450d437640 1 --2- 172.21.15.80:0/3874849520 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4508101870 0x7f4508198010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:33.471 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.470+0000 7f450d437640 1 --2- 172.21.15.80:0/3874849520 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4508100680 0x7f4508197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:33.471 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.470+0000 7f450d437640 1 -- 172.21.15.80:0/3874849520 >> 172.21.15.80:0/3874849520 conn(0x7f45080fbbf0 msgr2=0x7f45080fd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:33.471 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.470+0000 7f450d437640 1 -- 172.21.15.80:0/3874849520 shutdown_connections 2024-04-24T23:27:33.472 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.470+0000 7f450d437640 1 -- 172.21.15.80:0/3874849520 wait complete. 2024-04-24T23:27:33.480 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:33 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/3519414971' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:27:33.480 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:33 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/3384028671' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T23:27:33.480 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:33 smithi080 ceph-mon[29152]: pgmap v303: 129 pgs: 129 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:27:33.481 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:33 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/177005380' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T23:27:33.482 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-24T23:27:33.583 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.580+0000 7fc17f370640 1 -- 172.21.15.80:0/4217469043 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc178101890 msgr2=0x7fc178101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:33.583 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.580+0000 7fc17f370640 1 --2- 172.21.15.80:0/4217469043 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc178101890 0x7fc178101d10 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fc174009900 tx=0x7fc17402a410 comp rx=0 tx=0).stop 2024-04-24T23:27:33.584 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.580+0000 7fc17f370640 1 -- 172.21.15.80:0/4217469043 shutdown_connections 2024-04-24T23:27:33.584 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.580+0000 7fc17f370640 1 --2- 172.21.15.80:0/4217469043 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc178102250 0x7fc178108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:33.584 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.580+0000 7fc17f370640 1 --2- 172.21.15.80:0/4217469043 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc178101890 0x7fc178101d10 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:33.584 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.580+0000 7fc17f370640 1 --2- 172.21.15.80:0/4217469043 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1781006a0 0x7fc178100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:33.584 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.580+0000 7fc17f370640 1 -- 172.21.15.80:0/4217469043 >> 172.21.15.80:0/4217469043 conn(0x7fc1780fbc30 msgr2=0x7fc1780fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:33.584 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.580+0000 7fc17f370640 1 -- 172.21.15.80:0/4217469043 shutdown_connections 2024-04-24T23:27:33.584 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.580+0000 7fc17f370640 1 -- 172.21.15.80:0/4217469043 wait complete. 2024-04-24T23:27:33.584 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.580+0000 7fc17f370640 1 Processor -- start 2024-04-24T23:27:33.584 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.580+0000 7fc17f370640 1 -- start start 2024-04-24T23:27:33.584 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.581+0000 7fc17f370640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc1781006a0 0x7fc17806a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:33.584 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.581+0000 7fc17f370640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc178101890 0x7fc17819c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:33.584 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.581+0000 7fc17f370640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc178102250 0x7fc17819ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:33.584 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.581+0000 7fc17f370640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fc17810c380 con 0x7fc178102250 2024-04-24T23:27:33.585 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.581+0000 7fc17f370640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc17810c200 con 0x7fc178101890 2024-04-24T23:27:33.585 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.581+0000 7fc17f370640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc17810c500 con 0x7fc1781006a0 2024-04-24T23:27:33.585 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.581+0000 7fc17d90d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc178102250 0x7fc17819ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:33.585 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.581+0000 7fc17d90d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc178102250 0x7fc17819ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:50524/0 (socket says 172.21.15.80:50524) 2024-04-24T23:27:33.585 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.581+0000 7fc17d90d640 1 -- 172.21.15.80:0/3805201232 learned_addr learned my addr 172.21.15.80:0/3805201232 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:27:33.585 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.581+0000 7fc17c90b640 1 --2- 172.21.15.80:0/3805201232 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc178101890 0x7fc17819c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:33.585 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.581+0000 7fc17d10c640 1 --2- 172.21.15.80:0/3805201232 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc1781006a0 0x7fc17806a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:33.585 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.581+0000 7fc17d90d640 1 -- 172.21.15.80:0/3805201232 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc1781006a0 msgr2=0x7fc17806a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:33.585 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.581+0000 7fc17d90d640 1 --2- 172.21.15.80:0/3805201232 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc1781006a0 0x7fc17806a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:33.585 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.581+0000 7fc17d90d640 1 -- 172.21.15.80:0/3805201232 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc178101890 msgr2=0x7fc17819c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:33.585 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.581+0000 7fc17d90d640 1 --2- 172.21.15.80:0/3805201232 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc178101890 0x7fc17819c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:33.585 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.581+0000 7fc17d90d640 1 -- 172.21.15.80:0/3805201232 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc17819f7a0 con 0x7fc178102250 2024-04-24T23:27:33.585 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.581+0000 7fc17d90d640 1 --2- 172.21.15.80:0/3805201232 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc178102250 0x7fc17819ee90 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fc17000b970 tx=0x7fc17000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:33.585 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.581+0000 7fc16e7fc640 1 -- 172.21.15.80:0/3805201232 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc170019070 con 0x7fc178102250 2024-04-24T23:27:33.586 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.581+0000 7fc16e7fc640 1 -- 172.21.15.80:0/3805201232 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc17000cb40 con 0x7fc178102250 2024-04-24T23:27:33.586 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.581+0000 7fc17f370640 1 -- 172.21.15.80:0/3805201232 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc17819fa30 con 0x7fc178102250 2024-04-24T23:27:33.586 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.581+0000 7fc16e7fc640 1 -- 172.21.15.80:0/3805201232 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc170007650 con 0x7fc178102250 2024-04-24T23:27:33.586 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.581+0000 7fc17f370640 1 -- 172.21.15.80:0/3805201232 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc17819ffd0 con 0x7fc178102250 2024-04-24T23:27:33.586 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.582+0000 7fc17f370640 1 -- 172.21.15.80:0/3805201232 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1781079c0 con 0x7fc178102250 2024-04-24T23:27:33.587 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.583+0000 7fc16e7fc640 1 -- 172.21.15.80:0/3805201232 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fc17001c070 con 0x7fc178102250 2024-04-24T23:27:33.587 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.584+0000 7fc16e7fc640 1 --2- 172.21.15.80:0/3805201232 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fc1640a2b90 0x7fc1640a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:33.587 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.584+0000 7fc16e7fc640 1 -- 172.21.15.80:0/3805201232 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fc1700cecd0 con 0x7fc178102250 2024-04-24T23:27:33.587 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.584+0000 7fc17d10c640 1 --2- 172.21.15.80:0/3805201232 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fc1640a2b90 0x7fc1640a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:33.587 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.584+0000 7fc17d10c640 1 --2- 172.21.15.80:0/3805201232 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fc1640a2b90 0x7fc1640a5050 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc168005fd0 tx=0x7fc168005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:33.588 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.588+0000 7fc16e7fc640 1 -- 172.21.15.80:0/3805201232 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fc17009b3a0 con 0x7fc178102250 2024-04-24T23:27:33.617 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:33 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/3519414971' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:27:33.617 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:33 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/3384028671' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T23:27:33.617 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:33 smithi134 ceph-mon[32619]: pgmap v303: 129 pgs: 129 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:27:33.618 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:33 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/177005380' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T23:27:33.756 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:33 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/3519414971' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:27:33.756 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:33 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/3384028671' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T23:27:33.756 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:33 smithi148 ceph-mon[32528]: pgmap v303: 129 pgs: 129 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:27:33.756 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:33 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/177005380' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T23:27:33.776 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.775+0000 7fc17f370640 1 -- 172.21.15.80:0/3805201232 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fc17802d170 con 0x7fc178102250 2024-04-24T23:27:33.776 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.775+0000 7fc16e7fc640 1 -- 172.21.15.80:0/3805201232 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v90) ==== 81+0+586 (secure 0 0 0) 0x7fc17009f6a0 con 0x7fc178102250 2024-04-24T23:27:33.776 INFO:teuthology.orchestra.run.smithi080.stdout:172.21.15.80:0/2828378749 2024-04-25T23:20:06.793609+0000 2024-04-24T23:27:33.777 INFO:teuthology.orchestra.run.smithi080.stdout:172.21.15.80:6801/233364790 2024-04-25T23:20:06.793609+0000 2024-04-24T23:27:33.777 INFO:teuthology.orchestra.run.smithi080.stdout:172.21.15.80:6800/1614691410 2024-04-25T23:19:40.490692+0000 2024-04-24T23:27:33.777 INFO:teuthology.orchestra.run.smithi080.stdout:172.21.15.80:0/1406187145 2024-04-25T23:20:06.793609+0000 2024-04-24T23:27:33.777 INFO:teuthology.orchestra.run.smithi080.stdout:172.21.15.80:0/92953396 2024-04-25T23:19:40.490692+0000 2024-04-24T23:27:33.777 INFO:teuthology.orchestra.run.smithi080.stdout:172.21.15.80:0/598129432 2024-04-25T23:20:06.793609+0000 2024-04-24T23:27:33.777 INFO:teuthology.orchestra.run.smithi080.stdout:172.21.15.80:0/232193118 2024-04-25T23:19:40.490692+0000 2024-04-24T23:27:33.777 INFO:teuthology.orchestra.run.smithi080.stdout:172.21.15.80:6800/233364790 2024-04-25T23:20:06.793609+0000 2024-04-24T23:27:33.777 INFO:teuthology.orchestra.run.smithi080.stdout:172.21.15.80:6801/1614691410 2024-04-25T23:19:40.490692+0000 2024-04-24T23:27:33.777 INFO:teuthology.orchestra.run.smithi080.stdout:172.21.15.80:0/2081562529 2024-04-25T23:19:40.490692+0000 2024-04-24T23:27:33.777 INFO:teuthology.orchestra.run.smithi080.stderr:listed 10 entries 2024-04-24T23:27:33.778 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.777+0000 7fc17f370640 1 -- 172.21.15.80:0/3805201232 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fc1640a2b90 msgr2=0x7fc1640a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:33.778 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.777+0000 7fc17f370640 1 --2- 172.21.15.80:0/3805201232 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fc1640a2b90 0x7fc1640a5050 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc168005fd0 tx=0x7fc168005850 comp rx=0 tx=0).stop 2024-04-24T23:27:33.779 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.777+0000 7fc17f370640 1 -- 172.21.15.80:0/3805201232 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc178102250 msgr2=0x7fc17819ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:33.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.777+0000 7fc17f370640 1 --2- 172.21.15.80:0/3805201232 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc178102250 0x7fc17819ee90 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fc17000b970 tx=0x7fc17000be30 comp rx=0 tx=0).stop 2024-04-24T23:27:33.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.777+0000 7fc17f370640 1 -- 172.21.15.80:0/3805201232 shutdown_connections 2024-04-24T23:27:33.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.777+0000 7fc17f370640 1 --2- 172.21.15.80:0/3805201232 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fc1640a2b90 0x7fc1640a5050 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:33.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.777+0000 7fc17f370640 1 --2- 172.21.15.80:0/3805201232 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc178102250 0x7fc17819ee90 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:33.783 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.777+0000 7fc17f370640 1 --2- 172.21.15.80:0/3805201232 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc178101890 0x7fc17819c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:33.783 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.777+0000 7fc17f370640 1 --2- 172.21.15.80:0/3805201232 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc1781006a0 0x7fc17806a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:33.783 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.777+0000 7fc17f370640 1 -- 172.21.15.80:0/3805201232 >> 172.21.15.80:0/3805201232 conn(0x7fc1780fbc30 msgr2=0x7fc178104b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:33.783 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.777+0000 7fc17f370640 1 -- 172.21.15.80:0/3805201232 shutdown_connections 2024-04-24T23:27:33.783 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.777+0000 7fc17f370640 1 -- 172.21.15.80:0/3805201232 wait complete. 2024-04-24T23:27:33.789 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-24T23:27:33.789 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:27:33.805 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:27:33.946 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.941+0000 7f59f8941640 1 -- 172.21.15.80:0/2613603563 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f59f4102230 msgr2=0x7f59f4108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:33.946 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.941+0000 7f59f8941640 1 --2- 172.21.15.80:0/2613603563 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f59f4102230 0x7f59f4108ac0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f59ec009ca0 tx=0x7f59ec02a8c0 comp rx=0 tx=0).stop 2024-04-24T23:27:33.946 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.941+0000 7f59f8941640 1 -- 172.21.15.80:0/2613603563 shutdown_connections 2024-04-24T23:27:33.946 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.941+0000 7f59f8941640 1 --2- 172.21.15.80:0/2613603563 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f59f4102230 0x7f59f4108ac0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:33.946 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.941+0000 7f59f8941640 1 --2- 172.21.15.80:0/2613603563 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f59f4101870 0x7f59f4101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:33.946 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.941+0000 7f59f8941640 1 --2- 172.21.15.80:0/2613603563 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59f4100680 0x7f59f4100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:33.946 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.941+0000 7f59f8941640 1 -- 172.21.15.80:0/2613603563 >> 172.21.15.80:0/2613603563 conn(0x7f59f40fbbf0 msgr2=0x7f59f40fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:33.947 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.941+0000 7f59f8941640 1 -- 172.21.15.80:0/2613603563 shutdown_connections 2024-04-24T23:27:33.947 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.941+0000 7f59f8941640 1 -- 172.21.15.80:0/2613603563 wait complete. 2024-04-24T23:27:33.947 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.942+0000 7f59f8941640 1 Processor -- start 2024-04-24T23:27:33.947 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.942+0000 7f59f8941640 1 -- start start 2024-04-24T23:27:33.947 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.942+0000 7f59f8941640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f59f4100680 0x7f59f4197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:33.947 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.942+0000 7f59f8941640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f59f4101870 0x7f59f4197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:33.947 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.942+0000 7f59f8941640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59f4102230 0x7f59f419f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:33.947 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.942+0000 7f59f8941640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f59f410c380 con 0x7f59f4100680 2024-04-24T23:27:33.947 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.942+0000 7f59f8941640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f59f410c200 con 0x7f59f4101870 2024-04-24T23:27:33.947 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.942+0000 7f59f8941640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f59f410c500 con 0x7f59f4102230 2024-04-24T23:27:33.947 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.942+0000 7f59f259c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f59f4100680 0x7f59f4197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:33.947 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.942+0000 7f59f259c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f59f4100680 0x7f59f4197ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:50534/0 (socket says 172.21.15.80:50534) 2024-04-24T23:27:33.947 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.942+0000 7f59f259c640 1 -- 172.21.15.80:0/2145429120 learned_addr learned my addr 172.21.15.80:0/2145429120 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:27:33.947 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.943+0000 7f59f1d9b640 1 --2- 172.21.15.80:0/2145429120 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f59f4101870 0x7f59f4197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:33.948 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.943+0000 7f59f2d9d640 1 --2- 172.21.15.80:0/2145429120 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59f4102230 0x7f59f419f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:33.948 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.943+0000 7f59f259c640 1 -- 172.21.15.80:0/2145429120 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59f4102230 msgr2=0x7f59f419f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:33.948 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.943+0000 7f59f259c640 1 --2- 172.21.15.80:0/2145429120 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59f4102230 0x7f59f419f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:33.948 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.943+0000 7f59f259c640 1 -- 172.21.15.80:0/2145429120 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f59f4101870 msgr2=0x7f59f4197ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:33.948 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.943+0000 7f59f259c640 1 --2- 172.21.15.80:0/2145429120 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f59f4101870 0x7f59f4197ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:33.948 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.943+0000 7f59f259c640 1 -- 172.21.15.80:0/2145429120 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59f419f6d0 con 0x7f59f4100680 2024-04-24T23:27:33.948 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.943+0000 7f59f1d9b640 1 --2- 172.21.15.80:0/2145429120 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f59f4101870 0x7f59f4197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:27:33.948 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.943+0000 7f59f259c640 1 --2- 172.21.15.80:0/2145429120 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f59f4100680 0x7f59f4197ab0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f59e400d950 tx=0x7f59e400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:33.948 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.943+0000 7f59e37fe640 1 -- 172.21.15.80:0/2145429120 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59e4017070 con 0x7f59f4100680 2024-04-24T23:27:33.948 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.943+0000 7f59f8941640 1 -- 172.21.15.80:0/2145429120 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59f419f9c0 con 0x7f59f4100680 2024-04-24T23:27:33.948 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.943+0000 7f59e37fe640 1 -- 172.21.15.80:0/2145429120 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59e4009c30 con 0x7f59f4100680 2024-04-24T23:27:33.948 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.943+0000 7f59e37fe640 1 -- 172.21.15.80:0/2145429120 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59e400f600 con 0x7f59f4100680 2024-04-24T23:27:33.948 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.943+0000 7f59f8941640 1 -- 172.21.15.80:0/2145429120 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59f419ff90 con 0x7f59f4100680 2024-04-24T23:27:33.948 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.944+0000 7f59f8941640 1 -- 172.21.15.80:0/2145429120 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59f41a0160 con 0x7f59f4100680 2024-04-24T23:27:33.948 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.945+0000 7f59e37fe640 1 -- 172.21.15.80:0/2145429120 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f59e401a070 con 0x7f59f4100680 2024-04-24T23:27:33.949 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.946+0000 7f59e37fe640 1 --2- 172.21.15.80:0/2145429120 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f59dc0a2b90 0x7f59dc0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:33.949 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.946+0000 7f59f1d9b640 1 --2- 172.21.15.80:0/2145429120 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f59dc0a2b90 0x7f59dc0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:33.949 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.946+0000 7f59e37fe640 1 -- 172.21.15.80:0/2145429120 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f59e40cedb0 con 0x7f59f4100680 2024-04-24T23:27:33.949 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.946+0000 7f59f1d9b640 1 --2- 172.21.15.80:0/2145429120 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f59dc0a2b90 0x7f59dc0a5050 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f59f4198fc0 tx=0x7f59e8008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:33.950 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:33.949+0000 7f59e37fe640 1 -- 172.21.15.80:0/2145429120 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f59e4009de0 con 0x7f59f4100680 2024-04-24T23:27:34.124 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.123+0000 7f59f8941640 1 -- 172.21.15.80:0/2145429120 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f59f41a04f0 con 0x7f59f4100680 2024-04-24T23:27:34.126 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.125+0000 7f59e37fe640 1 -- 172.21.15.80:0/2145429120 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24124 (secure 0 0 0) 0x7f59e409bad0 con 0x7f59f4100680 2024-04-24T23:27:34.126 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-24T23:27:34.126 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":90,"fsid":"ec7885aa-0290-11ef-bc93-c7b262605968","created":"2024-04-24T23:19:05.586407+0000","modified":"2024-04-24T23:27:26.212213+0000","last_up_change":"2024-04-24T23:26:25.938316+0000","last_in_change":"2024-04-24T23:26:04.786699+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:22:33.124428+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:27:17.575788+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:27:18.492762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T23:27:22.438503+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a525741d-b46c-4eef-aa64-3f057cac1497","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":2597718112},{"type":"v1","addr":"172.21.15.80:6803","nonce":2597718112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":2597718112},{"type":"v1","addr":"172.21.15.80:6805","nonce":2597718112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2597718112},{"type":"v1","addr":"172.21.15.80:6809","nonce":2597718112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":2597718112},{"type":"v1","addr":"172.21.15.80:6807","nonce":2597718112}]},"public_addr":"172.21.15.80:6803/2597718112","cluster_addr":"172.21.15.80:6805/2597718112","heartbeat_back_addr":"172.21.15.80:6809/2597718112","heartbeat_front_addr":"172.21.15.80:6807/2597718112","state":["exists","up"]},{"osd":1,"uuid":"2d0f0d67-6db4-40ac-91cc-58604ab6795a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":2395610201},{"type":"v1","addr":"172.21.15.134:6801","nonce":2395610201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":2395610201},{"type":"v1","addr":"172.21.15.134:6803","nonce":2395610201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":2395610201},{"type":"v1","addr":"172.21.15.134:6807","nonce":2395610201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":2395610201},{"type":"v1","addr":"172.21.15.134:6805","nonce":2395610201}]},"public_addr":"172.21.15.134:6801/2395610201","cluster_addr":"172.21.15.134:6803/2395610201","heartbeat_back_addr":"172.21.15.134:6807/2395610201","heartbeat_front_addr":"172.21.15.134:6805/2395610201","state":["exists","up"]},{"osd":2,"uuid":"781fb879-fedc-4c84-82fb-59e073bc4cf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1629152187},{"type":"v1","addr":"172.21.15.148:6801","nonce":1629152187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1629152187},{"type":"v1","addr":"172.21.15.148:6803","nonce":1629152187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1629152187},{"type":"v1","addr":"172.21.15.148:6807","nonce":1629152187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1629152187},{"type":"v1","addr":"172.21.15.148:6805","nonce":1629152187}]},"public_addr":"172.21.15.148:6801/1629152187","cluster_addr":"172.21.15.148:6803/1629152187","heartbeat_back_addr":"172.21.15.148:6807/1629152187","heartbeat_front_addr":"172.21.15.148:6805/1629152187","state":["exists","up"]},{"osd":3,"uuid":"cd982757-f2f6-4296-b3e7-4cd40fbd8d9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":3128298510},{"type":"v1","addr":"172.21.15.80:6811","nonce":3128298510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":3128298510},{"type":"v1","addr":"172.21.15.80:6813","nonce":3128298510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":3128298510},{"type":"v1","addr":"172.21.15.80:6817","nonce":3128298510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":3128298510},{"type":"v1","addr":"172.21.15.80:6815","nonce":3128298510}]},"public_addr":"172.21.15.80:6811/3128298510","cluster_addr":"172.21.15.80:6813/3128298510","heartbeat_back_addr":"172.21.15.80:6817/3128298510","heartbeat_front_addr":"172.21.15.80:6815/3128298510","state":["exists","up"]},{"osd":4,"uuid":"dfc55a79-a7e8-417a-b898-ec44171a54af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1917556060},{"type":"v1","addr":"172.21.15.134:6809","nonce":1917556060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1917556060},{"type":"v1","addr":"172.21.15.134:6811","nonce":1917556060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1917556060},{"type":"v1","addr":"172.21.15.134:6815","nonce":1917556060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1917556060},{"type":"v1","addr":"172.21.15.134:6813","nonce":1917556060}]},"public_addr":"172.21.15.134:6809/1917556060","cluster_addr":"172.21.15.134:6811/1917556060","heartbeat_back_addr":"172.21.15.134:6815/1917556060","heartbeat_front_addr":"172.21.15.134:6813/1917556060","state":["exists","up"]},{"osd":5,"uuid":"b5491337-0694-440c-b679-312f3d14cf65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":2715951281},{"type":"v1","addr":"172.21.15.148:6809","nonce":2715951281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":2715951281},{"type":"v1","addr":"172.21.15.148:6811","nonce":2715951281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":2715951281},{"type":"v1","addr":"172.21.15.148:6815","nonce":2715951281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":2715951281},{"type":"v1","addr":"172.21.15.148:6813","nonce":2715951281}]},"public_addr":"172.21.15.148:6809/2715951281","cluster_addr":"172.21.15.148:6811/2715951281","heartbeat_back_addr":"172.21.15.148:6815/2715951281","heartbeat_front_addr":"172.21.15.148:6813/2715951281","state":["exists","up"]},{"osd":6,"uuid":"ba4b5df7-d86e-47cd-ab67-cf763c4e4d6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":953261724},{"type":"v1","addr":"172.21.15.80:6819","nonce":953261724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":953261724},{"type":"v1","addr":"172.21.15.80:6821","nonce":953261724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":953261724},{"type":"v1","addr":"172.21.15.80:6825","nonce":953261724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":953261724},{"type":"v1","addr":"172.21.15.80:6823","nonce":953261724}]},"public_addr":"172.21.15.80:6819/953261724","cluster_addr":"172.21.15.80:6821/953261724","heartbeat_back_addr":"172.21.15.80:6825/953261724","heartbeat_front_addr":"172.21.15.80:6823/953261724","state":["exists","up"]},{"osd":7,"uuid":"464a503a-75ae-49e7-ab6e-a7f9f2a6a307","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":108354844},{"type":"v1","addr":"172.21.15.134:6817","nonce":108354844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":108354844},{"type":"v1","addr":"172.21.15.134:6819","nonce":108354844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":108354844},{"type":"v1","addr":"172.21.15.134:6823","nonce":108354844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":108354844},{"type":"v1","addr":"172.21.15.134:6821","nonce":108354844}]},"public_addr":"172.21.15.134:6817/108354844","cluster_addr":"172.21.15.134:6819/108354844","heartbeat_back_addr":"172.21.15.134:6823/108354844","heartbeat_front_addr":"172.21.15.134:6821/108354844","state":["exists","up"]},{"osd":8,"uuid":"ce544b2e-4540-47c0-9997-500c45351330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3559655329},{"type":"v1","addr":"172.21.15.148:6817","nonce":3559655329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":3559655329},{"type":"v1","addr":"172.21.15.148:6819","nonce":3559655329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":3559655329},{"type":"v1","addr":"172.21.15.148:6823","nonce":3559655329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3559655329},{"type":"v1","addr":"172.21.15.148:6821","nonce":3559655329}]},"public_addr":"172.21.15.148:6817/3559655329","cluster_addr":"172.21.15.148:6819/3559655329","heartbeat_back_addr":"172.21.15.148:6823/3559655329","heartbeat_front_addr":"172.21.15.148:6821/3559655329","state":["exists","up"]},{"osd":9,"uuid":"06de3732-19b9-496c-a464-55322efa6759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":4019913545},{"type":"v1","addr":"172.21.15.80:6827","nonce":4019913545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":4019913545},{"type":"v1","addr":"172.21.15.80:6829","nonce":4019913545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":4019913545},{"type":"v1","addr":"172.21.15.80:6833","nonce":4019913545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":4019913545},{"type":"v1","addr":"172.21.15.80:6831","nonce":4019913545}]},"public_addr":"172.21.15.80:6827/4019913545","cluster_addr":"172.21.15.80:6829/4019913545","heartbeat_back_addr":"172.21.15.80:6833/4019913545","heartbeat_front_addr":"172.21.15.80:6831/4019913545","state":["exists","up"]},{"osd":10,"uuid":"8d92f0d9-5c64-429a-b750-c33ae0e070c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":647343766},{"type":"v1","addr":"172.21.15.134:6825","nonce":647343766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":647343766},{"type":"v1","addr":"172.21.15.134:6827","nonce":647343766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":647343766},{"type":"v1","addr":"172.21.15.134:6831","nonce":647343766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":647343766},{"type":"v1","addr":"172.21.15.134:6829","nonce":647343766}]},"public_addr":"172.21.15.134:6825/647343766","cluster_addr":"172.21.15.134:6827/647343766","heartbeat_back_addr":"172.21.15.134:6831/647343766","heartbeat_front_addr":"172.21.15.134:6829/647343766","state":["exists","up"]},{"osd":11,"uuid":"4c003a41-f65b-479f-bc80-b27d680cf936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":4068024116},{"type":"v1","addr":"172.21.15.148:6825","nonce":4068024116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":4068024116},{"type":"v1","addr":"172.21.15.148:6827","nonce":4068024116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":4068024116},{"type":"v1","addr":"172.21.15.148:6831","nonce":4068024116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":4068024116},{"type":"v1","addr":"172.21.15.148:6829","nonce":4068024116}]},"public_addr":"172.21.15.148:6825/4068024116","cluster_addr":"172.21.15.148:6827/4068024116","heartbeat_back_addr":"172.21.15.148:6831/4068024116","heartbeat_front_addr":"172.21.15.148:6829/4068024116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:21:42.433631+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:22:06.145933+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:22:30.726143+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:22:55.729961+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:23:21.280913+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:23:46.609983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:24:12.585247+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:24:38.359287+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:25:04.491232+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:25:30.788089+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:25:56.891078+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:26:23.899113+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.80:0/2828378749":"2024-04-25T23:20:06.793609+0000","172.21.15.80:6801/233364790":"2024-04-25T23:20:06.793609+0000","172.21.15.80:6800/1614691410":"2024-04-25T23:19:40.490692+0000","172.21.15.80:0/1406187145":"2024-04-25T23:20:06.793609+0000","172.21.15.80:0/92953396":"2024-04-25T23:19:40.490692+0000","172.21.15.80:0/598129432":"2024-04-25T23:20:06.793609+0000","172.21.15.80:0/232193118":"2024-04-25T23:19:40.490692+0000","172.21.15.80:6800/233364790":"2024-04-25T23:20:06.793609+0000","172.21.15.80:6801/1614691410":"2024-04-25T23:19:40.490692+0000","172.21.15.80:0/2081562529":"2024-04-25T23:19:40.490692+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:27:34.128 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.127+0000 7f59f8941640 1 -- 172.21.15.80:0/2145429120 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f59dc0a2b90 msgr2=0x7f59dc0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:34.128 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.127+0000 7f59f8941640 1 --2- 172.21.15.80:0/2145429120 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f59dc0a2b90 0x7f59dc0a5050 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f59f4198fc0 tx=0x7f59e8008040 comp rx=0 tx=0).stop 2024-04-24T23:27:34.128 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.127+0000 7f59f8941640 1 -- 172.21.15.80:0/2145429120 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f59f4100680 msgr2=0x7f59f4197ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:34.128 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.127+0000 7f59f8941640 1 --2- 172.21.15.80:0/2145429120 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f59f4100680 0x7f59f4197ab0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f59e400d950 tx=0x7f59e400de10 comp rx=0 tx=0).stop 2024-04-24T23:27:34.128 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.127+0000 7f59f8941640 1 -- 172.21.15.80:0/2145429120 shutdown_connections 2024-04-24T23:27:34.128 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.127+0000 7f59f8941640 1 --2- 172.21.15.80:0/2145429120 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f59dc0a2b90 0x7f59dc0a5050 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.128 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.127+0000 7f59f8941640 1 --2- 172.21.15.80:0/2145429120 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59f4102230 0x7f59f419f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.129 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.127+0000 7f59f8941640 1 --2- 172.21.15.80:0/2145429120 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f59f4101870 0x7f59f4197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.129 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.127+0000 7f59f8941640 1 --2- 172.21.15.80:0/2145429120 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f59f4100680 0x7f59f4197ab0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.129 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.127+0000 7f59f8941640 1 -- 172.21.15.80:0/2145429120 >> 172.21.15.80:0/2145429120 conn(0x7f59f40fbbf0 msgr2=0x7f59f40fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:34.129 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.127+0000 7f59f8941640 1 -- 172.21.15.80:0/2145429120 shutdown_connections 2024-04-24T23:27:34.129 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.127+0000 7f59f8941640 1 -- 172.21.15.80:0/2145429120 wait complete. 2024-04-24T23:27:34.139 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:22:33.124428+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T23:27:17.575788+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T23:27:18.492762+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-24T23:27:22.438503+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-24T23:27:34.139 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:27:34.242 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.239+0000 7fe007683640 1 -- 172.21.15.80:0/111714495 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe000102230 msgr2=0x7fe000108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:34.243 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.239+0000 7fe007683640 1 --2- 172.21.15.80:0/111714495 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe000102230 0x7fe000108ac0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fdff8009ca0 tx=0x7fdff802a8c0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.243 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.239+0000 7fe007683640 1 -- 172.21.15.80:0/111714495 shutdown_connections 2024-04-24T23:27:34.243 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.239+0000 7fe007683640 1 --2- 172.21.15.80:0/111714495 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe000102230 0x7fe000108ac0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.243 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.239+0000 7fe007683640 1 --2- 172.21.15.80:0/111714495 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe000101870 0x7fe000101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.243 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.239+0000 7fe007683640 1 --2- 172.21.15.80:0/111714495 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe000100680 0x7fe000100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.243 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.239+0000 7fe007683640 1 -- 172.21.15.80:0/111714495 >> 172.21.15.80:0/111714495 conn(0x7fe0000fbbf0 msgr2=0x7fe0000fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:34.243 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.239+0000 7fe007683640 1 -- 172.21.15.80:0/111714495 shutdown_connections 2024-04-24T23:27:34.243 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.239+0000 7fe007683640 1 -- 172.21.15.80:0/111714495 wait complete. 2024-04-24T23:27:34.243 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.240+0000 7fe007683640 1 Processor -- start 2024-04-24T23:27:34.243 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.240+0000 7fe007683640 1 -- start start 2024-04-24T23:27:34.244 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.240+0000 7fe007683640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe000100680 0x7fe000069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:34.244 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.240+0000 7fe007683640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe000101870 0x7fe00019c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:34.244 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.240+0000 7fe007683640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe000102230 0x7fe00019ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:34.244 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.240+0000 7fe007683640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fe00010c300 con 0x7fe000101870 2024-04-24T23:27:34.244 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.240+0000 7fe007683640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe00010c180 con 0x7fe000100680 2024-04-24T23:27:34.244 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.240+0000 7fe007683640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe00010c480 con 0x7fe000102230 2024-04-24T23:27:34.244 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.240+0000 7fe004c1e640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe000101870 0x7fe00019c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:34.244 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.240+0000 7fe004c1e640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe000101870 0x7fe00019c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:50556/0 (socket says 172.21.15.80:50556) 2024-04-24T23:27:34.244 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.240+0000 7fe004c1e640 1 -- 172.21.15.80:0/976457589 learned_addr learned my addr 172.21.15.80:0/976457589 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:27:34.244 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.240+0000 7fe00541f640 1 --2- 172.21.15.80:0/976457589 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe000100680 0x7fe000069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:34.244 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.240+0000 7fe004c1e640 1 -- 172.21.15.80:0/976457589 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe000102230 msgr2=0x7fe00019ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:34.244 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.240+0000 7fe004c1e640 1 --2- 172.21.15.80:0/976457589 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe000102230 0x7fe00019ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.245 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.240+0000 7fe004c1e640 1 -- 172.21.15.80:0/976457589 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe000100680 msgr2=0x7fe000069460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:34.245 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.240+0000 7fe004c1e640 1 --2- 172.21.15.80:0/976457589 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe000100680 0x7fe000069460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.245 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.240+0000 7fe004c1e640 1 -- 172.21.15.80:0/976457589 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe00019f640 con 0x7fe000101870 2024-04-24T23:27:34.245 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.240+0000 7fe004c1e640 1 --2- 172.21.15.80:0/976457589 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe000101870 0x7fe00019c740 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fdff000c970 tx=0x7fdff000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:34.245 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.240+0000 7fdff67fc640 1 -- 172.21.15.80:0/976457589 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdff0007bb0 con 0x7fe000101870 2024-04-24T23:27:34.246 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.240+0000 7fdff67fc640 1 -- 172.21.15.80:0/976457589 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdff0007d50 con 0x7fe000101870 2024-04-24T23:27:34.246 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.240+0000 7fdff67fc640 1 -- 172.21.15.80:0/976457589 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdff000f450 con 0x7fe000101870 2024-04-24T23:27:34.246 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.240+0000 7fe007683640 1 -- 172.21.15.80:0/976457589 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe00019f930 con 0x7fe000101870 2024-04-24T23:27:34.246 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.241+0000 7fe007683640 1 -- 172.21.15.80:0/976457589 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe00019fe30 con 0x7fe000101870 2024-04-24T23:27:34.246 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.241+0000 7fe007683640 1 -- 172.21.15.80:0/976457589 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe000107940 con 0x7fe000101870 2024-04-24T23:27:34.247 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.242+0000 7fdff67fc640 1 -- 172.21.15.80:0/976457589 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fdff0005020 con 0x7fe000101870 2024-04-24T23:27:34.247 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.243+0000 7fdff67fc640 1 --2- 172.21.15.80:0/976457589 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fdfec0a2b90 0x7fdfec0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:34.247 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.243+0000 7fe00541f640 1 --2- 172.21.15.80:0/976457589 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fdfec0a2b90 0x7fdfec0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:34.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.243+0000 7fdff67fc640 1 -- 172.21.15.80:0/976457589 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fdff00cf110 con 0x7fe000101870 2024-04-24T23:27:34.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.243+0000 7fe00541f640 1 --2- 172.21.15.80:0/976457589 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fdfec0a2b90 0x7fdfec0a5050 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fdffc005fd0 tx=0x7fdffc005ce0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:34.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.247+0000 7fdff67fc640 1 -- 172.21.15.80:0/976457589 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fdff0014030 con 0x7fe000101870 2024-04-24T23:27:34.425 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.424+0000 7fe007683640 1 -- 172.21.15.80:0/976457589 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe000062340 con 0x7fe000101870 2024-04-24T23:27:34.426 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.425+0000 7fdff67fc640 1 -- 172.21.15.80:0/976457589 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) ==== 93+0+10 (secure 0 0 0) 0x7fdff009be30 con 0x7fe000101870 2024-04-24T23:27:34.426 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 1 2024-04-24T23:27:34.428 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.427+0000 7fe007683640 1 -- 172.21.15.80:0/976457589 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fdfec0a2b90 msgr2=0x7fdfec0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:34.428 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.427+0000 7fe007683640 1 --2- 172.21.15.80:0/976457589 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fdfec0a2b90 0x7fdfec0a5050 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fdffc005fd0 tx=0x7fdffc005ce0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.428 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.427+0000 7fe007683640 1 -- 172.21.15.80:0/976457589 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe000101870 msgr2=0x7fe00019c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:34.428 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.427+0000 7fe007683640 1 --2- 172.21.15.80:0/976457589 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe000101870 0x7fe00019c740 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fdff000c970 tx=0x7fdff000ce30 comp rx=0 tx=0).stop 2024-04-24T23:27:34.428 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.427+0000 7fe007683640 1 -- 172.21.15.80:0/976457589 shutdown_connections 2024-04-24T23:27:34.428 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.427+0000 7fe007683640 1 --2- 172.21.15.80:0/976457589 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fdfec0a2b90 0x7fdfec0a5050 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.428 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.427+0000 7fe007683640 1 --2- 172.21.15.80:0/976457589 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe000102230 0x7fe00019ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.428 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.427+0000 7fe007683640 1 --2- 172.21.15.80:0/976457589 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe000101870 0x7fe00019c740 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.428 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.427+0000 7fe007683640 1 --2- 172.21.15.80:0/976457589 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe000100680 0x7fe000069460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.428 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.427+0000 7fe007683640 1 -- 172.21.15.80:0/976457589 >> 172.21.15.80:0/976457589 conn(0x7fe0000fbbf0 msgr2=0x7fe000104b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:34.428 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.427+0000 7fe007683640 1 -- 172.21.15.80:0/976457589 shutdown_connections 2024-04-24T23:27:34.428 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.427+0000 7fe007683640 1 -- 172.21.15.80:0/976457589 wait complete. 2024-04-24T23:27:34.437 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:34 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/3874849520' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T23:27:34.437 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:34 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/3805201232' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-24T23:27:34.438 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:34 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/2145429120' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:27:34.439 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T23:27:34.543 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.539+0000 7fad5527d640 1 -- 172.21.15.80:0/2397381113 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fad50100f60 msgr2=0x7fad501013e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:34.543 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.539+0000 7fad5527d640 1 --2- 172.21.15.80:0/2397381113 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fad50100f60 0x7fad501013e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fad44009900 tx=0x7fad4402b520 comp rx=0 tx=0).stop 2024-04-24T23:27:34.543 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.539+0000 7fad5527d640 1 -- 172.21.15.80:0/2397381113 shutdown_connections 2024-04-24T23:27:34.543 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.539+0000 7fad5527d640 1 --2- 172.21.15.80:0/2397381113 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad50101920 0x7fad50108240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.543 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.539+0000 7fad5527d640 1 --2- 172.21.15.80:0/2397381113 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fad50100f60 0x7fad501013e0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.543 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.539+0000 7fad5527d640 1 --2- 172.21.15.80:0/2397381113 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fad500ffe10 0x7fad50100230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.543 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.539+0000 7fad5527d640 1 -- 172.21.15.80:0/2397381113 >> 172.21.15.80:0/2397381113 conn(0x7fad500fb380 msgr2=0x7fad500fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:34.543 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.540+0000 7fad5527d640 1 -- 172.21.15.80:0/2397381113 shutdown_connections 2024-04-24T23:27:34.543 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.540+0000 7fad5527d640 1 -- 172.21.15.80:0/2397381113 wait complete. 2024-04-24T23:27:34.544 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.540+0000 7fad5527d640 1 Processor -- start 2024-04-24T23:27:34.544 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.540+0000 7fad5527d640 1 -- start start 2024-04-24T23:27:34.544 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.540+0000 7fad5527d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad500ffe10 0x7fad50197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:34.544 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.540+0000 7fad5527d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fad50100f60 0x7fad501980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:34.544 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.540+0000 7fad5527d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fad50101920 0x7fad5019f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:34.544 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.540+0000 7fad5527d640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fad5010b300 con 0x7fad50100f60 2024-04-24T23:27:34.544 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.540+0000 7fad5527d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fad5010b180 con 0x7fad50101920 2024-04-24T23:27:34.544 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.540+0000 7fad5527d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fad5010b480 con 0x7fad500ffe10 2024-04-24T23:27:34.544 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.541+0000 7fad4e7fc640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fad50100f60 0x7fad501980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:34.544 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.541+0000 7fad4e7fc640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fad50100f60 0x7fad501980c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:50568/0 (socket says 172.21.15.80:50568) 2024-04-24T23:27:34.544 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.541+0000 7fad4e7fc640 1 -- 172.21.15.80:0/2726121218 learned_addr learned my addr 172.21.15.80:0/2726121218 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:27:34.544 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.541+0000 7fad4effd640 1 --2- 172.21.15.80:0/2726121218 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad500ffe10 0x7fad50197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:34.544 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.541+0000 7fad4f7fe640 1 --2- 172.21.15.80:0/2726121218 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fad50101920 0x7fad5019f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:34.545 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.541+0000 7fad4e7fc640 1 -- 172.21.15.80:0/2726121218 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad500ffe10 msgr2=0x7fad50197b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:34.545 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.541+0000 7fad4e7fc640 1 --2- 172.21.15.80:0/2726121218 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad500ffe10 0x7fad50197b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.545 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.541+0000 7fad4e7fc640 1 -- 172.21.15.80:0/2726121218 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fad50101920 msgr2=0x7fad5019f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:34.545 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.541+0000 7fad4e7fc640 1 --2- 172.21.15.80:0/2726121218 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fad50101920 0x7fad5019f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.545 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.541+0000 7fad4e7fc640 1 -- 172.21.15.80:0/2726121218 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad5019f7a0 con 0x7fad50100f60 2024-04-24T23:27:34.545 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.541+0000 7fad4f7fe640 1 --2- 172.21.15.80:0/2726121218 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fad50101920 0x7fad5019f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:27:34.545 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.541+0000 7fad4e7fc640 1 --2- 172.21.15.80:0/2726121218 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fad50100f60 0x7fad501980c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fad44032440 tx=0x7fad44033c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:34.545 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.541+0000 7fad37fff640 1 -- 172.21.15.80:0/2726121218 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad44041070 con 0x7fad50100f60 2024-04-24T23:27:34.545 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.541+0000 7fad37fff640 1 -- 172.21.15.80:0/2726121218 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad44003cd0 con 0x7fad50100f60 2024-04-24T23:27:34.545 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.541+0000 7fad5527d640 1 -- 172.21.15.80:0/2726121218 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad5019fa30 con 0x7fad50100f60 2024-04-24T23:27:34.545 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.541+0000 7fad37fff640 1 -- 172.21.15.80:0/2726121218 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad44036040 con 0x7fad50100f60 2024-04-24T23:27:34.545 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.541+0000 7fad5527d640 1 -- 172.21.15.80:0/2726121218 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad5019ffa0 con 0x7fad50100f60 2024-04-24T23:27:34.546 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.542+0000 7fad5527d640 1 -- 172.21.15.80:0/2726121218 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad501a01d0 con 0x7fad50100f60 2024-04-24T23:27:34.546 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.543+0000 7fad37fff640 1 -- 172.21.15.80:0/2726121218 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fad44044070 con 0x7fad50100f60 2024-04-24T23:27:34.546 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.544+0000 7fad37fff640 1 --2- 172.21.15.80:0/2726121218 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fad380a2be0 0x7fad380a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:34.546 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.544+0000 7fad4effd640 1 --2- 172.21.15.80:0/2726121218 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fad380a2be0 0x7fad380a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:34.546 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.544+0000 7fad37fff640 1 -- 172.21.15.80:0/2726121218 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fad440ef710 con 0x7fad50100f60 2024-04-24T23:27:34.547 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.544+0000 7fad4effd640 1 --2- 172.21.15.80:0/2726121218 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fad380a2be0 0x7fad380a50a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fad40009990 tx=0x7fad40009110 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:34.549 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.548+0000 7fad37fff640 1 -- 172.21.15.80:0/2726121218 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fad440bc550 con 0x7fad50100f60 2024-04-24T23:27:34.697 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:34 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/3874849520' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T23:27:34.697 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:34 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/3805201232' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-24T23:27:34.697 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:34 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/2145429120' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:27:34.722 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.721+0000 7fad5527d640 1 -- 172.21.15.80:0/2726121218 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fad501a03a0 con 0x7fad50100f60 2024-04-24T23:27:34.723 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.722+0000 7fad37fff640 1 -- 172.21.15.80:0/2726121218 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) ==== 104+0+11 (secure 0 0 0) 0x7fad440bbe50 con 0x7fad50100f60 2024-04-24T23:27:34.723 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 32 2024-04-24T23:27:34.725 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.723+0000 7fad5527d640 1 -- 172.21.15.80:0/2726121218 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fad380a2be0 msgr2=0x7fad380a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:34.725 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.723+0000 7fad5527d640 1 --2- 172.21.15.80:0/2726121218 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fad380a2be0 0x7fad380a50a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fad40009990 tx=0x7fad40009110 comp rx=0 tx=0).stop 2024-04-24T23:27:34.725 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.723+0000 7fad5527d640 1 -- 172.21.15.80:0/2726121218 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fad50100f60 msgr2=0x7fad501980c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:34.725 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.724+0000 7fad5527d640 1 --2- 172.21.15.80:0/2726121218 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fad50100f60 0x7fad501980c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fad44032440 tx=0x7fad44033c60 comp rx=0 tx=0).stop 2024-04-24T23:27:34.725 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.724+0000 7fad5527d640 1 -- 172.21.15.80:0/2726121218 shutdown_connections 2024-04-24T23:27:34.725 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.724+0000 7fad5527d640 1 --2- 172.21.15.80:0/2726121218 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fad380a2be0 0x7fad380a50a0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.725 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.724+0000 7fad5527d640 1 --2- 172.21.15.80:0/2726121218 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fad50101920 0x7fad5019f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.725 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.724+0000 7fad5527d640 1 --2- 172.21.15.80:0/2726121218 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fad50100f60 0x7fad501980c0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.725 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.724+0000 7fad5527d640 1 --2- 172.21.15.80:0/2726121218 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad500ffe10 0x7fad50197b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.725 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.724+0000 7fad5527d640 1 -- 172.21.15.80:0/2726121218 >> 172.21.15.80:0/2726121218 conn(0x7fad500fb380 msgr2=0x7fad50106500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:34.726 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.724+0000 7fad5527d640 1 -- 172.21.15.80:0/2726121218 shutdown_connections 2024-04-24T23:27:34.726 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.724+0000 7fad5527d640 1 -- 172.21.15.80:0/2726121218 wait complete. 2024-04-24T23:27:34.736 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T23:27:34.756 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:34 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/3874849520' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T23:27:34.756 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:34 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/3805201232' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-24T23:27:34.756 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:34 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/2145429120' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:27:34.839 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.836+0000 7f766e031640 1 -- 172.21.15.80:0/1539816543 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7668102250 msgr2=0x7f7668108ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:34.839 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.836+0000 7f766e031640 1 --2- 172.21.15.80:0/1539816543 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7668102250 0x7f7668108ae0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f7660009ca0 tx=0x7f766002a8c0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.839 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.836+0000 7f766e031640 1 -- 172.21.15.80:0/1539816543 shutdown_connections 2024-04-24T23:27:34.839 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.836+0000 7f766e031640 1 --2- 172.21.15.80:0/1539816543 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7668102250 0x7f7668108ae0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.839 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.836+0000 7f766e031640 1 --2- 172.21.15.80:0/1539816543 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7668101890 0x7f7668101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.839 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.836+0000 7f766e031640 1 --2- 172.21.15.80:0/1539816543 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76681006a0 0x7f7668100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.840 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.836+0000 7f766e031640 1 -- 172.21.15.80:0/1539816543 >> 172.21.15.80:0/1539816543 conn(0x7f76680fbc10 msgr2=0x7f76680fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:34.840 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.836+0000 7f766e031640 1 -- 172.21.15.80:0/1539816543 shutdown_connections 2024-04-24T23:27:34.840 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.836+0000 7f766e031640 1 -- 172.21.15.80:0/1539816543 wait complete. 2024-04-24T23:27:34.840 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.836+0000 7f766e031640 1 Processor -- start 2024-04-24T23:27:34.840 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.837+0000 7f766e031640 1 -- start start 2024-04-24T23:27:34.840 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.837+0000 7f766e031640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76681006a0 0x7f7668197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:34.840 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.837+0000 7f766e031640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7668101890 0x7f7668197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:34.840 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.837+0000 7f766e031640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7668102250 0x7f766819f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:34.840 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.837+0000 7f766e031640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f766810d480 con 0x7f7668102250 2024-04-24T23:27:34.840 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.837+0000 7f766e031640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f766810d300 con 0x7f76681006a0 2024-04-24T23:27:34.840 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.837+0000 7f766e031640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f766810d600 con 0x7f7668101890 2024-04-24T23:27:34.840 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.837+0000 7f7667fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7668102250 0x7f766819f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:34.840 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.837+0000 7f7667fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7668102250 0x7f766819f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:50590/0 (socket says 172.21.15.80:50590) 2024-04-24T23:27:34.840 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.837+0000 7f7667fff640 1 -- 172.21.15.80:0/1253156776 learned_addr learned my addr 172.21.15.80:0/1253156776 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:27:34.841 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.837+0000 7f76677fe640 1 --2- 172.21.15.80:0/1253156776 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76681006a0 0x7f7668197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:34.841 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.837+0000 7f7666ffd640 1 --2- 172.21.15.80:0/1253156776 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7668101890 0x7f7668197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:34.841 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.837+0000 7f7667fff640 1 -- 172.21.15.80:0/1253156776 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7668101890 msgr2=0x7f7668197fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:34.841 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.837+0000 7f7667fff640 1 --2- 172.21.15.80:0/1253156776 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7668101890 0x7f7668197fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.841 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.837+0000 7f7667fff640 1 -- 172.21.15.80:0/1253156776 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76681006a0 msgr2=0x7f7668197aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:34.841 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.837+0000 7f7667fff640 1 --2- 172.21.15.80:0/1253156776 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76681006a0 0x7f7668197aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:34.841 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.837+0000 7f7667fff640 1 -- 172.21.15.80:0/1253156776 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f766819f6c0 con 0x7f7668102250 2024-04-24T23:27:34.841 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.837+0000 7f7667fff640 1 --2- 172.21.15.80:0/1253156776 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7668102250 0x7f766819f050 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f7660005e30 tx=0x7f7660031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:34.841 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.837+0000 7f7664ff9640 1 -- 172.21.15.80:0/1253156776 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7660003c30 con 0x7f7668102250 2024-04-24T23:27:34.841 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.838+0000 7f7664ff9640 1 -- 172.21.15.80:0/1253156776 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7660003dd0 con 0x7f7668102250 2024-04-24T23:27:34.841 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.838+0000 7f766e031640 1 -- 172.21.15.80:0/1253156776 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f766819f950 con 0x7f7668102250 2024-04-24T23:27:34.841 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.838+0000 7f7664ff9640 1 -- 172.21.15.80:0/1253156776 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f766002fde0 con 0x7f7668102250 2024-04-24T23:27:34.841 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.838+0000 7f766e031640 1 -- 172.21.15.80:0/1253156776 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f766819ff20 con 0x7f7668102250 2024-04-24T23:27:34.842 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.838+0000 7f766e031640 1 -- 172.21.15.80:0/1253156776 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76681079c0 con 0x7f7668102250 2024-04-24T23:27:34.842 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.839+0000 7f7664ff9640 1 -- 172.21.15.80:0/1253156776 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f7660004050 con 0x7f7668102250 2024-04-24T23:27:34.842 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.840+0000 7f7664ff9640 1 --2- 172.21.15.80:0/1253156776 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f76500a2b90 0x7f76500a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:34.843 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.840+0000 7f76677fe640 1 --2- 172.21.15.80:0/1253156776 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f76500a2b90 0x7f76500a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:34.843 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.840+0000 7f7664ff9640 1 -- 172.21.15.80:0/1253156776 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f76600f06a0 con 0x7f7668102250 2024-04-24T23:27:34.843 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.841+0000 7f76677fe640 1 --2- 172.21.15.80:0/1253156776 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f76500a2b90 0x7f76500a5050 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f76681016f0 tx=0x7f765800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:34.845 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:34.844+0000 7f7664ff9640 1 -- 172.21.15.80:0/1253156776 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f76600bd3c0 con 0x7f7668102250 2024-04-24T23:27:35.024 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:35.023+0000 7f766e031640 1 -- 172.21.15.80:0/1253156776 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f766802d390 con 0x7f7668102250 2024-04-24T23:27:35.025 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:35.024+0000 7f7664ff9640 1 -- 172.21.15.80:0/1253156776 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) ==== 100+0+11 (secure 0 0 0) 0x7f76600bccc0 con 0x7f7668102250 2024-04-24T23:27:35.025 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 64 2024-04-24T23:27:35.027 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:35.026+0000 7f766e031640 1 -- 172.21.15.80:0/1253156776 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f76500a2b90 msgr2=0x7f76500a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:35.027 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:35.026+0000 7f766e031640 1 --2- 172.21.15.80:0/1253156776 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f76500a2b90 0x7f76500a5050 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f76681016f0 tx=0x7f765800b390 comp rx=0 tx=0).stop 2024-04-24T23:27:35.027 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:35.026+0000 7f766e031640 1 -- 172.21.15.80:0/1253156776 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7668102250 msgr2=0x7f766819f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:35.027 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:35.026+0000 7f766e031640 1 --2- 172.21.15.80:0/1253156776 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7668102250 0x7f766819f050 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f7660005e30 tx=0x7f7660031ee0 comp rx=0 tx=0).stop 2024-04-24T23:27:35.028 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:35.026+0000 7f766e031640 1 -- 172.21.15.80:0/1253156776 shutdown_connections 2024-04-24T23:27:35.028 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:35.026+0000 7f766e031640 1 --2- 172.21.15.80:0/1253156776 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f76500a2b90 0x7f76500a5050 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:35.028 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:35.026+0000 7f766e031640 1 --2- 172.21.15.80:0/1253156776 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7668102250 0x7f766819f050 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:35.028 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:35.026+0000 7f766e031640 1 --2- 172.21.15.80:0/1253156776 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7668101890 0x7f7668197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:35.028 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:35.026+0000 7f766e031640 1 --2- 172.21.15.80:0/1253156776 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76681006a0 0x7f7668197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:35.028 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:35.026+0000 7f766e031640 1 -- 172.21.15.80:0/1253156776 >> 172.21.15.80:0/1253156776 conn(0x7f76680fbc10 msgr2=0x7f7668104b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:35.028 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:35.026+0000 7f766e031640 1 -- 172.21.15.80:0/1253156776 shutdown_connections 2024-04-24T23:27:35.028 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:35.026+0000 7f766e031640 1 -- 172.21.15.80:0/1253156776 wait complete. 2024-04-24T23:27:35.038 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-24T23:27:35.141 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:35.138+0000 7f03298ca640 1 -- 172.21.15.80:0/2482356484 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0324101880 msgr2=0x7f0324101d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:35.141 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:35.138+0000 7f03298ca640 1 --2- 172.21.15.80:0/2482356484 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0324101880 0x7f0324101d00 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f030c009900 tx=0x7f030c02a410 comp rx=0 tx=0).stop 2024-04-24T23:27:35.141 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:35.138+0000 7f03298ca640 1 -- 172.21.15.80:0/2482356484 shutdown_connections 2024-04-24T23:27:35.141 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:35.138+0000 7f03298ca640 1 --2- 172.21.15.80:0/2482356484 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0324102240 0x7f0324108ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:35.141 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:35.138+0000 7f03298ca640 1 --2- 172.21.15.80:0/2482356484 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0324101880 0x7f0324101d00 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:35.141 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:35.138+0000 7f03298ca640 1 --2- 172.21.15.80:0/2482356484 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0324100690 0x7f0324100ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:35.142 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:35.138+0000 7f03298ca640 1 -- 172.21.15.80:0/2482356484 >> 172.21.15.80:0/2482356484 conn(0x7f03240fbc00 msgr2=0x7f03240fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:35.142 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:35.138+0000 7f03298ca640 1 -- 172.21.15.80:0/2482356484 shutdown_connections 2024-04-24T23:27:35.142 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:35.138+0000 7f03298ca640 1 -- 172.21.15.80:0/2482356484 wait complete. 2024-04-24T23:27:35.142 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:35.138+0000 7f03298ca640 1 Processor -- start 2024-04-24T23:27:35.142 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:35.138+0000 7f03298ca640 1 -- start start 2024-04-24T23:27:35.142 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:35.139+0000 7f03298ca640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0324100690 0x7f032410b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:35.142 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:35.139+0000 7f03298ca640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0324101880 0x7f0324109650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:35.142 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:35.139+0000 7f03298ca640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0324102240 0x7f0324109b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:35.142 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:35.139+0000 7f03298ca640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f032410d430 con 0x7f0324100690 2024-04-24T23:27:35.142 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:35.139+0000 7f03298ca640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f032410d2b0 con 0x7f0324101880 2024-04-24T23:27:35.142 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:35.139+0000 7f03298ca640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f032410d5b0 con 0x7f0324102240 2024-04-24T23:27:35.142 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:35.139+0000 7f0322ffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0324100690 0x7f032410b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:35.142 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:35.139+0000 7f0322ffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0324100690 0x7f032410b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:50604/0 (socket says 172.21.15.80:50604) 2024-04-24T23:27:35.142 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:35.139+0000 7f0322ffd640 1 -- 172.21.15.80:0/3449959519 learned_addr learned my addr 172.21.15.80:0/3449959519 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:27:35.143 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:35.139+0000 7f03227fc640 1 --2- 172.21.15.80:0/3449959519 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0324101880 0x7f0324109650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:35.143 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:35.139+0000 7f0322ffd640 1 -- 172.21.15.80:0/3449959519 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0324102240 msgr2=0x7f0324109b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:35.143 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:35.139+0000 7f0322ffd640 1 --2- 172.21.15.80:0/3449959519 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0324102240 0x7f0324109b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:35.143 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:35.139+0000 7f0322ffd640 1 -- 172.21.15.80:0/3449959519 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0324101880 msgr2=0x7f0324109650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:35.143 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:35.139+0000 7f0322ffd640 1 --2- 172.21.15.80:0/3449959519 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0324101880 0x7f0324109650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:35.143 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:35.139+0000 7f0322ffd640 1 -- 172.21.15.80:0/3449959519 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f032410a420 con 0x7f0324100690 2024-04-24T23:27:35.143 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:35.139+0000 7f03227fc640 1 --2- 172.21.15.80:0/3449959519 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0324101880 0x7f0324109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:27:35.143 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:35.139+0000 7f0322ffd640 1 --2- 172.21.15.80:0/3449959519 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0324100690 0x7f032410b010 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f031400d950 tx=0x7f031400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:35.143 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:35.139+0000 7f03288c8640 1 -- 172.21.15.80:0/3449959519 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0314017070 con 0x7f0324100690 2024-04-24T23:27:35.143 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:35.139+0000 7f03288c8640 1 -- 172.21.15.80:0/3449959519 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0314009c30 con 0x7f0324100690 2024-04-24T23:27:35.143 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:35.139+0000 7f03288c8640 1 -- 172.21.15.80:0/3449959519 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f031400f600 con 0x7f0324100690 2024-04-24T23:27:35.143 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:35.139+0000 7f03298ca640 1 -- 172.21.15.80:0/3449959519 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03241a1bf0 con 0x7f0324100690 2024-04-24T23:27:35.143 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:35.139+0000 7f03298ca640 1 -- 172.21.15.80:0/3449959519 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03241a2110 con 0x7f0324100690 2024-04-24T23:27:35.144 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:35.140+0000 7f03298ca640 1 -- 172.21.15.80:0/3449959519 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03241079b0 con 0x7f0324100690 2024-04-24T23:27:35.144 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:35.141+0000 7f03288c8640 1 -- 172.21.15.80:0/3449959519 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f031401a070 con 0x7f0324100690 2024-04-24T23:27:35.144 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:35.142+0000 7f03288c8640 1 --2- 172.21.15.80:0/3449959519 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f03100a2c10 0x7f03100a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:35.144 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:35.142+0000 7f03227fc640 1 --2- 172.21.15.80:0/3449959519 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f03100a2c10 0x7f03100a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:35.145 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:35.142+0000 7f03288c8640 1 -- 172.21.15.80:0/3449959519 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f03140ced10 con 0x7f0324100690 2024-04-24T23:27:35.145 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:35.142+0000 7f03227fc640 1 --2- 172.21.15.80:0/3449959519 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f03100a2c10 0x7f03100a50d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f030c000c00 tx=0x7f030c0095f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:35.147 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:35.146+0000 7f03288c8640 1 -- 172.21.15.80:0/3449959519 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f0314009de0 con 0x7f0324100690 2024-04-24T23:27:35.323 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:35.321+0000 7f03298ca640 1 -- 172.21.15.80:0/3449959519 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f03240623e0 con 0x7f0324100690 2024-04-24T23:27:35.323 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:35.322+0000 7f03288c8640 1 -- 172.21.15.80:0/3449959519 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v90) ==== 103+0+11 (secure 0 0 0) 0x7f031409ba30 con 0x7f0324100690 2024-04-24T23:27:35.323 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 32 2024-04-24T23:27:35.326 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:35.324+0000 7f03298ca640 1 -- 172.21.15.80:0/3449959519 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f03100a2c10 msgr2=0x7f03100a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:35.326 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:35.324+0000 7f03298ca640 1 --2- 172.21.15.80:0/3449959519 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f03100a2c10 0x7f03100a50d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f030c000c00 tx=0x7f030c0095f0 comp rx=0 tx=0).stop 2024-04-24T23:27:35.326 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:35.324+0000 7f03298ca640 1 -- 172.21.15.80:0/3449959519 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0324100690 msgr2=0x7f032410b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:35.326 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:35.324+0000 7f03298ca640 1 --2- 172.21.15.80:0/3449959519 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0324100690 0x7f032410b010 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f031400d950 tx=0x7f031400de10 comp rx=0 tx=0).stop 2024-04-24T23:27:35.327 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:35.324+0000 7f03298ca640 1 -- 172.21.15.80:0/3449959519 shutdown_connections 2024-04-24T23:27:35.327 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:35.324+0000 7f03298ca640 1 --2- 172.21.15.80:0/3449959519 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f03100a2c10 0x7f03100a50d0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:35.327 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:35.324+0000 7f03298ca640 1 --2- 172.21.15.80:0/3449959519 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0324102240 0x7f0324109b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:35.327 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:35.324+0000 7f03298ca640 1 --2- 172.21.15.80:0/3449959519 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0324101880 0x7f0324109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:35.327 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:35.324+0000 7f03298ca640 1 --2- 172.21.15.80:0/3449959519 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0324100690 0x7f032410b010 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:35.327 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:35.324+0000 7f03298ca640 1 -- 172.21.15.80:0/3449959519 >> 172.21.15.80:0/3449959519 conn(0x7f03240fbc00 msgr2=0x7f0324104b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:35.327 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:35.324+0000 7f03298ca640 1 -- 172.21.15.80:0/3449959519 shutdown_connections 2024-04-24T23:27:35.327 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:35.324+0000 7f03298ca640 1 -- 172.21.15.80:0/3449959519 wait complete. 2024-04-24T23:27:35.336 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-24T23:27:35.438 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:35.435+0000 7f45d7d90640 1 -- 172.21.15.80:0/508584446 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f45d0102250 msgr2=0x7f45d0108ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:35.438 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:35.435+0000 7f45d7d90640 1 --2- 172.21.15.80:0/508584446 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f45d0102250 0x7f45d0108ae0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f45c0009900 tx=0x7f45c002a410 comp rx=0 tx=0).stop 2024-04-24T23:27:35.439 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:35.435+0000 7f45d7d90640 1 -- 172.21.15.80:0/508584446 shutdown_connections 2024-04-24T23:27:35.439 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:35.435+0000 7f45d7d90640 1 --2- 172.21.15.80:0/508584446 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f45d0102250 0x7f45d0108ae0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:35.439 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:35.435+0000 7f45d7d90640 1 --2- 172.21.15.80:0/508584446 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f45d0101890 0x7f45d0101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:35.439 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:35.435+0000 7f45d7d90640 1 --2- 172.21.15.80:0/508584446 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f45d01006a0 0x7f45d0100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:35.439 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:35.435+0000 7f45d7d90640 1 -- 172.21.15.80:0/508584446 >> 172.21.15.80:0/508584446 conn(0x7f45d00fbc30 msgr2=0x7f45d00fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:35.439 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:35.435+0000 7f45d7d90640 1 -- 172.21.15.80:0/508584446 shutdown_connections 2024-04-24T23:27:35.439 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:35.435+0000 7f45d7d90640 1 -- 172.21.15.80:0/508584446 wait complete. 2024-04-24T23:27:35.439 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:35.435+0000 7f45d7d90640 1 Processor -- start 2024-04-24T23:27:35.439 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:35.435+0000 7f45d7d90640 1 -- start start 2024-04-24T23:27:35.439 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:35.435+0000 7f45d7d90640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f45d01006a0 0x7f45d0197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:35.439 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:35.435+0000 7f45d7d90640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f45d0101890 0x7f45d01980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:35.439 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:35.436+0000 7f45d7d90640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f45d0102250 0x7f45d019f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:35.440 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:35.436+0000 7f45d7d90640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f45d010c380 con 0x7f45d0102250 2024-04-24T23:27:35.440 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:35.436+0000 7f45d7d90640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f45d010c200 con 0x7f45d0101890 2024-04-24T23:27:35.440 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:35.436+0000 7f45d7d90640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f45d010c500 con 0x7f45d01006a0 2024-04-24T23:27:35.440 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:35.436+0000 7f45d632d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f45d0102250 0x7f45d019f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:35.440 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:35.436+0000 7f45d632d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f45d0102250 0x7f45d019f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:50616/0 (socket says 172.21.15.80:50616) 2024-04-24T23:27:35.440 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:35.436+0000 7f45d632d640 1 -- 172.21.15.80:0/1298664909 learned_addr learned my addr 172.21.15.80:0/1298664909 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:27:35.440 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:35.436+0000 7f45d532b640 1 --2- 172.21.15.80:0/1298664909 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f45d0101890 0x7f45d01980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:35.440 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:35.436+0000 7f45d5b2c640 1 --2- 172.21.15.80:0/1298664909 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f45d01006a0 0x7f45d0197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:35.440 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:35.436+0000 7f45d632d640 1 -- 172.21.15.80:0/1298664909 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f45d01006a0 msgr2=0x7f45d0197b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:35.440 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:35.436+0000 7f45d632d640 1 --2- 172.21.15.80:0/1298664909 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f45d01006a0 0x7f45d0197b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:35.440 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:35.436+0000 7f45d632d640 1 -- 172.21.15.80:0/1298664909 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f45d0101890 msgr2=0x7f45d01980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:35.441 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:35.436+0000 7f45d632d640 1 --2- 172.21.15.80:0/1298664909 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f45d0101890 0x7f45d01980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:35.441 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:35.436+0000 7f45d632d640 1 -- 172.21.15.80:0/1298664909 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45d019f7a0 con 0x7f45d0102250 2024-04-24T23:27:35.441 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:35.436+0000 7f45d532b640 1 --2- 172.21.15.80:0/1298664909 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f45d0101890 0x7f45d01980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:27:35.441 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:35.436+0000 7f45d632d640 1 --2- 172.21.15.80:0/1298664909 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f45d0102250 0x7f45d019f130 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f45c0009230 tx=0x7f45c0004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:35.441 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:35.436+0000 7f45c6ffd640 1 -- 172.21.15.80:0/1298664909 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c003f030 con 0x7f45d0102250 2024-04-24T23:27:35.441 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:35.436+0000 7f45c6ffd640 1 -- 172.21.15.80:0/1298664909 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45c0003ce0 con 0x7f45d0102250 2024-04-24T23:27:35.441 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:35.436+0000 7f45d7d90640 1 -- 172.21.15.80:0/1298664909 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45d019fa30 con 0x7f45d0102250 2024-04-24T23:27:35.441 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:35.436+0000 7f45c6ffd640 1 -- 172.21.15.80:0/1298664909 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c0035070 con 0x7f45d0102250 2024-04-24T23:27:35.441 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:35.436+0000 7f45d7d90640 1 -- 172.21.15.80:0/1298664909 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45d019ffa0 con 0x7f45d0102250 2024-04-24T23:27:35.441 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:35.437+0000 7f45d7d90640 1 -- 172.21.15.80:0/1298664909 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45d01a01d0 con 0x7f45d0102250 2024-04-24T23:27:35.442 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:35.438+0000 7f45c6ffd640 1 -- 172.21.15.80:0/1298664909 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f45c0042030 con 0x7f45d0102250 2024-04-24T23:27:35.443 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:35.439+0000 7f45c6ffd640 1 --2- 172.21.15.80:0/1298664909 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f45b80a2b90 0x7f45b80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:35.443 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:35.439+0000 7f45d5b2c640 1 --2- 172.21.15.80:0/1298664909 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f45b80a2b90 0x7f45b80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:35.443 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:35.439+0000 7f45c6ffd640 1 -- 172.21.15.80:0/1298664909 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f45c00efd70 con 0x7f45d0102250 2024-04-24T23:27:35.443 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:35.439+0000 7f45d5b2c640 1 --2- 172.21.15.80:0/1298664909 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f45b80a2b90 0x7f45b80a5050 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f45c8005c40 tx=0x7f45c8005bd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:35.443 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:35.442+0000 7f45c6ffd640 1 -- 172.21.15.80:0/1298664909 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f45c00bca90 con 0x7f45d0102250 2024-04-24T23:27:35.634 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:35.633+0000 7f45d7d90640 1 -- 172.21.15.80:0/1298664909 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f45d01a0560 con 0x7f45d0102250 2024-04-24T23:27:35.634 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:35.633+0000 7f45c6ffd640 1 -- 172.21.15.80:0/1298664909 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v90) ==== 81+0+586 (secure 0 0 0) 0x7f45c00bc390 con 0x7f45d0102250 2024-04-24T23:27:35.635 INFO:teuthology.orchestra.run.smithi080.stdout:172.21.15.80:0/2828378749 2024-04-25T23:20:06.793609+0000 2024-04-24T23:27:35.635 INFO:teuthology.orchestra.run.smithi080.stdout:172.21.15.80:6801/233364790 2024-04-25T23:20:06.793609+0000 2024-04-24T23:27:35.635 INFO:teuthology.orchestra.run.smithi080.stdout:172.21.15.80:6800/1614691410 2024-04-25T23:19:40.490692+0000 2024-04-24T23:27:35.635 INFO:teuthology.orchestra.run.smithi080.stdout:172.21.15.80:0/1406187145 2024-04-25T23:20:06.793609+0000 2024-04-24T23:27:35.635 INFO:teuthology.orchestra.run.smithi080.stdout:172.21.15.80:0/92953396 2024-04-25T23:19:40.490692+0000 2024-04-24T23:27:35.635 INFO:teuthology.orchestra.run.smithi080.stdout:172.21.15.80:0/598129432 2024-04-25T23:20:06.793609+0000 2024-04-24T23:27:35.635 INFO:teuthology.orchestra.run.smithi080.stdout:172.21.15.80:0/232193118 2024-04-25T23:19:40.490692+0000 2024-04-24T23:27:35.635 INFO:teuthology.orchestra.run.smithi080.stdout:172.21.15.80:6800/233364790 2024-04-25T23:20:06.793609+0000 2024-04-24T23:27:35.635 INFO:teuthology.orchestra.run.smithi080.stdout:172.21.15.80:6801/1614691410 2024-04-25T23:19:40.490692+0000 2024-04-24T23:27:35.636 INFO:teuthology.orchestra.run.smithi080.stdout:172.21.15.80:0/2081562529 2024-04-25T23:19:40.490692+0000 2024-04-24T23:27:35.637 INFO:teuthology.orchestra.run.smithi080.stderr:listed 10 entries 2024-04-24T23:27:35.637 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:35.635+0000 7f45d7d90640 1 -- 172.21.15.80:0/1298664909 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f45b80a2b90 msgr2=0x7f45b80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:35.637 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:35.635+0000 7f45d7d90640 1 --2- 172.21.15.80:0/1298664909 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f45b80a2b90 0x7f45b80a5050 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f45c8005c40 tx=0x7f45c8005bd0 comp rx=0 tx=0).stop 2024-04-24T23:27:35.637 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:35.635+0000 7f45d7d90640 1 -- 172.21.15.80:0/1298664909 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f45d0102250 msgr2=0x7f45d019f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:35.637 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:35.635+0000 7f45d7d90640 1 --2- 172.21.15.80:0/1298664909 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f45d0102250 0x7f45d019f130 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f45c0009230 tx=0x7f45c0004360 comp rx=0 tx=0).stop 2024-04-24T23:27:35.637 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:35.635+0000 7f45d7d90640 1 -- 172.21.15.80:0/1298664909 shutdown_connections 2024-04-24T23:27:35.637 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:35.635+0000 7f45d7d90640 1 --2- 172.21.15.80:0/1298664909 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f45b80a2b90 0x7f45b80a5050 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:35.637 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:35.635+0000 7f45d7d90640 1 --2- 172.21.15.80:0/1298664909 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f45d0102250 0x7f45d019f130 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:35.637 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:35.635+0000 7f45d7d90640 1 --2- 172.21.15.80:0/1298664909 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f45d0101890 0x7f45d01980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:35.638 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:35.635+0000 7f45d7d90640 1 --2- 172.21.15.80:0/1298664909 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f45d01006a0 0x7f45d0197b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:35.638 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:35.635+0000 7f45d7d90640 1 -- 172.21.15.80:0/1298664909 >> 172.21.15.80:0/1298664909 conn(0x7f45d00fbc30 msgr2=0x7f45d00fd6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:35.638 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:35.635+0000 7f45d7d90640 1 -- 172.21.15.80:0/1298664909 shutdown_connections 2024-04-24T23:27:35.638 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:35.635+0000 7f45d7d90640 1 -- 172.21.15.80:0/1298664909 wait complete. 2024-04-24T23:27:35.646 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:35 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/976457589' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:27:35.646 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:35 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/2726121218' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T23:27:35.646 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:35 smithi080 ceph-mon[29152]: pgmap v304: 129 pgs: 129 active+clean; 581 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 317 B/s wr, 0 op/s 2024-04-24T23:27:35.646 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:35 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/1253156776' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T23:27:35.646 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:35 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/3449959519' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T23:27:35.647 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi080.front.sepia.ceph.com... 2024-04-24T23:27:35.648 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:27:35.648 DEBUG:teuthology.orchestra.run.smithi080:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-04-24T23:27:35.670 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi080.stderr:ceph-fuse[72649]: fuse finished with error 0 and tester_r 0 2024-04-24T23:27:35.672 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T23:27:35.697 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:35 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/976457589' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:27:35.697 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:35 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/2726121218' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T23:27:35.697 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:35 smithi134 ceph-mon[32619]: pgmap v304: 129 pgs: 129 active+clean; 581 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 317 B/s wr, 0 op/s 2024-04-24T23:27:35.697 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:35 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/1253156776' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T23:27:35.697 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:35 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/3449959519' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T23:27:35.756 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:35 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/976457589' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:27:35.756 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:35 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/2726121218' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T23:27:35.757 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:35 smithi148 ceph-mon[32528]: pgmap v304: 129 pgs: 129 active+clean; 581 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 317 B/s wr, 0 op/s 2024-04-24T23:27:35.757 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:35 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/1253156776' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T23:27:35.757 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:35 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/3449959519' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T23:27:36.697 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:36 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/1298664909' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-24T23:27:36.756 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:36 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/1298664909' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-24T23:27:36.788 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:36 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/1298664909' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-24T23:27:37.697 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:37 smithi134 ceph-mon[32619]: pgmap v305: 129 pgs: 129 active+clean; 581 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 287 B/s wr, 0 op/s 2024-04-24T23:27:37.756 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:37 smithi148 ceph-mon[32528]: pgmap v305: 129 pgs: 129 active+clean; 581 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 287 B/s wr, 0 op/s 2024-04-24T23:27:37.788 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:37 smithi080 ceph-mon[29152]: pgmap v305: 129 pgs: 129 active+clean; 581 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 287 B/s wr, 0 op/s 2024-04-24T23:27:40.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:39 smithi134 ceph-mon[32619]: pgmap v306: 129 pgs: 129 active+clean; 581 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-24T23:27:40.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:39 smithi148 ceph-mon[32528]: pgmap v306: 129 pgs: 129 active+clean; 581 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-24T23:27:40.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:39 smithi080 ceph-mon[29152]: pgmap v306: 129 pgs: 129 active+clean; 581 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-24T23:27:41.773 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi080.front.sepia.ceph.com 2024-04-24T23:27:41.774 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:27:41.774 DEBUG:teuthology.orchestra.run.smithi080:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-04-24T23:27:41.808 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:27:41.965 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:41.957+0000 7f3d9d716640 1 -- 172.21.15.80:0/3333200255 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3d98101880 msgr2=0x7f3d98101d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:41.965 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:41.957+0000 7f3d9d716640 1 --2- 172.21.15.80:0/3333200255 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3d98101880 0x7f3d98101d00 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f3d80009900 tx=0x7f3d8002a410 comp rx=0 tx=0).stop 2024-04-24T23:27:41.965 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:41.957+0000 7f3d9d716640 1 -- 172.21.15.80:0/3333200255 shutdown_connections 2024-04-24T23:27:41.966 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:41.957+0000 7f3d9d716640 1 --2- 172.21.15.80:0/3333200255 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d98102240 0x7f3d98108ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:41.966 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:41.957+0000 7f3d9d716640 1 --2- 172.21.15.80:0/3333200255 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3d98101880 0x7f3d98101d00 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:41.966 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:41.957+0000 7f3d9d716640 1 --2- 172.21.15.80:0/3333200255 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3d98100690 0x7f3d98100ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:41.966 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:41.957+0000 7f3d9d716640 1 -- 172.21.15.80:0/3333200255 >> 172.21.15.80:0/3333200255 conn(0x7f3d980fbc20 msgr2=0x7f3d980fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:41.966 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:41.957+0000 7f3d9d716640 1 -- 172.21.15.80:0/3333200255 shutdown_connections 2024-04-24T23:27:41.966 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:41.957+0000 7f3d9d716640 1 -- 172.21.15.80:0/3333200255 wait complete. 2024-04-24T23:27:41.966 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:41.957+0000 7f3d9d716640 1 Processor -- start 2024-04-24T23:27:41.966 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:41.958+0000 7f3d9d716640 1 -- start start 2024-04-24T23:27:41.967 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:41.958+0000 7f3d9d716640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3d98100690 0x7f3d98197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:41.967 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:41.958+0000 7f3d9d716640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d98101880 0x7f3d98197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:41.967 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:41.958+0000 7f3d9d716640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3d98102240 0x7f3d9819f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:41.967 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:41.958+0000 7f3d9d716640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f3d9810c360 con 0x7f3d98102240 2024-04-24T23:27:41.967 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:41.958+0000 7f3d9d716640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3d9810c1e0 con 0x7f3d98100690 2024-04-24T23:27:41.967 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:41.958+0000 7f3d9d716640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3d9810c4e0 con 0x7f3d98101880 2024-04-24T23:27:41.967 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:41.958+0000 7f3d977fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3d98102240 0x7f3d9819f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:41.967 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:41.958+0000 7f3d977fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3d98102240 0x7f3d9819f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:50636/0 (socket says 172.21.15.80:50636) 2024-04-24T23:27:41.968 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:41.958+0000 7f3d977fe640 1 -- 172.21.15.80:0/661818757 learned_addr learned my addr 172.21.15.80:0/661818757 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:27:41.968 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:41.958+0000 7f3d8e7fc640 1 --2- 172.21.15.80:0/661818757 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d98101880 0x7f3d98197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:41.968 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:41.958+0000 7f3d96ffd640 1 --2- 172.21.15.80:0/661818757 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3d98100690 0x7f3d98197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:41.968 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:41.958+0000 7f3d977fe640 1 -- 172.21.15.80:0/661818757 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d98101880 msgr2=0x7f3d98197fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:41.968 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:41.958+0000 7f3d977fe640 1 --2- 172.21.15.80:0/661818757 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d98101880 0x7f3d98197fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:41.968 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:41.958+0000 7f3d977fe640 1 -- 172.21.15.80:0/661818757 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3d98100690 msgr2=0x7f3d98197aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:41.968 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:41.958+0000 7f3d977fe640 1 --2- 172.21.15.80:0/661818757 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3d98100690 0x7f3d98197aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:41.968 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:41.958+0000 7f3d977fe640 1 -- 172.21.15.80:0/661818757 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d9819f6c0 con 0x7f3d98102240 2024-04-24T23:27:41.968 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:41.959+0000 7f3d977fe640 1 --2- 172.21.15.80:0/661818757 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3d98102240 0x7f3d9819f050 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f3d9000eaa0 tx=0x7f3d9000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:41.969 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:41.959+0000 7f3d94ff9640 1 -- 172.21.15.80:0/661818757 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d9000cdb0 con 0x7f3d98102240 2024-04-24T23:27:41.969 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:41.959+0000 7f3d94ff9640 1 -- 172.21.15.80:0/661818757 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d90005020 con 0x7f3d98102240 2024-04-24T23:27:41.969 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:41.959+0000 7f3d9d716640 1 -- 172.21.15.80:0/661818757 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d9819f9b0 con 0x7f3d98102240 2024-04-24T23:27:41.969 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:41.959+0000 7f3d94ff9640 1 -- 172.21.15.80:0/661818757 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d900116b0 con 0x7f3d98102240 2024-04-24T23:27:41.969 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:41.959+0000 7f3d9d716640 1 -- 172.21.15.80:0/661818757 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d9819fed0 con 0x7f3d98102240 2024-04-24T23:27:41.969 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:41.960+0000 7f3d9d716640 1 -- 172.21.15.80:0/661818757 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d981a01c0 con 0x7f3d98102240 2024-04-24T23:27:41.969 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:41.961+0000 7f3d94ff9640 1 -- 172.21.15.80:0/661818757 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f3d90005560 con 0x7f3d98102240 2024-04-24T23:27:41.969 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:41.962+0000 7f3d94ff9640 1 --2- 172.21.15.80:0/661818757 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f3d840a2b90 0x7f3d840a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:41.969 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:41.962+0000 7f3d96ffd640 1 --2- 172.21.15.80:0/661818757 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f3d840a2b90 0x7f3d840a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:41.969 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:41.962+0000 7f3d94ff9640 1 -- 172.21.15.80:0/661818757 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f3d900cf720 con 0x7f3d98102240 2024-04-24T23:27:41.969 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:41.962+0000 7f3d96ffd640 1 --2- 172.21.15.80:0/661818757 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f3d840a2b90 0x7f3d840a5050 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3d88005fd0 tx=0x7f3d8800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:41.970 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:41.965+0000 7f3d94ff9640 1 -- 172.21.15.80:0/661818757 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f3d90016030 con 0x7f3d98102240 2024-04-24T23:27:42.144 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:42.143+0000 7f3d9d716640 1 -- 172.21.15.80:0/661818757 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3d981a0390 con 0x7f3d98102240 2024-04-24T23:27:42.145 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:42.144+0000 7f3d94ff9640 1 -- 172.21.15.80:0/661818757 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24124 (secure 0 0 0) 0x7f3d9009bdf0 con 0x7f3d98102240 2024-04-24T23:27:42.146 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-24T23:27:42.146 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":90,"fsid":"ec7885aa-0290-11ef-bc93-c7b262605968","created":"2024-04-24T23:19:05.586407+0000","modified":"2024-04-24T23:27:26.212213+0000","last_up_change":"2024-04-24T23:26:25.938316+0000","last_in_change":"2024-04-24T23:26:04.786699+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:22:33.124428+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:27:17.575788+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:27:18.492762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T23:27:22.438503+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a525741d-b46c-4eef-aa64-3f057cac1497","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":2597718112},{"type":"v1","addr":"172.21.15.80:6803","nonce":2597718112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":2597718112},{"type":"v1","addr":"172.21.15.80:6805","nonce":2597718112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2597718112},{"type":"v1","addr":"172.21.15.80:6809","nonce":2597718112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":2597718112},{"type":"v1","addr":"172.21.15.80:6807","nonce":2597718112}]},"public_addr":"172.21.15.80:6803/2597718112","cluster_addr":"172.21.15.80:6805/2597718112","heartbeat_back_addr":"172.21.15.80:6809/2597718112","heartbeat_front_addr":"172.21.15.80:6807/2597718112","state":["exists","up"]},{"osd":1,"uuid":"2d0f0d67-6db4-40ac-91cc-58604ab6795a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":2395610201},{"type":"v1","addr":"172.21.15.134:6801","nonce":2395610201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":2395610201},{"type":"v1","addr":"172.21.15.134:6803","nonce":2395610201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":2395610201},{"type":"v1","addr":"172.21.15.134:6807","nonce":2395610201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":2395610201},{"type":"v1","addr":"172.21.15.134:6805","nonce":2395610201}]},"public_addr":"172.21.15.134:6801/2395610201","cluster_addr":"172.21.15.134:6803/2395610201","heartbeat_back_addr":"172.21.15.134:6807/2395610201","heartbeat_front_addr":"172.21.15.134:6805/2395610201","state":["exists","up"]},{"osd":2,"uuid":"781fb879-fedc-4c84-82fb-59e073bc4cf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1629152187},{"type":"v1","addr":"172.21.15.148:6801","nonce":1629152187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1629152187},{"type":"v1","addr":"172.21.15.148:6803","nonce":1629152187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1629152187},{"type":"v1","addr":"172.21.15.148:6807","nonce":1629152187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1629152187},{"type":"v1","addr":"172.21.15.148:6805","nonce":1629152187}]},"public_addr":"172.21.15.148:6801/1629152187","cluster_addr":"172.21.15.148:6803/1629152187","heartbeat_back_addr":"172.21.15.148:6807/1629152187","heartbeat_front_addr":"172.21.15.148:6805/1629152187","state":["exists","up"]},{"osd":3,"uuid":"cd982757-f2f6-4296-b3e7-4cd40fbd8d9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":3128298510},{"type":"v1","addr":"172.21.15.80:6811","nonce":3128298510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":3128298510},{"type":"v1","addr":"172.21.15.80:6813","nonce":3128298510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":3128298510},{"type":"v1","addr":"172.21.15.80:6817","nonce":3128298510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":3128298510},{"type":"v1","addr":"172.21.15.80:6815","nonce":3128298510}]},"public_addr":"172.21.15.80:6811/3128298510","cluster_addr":"172.21.15.80:6813/3128298510","heartbeat_back_addr":"172.21.15.80:6817/3128298510","heartbeat_front_addr":"172.21.15.80:6815/3128298510","state":["exists","up"]},{"osd":4,"uuid":"dfc55a79-a7e8-417a-b898-ec44171a54af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1917556060},{"type":"v1","addr":"172.21.15.134:6809","nonce":1917556060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1917556060},{"type":"v1","addr":"172.21.15.134:6811","nonce":1917556060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1917556060},{"type":"v1","addr":"172.21.15.134:6815","nonce":1917556060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1917556060},{"type":"v1","addr":"172.21.15.134:6813","nonce":1917556060}]},"public_addr":"172.21.15.134:6809/1917556060","cluster_addr":"172.21.15.134:6811/1917556060","heartbeat_back_addr":"172.21.15.134:6815/1917556060","heartbeat_front_addr":"172.21.15.134:6813/1917556060","state":["exists","up"]},{"osd":5,"uuid":"b5491337-0694-440c-b679-312f3d14cf65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":2715951281},{"type":"v1","addr":"172.21.15.148:6809","nonce":2715951281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":2715951281},{"type":"v1","addr":"172.21.15.148:6811","nonce":2715951281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":2715951281},{"type":"v1","addr":"172.21.15.148:6815","nonce":2715951281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":2715951281},{"type":"v1","addr":"172.21.15.148:6813","nonce":2715951281}]},"public_addr":"172.21.15.148:6809/2715951281","cluster_addr":"172.21.15.148:6811/2715951281","heartbeat_back_addr":"172.21.15.148:6815/2715951281","heartbeat_front_addr":"172.21.15.148:6813/2715951281","state":["exists","up"]},{"osd":6,"uuid":"ba4b5df7-d86e-47cd-ab67-cf763c4e4d6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":953261724},{"type":"v1","addr":"172.21.15.80:6819","nonce":953261724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":953261724},{"type":"v1","addr":"172.21.15.80:6821","nonce":953261724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":953261724},{"type":"v1","addr":"172.21.15.80:6825","nonce":953261724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":953261724},{"type":"v1","addr":"172.21.15.80:6823","nonce":953261724}]},"public_addr":"172.21.15.80:6819/953261724","cluster_addr":"172.21.15.80:6821/953261724","heartbeat_back_addr":"172.21.15.80:6825/953261724","heartbeat_front_addr":"172.21.15.80:6823/953261724","state":["exists","up"]},{"osd":7,"uuid":"464a503a-75ae-49e7-ab6e-a7f9f2a6a307","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":108354844},{"type":"v1","addr":"172.21.15.134:6817","nonce":108354844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":108354844},{"type":"v1","addr":"172.21.15.134:6819","nonce":108354844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":108354844},{"type":"v1","addr":"172.21.15.134:6823","nonce":108354844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":108354844},{"type":"v1","addr":"172.21.15.134:6821","nonce":108354844}]},"public_addr":"172.21.15.134:6817/108354844","cluster_addr":"172.21.15.134:6819/108354844","heartbeat_back_addr":"172.21.15.134:6823/108354844","heartbeat_front_addr":"172.21.15.134:6821/108354844","state":["exists","up"]},{"osd":8,"uuid":"ce544b2e-4540-47c0-9997-500c45351330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3559655329},{"type":"v1","addr":"172.21.15.148:6817","nonce":3559655329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":3559655329},{"type":"v1","addr":"172.21.15.148:6819","nonce":3559655329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":3559655329},{"type":"v1","addr":"172.21.15.148:6823","nonce":3559655329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3559655329},{"type":"v1","addr":"172.21.15.148:6821","nonce":3559655329}]},"public_addr":"172.21.15.148:6817/3559655329","cluster_addr":"172.21.15.148:6819/3559655329","heartbeat_back_addr":"172.21.15.148:6823/3559655329","heartbeat_front_addr":"172.21.15.148:6821/3559655329","state":["exists","up"]},{"osd":9,"uuid":"06de3732-19b9-496c-a464-55322efa6759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":4019913545},{"type":"v1","addr":"172.21.15.80:6827","nonce":4019913545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":4019913545},{"type":"v1","addr":"172.21.15.80:6829","nonce":4019913545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":4019913545},{"type":"v1","addr":"172.21.15.80:6833","nonce":4019913545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":4019913545},{"type":"v1","addr":"172.21.15.80:6831","nonce":4019913545}]},"public_addr":"172.21.15.80:6827/4019913545","cluster_addr":"172.21.15.80:6829/4019913545","heartbeat_back_addr":"172.21.15.80:6833/4019913545","heartbeat_front_addr":"172.21.15.80:6831/4019913545","state":["exists","up"]},{"osd":10,"uuid":"8d92f0d9-5c64-429a-b750-c33ae0e070c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":647343766},{"type":"v1","addr":"172.21.15.134:6825","nonce":647343766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":647343766},{"type":"v1","addr":"172.21.15.134:6827","nonce":647343766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":647343766},{"type":"v1","addr":"172.21.15.134:6831","nonce":647343766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":647343766},{"type":"v1","addr":"172.21.15.134:6829","nonce":647343766}]},"public_addr":"172.21.15.134:6825/647343766","cluster_addr":"172.21.15.134:6827/647343766","heartbeat_back_addr":"172.21.15.134:6831/647343766","heartbeat_front_addr":"172.21.15.134:6829/647343766","state":["exists","up"]},{"osd":11,"uuid":"4c003a41-f65b-479f-bc80-b27d680cf936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":4068024116},{"type":"v1","addr":"172.21.15.148:6825","nonce":4068024116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":4068024116},{"type":"v1","addr":"172.21.15.148:6827","nonce":4068024116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":4068024116},{"type":"v1","addr":"172.21.15.148:6831","nonce":4068024116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":4068024116},{"type":"v1","addr":"172.21.15.148:6829","nonce":4068024116}]},"public_addr":"172.21.15.148:6825/4068024116","cluster_addr":"172.21.15.148:6827/4068024116","heartbeat_back_addr":"172.21.15.148:6831/4068024116","heartbeat_front_addr":"172.21.15.148:6829/4068024116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:21:42.433631+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:22:06.145933+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:22:30.726143+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:22:55.729961+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:23:21.280913+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:23:46.609983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:24:12.585247+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:24:38.359287+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:25:04.491232+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:25:30.788089+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:25:56.891078+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:26:23.899113+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.80:0/2828378749":"2024-04-25T23:20:06.793609+0000","172.21.15.80:6801/233364790":"2024-04-25T23:20:06.793609+0000","172.21.15.80:6800/1614691410":"2024-04-25T23:19:40.490692+0000","172.21.15.80:0/1406187145":"2024-04-25T23:20:06.793609+0000","172.21.15.80:0/92953396":"2024-04-25T23:19:40.490692+0000","172.21.15.80:0/598129432":"2024-04-25T23:20:06.793609+0000","172.21.15.80:0/232193118":"2024-04-25T23:19:40.490692+0000","172.21.15.80:6800/233364790":"2024-04-25T23:20:06.793609+0000","172.21.15.80:6801/1614691410":"2024-04-25T23:19:40.490692+0000","172.21.15.80:0/2081562529":"2024-04-25T23:19:40.490692+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:27:42.148 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:42.146+0000 7f3d9d716640 1 -- 172.21.15.80:0/661818757 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f3d840a2b90 msgr2=0x7f3d840a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:42.148 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:42.146+0000 7f3d9d716640 1 --2- 172.21.15.80:0/661818757 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f3d840a2b90 0x7f3d840a5050 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3d88005fd0 tx=0x7f3d8800b410 comp rx=0 tx=0).stop 2024-04-24T23:27:42.148 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:42.146+0000 7f3d9d716640 1 -- 172.21.15.80:0/661818757 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3d98102240 msgr2=0x7f3d9819f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:42.148 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:42.146+0000 7f3d9d716640 1 --2- 172.21.15.80:0/661818757 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3d98102240 0x7f3d9819f050 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f3d9000eaa0 tx=0x7f3d9000ef60 comp rx=0 tx=0).stop 2024-04-24T23:27:42.148 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:42.146+0000 7f3d9d716640 1 -- 172.21.15.80:0/661818757 shutdown_connections 2024-04-24T23:27:42.148 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:42.146+0000 7f3d9d716640 1 --2- 172.21.15.80:0/661818757 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f3d840a2b90 0x7f3d840a5050 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:42.148 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:42.146+0000 7f3d9d716640 1 --2- 172.21.15.80:0/661818757 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3d98102240 0x7f3d9819f050 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:42.148 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:42.147+0000 7f3d9d716640 1 --2- 172.21.15.80:0/661818757 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d98101880 0x7f3d98197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:42.148 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:42.147+0000 7f3d9d716640 1 --2- 172.21.15.80:0/661818757 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3d98100690 0x7f3d98197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:42.148 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:42.147+0000 7f3d9d716640 1 -- 172.21.15.80:0/661818757 >> 172.21.15.80:0/661818757 conn(0x7f3d980fbc20 msgr2=0x7f3d980fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:42.148 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:42.147+0000 7f3d9d716640 1 -- 172.21.15.80:0/661818757 shutdown_connections 2024-04-24T23:27:42.149 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:42.147+0000 7f3d9d716640 1 -- 172.21.15.80:0/661818757 wait complete. 2024-04-24T23:27:42.157 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:41 smithi080 ceph-mon[29152]: pgmap v307: 129 pgs: 129 active+clean; 581 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-24T23:27:42.159 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:27:42.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:41 smithi134 ceph-mon[32619]: pgmap v307: 129 pgs: 129 active+clean; 581 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-24T23:27:42.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:41 smithi148 ceph-mon[32528]: pgmap v307: 129 pgs: 129 active+clean; 581 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-24T23:27:42.257 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:42.256+0000 7fce2b631640 1 -- 172.21.15.80:0/947079915 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fce24102240 msgr2=0x7fce24108ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:42.257 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:42.256+0000 7fce2b631640 1 --2- 172.21.15.80:0/947079915 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fce24102240 0x7fce24108ad0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fce1c009ca0 tx=0x7fce1c02a8c0 comp rx=0 tx=0).stop 2024-04-24T23:27:42.257 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:42.256+0000 7fce2b631640 1 -- 172.21.15.80:0/947079915 shutdown_connections 2024-04-24T23:27:42.257 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:42.256+0000 7fce2b631640 1 --2- 172.21.15.80:0/947079915 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fce24102240 0x7fce24108ad0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:42.258 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:42.256+0000 7fce2b631640 1 --2- 172.21.15.80:0/947079915 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce24101880 0x7fce24101d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:42.258 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:42.256+0000 7fce2b631640 1 --2- 172.21.15.80:0/947079915 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce24100690 0x7fce24100ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:42.258 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:42.256+0000 7fce2b631640 1 -- 172.21.15.80:0/947079915 >> 172.21.15.80:0/947079915 conn(0x7fce240fbc20 msgr2=0x7fce240fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:42.258 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:42.256+0000 7fce2b631640 1 -- 172.21.15.80:0/947079915 shutdown_connections 2024-04-24T23:27:42.258 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:42.256+0000 7fce2b631640 1 -- 172.21.15.80:0/947079915 wait complete. 2024-04-24T23:27:42.260 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:42.257+0000 7fce2b631640 1 Processor -- start 2024-04-24T23:27:42.260 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:42.257+0000 7fce2b631640 1 -- start start 2024-04-24T23:27:42.260 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:42.257+0000 7fce2b631640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fce24100690 0x7fce2406a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:42.260 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:42.257+0000 7fce2b631640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce24101880 0x7fce2419c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:42.260 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:42.257+0000 7fce2b631640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce24102240 0x7fce2419ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:42.260 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:42.257+0000 7fce2b631640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fce2410c380 con 0x7fce24100690 2024-04-24T23:27:42.260 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:42.257+0000 7fce2b631640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fce2410c200 con 0x7fce24101880 2024-04-24T23:27:42.260 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:42.257+0000 7fce2b631640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fce2410c500 con 0x7fce24102240 2024-04-24T23:27:42.261 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:42.257+0000 7fce293cd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fce24100690 0x7fce2406a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:42.261 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:42.257+0000 7fce293cd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fce24100690 0x7fce2406a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:50652/0 (socket says 172.21.15.80:50652) 2024-04-24T23:27:42.261 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:42.257+0000 7fce293cd640 1 -- 172.21.15.80:0/997207059 learned_addr learned my addr 172.21.15.80:0/997207059 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:27:42.261 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:42.257+0000 7fce29bce640 1 --2- 172.21.15.80:0/997207059 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce24102240 0x7fce2419ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:42.261 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:42.257+0000 7fce293cd640 1 -- 172.21.15.80:0/997207059 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce24102240 msgr2=0x7fce2419ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:42.261 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:42.257+0000 7fce293cd640 1 --2- 172.21.15.80:0/997207059 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce24102240 0x7fce2419ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:42.261 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:42.257+0000 7fce293cd640 1 -- 172.21.15.80:0/997207059 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce24101880 msgr2=0x7fce2419c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:42.261 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:42.257+0000 7fce293cd640 1 --2- 172.21.15.80:0/997207059 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce24101880 0x7fce2419c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:42.261 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:42.257+0000 7fce293cd640 1 -- 172.21.15.80:0/997207059 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce2419f6c0 con 0x7fce24100690 2024-04-24T23:27:42.261 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:42.257+0000 7fce29bce640 1 --2- 172.21.15.80:0/997207059 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce24102240 0x7fce2419ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:27:42.261 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:42.257+0000 7fce293cd640 1 --2- 172.21.15.80:0/997207059 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fce24100690 0x7fce2406a930 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fce2000b450 tx=0x7fce2000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:42.261 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:42.258+0000 7fce1a7fc640 1 -- 172.21.15.80:0/997207059 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce20015020 con 0x7fce24100690 2024-04-24T23:27:42.261 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:42.258+0000 7fce1a7fc640 1 -- 172.21.15.80:0/997207059 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce20011d70 con 0x7fce24100690 2024-04-24T23:27:42.261 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:42.258+0000 7fce2b631640 1 -- 172.21.15.80:0/997207059 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce2419f9b0 con 0x7fce24100690 2024-04-24T23:27:42.261 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:42.258+0000 7fce1a7fc640 1 -- 172.21.15.80:0/997207059 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce2000f340 con 0x7fce24100690 2024-04-24T23:27:42.262 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:42.258+0000 7fce2b631640 1 -- 172.21.15.80:0/997207059 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce2419ff80 con 0x7fce24100690 2024-04-24T23:27:42.262 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:42.259+0000 7fce2b631640 1 -- 172.21.15.80:0/997207059 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce24107950 con 0x7fce24100690 2024-04-24T23:27:42.263 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:42.260+0000 7fce1a7fc640 1 -- 172.21.15.80:0/997207059 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fce20018020 con 0x7fce24100690 2024-04-24T23:27:42.263 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:42.260+0000 7fce1a7fc640 1 --2- 172.21.15.80:0/997207059 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fce100a2b90 0x7fce100a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:42.263 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:42.260+0000 7fce28bcc640 1 --2- 172.21.15.80:0/997207059 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fce100a2b90 0x7fce100a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:42.263 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:42.260+0000 7fce1a7fc640 1 -- 172.21.15.80:0/997207059 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fce200ced90 con 0x7fce24100690 2024-04-24T23:27:42.263 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:42.261+0000 7fce28bcc640 1 --2- 172.21.15.80:0/997207059 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fce100a2b90 0x7fce100a5050 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fce14005fd0 tx=0x7fce14005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:42.265 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:42.265+0000 7fce1a7fc640 1 -- 172.21.15.80:0/997207059 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fce2001f020 con 0x7fce24100690 2024-04-24T23:27:42.442 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:42.441+0000 7fce2b631640 1 -- 172.21.15.80:0/997207059 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fce240623e0 con 0x7fce24100690 2024-04-24T23:27:42.443 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:42.442+0000 7fce1a7fc640 1 -- 172.21.15.80:0/997207059 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24124 (secure 0 0 0) 0x7fce2000d040 con 0x7fce24100690 2024-04-24T23:27:42.444 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-24T23:27:42.444 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":90,"fsid":"ec7885aa-0290-11ef-bc93-c7b262605968","created":"2024-04-24T23:19:05.586407+0000","modified":"2024-04-24T23:27:26.212213+0000","last_up_change":"2024-04-24T23:26:25.938316+0000","last_in_change":"2024-04-24T23:26:04.786699+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:22:33.124428+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:27:17.575788+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:27:18.492762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T23:27:22.438503+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a525741d-b46c-4eef-aa64-3f057cac1497","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":2597718112},{"type":"v1","addr":"172.21.15.80:6803","nonce":2597718112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":2597718112},{"type":"v1","addr":"172.21.15.80:6805","nonce":2597718112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2597718112},{"type":"v1","addr":"172.21.15.80:6809","nonce":2597718112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":2597718112},{"type":"v1","addr":"172.21.15.80:6807","nonce":2597718112}]},"public_addr":"172.21.15.80:6803/2597718112","cluster_addr":"172.21.15.80:6805/2597718112","heartbeat_back_addr":"172.21.15.80:6809/2597718112","heartbeat_front_addr":"172.21.15.80:6807/2597718112","state":["exists","up"]},{"osd":1,"uuid":"2d0f0d67-6db4-40ac-91cc-58604ab6795a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":2395610201},{"type":"v1","addr":"172.21.15.134:6801","nonce":2395610201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":2395610201},{"type":"v1","addr":"172.21.15.134:6803","nonce":2395610201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":2395610201},{"type":"v1","addr":"172.21.15.134:6807","nonce":2395610201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":2395610201},{"type":"v1","addr":"172.21.15.134:6805","nonce":2395610201}]},"public_addr":"172.21.15.134:6801/2395610201","cluster_addr":"172.21.15.134:6803/2395610201","heartbeat_back_addr":"172.21.15.134:6807/2395610201","heartbeat_front_addr":"172.21.15.134:6805/2395610201","state":["exists","up"]},{"osd":2,"uuid":"781fb879-fedc-4c84-82fb-59e073bc4cf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1629152187},{"type":"v1","addr":"172.21.15.148:6801","nonce":1629152187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1629152187},{"type":"v1","addr":"172.21.15.148:6803","nonce":1629152187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1629152187},{"type":"v1","addr":"172.21.15.148:6807","nonce":1629152187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1629152187},{"type":"v1","addr":"172.21.15.148:6805","nonce":1629152187}]},"public_addr":"172.21.15.148:6801/1629152187","cluster_addr":"172.21.15.148:6803/1629152187","heartbeat_back_addr":"172.21.15.148:6807/1629152187","heartbeat_front_addr":"172.21.15.148:6805/1629152187","state":["exists","up"]},{"osd":3,"uuid":"cd982757-f2f6-4296-b3e7-4cd40fbd8d9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":3128298510},{"type":"v1","addr":"172.21.15.80:6811","nonce":3128298510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":3128298510},{"type":"v1","addr":"172.21.15.80:6813","nonce":3128298510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":3128298510},{"type":"v1","addr":"172.21.15.80:6817","nonce":3128298510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":3128298510},{"type":"v1","addr":"172.21.15.80:6815","nonce":3128298510}]},"public_addr":"172.21.15.80:6811/3128298510","cluster_addr":"172.21.15.80:6813/3128298510","heartbeat_back_addr":"172.21.15.80:6817/3128298510","heartbeat_front_addr":"172.21.15.80:6815/3128298510","state":["exists","up"]},{"osd":4,"uuid":"dfc55a79-a7e8-417a-b898-ec44171a54af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1917556060},{"type":"v1","addr":"172.21.15.134:6809","nonce":1917556060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1917556060},{"type":"v1","addr":"172.21.15.134:6811","nonce":1917556060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1917556060},{"type":"v1","addr":"172.21.15.134:6815","nonce":1917556060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1917556060},{"type":"v1","addr":"172.21.15.134:6813","nonce":1917556060}]},"public_addr":"172.21.15.134:6809/1917556060","cluster_addr":"172.21.15.134:6811/1917556060","heartbeat_back_addr":"172.21.15.134:6815/1917556060","heartbeat_front_addr":"172.21.15.134:6813/1917556060","state":["exists","up"]},{"osd":5,"uuid":"b5491337-0694-440c-b679-312f3d14cf65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":2715951281},{"type":"v1","addr":"172.21.15.148:6809","nonce":2715951281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":2715951281},{"type":"v1","addr":"172.21.15.148:6811","nonce":2715951281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":2715951281},{"type":"v1","addr":"172.21.15.148:6815","nonce":2715951281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":2715951281},{"type":"v1","addr":"172.21.15.148:6813","nonce":2715951281}]},"public_addr":"172.21.15.148:6809/2715951281","cluster_addr":"172.21.15.148:6811/2715951281","heartbeat_back_addr":"172.21.15.148:6815/2715951281","heartbeat_front_addr":"172.21.15.148:6813/2715951281","state":["exists","up"]},{"osd":6,"uuid":"ba4b5df7-d86e-47cd-ab67-cf763c4e4d6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":953261724},{"type":"v1","addr":"172.21.15.80:6819","nonce":953261724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":953261724},{"type":"v1","addr":"172.21.15.80:6821","nonce":953261724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":953261724},{"type":"v1","addr":"172.21.15.80:6825","nonce":953261724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":953261724},{"type":"v1","addr":"172.21.15.80:6823","nonce":953261724}]},"public_addr":"172.21.15.80:6819/953261724","cluster_addr":"172.21.15.80:6821/953261724","heartbeat_back_addr":"172.21.15.80:6825/953261724","heartbeat_front_addr":"172.21.15.80:6823/953261724","state":["exists","up"]},{"osd":7,"uuid":"464a503a-75ae-49e7-ab6e-a7f9f2a6a307","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":108354844},{"type":"v1","addr":"172.21.15.134:6817","nonce":108354844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":108354844},{"type":"v1","addr":"172.21.15.134:6819","nonce":108354844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":108354844},{"type":"v1","addr":"172.21.15.134:6823","nonce":108354844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":108354844},{"type":"v1","addr":"172.21.15.134:6821","nonce":108354844}]},"public_addr":"172.21.15.134:6817/108354844","cluster_addr":"172.21.15.134:6819/108354844","heartbeat_back_addr":"172.21.15.134:6823/108354844","heartbeat_front_addr":"172.21.15.134:6821/108354844","state":["exists","up"]},{"osd":8,"uuid":"ce544b2e-4540-47c0-9997-500c45351330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3559655329},{"type":"v1","addr":"172.21.15.148:6817","nonce":3559655329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":3559655329},{"type":"v1","addr":"172.21.15.148:6819","nonce":3559655329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":3559655329},{"type":"v1","addr":"172.21.15.148:6823","nonce":3559655329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3559655329},{"type":"v1","addr":"172.21.15.148:6821","nonce":3559655329}]},"public_addr":"172.21.15.148:6817/3559655329","cluster_addr":"172.21.15.148:6819/3559655329","heartbeat_back_addr":"172.21.15.148:6823/3559655329","heartbeat_front_addr":"172.21.15.148:6821/3559655329","state":["exists","up"]},{"osd":9,"uuid":"06de3732-19b9-496c-a464-55322efa6759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":4019913545},{"type":"v1","addr":"172.21.15.80:6827","nonce":4019913545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":4019913545},{"type":"v1","addr":"172.21.15.80:6829","nonce":4019913545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":4019913545},{"type":"v1","addr":"172.21.15.80:6833","nonce":4019913545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":4019913545},{"type":"v1","addr":"172.21.15.80:6831","nonce":4019913545}]},"public_addr":"172.21.15.80:6827/4019913545","cluster_addr":"172.21.15.80:6829/4019913545","heartbeat_back_addr":"172.21.15.80:6833/4019913545","heartbeat_front_addr":"172.21.15.80:6831/4019913545","state":["exists","up"]},{"osd":10,"uuid":"8d92f0d9-5c64-429a-b750-c33ae0e070c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":647343766},{"type":"v1","addr":"172.21.15.134:6825","nonce":647343766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":647343766},{"type":"v1","addr":"172.21.15.134:6827","nonce":647343766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":647343766},{"type":"v1","addr":"172.21.15.134:6831","nonce":647343766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":647343766},{"type":"v1","addr":"172.21.15.134:6829","nonce":647343766}]},"public_addr":"172.21.15.134:6825/647343766","cluster_addr":"172.21.15.134:6827/647343766","heartbeat_back_addr":"172.21.15.134:6831/647343766","heartbeat_front_addr":"172.21.15.134:6829/647343766","state":["exists","up"]},{"osd":11,"uuid":"4c003a41-f65b-479f-bc80-b27d680cf936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":4068024116},{"type":"v1","addr":"172.21.15.148:6825","nonce":4068024116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":4068024116},{"type":"v1","addr":"172.21.15.148:6827","nonce":4068024116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":4068024116},{"type":"v1","addr":"172.21.15.148:6831","nonce":4068024116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":4068024116},{"type":"v1","addr":"172.21.15.148:6829","nonce":4068024116}]},"public_addr":"172.21.15.148:6825/4068024116","cluster_addr":"172.21.15.148:6827/4068024116","heartbeat_back_addr":"172.21.15.148:6831/4068024116","heartbeat_front_addr":"172.21.15.148:6829/4068024116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:21:42.433631+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:22:06.145933+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:22:30.726143+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:22:55.729961+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:23:21.280913+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:23:46.609983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:24:12.585247+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:24:38.359287+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:25:04.491232+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:25:30.788089+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:25:56.891078+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:26:23.899113+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.80:0/2828378749":"2024-04-25T23:20:06.793609+0000","172.21.15.80:6801/233364790":"2024-04-25T23:20:06.793609+0000","172.21.15.80:6800/1614691410":"2024-04-25T23:19:40.490692+0000","172.21.15.80:0/1406187145":"2024-04-25T23:20:06.793609+0000","172.21.15.80:0/92953396":"2024-04-25T23:19:40.490692+0000","172.21.15.80:0/598129432":"2024-04-25T23:20:06.793609+0000","172.21.15.80:0/232193118":"2024-04-25T23:19:40.490692+0000","172.21.15.80:6800/233364790":"2024-04-25T23:20:06.793609+0000","172.21.15.80:6801/1614691410":"2024-04-25T23:19:40.490692+0000","172.21.15.80:0/2081562529":"2024-04-25T23:19:40.490692+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:27:42.446 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:42.445+0000 7fce2b631640 1 -- 172.21.15.80:0/997207059 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fce100a2b90 msgr2=0x7fce100a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:42.446 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:42.445+0000 7fce2b631640 1 --2- 172.21.15.80:0/997207059 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fce100a2b90 0x7fce100a5050 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fce14005fd0 tx=0x7fce14005850 comp rx=0 tx=0).stop 2024-04-24T23:27:42.446 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:42.445+0000 7fce2b631640 1 -- 172.21.15.80:0/997207059 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fce24100690 msgr2=0x7fce2406a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:42.446 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:42.445+0000 7fce2b631640 1 --2- 172.21.15.80:0/997207059 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fce24100690 0x7fce2406a930 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fce2000b450 tx=0x7fce2000b910 comp rx=0 tx=0).stop 2024-04-24T23:27:42.446 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:42.445+0000 7fce2b631640 1 -- 172.21.15.80:0/997207059 shutdown_connections 2024-04-24T23:27:42.446 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:42.445+0000 7fce2b631640 1 --2- 172.21.15.80:0/997207059 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fce100a2b90 0x7fce100a5050 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:42.446 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:42.445+0000 7fce2b631640 1 --2- 172.21.15.80:0/997207059 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce24102240 0x7fce2419ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:42.446 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:42.445+0000 7fce2b631640 1 --2- 172.21.15.80:0/997207059 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce24101880 0x7fce2419c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:42.447 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:42.445+0000 7fce2b631640 1 --2- 172.21.15.80:0/997207059 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fce24100690 0x7fce2406a930 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:42.447 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:42.445+0000 7fce2b631640 1 -- 172.21.15.80:0/997207059 >> 172.21.15.80:0/997207059 conn(0x7fce240fbc20 msgr2=0x7fce24104b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:42.447 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:42.445+0000 7fce2b631640 1 -- 172.21.15.80:0/997207059 shutdown_connections 2024-04-24T23:27:42.447 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:42.445+0000 7fce2b631640 1 -- 172.21.15.80:0/997207059 wait complete. 2024-04-24T23:27:42.457 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-24T23:27:42.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:42.557+0000 7f200dfee640 1 -- 172.21.15.80:0/2942620050 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f20080ffe10 msgr2=0x7f2008100230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:42.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:42.557+0000 7f200dfee640 1 --2- 172.21.15.80:0/2942620050 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f20080ffe10 0x7f2008100230 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f1ff8009900 tx=0x7f1ff802a410 comp rx=0 tx=0).stop 2024-04-24T23:27:42.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:42.557+0000 7f200dfee640 1 -- 172.21.15.80:0/2942620050 shutdown_connections 2024-04-24T23:27:42.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:42.557+0000 7f200dfee640 1 --2- 172.21.15.80:0/2942620050 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2008101920 0x7f2008108240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:42.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:42.557+0000 7f200dfee640 1 --2- 172.21.15.80:0/2942620050 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2008100f60 0x7f20081013e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:42.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:42.557+0000 7f200dfee640 1 --2- 172.21.15.80:0/2942620050 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f20080ffe10 0x7f2008100230 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:42.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:42.557+0000 7f200dfee640 1 -- 172.21.15.80:0/2942620050 >> 172.21.15.80:0/2942620050 conn(0x7f20080fb380 msgr2=0x7f20080fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:42.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:42.557+0000 7f200dfee640 1 -- 172.21.15.80:0/2942620050 shutdown_connections 2024-04-24T23:27:42.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:42.557+0000 7f200dfee640 1 -- 172.21.15.80:0/2942620050 wait complete. 2024-04-24T23:27:42.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:42.557+0000 7f200dfee640 1 Processor -- start 2024-04-24T23:27:42.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:42.558+0000 7f200dfee640 1 -- start start 2024-04-24T23:27:42.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:42.558+0000 7f200dfee640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20080ffe10 0x7f20081097c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:42.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:42.558+0000 7f200dfee640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2008100f60 0x7f200819c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:42.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:42.558+0000 7f200dfee640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2008101920 0x7f200819ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:42.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:42.558+0000 7f200dfee640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f200810c3b0 con 0x7f2008101920 2024-04-24T23:27:42.562 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:42.558+0000 7f200dfee640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f200810c230 con 0x7f2008100f60 2024-04-24T23:27:42.562 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:42.558+0000 7f200dfee640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f200810c530 con 0x7f20080ffe10 2024-04-24T23:27:42.562 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:42.558+0000 7f2007fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2008101920 0x7f200819ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:42.562 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:42.558+0000 7f2007fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2008101920 0x7f200819ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:37302/0 (socket says 172.21.15.80:37302) 2024-04-24T23:27:42.562 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:42.558+0000 7f2007fff640 1 -- 172.21.15.80:0/1382410285 learned_addr learned my addr 172.21.15.80:0/1382410285 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:27:42.562 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:42.558+0000 7f20077fe640 1 --2- 172.21.15.80:0/1382410285 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20080ffe10 0x7f20081097c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:42.562 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:42.558+0000 7f2006ffd640 1 --2- 172.21.15.80:0/1382410285 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2008100f60 0x7f200819c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:42.562 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:42.558+0000 7f2007fff640 1 -- 172.21.15.80:0/1382410285 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20080ffe10 msgr2=0x7f20081097c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:42.562 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:42.558+0000 7f2007fff640 1 --2- 172.21.15.80:0/1382410285 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20080ffe10 0x7f20081097c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:42.562 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:42.558+0000 7f2007fff640 1 -- 172.21.15.80:0/1382410285 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2008100f60 msgr2=0x7f200819c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:42.562 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:42.558+0000 7f2007fff640 1 --2- 172.21.15.80:0/1382410285 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2008100f60 0x7f200819c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:42.562 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:42.558+0000 7f2007fff640 1 -- 172.21.15.80:0/1382410285 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f200819f750 con 0x7f2008101920 2024-04-24T23:27:42.562 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:42.558+0000 7f2006ffd640 1 --2- 172.21.15.80:0/1382410285 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2008100f60 0x7f200819c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:27:42.562 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:42.558+0000 7f2007fff640 1 --2- 172.21.15.80:0/1382410285 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2008101920 0x7f200819ee90 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f200000eaa0 tx=0x7f200000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:42.562 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:42.559+0000 7f2004ff9640 1 -- 172.21.15.80:0/1382410285 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f200000cdb0 con 0x7f2008101920 2024-04-24T23:27:42.563 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:42.559+0000 7f2004ff9640 1 -- 172.21.15.80:0/1382410285 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2000005020 con 0x7f2008101920 2024-04-24T23:27:42.563 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:42.559+0000 7f200dfee640 1 -- 172.21.15.80:0/1382410285 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f200819fa40 con 0x7f2008101920 2024-04-24T23:27:42.563 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:42.559+0000 7f2004ff9640 1 -- 172.21.15.80:0/1382410285 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20000116b0 con 0x7f2008101920 2024-04-24T23:27:42.563 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:42.559+0000 7f200dfee640 1 -- 172.21.15.80:0/1382410285 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f200819ffc0 con 0x7f2008101920 2024-04-24T23:27:42.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:42.560+0000 7f200dfee640 1 -- 172.21.15.80:0/1382410285 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2008107120 con 0x7f2008101920 2024-04-24T23:27:42.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:42.561+0000 7f2004ff9640 1 -- 172.21.15.80:0/1382410285 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f2000005560 con 0x7f2008101920 2024-04-24T23:27:42.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:42.562+0000 7f2004ff9640 1 --2- 172.21.15.80:0/1382410285 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f1ff00a2b90 0x7f1ff00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:42.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:42.562+0000 7f20077fe640 1 --2- 172.21.15.80:0/1382410285 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f1ff00a2b90 0x7f1ff00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:42.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:42.562+0000 7f2004ff9640 1 -- 172.21.15.80:0/1382410285 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f20000cf4d0 con 0x7f2008101920 2024-04-24T23:27:42.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:42.562+0000 7f20077fe640 1 --2- 172.21.15.80:0/1382410285 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f1ff00a2b90 0x7f1ff00a5050 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1ff80314e0 tx=0x7f1ff8000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:42.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:42.566+0000 7f2004ff9640 1 -- 172.21.15.80:0/1382410285 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f2000016030 con 0x7f2008101920 2024-04-24T23:27:42.778 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:42.777+0000 7f200dfee640 1 -- 172.21.15.80:0/1382410285 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f2008062340 con 0x7f2008101920 2024-04-24T23:27:42.954 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:42.953+0000 7f2004ff9640 1 -- 172.21.15.80:0/1382410285 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) ==== 118+0+0 (secure 0 0 0) 0x7f200009c1f0 con 0x7f2008101920 2024-04-24T23:27:42.958 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:42.956+0000 7f200dfee640 1 -- 172.21.15.80:0/1382410285 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f1ff00a2b90 msgr2=0x7f1ff00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:42.958 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:42.956+0000 7f200dfee640 1 --2- 172.21.15.80:0/1382410285 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f1ff00a2b90 0x7f1ff00a5050 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1ff80314e0 tx=0x7f1ff8000bc0 comp rx=0 tx=0).stop 2024-04-24T23:27:42.958 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:42.956+0000 7f200dfee640 1 -- 172.21.15.80:0/1382410285 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2008101920 msgr2=0x7f200819ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:42.958 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:42.956+0000 7f200dfee640 1 --2- 172.21.15.80:0/1382410285 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2008101920 0x7f200819ee90 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f200000eaa0 tx=0x7f200000ef60 comp rx=0 tx=0).stop 2024-04-24T23:27:42.958 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:42.956+0000 7f200dfee640 1 -- 172.21.15.80:0/1382410285 shutdown_connections 2024-04-24T23:27:42.958 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:42.956+0000 7f200dfee640 1 --2- 172.21.15.80:0/1382410285 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f1ff00a2b90 0x7f1ff00a5050 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:42.958 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:42.956+0000 7f200dfee640 1 --2- 172.21.15.80:0/1382410285 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2008101920 0x7f200819ee90 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:42.958 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:42.956+0000 7f200dfee640 1 --2- 172.21.15.80:0/1382410285 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2008100f60 0x7f200819c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:42.959 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:42.956+0000 7f200dfee640 1 --2- 172.21.15.80:0/1382410285 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20080ffe10 0x7f20081097c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:42.959 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:42.956+0000 7f200dfee640 1 -- 172.21.15.80:0/1382410285 >> 172.21.15.80:0/1382410285 conn(0x7f20080fb380 msgr2=0x7f20081042f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:42.959 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:42.956+0000 7f200dfee640 1 -- 172.21.15.80:0/1382410285 shutdown_connections 2024-04-24T23:27:42.959 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:42.956+0000 7f200dfee640 1 -- 172.21.15.80:0/1382410285 wait complete. 2024-04-24T23:27:42.968 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-04-24T23:27:42.968 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2024-04-24T23:27:43.072 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:43.069+0000 7fbfdd9c8640 1 -- 172.21.15.80:0/2560025718 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbfd8102240 msgr2=0x7fbfd8108ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:43.072 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:43.069+0000 7fbfdd9c8640 1 --2- 172.21.15.80:0/2560025718 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbfd8102240 0x7fbfd8108ad0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fbfd0009ca0 tx=0x7fbfd002a8c0 comp rx=0 tx=0).stop 2024-04-24T23:27:43.072 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:43.069+0000 7fbfdd9c8640 1 -- 172.21.15.80:0/2560025718 shutdown_connections 2024-04-24T23:27:43.072 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:43.069+0000 7fbfdd9c8640 1 --2- 172.21.15.80:0/2560025718 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbfd8102240 0x7fbfd8108ad0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:43.072 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:43.069+0000 7fbfdd9c8640 1 --2- 172.21.15.80:0/2560025718 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbfd8101880 0x7fbfd8101d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:43.072 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:43.069+0000 7fbfdd9c8640 1 --2- 172.21.15.80:0/2560025718 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbfd8100690 0x7fbfd8100ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:43.072 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:43.069+0000 7fbfdd9c8640 1 -- 172.21.15.80:0/2560025718 >> 172.21.15.80:0/2560025718 conn(0x7fbfd80fbc00 msgr2=0x7fbfd80fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:43.072 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:43.069+0000 7fbfdd9c8640 1 -- 172.21.15.80:0/2560025718 shutdown_connections 2024-04-24T23:27:43.073 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:43.069+0000 7fbfdd9c8640 1 -- 172.21.15.80:0/2560025718 wait complete. 2024-04-24T23:27:43.073 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:43.069+0000 7fbfdd9c8640 1 Processor -- start 2024-04-24T23:27:43.073 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:43.069+0000 7fbfdd9c8640 1 -- start start 2024-04-24T23:27:43.073 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:43.069+0000 7fbfdd9c8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbfd8100690 0x7fbfd806a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:43.073 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:43.069+0000 7fbfdd9c8640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbfd8101880 0x7fbfd819c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:43.073 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:43.069+0000 7fbfdd9c8640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbfd8102240 0x7fbfd819ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:43.073 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:43.070+0000 7fbfdd9c8640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd810c400 con 0x7fbfd8101880 2024-04-24T23:27:43.073 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:43.070+0000 7fbfdd9c8640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd810c280 con 0x7fbfd8102240 2024-04-24T23:27:43.073 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:43.070+0000 7fbfdd9c8640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd810c580 con 0x7fbfd8100690 2024-04-24T23:27:43.073 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:43.070+0000 7fbfd67fc640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbfd8101880 0x7fbfd819c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:43.073 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:43.070+0000 7fbfd67fc640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbfd8101880 0x7fbfd819c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:37322/0 (socket says 172.21.15.80:37322) 2024-04-24T23:27:43.073 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:43.070+0000 7fbfd67fc640 1 -- 172.21.15.80:0/2502468570 learned_addr learned my addr 172.21.15.80:0/2502468570 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:27:43.073 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:43.070+0000 7fbfd77fe640 1 --2- 172.21.15.80:0/2502468570 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbfd8102240 0x7fbfd819ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:43.074 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:43.070+0000 7fbfd6ffd640 1 --2- 172.21.15.80:0/2502468570 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbfd8100690 0x7fbfd806a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:43.074 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:43.070+0000 7fbfd67fc640 1 -- 172.21.15.80:0/2502468570 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbfd8100690 msgr2=0x7fbfd806a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:43.074 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:43.070+0000 7fbfd67fc640 1 --2- 172.21.15.80:0/2502468570 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbfd8100690 0x7fbfd806a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:43.074 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:43.070+0000 7fbfd67fc640 1 -- 172.21.15.80:0/2502468570 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbfd8102240 msgr2=0x7fbfd819ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:43.074 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:43.070+0000 7fbfd67fc640 1 --2- 172.21.15.80:0/2502468570 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbfd8102240 0x7fbfd819ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:43.074 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:43.070+0000 7fbfd67fc640 1 -- 172.21.15.80:0/2502468570 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfd819f830 con 0x7fbfd8101880 2024-04-24T23:27:43.074 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:43.070+0000 7fbfd67fc640 1 --2- 172.21.15.80:0/2502468570 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbfd8101880 0x7fbfd819c930 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fbfcc00b4a0 tx=0x7fbfcc00b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:43.074 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:43.070+0000 7fbfdc9c6640 1 -- 172.21.15.80:0/2502468570 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfcc015020 con 0x7fbfd8101880 2024-04-24T23:27:43.074 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:43.070+0000 7fbfdc9c6640 1 -- 172.21.15.80:0/2502468570 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbfcc011d10 con 0x7fbfd8101880 2024-04-24T23:27:43.074 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:43.070+0000 7fbfdd9c8640 1 -- 172.21.15.80:0/2502468570 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfd819fb20 con 0x7fbfd8101880 2024-04-24T23:27:43.074 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:43.070+0000 7fbfdc9c6640 1 -- 172.21.15.80:0/2502468570 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfcc00f2e0 con 0x7fbfd8101880 2024-04-24T23:27:43.074 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:43.070+0000 7fbfdd9c8640 1 -- 172.21.15.80:0/2502468570 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfd81a0070 con 0x7fbfd8101880 2024-04-24T23:27:43.074 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:43.071+0000 7fbfdd9c8640 1 -- 172.21.15.80:0/2502468570 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfd8107a80 con 0x7fbfd8101880 2024-04-24T23:27:43.075 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:43.072+0000 7fbfdc9c6640 1 -- 172.21.15.80:0/2502468570 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fbfcc018020 con 0x7fbfd8101880 2024-04-24T23:27:43.076 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:43.073+0000 7fbfdc9c6640 1 --2- 172.21.15.80:0/2502468570 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fbfc00a2b90 0x7fbfc00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:43.076 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:43.073+0000 7fbfd6ffd640 1 --2- 172.21.15.80:0/2502468570 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fbfc00a2b90 0x7fbfc00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:43.076 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:43.073+0000 7fbfdc9c6640 1 -- 172.21.15.80:0/2502468570 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fbfcc0cee90 con 0x7fbfd8101880 2024-04-24T23:27:43.076 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:43.073+0000 7fbfd6ffd640 1 --2- 172.21.15.80:0/2502468570 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fbfc00a2b90 0x7fbfc00a5050 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fbfd81016e0 tx=0x7fbfc8009110 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:43.077 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:43.076+0000 7fbfdc9c6640 1 -- 172.21.15.80:0/2502468570 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fbfcc01f020 con 0x7fbfd8101880 2024-04-24T23:27:43.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:42 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/661818757' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:27:43.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:42 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/997207059' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:27:43.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:42 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/1382410285' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-24T23:27:43.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:42 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/661818757' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:27:43.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:42 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/997207059' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:27:43.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:42 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/1382410285' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-24T23:27:43.289 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:43.287+0000 7fbfdd9c8640 1 -- 172.21.15.80:0/2502468570 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7fbfd802d370 con 0x7fbfd8101880 2024-04-24T23:27:43.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:42 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/661818757' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:27:43.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:42 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/997207059' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:27:43.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:42 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/1382410285' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-24T23:27:43.955 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:43.955+0000 7fbfdc9c6640 1 -- 172.21.15.80:0/2502468570 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v19) ==== 105+0+0 (secure 0 0 0) 0x7fbfcc09bbb0 con 0x7fbfd8101880 2024-04-24T23:27:43.958 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:43.957+0000 7fbfdd9c8640 1 -- 172.21.15.80:0/2502468570 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fbfc00a2b90 msgr2=0x7fbfc00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:43.958 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:43.957+0000 7fbfdd9c8640 1 --2- 172.21.15.80:0/2502468570 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fbfc00a2b90 0x7fbfc00a5050 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fbfd81016e0 tx=0x7fbfc8009110 comp rx=0 tx=0).stop 2024-04-24T23:27:43.958 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:43.957+0000 7fbfdd9c8640 1 -- 172.21.15.80:0/2502468570 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbfd8101880 msgr2=0x7fbfd819c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:43.959 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:43.957+0000 7fbfdd9c8640 1 --2- 172.21.15.80:0/2502468570 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbfd8101880 0x7fbfd819c930 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fbfcc00b4a0 tx=0x7fbfcc00b8a0 comp rx=0 tx=0).stop 2024-04-24T23:27:43.959 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:43.957+0000 7fbfdd9c8640 1 -- 172.21.15.80:0/2502468570 shutdown_connections 2024-04-24T23:27:43.959 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:43.957+0000 7fbfdd9c8640 1 --2- 172.21.15.80:0/2502468570 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fbfc00a2b90 0x7fbfc00a5050 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:43.959 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:43.957+0000 7fbfdd9c8640 1 --2- 172.21.15.80:0/2502468570 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbfd8102240 0x7fbfd819ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:43.959 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:43.957+0000 7fbfdd9c8640 1 --2- 172.21.15.80:0/2502468570 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbfd8101880 0x7fbfd819c930 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:43.959 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:43.957+0000 7fbfdd9c8640 1 --2- 172.21.15.80:0/2502468570 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbfd8100690 0x7fbfd806a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:43.959 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:43.957+0000 7fbfdd9c8640 1 -- 172.21.15.80:0/2502468570 >> 172.21.15.80:0/2502468570 conn(0x7fbfd80fbc00 msgr2=0x7fbfd8104c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:43.959 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:43.957+0000 7fbfdd9c8640 1 -- 172.21.15.80:0/2502468570 shutdown_connections 2024-04-24T23:27:43.959 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:43.957+0000 7fbfdd9c8640 1 -- 172.21.15.80:0/2502468570 wait complete. 2024-04-24T23:27:43.972 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-04-24T23:27:44.080 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:44.077+0000 7fc2c9db7640 1 -- 172.21.15.80:0/4276740700 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc2c4102230 msgr2=0x7fc2c4108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:44.080 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:44.077+0000 7fc2c9db7640 1 --2- 172.21.15.80:0/4276740700 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc2c4102230 0x7fc2c4108ac0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc009ca0 tx=0x7fc2bc02a8c0 comp rx=0 tx=0).stop 2024-04-24T23:27:44.081 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:44.077+0000 7fc2c9db7640 1 -- 172.21.15.80:0/4276740700 shutdown_connections 2024-04-24T23:27:44.081 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:44.077+0000 7fc2c9db7640 1 --2- 172.21.15.80:0/4276740700 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc2c4102230 0x7fc2c4108ac0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:44.081 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:44.077+0000 7fc2c9db7640 1 --2- 172.21.15.80:0/4276740700 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2c4101870 0x7fc2c4101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:44.081 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:44.077+0000 7fc2c9db7640 1 --2- 172.21.15.80:0/4276740700 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc2c4100680 0x7fc2c4100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:44.081 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:44.077+0000 7fc2c9db7640 1 -- 172.21.15.80:0/4276740700 >> 172.21.15.80:0/4276740700 conn(0x7fc2c40fbbf0 msgr2=0x7fc2c40fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:44.081 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:44.077+0000 7fc2c9db7640 1 -- 172.21.15.80:0/4276740700 shutdown_connections 2024-04-24T23:27:44.081 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:44.077+0000 7fc2c9db7640 1 -- 172.21.15.80:0/4276740700 wait complete. 2024-04-24T23:27:44.081 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:44.078+0000 7fc2c9db7640 1 Processor -- start 2024-04-24T23:27:44.081 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:44.078+0000 7fc2c9db7640 1 -- start start 2024-04-24T23:27:44.081 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:44.078+0000 7fc2c9db7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2c4100680 0x7fc2c4196980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:44.081 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:44.078+0000 7fc2c9db7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc2c4101870 0x7fc2c4196ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:44.081 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:44.078+0000 7fc2c9db7640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc2c4102230 0x7fc2c419df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:44.081 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:44.078+0000 7fc2c9db7640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fc2c410c300 con 0x7fc2c4102230 2024-04-24T23:27:44.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:44.078+0000 7fc2c9db7640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc2c410c180 con 0x7fc2c4101870 2024-04-24T23:27:44.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:44.078+0000 7fc2c9db7640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc2c410c480 con 0x7fc2c4100680 2024-04-24T23:27:44.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:44.078+0000 7fc2c3fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc2c4102230 0x7fc2c419df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:44.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:44.078+0000 7fc2c3fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc2c4102230 0x7fc2c419df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:37352/0 (socket says 172.21.15.80:37352) 2024-04-24T23:27:44.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:44.078+0000 7fc2c3fff640 1 -- 172.21.15.80:0/598878835 learned_addr learned my addr 172.21.15.80:0/598878835 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:27:44.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:44.078+0000 7fc2c2ffd640 1 --2- 172.21.15.80:0/598878835 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc2c4101870 0x7fc2c4196ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:44.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:44.078+0000 7fc2c3fff640 1 -- 172.21.15.80:0/598878835 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2c4100680 msgr2=0x7fc2c4196980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:44.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:44.078+0000 7fc2c3fff640 1 --2- 172.21.15.80:0/598878835 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2c4100680 0x7fc2c4196980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:44.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:44.078+0000 7fc2c3fff640 1 -- 172.21.15.80:0/598878835 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc2c4101870 msgr2=0x7fc2c4196ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:44.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:44.078+0000 7fc2c3fff640 1 --2- 172.21.15.80:0/598878835 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc2c4101870 0x7fc2c4196ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:44.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:44.078+0000 7fc2c3fff640 1 -- 172.21.15.80:0/598878835 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2c419e5a0 con 0x7fc2c4102230 2024-04-24T23:27:44.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:44.079+0000 7fc2c3fff640 1 --2- 172.21.15.80:0/598878835 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc2c4102230 0x7fc2c419df30 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc0098d0 tx=0x7fc2bc02f840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:44.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:44.079+0000 7fc2c0ff9640 1 -- 172.21.15.80:0/598878835 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2bc02fc30 con 0x7fc2c4102230 2024-04-24T23:27:44.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:44.079+0000 7fc2c9db7640 1 -- 172.21.15.80:0/598878835 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2c419e830 con 0x7fc2c4102230 2024-04-24T23:27:44.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:44.079+0000 7fc2c0ff9640 1 -- 172.21.15.80:0/598878835 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2bc003e50 con 0x7fc2c4102230 2024-04-24T23:27:44.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:44.079+0000 7fc2c0ff9640 1 -- 172.21.15.80:0/598878835 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2bc0305c0 con 0x7fc2c4102230 2024-04-24T23:27:44.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:44.079+0000 7fc2c9db7640 1 -- 172.21.15.80:0/598878835 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2c419edd0 con 0x7fc2c4102230 2024-04-24T23:27:44.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:44.080+0000 7fc2c9db7640 1 -- 172.21.15.80:0/598878835 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2c4107a70 con 0x7fc2c4102230 2024-04-24T23:27:44.084 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:44.081+0000 7fc2c0ff9640 1 -- 172.21.15.80:0/598878835 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fc2bc003970 con 0x7fc2c4102230 2024-04-24T23:27:44.084 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:44.081+0000 7fc2c0ff9640 1 --2- 172.21.15.80:0/598878835 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fc2ac0a2b90 0x7fc2ac0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:44.084 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:44.082+0000 7fc2c37fe640 1 --2- 172.21.15.80:0/598878835 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fc2ac0a2b90 0x7fc2ac0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:44.084 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:44.082+0000 7fc2c0ff9640 1 -- 172.21.15.80:0/598878835 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fc2bc0f0890 con 0x7fc2c4102230 2024-04-24T23:27:44.084 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:44.082+0000 7fc2c37fe640 1 --2- 172.21.15.80:0/598878835 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fc2ac0a2b90 0x7fc2ac0a5050 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc2c41016d0 tx=0x7fc2b4005c10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:44.086 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:44.085+0000 7fc2c0ff9640 1 -- 172.21.15.80:0/598878835 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fc2bc0bd5b0 con 0x7fc2c4102230 2024-04-24T23:27:44.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:43 smithi148 ceph-mon[32528]: pgmap v308: 129 pgs: 129 active+clean; 581 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-24T23:27:44.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:43 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/1382410285' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-24T23:27:44.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:43 smithi148 ceph-mon[32528]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-04-24T23:27:44.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:43 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/2502468570' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-24T23:27:44.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:43 smithi080 ceph-mon[29152]: pgmap v308: 129 pgs: 129 active+clean; 581 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-24T23:27:44.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:43 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/1382410285' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-24T23:27:44.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:43 smithi080 ceph-mon[29152]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-04-24T23:27:44.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:43 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/2502468570' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-24T23:27:44.298 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:44.297+0000 7fc2c9db7640 1 -- 172.21.15.80:0/598878835 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7fc2c4062340 con 0x7fc2c4102230 2024-04-24T23:27:44.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:43 smithi134 ceph-mon[32619]: pgmap v308: 129 pgs: 129 active+clean; 581 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-24T23:27:44.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:43 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/1382410285' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-24T23:27:44.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:43 smithi134 ceph-mon[32619]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-04-24T23:27:44.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:43 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/2502468570' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-24T23:27:44.968 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:44.967+0000 7fc2c0ff9640 1 -- 172.21.15.80:0/598878835 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v21) ==== 121+0+0 (secure 0 0 0) 0x7fc2bc0bceb0 con 0x7fc2c4102230 2024-04-24T23:27:44.971 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:44.970+0000 7fc2c9db7640 1 -- 172.21.15.80:0/598878835 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fc2ac0a2b90 msgr2=0x7fc2ac0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:44.971 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:44.970+0000 7fc2c9db7640 1 --2- 172.21.15.80:0/598878835 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fc2ac0a2b90 0x7fc2ac0a5050 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc2c41016d0 tx=0x7fc2b4005c10 comp rx=0 tx=0).stop 2024-04-24T23:27:44.971 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:44.970+0000 7fc2c9db7640 1 -- 172.21.15.80:0/598878835 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc2c4102230 msgr2=0x7fc2c419df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:44.971 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:44.970+0000 7fc2c9db7640 1 --2- 172.21.15.80:0/598878835 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc2c4102230 0x7fc2c419df30 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc0098d0 tx=0x7fc2bc02f840 comp rx=0 tx=0).stop 2024-04-24T23:27:44.971 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:44.970+0000 7fc2c9db7640 1 -- 172.21.15.80:0/598878835 shutdown_connections 2024-04-24T23:27:44.971 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:44.970+0000 7fc2c9db7640 1 --2- 172.21.15.80:0/598878835 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fc2ac0a2b90 0x7fc2ac0a5050 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:44.971 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:44.970+0000 7fc2c9db7640 1 --2- 172.21.15.80:0/598878835 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc2c4102230 0x7fc2c419df30 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:44.971 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:44.970+0000 7fc2c9db7640 1 --2- 172.21.15.80:0/598878835 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc2c4101870 0x7fc2c4196ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:44.971 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:44.970+0000 7fc2c9db7640 1 --2- 172.21.15.80:0/598878835 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2c4100680 0x7fc2c4196980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:44.971 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:44.970+0000 7fc2c9db7640 1 -- 172.21.15.80:0/598878835 >> 172.21.15.80:0/598878835 conn(0x7fc2c40fbbf0 msgr2=0x7fc2c4104c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:44.971 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:44.970+0000 7fc2c9db7640 1 -- 172.21.15.80:0/598878835 shutdown_connections 2024-04-24T23:27:44.972 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:44.970+0000 7fc2c9db7640 1 -- 172.21.15.80:0/598878835 wait complete. 2024-04-24T23:27:44.987 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-04-24T23:27:45.095 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:45.091+0000 7f5a09fee640 1 -- 172.21.15.80:0/569326879 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5a04100680 msgr2=0x7f5a04100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:45.095 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:45.091+0000 7f5a09fee640 1 --2- 172.21.15.80:0/569326879 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5a04100680 0x7f5a04100aa0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f59f4009900 tx=0x7f59f402a410 comp rx=0 tx=0).stop 2024-04-24T23:27:45.095 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:45.091+0000 7f5a09fee640 1 -- 172.21.15.80:0/569326879 shutdown_connections 2024-04-24T23:27:45.095 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:45.091+0000 7f5a09fee640 1 --2- 172.21.15.80:0/569326879 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a04102230 0x7f5a04108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:45.095 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:45.091+0000 7f5a09fee640 1 --2- 172.21.15.80:0/569326879 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a04101870 0x7f5a04101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:45.095 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:45.091+0000 7f5a09fee640 1 --2- 172.21.15.80:0/569326879 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5a04100680 0x7f5a04100aa0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:45.095 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:45.091+0000 7f5a09fee640 1 -- 172.21.15.80:0/569326879 >> 172.21.15.80:0/569326879 conn(0x7f5a040fbbf0 msgr2=0x7f5a040fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:45.095 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:45.091+0000 7f5a09fee640 1 -- 172.21.15.80:0/569326879 shutdown_connections 2024-04-24T23:27:45.096 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:45.091+0000 7f5a09fee640 1 -- 172.21.15.80:0/569326879 wait complete. 2024-04-24T23:27:45.096 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:45.091+0000 7f5a09fee640 1 Processor -- start 2024-04-24T23:27:45.096 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:45.092+0000 7f5a09fee640 1 -- start start 2024-04-24T23:27:45.096 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:45.092+0000 7f5a09fee640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5a04100680 0x7f5a04197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:45.096 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:45.092+0000 7f5a09fee640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a04101870 0x7f5a04198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:45.096 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:45.092+0000 7f5a09fee640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a04102230 0x7f5a0419f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:45.096 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:45.092+0000 7f5a09fee640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f5a0410c370 con 0x7f5a04100680 2024-04-24T23:27:45.096 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:45.092+0000 7f5a09fee640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5a0410c1f0 con 0x7f5a04102230 2024-04-24T23:27:45.097 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:45.092+0000 7f5a09fee640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5a0410c4f0 con 0x7f5a04101870 2024-04-24T23:27:45.097 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:45.092+0000 7f5a037fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5a04100680 0x7f5a04197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:45.097 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:45.092+0000 7f5a037fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5a04100680 0x7f5a04197b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:37360/0 (socket says 172.21.15.80:37360) 2024-04-24T23:27:45.097 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:45.092+0000 7f5a037fe640 1 -- 172.21.15.80:0/563957532 learned_addr learned my addr 172.21.15.80:0/563957532 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:27:45.097 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:45.092+0000 7f5a02ffd640 1 --2- 172.21.15.80:0/563957532 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a04101870 0x7f5a04198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:45.097 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:45.092+0000 7f5a037fe640 1 -- 172.21.15.80:0/563957532 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a04101870 msgr2=0x7f5a04198080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:45.097 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:45.092+0000 7f5a037fe640 1 --2- 172.21.15.80:0/563957532 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a04101870 0x7f5a04198080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:45.097 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:45.092+0000 7f5a037fe640 1 -- 172.21.15.80:0/563957532 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a04102230 msgr2=0x7f5a0419f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:45.097 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:45.092+0000 7f5a037fe640 1 --2- 172.21.15.80:0/563957532 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a04102230 0x7f5a0419f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:45.097 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:45.092+0000 7f5a037fe640 1 -- 172.21.15.80:0/563957532 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a0419f760 con 0x7f5a04100680 2024-04-24T23:27:45.097 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:45.092+0000 7f5a037fe640 1 --2- 172.21.15.80:0/563957532 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5a04100680 0x7f5a04197b40 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f59f4031610 tx=0x7f59f4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:45.097 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:45.092+0000 7f5a00ff9640 1 -- 172.21.15.80:0/563957532 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59f403f030 con 0x7f5a04100680 2024-04-24T23:27:45.097 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:45.093+0000 7f5a09fee640 1 -- 172.21.15.80:0/563957532 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a0419f9f0 con 0x7f5a04100680 2024-04-24T23:27:45.097 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:45.093+0000 7f5a00ff9640 1 -- 172.21.15.80:0/563957532 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59f4003ad0 con 0x7f5a04100680 2024-04-24T23:27:45.098 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:45.093+0000 7f5a00ff9640 1 -- 172.21.15.80:0/563957532 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59f4035070 con 0x7f5a04100680 2024-04-24T23:27:45.098 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:45.093+0000 7f5a09fee640 1 -- 172.21.15.80:0/563957532 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a0419ff60 con 0x7f5a04100680 2024-04-24T23:27:45.098 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:45.093+0000 7f5a09fee640 1 -- 172.21.15.80:0/563957532 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a041a0130 con 0x7f5a04100680 2024-04-24T23:27:45.099 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:45.094+0000 7f5a00ff9640 1 -- 172.21.15.80:0/563957532 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f59f4042030 con 0x7f5a04100680 2024-04-24T23:27:45.099 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:45.095+0000 7f5a00ff9640 1 --2- 172.21.15.80:0/563957532 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f59ec0a2b90 0x7f59ec0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:45.099 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:45.095+0000 7f5a02ffd640 1 --2- 172.21.15.80:0/563957532 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f59ec0a2b90 0x7f59ec0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:45.099 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:45.095+0000 7f5a00ff9640 1 -- 172.21.15.80:0/563957532 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f59f40efa00 con 0x7f5a04100680 2024-04-24T23:27:45.099 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:45.096+0000 7f5a02ffd640 1 --2- 172.21.15.80:0/563957532 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f59ec0a2b90 0x7f59ec0a5050 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5a04199050 tx=0x7f59f8009110 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:45.100 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:45.100+0000 7f5a00ff9640 1 -- 172.21.15.80:0/563957532 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f59f40bc720 con 0x7f5a04100680 2024-04-24T23:27:45.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:44 smithi148 ceph-mon[32528]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-24T23:27:45.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:44 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/2502468570' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-04-24T23:27:45.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:44 smithi148 ceph-mon[32528]: daemon mds.b assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-24T23:27:45.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:44 smithi148 ceph-mon[32528]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-04-24T23:27:45.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:44 smithi148 ceph-mon[32528]: fsmap cephfs:2 {0=i=up:active,1=b=up:creating} 10 up:standby 2024-04-24T23:27:45.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:44 smithi148 ceph-mon[32528]: daemon mds.b is now active in filesystem cephfs as rank 1 2024-04-24T23:27:45.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:44 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/598878835' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-24T23:27:45.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:44 smithi080 ceph-mon[29152]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-24T23:27:45.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:44 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/2502468570' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-04-24T23:27:45.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:44 smithi080 ceph-mon[29152]: daemon mds.b assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-24T23:27:45.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:44 smithi080 ceph-mon[29152]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-04-24T23:27:45.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:44 smithi080 ceph-mon[29152]: fsmap cephfs:2 {0=i=up:active,1=b=up:creating} 10 up:standby 2024-04-24T23:27:45.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:44 smithi080 ceph-mon[29152]: daemon mds.b is now active in filesystem cephfs as rank 1 2024-04-24T23:27:45.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:44 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/598878835' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-24T23:27:45.316 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:45.315+0000 7f5a09fee640 1 -- 172.21.15.80:0/563957532 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f5a041a03a0 con 0x7f5a04100680 2024-04-24T23:27:45.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:44 smithi134 ceph-mon[32619]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-24T23:27:45.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:44 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/2502468570' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-04-24T23:27:45.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:44 smithi134 ceph-mon[32619]: daemon mds.b assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-24T23:27:45.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:44 smithi134 ceph-mon[32619]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-04-24T23:27:45.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:44 smithi134 ceph-mon[32619]: fsmap cephfs:2 {0=i=up:active,1=b=up:creating} 10 up:standby 2024-04-24T23:27:45.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:44 smithi134 ceph-mon[32619]: daemon mds.b is now active in filesystem cephfs as rank 1 2024-04-24T23:27:45.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:44 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/598878835' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-24T23:27:45.979 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:45.977+0000 7f5a00ff9640 1 -- 172.21.15.80:0/563957532 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v23) ==== 115+0+0 (secure 0 0 0) 0x7f59f40bc020 con 0x7f5a04100680 2024-04-24T23:27:45.982 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:45.980+0000 7f5a09fee640 1 -- 172.21.15.80:0/563957532 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f59ec0a2b90 msgr2=0x7f59ec0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:45.982 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:45.980+0000 7f5a09fee640 1 --2- 172.21.15.80:0/563957532 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f59ec0a2b90 0x7f59ec0a5050 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5a04199050 tx=0x7f59f8009110 comp rx=0 tx=0).stop 2024-04-24T23:27:45.982 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:45.980+0000 7f5a09fee640 1 -- 172.21.15.80:0/563957532 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5a04100680 msgr2=0x7f5a04197b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:45.983 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:45.980+0000 7f5a09fee640 1 --2- 172.21.15.80:0/563957532 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5a04100680 0x7f5a04197b40 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f59f4031610 tx=0x7f59f4004270 comp rx=0 tx=0).stop 2024-04-24T23:27:45.983 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:45.980+0000 7f5a09fee640 1 -- 172.21.15.80:0/563957532 shutdown_connections 2024-04-24T23:27:45.983 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:45.980+0000 7f5a09fee640 1 --2- 172.21.15.80:0/563957532 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f59ec0a2b90 0x7f59ec0a5050 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:45.983 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:45.980+0000 7f5a09fee640 1 --2- 172.21.15.80:0/563957532 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a04102230 0x7f5a0419f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:45.983 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:45.980+0000 7f5a09fee640 1 --2- 172.21.15.80:0/563957532 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a04101870 0x7f5a04198080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:45.983 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:45.980+0000 7f5a09fee640 1 --2- 172.21.15.80:0/563957532 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5a04100680 0x7f5a04197b40 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:45.983 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:45.980+0000 7f5a09fee640 1 -- 172.21.15.80:0/563957532 >> 172.21.15.80:0/563957532 conn(0x7f5a040fbbf0 msgr2=0x7f5a040fd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:45.983 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:45.980+0000 7f5a09fee640 1 -- 172.21.15.80:0/563957532 shutdown_connections 2024-04-24T23:27:45.983 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:45.980+0000 7f5a09fee640 1 -- 172.21.15.80:0/563957532 wait complete. 2024-04-24T23:27:45.996 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--size 25000000000'} subvols for filesystem 'cephfs' 2024-04-24T23:27:45.997 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-04-24T23:27:46.111 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.104+0000 7fad1d3e7640 1 -- 172.21.15.80:0/1020970910 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fad18102d90 msgr2=0x7fad181031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:46.252 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.104+0000 7fad1d3e7640 1 --2- 172.21.15.80:0/1020970910 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fad18102d90 0x7fad181031b0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fad08009900 tx=0x7fad0802a410 comp rx=0 tx=0).stop 2024-04-24T23:27:46.254 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.104+0000 7fad1d3e7640 1 -- 172.21.15.80:0/1020970910 shutdown_connections 2024-04-24T23:27:46.254 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.104+0000 7fad1d3e7640 1 --2- 172.21.15.80:0/1020970910 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad181001c0 0x7fad18109d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:46.254 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.104+0000 7fad1d3e7640 1 --2- 172.21.15.80:0/1020970910 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fad180ff800 0x7fad180ffc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:46.254 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.104+0000 7fad1d3e7640 1 --2- 172.21.15.80:0/1020970910 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fad18102d90 0x7fad181031b0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:46.254 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.104+0000 7fad1d3e7640 1 -- 172.21.15.80:0/1020970910 >> 172.21.15.80:0/1020970910 conn(0x7fad180fb380 msgr2=0x7fad180fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:46.254 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.104+0000 7fad1d3e7640 1 -- 172.21.15.80:0/1020970910 shutdown_connections 2024-04-24T23:27:46.254 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.104+0000 7fad1d3e7640 1 -- 172.21.15.80:0/1020970910 wait complete. 2024-04-24T23:27:46.254 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.104+0000 7fad1d3e7640 1 Processor -- start 2024-04-24T23:27:46.254 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.105+0000 7fad1d3e7640 1 -- start start 2024-04-24T23:27:46.254 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.105+0000 7fad1d3e7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad180ff800 0x7fad18102490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:46.254 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.105+0000 7fad1d3e7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fad181001c0 0x7fad18100ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:46.255 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.105+0000 7fad1d3e7640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fad18102d90 0x7fad18101010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:46.255 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.105+0000 7fad1d3e7640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fad1810d410 con 0x7fad18102d90 2024-04-24T23:27:46.255 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.105+0000 7fad1d3e7640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fad1810d290 con 0x7fad181001c0 2024-04-24T23:27:46.255 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.105+0000 7fad1d3e7640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fad1810d590 con 0x7fad180ff800 2024-04-24T23:27:46.255 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.105+0000 7fad177fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fad18102d90 0x7fad18101010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:46.255 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.105+0000 7fad177fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fad18102d90 0x7fad18101010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:37370/0 (socket says 172.21.15.80:37370) 2024-04-24T23:27:46.255 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.105+0000 7fad177fe640 1 -- 172.21.15.80:0/1769165244 learned_addr learned my addr 172.21.15.80:0/1769165244 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:27:46.255 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.105+0000 7fad16ffd640 1 --2- 172.21.15.80:0/1769165244 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad180ff800 0x7fad18102490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:46.255 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.105+0000 7fad177fe640 1 -- 172.21.15.80:0/1769165244 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad180ff800 msgr2=0x7fad18102490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:46.255 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.105+0000 7fad177fe640 1 --2- 172.21.15.80:0/1769165244 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad180ff800 0x7fad18102490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:46.255 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.105+0000 7fad167fc640 1 --2- 172.21.15.80:0/1769165244 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fad181001c0 0x7fad18100ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:46.255 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.105+0000 7fad177fe640 1 -- 172.21.15.80:0/1769165244 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fad181001c0 msgr2=0x7fad18100ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:46.255 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.105+0000 7fad177fe640 1 --2- 172.21.15.80:0/1769165244 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fad181001c0 0x7fad18100ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:46.255 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.105+0000 7fad177fe640 1 -- 172.21.15.80:0/1769165244 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad181018a0 con 0x7fad18102d90 2024-04-24T23:27:46.256 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.105+0000 7fad177fe640 1 --2- 172.21.15.80:0/1769165244 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fad18102d90 0x7fad18101010 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fad1000eb10 tx=0x7fad1000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:46.256 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.106+0000 7facfffff640 1 -- 172.21.15.80:0/1769165244 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad10004b20 con 0x7fad18102d90 2024-04-24T23:27:46.256 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.106+0000 7fad1d3e7640 1 -- 172.21.15.80:0/1769165244 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad181a1ba0 con 0x7fad18102d90 2024-04-24T23:27:46.256 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.106+0000 7facfffff640 1 -- 172.21.15.80:0/1769165244 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad10004cc0 con 0x7fad18102d90 2024-04-24T23:27:46.256 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.106+0000 7facfffff640 1 -- 172.21.15.80:0/1769165244 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad10010760 con 0x7fad18102d90 2024-04-24T23:27:46.256 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.106+0000 7fad1d3e7640 1 -- 172.21.15.80:0/1769165244 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad181a20f0 con 0x7fad18102d90 2024-04-24T23:27:46.256 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.107+0000 7fad1d3e7640 1 -- 172.21.15.80:0/1769165244 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad18108ca0 con 0x7fad18102d90 2024-04-24T23:27:46.256 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.108+0000 7facfffff640 1 -- 172.21.15.80:0/1769165244 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fad10005020 con 0x7fad18102d90 2024-04-24T23:27:46.256 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.108+0000 7facfffff640 1 --2- 172.21.15.80:0/1769165244 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fad000a2b90 0x7fad000a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:46.256 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.108+0000 7fad16ffd640 1 --2- 172.21.15.80:0/1769165244 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fad000a2b90 0x7fad000a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:46.256 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.108+0000 7facfffff640 1 -- 172.21.15.80:0/1769165244 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fad100cfa20 con 0x7fad18102d90 2024-04-24T23:27:46.256 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.109+0000 7fad16ffd640 1 --2- 172.21.15.80:0/1769165244 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fad000a2b90 0x7fad000a5050 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fad08005ab0 tx=0x7fad08000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:46.257 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.113+0000 7facfffff640 1 -- 172.21.15.80:0/1769165244 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fad1009c740 con 0x7fad18102d90 2024-04-24T23:27:46.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:45 smithi148 ceph-mon[32528]: pgmap v309: 129 pgs: 129 active+clean; 581 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-24T23:27:46.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:45 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/598878835' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-24T23:27:46.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:45 smithi148 ceph-mon[32528]: mds.? [v2:172.21.15.134:6838/1743815021,v1:172.21.15.134:6839/1743815021] up:active 2024-04-24T23:27:46.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:45 smithi148 ceph-mon[32528]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-24T23:27:46.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:45 smithi148 ceph-mon[32528]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-24T23:27:46.258 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:45 smithi148 ceph-mon[32528]: Cluster is now healthy 2024-04-24T23:27:46.258 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:45 smithi148 ceph-mon[32528]: fsmap cephfs:2 {0=i=up:active,1=b=up:active} 10 up:standby 2024-04-24T23:27:46.258 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:45 smithi148 ceph-mon[32528]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:creating} 9 up:standby 2024-04-24T23:27:46.258 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:45 smithi148 ceph-mon[32528]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-04-24T23:27:46.258 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:45 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/563957532' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-24T23:27:46.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:45 smithi080 ceph-mon[29152]: pgmap v309: 129 pgs: 129 active+clean; 581 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-24T23:27:46.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:45 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/598878835' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-24T23:27:46.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:45 smithi080 ceph-mon[29152]: mds.? [v2:172.21.15.134:6838/1743815021,v1:172.21.15.134:6839/1743815021] up:active 2024-04-24T23:27:46.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:45 smithi080 ceph-mon[29152]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-24T23:27:46.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:45 smithi080 ceph-mon[29152]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-24T23:27:46.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:45 smithi080 ceph-mon[29152]: Cluster is now healthy 2024-04-24T23:27:46.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:45 smithi080 ceph-mon[29152]: fsmap cephfs:2 {0=i=up:active,1=b=up:active} 10 up:standby 2024-04-24T23:27:46.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:45 smithi080 ceph-mon[29152]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:creating} 9 up:standby 2024-04-24T23:27:46.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:45 smithi080 ceph-mon[29152]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-04-24T23:27:46.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:45 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/563957532' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-24T23:27:46.315 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.314+0000 7fad1d3e7640 1 -- 172.21.15.80:0/1769165244 --> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fad18061b80 con 0x7fad000a2b90 2024-04-24T23:27:46.342 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.341+0000 7facfffff640 1 -- 172.21.15.80:0/1769165244 <== mgr.14152 v2:172.21.15.80:6800/3830596922 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fad18061b80 con 0x7fad000a2b90 2024-04-24T23:27:46.344 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.343+0000 7fad1d3e7640 1 -- 172.21.15.80:0/1769165244 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fad000a2b90 msgr2=0x7fad000a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:46.344 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.343+0000 7fad1d3e7640 1 --2- 172.21.15.80:0/1769165244 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fad000a2b90 0x7fad000a5050 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fad08005ab0 tx=0x7fad08000bc0 comp rx=0 tx=0).stop 2024-04-24T23:27:46.344 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.343+0000 7fad1d3e7640 1 -- 172.21.15.80:0/1769165244 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fad18102d90 msgr2=0x7fad18101010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:46.345 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.343+0000 7fad1d3e7640 1 --2- 172.21.15.80:0/1769165244 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fad18102d90 0x7fad18101010 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fad1000eb10 tx=0x7fad1000c2d0 comp rx=0 tx=0).stop 2024-04-24T23:27:46.345 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.343+0000 7fad1d3e7640 1 -- 172.21.15.80:0/1769165244 shutdown_connections 2024-04-24T23:27:46.345 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.343+0000 7fad1d3e7640 1 --2- 172.21.15.80:0/1769165244 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fad000a2b90 0x7fad000a5050 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:46.345 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.343+0000 7fad1d3e7640 1 --2- 172.21.15.80:0/1769165244 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fad18102d90 0x7fad18101010 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:46.345 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.343+0000 7fad1d3e7640 1 --2- 172.21.15.80:0/1769165244 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fad181001c0 0x7fad18100ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:46.345 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.343+0000 7fad1d3e7640 1 --2- 172.21.15.80:0/1769165244 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad180ff800 0x7fad18102490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:46.345 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.343+0000 7fad1d3e7640 1 -- 172.21.15.80:0/1769165244 >> 172.21.15.80:0/1769165244 conn(0x7fad180fb380 msgr2=0x7fad18105e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:46.345 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.343+0000 7fad1d3e7640 1 -- 172.21.15.80:0/1769165244 shutdown_connections 2024-04-24T23:27:46.345 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.343+0000 7fad1d3e7640 1 -- 172.21.15.80:0/1769165244 wait complete. 2024-04-24T23:27:46.355 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-04-24T23:27:46.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:45 smithi134 ceph-mon[32619]: pgmap v309: 129 pgs: 129 active+clean; 581 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-24T23:27:46.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:45 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/598878835' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-24T23:27:46.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:45 smithi134 ceph-mon[32619]: mds.? [v2:172.21.15.134:6838/1743815021,v1:172.21.15.134:6839/1743815021] up:active 2024-04-24T23:27:46.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:45 smithi134 ceph-mon[32619]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-24T23:27:46.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:45 smithi134 ceph-mon[32619]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-24T23:27:46.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:45 smithi134 ceph-mon[32619]: Cluster is now healthy 2024-04-24T23:27:46.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:45 smithi134 ceph-mon[32619]: fsmap cephfs:2 {0=i=up:active,1=b=up:active} 10 up:standby 2024-04-24T23:27:46.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:45 smithi134 ceph-mon[32619]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:creating} 9 up:standby 2024-04-24T23:27:46.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:45 smithi134 ceph-mon[32619]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-04-24T23:27:46.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:45 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/563957532' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-24T23:27:46.460 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.457+0000 7fbe2123f640 1 -- 172.21.15.80:0/391180650 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbe1c101890 msgr2=0x7fbe1c101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:46.460 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.457+0000 7fbe2123f640 1 --2- 172.21.15.80:0/391180650 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbe1c101890 0x7fbe1c101d10 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fbe04009900 tx=0x7fbe0402a410 comp rx=0 tx=0).stop 2024-04-24T23:27:46.461 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.457+0000 7fbe2123f640 1 -- 172.21.15.80:0/391180650 shutdown_connections 2024-04-24T23:27:46.461 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.457+0000 7fbe2123f640 1 --2- 172.21.15.80:0/391180650 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe1c102250 0x7fbe1c108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:46.461 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.457+0000 7fbe2123f640 1 --2- 172.21.15.80:0/391180650 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbe1c101890 0x7fbe1c101d10 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:46.461 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.457+0000 7fbe2123f640 1 --2- 172.21.15.80:0/391180650 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbe1c1006a0 0x7fbe1c100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:46.461 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.457+0000 7fbe2123f640 1 -- 172.21.15.80:0/391180650 >> 172.21.15.80:0/391180650 conn(0x7fbe1c0fbc30 msgr2=0x7fbe1c0fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:46.461 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.457+0000 7fbe2123f640 1 -- 172.21.15.80:0/391180650 shutdown_connections 2024-04-24T23:27:46.461 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.457+0000 7fbe2123f640 1 -- 172.21.15.80:0/391180650 wait complete. 2024-04-24T23:27:46.461 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.458+0000 7fbe2123f640 1 Processor -- start 2024-04-24T23:27:46.461 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.458+0000 7fbe2123f640 1 -- start start 2024-04-24T23:27:46.461 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.458+0000 7fbe2123f640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbe1c1006a0 0x7fbe1c196a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:46.461 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.458+0000 7fbe2123f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbe1c101890 0x7fbe1c196f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:46.461 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.458+0000 7fbe2123f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe1c102250 0x7fbe1c19dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:46.461 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.458+0000 7fbe2123f640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fbe1c10c320 con 0x7fbe1c1006a0 2024-04-24T23:27:46.461 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.458+0000 7fbe2123f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbe1c10c1a0 con 0x7fbe1c102250 2024-04-24T23:27:46.462 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.458+0000 7fbe2123f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fbe1c10c4a0 con 0x7fbe1c101890 2024-04-24T23:27:46.462 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.458+0000 7fbe1ad9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbe1c1006a0 0x7fbe1c196a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:46.462 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.458+0000 7fbe1ad9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbe1c1006a0 0x7fbe1c196a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:37412/0 (socket says 172.21.15.80:37412) 2024-04-24T23:27:46.462 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.458+0000 7fbe1ad9d640 1 -- 172.21.15.80:0/3368332141 learned_addr learned my addr 172.21.15.80:0/3368332141 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:27:46.462 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.458+0000 7fbe1259c640 1 --2- 172.21.15.80:0/3368332141 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbe1c101890 0x7fbe1c196f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:46.462 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.458+0000 7fbe1b59e640 1 --2- 172.21.15.80:0/3368332141 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe1c102250 0x7fbe1c19dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:46.462 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.458+0000 7fbe1ad9d640 1 -- 172.21.15.80:0/3368332141 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbe1c101890 msgr2=0x7fbe1c196f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:46.462 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.458+0000 7fbe1ad9d640 1 --2- 172.21.15.80:0/3368332141 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbe1c101890 0x7fbe1c196f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:46.462 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.458+0000 7fbe1ad9d640 1 -- 172.21.15.80:0/3368332141 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe1c102250 msgr2=0x7fbe1c19dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:46.462 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.458+0000 7fbe1ad9d640 1 --2- 172.21.15.80:0/3368332141 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe1c102250 0x7fbe1c19dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:46.462 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.458+0000 7fbe1ad9d640 1 -- 172.21.15.80:0/3368332141 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe1c19e630 con 0x7fbe1c1006a0 2024-04-24T23:27:46.462 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.459+0000 7fbe1ad9d640 1 --2- 172.21.15.80:0/3368332141 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbe1c1006a0 0x7fbe1c196a10 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c00e9c0 tx=0x7fbe0c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:46.462 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.459+0000 7fbe18d99640 1 -- 172.21.15.80:0/3368332141 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe0c00cd80 con 0x7fbe1c1006a0 2024-04-24T23:27:46.462 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.459+0000 7fbe18d99640 1 -- 172.21.15.80:0/3368332141 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe0c0035f0 con 0x7fbe1c1006a0 2024-04-24T23:27:46.463 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.459+0000 7fbe2123f640 1 -- 172.21.15.80:0/3368332141 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe1c19e920 con 0x7fbe1c1006a0 2024-04-24T23:27:46.463 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.459+0000 7fbe18d99640 1 -- 172.21.15.80:0/3368332141 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe0c010690 con 0x7fbe1c1006a0 2024-04-24T23:27:46.463 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.459+0000 7fbe2123f640 1 -- 172.21.15.80:0/3368332141 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe1c19edf0 con 0x7fbe1c1006a0 2024-04-24T23:27:46.464 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.460+0000 7fbe2123f640 1 -- 172.21.15.80:0/3368332141 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe1c107a80 con 0x7fbe1c1006a0 2024-04-24T23:27:46.464 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.461+0000 7fbe18d99640 1 -- 172.21.15.80:0/3368332141 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fbe0c0037a0 con 0x7fbe1c1006a0 2024-04-24T23:27:46.464 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.461+0000 7fbe18d99640 1 --2- 172.21.15.80:0/3368332141 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fbe080a2b90 0x7fbe080a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:46.464 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.462+0000 7fbe1259c640 1 --2- 172.21.15.80:0/3368332141 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fbe080a2b90 0x7fbe080a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:46.464 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.462+0000 7fbe18d99640 1 -- 172.21.15.80:0/3368332141 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fbe0c0cf5d0 con 0x7fbe1c1006a0 2024-04-24T23:27:46.464 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.462+0000 7fbe1259c640 1 --2- 172.21.15.80:0/3368332141 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fbe080a2b90 0x7fbe080a5050 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fbe040314e0 tx=0x7fbe04000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:46.467 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.466+0000 7fbe18d99640 1 -- 172.21.15.80:0/3368332141 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fbe0c09c2f0 con 0x7fbe1c1006a0 2024-04-24T23:27:46.667 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.666+0000 7fbe2123f640 1 -- 172.21.15.80:0/3368332141 --> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fbe1c061b80 con 0x7fbe080a2b90 2024-04-24T23:27:46.691 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.691+0000 7fbe18d99640 1 -- 172.21.15.80:0/3368332141 <== mgr.14152 v2:172.21.15.80:6800/3830596922 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fbe1c061b80 con 0x7fbe080a2b90 2024-04-24T23:27:46.694 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.692+0000 7fbe2123f640 1 -- 172.21.15.80:0/3368332141 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fbe080a2b90 msgr2=0x7fbe080a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:46.694 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.693+0000 7fbe2123f640 1 --2- 172.21.15.80:0/3368332141 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fbe080a2b90 0x7fbe080a5050 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fbe040314e0 tx=0x7fbe04000bc0 comp rx=0 tx=0).stop 2024-04-24T23:27:46.694 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.693+0000 7fbe2123f640 1 -- 172.21.15.80:0/3368332141 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbe1c1006a0 msgr2=0x7fbe1c196a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:46.694 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.693+0000 7fbe2123f640 1 --2- 172.21.15.80:0/3368332141 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbe1c1006a0 0x7fbe1c196a10 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c00e9c0 tx=0x7fbe0c00ee80 comp rx=0 tx=0).stop 2024-04-24T23:27:46.694 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.693+0000 7fbe2123f640 1 -- 172.21.15.80:0/3368332141 shutdown_connections 2024-04-24T23:27:46.694 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.693+0000 7fbe2123f640 1 --2- 172.21.15.80:0/3368332141 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fbe080a2b90 0x7fbe080a5050 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:46.694 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.693+0000 7fbe2123f640 1 --2- 172.21.15.80:0/3368332141 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe1c102250 0x7fbe1c19dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:46.694 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.693+0000 7fbe2123f640 1 --2- 172.21.15.80:0/3368332141 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbe1c101890 0x7fbe1c196f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:46.694 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.693+0000 7fbe2123f640 1 --2- 172.21.15.80:0/3368332141 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbe1c1006a0 0x7fbe1c196a10 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:46.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.693+0000 7fbe2123f640 1 -- 172.21.15.80:0/3368332141 >> 172.21.15.80:0/3368332141 conn(0x7fbe1c0fbc30 msgr2=0x7fbe1c104c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:46.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.693+0000 7fbe2123f640 1 -- 172.21.15.80:0/3368332141 shutdown_connections 2024-04-24T23:27:46.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.693+0000 7fbe2123f640 1 -- 172.21.15.80:0/3368332141 wait complete. 2024-04-24T23:27:46.704 INFO:journalctl@ceph.mgr.x.smithi080.stdout:Apr 24 23:27:46 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-x[29381]: 2024-04-24T23:27:46.319+0000 7f5d164f0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T23:27:46.704 INFO:journalctl@ceph.mgr.x.smithi080.stdout:Apr 24 23:27:46 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-x[29381]: 2024-04-24T23:27:46.319+0000 7f5d164f0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T23:27:46.704 INFO:journalctl@ceph.mgr.x.smithi080.stdout:Apr 24 23:27:46 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-x[29381]: 2024-04-24T23:27:46.319+0000 7f5d164f0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T23:27:46.704 INFO:journalctl@ceph.mgr.x.smithi080.stdout:Apr 24 23:27:46 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-x[29381]: 2024-04-24T23:27:46.319+0000 7f5d164f0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T23:27:46.704 INFO:journalctl@ceph.mgr.x.smithi080.stdout:Apr 24 23:27:46 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-x[29381]: 2024-04-24T23:27:46.319+0000 7f5d164f0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T23:27:46.705 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-04-24T23:27:46.810 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.807+0000 7f36ed5b7640 1 -- 172.21.15.80:0/1571907006 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f36e8102230 msgr2=0x7f36e8108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:46.810 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.807+0000 7f36ed5b7640 1 --2- 172.21.15.80:0/1571907006 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f36e8102230 0x7f36e8108ac0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f36e0009ca0 tx=0x7f36e002a8c0 comp rx=0 tx=0).stop 2024-04-24T23:27:46.810 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.807+0000 7f36ed5b7640 1 -- 172.21.15.80:0/1571907006 shutdown_connections 2024-04-24T23:27:46.810 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.807+0000 7f36ed5b7640 1 --2- 172.21.15.80:0/1571907006 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f36e8102230 0x7f36e8108ac0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:46.811 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.807+0000 7f36ed5b7640 1 --2- 172.21.15.80:0/1571907006 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36e8101870 0x7f36e8101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:46.811 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.807+0000 7f36ed5b7640 1 --2- 172.21.15.80:0/1571907006 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f36e8100680 0x7f36e8100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:46.811 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.807+0000 7f36ed5b7640 1 -- 172.21.15.80:0/1571907006 >> 172.21.15.80:0/1571907006 conn(0x7f36e80fbbf0 msgr2=0x7f36e80fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:46.811 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.807+0000 7f36ed5b7640 1 -- 172.21.15.80:0/1571907006 shutdown_connections 2024-04-24T23:27:46.811 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.807+0000 7f36ed5b7640 1 -- 172.21.15.80:0/1571907006 wait complete. 2024-04-24T23:27:46.811 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.807+0000 7f36ed5b7640 1 Processor -- start 2024-04-24T23:27:46.811 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.807+0000 7f36ed5b7640 1 -- start start 2024-04-24T23:27:46.811 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.807+0000 7f36ed5b7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f36e8100680 0x7f36e8069440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:46.811 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.807+0000 7f36ed5b7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36e8101870 0x7f36e819ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:46.811 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.808+0000 7f36ed5b7640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f36e8102230 0x7f36e81a1120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:46.811 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.808+0000 7f36ed5b7640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f36e810c300 con 0x7f36e8102230 2024-04-24T23:27:46.811 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.808+0000 7f36ed5b7640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f36e810c180 con 0x7f36e8100680 2024-04-24T23:27:46.811 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.808+0000 7f36ed5b7640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f36e810c480 con 0x7f36e8101870 2024-04-24T23:27:46.811 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.808+0000 7f36e77fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f36e8102230 0x7f36e81a1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:46.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.808+0000 7f36e77fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f36e8102230 0x7f36e81a1120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:37430/0 (socket says 172.21.15.80:37430) 2024-04-24T23:27:46.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.808+0000 7f36e77fe640 1 -- 172.21.15.80:0/896792367 learned_addr learned my addr 172.21.15.80:0/896792367 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:27:46.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.808+0000 7f36dffff640 1 --2- 172.21.15.80:0/896792367 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36e8101870 0x7f36e819ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:46.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.808+0000 7f36e6ffd640 1 --2- 172.21.15.80:0/896792367 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f36e8100680 0x7f36e8069440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:46.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.808+0000 7f36e77fe640 1 -- 172.21.15.80:0/896792367 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36e8101870 msgr2=0x7f36e819ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:46.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.808+0000 7f36e77fe640 1 --2- 172.21.15.80:0/896792367 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36e8101870 0x7f36e819ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:46.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.808+0000 7f36e77fe640 1 -- 172.21.15.80:0/896792367 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f36e8100680 msgr2=0x7f36e8069440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:46.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.808+0000 7f36e77fe640 1 --2- 172.21.15.80:0/896792367 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f36e8100680 0x7f36e8069440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:46.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.808+0000 7f36e77fe640 1 -- 172.21.15.80:0/896792367 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36e81a1910 con 0x7f36e8102230 2024-04-24T23:27:46.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.808+0000 7f36e77fe640 1 --2- 172.21.15.80:0/896792367 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f36e8102230 0x7f36e81a1120 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f36e0031ae0 tx=0x7f36e0031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:46.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.808+0000 7f36e4ff9640 1 -- 172.21.15.80:0/896792367 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36e0003c30 con 0x7f36e8102230 2024-04-24T23:27:46.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.808+0000 7f36e4ff9640 1 -- 172.21.15.80:0/896792367 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36e0003dd0 con 0x7f36e8102230 2024-04-24T23:27:46.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.808+0000 7f36ed5b7640 1 -- 172.21.15.80:0/896792367 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36e81a1b40 con 0x7f36e8102230 2024-04-24T23:27:46.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.808+0000 7f36e4ff9640 1 -- 172.21.15.80:0/896792367 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36e002fcb0 con 0x7f36e8102230 2024-04-24T23:27:46.813 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.808+0000 7f36ed5b7640 1 -- 172.21.15.80:0/896792367 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36e81a2090 con 0x7f36e8102230 2024-04-24T23:27:46.813 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.809+0000 7f36ed5b7640 1 -- 172.21.15.80:0/896792367 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36e8107a70 con 0x7f36e8102230 2024-04-24T23:27:46.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.810+0000 7f36e4ff9640 1 -- 172.21.15.80:0/896792367 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f36e0004050 con 0x7f36e8102230 2024-04-24T23:27:46.815 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.811+0000 7f36e4ff9640 1 --2- 172.21.15.80:0/896792367 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f36d40a2bc0 0x7f36d40a5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:46.815 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.811+0000 7f36e6ffd640 1 --2- 172.21.15.80:0/896792367 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f36d40a2bc0 0x7f36d40a5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:46.815 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.811+0000 7f36e4ff9640 1 -- 172.21.15.80:0/896792367 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f36e00f0530 con 0x7f36e8102230 2024-04-24T23:27:46.815 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.811+0000 7f36e6ffd640 1 --2- 172.21.15.80:0/896792367 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f36d40a2bc0 0x7f36d40a5080 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f36d8005db0 tx=0x7f36d8005d40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:46.816 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.816+0000 7f36e4ff9640 1 -- 172.21.15.80:0/896792367 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f36e00bd370 con 0x7f36e8102230 2024-04-24T23:27:46.999 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:46.998+0000 7f36e4ff9640 1 -- 172.21.15.80:0/896792367 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f36e00b92e0 con 0x7f36e8102230 2024-04-24T23:27:47.025 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.024+0000 7f36ed5b7640 1 -- 172.21.15.80:0/896792367 --> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f36e8061b80 con 0x7f36d40a2bc0 2024-04-24T23:27:47.030 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.029+0000 7f36e4ff9640 1 -- 172.21.15.80:0/896792367 <== mgr.14152 v2:172.21.15.80:6800/3830596922 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f36e8061b80 con 0x7f36d40a2bc0 2024-04-24T23:27:47.030 INFO:teuthology.orchestra.run.smithi080.stdout:/volumes/qa/sv_0/58614f39-f067-46f1-bcb5-6cf10714f9fe 2024-04-24T23:27:47.032 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.031+0000 7f36ed5b7640 1 -- 172.21.15.80:0/896792367 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f36d40a2bc0 msgr2=0x7f36d40a5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:47.032 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.031+0000 7f36ed5b7640 1 --2- 172.21.15.80:0/896792367 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f36d40a2bc0 0x7f36d40a5080 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f36d8005db0 tx=0x7f36d8005d40 comp rx=0 tx=0).stop 2024-04-24T23:27:47.032 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.031+0000 7f36ed5b7640 1 -- 172.21.15.80:0/896792367 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f36e8102230 msgr2=0x7f36e81a1120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:47.032 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.031+0000 7f36ed5b7640 1 --2- 172.21.15.80:0/896792367 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f36e8102230 0x7f36e81a1120 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f36e0031ae0 tx=0x7f36e0031ee0 comp rx=0 tx=0).stop 2024-04-24T23:27:47.033 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.031+0000 7f36ed5b7640 1 -- 172.21.15.80:0/896792367 shutdown_connections 2024-04-24T23:27:47.033 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.031+0000 7f36ed5b7640 1 --2- 172.21.15.80:0/896792367 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f36d40a2bc0 0x7f36d40a5080 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:47.033 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.031+0000 7f36ed5b7640 1 --2- 172.21.15.80:0/896792367 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f36e8102230 0x7f36e81a1120 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:47.033 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.031+0000 7f36ed5b7640 1 --2- 172.21.15.80:0/896792367 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36e8101870 0x7f36e819ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:47.033 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.031+0000 7f36ed5b7640 1 --2- 172.21.15.80:0/896792367 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f36e8100680 0x7f36e8069440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:47.033 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.031+0000 7f36ed5b7640 1 -- 172.21.15.80:0/896792367 >> 172.21.15.80:0/896792367 conn(0x7f36e80fbbf0 msgr2=0x7f36e8104c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:47.033 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.031+0000 7f36ed5b7640 1 -- 172.21.15.80:0/896792367 shutdown_connections 2024-04-24T23:27:47.033 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.031+0000 7f36ed5b7640 1 -- 172.21.15.80:0/896792367 wait complete. 2024-04-24T23:27:47.043 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-04-24T23:27:47.150 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.145+0000 7f4cd803f640 1 -- 172.21.15.80:0/2679323427 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4cd0101870 msgr2=0x7f4cd0101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:47.150 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.145+0000 7f4cd803f640 1 --2- 172.21.15.80:0/2679323427 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4cd0101870 0x7f4cd0101cf0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f4cc0009900 tx=0x7f4cc002a410 comp rx=0 tx=0).stop 2024-04-24T23:27:47.150 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.145+0000 7f4cd803f640 1 -- 172.21.15.80:0/2679323427 shutdown_connections 2024-04-24T23:27:47.150 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.145+0000 7f4cd803f640 1 --2- 172.21.15.80:0/2679323427 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4cd0102230 0x7f4cd0108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:47.150 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.145+0000 7f4cd803f640 1 --2- 172.21.15.80:0/2679323427 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4cd0101870 0x7f4cd0101cf0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:47.150 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.145+0000 7f4cd803f640 1 --2- 172.21.15.80:0/2679323427 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4cd0100680 0x7f4cd0100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:47.150 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.145+0000 7f4cd803f640 1 -- 172.21.15.80:0/2679323427 >> 172.21.15.80:0/2679323427 conn(0x7f4cd00fbbf0 msgr2=0x7f4cd00fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:47.150 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.145+0000 7f4cd803f640 1 -- 172.21.15.80:0/2679323427 shutdown_connections 2024-04-24T23:27:47.150 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.145+0000 7f4cd803f640 1 -- 172.21.15.80:0/2679323427 wait complete. 2024-04-24T23:27:47.150 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.145+0000 7f4cd803f640 1 Processor -- start 2024-04-24T23:27:47.151 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.145+0000 7f4cd803f640 1 -- start start 2024-04-24T23:27:47.151 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.145+0000 7f4cd803f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4cd0100680 0x7f4cd0197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:47.151 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.145+0000 7f4cd803f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4cd0101870 0x7f4cd0198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:47.151 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.145+0000 7f4cd803f640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4cd0102230 0x7f4cd019f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:47.151 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.145+0000 7f4cd803f640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f4cd010c300 con 0x7f4cd0102230 2024-04-24T23:27:47.151 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.145+0000 7f4cd803f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4cd010c180 con 0x7f4cd0101870 2024-04-24T23:27:47.151 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.145+0000 7f4cd803f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4cd010c480 con 0x7f4cd0100680 2024-04-24T23:27:47.151 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.146+0000 7f4cd65dc640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4cd0102230 0x7f4cd019f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:47.151 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.146+0000 7f4cd65dc640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4cd0102230 0x7f4cd019f0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:37460/0 (socket says 172.21.15.80:37460) 2024-04-24T23:27:47.151 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.146+0000 7f4cd65dc640 1 -- 172.21.15.80:0/3842715094 learned_addr learned my addr 172.21.15.80:0/3842715094 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:27:47.151 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.146+0000 7f4cd5ddb640 1 --2- 172.21.15.80:0/3842715094 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4cd0100680 0x7f4cd0197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:47.151 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.146+0000 7f4cd55da640 1 --2- 172.21.15.80:0/3842715094 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4cd0101870 0x7f4cd0198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:47.151 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.146+0000 7f4cd65dc640 1 -- 172.21.15.80:0/3842715094 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4cd0100680 msgr2=0x7f4cd0197b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:47.152 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.146+0000 7f4cd65dc640 1 --2- 172.21.15.80:0/3842715094 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4cd0100680 0x7f4cd0197b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:47.152 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.146+0000 7f4cd65dc640 1 -- 172.21.15.80:0/3842715094 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4cd0101870 msgr2=0x7f4cd0198060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:47.152 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.146+0000 7f4cd65dc640 1 --2- 172.21.15.80:0/3842715094 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4cd0101870 0x7f4cd0198060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:47.152 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.146+0000 7f4cd65dc640 1 -- 172.21.15.80:0/3842715094 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cd019f740 con 0x7f4cd0102230 2024-04-24T23:27:47.152 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.146+0000 7f4cd5ddb640 1 --2- 172.21.15.80:0/3842715094 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4cd0100680 0x7f4cd0197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:27:47.152 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.146+0000 7f4cd55da640 1 --2- 172.21.15.80:0/3842715094 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4cd0101870 0x7f4cd0198060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:27:47.152 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.146+0000 7f4cd65dc640 1 --2- 172.21.15.80:0/3842715094 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4cd0102230 0x7f4cd019f0d0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f4cc8009fd0 tx=0x7f4cc800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:47.152 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.146+0000 7f4cc6ffd640 1 -- 172.21.15.80:0/3842715094 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cc8017070 con 0x7f4cd0102230 2024-04-24T23:27:47.152 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.146+0000 7f4cc6ffd640 1 -- 172.21.15.80:0/3842715094 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4cc8003e50 con 0x7f4cd0102230 2024-04-24T23:27:47.152 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.146+0000 7f4cd803f640 1 -- 172.21.15.80:0/3842715094 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cd019fa30 con 0x7f4cd0102230 2024-04-24T23:27:47.152 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.146+0000 7f4cc6ffd640 1 -- 172.21.15.80:0/3842715094 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cc80106c0 con 0x7f4cd0102230 2024-04-24T23:27:47.152 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.146+0000 7f4cd803f640 1 -- 172.21.15.80:0/3842715094 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cd019ff30 con 0x7f4cd0102230 2024-04-24T23:27:47.152 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.147+0000 7f4cd803f640 1 -- 172.21.15.80:0/3842715094 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4cd01a0160 con 0x7f4cd0102230 2024-04-24T23:27:47.152 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.148+0000 7f4cc6ffd640 1 -- 172.21.15.80:0/3842715094 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f4cc8003970 con 0x7f4cd0102230 2024-04-24T23:27:47.153 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.149+0000 7f4cc6ffd640 1 --2- 172.21.15.80:0/3842715094 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f4cbc0a2c20 0x7f4cbc0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:47.153 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.149+0000 7f4cd5ddb640 1 --2- 172.21.15.80:0/3842715094 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f4cbc0a2c20 0x7f4cbc0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:47.153 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.149+0000 7f4cc6ffd640 1 -- 172.21.15.80:0/3842715094 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f4cc80cf260 con 0x7f4cd0102230 2024-04-24T23:27:47.153 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.149+0000 7f4cd5ddb640 1 --2- 172.21.15.80:0/3842715094 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f4cbc0a2c20 0x7f4cbc0a50e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc0058c0 tx=0x7f4ccc005ec0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:47.153 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.152+0000 7f4cc6ffd640 1 -- 172.21.15.80:0/3842715094 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f4cc809be20 con 0x7f4cd0102230 2024-04-24T23:27:47.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:46 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/563957532' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-24T23:27:47.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:46 smithi148 ceph-mon[32528]: mds.? [v2:172.21.15.148:6836/856528403,v1:172.21.15.148:6837/856528403] up:active 2024-04-24T23:27:47.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:46 smithi148 ceph-mon[32528]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 9 up:standby 2024-04-24T23:27:47.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:46 smithi148 ceph-mon[32528]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T23:27:47.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:46 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/563957532' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-24T23:27:47.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:46 smithi080 ceph-mon[29152]: mds.? [v2:172.21.15.148:6836/856528403,v1:172.21.15.148:6837/856528403] up:active 2024-04-24T23:27:47.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:46 smithi080 ceph-mon[29152]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 9 up:standby 2024-04-24T23:27:47.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:46 smithi080 ceph-mon[29152]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T23:27:47.356 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.355+0000 7f4cd803f640 1 -- 172.21.15.80:0/3842715094 --> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f4cd00fde20 con 0x7f4cbc0a2c20 2024-04-24T23:27:47.382 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.381+0000 7f4cc6ffd640 1 -- 172.21.15.80:0/3842715094 <== mgr.14152 v2:172.21.15.80:6800/3830596922 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f4cd00fde20 con 0x7f4cbc0a2c20 2024-04-24T23:27:47.384 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.383+0000 7f4cd803f640 1 -- 172.21.15.80:0/3842715094 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f4cbc0a2c20 msgr2=0x7f4cbc0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:47.385 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.383+0000 7f4cd803f640 1 --2- 172.21.15.80:0/3842715094 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f4cbc0a2c20 0x7f4cbc0a50e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc0058c0 tx=0x7f4ccc005ec0 comp rx=0 tx=0).stop 2024-04-24T23:27:47.385 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.383+0000 7f4cd803f640 1 -- 172.21.15.80:0/3842715094 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4cd0102230 msgr2=0x7f4cd019f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:47.385 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.383+0000 7f4cd803f640 1 --2- 172.21.15.80:0/3842715094 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4cd0102230 0x7f4cd019f0d0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f4cc8009fd0 tx=0x7f4cc800b260 comp rx=0 tx=0).stop 2024-04-24T23:27:47.385 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.383+0000 7f4cd803f640 1 -- 172.21.15.80:0/3842715094 shutdown_connections 2024-04-24T23:27:47.385 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.383+0000 7f4cd803f640 1 --2- 172.21.15.80:0/3842715094 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f4cbc0a2c20 0x7f4cbc0a50e0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:47.385 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.383+0000 7f4cd803f640 1 --2- 172.21.15.80:0/3842715094 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4cd0102230 0x7f4cd019f0d0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:47.385 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.383+0000 7f4cd803f640 1 --2- 172.21.15.80:0/3842715094 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4cd0101870 0x7f4cd0198060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:47.385 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.383+0000 7f4cd803f640 1 --2- 172.21.15.80:0/3842715094 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4cd0100680 0x7f4cd0197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:47.385 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.383+0000 7f4cd803f640 1 -- 172.21.15.80:0/3842715094 >> 172.21.15.80:0/3842715094 conn(0x7f4cd00fbbf0 msgr2=0x7f4cd00fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:47.385 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.383+0000 7f4cd803f640 1 -- 172.21.15.80:0/3842715094 shutdown_connections 2024-04-24T23:27:47.385 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.383+0000 7f4cd803f640 1 -- 172.21.15.80:0/3842715094 wait complete. 2024-04-24T23:27:47.395 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-04-24T23:27:47.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:46 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/563957532' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-24T23:27:47.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:46 smithi134 ceph-mon[32619]: mds.? [v2:172.21.15.148:6836/856528403,v1:172.21.15.148:6837/856528403] up:active 2024-04-24T23:27:47.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:46 smithi134 ceph-mon[32619]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 9 up:standby 2024-04-24T23:27:47.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:46 smithi134 ceph-mon[32619]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T23:27:47.504 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.497+0000 7fa5bcbb0640 1 -- 172.21.15.80:0/3956465770 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa5b8102250 msgr2=0x7fa5b8108ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:47.505 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.497+0000 7fa5bcbb0640 1 --2- 172.21.15.80:0/3956465770 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa5b8102250 0x7fa5b8108ae0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fa5a8009900 tx=0x7fa5a802a410 comp rx=0 tx=0).stop 2024-04-24T23:27:47.505 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.497+0000 7fa5bcbb0640 1 -- 172.21.15.80:0/3956465770 shutdown_connections 2024-04-24T23:27:47.505 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.497+0000 7fa5bcbb0640 1 --2- 172.21.15.80:0/3956465770 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa5b8102250 0x7fa5b8108ae0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:47.505 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.497+0000 7fa5bcbb0640 1 --2- 172.21.15.80:0/3956465770 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5b8101890 0x7fa5b8101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:47.505 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.497+0000 7fa5bcbb0640 1 --2- 172.21.15.80:0/3956465770 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa5b81006a0 0x7fa5b8100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:47.505 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.497+0000 7fa5bcbb0640 1 -- 172.21.15.80:0/3956465770 >> 172.21.15.80:0/3956465770 conn(0x7fa5b80fbc10 msgr2=0x7fa5b80fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:47.505 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.497+0000 7fa5bcbb0640 1 -- 172.21.15.80:0/3956465770 shutdown_connections 2024-04-24T23:27:47.505 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.497+0000 7fa5bcbb0640 1 -- 172.21.15.80:0/3956465770 wait complete. 2024-04-24T23:27:47.505 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.497+0000 7fa5bcbb0640 1 Processor -- start 2024-04-24T23:27:47.505 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.498+0000 7fa5bcbb0640 1 -- start start 2024-04-24T23:27:47.505 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.498+0000 7fa5bcbb0640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa5b81006a0 0x7fa5b8197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:47.505 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.498+0000 7fa5bcbb0640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5b8101890 0x7fa5b8197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:47.506 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.498+0000 7fa5bcbb0640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa5b8102250 0x7fa5b819efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:47.506 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.498+0000 7fa5bcbb0640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fa5b810c320 con 0x7fa5b81006a0 2024-04-24T23:27:47.506 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.498+0000 7fa5bcbb0640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa5b810c1a0 con 0x7fa5b8102250 2024-04-24T23:27:47.506 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.498+0000 7fa5bcbb0640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa5b810c4a0 con 0x7fa5b8101890 2024-04-24T23:27:47.506 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.498+0000 7fa5b659c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa5b81006a0 0x7fa5b8197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:47.506 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.498+0000 7fa5b659c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa5b81006a0 0x7fa5b8197a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:37484/0 (socket says 172.21.15.80:37484) 2024-04-24T23:27:47.506 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.498+0000 7fa5b659c640 1 -- 172.21.15.80:0/3399062221 learned_addr learned my addr 172.21.15.80:0/3399062221 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:27:47.506 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.498+0000 7fa5b5d9b640 1 --2- 172.21.15.80:0/3399062221 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5b8101890 0x7fa5b8197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:47.506 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.498+0000 7fa5b6d9d640 1 --2- 172.21.15.80:0/3399062221 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa5b8102250 0x7fa5b819efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:47.506 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.498+0000 7fa5b659c640 1 -- 172.21.15.80:0/3399062221 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5b8101890 msgr2=0x7fa5b8197f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:47.506 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.498+0000 7fa5b659c640 1 --2- 172.21.15.80:0/3399062221 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5b8101890 0x7fa5b8197f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:47.506 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.498+0000 7fa5b659c640 1 -- 172.21.15.80:0/3399062221 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa5b8102250 msgr2=0x7fa5b819efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:47.506 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.498+0000 7fa5b659c640 1 --2- 172.21.15.80:0/3399062221 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa5b8102250 0x7fa5b819efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:47.506 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.498+0000 7fa5b659c640 1 -- 172.21.15.80:0/3399062221 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5b819f640 con 0x7fa5b81006a0 2024-04-24T23:27:47.507 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.498+0000 7fa5b6d9d640 1 --2- 172.21.15.80:0/3399062221 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa5b8102250 0x7fa5b819efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:27:47.507 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.498+0000 7fa5b659c640 1 --2- 172.21.15.80:0/3399062221 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa5b81006a0 0x7fa5b8197a20 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fa5b000c970 tx=0x7fa5b000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:47.507 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.498+0000 7fa5a77fe640 1 -- 172.21.15.80:0/3399062221 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b0007c10 con 0x7fa5b81006a0 2024-04-24T23:27:47.507 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.498+0000 7fa5bcbb0640 1 -- 172.21.15.80:0/3399062221 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5b819f930 con 0x7fa5b81006a0 2024-04-24T23:27:47.507 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.498+0000 7fa5a77fe640 1 -- 172.21.15.80:0/3399062221 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5b0007db0 con 0x7fa5b81006a0 2024-04-24T23:27:47.507 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.498+0000 7fa5a77fe640 1 -- 172.21.15.80:0/3399062221 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b000f8a0 con 0x7fa5b81006a0 2024-04-24T23:27:47.507 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.499+0000 7fa5bcbb0640 1 -- 172.21.15.80:0/3399062221 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5b819fed0 con 0x7fa5b81006a0 2024-04-24T23:27:47.507 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.499+0000 7fa5bcbb0640 1 -- 172.21.15.80:0/3399062221 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5b81a0160 con 0x7fa5b81006a0 2024-04-24T23:27:47.507 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.501+0000 7fa5a77fe640 1 -- 172.21.15.80:0/3399062221 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fa5b000fa40 con 0x7fa5b81006a0 2024-04-24T23:27:47.507 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.501+0000 7fa5a77fe640 1 --2- 172.21.15.80:0/3399062221 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fa5a00a2c20 0x7fa5a00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:47.507 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.501+0000 7fa5b5d9b640 1 --2- 172.21.15.80:0/3399062221 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fa5a00a2c20 0x7fa5a00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:47.507 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.501+0000 7fa5a77fe640 1 -- 172.21.15.80:0/3399062221 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fa5b00cf9a0 con 0x7fa5b81006a0 2024-04-24T23:27:47.507 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.502+0000 7fa5b5d9b640 1 --2- 172.21.15.80:0/3399062221 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fa5a00a2c20 0x7fa5a00a50e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa5b8198f30 tx=0x7fa5ac009680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:47.508 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.505+0000 7fa5a77fe640 1 -- 172.21.15.80:0/3399062221 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa5b009c680 con 0x7fa5b81006a0 2024-04-24T23:27:47.707 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.706+0000 7fa5bcbb0640 1 -- 172.21.15.80:0/3399062221 --> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7fa5b80fdd30 con 0x7fa5a00a2c20 2024-04-24T23:27:47.712 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.712+0000 7fa5a77fe640 1 -- 172.21.15.80:0/3399062221 <== mgr.14152 v2:172.21.15.80:6800/3830596922 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fa5b80fdd30 con 0x7fa5a00a2c20 2024-04-24T23:27:47.713 INFO:teuthology.orchestra.run.smithi080.stdout:/volumes/qa/sv_1/521b5c7e-1c80-4acd-899a-7184a5da80b8 2024-04-24T23:27:47.715 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.713+0000 7fa5bcbb0640 1 -- 172.21.15.80:0/3399062221 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fa5a00a2c20 msgr2=0x7fa5a00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:47.715 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.713+0000 7fa5bcbb0640 1 --2- 172.21.15.80:0/3399062221 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fa5a00a2c20 0x7fa5a00a50e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa5b8198f30 tx=0x7fa5ac009680 comp rx=0 tx=0).stop 2024-04-24T23:27:47.715 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.713+0000 7fa5bcbb0640 1 -- 172.21.15.80:0/3399062221 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa5b81006a0 msgr2=0x7fa5b8197a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:47.715 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.713+0000 7fa5bcbb0640 1 --2- 172.21.15.80:0/3399062221 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa5b81006a0 0x7fa5b8197a20 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fa5b000c970 tx=0x7fa5b000ce30 comp rx=0 tx=0).stop 2024-04-24T23:27:47.715 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.713+0000 7fa5bcbb0640 1 -- 172.21.15.80:0/3399062221 shutdown_connections 2024-04-24T23:27:47.715 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.713+0000 7fa5bcbb0640 1 --2- 172.21.15.80:0/3399062221 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fa5a00a2c20 0x7fa5a00a50e0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:47.715 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.713+0000 7fa5bcbb0640 1 --2- 172.21.15.80:0/3399062221 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa5b8102250 0x7fa5b819efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:47.715 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.713+0000 7fa5bcbb0640 1 --2- 172.21.15.80:0/3399062221 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5b8101890 0x7fa5b8197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:47.715 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.713+0000 7fa5bcbb0640 1 --2- 172.21.15.80:0/3399062221 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa5b81006a0 0x7fa5b8197a20 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:47.715 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.714+0000 7fa5bcbb0640 1 -- 172.21.15.80:0/3399062221 >> 172.21.15.80:0/3399062221 conn(0x7fa5b80fbc10 msgr2=0x7fa5b80fd620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:47.715 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.714+0000 7fa5bcbb0640 1 -- 172.21.15.80:0/3399062221 shutdown_connections 2024-04-24T23:27:47.716 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.714+0000 7fa5bcbb0640 1 -- 172.21.15.80:0/3399062221 wait complete. 2024-04-24T23:27:47.725 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:27:47.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.821+0000 7fa1c9929640 1 -- 172.21.15.80:0/2032007926 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa1c4101930 msgr2=0x7fa1c4108250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:47.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.821+0000 7fa1c9929640 1 --2- 172.21.15.80:0/2032007926 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa1c4101930 0x7fa1c4108250 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fa1b4009900 tx=0x7fa1b402a410 comp rx=0 tx=0).stop 2024-04-24T23:27:47.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.822+0000 7fa1c9929640 1 -- 172.21.15.80:0/2032007926 shutdown_connections 2024-04-24T23:27:47.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.822+0000 7fa1c9929640 1 --2- 172.21.15.80:0/2032007926 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa1c4101930 0x7fa1c4108250 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:47.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.822+0000 7fa1c9929640 1 --2- 172.21.15.80:0/2032007926 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa1c4100f70 0x7fa1c41013f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:47.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.822+0000 7fa1c9929640 1 --2- 172.21.15.80:0/2032007926 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1c40ffe20 0x7fa1c4100240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:47.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.822+0000 7fa1c9929640 1 -- 172.21.15.80:0/2032007926 >> 172.21.15.80:0/2032007926 conn(0x7fa1c40fb3b0 msgr2=0x7fa1c40fd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:47.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.822+0000 7fa1c9929640 1 -- 172.21.15.80:0/2032007926 shutdown_connections 2024-04-24T23:27:47.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.822+0000 7fa1c9929640 1 -- 172.21.15.80:0/2032007926 wait complete. 2024-04-24T23:27:47.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.822+0000 7fa1c9929640 1 Processor -- start 2024-04-24T23:27:47.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.822+0000 7fa1c9929640 1 -- start start 2024-04-24T23:27:47.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.822+0000 7fa1c9929640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa1c40ffe20 0x7fa1c4197a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:47.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.822+0000 7fa1c9929640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa1c4100f70 0x7fa1c4197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:47.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.822+0000 7fa1c9929640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1c4101930 0x7fa1c419efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:47.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.822+0000 7fa1c9929640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fa1c410c330 con 0x7fa1c40ffe20 2024-04-24T23:27:47.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.822+0000 7fa1c9929640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa1c410c1b0 con 0x7fa1c4100f70 2024-04-24T23:27:47.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.822+0000 7fa1c9929640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa1c410c4b0 con 0x7fa1c4101930 2024-04-24T23:27:47.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.823+0000 7fa1c2ffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa1c40ffe20 0x7fa1c4197a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:47.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.823+0000 7fa1c2ffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa1c40ffe20 0x7fa1c4197a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:37506/0 (socket says 172.21.15.80:37506) 2024-04-24T23:27:47.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.823+0000 7fa1c2ffd640 1 -- 172.21.15.80:0/476755511 learned_addr learned my addr 172.21.15.80:0/476755511 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:27:47.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.823+0000 7fa1c37fe640 1 --2- 172.21.15.80:0/476755511 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1c4101930 0x7fa1c419efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:47.827 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.823+0000 7fa1c27fc640 1 --2- 172.21.15.80:0/476755511 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa1c4100f70 0x7fa1c4197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:47.827 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.823+0000 7fa1c2ffd640 1 -- 172.21.15.80:0/476755511 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1c4101930 msgr2=0x7fa1c419efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:47.827 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.823+0000 7fa1c2ffd640 1 --2- 172.21.15.80:0/476755511 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1c4101930 0x7fa1c419efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:47.827 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.823+0000 7fa1c2ffd640 1 -- 172.21.15.80:0/476755511 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa1c4100f70 msgr2=0x7fa1c4197f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:47.827 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.823+0000 7fa1c2ffd640 1 --2- 172.21.15.80:0/476755511 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa1c4100f70 0x7fa1c4197f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:47.827 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.823+0000 7fa1c2ffd640 1 -- 172.21.15.80:0/476755511 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1c419f650 con 0x7fa1c40ffe20 2024-04-24T23:27:47.827 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.823+0000 7fa1c2ffd640 1 --2- 172.21.15.80:0/476755511 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa1c40ffe20 0x7fa1c4197a30 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fa1bc00d950 tx=0x7fa1bc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:47.827 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.823+0000 7fa1c8927640 1 -- 172.21.15.80:0/476755511 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1bc017070 con 0x7fa1c40ffe20 2024-04-24T23:27:47.827 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.823+0000 7fa1c8927640 1 -- 172.21.15.80:0/476755511 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1bc009c30 con 0x7fa1c40ffe20 2024-04-24T23:27:47.827 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.823+0000 7fa1c9929640 1 -- 172.21.15.80:0/476755511 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1c419f940 con 0x7fa1c40ffe20 2024-04-24T23:27:47.827 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.823+0000 7fa1c8927640 1 -- 172.21.15.80:0/476755511 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1bc00f6b0 con 0x7fa1c40ffe20 2024-04-24T23:27:47.827 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.823+0000 7fa1c9929640 1 -- 172.21.15.80:0/476755511 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1c419fe90 con 0x7fa1c40ffe20 2024-04-24T23:27:47.827 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.824+0000 7fa1c9929640 1 -- 172.21.15.80:0/476755511 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1c40689b0 con 0x7fa1c40ffe20 2024-04-24T23:27:47.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.825+0000 7fa1c8927640 1 -- 172.21.15.80:0/476755511 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fa1bc0109d0 con 0x7fa1c40ffe20 2024-04-24T23:27:47.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.826+0000 7fa1c8927640 1 --2- 172.21.15.80:0/476755511 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fa1ac0a2c20 0x7fa1ac0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:47.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.826+0000 7fa1c27fc640 1 --2- 172.21.15.80:0/476755511 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fa1ac0a2c20 0x7fa1ac0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:47.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.826+0000 7fa1c8927640 1 -- 172.21.15.80:0/476755511 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fa1bc0cefa0 con 0x7fa1c40ffe20 2024-04-24T23:27:47.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.826+0000 7fa1c27fc640 1 --2- 172.21.15.80:0/476755511 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fa1ac0a2c20 0x7fa1ac0a50e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa1c4198f40 tx=0x7fa1b8009670 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:47.830 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:47.829+0000 7fa1c8927640 1 -- 172.21.15.80:0/476755511 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa1bc009de0 con 0x7fa1c40ffe20 2024-04-24T23:27:48.042 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.041+0000 7fa1c9929640 1 -- 172.21.15.80:0/476755511 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa1c41a0350 con 0x7fa1c40ffe20 2024-04-24T23:27:48.043 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.042+0000 7fa1c8927640 1 -- 172.21.15.80:0/476755511 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11872 (secure 0 0 0) 0x7fa1bc09b510 con 0x7fa1c40ffe20 2024-04-24T23:27:48.043 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-24T23:27:48.043 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/891591085","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":891591085},{"type":"v1","addr":"172.21.15.80:6835","nonce":891591085}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6837/259529026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6836","nonce":259529026},{"type":"v1","addr":"172.21.15.80:6837","nonce":259529026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14646,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6839/4194284854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6838","nonce":4194284854},{"type":"v1","addr":"172.21.15.80:6839","nonce":4194284854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14658,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6841/3145730958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6840","nonce":3145730958},{"type":"v1","addr":"172.21.15.80:6841","nonce":3145730958}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24395,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/3736784385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3736784385},{"type":"v1","addr":"172.21.15.134:6833","nonce":3736784385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/124777462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":124777462},{"type":"v1","addr":"172.21.15.148:6833","nonce":124777462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T23:27:19.300566+0000","modified":"2024-04-24T23:27:45.981839+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24496,"mds_1":24488,"mds_2":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/1399801120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1399801120},{"type":"v1","addr":"172.21.15.134:6835","nonce":1399801120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/430512704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":430512704},{"type":"v1","addr":"172.21.15.134:6837","nonce":430512704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/3002319310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3002319310},{"type":"v1","addr":"172.21.15.148:6835","nonce":3002319310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.148:6837/856528403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":856528403},{"type":"v1","addr":"172.21.15.148:6837","nonce":856528403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6839/1743815021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":1743815021},{"type":"v1","addr":"172.21.15.134:6839","nonce":1743815021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.148:6839/2952757594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":2952757594},{"type":"v1","addr":"172.21.15.148:6839","nonce":2952757594}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24496,"qdb_cluster":[24472,24488,24496]},"id":1}]} 2024-04-24T23:27:48.044 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 24 2024-04-24T23:27:48.046 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.044+0000 7fa1c9929640 1 -- 172.21.15.80:0/476755511 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fa1ac0a2c20 msgr2=0x7fa1ac0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:48.046 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.044+0000 7fa1c9929640 1 --2- 172.21.15.80:0/476755511 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fa1ac0a2c20 0x7fa1ac0a50e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa1c4198f40 tx=0x7fa1b8009670 comp rx=0 tx=0).stop 2024-04-24T23:27:48.046 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.044+0000 7fa1c9929640 1 -- 172.21.15.80:0/476755511 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa1c40ffe20 msgr2=0x7fa1c4197a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:48.046 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.044+0000 7fa1c9929640 1 --2- 172.21.15.80:0/476755511 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa1c40ffe20 0x7fa1c4197a30 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fa1bc00d950 tx=0x7fa1bc00de10 comp rx=0 tx=0).stop 2024-04-24T23:27:48.046 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.044+0000 7fa1c9929640 1 -- 172.21.15.80:0/476755511 shutdown_connections 2024-04-24T23:27:48.046 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.044+0000 7fa1c9929640 1 --2- 172.21.15.80:0/476755511 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fa1ac0a2c20 0x7fa1ac0a50e0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:48.046 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.044+0000 7fa1c9929640 1 --2- 172.21.15.80:0/476755511 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1c4101930 0x7fa1c419efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:48.046 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.044+0000 7fa1c9929640 1 --2- 172.21.15.80:0/476755511 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa1c4100f70 0x7fa1c4197f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:48.046 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.044+0000 7fa1c9929640 1 --2- 172.21.15.80:0/476755511 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa1c40ffe20 0x7fa1c4197a30 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:48.046 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.044+0000 7fa1c9929640 1 -- 172.21.15.80:0/476755511 >> 172.21.15.80:0/476755511 conn(0x7fa1c40fb3b0 msgr2=0x7fa1c4108c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:48.046 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.044+0000 7fa1c9929640 1 -- 172.21.15.80:0/476755511 shutdown_connections 2024-04-24T23:27:48.047 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.044+0000 7fa1c9929640 1 -- 172.21.15.80:0/476755511 wait complete. 2024-04-24T23:27:48.057 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:27:48.157 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.154+0000 7f23ea8b3640 1 -- 172.21.15.80:0/873132388 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f23e4102200 msgr2=0x7f23e41045f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:48.158 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.154+0000 7f23ea8b3640 1 --2- 172.21.15.80:0/873132388 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f23e4102200 0x7f23e41045f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f23d4009900 tx=0x7f23d402a410 comp rx=0 tx=0).stop 2024-04-24T23:27:48.158 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.154+0000 7f23ea8b3640 1 -- 172.21.15.80:0/873132388 shutdown_connections 2024-04-24T23:27:48.158 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.154+0000 7f23ea8b3640 1 --2- 172.21.15.80:0/873132388 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f23e4107460 0x7f23e41098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:48.158 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.154+0000 7f23ea8b3640 1 --2- 172.21.15.80:0/873132388 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f23e4104b30 0x7f23e4106f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:48.158 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.154+0000 7f23ea8b3640 1 --2- 172.21.15.80:0/873132388 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f23e4102200 0x7f23e41045f0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:48.158 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.154+0000 7f23ea8b3640 1 -- 172.21.15.80:0/873132388 >> 172.21.15.80:0/873132388 conn(0x7f23e40fbbf0 msgr2=0x7f23e40fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:48.158 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.154+0000 7f23ea8b3640 1 -- 172.21.15.80:0/873132388 shutdown_connections 2024-04-24T23:27:48.158 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.154+0000 7f23ea8b3640 1 -- 172.21.15.80:0/873132388 wait complete. 2024-04-24T23:27:48.158 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.155+0000 7f23ea8b3640 1 Processor -- start 2024-04-24T23:27:48.158 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.155+0000 7f23ea8b3640 1 -- start start 2024-04-24T23:27:48.158 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.155+0000 7f23ea8b3640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f23e4102200 0x7f23e4197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:48.158 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.155+0000 7f23ea8b3640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f23e4104b30 0x7f23e4198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:48.159 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.155+0000 7f23ea8b3640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f23e4107460 0x7f23e419f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:48.159 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.155+0000 7f23ea8b3640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f23e410d430 con 0x7f23e4107460 2024-04-24T23:27:48.159 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.155+0000 7f23ea8b3640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f23e410d2b0 con 0x7f23e4102200 2024-04-24T23:27:48.159 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.155+0000 7f23ea8b3640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f23e410d5b0 con 0x7f23e4104b30 2024-04-24T23:27:48.159 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.155+0000 7f23e8e50640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f23e4107460 0x7f23e419f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:48.159 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.155+0000 7f23e8e50640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f23e4107460 0x7f23e419f0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:37518/0 (socket says 172.21.15.80:37518) 2024-04-24T23:27:48.159 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.155+0000 7f23e8e50640 1 -- 172.21.15.80:0/3908477863 learned_addr learned my addr 172.21.15.80:0/3908477863 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:27:48.159 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.155+0000 7f23e3fff640 1 --2- 172.21.15.80:0/3908477863 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f23e4102200 0x7f23e4197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:48.159 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.156+0000 7f23e37fe640 1 --2- 172.21.15.80:0/3908477863 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f23e4104b30 0x7f23e4198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:48.159 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.156+0000 7f23e8e50640 1 -- 172.21.15.80:0/3908477863 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f23e4104b30 msgr2=0x7f23e4198070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:48.159 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.156+0000 7f23e8e50640 1 --2- 172.21.15.80:0/3908477863 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f23e4104b30 0x7f23e4198070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:48.159 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.156+0000 7f23e8e50640 1 -- 172.21.15.80:0/3908477863 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f23e4102200 msgr2=0x7f23e4197b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:48.159 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.156+0000 7f23e8e50640 1 --2- 172.21.15.80:0/3908477863 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f23e4102200 0x7f23e4197b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:48.159 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.156+0000 7f23e8e50640 1 -- 172.21.15.80:0/3908477863 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23e419f750 con 0x7f23e4107460 2024-04-24T23:27:48.159 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.156+0000 7f23e8e50640 1 --2- 172.21.15.80:0/3908477863 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f23e4107460 0x7f23e419f0e0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f23dc00cad0 tx=0x7f23dc00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:48.160 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.156+0000 7f23e17fa640 1 -- 172.21.15.80:0/3908477863 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23dc00ae20 con 0x7f23e4107460 2024-04-24T23:27:48.160 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.156+0000 7f23ea8b3640 1 -- 172.21.15.80:0/3908477863 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23e419fa40 con 0x7f23e4107460 2024-04-24T23:27:48.160 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.156+0000 7f23e17fa640 1 -- 172.21.15.80:0/3908477863 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23dc005020 con 0x7f23e4107460 2024-04-24T23:27:48.160 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.156+0000 7f23e17fa640 1 -- 172.21.15.80:0/3908477863 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23dc00f6f0 con 0x7f23e4107460 2024-04-24T23:27:48.160 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.156+0000 7f23ea8b3640 1 -- 172.21.15.80:0/3908477863 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23e419ff90 con 0x7f23e4107460 2024-04-24T23:27:48.161 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.157+0000 7f23ea8b3640 1 -- 172.21.15.80:0/3908477863 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23e40fd6e0 con 0x7f23e4107460 2024-04-24T23:27:48.161 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.158+0000 7f23e17fa640 1 -- 172.21.15.80:0/3908477863 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f23dc0051d0 con 0x7f23e4107460 2024-04-24T23:27:48.161 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.159+0000 7f23e17fa640 1 --2- 172.21.15.80:0/3908477863 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f23cc0a2c20 0x7f23cc0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:48.161 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.159+0000 7f23e3fff640 1 --2- 172.21.15.80:0/3908477863 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f23cc0a2c20 0x7f23cc0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:48.161 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.159+0000 7f23e17fa640 1 -- 172.21.15.80:0/3908477863 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f23dc0cf450 con 0x7f23e4107460 2024-04-24T23:27:48.161 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.159+0000 7f23e3fff640 1 --2- 172.21.15.80:0/3908477863 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f23cc0a2c20 0x7f23cc0a50e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f23d40314e0 tx=0x7f23d4000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:48.164 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.163+0000 7f23e17fa640 1 -- 172.21.15.80:0/3908477863 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f23dc014030 con 0x7f23e4107460 2024-04-24T23:27:48.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:47 smithi148 ceph-mon[32528]: from='client.14913 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:27:48.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:47 smithi148 ceph-mon[32528]: from='client.14925 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:27:48.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:47 smithi148 ceph-mon[32528]: pgmap v310: 129 pgs: 129 active+clean; 581 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s rd, 938 B/s wr, 2 op/s 2024-04-24T23:27:48.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:47 smithi148 ceph-mon[32528]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-24T23:27:48.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:47 smithi148 ceph-mon[32528]: from='client.14931 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:27:48.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:47 smithi080 ceph-mon[29152]: from='client.14913 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:27:48.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:47 smithi080 ceph-mon[29152]: from='client.14925 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:27:48.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:47 smithi080 ceph-mon[29152]: pgmap v310: 129 pgs: 129 active+clean; 581 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s rd, 938 B/s wr, 2 op/s 2024-04-24T23:27:48.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:47 smithi080 ceph-mon[29152]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-24T23:27:48.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:47 smithi080 ceph-mon[29152]: from='client.14931 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:27:48.346 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.345+0000 7f23ea8b3640 1 -- 172.21.15.80:0/3908477863 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f23e402d3c0 con 0x7f23e4107460 2024-04-24T23:27:48.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.346+0000 7f23e17fa640 1 -- 172.21.15.80:0/3908477863 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24124 (secure 0 0 0) 0x7f23dc09c010 con 0x7f23e4107460 2024-04-24T23:27:48.348 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-24T23:27:48.348 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":90,"fsid":"ec7885aa-0290-11ef-bc93-c7b262605968","created":"2024-04-24T23:19:05.586407+0000","modified":"2024-04-24T23:27:26.212213+0000","last_up_change":"2024-04-24T23:26:25.938316+0000","last_in_change":"2024-04-24T23:26:04.786699+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:22:33.124428+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:27:17.575788+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:27:18.492762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T23:27:22.438503+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a525741d-b46c-4eef-aa64-3f057cac1497","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":2597718112},{"type":"v1","addr":"172.21.15.80:6803","nonce":2597718112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":2597718112},{"type":"v1","addr":"172.21.15.80:6805","nonce":2597718112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2597718112},{"type":"v1","addr":"172.21.15.80:6809","nonce":2597718112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":2597718112},{"type":"v1","addr":"172.21.15.80:6807","nonce":2597718112}]},"public_addr":"172.21.15.80:6803/2597718112","cluster_addr":"172.21.15.80:6805/2597718112","heartbeat_back_addr":"172.21.15.80:6809/2597718112","heartbeat_front_addr":"172.21.15.80:6807/2597718112","state":["exists","up"]},{"osd":1,"uuid":"2d0f0d67-6db4-40ac-91cc-58604ab6795a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":2395610201},{"type":"v1","addr":"172.21.15.134:6801","nonce":2395610201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":2395610201},{"type":"v1","addr":"172.21.15.134:6803","nonce":2395610201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":2395610201},{"type":"v1","addr":"172.21.15.134:6807","nonce":2395610201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":2395610201},{"type":"v1","addr":"172.21.15.134:6805","nonce":2395610201}]},"public_addr":"172.21.15.134:6801/2395610201","cluster_addr":"172.21.15.134:6803/2395610201","heartbeat_back_addr":"172.21.15.134:6807/2395610201","heartbeat_front_addr":"172.21.15.134:6805/2395610201","state":["exists","up"]},{"osd":2,"uuid":"781fb879-fedc-4c84-82fb-59e073bc4cf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1629152187},{"type":"v1","addr":"172.21.15.148:6801","nonce":1629152187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1629152187},{"type":"v1","addr":"172.21.15.148:6803","nonce":1629152187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1629152187},{"type":"v1","addr":"172.21.15.148:6807","nonce":1629152187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1629152187},{"type":"v1","addr":"172.21.15.148:6805","nonce":1629152187}]},"public_addr":"172.21.15.148:6801/1629152187","cluster_addr":"172.21.15.148:6803/1629152187","heartbeat_back_addr":"172.21.15.148:6807/1629152187","heartbeat_front_addr":"172.21.15.148:6805/1629152187","state":["exists","up"]},{"osd":3,"uuid":"cd982757-f2f6-4296-b3e7-4cd40fbd8d9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":3128298510},{"type":"v1","addr":"172.21.15.80:6811","nonce":3128298510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":3128298510},{"type":"v1","addr":"172.21.15.80:6813","nonce":3128298510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":3128298510},{"type":"v1","addr":"172.21.15.80:6817","nonce":3128298510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":3128298510},{"type":"v1","addr":"172.21.15.80:6815","nonce":3128298510}]},"public_addr":"172.21.15.80:6811/3128298510","cluster_addr":"172.21.15.80:6813/3128298510","heartbeat_back_addr":"172.21.15.80:6817/3128298510","heartbeat_front_addr":"172.21.15.80:6815/3128298510","state":["exists","up"]},{"osd":4,"uuid":"dfc55a79-a7e8-417a-b898-ec44171a54af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1917556060},{"type":"v1","addr":"172.21.15.134:6809","nonce":1917556060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1917556060},{"type":"v1","addr":"172.21.15.134:6811","nonce":1917556060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1917556060},{"type":"v1","addr":"172.21.15.134:6815","nonce":1917556060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1917556060},{"type":"v1","addr":"172.21.15.134:6813","nonce":1917556060}]},"public_addr":"172.21.15.134:6809/1917556060","cluster_addr":"172.21.15.134:6811/1917556060","heartbeat_back_addr":"172.21.15.134:6815/1917556060","heartbeat_front_addr":"172.21.15.134:6813/1917556060","state":["exists","up"]},{"osd":5,"uuid":"b5491337-0694-440c-b679-312f3d14cf65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":2715951281},{"type":"v1","addr":"172.21.15.148:6809","nonce":2715951281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":2715951281},{"type":"v1","addr":"172.21.15.148:6811","nonce":2715951281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":2715951281},{"type":"v1","addr":"172.21.15.148:6815","nonce":2715951281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":2715951281},{"type":"v1","addr":"172.21.15.148:6813","nonce":2715951281}]},"public_addr":"172.21.15.148:6809/2715951281","cluster_addr":"172.21.15.148:6811/2715951281","heartbeat_back_addr":"172.21.15.148:6815/2715951281","heartbeat_front_addr":"172.21.15.148:6813/2715951281","state":["exists","up"]},{"osd":6,"uuid":"ba4b5df7-d86e-47cd-ab67-cf763c4e4d6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":953261724},{"type":"v1","addr":"172.21.15.80:6819","nonce":953261724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":953261724},{"type":"v1","addr":"172.21.15.80:6821","nonce":953261724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":953261724},{"type":"v1","addr":"172.21.15.80:6825","nonce":953261724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":953261724},{"type":"v1","addr":"172.21.15.80:6823","nonce":953261724}]},"public_addr":"172.21.15.80:6819/953261724","cluster_addr":"172.21.15.80:6821/953261724","heartbeat_back_addr":"172.21.15.80:6825/953261724","heartbeat_front_addr":"172.21.15.80:6823/953261724","state":["exists","up"]},{"osd":7,"uuid":"464a503a-75ae-49e7-ab6e-a7f9f2a6a307","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":108354844},{"type":"v1","addr":"172.21.15.134:6817","nonce":108354844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":108354844},{"type":"v1","addr":"172.21.15.134:6819","nonce":108354844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":108354844},{"type":"v1","addr":"172.21.15.134:6823","nonce":108354844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":108354844},{"type":"v1","addr":"172.21.15.134:6821","nonce":108354844}]},"public_addr":"172.21.15.134:6817/108354844","cluster_addr":"172.21.15.134:6819/108354844","heartbeat_back_addr":"172.21.15.134:6823/108354844","heartbeat_front_addr":"172.21.15.134:6821/108354844","state":["exists","up"]},{"osd":8,"uuid":"ce544b2e-4540-47c0-9997-500c45351330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3559655329},{"type":"v1","addr":"172.21.15.148:6817","nonce":3559655329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":3559655329},{"type":"v1","addr":"172.21.15.148:6819","nonce":3559655329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":3559655329},{"type":"v1","addr":"172.21.15.148:6823","nonce":3559655329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3559655329},{"type":"v1","addr":"172.21.15.148:6821","nonce":3559655329}]},"public_addr":"172.21.15.148:6817/3559655329","cluster_addr":"172.21.15.148:6819/3559655329","heartbeat_back_addr":"172.21.15.148:6823/3559655329","heartbeat_front_addr":"172.21.15.148:6821/3559655329","state":["exists","up"]},{"osd":9,"uuid":"06de3732-19b9-496c-a464-55322efa6759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":4019913545},{"type":"v1","addr":"172.21.15.80:6827","nonce":4019913545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":4019913545},{"type":"v1","addr":"172.21.15.80:6829","nonce":4019913545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":4019913545},{"type":"v1","addr":"172.21.15.80:6833","nonce":4019913545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":4019913545},{"type":"v1","addr":"172.21.15.80:6831","nonce":4019913545}]},"public_addr":"172.21.15.80:6827/4019913545","cluster_addr":"172.21.15.80:6829/4019913545","heartbeat_back_addr":"172.21.15.80:6833/4019913545","heartbeat_front_addr":"172.21.15.80:6831/4019913545","state":["exists","up"]},{"osd":10,"uuid":"8d92f0d9-5c64-429a-b750-c33ae0e070c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":647343766},{"type":"v1","addr":"172.21.15.134:6825","nonce":647343766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":647343766},{"type":"v1","addr":"172.21.15.134:6827","nonce":647343766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":647343766},{"type":"v1","addr":"172.21.15.134:6831","nonce":647343766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":647343766},{"type":"v1","addr":"172.21.15.134:6829","nonce":647343766}]},"public_addr":"172.21.15.134:6825/647343766","cluster_addr":"172.21.15.134:6827/647343766","heartbeat_back_addr":"172.21.15.134:6831/647343766","heartbeat_front_addr":"172.21.15.134:6829/647343766","state":["exists","up"]},{"osd":11,"uuid":"4c003a41-f65b-479f-bc80-b27d680cf936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":4068024116},{"type":"v1","addr":"172.21.15.148:6825","nonce":4068024116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":4068024116},{"type":"v1","addr":"172.21.15.148:6827","nonce":4068024116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":4068024116},{"type":"v1","addr":"172.21.15.148:6831","nonce":4068024116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":4068024116},{"type":"v1","addr":"172.21.15.148:6829","nonce":4068024116}]},"public_addr":"172.21.15.148:6825/4068024116","cluster_addr":"172.21.15.148:6827/4068024116","heartbeat_back_addr":"172.21.15.148:6831/4068024116","heartbeat_front_addr":"172.21.15.148:6829/4068024116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:21:42.433631+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:22:06.145933+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:22:30.726143+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:22:55.729961+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:23:21.280913+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:23:46.609983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:24:12.585247+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:24:38.359287+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:25:04.491232+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:25:30.788089+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:25:56.891078+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:26:23.899113+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.80:0/2828378749":"2024-04-25T23:20:06.793609+0000","172.21.15.80:6801/233364790":"2024-04-25T23:20:06.793609+0000","172.21.15.80:6800/1614691410":"2024-04-25T23:19:40.490692+0000","172.21.15.80:0/1406187145":"2024-04-25T23:20:06.793609+0000","172.21.15.80:0/92953396":"2024-04-25T23:19:40.490692+0000","172.21.15.80:0/598129432":"2024-04-25T23:20:06.793609+0000","172.21.15.80:0/232193118":"2024-04-25T23:19:40.490692+0000","172.21.15.80:6800/233364790":"2024-04-25T23:20:06.793609+0000","172.21.15.80:6801/1614691410":"2024-04-25T23:19:40.490692+0000","172.21.15.80:0/2081562529":"2024-04-25T23:19:40.490692+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:27:48.350 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.350+0000 7f23ea8b3640 1 -- 172.21.15.80:0/3908477863 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f23cc0a2c20 msgr2=0x7f23cc0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:48.350 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.350+0000 7f23ea8b3640 1 --2- 172.21.15.80:0/3908477863 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f23cc0a2c20 0x7f23cc0a50e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f23d40314e0 tx=0x7f23d4000bc0 comp rx=0 tx=0).stop 2024-04-24T23:27:48.350 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.350+0000 7f23ea8b3640 1 -- 172.21.15.80:0/3908477863 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f23e4107460 msgr2=0x7f23e419f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:48.350 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.350+0000 7f23ea8b3640 1 --2- 172.21.15.80:0/3908477863 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f23e4107460 0x7f23e419f0e0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f23dc00cad0 tx=0x7f23dc00cf90 comp rx=0 tx=0).stop 2024-04-24T23:27:48.350 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.350+0000 7f23ea8b3640 1 -- 172.21.15.80:0/3908477863 shutdown_connections 2024-04-24T23:27:48.350 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.350+0000 7f23ea8b3640 1 --2- 172.21.15.80:0/3908477863 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f23cc0a2c20 0x7f23cc0a50e0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:48.350 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.350+0000 7f23ea8b3640 1 --2- 172.21.15.80:0/3908477863 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f23e4107460 0x7f23e419f0e0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:48.350 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.350+0000 7f23ea8b3640 1 --2- 172.21.15.80:0/3908477863 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f23e4104b30 0x7f23e4198070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:48.350 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.350+0000 7f23ea8b3640 1 --2- 172.21.15.80:0/3908477863 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f23e4102200 0x7f23e4197b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:48.351 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.350+0000 7f23ea8b3640 1 -- 172.21.15.80:0/3908477863 >> 172.21.15.80:0/3908477863 conn(0x7f23e40fbbf0 msgr2=0x7f23e4102b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:48.351 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.350+0000 7f23ea8b3640 1 -- 172.21.15.80:0/3908477863 shutdown_connections 2024-04-24T23:27:48.351 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.350+0000 7f23ea8b3640 1 -- 172.21.15.80:0/3908477863 wait complete. 2024-04-24T23:27:48.362 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-24T23:27:48.362 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T23:27:48.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:47 smithi134 ceph-mon[32619]: from='client.14913 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:27:48.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:47 smithi134 ceph-mon[32619]: from='client.14925 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:27:48.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:47 smithi134 ceph-mon[32619]: pgmap v310: 129 pgs: 129 active+clean; 581 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s rd, 938 B/s wr, 2 op/s 2024-04-24T23:27:48.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:47 smithi134 ceph-mon[32619]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-24T23:27:48.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:47 smithi134 ceph-mon[32619]: from='client.14931 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:27:48.462 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.459+0000 7f90c05f9640 1 -- 172.21.15.80:0/1122850685 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f90b81017b0 msgr2=0x7f90b8101bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:48.462 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.459+0000 7f90c05f9640 1 --2- 172.21.15.80:0/1122850685 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f90b81017b0 0x7f90b8101bd0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f90b4009900 tx=0x7f90b402a410 comp rx=0 tx=0).stop 2024-04-24T23:27:48.462 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.459+0000 7f90c05f9640 1 -- 172.21.15.80:0/1122850685 shutdown_connections 2024-04-24T23:27:48.462 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.459+0000 7f90c05f9640 1 --2- 172.21.15.80:0/1122850685 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90b8103360 0x7f90b8109bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:48.462 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.459+0000 7f90c05f9640 1 --2- 172.21.15.80:0/1122850685 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90b81029a0 0x7f90b8102e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:48.462 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.459+0000 7f90c05f9640 1 --2- 172.21.15.80:0/1122850685 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f90b81017b0 0x7f90b8101bd0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:48.462 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.459+0000 7f90c05f9640 1 -- 172.21.15.80:0/1122850685 >> 172.21.15.80:0/1122850685 conn(0x7f90b80fcd40 msgr2=0x7f90b80ff160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:48.463 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.459+0000 7f90c05f9640 1 -- 172.21.15.80:0/1122850685 shutdown_connections 2024-04-24T23:27:48.463 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.459+0000 7f90c05f9640 1 -- 172.21.15.80:0/1122850685 wait complete. 2024-04-24T23:27:48.463 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.460+0000 7f90c05f9640 1 Processor -- start 2024-04-24T23:27:48.463 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.460+0000 7f90c05f9640 1 -- start start 2024-04-24T23:27:48.463 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.460+0000 7f90c05f9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90b81017b0 0x7f90b81979f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:48.463 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.460+0000 7f90c05f9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90b81029a0 0x7f90b8197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:48.463 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.460+0000 7f90c05f9640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f90b8103360 0x7f90b819efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:48.463 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.460+0000 7f90c05f9640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f90b810d430 con 0x7f90b8103360 2024-04-24T23:27:48.463 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.460+0000 7f90c05f9640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f90b810d2b0 con 0x7f90b81017b0 2024-04-24T23:27:48.463 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.460+0000 7f90c05f9640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f90b810d5b0 con 0x7f90b81029a0 2024-04-24T23:27:48.463 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.460+0000 7f90beb96640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f90b8103360 0x7f90b819efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:48.463 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.460+0000 7f90beb96640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f90b8103360 0x7f90b819efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:37540/0 (socket says 172.21.15.80:37540) 2024-04-24T23:27:48.463 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.460+0000 7f90beb96640 1 -- 172.21.15.80:0/129462227 learned_addr learned my addr 172.21.15.80:0/129462227 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:27:48.463 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.460+0000 7f90bdb94640 1 --2- 172.21.15.80:0/129462227 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90b81029a0 0x7f90b8197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:48.464 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.460+0000 7f90be395640 1 --2- 172.21.15.80:0/129462227 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90b81017b0 0x7f90b81979f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:48.464 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.460+0000 7f90beb96640 1 -- 172.21.15.80:0/129462227 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90b81029a0 msgr2=0x7f90b8197f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:48.464 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.460+0000 7f90beb96640 1 --2- 172.21.15.80:0/129462227 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90b81029a0 0x7f90b8197f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:48.464 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.460+0000 7f90beb96640 1 -- 172.21.15.80:0/129462227 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90b81017b0 msgr2=0x7f90b81979f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:48.464 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.460+0000 7f90beb96640 1 --2- 172.21.15.80:0/129462227 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90b81017b0 0x7f90b81979f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:48.464 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.460+0000 7f90beb96640 1 -- 172.21.15.80:0/129462227 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90b819f610 con 0x7f90b8103360 2024-04-24T23:27:48.464 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.460+0000 7f90beb96640 1 --2- 172.21.15.80:0/129462227 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f90b8103360 0x7f90b819efa0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f90b000eaa0 tx=0x7f90b000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:48.464 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.461+0000 7f90af7fe640 1 -- 172.21.15.80:0/129462227 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90b0004d20 con 0x7f90b8103360 2024-04-24T23:27:48.464 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.461+0000 7f90af7fe640 1 -- 172.21.15.80:0/129462227 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90b0005020 con 0x7f90b8103360 2024-04-24T23:27:48.464 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.461+0000 7f90c05f9640 1 -- 172.21.15.80:0/129462227 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90b819f900 con 0x7f90b8103360 2024-04-24T23:27:48.464 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.461+0000 7f90af7fe640 1 -- 172.21.15.80:0/129462227 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90b00116d0 con 0x7f90b8103360 2024-04-24T23:27:48.464 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.461+0000 7f90c05f9640 1 -- 172.21.15.80:0/129462227 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90b819fe50 con 0x7f90b8103360 2024-04-24T23:27:48.465 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.462+0000 7f90c05f9640 1 -- 172.21.15.80:0/129462227 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90b8108a70 con 0x7f90b8103360 2024-04-24T23:27:48.465 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.463+0000 7f90af7fe640 1 -- 172.21.15.80:0/129462227 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f90b00051d0 con 0x7f90b8103360 2024-04-24T23:27:48.465 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.463+0000 7f90af7fe640 1 --2- 172.21.15.80:0/129462227 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f90a40a2c20 0x7f90a40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:48.465 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.463+0000 7f90be395640 1 --2- 172.21.15.80:0/129462227 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f90a40a2c20 0x7f90a40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:48.466 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.463+0000 7f90af7fe640 1 -- 172.21.15.80:0/129462227 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f90b00cf650 con 0x7f90b8103360 2024-04-24T23:27:48.466 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.464+0000 7f90be395640 1 --2- 172.21.15.80:0/129462227 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f90a40a2c20 0x7f90a40a50e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f90b4006010 tx=0x7f90b4000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:48.468 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.468+0000 7f90af7fe640 1 -- 172.21.15.80:0/129462227 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f90b0016030 con 0x7f90b8103360 2024-04-24T23:27:48.640 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.639+0000 7f90c05f9640 1 -- 172.21.15.80:0/129462227 --> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f90b8061b80 con 0x7f90a40a2c20 2024-04-24T23:27:48.650 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.649+0000 7f90af7fe640 1 -- 172.21.15.80:0/129462227 <== mgr.14152 v2:172.21.15.80:6800/3830596922 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340529 (secure 0 0 0) 0x7f90b8061b80 con 0x7f90a40a2c20 2024-04-24T23:27:48.655 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-24T23:27:48.657 INFO:teuthology.orchestra.run.smithi080.stderr:dumped all 2024-04-24T23:27:48.659 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.654+0000 7f90c05f9640 1 -- 172.21.15.80:0/129462227 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f90a40a2c20 msgr2=0x7f90a40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:48.659 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.654+0000 7f90c05f9640 1 --2- 172.21.15.80:0/129462227 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f90a40a2c20 0x7f90a40a50e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f90b4006010 tx=0x7f90b4000bc0 comp rx=0 tx=0).stop 2024-04-24T23:27:48.659 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.654+0000 7f90c05f9640 1 -- 172.21.15.80:0/129462227 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f90b8103360 msgr2=0x7f90b819efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:48.659 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.654+0000 7f90c05f9640 1 --2- 172.21.15.80:0/129462227 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f90b8103360 0x7f90b819efa0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f90b000eaa0 tx=0x7f90b000eea0 comp rx=0 tx=0).stop 2024-04-24T23:27:48.659 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.654+0000 7f90c05f9640 1 -- 172.21.15.80:0/129462227 shutdown_connections 2024-04-24T23:27:48.659 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.654+0000 7f90c05f9640 1 --2- 172.21.15.80:0/129462227 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f90a40a2c20 0x7f90a40a50e0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:48.660 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.654+0000 7f90c05f9640 1 --2- 172.21.15.80:0/129462227 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f90b8103360 0x7f90b819efa0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:48.660 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.654+0000 7f90c05f9640 1 --2- 172.21.15.80:0/129462227 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90b81029a0 0x7f90b8197f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:48.660 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.654+0000 7f90c05f9640 1 --2- 172.21.15.80:0/129462227 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90b81017b0 0x7f90b81979f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:48.660 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.654+0000 7f90c05f9640 1 -- 172.21.15.80:0/129462227 >> 172.21.15.80:0/129462227 conn(0x7f90b80fcd40 msgr2=0x7f90b8105c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:48.660 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.654+0000 7f90c05f9640 1 -- 172.21.15.80:0/129462227 shutdown_connections 2024-04-24T23:27:48.660 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.654+0000 7f90c05f9640 1 -- 172.21.15.80:0/129462227 wait complete. 2024-04-24T23:27:48.665 INFO:teuthology.orchestra.run.smithi080.stdout:{"pg_ready":true,"pg_map":{"version":310,"stamp":"2024-04-24T23:27:46.886055+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595142,"num_objects":34,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":0,"num_read":131,"num_read_kb":114,"num_write":251,"num_write_kb":2966,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":199,"ondisk_log_size":199,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":365044,"kb_used_data":11688,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124576780,"statfs":{"total":1151940427776,"available":1151566622720,"internally_reserved":0,"allocated":11968512,"data_stored":7625826,"data_compressed":318897,"data_compressed_allocated":430080,"data_compressed_original":2359296,"omap_allocated":19141,"internal_metadata":361674043},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":275,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":23,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003779"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.888099+0000","last_change":"2024-04-24T23:27:19.978807+0000","last_active":"2024-04-24T23:27:26.888099+0000","last_peered":"2024-04-24T23:27:26.888099+0000","last_clean":"2024-04-24T23:27:26.888099+0000","last_became_active":"2024-04-24T23:27:19.978436+0000","last_became_peered":"2024-04-24T23:27:19.978436+0000","last_unstale":"2024-04-24T23:27:26.888099+0000","last_undegraded":"2024-04-24T23:27:26.888099+0000","last_fullsized":"2024-04-24T23:27:26.888099+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:00:13.126518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.853244+0000","last_change":"2024-04-24T23:27:19.980327+0000","last_active":"2024-04-24T23:27:26.853244+0000","last_peered":"2024-04-24T23:27:26.853244+0000","last_clean":"2024-04-24T23:27:26.853244+0000","last_became_active":"2024-04-24T23:27:19.979912+0000","last_became_peered":"2024-04-24T23:27:19.979912+0000","last_unstale":"2024-04-24T23:27:26.853244+0000","last_undegraded":"2024-04-24T23:27:26.853244+0000","last_fullsized":"2024-04-24T23:27:26.853244+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:25:41.657205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.853033+0000","last_change":"2024-04-24T23:27:19.976451+0000","last_active":"2024-04-24T23:27:26.853033+0000","last_peered":"2024-04-24T23:27:26.853033+0000","last_clean":"2024-04-24T23:27:26.853033+0000","last_became_active":"2024-04-24T23:27:19.975976+0000","last_became_peered":"2024-04-24T23:27:19.975976+0000","last_unstale":"2024-04-24T23:27:26.853033+0000","last_undegraded":"2024-04-24T23:27:26.853033+0000","last_fullsized":"2024-04-24T23:27:26.853033+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:14:51.605217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.852139+0000","last_change":"2024-04-24T23:27:19.991245+0000","last_active":"2024-04-24T23:27:26.852139+0000","last_peered":"2024-04-24T23:27:26.852139+0000","last_clean":"2024-04-24T23:27:26.852139+0000","last_became_active":"2024-04-24T23:27:19.990448+0000","last_became_peered":"2024-04-24T23:27:19.990448+0000","last_unstale":"2024-04-24T23:27:26.852139+0000","last_undegraded":"2024-04-24T23:27:26.852139+0000","last_fullsized":"2024-04-24T23:27:26.852139+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:03:21.013005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.851092+0000","last_change":"2024-04-24T23:27:19.977664+0000","last_active":"2024-04-24T23:27:26.851092+0000","last_peered":"2024-04-24T23:27:26.851092+0000","last_clean":"2024-04-24T23:27:26.851092+0000","last_became_active":"2024-04-24T23:27:19.977300+0000","last_became_peered":"2024-04-24T23:27:19.977300+0000","last_unstale":"2024-04-24T23:27:26.851092+0000","last_undegraded":"2024-04-24T23:27:26.851092+0000","last_fullsized":"2024-04-24T23:27:26.851092+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:19:27.229712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.852801+0000","last_change":"2024-04-24T23:27:19.989994+0000","last_active":"2024-04-24T23:27:26.852801+0000","last_peered":"2024-04-24T23:27:26.852801+0000","last_clean":"2024-04-24T23:27:26.852801+0000","last_became_active":"2024-04-24T23:27:19.989639+0000","last_became_peered":"2024-04-24T23:27:19.989639+0000","last_unstale":"2024-04-24T23:27:26.852801+0000","last_undegraded":"2024-04-24T23:27:26.852801+0000","last_fullsized":"2024-04-24T23:27:26.852801+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:20:43.931086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.216561+0000","last_change":"2024-04-24T23:27:19.992886+0000","last_active":"2024-04-24T23:27:26.216561+0000","last_peered":"2024-04-24T23:27:26.216561+0000","last_clean":"2024-04-24T23:27:26.216561+0000","last_became_active":"2024-04-24T23:27:19.992438+0000","last_became_peered":"2024-04-24T23:27:19.992438+0000","last_unstale":"2024-04-24T23:27:26.216561+0000","last_undegraded":"2024-04-24T23:27:26.216561+0000","last_fullsized":"2024-04-24T23:27:26.216561+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:02:34.110984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.850659+0000","last_change":"2024-04-24T23:27:19.981592+0000","last_active":"2024-04-24T23:27:26.850659+0000","last_peered":"2024-04-24T23:27:26.850659+0000","last_clean":"2024-04-24T23:27:26.850659+0000","last_became_active":"2024-04-24T23:27:19.981248+0000","last_became_peered":"2024-04-24T23:27:19.981248+0000","last_unstale":"2024-04-24T23:27:26.850659+0000","last_undegraded":"2024-04-24T23:27:26.850659+0000","last_fullsized":"2024-04-24T23:27:26.850659+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:53:31.191318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.852494+0000","last_change":"2024-04-24T23:27:19.975995+0000","last_active":"2024-04-24T23:27:26.852494+0000","last_peered":"2024-04-24T23:27:26.852494+0000","last_clean":"2024-04-24T23:27:26.852494+0000","last_became_active":"2024-04-24T23:27:19.975614+0000","last_became_peered":"2024-04-24T23:27:19.975614+0000","last_unstale":"2024-04-24T23:27:26.852494+0000","last_undegraded":"2024-04-24T23:27:26.852494+0000","last_fullsized":"2024-04-24T23:27:26.852494+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:33:22.439998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.853019+0000","last_change":"2024-04-24T23:27:19.991265+0000","last_active":"2024-04-24T23:27:26.853019+0000","last_peered":"2024-04-24T23:27:26.853019+0000","last_clean":"2024-04-24T23:27:26.853019+0000","last_became_active":"2024-04-24T23:27:19.990928+0000","last_became_peered":"2024-04-24T23:27:19.990928+0000","last_unstale":"2024-04-24T23:27:26.853019+0000","last_undegraded":"2024-04-24T23:27:26.853019+0000","last_fullsized":"2024-04-24T23:27:26.853019+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:20:41.244669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.217102+0000","last_change":"2024-04-24T23:27:19.985182+0000","last_active":"2024-04-24T23:27:26.217102+0000","last_peered":"2024-04-24T23:27:26.217102+0000","last_clean":"2024-04-24T23:27:26.217102+0000","last_became_active":"2024-04-24T23:27:19.984600+0000","last_became_peered":"2024-04-24T23:27:19.984600+0000","last_unstale":"2024-04-24T23:27:26.217102+0000","last_undegraded":"2024-04-24T23:27:26.217102+0000","last_fullsized":"2024-04-24T23:27:26.217102+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:16:09.919736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.851979+0000","last_change":"2024-04-24T23:27:19.994271+0000","last_active":"2024-04-24T23:27:26.851979+0000","last_peered":"2024-04-24T23:27:26.851979+0000","last_clean":"2024-04-24T23:27:26.851979+0000","last_became_active":"2024-04-24T23:27:19.993926+0000","last_became_peered":"2024-04-24T23:27:19.993926+0000","last_unstale":"2024-04-24T23:27:26.851979+0000","last_undegraded":"2024-04-24T23:27:26.851979+0000","last_fullsized":"2024-04-24T23:27:26.851979+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:49:07.500370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.215566+0000","last_change":"2024-04-24T23:27:19.984230+0000","last_active":"2024-04-24T23:27:26.215566+0000","last_peered":"2024-04-24T23:27:26.215566+0000","last_clean":"2024-04-24T23:27:26.215566+0000","last_became_active":"2024-04-24T23:27:19.983902+0000","last_became_peered":"2024-04-24T23:27:19.983902+0000","last_unstale":"2024-04-24T23:27:26.215566+0000","last_undegraded":"2024-04-24T23:27:26.215566+0000","last_fullsized":"2024-04-24T23:27:26.215566+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:12:40.985092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.852594+0000","last_change":"2024-04-24T23:27:19.979822+0000","last_active":"2024-04-24T23:27:26.852594+0000","last_peered":"2024-04-24T23:27:26.852594+0000","last_clean":"2024-04-24T23:27:26.852594+0000","last_became_active":"2024-04-24T23:27:19.979452+0000","last_became_peered":"2024-04-24T23:27:19.979452+0000","last_unstale":"2024-04-24T23:27:26.852594+0000","last_undegraded":"2024-04-24T23:27:26.852594+0000","last_fullsized":"2024-04-24T23:27:26.852594+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:08:53.945221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.852737+0000","last_change":"2024-04-24T23:27:19.986313+0000","last_active":"2024-04-24T23:27:26.852737+0000","last_peered":"2024-04-24T23:27:26.852737+0000","last_clean":"2024-04-24T23:27:26.852737+0000","last_became_active":"2024-04-24T23:27:19.985947+0000","last_became_peered":"2024-04-24T23:27:19.985947+0000","last_unstale":"2024-04-24T23:27:26.852737+0000","last_undegraded":"2024-04-24T23:27:26.852737+0000","last_fullsized":"2024-04-24T23:27:26.852737+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:32:10.557949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.851170+0000","last_change":"2024-04-24T23:27:19.984346+0000","last_active":"2024-04-24T23:27:26.851170+0000","last_peered":"2024-04-24T23:27:26.851170+0000","last_clean":"2024-04-24T23:27:26.851170+0000","last_became_active":"2024-04-24T23:27:19.983999+0000","last_became_peered":"2024-04-24T23:27:19.983999+0000","last_unstale":"2024-04-24T23:27:26.851170+0000","last_undegraded":"2024-04-24T23:27:26.851170+0000","last_fullsized":"2024-04-24T23:27:26.851170+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:15:35.329454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.887975+0000","last_change":"2024-04-24T23:27:19.979939+0000","last_active":"2024-04-24T23:27:26.887975+0000","last_peered":"2024-04-24T23:27:26.887975+0000","last_clean":"2024-04-24T23:27:26.887975+0000","last_became_active":"2024-04-24T23:27:19.979600+0000","last_became_peered":"2024-04-24T23:27:19.979600+0000","last_unstale":"2024-04-24T23:27:26.887975+0000","last_undegraded":"2024-04-24T23:27:26.887975+0000","last_fullsized":"2024-04-24T23:27:26.887975+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:37:52.250143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.852978+0000","last_change":"2024-04-24T23:27:19.984797+0000","last_active":"2024-04-24T23:27:26.852978+0000","last_peered":"2024-04-24T23:27:26.852978+0000","last_clean":"2024-04-24T23:27:26.852978+0000","last_became_active":"2024-04-24T23:27:19.983881+0000","last_became_peered":"2024-04-24T23:27:19.983881+0000","last_unstale":"2024-04-24T23:27:26.852978+0000","last_undegraded":"2024-04-24T23:27:26.852978+0000","last_fullsized":"2024-04-24T23:27:26.852978+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:17:24.630821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.283765+0000","last_change":"2024-04-24T23:27:19.977558+0000","last_active":"2024-04-24T23:27:26.283765+0000","last_peered":"2024-04-24T23:27:26.283765+0000","last_clean":"2024-04-24T23:27:26.283765+0000","last_became_active":"2024-04-24T23:27:19.977220+0000","last_became_peered":"2024-04-24T23:27:19.977220+0000","last_unstale":"2024-04-24T23:27:26.283765+0000","last_undegraded":"2024-04-24T23:27:26.283765+0000","last_fullsized":"2024-04-24T23:27:26.283765+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:38:49.615455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.283511+0000","last_change":"2024-04-24T23:27:24.234304+0000","last_active":"2024-04-24T23:27:26.283511+0000","last_peered":"2024-04-24T23:27:26.283511+0000","last_clean":"2024-04-24T23:27:26.283511+0000","last_became_active":"2024-04-24T23:27:24.233865+0000","last_became_peered":"2024-04-24T23:27:24.233865+0000","last_unstale":"2024-04-24T23:27:26.283511+0000","last_undegraded":"2024-04-24T23:27:26.283511+0000","last_fullsized":"2024-04-24T23:27:26.283511+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:46:20.853060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"84'4","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.853374+0000","last_change":"2024-04-24T23:27:18.970613+0000","last_active":"2024-04-24T23:27:26.853374+0000","last_peered":"2024-04-24T23:27:26.853374+0000","last_clean":"2024-04-24T23:27:26.853374+0000","last_became_active":"2024-04-24T23:27:18.970173+0000","last_became_peered":"2024-04-24T23:27:18.970173+0000","last_unstale":"2024-04-24T23:27:26.853374+0000","last_undegraded":"2024-04-24T23:27:26.853374+0000","last_fullsized":"2024-04-24T23:27:26.853374+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:12:35.187309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.851565+0000","last_change":"2024-04-24T23:27:19.981571+0000","last_active":"2024-04-24T23:27:26.851565+0000","last_peered":"2024-04-24T23:27:26.851565+0000","last_clean":"2024-04-24T23:27:26.851565+0000","last_became_active":"2024-04-24T23:27:19.981007+0000","last_became_peered":"2024-04-24T23:27:19.981007+0000","last_unstale":"2024-04-24T23:27:26.851565+0000","last_undegraded":"2024-04-24T23:27:26.851565+0000","last_fullsized":"2024-04-24T23:27:26.851565+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:38:30.023755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.851355+0000","last_change":"2024-04-24T23:27:24.234134+0000","last_active":"2024-04-24T23:27:26.851355+0000","last_peered":"2024-04-24T23:27:26.851355+0000","last_clean":"2024-04-24T23:27:26.851355+0000","last_became_active":"2024-04-24T23:27:24.233650+0000","last_became_peered":"2024-04-24T23:27:24.233650+0000","last_unstale":"2024-04-24T23:27:26.851355+0000","last_undegraded":"2024-04-24T23:27:26.851355+0000","last_fullsized":"2024-04-24T23:27:26.851355+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:04:54.191186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"84'1","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.851925+0000","last_change":"2024-04-24T23:27:18.969572+0000","last_active":"2024-04-24T23:27:26.851925+0000","last_peered":"2024-04-24T23:27:26.851925+0000","last_clean":"2024-04-24T23:27:26.851925+0000","last_became_active":"2024-04-24T23:27:18.968897+0000","last_became_peered":"2024-04-24T23:27:18.968897+0000","last_unstale":"2024-04-24T23:27:26.851925+0000","last_undegraded":"2024-04-24T23:27:26.851925+0000","last_fullsized":"2024-04-24T23:27:26.851925+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:42:51.380386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.852983+0000","last_change":"2024-04-24T23:27:19.977290+0000","last_active":"2024-04-24T23:27:26.852983+0000","last_peered":"2024-04-24T23:27:26.852983+0000","last_clean":"2024-04-24T23:27:26.852983+0000","last_became_active":"2024-04-24T23:27:19.976802+0000","last_became_peered":"2024-04-24T23:27:19.976802+0000","last_unstale":"2024-04-24T23:27:26.852983+0000","last_undegraded":"2024-04-24T23:27:26.852983+0000","last_fullsized":"2024-04-24T23:27:26.852983+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:55:15.827279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.851496+0000","last_change":"2024-04-24T23:27:24.235706+0000","last_active":"2024-04-24T23:27:26.851496+0000","last_peered":"2024-04-24T23:27:26.851496+0000","last_clean":"2024-04-24T23:27:26.851496+0000","last_became_active":"2024-04-24T23:27:24.235367+0000","last_became_peered":"2024-04-24T23:27:24.235367+0000","last_unstale":"2024-04-24T23:27:26.851496+0000","last_undegraded":"2024-04-24T23:27:26.851496+0000","last_fullsized":"2024-04-24T23:27:26.851496+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:17.444845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:44.988135+0000","last_change":"2024-04-24T23:27:18.974428+0000","last_active":"2024-04-24T23:27:44.988135+0000","last_peered":"2024-04-24T23:27:44.988135+0000","last_clean":"2024-04-24T23:27:44.988135+0000","last_became_active":"2024-04-24T23:27:18.974100+0000","last_became_peered":"2024-04-24T23:27:18.974100+0000","last_unstale":"2024-04-24T23:27:44.988135+0000","last_undegraded":"2024-04-24T23:27:44.988135+0000","last_fullsized":"2024-04-24T23:27:44.988135+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:55:36.070904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.852155+0000","last_change":"2024-04-24T23:27:19.975222+0000","last_active":"2024-04-24T23:27:26.852155+0000","last_peered":"2024-04-24T23:27:26.852155+0000","last_clean":"2024-04-24T23:27:26.852155+0000","last_became_active":"2024-04-24T23:27:19.974881+0000","last_became_peered":"2024-04-24T23:27:19.974881+0000","last_unstale":"2024-04-24T23:27:26.852155+0000","last_undegraded":"2024-04-24T23:27:26.852155+0000","last_fullsized":"2024-04-24T23:27:26.852155+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:16:17.884748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.851380+0000","last_change":"2024-04-24T23:27:24.232445+0000","last_active":"2024-04-24T23:27:26.851380+0000","last_peered":"2024-04-24T23:27:26.851380+0000","last_clean":"2024-04-24T23:27:26.851380+0000","last_became_active":"2024-04-24T23:27:24.231754+0000","last_became_peered":"2024-04-24T23:27:24.231754+0000","last_unstale":"2024-04-24T23:27:26.851380+0000","last_undegraded":"2024-04-24T23:27:26.851380+0000","last_fullsized":"2024-04-24T23:27:26.851380+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:45:43.630508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.851160+0000","last_change":"2024-04-24T23:27:18.966558+0000","last_active":"2024-04-24T23:27:26.851160+0000","last_peered":"2024-04-24T23:27:26.851160+0000","last_clean":"2024-04-24T23:27:26.851160+0000","last_became_active":"2024-04-24T23:27:18.966212+0000","last_became_peered":"2024-04-24T23:27:18.966212+0000","last_unstale":"2024-04-24T23:27:26.851160+0000","last_undegraded":"2024-04-24T23:27:26.851160+0000","last_fullsized":"2024-04-24T23:27:26.851160+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:24:17.262587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.216910+0000","last_change":"2024-04-24T23:27:19.976811+0000","last_active":"2024-04-24T23:27:26.216910+0000","last_peered":"2024-04-24T23:27:26.216910+0000","last_clean":"2024-04-24T23:27:26.216910+0000","last_became_active":"2024-04-24T23:27:19.976456+0000","last_became_peered":"2024-04-24T23:27:19.976456+0000","last_unstale":"2024-04-24T23:27:26.216910+0000","last_undegraded":"2024-04-24T23:27:26.216910+0000","last_fullsized":"2024-04-24T23:27:26.216910+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:43:13.755833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.283753+0000","last_change":"2024-04-24T23:27:24.232136+0000","last_active":"2024-04-24T23:27:26.283753+0000","last_peered":"2024-04-24T23:27:26.283753+0000","last_clean":"2024-04-24T23:27:26.283753+0000","last_became_active":"2024-04-24T23:27:24.231459+0000","last_became_peered":"2024-04-24T23:27:24.231459+0000","last_unstale":"2024-04-24T23:27:26.283753+0000","last_undegraded":"2024-04-24T23:27:26.283753+0000","last_fullsized":"2024-04-24T23:27:26.283753+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:51:51.114787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"84'1","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.216545+0000","last_change":"2024-04-24T23:27:18.975333+0000","last_active":"2024-04-24T23:27:26.216545+0000","last_peered":"2024-04-24T23:27:26.216545+0000","last_clean":"2024-04-24T23:27:26.216545+0000","last_became_active":"2024-04-24T23:27:18.974869+0000","last_became_peered":"2024-04-24T23:27:18.974869+0000","last_unstale":"2024-04-24T23:27:26.216545+0000","last_undegraded":"2024-04-24T23:27:26.216545+0000","last_fullsized":"2024-04-24T23:27:26.216545+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:30:26.758596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.851098+0000","last_change":"2024-04-24T23:27:19.974989+0000","last_active":"2024-04-24T23:27:26.851098+0000","last_peered":"2024-04-24T23:27:26.851098+0000","last_clean":"2024-04-24T23:27:26.851098+0000","last_became_active":"2024-04-24T23:27:19.974312+0000","last_became_peered":"2024-04-24T23:27:19.974312+0000","last_unstale":"2024-04-24T23:27:26.851098+0000","last_undegraded":"2024-04-24T23:27:26.851098+0000","last_fullsized":"2024-04-24T23:27:26.851098+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:16:39.685430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.283739+0000","last_change":"2024-04-24T23:27:24.229404+0000","last_active":"2024-04-24T23:27:26.283739+0000","last_peered":"2024-04-24T23:27:26.283739+0000","last_clean":"2024-04-24T23:27:26.283739+0000","last_became_active":"2024-04-24T23:27:24.228905+0000","last_became_peered":"2024-04-24T23:27:24.228905+0000","last_unstale":"2024-04-24T23:27:26.283739+0000","last_undegraded":"2024-04-24T23:27:26.283739+0000","last_fullsized":"2024-04-24T23:27:26.283739+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:59:25.069173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"84'1","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.852773+0000","last_change":"2024-04-24T23:27:18.971313+0000","last_active":"2024-04-24T23:27:26.852773+0000","last_peered":"2024-04-24T23:27:26.852773+0000","last_clean":"2024-04-24T23:27:26.852773+0000","last_became_active":"2024-04-24T23:27:18.970888+0000","last_became_peered":"2024-04-24T23:27:18.970888+0000","last_unstale":"2024-04-24T23:27:26.852773+0000","last_undegraded":"2024-04-24T23:27:26.852773+0000","last_fullsized":"2024-04-24T23:27:26.852773+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:34:20.118693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.216658+0000","last_change":"2024-04-24T23:27:19.978939+0000","last_active":"2024-04-24T23:27:26.216658+0000","last_peered":"2024-04-24T23:27:26.216658+0000","last_clean":"2024-04-24T23:27:26.216658+0000","last_became_active":"2024-04-24T23:27:19.978556+0000","last_became_peered":"2024-04-24T23:27:19.978556+0000","last_unstale":"2024-04-24T23:27:26.216658+0000","last_undegraded":"2024-04-24T23:27:26.216658+0000","last_fullsized":"2024-04-24T23:27:26.216658+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:28:43.118561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.887964+0000","last_change":"2024-04-24T23:27:24.232384+0000","last_active":"2024-04-24T23:27:26.887964+0000","last_peered":"2024-04-24T23:27:26.887964+0000","last_clean":"2024-04-24T23:27:26.887964+0000","last_became_active":"2024-04-24T23:27:24.231497+0000","last_became_peered":"2024-04-24T23:27:24.231497+0000","last_unstale":"2024-04-24T23:27:26.887964+0000","last_undegraded":"2024-04-24T23:27:26.887964+0000","last_fullsized":"2024-04-24T23:27:26.887964+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:26:54.716821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.215385+0000","last_change":"2024-04-24T23:27:18.968949+0000","last_active":"2024-04-24T23:27:26.215385+0000","last_peered":"2024-04-24T23:27:26.215385+0000","last_clean":"2024-04-24T23:27:26.215385+0000","last_became_active":"2024-04-24T23:27:18.968612+0000","last_became_peered":"2024-04-24T23:27:18.968612+0000","last_unstale":"2024-04-24T23:27:26.215385+0000","last_undegraded":"2024-04-24T23:27:26.215385+0000","last_fullsized":"2024-04-24T23:27:26.215385+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:03:45.908080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.851074+0000","last_change":"2024-04-24T23:27:19.979670+0000","last_active":"2024-04-24T23:27:26.851074+0000","last_peered":"2024-04-24T23:27:26.851074+0000","last_clean":"2024-04-24T23:27:26.851074+0000","last_became_active":"2024-04-24T23:27:19.979277+0000","last_became_peered":"2024-04-24T23:27:19.979277+0000","last_unstale":"2024-04-24T23:27:26.851074+0000","last_undegraded":"2024-04-24T23:27:26.851074+0000","last_fullsized":"2024-04-24T23:27:26.851074+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:05:54.958747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.853095+0000","last_change":"2024-04-24T23:27:24.237085+0000","last_active":"2024-04-24T23:27:26.853095+0000","last_peered":"2024-04-24T23:27:26.853095+0000","last_clean":"2024-04-24T23:27:26.853095+0000","last_became_active":"2024-04-24T23:27:24.236671+0000","last_became_peered":"2024-04-24T23:27:24.236671+0000","last_unstale":"2024-04-24T23:27:26.853095+0000","last_undegraded":"2024-04-24T23:27:26.853095+0000","last_fullsized":"2024-04-24T23:27:26.853095+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:48:38.482374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.216335+0000","last_change":"2024-04-24T23:27:18.974146+0000","last_active":"2024-04-24T23:27:26.216335+0000","last_peered":"2024-04-24T23:27:26.216335+0000","last_clean":"2024-04-24T23:27:26.216335+0000","last_became_active":"2024-04-24T23:27:18.973814+0000","last_became_peered":"2024-04-24T23:27:18.973814+0000","last_unstale":"2024-04-24T23:27:26.216335+0000","last_undegraded":"2024-04-24T23:27:26.216335+0000","last_fullsized":"2024-04-24T23:27:26.216335+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:22:52.409448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.851753+0000","last_change":"2024-04-24T23:27:19.984325+0000","last_active":"2024-04-24T23:27:26.851753+0000","last_peered":"2024-04-24T23:27:26.851753+0000","last_clean":"2024-04-24T23:27:26.851753+0000","last_became_active":"2024-04-24T23:27:19.983918+0000","last_became_peered":"2024-04-24T23:27:19.983918+0000","last_unstale":"2024-04-24T23:27:26.851753+0000","last_undegraded":"2024-04-24T23:27:26.851753+0000","last_fullsized":"2024-04-24T23:27:26.851753+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:24:07.640983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.851460+0000","last_change":"2024-04-24T23:27:24.223561+0000","last_active":"2024-04-24T23:27:26.851460+0000","last_peered":"2024-04-24T23:27:26.851460+0000","last_clean":"2024-04-24T23:27:26.851460+0000","last_became_active":"2024-04-24T23:27:24.223127+0000","last_became_peered":"2024-04-24T23:27:24.223127+0000","last_unstale":"2024-04-24T23:27:26.851460+0000","last_undegraded":"2024-04-24T23:27:26.851460+0000","last_fullsized":"2024-04-24T23:27:26.851460+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:31:28.079633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.852272+0000","last_change":"2024-04-24T23:27:18.971374+0000","last_active":"2024-04-24T23:27:26.852272+0000","last_peered":"2024-04-24T23:27:26.852272+0000","last_clean":"2024-04-24T23:27:26.852272+0000","last_became_active":"2024-04-24T23:27:18.970818+0000","last_became_peered":"2024-04-24T23:27:18.970818+0000","last_unstale":"2024-04-24T23:27:26.852272+0000","last_undegraded":"2024-04-24T23:27:26.852272+0000","last_fullsized":"2024-04-24T23:27:26.852272+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:12:17.547968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.852494+0000","last_change":"2024-04-24T23:27:19.980275+0000","last_active":"2024-04-24T23:27:26.852494+0000","last_peered":"2024-04-24T23:27:26.852494+0000","last_clean":"2024-04-24T23:27:26.852494+0000","last_became_active":"2024-04-24T23:27:19.979925+0000","last_became_peered":"2024-04-24T23:27:19.979925+0000","last_unstale":"2024-04-24T23:27:26.852494+0000","last_undegraded":"2024-04-24T23:27:26.852494+0000","last_fullsized":"2024-04-24T23:27:26.852494+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:21:10.681169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.851405+0000","last_change":"2024-04-24T23:27:24.231389+0000","last_active":"2024-04-24T23:27:26.851405+0000","last_peered":"2024-04-24T23:27:26.851405+0000","last_clean":"2024-04-24T23:27:26.851405+0000","last_became_active":"2024-04-24T23:27:24.230973+0000","last_became_peered":"2024-04-24T23:27:24.230973+0000","last_unstale":"2024-04-24T23:27:26.851405+0000","last_undegraded":"2024-04-24T23:27:26.851405+0000","last_fullsized":"2024-04-24T23:27:26.851405+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:35.943122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:43.975849+0000","last_change":"2024-04-24T23:27:18.977009+0000","last_active":"2024-04-24T23:27:43.975849+0000","last_peered":"2024-04-24T23:27:43.975849+0000","last_clean":"2024-04-24T23:27:43.975849+0000","last_became_active":"2024-04-24T23:27:18.976667+0000","last_became_peered":"2024-04-24T23:27:18.976667+0000","last_unstale":"2024-04-24T23:27:43.975849+0000","last_undegraded":"2024-04-24T23:27:43.975849+0000","last_fullsized":"2024-04-24T23:27:43.975849+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:19:47.817961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.853087+0000","last_change":"2024-04-24T23:27:19.988589+0000","last_active":"2024-04-24T23:27:26.853087+0000","last_peered":"2024-04-24T23:27:26.853087+0000","last_clean":"2024-04-24T23:27:26.853087+0000","last_became_active":"2024-04-24T23:27:19.988261+0000","last_became_peered":"2024-04-24T23:27:19.988261+0000","last_unstale":"2024-04-24T23:27:26.853087+0000","last_undegraded":"2024-04-24T23:27:26.853087+0000","last_fullsized":"2024-04-24T23:27:26.853087+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:44:15.809776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.852146+0000","last_change":"2024-04-24T23:27:24.231256+0000","last_active":"2024-04-24T23:27:26.852146+0000","last_peered":"2024-04-24T23:27:26.852146+0000","last_clean":"2024-04-24T23:27:26.852146+0000","last_became_active":"2024-04-24T23:27:24.230545+0000","last_became_peered":"2024-04-24T23:27:24.230545+0000","last_unstale":"2024-04-24T23:27:26.852146+0000","last_undegraded":"2024-04-24T23:27:26.852146+0000","last_fullsized":"2024-04-24T23:27:26.852146+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:43:40.460473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"84'3","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.216810+0000","last_change":"2024-04-24T23:27:18.968805+0000","last_active":"2024-04-24T23:27:26.216810+0000","last_peered":"2024-04-24T23:27:26.216810+0000","last_clean":"2024-04-24T23:27:26.216810+0000","last_became_active":"2024-04-24T23:27:18.968423+0000","last_became_peered":"2024-04-24T23:27:18.968423+0000","last_unstale":"2024-04-24T23:27:26.216810+0000","last_undegraded":"2024-04-24T23:27:26.216810+0000","last_fullsized":"2024-04-24T23:27:26.216810+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:14:04.077773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.852319+0000","last_change":"2024-04-24T23:27:19.981986+0000","last_active":"2024-04-24T23:27:26.852319+0000","last_peered":"2024-04-24T23:27:26.852319+0000","last_clean":"2024-04-24T23:27:26.852319+0000","last_became_active":"2024-04-24T23:27:19.981508+0000","last_became_peered":"2024-04-24T23:27:19.981508+0000","last_unstale":"2024-04-24T23:27:26.852319+0000","last_undegraded":"2024-04-24T23:27:26.852319+0000","last_fullsized":"2024-04-24T23:27:26.852319+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:59:21.707481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.852123+0000","last_change":"2024-04-24T23:27:24.225481+0000","last_active":"2024-04-24T23:27:26.852123+0000","last_peered":"2024-04-24T23:27:26.852123+0000","last_clean":"2024-04-24T23:27:26.852123+0000","last_became_active":"2024-04-24T23:27:24.224363+0000","last_became_peered":"2024-04-24T23:27:24.224363+0000","last_unstale":"2024-04-24T23:27:26.852123+0000","last_undegraded":"2024-04-24T23:27:26.852123+0000","last_fullsized":"2024-04-24T23:27:26.852123+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:36:17.715596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"90'10","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:35.673929+0000","last_change":"2024-04-24T23:27:18.968264+0000","last_active":"2024-04-24T23:27:35.673929+0000","last_peered":"2024-04-24T23:27:35.673929+0000","last_clean":"2024-04-24T23:27:35.673929+0000","last_became_active":"2024-04-24T23:27:18.967856+0000","last_became_peered":"2024-04-24T23:27:18.967856+0000","last_unstale":"2024-04-24T23:27:35.673929+0000","last_undegraded":"2024-04-24T23:27:35.673929+0000","last_fullsized":"2024-04-24T23:27:35.673929+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:48:44.347254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.851422+0000","last_change":"2024-04-24T23:27:19.984719+0000","last_active":"2024-04-24T23:27:26.851422+0000","last_peered":"2024-04-24T23:27:26.851422+0000","last_clean":"2024-04-24T23:27:26.851422+0000","last_became_active":"2024-04-24T23:27:19.984386+0000","last_became_peered":"2024-04-24T23:27:19.984386+0000","last_unstale":"2024-04-24T23:27:26.851422+0000","last_undegraded":"2024-04-24T23:27:26.851422+0000","last_fullsized":"2024-04-24T23:27:26.851422+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:02:13.739759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.217239+0000","last_change":"2024-04-24T23:27:24.230795+0000","last_active":"2024-04-24T23:27:26.217239+0000","last_peered":"2024-04-24T23:27:26.217239+0000","last_clean":"2024-04-24T23:27:26.217239+0000","last_became_active":"2024-04-24T23:27:24.230360+0000","last_became_peered":"2024-04-24T23:27:24.230360+0000","last_unstale":"2024-04-24T23:27:26.217239+0000","last_undegraded":"2024-04-24T23:27:26.217239+0000","last_fullsized":"2024-04-24T23:27:26.217239+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:51:24.875668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"84'2","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.283530+0000","last_change":"2024-04-24T23:27:18.964964+0000","last_active":"2024-04-24T23:27:26.283530+0000","last_peered":"2024-04-24T23:27:26.283530+0000","last_clean":"2024-04-24T23:27:26.283530+0000","last_became_active":"2024-04-24T23:27:18.964632+0000","last_became_peered":"2024-04-24T23:27:18.964632+0000","last_unstale":"2024-04-24T23:27:26.283530+0000","last_undegraded":"2024-04-24T23:27:26.283530+0000","last_fullsized":"2024-04-24T23:27:26.283530+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:52:23.144629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.215746+0000","last_change":"2024-04-24T23:27:19.987258+0000","last_active":"2024-04-24T23:27:26.215746+0000","last_peered":"2024-04-24T23:27:26.215746+0000","last_clean":"2024-04-24T23:27:26.215746+0000","last_became_active":"2024-04-24T23:27:19.986864+0000","last_became_peered":"2024-04-24T23:27:19.986864+0000","last_unstale":"2024-04-24T23:27:26.215746+0000","last_undegraded":"2024-04-24T23:27:26.215746+0000","last_fullsized":"2024-04-24T23:27:26.215746+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:57:40.503506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.887768+0000","last_change":"2024-04-24T23:27:24.223758+0000","last_active":"2024-04-24T23:27:26.887768+0000","last_peered":"2024-04-24T23:27:26.887768+0000","last_clean":"2024-04-24T23:27:26.887768+0000","last_became_active":"2024-04-24T23:27:24.223343+0000","last_became_peered":"2024-04-24T23:27:24.223343+0000","last_unstale":"2024-04-24T23:27:26.887768+0000","last_undegraded":"2024-04-24T23:27:26.887768+0000","last_fullsized":"2024-04-24T23:27:26.887768+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:41:52.620132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.216550+0000","last_change":"2024-04-24T23:27:18.974229+0000","last_active":"2024-04-24T23:27:26.216550+0000","last_peered":"2024-04-24T23:27:26.216550+0000","last_clean":"2024-04-24T23:27:26.216550+0000","last_became_active":"2024-04-24T23:27:18.973893+0000","last_became_peered":"2024-04-24T23:27:18.973893+0000","last_unstale":"2024-04-24T23:27:26.216550+0000","last_undegraded":"2024-04-24T23:27:26.216550+0000","last_fullsized":"2024-04-24T23:27:26.216550+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:41:45.699709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.217043+0000","last_change":"2024-04-24T23:27:19.986910+0000","last_active":"2024-04-24T23:27:26.217043+0000","last_peered":"2024-04-24T23:27:26.217043+0000","last_clean":"2024-04-24T23:27:26.217043+0000","last_became_active":"2024-04-24T23:27:19.986556+0000","last_became_peered":"2024-04-24T23:27:19.986556+0000","last_unstale":"2024-04-24T23:27:26.217043+0000","last_undegraded":"2024-04-24T23:27:26.217043+0000","last_fullsized":"2024-04-24T23:27:26.217043+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:06:28.506556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.852652+0000","last_change":"2024-04-24T23:27:24.233115+0000","last_active":"2024-04-24T23:27:26.852652+0000","last_peered":"2024-04-24T23:27:26.852652+0000","last_clean":"2024-04-24T23:27:26.852652+0000","last_became_active":"2024-04-24T23:27:24.232714+0000","last_became_peered":"2024-04-24T23:27:24.232714+0000","last_unstale":"2024-04-24T23:27:26.852652+0000","last_undegraded":"2024-04-24T23:27:26.852652+0000","last_fullsized":"2024-04-24T23:27:26.852652+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:04.365515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.851468+0000","last_change":"2024-04-24T23:27:18.966426+0000","last_active":"2024-04-24T23:27:26.851468+0000","last_peered":"2024-04-24T23:27:26.851468+0000","last_clean":"2024-04-24T23:27:26.851468+0000","last_became_active":"2024-04-24T23:27:18.966070+0000","last_became_peered":"2024-04-24T23:27:18.966070+0000","last_unstale":"2024-04-24T23:27:26.851468+0000","last_undegraded":"2024-04-24T23:27:26.851468+0000","last_fullsized":"2024-04-24T23:27:26.851468+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:56:21.679639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.216614+0000","last_change":"2024-04-24T23:27:19.991829+0000","last_active":"2024-04-24T23:27:26.216614+0000","last_peered":"2024-04-24T23:27:26.216614+0000","last_clean":"2024-04-24T23:27:26.216614+0000","last_became_active":"2024-04-24T23:27:19.991492+0000","last_became_peered":"2024-04-24T23:27:19.991492+0000","last_unstale":"2024-04-24T23:27:26.216614+0000","last_undegraded":"2024-04-24T23:27:26.216614+0000","last_fullsized":"2024-04-24T23:27:26.216614+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:23:45.195217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.216307+0000","last_change":"2024-04-24T23:27:24.229794+0000","last_active":"2024-04-24T23:27:26.216307+0000","last_peered":"2024-04-24T23:27:26.216307+0000","last_clean":"2024-04-24T23:27:26.216307+0000","last_became_active":"2024-04-24T23:27:24.229410+0000","last_became_peered":"2024-04-24T23:27:24.229410+0000","last_unstale":"2024-04-24T23:27:26.216307+0000","last_undegraded":"2024-04-24T23:27:26.216307+0000","last_fullsized":"2024-04-24T23:27:26.216307+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:00:53.188307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:44.988890+0000","last_change":"2024-04-24T23:27:18.969071+0000","last_active":"2024-04-24T23:27:44.988890+0000","last_peered":"2024-04-24T23:27:44.988890+0000","last_clean":"2024-04-24T23:27:44.988890+0000","last_became_active":"2024-04-24T23:27:18.968671+0000","last_became_peered":"2024-04-24T23:27:18.968671+0000","last_unstale":"2024-04-24T23:27:44.988890+0000","last_undegraded":"2024-04-24T23:27:44.988890+0000","last_fullsized":"2024-04-24T23:27:44.988890+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:50:23.743556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.216391+0000","last_change":"2024-04-24T23:27:19.982608+0000","last_active":"2024-04-24T23:27:26.216391+0000","last_peered":"2024-04-24T23:27:26.216391+0000","last_clean":"2024-04-24T23:27:26.216391+0000","last_became_active":"2024-04-24T23:27:19.982270+0000","last_became_peered":"2024-04-24T23:27:19.982270+0000","last_unstale":"2024-04-24T23:27:26.216391+0000","last_undegraded":"2024-04-24T23:27:26.216391+0000","last_fullsized":"2024-04-24T23:27:26.216391+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:01:10.525164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.216378+0000","last_change":"2024-04-24T23:27:24.236143+0000","last_active":"2024-04-24T23:27:26.216378+0000","last_peered":"2024-04-24T23:27:26.216378+0000","last_clean":"2024-04-24T23:27:26.216378+0000","last_became_active":"2024-04-24T23:27:24.235724+0000","last_became_peered":"2024-04-24T23:27:24.235724+0000","last_unstale":"2024-04-24T23:27:26.216378+0000","last_undegraded":"2024-04-24T23:27:26.216378+0000","last_fullsized":"2024-04-24T23:27:26.216378+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:55:41.317044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:45.983035+0000","last_change":"2024-04-24T23:27:18.971312+0000","last_active":"2024-04-24T23:27:45.983035+0000","last_peered":"2024-04-24T23:27:45.983035+0000","last_clean":"2024-04-24T23:27:45.983035+0000","last_became_active":"2024-04-24T23:27:18.970929+0000","last_became_peered":"2024-04-24T23:27:18.970929+0000","last_unstale":"2024-04-24T23:27:45.983035+0000","last_undegraded":"2024-04-24T23:27:45.983035+0000","last_fullsized":"2024-04-24T23:27:45.983035+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:54:20.318916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.887848+0000","last_change":"2024-04-24T23:27:19.981169+0000","last_active":"2024-04-24T23:27:26.887848+0000","last_peered":"2024-04-24T23:27:26.887848+0000","last_clean":"2024-04-24T23:27:26.887848+0000","last_became_active":"2024-04-24T23:27:19.980820+0000","last_became_peered":"2024-04-24T23:27:19.980820+0000","last_unstale":"2024-04-24T23:27:26.887848+0000","last_undegraded":"2024-04-24T23:27:26.887848+0000","last_fullsized":"2024-04-24T23:27:26.887848+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:51:00.464488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.851121+0000","last_change":"2024-04-24T23:27:24.718948+0000","last_active":"2024-04-24T23:27:26.851121+0000","last_peered":"2024-04-24T23:27:26.851121+0000","last_clean":"2024-04-24T23:27:26.851121+0000","last_became_active":"2024-04-24T23:27:24.718424+0000","last_became_peered":"2024-04-24T23:27:24.718424+0000","last_unstale":"2024-04-24T23:27:26.851121+0000","last_undegraded":"2024-04-24T23:27:26.851121+0000","last_fullsized":"2024-04-24T23:27:26.851121+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:22:01.984976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.852462+0000","last_change":"2024-04-24T23:27:18.957073+0000","last_active":"2024-04-24T23:27:26.852462+0000","last_peered":"2024-04-24T23:27:26.852462+0000","last_clean":"2024-04-24T23:27:26.852462+0000","last_became_active":"2024-04-24T23:27:18.956713+0000","last_became_peered":"2024-04-24T23:27:18.956713+0000","last_unstale":"2024-04-24T23:27:26.852462+0000","last_undegraded":"2024-04-24T23:27:26.852462+0000","last_fullsized":"2024-04-24T23:27:26.852462+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:16:21.015902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.852990+0000","last_change":"2024-04-24T23:27:19.985058+0000","last_active":"2024-04-24T23:27:26.852990+0000","last_peered":"2024-04-24T23:27:26.852990+0000","last_clean":"2024-04-24T23:27:26.852990+0000","last_became_active":"2024-04-24T23:27:19.984675+0000","last_became_peered":"2024-04-24T23:27:19.984675+0000","last_unstale":"2024-04-24T23:27:26.852990+0000","last_undegraded":"2024-04-24T23:27:26.852990+0000","last_fullsized":"2024-04-24T23:27:26.852990+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:22:35.252839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.215919+0000","last_change":"2024-04-24T23:27:24.236117+0000","last_active":"2024-04-24T23:27:26.215919+0000","last_peered":"2024-04-24T23:27:26.215919+0000","last_clean":"2024-04-24T23:27:26.215919+0000","last_became_active":"2024-04-24T23:27:24.235720+0000","last_became_peered":"2024-04-24T23:27:24.235720+0000","last_unstale":"2024-04-24T23:27:26.215919+0000","last_undegraded":"2024-04-24T23:27:26.215919+0000","last_fullsized":"2024-04-24T23:27:26.215919+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:46:40.604988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.216942+0000","last_change":"2024-04-24T23:27:18.973690+0000","last_active":"2024-04-24T23:27:26.216942+0000","last_peered":"2024-04-24T23:27:26.216942+0000","last_clean":"2024-04-24T23:27:26.216942+0000","last_became_active":"2024-04-24T23:27:18.973182+0000","last_became_peered":"2024-04-24T23:27:18.973182+0000","last_unstale":"2024-04-24T23:27:26.216942+0000","last_undegraded":"2024-04-24T23:27:26.216942+0000","last_fullsized":"2024-04-24T23:27:26.216942+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:00:44.809880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.851854+0000","last_change":"2024-04-24T23:27:19.991252+0000","last_active":"2024-04-24T23:27:26.851854+0000","last_peered":"2024-04-24T23:27:26.851854+0000","last_clean":"2024-04-24T23:27:26.851854+0000","last_became_active":"2024-04-24T23:27:19.990892+0000","last_became_peered":"2024-04-24T23:27:19.990892+0000","last_unstale":"2024-04-24T23:27:26.851854+0000","last_undegraded":"2024-04-24T23:27:26.851854+0000","last_fullsized":"2024-04-24T23:27:26.851854+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:29:37.835521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.216074+0000","last_change":"2024-04-24T23:27:24.718929+0000","last_active":"2024-04-24T23:27:26.216074+0000","last_peered":"2024-04-24T23:27:26.216074+0000","last_clean":"2024-04-24T23:27:26.216074+0000","last_became_active":"2024-04-24T23:27:24.718486+0000","last_became_peered":"2024-04-24T23:27:24.718486+0000","last_unstale":"2024-04-24T23:27:26.216074+0000","last_undegraded":"2024-04-24T23:27:26.216074+0000","last_fullsized":"2024-04-24T23:27:26.216074+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:31:13.359829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.216502+0000","last_change":"2024-04-24T23:27:19.983181+0000","last_active":"2024-04-24T23:27:26.216502+0000","last_peered":"2024-04-24T23:27:26.216502+0000","last_clean":"2024-04-24T23:27:26.216502+0000","last_became_active":"2024-04-24T23:27:19.981764+0000","last_became_peered":"2024-04-24T23:27:19.981764+0000","last_unstale":"2024-04-24T23:27:26.216502+0000","last_undegraded":"2024-04-24T23:27:26.216502+0000","last_fullsized":"2024-04-24T23:27:26.216502+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:01:20.653572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'1","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.215356+0000","last_change":"2024-04-24T23:27:18.976351+0000","last_active":"2024-04-24T23:27:26.215356+0000","last_peered":"2024-04-24T23:27:26.215356+0000","last_clean":"2024-04-24T23:27:26.215356+0000","last_became_active":"2024-04-24T23:27:18.975552+0000","last_became_peered":"2024-04-24T23:27:18.975552+0000","last_unstale":"2024-04-24T23:27:26.215356+0000","last_undegraded":"2024-04-24T23:27:26.215356+0000","last_fullsized":"2024-04-24T23:27:26.215356+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:26:29.551439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.216929+0000","last_change":"2024-04-24T23:27:24.229528+0000","last_active":"2024-04-24T23:27:26.216929+0000","last_peered":"2024-04-24T23:27:26.216929+0000","last_clean":"2024-04-24T23:27:26.216929+0000","last_became_active":"2024-04-24T23:27:24.229099+0000","last_became_peered":"2024-04-24T23:27:24.229099+0000","last_unstale":"2024-04-24T23:27:26.216929+0000","last_undegraded":"2024-04-24T23:27:26.216929+0000","last_fullsized":"2024-04-24T23:27:26.216929+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:51:51.421912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.852190+0000","last_change":"2024-04-24T23:27:19.980094+0000","last_active":"2024-04-24T23:27:26.852190+0000","last_peered":"2024-04-24T23:27:26.852190+0000","last_clean":"2024-04-24T23:27:26.852190+0000","last_became_active":"2024-04-24T23:27:19.979710+0000","last_became_peered":"2024-04-24T23:27:19.979710+0000","last_unstale":"2024-04-24T23:27:26.852190+0000","last_undegraded":"2024-04-24T23:27:26.852190+0000","last_fullsized":"2024-04-24T23:27:26.852190+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:50:37.920235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.851996+0000","last_change":"2024-04-24T23:27:18.963224+0000","last_active":"2024-04-24T23:27:26.851996+0000","last_peered":"2024-04-24T23:27:26.851996+0000","last_clean":"2024-04-24T23:27:26.851996+0000","last_became_active":"2024-04-24T23:27:18.962794+0000","last_became_peered":"2024-04-24T23:27:18.962794+0000","last_unstale":"2024-04-24T23:27:26.851996+0000","last_undegraded":"2024-04-24T23:27:26.851996+0000","last_fullsized":"2024-04-24T23:27:26.851996+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:05:36.869062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":349,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.216468+0000","last_change":"2024-04-24T23:26:02.307599+0000","last_active":"2024-04-24T23:27:26.216468+0000","last_peered":"2024-04-24T23:27:26.216468+0000","last_clean":"2024-04-24T23:27:26.216468+0000","last_became_active":"2024-04-24T23:26:02.307186+0000","last_became_peered":"2024-04-24T23:26:02.307186+0000","last_unstale":"2024-04-24T23:27:26.216468+0000","last_undegraded":"2024-04-24T23:27:26.216468+0000","last_fullsized":"2024-04-24T23:27:26.216468+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:22:33.824274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:22:33.824274+0000","last_clean_scrub_stamp":"2024-04-24T23:22:33.824274+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:19:26.789955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.216990+0000","last_change":"2024-04-24T23:27:24.719315+0000","last_active":"2024-04-24T23:27:26.216990+0000","last_peered":"2024-04-24T23:27:26.216990+0000","last_clean":"2024-04-24T23:27:26.216990+0000","last_became_active":"2024-04-24T23:27:24.718868+0000","last_became_peered":"2024-04-24T23:27:24.718868+0000","last_unstale":"2024-04-24T23:27:26.216990+0000","last_undegraded":"2024-04-24T23:27:26.216990+0000","last_fullsized":"2024-04-24T23:27:26.216990+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:43:32.122873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.215281+0000","last_change":"2024-04-24T23:27:19.983711+0000","last_active":"2024-04-24T23:27:26.215281+0000","last_peered":"2024-04-24T23:27:26.215281+0000","last_clean":"2024-04-24T23:27:26.215281+0000","last_became_active":"2024-04-24T23:27:19.983302+0000","last_became_peered":"2024-04-24T23:27:19.983302+0000","last_unstale":"2024-04-24T23:27:26.215281+0000","last_undegraded":"2024-04-24T23:27:26.215281+0000","last_fullsized":"2024-04-24T23:27:26.215281+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:44:49.251679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'1","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.216737+0000","last_change":"2024-04-24T23:27:18.976766+0000","last_active":"2024-04-24T23:27:26.216737+0000","last_peered":"2024-04-24T23:27:26.216737+0000","last_clean":"2024-04-24T23:27:26.216737+0000","last_became_active":"2024-04-24T23:27:18.976420+0000","last_became_peered":"2024-04-24T23:27:18.976420+0000","last_unstale":"2024-04-24T23:27:26.216737+0000","last_undegraded":"2024-04-24T23:27:26.216737+0000","last_fullsized":"2024-04-24T23:27:26.216737+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:59:34.262774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.887765+0000","last_change":"2024-04-24T23:27:24.226979+0000","last_active":"2024-04-24T23:27:26.887765+0000","last_peered":"2024-04-24T23:27:26.887765+0000","last_clean":"2024-04-24T23:27:26.887765+0000","last_became_active":"2024-04-24T23:27:24.226596+0000","last_became_peered":"2024-04-24T23:27:24.226596+0000","last_unstale":"2024-04-24T23:27:26.887765+0000","last_undegraded":"2024-04-24T23:27:26.887765+0000","last_fullsized":"2024-04-24T23:27:26.887765+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:50:48.585067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:43.973625+0000","last_change":"2024-04-24T23:27:18.970811+0000","last_active":"2024-04-24T23:27:43.973625+0000","last_peered":"2024-04-24T23:27:43.973625+0000","last_clean":"2024-04-24T23:27:43.973625+0000","last_became_active":"2024-04-24T23:27:18.970292+0000","last_became_peered":"2024-04-24T23:27:18.970292+0000","last_unstale":"2024-04-24T23:27:43.973625+0000","last_undegraded":"2024-04-24T23:27:43.973625+0000","last_fullsized":"2024-04-24T23:27:43.973625+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:06:37.556527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.215988+0000","last_change":"2024-04-24T23:27:19.975455+0000","last_active":"2024-04-24T23:27:26.215988+0000","last_peered":"2024-04-24T23:27:26.215988+0000","last_clean":"2024-04-24T23:27:26.215988+0000","last_became_active":"2024-04-24T23:27:19.973504+0000","last_became_peered":"2024-04-24T23:27:19.973504+0000","last_unstale":"2024-04-24T23:27:26.215988+0000","last_undegraded":"2024-04-24T23:27:26.215988+0000","last_fullsized":"2024-04-24T23:27:26.215988+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:50:34.450370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.850941+0000","last_change":"2024-04-24T23:27:24.719253+0000","last_active":"2024-04-24T23:27:26.850941+0000","last_peered":"2024-04-24T23:27:26.850941+0000","last_clean":"2024-04-24T23:27:26.850941+0000","last_became_active":"2024-04-24T23:27:24.718357+0000","last_became_peered":"2024-04-24T23:27:24.718357+0000","last_unstale":"2024-04-24T23:27:26.850941+0000","last_undegraded":"2024-04-24T23:27:26.850941+0000","last_fullsized":"2024-04-24T23:27:26.850941+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:31:03.798037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"85'1","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.853106+0000","last_change":"2024-04-24T23:27:18.974310+0000","last_active":"2024-04-24T23:27:26.853106+0000","last_peered":"2024-04-24T23:27:26.853106+0000","last_clean":"2024-04-24T23:27:26.853106+0000","last_became_active":"2024-04-24T23:27:18.973956+0000","last_became_peered":"2024-04-24T23:27:18.973956+0000","last_unstale":"2024-04-24T23:27:26.853106+0000","last_undegraded":"2024-04-24T23:27:26.853106+0000","last_fullsized":"2024-04-24T23:27:26.853106+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:44:27.233155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.851172+0000","last_change":"2024-04-24T23:27:19.985763+0000","last_active":"2024-04-24T23:27:26.851172+0000","last_peered":"2024-04-24T23:27:26.851172+0000","last_clean":"2024-04-24T23:27:26.851172+0000","last_became_active":"2024-04-24T23:27:19.985413+0000","last_became_peered":"2024-04-24T23:27:19.985413+0000","last_unstale":"2024-04-24T23:27:26.851172+0000","last_undegraded":"2024-04-24T23:27:26.851172+0000","last_fullsized":"2024-04-24T23:27:26.851172+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:04:54.910407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.283200+0000","last_change":"2024-04-24T23:27:24.225615+0000","last_active":"2024-04-24T23:27:26.283200+0000","last_peered":"2024-04-24T23:27:26.283200+0000","last_clean":"2024-04-24T23:27:26.283200+0000","last_became_active":"2024-04-24T23:27:24.225029+0000","last_became_peered":"2024-04-24T23:27:24.225029+0000","last_unstale":"2024-04-24T23:27:26.283200+0000","last_undegraded":"2024-04-24T23:27:26.283200+0000","last_fullsized":"2024-04-24T23:27:26.283200+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:56:02.956669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"84'2","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.851720+0000","last_change":"2024-04-24T23:27:18.963235+0000","last_active":"2024-04-24T23:27:26.851720+0000","last_peered":"2024-04-24T23:27:26.851720+0000","last_clean":"2024-04-24T23:27:26.851720+0000","last_became_active":"2024-04-24T23:27:18.962835+0000","last_became_peered":"2024-04-24T23:27:18.962835+0000","last_unstale":"2024-04-24T23:27:26.851720+0000","last_undegraded":"2024-04-24T23:27:26.851720+0000","last_fullsized":"2024-04-24T23:27:26.851720+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:34:29.376856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.853191+0000","last_change":"2024-04-24T23:27:19.987386+0000","last_active":"2024-04-24T23:27:26.853191+0000","last_peered":"2024-04-24T23:27:26.853191+0000","last_clean":"2024-04-24T23:27:26.853191+0000","last_became_active":"2024-04-24T23:27:19.987003+0000","last_became_peered":"2024-04-24T23:27:19.987003+0000","last_unstale":"2024-04-24T23:27:26.853191+0000","last_undegraded":"2024-04-24T23:27:26.853191+0000","last_fullsized":"2024-04-24T23:27:26.853191+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:20:25.635732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.851481+0000","last_change":"2024-04-24T23:27:24.719291+0000","last_active":"2024-04-24T23:27:26.851481+0000","last_peered":"2024-04-24T23:27:26.851481+0000","last_clean":"2024-04-24T23:27:26.851481+0000","last_became_active":"2024-04-24T23:27:24.718796+0000","last_became_peered":"2024-04-24T23:27:24.718796+0000","last_unstale":"2024-04-24T23:27:26.851481+0000","last_undegraded":"2024-04-24T23:27:26.851481+0000","last_fullsized":"2024-04-24T23:27:26.851481+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:20:25.135902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:43.974506+0000","last_change":"2024-04-24T23:27:18.967856+0000","last_active":"2024-04-24T23:27:43.974506+0000","last_peered":"2024-04-24T23:27:43.974506+0000","last_clean":"2024-04-24T23:27:43.974506+0000","last_became_active":"2024-04-24T23:27:18.967497+0000","last_became_peered":"2024-04-24T23:27:18.967497+0000","last_unstale":"2024-04-24T23:27:43.974506+0000","last_undegraded":"2024-04-24T23:27:43.974506+0000","last_fullsized":"2024-04-24T23:27:43.974506+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:21:29.841791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.850963+0000","last_change":"2024-04-24T23:27:19.981302+0000","last_active":"2024-04-24T23:27:26.850963+0000","last_peered":"2024-04-24T23:27:26.850963+0000","last_clean":"2024-04-24T23:27:26.850963+0000","last_became_active":"2024-04-24T23:27:19.980938+0000","last_became_peered":"2024-04-24T23:27:19.980938+0000","last_unstale":"2024-04-24T23:27:26.850963+0000","last_undegraded":"2024-04-24T23:27:26.850963+0000","last_fullsized":"2024-04-24T23:27:26.850963+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:24:53.939882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.853469+0000","last_change":"2024-04-24T23:27:24.224626+0000","last_active":"2024-04-24T23:27:26.853469+0000","last_peered":"2024-04-24T23:27:26.853469+0000","last_clean":"2024-04-24T23:27:26.853469+0000","last_became_active":"2024-04-24T23:27:24.224175+0000","last_became_peered":"2024-04-24T23:27:24.224175+0000","last_unstale":"2024-04-24T23:27:26.853469+0000","last_undegraded":"2024-04-24T23:27:26.853469+0000","last_fullsized":"2024-04-24T23:27:26.853469+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:54:44.155637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.851237+0000","last_change":"2024-04-24T23:27:18.975662+0000","last_active":"2024-04-24T23:27:26.851237+0000","last_peered":"2024-04-24T23:27:26.851237+0000","last_clean":"2024-04-24T23:27:26.851237+0000","last_became_active":"2024-04-24T23:27:18.975326+0000","last_became_peered":"2024-04-24T23:27:18.975326+0000","last_unstale":"2024-04-24T23:27:26.851237+0000","last_undegraded":"2024-04-24T23:27:26.851237+0000","last_fullsized":"2024-04-24T23:27:26.851237+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:39.440866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.887561+0000","last_change":"2024-04-24T23:27:19.978818+0000","last_active":"2024-04-24T23:27:26.887561+0000","last_peered":"2024-04-24T23:27:26.887561+0000","last_clean":"2024-04-24T23:27:26.887561+0000","last_became_active":"2024-04-24T23:27:19.978107+0000","last_became_peered":"2024-04-24T23:27:19.978107+0000","last_unstale":"2024-04-24T23:27:26.887561+0000","last_undegraded":"2024-04-24T23:27:26.887561+0000","last_fullsized":"2024-04-24T23:27:26.887561+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:22:25.797996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.851650+0000","last_change":"2024-04-24T23:27:24.234504+0000","last_active":"2024-04-24T23:27:26.851650+0000","last_peered":"2024-04-24T23:27:26.851650+0000","last_clean":"2024-04-24T23:27:26.851650+0000","last_became_active":"2024-04-24T23:27:24.234019+0000","last_became_peered":"2024-04-24T23:27:24.234019+0000","last_unstale":"2024-04-24T23:27:26.851650+0000","last_undegraded":"2024-04-24T23:27:26.851650+0000","last_fullsized":"2024-04-24T23:27:26.851650+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:06:19.101515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.283112+0000","last_change":"2024-04-24T23:27:18.967165+0000","last_active":"2024-04-24T23:27:26.283112+0000","last_peered":"2024-04-24T23:27:26.283112+0000","last_clean":"2024-04-24T23:27:26.283112+0000","last_became_active":"2024-04-24T23:27:18.966628+0000","last_became_peered":"2024-04-24T23:27:18.966628+0000","last_unstale":"2024-04-24T23:27:26.283112+0000","last_undegraded":"2024-04-24T23:27:26.283112+0000","last_fullsized":"2024-04-24T23:27:26.283112+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:12:20.520777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.851816+0000","last_change":"2024-04-24T23:27:19.988069+0000","last_active":"2024-04-24T23:27:26.851816+0000","last_peered":"2024-04-24T23:27:26.851816+0000","last_clean":"2024-04-24T23:27:26.851816+0000","last_became_active":"2024-04-24T23:27:19.987660+0000","last_became_peered":"2024-04-24T23:27:19.987660+0000","last_unstale":"2024-04-24T23:27:26.851816+0000","last_undegraded":"2024-04-24T23:27:26.851816+0000","last_fullsized":"2024-04-24T23:27:26.851816+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:21:03.505313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.851684+0000","last_change":"2024-04-24T23:27:24.719041+0000","last_active":"2024-04-24T23:27:26.851684+0000","last_peered":"2024-04-24T23:27:26.851684+0000","last_clean":"2024-04-24T23:27:26.851684+0000","last_became_active":"2024-04-24T23:27:24.718523+0000","last_became_peered":"2024-04-24T23:27:24.718523+0000","last_unstale":"2024-04-24T23:27:26.851684+0000","last_undegraded":"2024-04-24T23:27:26.851684+0000","last_fullsized":"2024-04-24T23:27:26.851684+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:27:57.396800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:43.975785+0000","last_change":"2024-04-24T23:27:18.975034+0000","last_active":"2024-04-24T23:27:43.975785+0000","last_peered":"2024-04-24T23:27:43.975785+0000","last_clean":"2024-04-24T23:27:43.975785+0000","last_became_active":"2024-04-24T23:27:18.973764+0000","last_became_peered":"2024-04-24T23:27:18.973764+0000","last_unstale":"2024-04-24T23:27:43.975785+0000","last_undegraded":"2024-04-24T23:27:43.975785+0000","last_fullsized":"2024-04-24T23:27:43.975785+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:43:23.451904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.216982+0000","last_change":"2024-04-24T23:27:19.984948+0000","last_active":"2024-04-24T23:27:26.216982+0000","last_peered":"2024-04-24T23:27:26.216982+0000","last_clean":"2024-04-24T23:27:26.216982+0000","last_became_active":"2024-04-24T23:27:19.984484+0000","last_became_peered":"2024-04-24T23:27:19.984484+0000","last_unstale":"2024-04-24T23:27:26.216982+0000","last_undegraded":"2024-04-24T23:27:26.216982+0000","last_fullsized":"2024-04-24T23:27:26.216982+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:17:43.335983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.852122+0000","last_change":"2024-04-24T23:27:24.231307+0000","last_active":"2024-04-24T23:27:26.852122+0000","last_peered":"2024-04-24T23:27:26.852122+0000","last_clean":"2024-04-24T23:27:26.852122+0000","last_became_active":"2024-04-24T23:27:24.230848+0000","last_became_peered":"2024-04-24T23:27:24.230848+0000","last_unstale":"2024-04-24T23:27:26.852122+0000","last_undegraded":"2024-04-24T23:27:26.852122+0000","last_fullsized":"2024-04-24T23:27:26.852122+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:28.586230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.283769+0000","last_change":"2024-04-24T23:27:18.964232+0000","last_active":"2024-04-24T23:27:26.283769+0000","last_peered":"2024-04-24T23:27:26.283769+0000","last_clean":"2024-04-24T23:27:26.283769+0000","last_became_active":"2024-04-24T23:27:18.963855+0000","last_became_peered":"2024-04-24T23:27:18.963855+0000","last_unstale":"2024-04-24T23:27:26.283769+0000","last_undegraded":"2024-04-24T23:27:26.283769+0000","last_fullsized":"2024-04-24T23:27:26.283769+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:02:21.000326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.217169+0000","last_change":"2024-04-24T23:27:19.984354+0000","last_active":"2024-04-24T23:27:26.217169+0000","last_peered":"2024-04-24T23:27:26.217169+0000","last_clean":"2024-04-24T23:27:26.217169+0000","last_became_active":"2024-04-24T23:27:19.983972+0000","last_became_peered":"2024-04-24T23:27:19.983972+0000","last_unstale":"2024-04-24T23:27:26.217169+0000","last_undegraded":"2024-04-24T23:27:26.217169+0000","last_fullsized":"2024-04-24T23:27:26.217169+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:28:28.390283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.852637+0000","last_change":"2024-04-24T23:27:24.238404+0000","last_active":"2024-04-24T23:27:26.852637+0000","last_peered":"2024-04-24T23:27:26.852637+0000","last_clean":"2024-04-24T23:27:26.852637+0000","last_became_active":"2024-04-24T23:27:24.237976+0000","last_became_peered":"2024-04-24T23:27:24.237976+0000","last_unstale":"2024-04-24T23:27:26.852637+0000","last_undegraded":"2024-04-24T23:27:26.852637+0000","last_fullsized":"2024-04-24T23:27:26.852637+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:19:31.476789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.216947+0000","last_change":"2024-04-24T23:27:18.971259+0000","last_active":"2024-04-24T23:27:26.216947+0000","last_peered":"2024-04-24T23:27:26.216947+0000","last_clean":"2024-04-24T23:27:26.216947+0000","last_became_active":"2024-04-24T23:27:18.970856+0000","last_became_peered":"2024-04-24T23:27:18.970856+0000","last_unstale":"2024-04-24T23:27:26.216947+0000","last_undegraded":"2024-04-24T23:27:26.216947+0000","last_fullsized":"2024-04-24T23:27:26.216947+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:54:58.642528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.216739+0000","last_change":"2024-04-24T23:27:19.986320+0000","last_active":"2024-04-24T23:27:26.216739+0000","last_peered":"2024-04-24T23:27:26.216739+0000","last_clean":"2024-04-24T23:27:26.216739+0000","last_became_active":"2024-04-24T23:27:19.985951+0000","last_became_peered":"2024-04-24T23:27:19.985951+0000","last_unstale":"2024-04-24T23:27:26.216739+0000","last_undegraded":"2024-04-24T23:27:26.216739+0000","last_fullsized":"2024-04-24T23:27:26.216739+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:05:35.427735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.216118+0000","last_change":"2024-04-24T23:27:24.228149+0000","last_active":"2024-04-24T23:27:26.216118+0000","last_peered":"2024-04-24T23:27:26.216118+0000","last_clean":"2024-04-24T23:27:26.216118+0000","last_became_active":"2024-04-24T23:27:24.227683+0000","last_became_peered":"2024-04-24T23:27:24.227683+0000","last_unstale":"2024-04-24T23:27:26.216118+0000","last_undegraded":"2024-04-24T23:27:26.216118+0000","last_fullsized":"2024-04-24T23:27:26.216118+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:55:50.408834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.216168+0000","last_change":"2024-04-24T23:27:18.976047+0000","last_active":"2024-04-24T23:27:26.216168+0000","last_peered":"2024-04-24T23:27:26.216168+0000","last_clean":"2024-04-24T23:27:26.216168+0000","last_became_active":"2024-04-24T23:27:18.975634+0000","last_became_peered":"2024-04-24T23:27:18.975634+0000","last_unstale":"2024-04-24T23:27:26.216168+0000","last_undegraded":"2024-04-24T23:27:26.216168+0000","last_fullsized":"2024-04-24T23:27:26.216168+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:04:35.528811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.217298+0000","last_change":"2024-04-24T23:27:19.990935+0000","last_active":"2024-04-24T23:27:26.217298+0000","last_peered":"2024-04-24T23:27:26.217298+0000","last_clean":"2024-04-24T23:27:26.217298+0000","last_became_active":"2024-04-24T23:27:19.990524+0000","last_became_peered":"2024-04-24T23:27:19.990524+0000","last_unstale":"2024-04-24T23:27:26.217298+0000","last_undegraded":"2024-04-24T23:27:26.217298+0000","last_fullsized":"2024-04-24T23:27:26.217298+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:54:02.463248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.887650+0000","last_change":"2024-04-24T23:27:19.979142+0000","last_active":"2024-04-24T23:27:26.887650+0000","last_peered":"2024-04-24T23:27:26.887650+0000","last_clean":"2024-04-24T23:27:26.887650+0000","last_became_active":"2024-04-24T23:27:19.978799+0000","last_became_peered":"2024-04-24T23:27:19.978799+0000","last_unstale":"2024-04-24T23:27:26.887650+0000","last_undegraded":"2024-04-24T23:27:26.887650+0000","last_fullsized":"2024-04-24T23:27:26.887650+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:07:19.046313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.283243+0000","last_change":"2024-04-24T23:27:19.986448+0000","last_active":"2024-04-24T23:27:26.283243+0000","last_peered":"2024-04-24T23:27:26.283243+0000","last_clean":"2024-04-24T23:27:26.283243+0000","last_became_active":"2024-04-24T23:27:19.986110+0000","last_became_peered":"2024-04-24T23:27:19.986110+0000","last_unstale":"2024-04-24T23:27:26.283243+0000","last_undegraded":"2024-04-24T23:27:26.283243+0000","last_fullsized":"2024-04-24T23:27:26.283243+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:21:25.819696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.283117+0000","last_change":"2024-04-24T23:27:19.980008+0000","last_active":"2024-04-24T23:27:26.283117+0000","last_peered":"2024-04-24T23:27:26.283117+0000","last_clean":"2024-04-24T23:27:26.283117+0000","last_became_active":"2024-04-24T23:27:19.979424+0000","last_became_peered":"2024-04-24T23:27:19.979424+0000","last_unstale":"2024-04-24T23:27:26.283117+0000","last_undegraded":"2024-04-24T23:27:26.283117+0000","last_fullsized":"2024-04-24T23:27:26.283117+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:44.449245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.852862+0000","last_change":"2024-04-24T23:27:19.987574+0000","last_active":"2024-04-24T23:27:26.852862+0000","last_peered":"2024-04-24T23:27:26.852862+0000","last_clean":"2024-04-24T23:27:26.852862+0000","last_became_active":"2024-04-24T23:27:19.987242+0000","last_became_peered":"2024-04-24T23:27:19.987242+0000","last_unstale":"2024-04-24T23:27:26.852862+0000","last_undegraded":"2024-04-24T23:27:26.852862+0000","last_fullsized":"2024-04-24T23:27:26.852862+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:39:10.777152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.216054+0000","last_change":"2024-04-24T23:27:19.980983+0000","last_active":"2024-04-24T23:27:26.216054+0000","last_peered":"2024-04-24T23:27:26.216054+0000","last_clean":"2024-04-24T23:27:26.216054+0000","last_became_active":"2024-04-24T23:27:19.980659+0000","last_became_peered":"2024-04-24T23:27:19.980659+0000","last_unstale":"2024-04-24T23:27:26.216054+0000","last_undegraded":"2024-04-24T23:27:26.216054+0000","last_fullsized":"2024-04-24T23:27:26.216054+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:02:52.647687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.888064+0000","last_change":"2024-04-24T23:27:19.988661+0000","last_active":"2024-04-24T23:27:26.888064+0000","last_peered":"2024-04-24T23:27:26.888064+0000","last_clean":"2024-04-24T23:27:26.888064+0000","last_became_active":"2024-04-24T23:27:19.988137+0000","last_became_peered":"2024-04-24T23:27:19.988137+0000","last_unstale":"2024-04-24T23:27:26.888064+0000","last_undegraded":"2024-04-24T23:27:26.888064+0000","last_fullsized":"2024-04-24T23:27:26.888064+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:47:59.736882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.283984+0000","last_change":"2024-04-24T23:27:19.985864+0000","last_active":"2024-04-24T23:27:26.283984+0000","last_peered":"2024-04-24T23:27:26.283984+0000","last_clean":"2024-04-24T23:27:26.283984+0000","last_became_active":"2024-04-24T23:27:19.984877+0000","last_became_peered":"2024-04-24T23:27:19.984877+0000","last_unstale":"2024-04-24T23:27:26.283984+0000","last_undegraded":"2024-04-24T23:27:26.283984+0000","last_fullsized":"2024-04-24T23:27:26.283984+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:14:43.458970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.852307+0000","last_change":"2024-04-24T23:27:19.988831+0000","last_active":"2024-04-24T23:27:26.852307+0000","last_peered":"2024-04-24T23:27:26.852307+0000","last_clean":"2024-04-24T23:27:26.852307+0000","last_became_active":"2024-04-24T23:27:19.988478+0000","last_became_peered":"2024-04-24T23:27:19.988478+0000","last_unstale":"2024-04-24T23:27:26.852307+0000","last_undegraded":"2024-04-24T23:27:26.852307+0000","last_fullsized":"2024-04-24T23:27:26.852307+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:18:37.488710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.216350+0000","last_change":"2024-04-24T23:27:19.983553+0000","last_active":"2024-04-24T23:27:26.216350+0000","last_peered":"2024-04-24T23:27:26.216350+0000","last_clean":"2024-04-24T23:27:26.216350+0000","last_became_active":"2024-04-24T23:27:19.981042+0000","last_became_peered":"2024-04-24T23:27:19.981042+0000","last_unstale":"2024-04-24T23:27:26.216350+0000","last_undegraded":"2024-04-24T23:27:26.216350+0000","last_fullsized":"2024-04-24T23:27:26.216350+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:09:42.608238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.851121+0000","last_change":"2024-04-24T23:27:19.985425+0000","last_active":"2024-04-24T23:27:26.851121+0000","last_peered":"2024-04-24T23:27:26.851121+0000","last_clean":"2024-04-24T23:27:26.851121+0000","last_became_active":"2024-04-24T23:27:19.985094+0000","last_became_peered":"2024-04-24T23:27:19.985094+0000","last_unstale":"2024-04-24T23:27:26.851121+0000","last_undegraded":"2024-04-24T23:27:26.851121+0000","last_fullsized":"2024-04-24T23:27:26.851121+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:07:15.612247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.852516+0000","last_change":"2024-04-24T23:27:19.988224+0000","last_active":"2024-04-24T23:27:26.852516+0000","last_peered":"2024-04-24T23:27:26.852516+0000","last_clean":"2024-04-24T23:27:26.852516+0000","last_became_active":"2024-04-24T23:27:19.987820+0000","last_became_peered":"2024-04-24T23:27:19.987820+0000","last_unstale":"2024-04-24T23:27:26.852516+0000","last_undegraded":"2024-04-24T23:27:26.852516+0000","last_fullsized":"2024-04-24T23:27:26.852516+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:27.653290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.215564+0000","last_change":"2024-04-24T23:27:19.989672+0000","last_active":"2024-04-24T23:27:26.215564+0000","last_peered":"2024-04-24T23:27:26.215564+0000","last_clean":"2024-04-24T23:27:26.215564+0000","last_became_active":"2024-04-24T23:27:19.989337+0000","last_became_peered":"2024-04-24T23:27:19.989337+0000","last_unstale":"2024-04-24T23:27:26.215564+0000","last_undegraded":"2024-04-24T23:27:26.215564+0000","last_fullsized":"2024-04-24T23:27:26.215564+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:07:13.399366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.887762+0000","last_change":"2024-04-24T23:27:19.987703+0000","last_active":"2024-04-24T23:27:26.887762+0000","last_peered":"2024-04-24T23:27:26.887762+0000","last_clean":"2024-04-24T23:27:26.887762+0000","last_became_active":"2024-04-24T23:27:19.987355+0000","last_became_peered":"2024-04-24T23:27:19.987355+0000","last_unstale":"2024-04-24T23:27:26.887762+0000","last_undegraded":"2024-04-24T23:27:26.887762+0000","last_fullsized":"2024-04-24T23:27:26.887762+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:03:15.812726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":280134,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4774,"num_objects":32,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":70,"num_write_kb":28,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":15862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":62,"ondisk_log_size":62,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383699,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":978944,"data_stored":487204,"data_compressed":17667,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645335,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1048576,"data_stored":1081026,"data_compressed":53298,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808861,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":978944,"data_stored":487204,"data_compressed":17667,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070499,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":995328,"data_stored":487360,"data_compressed":17667,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332135,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1036288,"data_stored":1078194,"data_compressed":53298,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495661,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":974848,"data_stored":486622,"data_compressed":17667,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574918,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36300,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708852,"statfs":{"total":95995035648,"available":95957864448,"internally_reserved":0,"allocated":987136,"data_stored":490464,"data_compressed":17667,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738442,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":486712,"data_compressed":17667,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411393,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":991232,"data_stored":490498,"data_compressed":17667,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182460,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":983040,"data_stored":486726,"data_compressed":17667,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920823,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":978944,"data_stored":486644,"data_compressed":17667,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757298,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1036288,"data_stored":1077172,"data_compressed":53298,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33993,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33993,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33993,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35631,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35631,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35631,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35631,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35631,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":4036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T23:27:48.673 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T23:27:48.806 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.803+0000 7f7daa1ce640 1 -- 172.21.15.80:0/338314189 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7da41006a0 msgr2=0x7f7da4100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:48.806 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.803+0000 7f7daa1ce640 1 --2- 172.21.15.80:0/338314189 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7da41006a0 0x7f7da4100ac0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f7d98009900 tx=0x7f7d9802a410 comp rx=0 tx=0).stop 2024-04-24T23:27:48.806 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.803+0000 7f7daa1ce640 1 -- 172.21.15.80:0/338314189 shutdown_connections 2024-04-24T23:27:48.806 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.803+0000 7f7daa1ce640 1 --2- 172.21.15.80:0/338314189 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7da4102250 0x7f7da4108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:48.806 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.803+0000 7f7daa1ce640 1 --2- 172.21.15.80:0/338314189 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7da4101890 0x7f7da4101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:48.806 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.803+0000 7f7daa1ce640 1 --2- 172.21.15.80:0/338314189 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7da41006a0 0x7f7da4100ac0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:48.806 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.803+0000 7f7daa1ce640 1 -- 172.21.15.80:0/338314189 >> 172.21.15.80:0/338314189 conn(0x7f7da40fbc30 msgr2=0x7f7da40fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:48.807 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.803+0000 7f7daa1ce640 1 -- 172.21.15.80:0/338314189 shutdown_connections 2024-04-24T23:27:48.807 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.803+0000 7f7daa1ce640 1 -- 172.21.15.80:0/338314189 wait complete. 2024-04-24T23:27:48.807 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.803+0000 7f7daa1ce640 1 Processor -- start 2024-04-24T23:27:48.807 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.804+0000 7f7daa1ce640 1 -- start start 2024-04-24T23:27:48.807 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.804+0000 7f7daa1ce640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7da41006a0 0x7f7da406a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:48.807 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.804+0000 7f7daa1ce640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7da4101890 0x7f7da419c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:48.807 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.804+0000 7f7daa1ce640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7da4102250 0x7f7da419ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:48.807 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.804+0000 7f7daa1ce640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f7da410c3f0 con 0x7f7da4101890 2024-04-24T23:27:48.807 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.804+0000 7f7daa1ce640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7da410c270 con 0x7f7da41006a0 2024-04-24T23:27:48.807 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.804+0000 7f7daa1ce640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f7da410c570 con 0x7f7da4102250 2024-04-24T23:27:48.807 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.804+0000 7f7da2ffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7da4101890 0x7f7da419c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:48.807 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.804+0000 7f7da2ffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7da4101890 0x7f7da419c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:37568/0 (socket says 172.21.15.80:37568) 2024-04-24T23:27:48.807 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.804+0000 7f7da2ffd640 1 -- 172.21.15.80:0/1123555111 learned_addr learned my addr 172.21.15.80:0/1123555111 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:27:48.807 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.804+0000 7f7da37fe640 1 --2- 172.21.15.80:0/1123555111 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7da41006a0 0x7f7da406a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:48.807 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.804+0000 7f7da3fff640 1 --2- 172.21.15.80:0/1123555111 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7da4102250 0x7f7da419ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:48.808 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.804+0000 7f7da2ffd640 1 -- 172.21.15.80:0/1123555111 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7da4102250 msgr2=0x7f7da419ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:48.808 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.804+0000 7f7da2ffd640 1 --2- 172.21.15.80:0/1123555111 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7da4102250 0x7f7da419ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:48.808 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.804+0000 7f7da2ffd640 1 -- 172.21.15.80:0/1123555111 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7da41006a0 msgr2=0x7f7da406a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:48.808 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.804+0000 7f7da2ffd640 1 --2- 172.21.15.80:0/1123555111 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7da41006a0 0x7f7da406a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:48.808 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.804+0000 7f7da2ffd640 1 -- 172.21.15.80:0/1123555111 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7da419f890 con 0x7f7da4101890 2024-04-24T23:27:48.808 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.804+0000 7f7da37fe640 1 --2- 172.21.15.80:0/1123555111 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7da41006a0 0x7f7da406a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:27:48.808 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.804+0000 7f7da2ffd640 1 --2- 172.21.15.80:0/1123555111 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7da4101890 0x7f7da419c960 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f7d9400b4a0 tx=0x7f7d9400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:48.808 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.805+0000 7f7da0ff9640 1 -- 172.21.15.80:0/1123555111 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d94014070 con 0x7f7da4101890 2024-04-24T23:27:48.808 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.805+0000 7f7daa1ce640 1 -- 172.21.15.80:0/1123555111 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7da419fb20 con 0x7f7da4101890 2024-04-24T23:27:48.808 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.805+0000 7f7da0ff9640 1 -- 172.21.15.80:0/1123555111 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d94010af0 con 0x7f7da4101890 2024-04-24T23:27:48.808 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.805+0000 7f7da0ff9640 1 -- 172.21.15.80:0/1123555111 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d94017d10 con 0x7f7da4101890 2024-04-24T23:27:48.808 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.805+0000 7f7daa1ce640 1 -- 172.21.15.80:0/1123555111 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7da41a00c0 con 0x7f7da4101890 2024-04-24T23:27:48.809 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.806+0000 7f7daa1ce640 1 -- 172.21.15.80:0/1123555111 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7da4107a90 con 0x7f7da4101890 2024-04-24T23:27:48.809 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.807+0000 7f7da0ff9640 1 -- 172.21.15.80:0/1123555111 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f7d94017070 con 0x7f7da4101890 2024-04-24T23:27:48.809 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.807+0000 7f7da0ff9640 1 --2- 172.21.15.80:0/1123555111 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f7d8c0a2c20 0x7f7d8c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:48.809 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.807+0000 7f7da37fe640 1 --2- 172.21.15.80:0/1123555111 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f7d8c0a2c20 0x7f7d8c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:48.810 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.808+0000 7f7da0ff9640 1 -- 172.21.15.80:0/1123555111 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f7d940ce720 con 0x7f7da4101890 2024-04-24T23:27:48.810 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.808+0000 7f7da37fe640 1 --2- 172.21.15.80:0/1123555111 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f7d8c0a2c20 0x7f7d8c0a50e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7d98000c00 tx=0x7f7d98031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:48.813 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.813+0000 7f7da0ff9640 1 -- 172.21.15.80:0/1123555111 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f7d9409b3f0 con 0x7f7da4101890 2024-04-24T23:27:48.985 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.984+0000 7f7daa1ce640 1 -- 172.21.15.80:0/1123555111 --> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7da4061b80 con 0x7f7d8c0a2c20 2024-04-24T23:27:48.995 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.994+0000 7f7da0ff9640 1 -- 172.21.15.80:0/1123555111 <== mgr.14152 v2:172.21.15.80:6800/3830596922 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340614 (secure 0 0 0) 0x7f7da4061b80 con 0x7f7d8c0a2c20 2024-04-24T23:27:48.999 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-24T23:27:49.000 INFO:teuthology.orchestra.run.smithi080.stderr:dumped all 2024-04-24T23:27:49.002 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.997+0000 7f7daa1ce640 1 -- 172.21.15.80:0/1123555111 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f7d8c0a2c20 msgr2=0x7f7d8c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:49.002 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.997+0000 7f7daa1ce640 1 --2- 172.21.15.80:0/1123555111 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f7d8c0a2c20 0x7f7d8c0a50e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7d98000c00 tx=0x7f7d98031400 comp rx=0 tx=0).stop 2024-04-24T23:27:49.002 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.997+0000 7f7daa1ce640 1 -- 172.21.15.80:0/1123555111 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7da4101890 msgr2=0x7f7da419c960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:49.002 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.997+0000 7f7daa1ce640 1 --2- 172.21.15.80:0/1123555111 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7da4101890 0x7f7da419c960 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f7d9400b4a0 tx=0x7f7d9400b960 comp rx=0 tx=0).stop 2024-04-24T23:27:49.003 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.998+0000 7f7daa1ce640 1 -- 172.21.15.80:0/1123555111 shutdown_connections 2024-04-24T23:27:49.003 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.998+0000 7f7daa1ce640 1 --2- 172.21.15.80:0/1123555111 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f7d8c0a2c20 0x7f7d8c0a50e0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:49.003 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.998+0000 7f7daa1ce640 1 --2- 172.21.15.80:0/1123555111 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7da4102250 0x7f7da419ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:49.003 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.998+0000 7f7daa1ce640 1 --2- 172.21.15.80:0/1123555111 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7da4101890 0x7f7da419c960 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:49.003 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.998+0000 7f7daa1ce640 1 --2- 172.21.15.80:0/1123555111 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7da41006a0 0x7f7da406a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:49.003 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.998+0000 7f7daa1ce640 1 -- 172.21.15.80:0/1123555111 >> 172.21.15.80:0/1123555111 conn(0x7f7da40fbc30 msgr2=0x7f7da4104c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:49.003 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.998+0000 7f7daa1ce640 1 -- 172.21.15.80:0/1123555111 shutdown_connections 2024-04-24T23:27:49.003 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:48.998+0000 7f7daa1ce640 1 -- 172.21.15.80:0/1123555111 wait complete. 2024-04-24T23:27:49.010 INFO:teuthology.orchestra.run.smithi080.stdout:{"pg_ready":true,"pg_map":{"version":311,"stamp":"2024-04-24T23:27:48.886470+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":634128,"num_objects":52,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":155,"num_read_kb":169,"num_write":305,"num_write_kb":3014,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":33,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":247,"ondisk_log_size":247,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":365188,"kb_used_data":11852,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124576636,"statfs":{"total":1151940427776,"available":1151566475264,"internally_reserved":0,"allocated":12136448,"data_stored":7716955,"data_compressed":318897,"data_compressed_allocated":430080,"data_compressed_original":2359296,"omap_allocated":19141,"internal_metadata":361674043},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":39261,"num_objects":28,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":29,"num_read_kb":60,"num_write":77,"num_write_kb":59,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":19,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003421"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.888099+0000","last_change":"2024-04-24T23:27:19.978807+0000","last_active":"2024-04-24T23:27:26.888099+0000","last_peered":"2024-04-24T23:27:26.888099+0000","last_clean":"2024-04-24T23:27:26.888099+0000","last_became_active":"2024-04-24T23:27:19.978436+0000","last_became_peered":"2024-04-24T23:27:19.978436+0000","last_unstale":"2024-04-24T23:27:26.888099+0000","last_undegraded":"2024-04-24T23:27:26.888099+0000","last_fullsized":"2024-04-24T23:27:26.888099+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:00:13.126518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.853244+0000","last_change":"2024-04-24T23:27:19.980327+0000","last_active":"2024-04-24T23:27:26.853244+0000","last_peered":"2024-04-24T23:27:26.853244+0000","last_clean":"2024-04-24T23:27:26.853244+0000","last_became_active":"2024-04-24T23:27:19.979912+0000","last_became_peered":"2024-04-24T23:27:19.979912+0000","last_unstale":"2024-04-24T23:27:26.853244+0000","last_undegraded":"2024-04-24T23:27:26.853244+0000","last_fullsized":"2024-04-24T23:27:26.853244+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:25:41.657205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.853033+0000","last_change":"2024-04-24T23:27:19.976451+0000","last_active":"2024-04-24T23:27:26.853033+0000","last_peered":"2024-04-24T23:27:26.853033+0000","last_clean":"2024-04-24T23:27:26.853033+0000","last_became_active":"2024-04-24T23:27:19.975976+0000","last_became_peered":"2024-04-24T23:27:19.975976+0000","last_unstale":"2024-04-24T23:27:26.853033+0000","last_undegraded":"2024-04-24T23:27:26.853033+0000","last_fullsized":"2024-04-24T23:27:26.853033+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:14:51.605217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.852139+0000","last_change":"2024-04-24T23:27:19.991245+0000","last_active":"2024-04-24T23:27:26.852139+0000","last_peered":"2024-04-24T23:27:26.852139+0000","last_clean":"2024-04-24T23:27:26.852139+0000","last_became_active":"2024-04-24T23:27:19.990448+0000","last_became_peered":"2024-04-24T23:27:19.990448+0000","last_unstale":"2024-04-24T23:27:26.852139+0000","last_undegraded":"2024-04-24T23:27:26.852139+0000","last_fullsized":"2024-04-24T23:27:26.852139+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:03:21.013005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.851092+0000","last_change":"2024-04-24T23:27:19.977664+0000","last_active":"2024-04-24T23:27:26.851092+0000","last_peered":"2024-04-24T23:27:26.851092+0000","last_clean":"2024-04-24T23:27:26.851092+0000","last_became_active":"2024-04-24T23:27:19.977300+0000","last_became_peered":"2024-04-24T23:27:19.977300+0000","last_unstale":"2024-04-24T23:27:26.851092+0000","last_undegraded":"2024-04-24T23:27:26.851092+0000","last_fullsized":"2024-04-24T23:27:26.851092+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:19:27.229712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.852801+0000","last_change":"2024-04-24T23:27:19.989994+0000","last_active":"2024-04-24T23:27:26.852801+0000","last_peered":"2024-04-24T23:27:26.852801+0000","last_clean":"2024-04-24T23:27:26.852801+0000","last_became_active":"2024-04-24T23:27:19.989639+0000","last_became_peered":"2024-04-24T23:27:19.989639+0000","last_unstale":"2024-04-24T23:27:26.852801+0000","last_undegraded":"2024-04-24T23:27:26.852801+0000","last_fullsized":"2024-04-24T23:27:26.852801+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:20:43.931086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.216561+0000","last_change":"2024-04-24T23:27:19.992886+0000","last_active":"2024-04-24T23:27:26.216561+0000","last_peered":"2024-04-24T23:27:26.216561+0000","last_clean":"2024-04-24T23:27:26.216561+0000","last_became_active":"2024-04-24T23:27:19.992438+0000","last_became_peered":"2024-04-24T23:27:19.992438+0000","last_unstale":"2024-04-24T23:27:26.216561+0000","last_undegraded":"2024-04-24T23:27:26.216561+0000","last_fullsized":"2024-04-24T23:27:26.216561+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:02:34.110984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.850659+0000","last_change":"2024-04-24T23:27:19.981592+0000","last_active":"2024-04-24T23:27:26.850659+0000","last_peered":"2024-04-24T23:27:26.850659+0000","last_clean":"2024-04-24T23:27:26.850659+0000","last_became_active":"2024-04-24T23:27:19.981248+0000","last_became_peered":"2024-04-24T23:27:19.981248+0000","last_unstale":"2024-04-24T23:27:26.850659+0000","last_undegraded":"2024-04-24T23:27:26.850659+0000","last_fullsized":"2024-04-24T23:27:26.850659+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:53:31.191318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.852494+0000","last_change":"2024-04-24T23:27:19.975995+0000","last_active":"2024-04-24T23:27:26.852494+0000","last_peered":"2024-04-24T23:27:26.852494+0000","last_clean":"2024-04-24T23:27:26.852494+0000","last_became_active":"2024-04-24T23:27:19.975614+0000","last_became_peered":"2024-04-24T23:27:19.975614+0000","last_unstale":"2024-04-24T23:27:26.852494+0000","last_undegraded":"2024-04-24T23:27:26.852494+0000","last_fullsized":"2024-04-24T23:27:26.852494+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:33:22.439998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.853019+0000","last_change":"2024-04-24T23:27:19.991265+0000","last_active":"2024-04-24T23:27:26.853019+0000","last_peered":"2024-04-24T23:27:26.853019+0000","last_clean":"2024-04-24T23:27:26.853019+0000","last_became_active":"2024-04-24T23:27:19.990928+0000","last_became_peered":"2024-04-24T23:27:19.990928+0000","last_unstale":"2024-04-24T23:27:26.853019+0000","last_undegraded":"2024-04-24T23:27:26.853019+0000","last_fullsized":"2024-04-24T23:27:26.853019+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:20:41.244669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.217102+0000","last_change":"2024-04-24T23:27:19.985182+0000","last_active":"2024-04-24T23:27:26.217102+0000","last_peered":"2024-04-24T23:27:26.217102+0000","last_clean":"2024-04-24T23:27:26.217102+0000","last_became_active":"2024-04-24T23:27:19.984600+0000","last_became_peered":"2024-04-24T23:27:19.984600+0000","last_unstale":"2024-04-24T23:27:26.217102+0000","last_undegraded":"2024-04-24T23:27:26.217102+0000","last_fullsized":"2024-04-24T23:27:26.217102+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:16:09.919736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.851979+0000","last_change":"2024-04-24T23:27:19.994271+0000","last_active":"2024-04-24T23:27:26.851979+0000","last_peered":"2024-04-24T23:27:26.851979+0000","last_clean":"2024-04-24T23:27:26.851979+0000","last_became_active":"2024-04-24T23:27:19.993926+0000","last_became_peered":"2024-04-24T23:27:19.993926+0000","last_unstale":"2024-04-24T23:27:26.851979+0000","last_undegraded":"2024-04-24T23:27:26.851979+0000","last_fullsized":"2024-04-24T23:27:26.851979+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:49:07.500370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.215566+0000","last_change":"2024-04-24T23:27:19.984230+0000","last_active":"2024-04-24T23:27:26.215566+0000","last_peered":"2024-04-24T23:27:26.215566+0000","last_clean":"2024-04-24T23:27:26.215566+0000","last_became_active":"2024-04-24T23:27:19.983902+0000","last_became_peered":"2024-04-24T23:27:19.983902+0000","last_unstale":"2024-04-24T23:27:26.215566+0000","last_undegraded":"2024-04-24T23:27:26.215566+0000","last_fullsized":"2024-04-24T23:27:26.215566+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:12:40.985092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.852594+0000","last_change":"2024-04-24T23:27:19.979822+0000","last_active":"2024-04-24T23:27:26.852594+0000","last_peered":"2024-04-24T23:27:26.852594+0000","last_clean":"2024-04-24T23:27:26.852594+0000","last_became_active":"2024-04-24T23:27:19.979452+0000","last_became_peered":"2024-04-24T23:27:19.979452+0000","last_unstale":"2024-04-24T23:27:26.852594+0000","last_undegraded":"2024-04-24T23:27:26.852594+0000","last_fullsized":"2024-04-24T23:27:26.852594+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:08:53.945221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.852737+0000","last_change":"2024-04-24T23:27:19.986313+0000","last_active":"2024-04-24T23:27:26.852737+0000","last_peered":"2024-04-24T23:27:26.852737+0000","last_clean":"2024-04-24T23:27:26.852737+0000","last_became_active":"2024-04-24T23:27:19.985947+0000","last_became_peered":"2024-04-24T23:27:19.985947+0000","last_unstale":"2024-04-24T23:27:26.852737+0000","last_undegraded":"2024-04-24T23:27:26.852737+0000","last_fullsized":"2024-04-24T23:27:26.852737+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:32:10.557949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.851170+0000","last_change":"2024-04-24T23:27:19.984346+0000","last_active":"2024-04-24T23:27:26.851170+0000","last_peered":"2024-04-24T23:27:26.851170+0000","last_clean":"2024-04-24T23:27:26.851170+0000","last_became_active":"2024-04-24T23:27:19.983999+0000","last_became_peered":"2024-04-24T23:27:19.983999+0000","last_unstale":"2024-04-24T23:27:26.851170+0000","last_undegraded":"2024-04-24T23:27:26.851170+0000","last_fullsized":"2024-04-24T23:27:26.851170+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:15:35.329454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.887975+0000","last_change":"2024-04-24T23:27:19.979939+0000","last_active":"2024-04-24T23:27:26.887975+0000","last_peered":"2024-04-24T23:27:26.887975+0000","last_clean":"2024-04-24T23:27:26.887975+0000","last_became_active":"2024-04-24T23:27:19.979600+0000","last_became_peered":"2024-04-24T23:27:19.979600+0000","last_unstale":"2024-04-24T23:27:26.887975+0000","last_undegraded":"2024-04-24T23:27:26.887975+0000","last_fullsized":"2024-04-24T23:27:26.887975+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:37:52.250143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.852978+0000","last_change":"2024-04-24T23:27:19.984797+0000","last_active":"2024-04-24T23:27:26.852978+0000","last_peered":"2024-04-24T23:27:26.852978+0000","last_clean":"2024-04-24T23:27:26.852978+0000","last_became_active":"2024-04-24T23:27:19.983881+0000","last_became_peered":"2024-04-24T23:27:19.983881+0000","last_unstale":"2024-04-24T23:27:26.852978+0000","last_undegraded":"2024-04-24T23:27:26.852978+0000","last_fullsized":"2024-04-24T23:27:26.852978+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:17:24.630821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.283765+0000","last_change":"2024-04-24T23:27:19.977558+0000","last_active":"2024-04-24T23:27:26.283765+0000","last_peered":"2024-04-24T23:27:26.283765+0000","last_clean":"2024-04-24T23:27:26.283765+0000","last_became_active":"2024-04-24T23:27:19.977220+0000","last_became_peered":"2024-04-24T23:27:19.977220+0000","last_unstale":"2024-04-24T23:27:26.283765+0000","last_undegraded":"2024-04-24T23:27:26.283765+0000","last_fullsized":"2024-04-24T23:27:26.283765+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:38:49.615455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.283511+0000","last_change":"2024-04-24T23:27:24.234304+0000","last_active":"2024-04-24T23:27:26.283511+0000","last_peered":"2024-04-24T23:27:26.283511+0000","last_clean":"2024-04-24T23:27:26.283511+0000","last_became_active":"2024-04-24T23:27:24.233865+0000","last_became_peered":"2024-04-24T23:27:24.233865+0000","last_unstale":"2024-04-24T23:27:26.283511+0000","last_undegraded":"2024-04-24T23:27:26.283511+0000","last_fullsized":"2024-04-24T23:27:26.283511+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:46:20.853060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'9","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:44.988037+0000","last_change":"2024-04-24T23:27:18.970613+0000","last_active":"2024-04-24T23:27:44.988037+0000","last_peered":"2024-04-24T23:27:44.988037+0000","last_clean":"2024-04-24T23:27:44.988037+0000","last_became_active":"2024-04-24T23:27:18.970173+0000","last_became_peered":"2024-04-24T23:27:18.970173+0000","last_unstale":"2024-04-24T23:27:44.988037+0000","last_undegraded":"2024-04-24T23:27:44.988037+0000","last_fullsized":"2024-04-24T23:27:44.988037+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:12:35.187309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.851565+0000","last_change":"2024-04-24T23:27:19.981571+0000","last_active":"2024-04-24T23:27:26.851565+0000","last_peered":"2024-04-24T23:27:26.851565+0000","last_clean":"2024-04-24T23:27:26.851565+0000","last_became_active":"2024-04-24T23:27:19.981007+0000","last_became_peered":"2024-04-24T23:27:19.981007+0000","last_unstale":"2024-04-24T23:27:26.851565+0000","last_undegraded":"2024-04-24T23:27:26.851565+0000","last_fullsized":"2024-04-24T23:27:26.851565+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:38:30.023755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.851355+0000","last_change":"2024-04-24T23:27:24.234134+0000","last_active":"2024-04-24T23:27:26.851355+0000","last_peered":"2024-04-24T23:27:26.851355+0000","last_clean":"2024-04-24T23:27:26.851355+0000","last_became_active":"2024-04-24T23:27:24.233650+0000","last_became_peered":"2024-04-24T23:27:24.233650+0000","last_unstale":"2024-04-24T23:27:26.851355+0000","last_undegraded":"2024-04-24T23:27:26.851355+0000","last_fullsized":"2024-04-24T23:27:26.851355+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:04:54.191186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.851973+0000","last_change":"2024-04-24T23:27:18.969572+0000","last_active":"2024-04-24T23:27:26.851973+0000","last_peered":"2024-04-24T23:27:26.851973+0000","last_clean":"2024-04-24T23:27:26.851973+0000","last_became_active":"2024-04-24T23:27:18.968897+0000","last_became_peered":"2024-04-24T23:27:18.968897+0000","last_unstale":"2024-04-24T23:27:26.851973+0000","last_undegraded":"2024-04-24T23:27:26.851973+0000","last_fullsized":"2024-04-24T23:27:26.851973+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:42:51.380386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.852983+0000","last_change":"2024-04-24T23:27:19.977290+0000","last_active":"2024-04-24T23:27:26.852983+0000","last_peered":"2024-04-24T23:27:26.852983+0000","last_clean":"2024-04-24T23:27:26.852983+0000","last_became_active":"2024-04-24T23:27:19.976802+0000","last_became_peered":"2024-04-24T23:27:19.976802+0000","last_unstale":"2024-04-24T23:27:26.852983+0000","last_undegraded":"2024-04-24T23:27:26.852983+0000","last_fullsized":"2024-04-24T23:27:26.852983+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:55:15.827279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.851496+0000","last_change":"2024-04-24T23:27:24.235706+0000","last_active":"2024-04-24T23:27:26.851496+0000","last_peered":"2024-04-24T23:27:26.851496+0000","last_clean":"2024-04-24T23:27:26.851496+0000","last_became_active":"2024-04-24T23:27:24.235367+0000","last_became_peered":"2024-04-24T23:27:24.235367+0000","last_unstale":"2024-04-24T23:27:26.851496+0000","last_undegraded":"2024-04-24T23:27:26.851496+0000","last_fullsized":"2024-04-24T23:27:26.851496+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:17.444845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:44.988135+0000","last_change":"2024-04-24T23:27:18.974428+0000","last_active":"2024-04-24T23:27:44.988135+0000","last_peered":"2024-04-24T23:27:44.988135+0000","last_clean":"2024-04-24T23:27:44.988135+0000","last_became_active":"2024-04-24T23:27:18.974100+0000","last_became_peered":"2024-04-24T23:27:18.974100+0000","last_unstale":"2024-04-24T23:27:44.988135+0000","last_undegraded":"2024-04-24T23:27:44.988135+0000","last_fullsized":"2024-04-24T23:27:44.988135+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:55:36.070904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.852155+0000","last_change":"2024-04-24T23:27:19.975222+0000","last_active":"2024-04-24T23:27:26.852155+0000","last_peered":"2024-04-24T23:27:26.852155+0000","last_clean":"2024-04-24T23:27:26.852155+0000","last_became_active":"2024-04-24T23:27:19.974881+0000","last_became_peered":"2024-04-24T23:27:19.974881+0000","last_unstale":"2024-04-24T23:27:26.852155+0000","last_undegraded":"2024-04-24T23:27:26.852155+0000","last_fullsized":"2024-04-24T23:27:26.852155+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:16:17.884748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.851380+0000","last_change":"2024-04-24T23:27:24.232445+0000","last_active":"2024-04-24T23:27:26.851380+0000","last_peered":"2024-04-24T23:27:26.851380+0000","last_clean":"2024-04-24T23:27:26.851380+0000","last_became_active":"2024-04-24T23:27:24.231754+0000","last_became_peered":"2024-04-24T23:27:24.231754+0000","last_unstale":"2024-04-24T23:27:26.851380+0000","last_undegraded":"2024-04-24T23:27:26.851380+0000","last_fullsized":"2024-04-24T23:27:26.851380+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:45:43.630508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.851160+0000","last_change":"2024-04-24T23:27:18.966558+0000","last_active":"2024-04-24T23:27:26.851160+0000","last_peered":"2024-04-24T23:27:26.851160+0000","last_clean":"2024-04-24T23:27:26.851160+0000","last_became_active":"2024-04-24T23:27:18.966212+0000","last_became_peered":"2024-04-24T23:27:18.966212+0000","last_unstale":"2024-04-24T23:27:26.851160+0000","last_undegraded":"2024-04-24T23:27:26.851160+0000","last_fullsized":"2024-04-24T23:27:26.851160+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:24:17.262587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.216910+0000","last_change":"2024-04-24T23:27:19.976811+0000","last_active":"2024-04-24T23:27:26.216910+0000","last_peered":"2024-04-24T23:27:26.216910+0000","last_clean":"2024-04-24T23:27:26.216910+0000","last_became_active":"2024-04-24T23:27:19.976456+0000","last_became_peered":"2024-04-24T23:27:19.976456+0000","last_unstale":"2024-04-24T23:27:26.216910+0000","last_undegraded":"2024-04-24T23:27:26.216910+0000","last_fullsized":"2024-04-24T23:27:26.216910+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:43:13.755833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.283753+0000","last_change":"2024-04-24T23:27:24.232136+0000","last_active":"2024-04-24T23:27:26.283753+0000","last_peered":"2024-04-24T23:27:26.283753+0000","last_clean":"2024-04-24T23:27:26.283753+0000","last_became_active":"2024-04-24T23:27:24.231459+0000","last_became_peered":"2024-04-24T23:27:24.231459+0000","last_unstale":"2024-04-24T23:27:26.283753+0000","last_undegraded":"2024-04-24T23:27:26.283753+0000","last_fullsized":"2024-04-24T23:27:26.283753+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:51:51.114787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'6","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:46.995908+0000","last_change":"2024-04-24T23:27:18.975333+0000","last_active":"2024-04-24T23:27:46.995908+0000","last_peered":"2024-04-24T23:27:46.995908+0000","last_clean":"2024-04-24T23:27:46.995908+0000","last_became_active":"2024-04-24T23:27:18.974869+0000","last_became_peered":"2024-04-24T23:27:18.974869+0000","last_unstale":"2024-04-24T23:27:46.995908+0000","last_undegraded":"2024-04-24T23:27:46.995908+0000","last_fullsized":"2024-04-24T23:27:46.995908+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:30:26.758596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.851098+0000","last_change":"2024-04-24T23:27:19.974989+0000","last_active":"2024-04-24T23:27:26.851098+0000","last_peered":"2024-04-24T23:27:26.851098+0000","last_clean":"2024-04-24T23:27:26.851098+0000","last_became_active":"2024-04-24T23:27:19.974312+0000","last_became_peered":"2024-04-24T23:27:19.974312+0000","last_unstale":"2024-04-24T23:27:26.851098+0000","last_undegraded":"2024-04-24T23:27:26.851098+0000","last_fullsized":"2024-04-24T23:27:26.851098+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:16:39.685430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.283739+0000","last_change":"2024-04-24T23:27:24.229404+0000","last_active":"2024-04-24T23:27:26.283739+0000","last_peered":"2024-04-24T23:27:26.283739+0000","last_clean":"2024-04-24T23:27:26.283739+0000","last_became_active":"2024-04-24T23:27:24.228905+0000","last_became_peered":"2024-04-24T23:27:24.228905+0000","last_unstale":"2024-04-24T23:27:26.283739+0000","last_undegraded":"2024-04-24T23:27:26.283739+0000","last_fullsized":"2024-04-24T23:27:26.283739+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:59:25.069173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:44.987908+0000","last_change":"2024-04-24T23:27:18.971313+0000","last_active":"2024-04-24T23:27:44.987908+0000","last_peered":"2024-04-24T23:27:44.987908+0000","last_clean":"2024-04-24T23:27:44.987908+0000","last_became_active":"2024-04-24T23:27:18.970888+0000","last_became_peered":"2024-04-24T23:27:18.970888+0000","last_unstale":"2024-04-24T23:27:44.987908+0000","last_undegraded":"2024-04-24T23:27:44.987908+0000","last_fullsized":"2024-04-24T23:27:44.987908+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:34:20.118693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.216658+0000","last_change":"2024-04-24T23:27:19.978939+0000","last_active":"2024-04-24T23:27:26.216658+0000","last_peered":"2024-04-24T23:27:26.216658+0000","last_clean":"2024-04-24T23:27:26.216658+0000","last_became_active":"2024-04-24T23:27:19.978556+0000","last_became_peered":"2024-04-24T23:27:19.978556+0000","last_unstale":"2024-04-24T23:27:26.216658+0000","last_undegraded":"2024-04-24T23:27:26.216658+0000","last_fullsized":"2024-04-24T23:27:26.216658+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:28:43.118561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.887964+0000","last_change":"2024-04-24T23:27:24.232384+0000","last_active":"2024-04-24T23:27:26.887964+0000","last_peered":"2024-04-24T23:27:26.887964+0000","last_clean":"2024-04-24T23:27:26.887964+0000","last_became_active":"2024-04-24T23:27:24.231497+0000","last_became_peered":"2024-04-24T23:27:24.231497+0000","last_unstale":"2024-04-24T23:27:26.887964+0000","last_undegraded":"2024-04-24T23:27:26.887964+0000","last_fullsized":"2024-04-24T23:27:26.887964+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:26:54.716821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.215385+0000","last_change":"2024-04-24T23:27:18.968949+0000","last_active":"2024-04-24T23:27:26.215385+0000","last_peered":"2024-04-24T23:27:26.215385+0000","last_clean":"2024-04-24T23:27:26.215385+0000","last_became_active":"2024-04-24T23:27:18.968612+0000","last_became_peered":"2024-04-24T23:27:18.968612+0000","last_unstale":"2024-04-24T23:27:26.215385+0000","last_undegraded":"2024-04-24T23:27:26.215385+0000","last_fullsized":"2024-04-24T23:27:26.215385+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:03:45.908080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.851074+0000","last_change":"2024-04-24T23:27:19.979670+0000","last_active":"2024-04-24T23:27:26.851074+0000","last_peered":"2024-04-24T23:27:26.851074+0000","last_clean":"2024-04-24T23:27:26.851074+0000","last_became_active":"2024-04-24T23:27:19.979277+0000","last_became_peered":"2024-04-24T23:27:19.979277+0000","last_unstale":"2024-04-24T23:27:26.851074+0000","last_undegraded":"2024-04-24T23:27:26.851074+0000","last_fullsized":"2024-04-24T23:27:26.851074+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:05:54.958747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.853095+0000","last_change":"2024-04-24T23:27:24.237085+0000","last_active":"2024-04-24T23:27:26.853095+0000","last_peered":"2024-04-24T23:27:26.853095+0000","last_clean":"2024-04-24T23:27:26.853095+0000","last_became_active":"2024-04-24T23:27:24.236671+0000","last_became_peered":"2024-04-24T23:27:24.236671+0000","last_unstale":"2024-04-24T23:27:26.853095+0000","last_undegraded":"2024-04-24T23:27:26.853095+0000","last_fullsized":"2024-04-24T23:27:26.853095+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:48:38.482374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.216398+0000","last_change":"2024-04-24T23:27:18.974146+0000","last_active":"2024-04-24T23:27:26.216398+0000","last_peered":"2024-04-24T23:27:26.216398+0000","last_clean":"2024-04-24T23:27:26.216398+0000","last_became_active":"2024-04-24T23:27:18.973814+0000","last_became_peered":"2024-04-24T23:27:18.973814+0000","last_unstale":"2024-04-24T23:27:26.216398+0000","last_undegraded":"2024-04-24T23:27:26.216398+0000","last_fullsized":"2024-04-24T23:27:26.216398+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:22:52.409448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.851753+0000","last_change":"2024-04-24T23:27:19.984325+0000","last_active":"2024-04-24T23:27:26.851753+0000","last_peered":"2024-04-24T23:27:26.851753+0000","last_clean":"2024-04-24T23:27:26.851753+0000","last_became_active":"2024-04-24T23:27:19.983918+0000","last_became_peered":"2024-04-24T23:27:19.983918+0000","last_unstale":"2024-04-24T23:27:26.851753+0000","last_undegraded":"2024-04-24T23:27:26.851753+0000","last_fullsized":"2024-04-24T23:27:26.851753+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:24:07.640983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.851460+0000","last_change":"2024-04-24T23:27:24.223561+0000","last_active":"2024-04-24T23:27:26.851460+0000","last_peered":"2024-04-24T23:27:26.851460+0000","last_clean":"2024-04-24T23:27:26.851460+0000","last_became_active":"2024-04-24T23:27:24.223127+0000","last_became_peered":"2024-04-24T23:27:24.223127+0000","last_unstale":"2024-04-24T23:27:26.851460+0000","last_undegraded":"2024-04-24T23:27:26.851460+0000","last_fullsized":"2024-04-24T23:27:26.851460+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:31:28.079633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"90'3","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:46.994272+0000","last_change":"2024-04-24T23:27:18.971374+0000","last_active":"2024-04-24T23:27:46.994272+0000","last_peered":"2024-04-24T23:27:46.994272+0000","last_clean":"2024-04-24T23:27:46.994272+0000","last_became_active":"2024-04-24T23:27:18.970818+0000","last_became_peered":"2024-04-24T23:27:18.970818+0000","last_unstale":"2024-04-24T23:27:46.994272+0000","last_undegraded":"2024-04-24T23:27:46.994272+0000","last_fullsized":"2024-04-24T23:27:46.994272+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:12:17.547968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.852494+0000","last_change":"2024-04-24T23:27:19.980275+0000","last_active":"2024-04-24T23:27:26.852494+0000","last_peered":"2024-04-24T23:27:26.852494+0000","last_clean":"2024-04-24T23:27:26.852494+0000","last_became_active":"2024-04-24T23:27:19.979925+0000","last_became_peered":"2024-04-24T23:27:19.979925+0000","last_unstale":"2024-04-24T23:27:26.852494+0000","last_undegraded":"2024-04-24T23:27:26.852494+0000","last_fullsized":"2024-04-24T23:27:26.852494+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:21:10.681169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.851405+0000","last_change":"2024-04-24T23:27:24.231389+0000","last_active":"2024-04-24T23:27:26.851405+0000","last_peered":"2024-04-24T23:27:26.851405+0000","last_clean":"2024-04-24T23:27:26.851405+0000","last_became_active":"2024-04-24T23:27:24.230973+0000","last_became_peered":"2024-04-24T23:27:24.230973+0000","last_unstale":"2024-04-24T23:27:26.851405+0000","last_undegraded":"2024-04-24T23:27:26.851405+0000","last_fullsized":"2024-04-24T23:27:26.851405+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:35.943122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:43.975849+0000","last_change":"2024-04-24T23:27:18.977009+0000","last_active":"2024-04-24T23:27:43.975849+0000","last_peered":"2024-04-24T23:27:43.975849+0000","last_clean":"2024-04-24T23:27:43.975849+0000","last_became_active":"2024-04-24T23:27:18.976667+0000","last_became_peered":"2024-04-24T23:27:18.976667+0000","last_unstale":"2024-04-24T23:27:43.975849+0000","last_undegraded":"2024-04-24T23:27:43.975849+0000","last_fullsized":"2024-04-24T23:27:43.975849+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:19:47.817961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.853087+0000","last_change":"2024-04-24T23:27:19.988589+0000","last_active":"2024-04-24T23:27:26.853087+0000","last_peered":"2024-04-24T23:27:26.853087+0000","last_clean":"2024-04-24T23:27:26.853087+0000","last_became_active":"2024-04-24T23:27:19.988261+0000","last_became_peered":"2024-04-24T23:27:19.988261+0000","last_unstale":"2024-04-24T23:27:26.853087+0000","last_undegraded":"2024-04-24T23:27:26.853087+0000","last_fullsized":"2024-04-24T23:27:26.853087+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:44:15.809776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.852146+0000","last_change":"2024-04-24T23:27:24.231256+0000","last_active":"2024-04-24T23:27:26.852146+0000","last_peered":"2024-04-24T23:27:26.852146+0000","last_clean":"2024-04-24T23:27:26.852146+0000","last_became_active":"2024-04-24T23:27:24.230545+0000","last_became_peered":"2024-04-24T23:27:24.230545+0000","last_unstale":"2024-04-24T23:27:26.852146+0000","last_undegraded":"2024-04-24T23:27:26.852146+0000","last_fullsized":"2024-04-24T23:27:26.852146+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:43:40.460473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:43.975011+0000","last_change":"2024-04-24T23:27:18.968805+0000","last_active":"2024-04-24T23:27:43.975011+0000","last_peered":"2024-04-24T23:27:43.975011+0000","last_clean":"2024-04-24T23:27:43.975011+0000","last_became_active":"2024-04-24T23:27:18.968423+0000","last_became_peered":"2024-04-24T23:27:18.968423+0000","last_unstale":"2024-04-24T23:27:43.975011+0000","last_undegraded":"2024-04-24T23:27:43.975011+0000","last_fullsized":"2024-04-24T23:27:43.975011+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:14:04.077773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.852319+0000","last_change":"2024-04-24T23:27:19.981986+0000","last_active":"2024-04-24T23:27:26.852319+0000","last_peered":"2024-04-24T23:27:26.852319+0000","last_clean":"2024-04-24T23:27:26.852319+0000","last_became_active":"2024-04-24T23:27:19.981508+0000","last_became_peered":"2024-04-24T23:27:19.981508+0000","last_unstale":"2024-04-24T23:27:26.852319+0000","last_undegraded":"2024-04-24T23:27:26.852319+0000","last_fullsized":"2024-04-24T23:27:26.852319+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:59:21.707481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.852123+0000","last_change":"2024-04-24T23:27:24.225481+0000","last_active":"2024-04-24T23:27:26.852123+0000","last_peered":"2024-04-24T23:27:26.852123+0000","last_clean":"2024-04-24T23:27:26.852123+0000","last_became_active":"2024-04-24T23:27:24.224363+0000","last_became_peered":"2024-04-24T23:27:24.224363+0000","last_unstale":"2024-04-24T23:27:26.852123+0000","last_undegraded":"2024-04-24T23:27:26.852123+0000","last_fullsized":"2024-04-24T23:27:26.852123+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:36:17.715596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"90'23","reported_seq":58,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:47.999206+0000","last_change":"2024-04-24T23:27:18.968264+0000","last_active":"2024-04-24T23:27:47.999206+0000","last_peered":"2024-04-24T23:27:47.999206+0000","last_clean":"2024-04-24T23:27:47.999206+0000","last_became_active":"2024-04-24T23:27:18.967856+0000","last_became_peered":"2024-04-24T23:27:18.967856+0000","last_unstale":"2024-04-24T23:27:47.999206+0000","last_undegraded":"2024-04-24T23:27:47.999206+0000","last_fullsized":"2024-04-24T23:27:47.999206+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:48:44.347254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40471,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":10,"num_read_kb":44,"num_write":24,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.851422+0000","last_change":"2024-04-24T23:27:19.984719+0000","last_active":"2024-04-24T23:27:26.851422+0000","last_peered":"2024-04-24T23:27:26.851422+0000","last_clean":"2024-04-24T23:27:26.851422+0000","last_became_active":"2024-04-24T23:27:19.984386+0000","last_became_peered":"2024-04-24T23:27:19.984386+0000","last_unstale":"2024-04-24T23:27:26.851422+0000","last_undegraded":"2024-04-24T23:27:26.851422+0000","last_fullsized":"2024-04-24T23:27:26.851422+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:02:13.739759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.217239+0000","last_change":"2024-04-24T23:27:24.230795+0000","last_active":"2024-04-24T23:27:26.217239+0000","last_peered":"2024-04-24T23:27:26.217239+0000","last_clean":"2024-04-24T23:27:26.217239+0000","last_became_active":"2024-04-24T23:27:24.230360+0000","last_became_peered":"2024-04-24T23:27:24.230360+0000","last_unstale":"2024-04-24T23:27:26.217239+0000","last_undegraded":"2024-04-24T23:27:26.217239+0000","last_fullsized":"2024-04-24T23:27:26.217239+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:51:24.875668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:44.987562+0000","last_change":"2024-04-24T23:27:18.964964+0000","last_active":"2024-04-24T23:27:44.987562+0000","last_peered":"2024-04-24T23:27:44.987562+0000","last_clean":"2024-04-24T23:27:44.987562+0000","last_became_active":"2024-04-24T23:27:18.964632+0000","last_became_peered":"2024-04-24T23:27:18.964632+0000","last_unstale":"2024-04-24T23:27:44.987562+0000","last_undegraded":"2024-04-24T23:27:44.987562+0000","last_fullsized":"2024-04-24T23:27:44.987562+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:52:23.144629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.215746+0000","last_change":"2024-04-24T23:27:19.987258+0000","last_active":"2024-04-24T23:27:26.215746+0000","last_peered":"2024-04-24T23:27:26.215746+0000","last_clean":"2024-04-24T23:27:26.215746+0000","last_became_active":"2024-04-24T23:27:19.986864+0000","last_became_peered":"2024-04-24T23:27:19.986864+0000","last_unstale":"2024-04-24T23:27:26.215746+0000","last_undegraded":"2024-04-24T23:27:26.215746+0000","last_fullsized":"2024-04-24T23:27:26.215746+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:57:40.503506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.887768+0000","last_change":"2024-04-24T23:27:24.223758+0000","last_active":"2024-04-24T23:27:26.887768+0000","last_peered":"2024-04-24T23:27:26.887768+0000","last_clean":"2024-04-24T23:27:26.887768+0000","last_became_active":"2024-04-24T23:27:24.223343+0000","last_became_peered":"2024-04-24T23:27:24.223343+0000","last_unstale":"2024-04-24T23:27:26.887768+0000","last_undegraded":"2024-04-24T23:27:26.887768+0000","last_fullsized":"2024-04-24T23:27:26.887768+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:41:52.620132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:43.975855+0000","last_change":"2024-04-24T23:27:18.974229+0000","last_active":"2024-04-24T23:27:43.975855+0000","last_peered":"2024-04-24T23:27:43.975855+0000","last_clean":"2024-04-24T23:27:43.975855+0000","last_became_active":"2024-04-24T23:27:18.973893+0000","last_became_peered":"2024-04-24T23:27:18.973893+0000","last_unstale":"2024-04-24T23:27:43.975855+0000","last_undegraded":"2024-04-24T23:27:43.975855+0000","last_fullsized":"2024-04-24T23:27:43.975855+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:41:45.699709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.217043+0000","last_change":"2024-04-24T23:27:19.986910+0000","last_active":"2024-04-24T23:27:26.217043+0000","last_peered":"2024-04-24T23:27:26.217043+0000","last_clean":"2024-04-24T23:27:26.217043+0000","last_became_active":"2024-04-24T23:27:19.986556+0000","last_became_peered":"2024-04-24T23:27:19.986556+0000","last_unstale":"2024-04-24T23:27:26.217043+0000","last_undegraded":"2024-04-24T23:27:26.217043+0000","last_fullsized":"2024-04-24T23:27:26.217043+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:06:28.506556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.852652+0000","last_change":"2024-04-24T23:27:24.233115+0000","last_active":"2024-04-24T23:27:26.852652+0000","last_peered":"2024-04-24T23:27:26.852652+0000","last_clean":"2024-04-24T23:27:26.852652+0000","last_became_active":"2024-04-24T23:27:24.232714+0000","last_became_peered":"2024-04-24T23:27:24.232714+0000","last_unstale":"2024-04-24T23:27:26.852652+0000","last_undegraded":"2024-04-24T23:27:26.852652+0000","last_fullsized":"2024-04-24T23:27:26.852652+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:04.365515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.851468+0000","last_change":"2024-04-24T23:27:18.966426+0000","last_active":"2024-04-24T23:27:26.851468+0000","last_peered":"2024-04-24T23:27:26.851468+0000","last_clean":"2024-04-24T23:27:26.851468+0000","last_became_active":"2024-04-24T23:27:18.966070+0000","last_became_peered":"2024-04-24T23:27:18.966070+0000","last_unstale":"2024-04-24T23:27:26.851468+0000","last_undegraded":"2024-04-24T23:27:26.851468+0000","last_fullsized":"2024-04-24T23:27:26.851468+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:56:21.679639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.216614+0000","last_change":"2024-04-24T23:27:19.991829+0000","last_active":"2024-04-24T23:27:26.216614+0000","last_peered":"2024-04-24T23:27:26.216614+0000","last_clean":"2024-04-24T23:27:26.216614+0000","last_became_active":"2024-04-24T23:27:19.991492+0000","last_became_peered":"2024-04-24T23:27:19.991492+0000","last_unstale":"2024-04-24T23:27:26.216614+0000","last_undegraded":"2024-04-24T23:27:26.216614+0000","last_fullsized":"2024-04-24T23:27:26.216614+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:23:45.195217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.216307+0000","last_change":"2024-04-24T23:27:24.229794+0000","last_active":"2024-04-24T23:27:26.216307+0000","last_peered":"2024-04-24T23:27:26.216307+0000","last_clean":"2024-04-24T23:27:26.216307+0000","last_became_active":"2024-04-24T23:27:24.229410+0000","last_became_peered":"2024-04-24T23:27:24.229410+0000","last_unstale":"2024-04-24T23:27:26.216307+0000","last_undegraded":"2024-04-24T23:27:26.216307+0000","last_fullsized":"2024-04-24T23:27:26.216307+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:00:53.188307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:44.988890+0000","last_change":"2024-04-24T23:27:18.969071+0000","last_active":"2024-04-24T23:27:44.988890+0000","last_peered":"2024-04-24T23:27:44.988890+0000","last_clean":"2024-04-24T23:27:44.988890+0000","last_became_active":"2024-04-24T23:27:18.968671+0000","last_became_peered":"2024-04-24T23:27:18.968671+0000","last_unstale":"2024-04-24T23:27:44.988890+0000","last_undegraded":"2024-04-24T23:27:44.988890+0000","last_fullsized":"2024-04-24T23:27:44.988890+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:50:23.743556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.216391+0000","last_change":"2024-04-24T23:27:19.982608+0000","last_active":"2024-04-24T23:27:26.216391+0000","last_peered":"2024-04-24T23:27:26.216391+0000","last_clean":"2024-04-24T23:27:26.216391+0000","last_became_active":"2024-04-24T23:27:19.982270+0000","last_became_peered":"2024-04-24T23:27:19.982270+0000","last_unstale":"2024-04-24T23:27:26.216391+0000","last_undegraded":"2024-04-24T23:27:26.216391+0000","last_fullsized":"2024-04-24T23:27:26.216391+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:01:10.525164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.216378+0000","last_change":"2024-04-24T23:27:24.236143+0000","last_active":"2024-04-24T23:27:26.216378+0000","last_peered":"2024-04-24T23:27:26.216378+0000","last_clean":"2024-04-24T23:27:26.216378+0000","last_became_active":"2024-04-24T23:27:24.235724+0000","last_became_peered":"2024-04-24T23:27:24.235724+0000","last_unstale":"2024-04-24T23:27:26.216378+0000","last_undegraded":"2024-04-24T23:27:26.216378+0000","last_fullsized":"2024-04-24T23:27:26.216378+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:55:41.317044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:45.983035+0000","last_change":"2024-04-24T23:27:18.971312+0000","last_active":"2024-04-24T23:27:45.983035+0000","last_peered":"2024-04-24T23:27:45.983035+0000","last_clean":"2024-04-24T23:27:45.983035+0000","last_became_active":"2024-04-24T23:27:18.970929+0000","last_became_peered":"2024-04-24T23:27:18.970929+0000","last_unstale":"2024-04-24T23:27:45.983035+0000","last_undegraded":"2024-04-24T23:27:45.983035+0000","last_fullsized":"2024-04-24T23:27:45.983035+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:54:20.318916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.887848+0000","last_change":"2024-04-24T23:27:19.981169+0000","last_active":"2024-04-24T23:27:26.887848+0000","last_peered":"2024-04-24T23:27:26.887848+0000","last_clean":"2024-04-24T23:27:26.887848+0000","last_became_active":"2024-04-24T23:27:19.980820+0000","last_became_peered":"2024-04-24T23:27:19.980820+0000","last_unstale":"2024-04-24T23:27:26.887848+0000","last_undegraded":"2024-04-24T23:27:26.887848+0000","last_fullsized":"2024-04-24T23:27:26.887848+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:51:00.464488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.851121+0000","last_change":"2024-04-24T23:27:24.718948+0000","last_active":"2024-04-24T23:27:26.851121+0000","last_peered":"2024-04-24T23:27:26.851121+0000","last_clean":"2024-04-24T23:27:26.851121+0000","last_became_active":"2024-04-24T23:27:24.718424+0000","last_became_peered":"2024-04-24T23:27:24.718424+0000","last_unstale":"2024-04-24T23:27:26.851121+0000","last_undegraded":"2024-04-24T23:27:26.851121+0000","last_fullsized":"2024-04-24T23:27:26.851121+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:22:01.984976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.852462+0000","last_change":"2024-04-24T23:27:18.957073+0000","last_active":"2024-04-24T23:27:26.852462+0000","last_peered":"2024-04-24T23:27:26.852462+0000","last_clean":"2024-04-24T23:27:26.852462+0000","last_became_active":"2024-04-24T23:27:18.956713+0000","last_became_peered":"2024-04-24T23:27:18.956713+0000","last_unstale":"2024-04-24T23:27:26.852462+0000","last_undegraded":"2024-04-24T23:27:26.852462+0000","last_fullsized":"2024-04-24T23:27:26.852462+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:16:21.015902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.852990+0000","last_change":"2024-04-24T23:27:19.985058+0000","last_active":"2024-04-24T23:27:26.852990+0000","last_peered":"2024-04-24T23:27:26.852990+0000","last_clean":"2024-04-24T23:27:26.852990+0000","last_became_active":"2024-04-24T23:27:19.984675+0000","last_became_peered":"2024-04-24T23:27:19.984675+0000","last_unstale":"2024-04-24T23:27:26.852990+0000","last_undegraded":"2024-04-24T23:27:26.852990+0000","last_fullsized":"2024-04-24T23:27:26.852990+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:22:35.252839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.215919+0000","last_change":"2024-04-24T23:27:24.236117+0000","last_active":"2024-04-24T23:27:26.215919+0000","last_peered":"2024-04-24T23:27:26.215919+0000","last_clean":"2024-04-24T23:27:26.215919+0000","last_became_active":"2024-04-24T23:27:24.235720+0000","last_became_peered":"2024-04-24T23:27:24.235720+0000","last_unstale":"2024-04-24T23:27:26.215919+0000","last_undegraded":"2024-04-24T23:27:26.215919+0000","last_fullsized":"2024-04-24T23:27:26.215919+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:46:40.604988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.216942+0000","last_change":"2024-04-24T23:27:18.973690+0000","last_active":"2024-04-24T23:27:26.216942+0000","last_peered":"2024-04-24T23:27:26.216942+0000","last_clean":"2024-04-24T23:27:26.216942+0000","last_became_active":"2024-04-24T23:27:18.973182+0000","last_became_peered":"2024-04-24T23:27:18.973182+0000","last_unstale":"2024-04-24T23:27:26.216942+0000","last_undegraded":"2024-04-24T23:27:26.216942+0000","last_fullsized":"2024-04-24T23:27:26.216942+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:00:44.809880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.851854+0000","last_change":"2024-04-24T23:27:19.991252+0000","last_active":"2024-04-24T23:27:26.851854+0000","last_peered":"2024-04-24T23:27:26.851854+0000","last_clean":"2024-04-24T23:27:26.851854+0000","last_became_active":"2024-04-24T23:27:19.990892+0000","last_became_peered":"2024-04-24T23:27:19.990892+0000","last_unstale":"2024-04-24T23:27:26.851854+0000","last_undegraded":"2024-04-24T23:27:26.851854+0000","last_fullsized":"2024-04-24T23:27:26.851854+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:29:37.835521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.216074+0000","last_change":"2024-04-24T23:27:24.718929+0000","last_active":"2024-04-24T23:27:26.216074+0000","last_peered":"2024-04-24T23:27:26.216074+0000","last_clean":"2024-04-24T23:27:26.216074+0000","last_became_active":"2024-04-24T23:27:24.718486+0000","last_became_peered":"2024-04-24T23:27:24.718486+0000","last_unstale":"2024-04-24T23:27:26.216074+0000","last_undegraded":"2024-04-24T23:27:26.216074+0000","last_fullsized":"2024-04-24T23:27:26.216074+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:31:13.359829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.216502+0000","last_change":"2024-04-24T23:27:19.983181+0000","last_active":"2024-04-24T23:27:26.216502+0000","last_peered":"2024-04-24T23:27:26.216502+0000","last_clean":"2024-04-24T23:27:26.216502+0000","last_became_active":"2024-04-24T23:27:19.981764+0000","last_became_peered":"2024-04-24T23:27:19.981764+0000","last_unstale":"2024-04-24T23:27:26.216502+0000","last_undegraded":"2024-04-24T23:27:26.216502+0000","last_fullsized":"2024-04-24T23:27:26.216502+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:01:20.653572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'1","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.215356+0000","last_change":"2024-04-24T23:27:18.976351+0000","last_active":"2024-04-24T23:27:26.215356+0000","last_peered":"2024-04-24T23:27:26.215356+0000","last_clean":"2024-04-24T23:27:26.215356+0000","last_became_active":"2024-04-24T23:27:18.975552+0000","last_became_peered":"2024-04-24T23:27:18.975552+0000","last_unstale":"2024-04-24T23:27:26.215356+0000","last_undegraded":"2024-04-24T23:27:26.215356+0000","last_fullsized":"2024-04-24T23:27:26.215356+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:26:29.551439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.216929+0000","last_change":"2024-04-24T23:27:24.229528+0000","last_active":"2024-04-24T23:27:26.216929+0000","last_peered":"2024-04-24T23:27:26.216929+0000","last_clean":"2024-04-24T23:27:26.216929+0000","last_became_active":"2024-04-24T23:27:24.229099+0000","last_became_peered":"2024-04-24T23:27:24.229099+0000","last_unstale":"2024-04-24T23:27:26.216929+0000","last_undegraded":"2024-04-24T23:27:26.216929+0000","last_fullsized":"2024-04-24T23:27:26.216929+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:51:51.421912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.852190+0000","last_change":"2024-04-24T23:27:19.980094+0000","last_active":"2024-04-24T23:27:26.852190+0000","last_peered":"2024-04-24T23:27:26.852190+0000","last_clean":"2024-04-24T23:27:26.852190+0000","last_became_active":"2024-04-24T23:27:19.979710+0000","last_became_peered":"2024-04-24T23:27:19.979710+0000","last_unstale":"2024-04-24T23:27:26.852190+0000","last_undegraded":"2024-04-24T23:27:26.852190+0000","last_fullsized":"2024-04-24T23:27:26.852190+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:50:37.920235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.851996+0000","last_change":"2024-04-24T23:27:18.963224+0000","last_active":"2024-04-24T23:27:26.851996+0000","last_peered":"2024-04-24T23:27:26.851996+0000","last_clean":"2024-04-24T23:27:26.851996+0000","last_became_active":"2024-04-24T23:27:18.962794+0000","last_became_peered":"2024-04-24T23:27:18.962794+0000","last_unstale":"2024-04-24T23:27:26.851996+0000","last_undegraded":"2024-04-24T23:27:26.851996+0000","last_fullsized":"2024-04-24T23:27:26.851996+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:05:36.869062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":349,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.216468+0000","last_change":"2024-04-24T23:26:02.307599+0000","last_active":"2024-04-24T23:27:26.216468+0000","last_peered":"2024-04-24T23:27:26.216468+0000","last_clean":"2024-04-24T23:27:26.216468+0000","last_became_active":"2024-04-24T23:26:02.307186+0000","last_became_peered":"2024-04-24T23:26:02.307186+0000","last_unstale":"2024-04-24T23:27:26.216468+0000","last_undegraded":"2024-04-24T23:27:26.216468+0000","last_fullsized":"2024-04-24T23:27:26.216468+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:22:33.824274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:22:33.824274+0000","last_clean_scrub_stamp":"2024-04-24T23:22:33.824274+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:19:26.789955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.216990+0000","last_change":"2024-04-24T23:27:24.719315+0000","last_active":"2024-04-24T23:27:26.216990+0000","last_peered":"2024-04-24T23:27:26.216990+0000","last_clean":"2024-04-24T23:27:26.216990+0000","last_became_active":"2024-04-24T23:27:24.718868+0000","last_became_peered":"2024-04-24T23:27:24.718868+0000","last_unstale":"2024-04-24T23:27:26.216990+0000","last_undegraded":"2024-04-24T23:27:26.216990+0000","last_fullsized":"2024-04-24T23:27:26.216990+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:43:32.122873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.215281+0000","last_change":"2024-04-24T23:27:19.983711+0000","last_active":"2024-04-24T23:27:26.215281+0000","last_peered":"2024-04-24T23:27:26.215281+0000","last_clean":"2024-04-24T23:27:26.215281+0000","last_became_active":"2024-04-24T23:27:19.983302+0000","last_became_peered":"2024-04-24T23:27:19.983302+0000","last_unstale":"2024-04-24T23:27:26.215281+0000","last_undegraded":"2024-04-24T23:27:26.215281+0000","last_fullsized":"2024-04-24T23:27:26.215281+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:44:49.251679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'1","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.216737+0000","last_change":"2024-04-24T23:27:18.976766+0000","last_active":"2024-04-24T23:27:26.216737+0000","last_peered":"2024-04-24T23:27:26.216737+0000","last_clean":"2024-04-24T23:27:26.216737+0000","last_became_active":"2024-04-24T23:27:18.976420+0000","last_became_peered":"2024-04-24T23:27:18.976420+0000","last_unstale":"2024-04-24T23:27:26.216737+0000","last_undegraded":"2024-04-24T23:27:26.216737+0000","last_fullsized":"2024-04-24T23:27:26.216737+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:59:34.262774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.887765+0000","last_change":"2024-04-24T23:27:24.226979+0000","last_active":"2024-04-24T23:27:26.887765+0000","last_peered":"2024-04-24T23:27:26.887765+0000","last_clean":"2024-04-24T23:27:26.887765+0000","last_became_active":"2024-04-24T23:27:24.226596+0000","last_became_peered":"2024-04-24T23:27:24.226596+0000","last_unstale":"2024-04-24T23:27:26.887765+0000","last_undegraded":"2024-04-24T23:27:26.887765+0000","last_fullsized":"2024-04-24T23:27:26.887765+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:50:48.585067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:43.973625+0000","last_change":"2024-04-24T23:27:18.970811+0000","last_active":"2024-04-24T23:27:43.973625+0000","last_peered":"2024-04-24T23:27:43.973625+0000","last_clean":"2024-04-24T23:27:43.973625+0000","last_became_active":"2024-04-24T23:27:18.970292+0000","last_became_peered":"2024-04-24T23:27:18.970292+0000","last_unstale":"2024-04-24T23:27:43.973625+0000","last_undegraded":"2024-04-24T23:27:43.973625+0000","last_fullsized":"2024-04-24T23:27:43.973625+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:06:37.556527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.215988+0000","last_change":"2024-04-24T23:27:19.975455+0000","last_active":"2024-04-24T23:27:26.215988+0000","last_peered":"2024-04-24T23:27:26.215988+0000","last_clean":"2024-04-24T23:27:26.215988+0000","last_became_active":"2024-04-24T23:27:19.973504+0000","last_became_peered":"2024-04-24T23:27:19.973504+0000","last_unstale":"2024-04-24T23:27:26.215988+0000","last_undegraded":"2024-04-24T23:27:26.215988+0000","last_fullsized":"2024-04-24T23:27:26.215988+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:50:34.450370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.850941+0000","last_change":"2024-04-24T23:27:24.719253+0000","last_active":"2024-04-24T23:27:26.850941+0000","last_peered":"2024-04-24T23:27:26.850941+0000","last_clean":"2024-04-24T23:27:26.850941+0000","last_became_active":"2024-04-24T23:27:24.718357+0000","last_became_peered":"2024-04-24T23:27:24.718357+0000","last_unstale":"2024-04-24T23:27:26.850941+0000","last_undegraded":"2024-04-24T23:27:26.850941+0000","last_fullsized":"2024-04-24T23:27:26.850941+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:31:03.798037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'4","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:44.988829+0000","last_change":"2024-04-24T23:27:18.974310+0000","last_active":"2024-04-24T23:27:44.988829+0000","last_peered":"2024-04-24T23:27:44.988829+0000","last_clean":"2024-04-24T23:27:44.988829+0000","last_became_active":"2024-04-24T23:27:18.973956+0000","last_became_peered":"2024-04-24T23:27:18.973956+0000","last_unstale":"2024-04-24T23:27:44.988829+0000","last_undegraded":"2024-04-24T23:27:44.988829+0000","last_fullsized":"2024-04-24T23:27:44.988829+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:44:27.233155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.851172+0000","last_change":"2024-04-24T23:27:19.985763+0000","last_active":"2024-04-24T23:27:26.851172+0000","last_peered":"2024-04-24T23:27:26.851172+0000","last_clean":"2024-04-24T23:27:26.851172+0000","last_became_active":"2024-04-24T23:27:19.985413+0000","last_became_peered":"2024-04-24T23:27:19.985413+0000","last_unstale":"2024-04-24T23:27:26.851172+0000","last_undegraded":"2024-04-24T23:27:26.851172+0000","last_fullsized":"2024-04-24T23:27:26.851172+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:04:54.910407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.283200+0000","last_change":"2024-04-24T23:27:24.225615+0000","last_active":"2024-04-24T23:27:26.283200+0000","last_peered":"2024-04-24T23:27:26.283200+0000","last_clean":"2024-04-24T23:27:26.283200+0000","last_became_active":"2024-04-24T23:27:24.225029+0000","last_became_peered":"2024-04-24T23:27:24.225029+0000","last_unstale":"2024-04-24T23:27:26.283200+0000","last_undegraded":"2024-04-24T23:27:26.283200+0000","last_fullsized":"2024-04-24T23:27:26.283200+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:56:02.956669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"84'2","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.851720+0000","last_change":"2024-04-24T23:27:18.963235+0000","last_active":"2024-04-24T23:27:26.851720+0000","last_peered":"2024-04-24T23:27:26.851720+0000","last_clean":"2024-04-24T23:27:26.851720+0000","last_became_active":"2024-04-24T23:27:18.962835+0000","last_became_peered":"2024-04-24T23:27:18.962835+0000","last_unstale":"2024-04-24T23:27:26.851720+0000","last_undegraded":"2024-04-24T23:27:26.851720+0000","last_fullsized":"2024-04-24T23:27:26.851720+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:34:29.376856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.853191+0000","last_change":"2024-04-24T23:27:19.987386+0000","last_active":"2024-04-24T23:27:26.853191+0000","last_peered":"2024-04-24T23:27:26.853191+0000","last_clean":"2024-04-24T23:27:26.853191+0000","last_became_active":"2024-04-24T23:27:19.987003+0000","last_became_peered":"2024-04-24T23:27:19.987003+0000","last_unstale":"2024-04-24T23:27:26.853191+0000","last_undegraded":"2024-04-24T23:27:26.853191+0000","last_fullsized":"2024-04-24T23:27:26.853191+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:20:25.635732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.851481+0000","last_change":"2024-04-24T23:27:24.719291+0000","last_active":"2024-04-24T23:27:26.851481+0000","last_peered":"2024-04-24T23:27:26.851481+0000","last_clean":"2024-04-24T23:27:26.851481+0000","last_became_active":"2024-04-24T23:27:24.718796+0000","last_became_peered":"2024-04-24T23:27:24.718796+0000","last_unstale":"2024-04-24T23:27:26.851481+0000","last_undegraded":"2024-04-24T23:27:26.851481+0000","last_fullsized":"2024-04-24T23:27:26.851481+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:20:25.135902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:43.974506+0000","last_change":"2024-04-24T23:27:18.967856+0000","last_active":"2024-04-24T23:27:43.974506+0000","last_peered":"2024-04-24T23:27:43.974506+0000","last_clean":"2024-04-24T23:27:43.974506+0000","last_became_active":"2024-04-24T23:27:18.967497+0000","last_became_peered":"2024-04-24T23:27:18.967497+0000","last_unstale":"2024-04-24T23:27:43.974506+0000","last_undegraded":"2024-04-24T23:27:43.974506+0000","last_fullsized":"2024-04-24T23:27:43.974506+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:21:29.841791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.850963+0000","last_change":"2024-04-24T23:27:19.981302+0000","last_active":"2024-04-24T23:27:26.850963+0000","last_peered":"2024-04-24T23:27:26.850963+0000","last_clean":"2024-04-24T23:27:26.850963+0000","last_became_active":"2024-04-24T23:27:19.980938+0000","last_became_peered":"2024-04-24T23:27:19.980938+0000","last_unstale":"2024-04-24T23:27:26.850963+0000","last_undegraded":"2024-04-24T23:27:26.850963+0000","last_fullsized":"2024-04-24T23:27:26.850963+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:24:53.939882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.853469+0000","last_change":"2024-04-24T23:27:24.224626+0000","last_active":"2024-04-24T23:27:26.853469+0000","last_peered":"2024-04-24T23:27:26.853469+0000","last_clean":"2024-04-24T23:27:26.853469+0000","last_became_active":"2024-04-24T23:27:24.224175+0000","last_became_peered":"2024-04-24T23:27:24.224175+0000","last_unstale":"2024-04-24T23:27:26.853469+0000","last_undegraded":"2024-04-24T23:27:26.853469+0000","last_fullsized":"2024-04-24T23:27:26.853469+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:54:44.155637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"90'3","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:43.975683+0000","last_change":"2024-04-24T23:27:18.975662+0000","last_active":"2024-04-24T23:27:43.975683+0000","last_peered":"2024-04-24T23:27:43.975683+0000","last_clean":"2024-04-24T23:27:43.975683+0000","last_became_active":"2024-04-24T23:27:18.975326+0000","last_became_peered":"2024-04-24T23:27:18.975326+0000","last_unstale":"2024-04-24T23:27:43.975683+0000","last_undegraded":"2024-04-24T23:27:43.975683+0000","last_fullsized":"2024-04-24T23:27:43.975683+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:39.440866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.887561+0000","last_change":"2024-04-24T23:27:19.978818+0000","last_active":"2024-04-24T23:27:26.887561+0000","last_peered":"2024-04-24T23:27:26.887561+0000","last_clean":"2024-04-24T23:27:26.887561+0000","last_became_active":"2024-04-24T23:27:19.978107+0000","last_became_peered":"2024-04-24T23:27:19.978107+0000","last_unstale":"2024-04-24T23:27:26.887561+0000","last_undegraded":"2024-04-24T23:27:26.887561+0000","last_fullsized":"2024-04-24T23:27:26.887561+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:22:25.797996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.851650+0000","last_change":"2024-04-24T23:27:24.234504+0000","last_active":"2024-04-24T23:27:26.851650+0000","last_peered":"2024-04-24T23:27:26.851650+0000","last_clean":"2024-04-24T23:27:26.851650+0000","last_became_active":"2024-04-24T23:27:24.234019+0000","last_became_peered":"2024-04-24T23:27:24.234019+0000","last_unstale":"2024-04-24T23:27:26.851650+0000","last_undegraded":"2024-04-24T23:27:26.851650+0000","last_fullsized":"2024-04-24T23:27:26.851650+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:06:19.101515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'1","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.283225+0000","last_change":"2024-04-24T23:27:18.967165+0000","last_active":"2024-04-24T23:27:26.283225+0000","last_peered":"2024-04-24T23:27:26.283225+0000","last_clean":"2024-04-24T23:27:26.283225+0000","last_became_active":"2024-04-24T23:27:18.966628+0000","last_became_peered":"2024-04-24T23:27:18.966628+0000","last_unstale":"2024-04-24T23:27:26.283225+0000","last_undegraded":"2024-04-24T23:27:26.283225+0000","last_fullsized":"2024-04-24T23:27:26.283225+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:12:20.520777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.851816+0000","last_change":"2024-04-24T23:27:19.988069+0000","last_active":"2024-04-24T23:27:26.851816+0000","last_peered":"2024-04-24T23:27:26.851816+0000","last_clean":"2024-04-24T23:27:26.851816+0000","last_became_active":"2024-04-24T23:27:19.987660+0000","last_became_peered":"2024-04-24T23:27:19.987660+0000","last_unstale":"2024-04-24T23:27:26.851816+0000","last_undegraded":"2024-04-24T23:27:26.851816+0000","last_fullsized":"2024-04-24T23:27:26.851816+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:21:03.505313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.851684+0000","last_change":"2024-04-24T23:27:24.719041+0000","last_active":"2024-04-24T23:27:26.851684+0000","last_peered":"2024-04-24T23:27:26.851684+0000","last_clean":"2024-04-24T23:27:26.851684+0000","last_became_active":"2024-04-24T23:27:24.718523+0000","last_became_peered":"2024-04-24T23:27:24.718523+0000","last_unstale":"2024-04-24T23:27:26.851684+0000","last_undegraded":"2024-04-24T23:27:26.851684+0000","last_fullsized":"2024-04-24T23:27:26.851684+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:27:57.396800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:43.975785+0000","last_change":"2024-04-24T23:27:18.975034+0000","last_active":"2024-04-24T23:27:43.975785+0000","last_peered":"2024-04-24T23:27:43.975785+0000","last_clean":"2024-04-24T23:27:43.975785+0000","last_became_active":"2024-04-24T23:27:18.973764+0000","last_became_peered":"2024-04-24T23:27:18.973764+0000","last_unstale":"2024-04-24T23:27:43.975785+0000","last_undegraded":"2024-04-24T23:27:43.975785+0000","last_fullsized":"2024-04-24T23:27:43.975785+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:43:23.451904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.216982+0000","last_change":"2024-04-24T23:27:19.984948+0000","last_active":"2024-04-24T23:27:26.216982+0000","last_peered":"2024-04-24T23:27:26.216982+0000","last_clean":"2024-04-24T23:27:26.216982+0000","last_became_active":"2024-04-24T23:27:19.984484+0000","last_became_peered":"2024-04-24T23:27:19.984484+0000","last_unstale":"2024-04-24T23:27:26.216982+0000","last_undegraded":"2024-04-24T23:27:26.216982+0000","last_fullsized":"2024-04-24T23:27:26.216982+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:17:43.335983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.852122+0000","last_change":"2024-04-24T23:27:24.231307+0000","last_active":"2024-04-24T23:27:26.852122+0000","last_peered":"2024-04-24T23:27:26.852122+0000","last_clean":"2024-04-24T23:27:26.852122+0000","last_became_active":"2024-04-24T23:27:24.230848+0000","last_became_peered":"2024-04-24T23:27:24.230848+0000","last_unstale":"2024-04-24T23:27:26.852122+0000","last_undegraded":"2024-04-24T23:27:26.852122+0000","last_fullsized":"2024-04-24T23:27:26.852122+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:28.586230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:44.989274+0000","last_change":"2024-04-24T23:27:18.964232+0000","last_active":"2024-04-24T23:27:44.989274+0000","last_peered":"2024-04-24T23:27:44.989274+0000","last_clean":"2024-04-24T23:27:44.989274+0000","last_became_active":"2024-04-24T23:27:18.963855+0000","last_became_peered":"2024-04-24T23:27:18.963855+0000","last_unstale":"2024-04-24T23:27:44.989274+0000","last_undegraded":"2024-04-24T23:27:44.989274+0000","last_fullsized":"2024-04-24T23:27:44.989274+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:02:21.000326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.217169+0000","last_change":"2024-04-24T23:27:19.984354+0000","last_active":"2024-04-24T23:27:26.217169+0000","last_peered":"2024-04-24T23:27:26.217169+0000","last_clean":"2024-04-24T23:27:26.217169+0000","last_became_active":"2024-04-24T23:27:19.983972+0000","last_became_peered":"2024-04-24T23:27:19.983972+0000","last_unstale":"2024-04-24T23:27:26.217169+0000","last_undegraded":"2024-04-24T23:27:26.217169+0000","last_fullsized":"2024-04-24T23:27:26.217169+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:28:28.390283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.852637+0000","last_change":"2024-04-24T23:27:24.238404+0000","last_active":"2024-04-24T23:27:26.852637+0000","last_peered":"2024-04-24T23:27:26.852637+0000","last_clean":"2024-04-24T23:27:26.852637+0000","last_became_active":"2024-04-24T23:27:24.237976+0000","last_became_peered":"2024-04-24T23:27:24.237976+0000","last_unstale":"2024-04-24T23:27:26.852637+0000","last_undegraded":"2024-04-24T23:27:26.852637+0000","last_fullsized":"2024-04-24T23:27:26.852637+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:19:31.476789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.217010+0000","last_change":"2024-04-24T23:27:18.971259+0000","last_active":"2024-04-24T23:27:26.217010+0000","last_peered":"2024-04-24T23:27:26.217010+0000","last_clean":"2024-04-24T23:27:26.217010+0000","last_became_active":"2024-04-24T23:27:18.970856+0000","last_became_peered":"2024-04-24T23:27:18.970856+0000","last_unstale":"2024-04-24T23:27:26.217010+0000","last_undegraded":"2024-04-24T23:27:26.217010+0000","last_fullsized":"2024-04-24T23:27:26.217010+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:54:58.642528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.216739+0000","last_change":"2024-04-24T23:27:19.986320+0000","last_active":"2024-04-24T23:27:26.216739+0000","last_peered":"2024-04-24T23:27:26.216739+0000","last_clean":"2024-04-24T23:27:26.216739+0000","last_became_active":"2024-04-24T23:27:19.985951+0000","last_became_peered":"2024-04-24T23:27:19.985951+0000","last_unstale":"2024-04-24T23:27:26.216739+0000","last_undegraded":"2024-04-24T23:27:26.216739+0000","last_fullsized":"2024-04-24T23:27:26.216739+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:05:35.427735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.216118+0000","last_change":"2024-04-24T23:27:24.228149+0000","last_active":"2024-04-24T23:27:26.216118+0000","last_peered":"2024-04-24T23:27:26.216118+0000","last_clean":"2024-04-24T23:27:26.216118+0000","last_became_active":"2024-04-24T23:27:24.227683+0000","last_became_peered":"2024-04-24T23:27:24.227683+0000","last_unstale":"2024-04-24T23:27:26.216118+0000","last_undegraded":"2024-04-24T23:27:26.216118+0000","last_fullsized":"2024-04-24T23:27:26.216118+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:55:50.408834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.216168+0000","last_change":"2024-04-24T23:27:18.976047+0000","last_active":"2024-04-24T23:27:26.216168+0000","last_peered":"2024-04-24T23:27:26.216168+0000","last_clean":"2024-04-24T23:27:26.216168+0000","last_became_active":"2024-04-24T23:27:18.975634+0000","last_became_peered":"2024-04-24T23:27:18.975634+0000","last_unstale":"2024-04-24T23:27:26.216168+0000","last_undegraded":"2024-04-24T23:27:26.216168+0000","last_fullsized":"2024-04-24T23:27:26.216168+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:04:35.528811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.217298+0000","last_change":"2024-04-24T23:27:19.990935+0000","last_active":"2024-04-24T23:27:26.217298+0000","last_peered":"2024-04-24T23:27:26.217298+0000","last_clean":"2024-04-24T23:27:26.217298+0000","last_became_active":"2024-04-24T23:27:19.990524+0000","last_became_peered":"2024-04-24T23:27:19.990524+0000","last_unstale":"2024-04-24T23:27:26.217298+0000","last_undegraded":"2024-04-24T23:27:26.217298+0000","last_fullsized":"2024-04-24T23:27:26.217298+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:54:02.463248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.887650+0000","last_change":"2024-04-24T23:27:19.979142+0000","last_active":"2024-04-24T23:27:26.887650+0000","last_peered":"2024-04-24T23:27:26.887650+0000","last_clean":"2024-04-24T23:27:26.887650+0000","last_became_active":"2024-04-24T23:27:19.978799+0000","last_became_peered":"2024-04-24T23:27:19.978799+0000","last_unstale":"2024-04-24T23:27:26.887650+0000","last_undegraded":"2024-04-24T23:27:26.887650+0000","last_fullsized":"2024-04-24T23:27:26.887650+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:07:19.046313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.283243+0000","last_change":"2024-04-24T23:27:19.986448+0000","last_active":"2024-04-24T23:27:26.283243+0000","last_peered":"2024-04-24T23:27:26.283243+0000","last_clean":"2024-04-24T23:27:26.283243+0000","last_became_active":"2024-04-24T23:27:19.986110+0000","last_became_peered":"2024-04-24T23:27:19.986110+0000","last_unstale":"2024-04-24T23:27:26.283243+0000","last_undegraded":"2024-04-24T23:27:26.283243+0000","last_fullsized":"2024-04-24T23:27:26.283243+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:21:25.819696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.283117+0000","last_change":"2024-04-24T23:27:19.980008+0000","last_active":"2024-04-24T23:27:26.283117+0000","last_peered":"2024-04-24T23:27:26.283117+0000","last_clean":"2024-04-24T23:27:26.283117+0000","last_became_active":"2024-04-24T23:27:19.979424+0000","last_became_peered":"2024-04-24T23:27:19.979424+0000","last_unstale":"2024-04-24T23:27:26.283117+0000","last_undegraded":"2024-04-24T23:27:26.283117+0000","last_fullsized":"2024-04-24T23:27:26.283117+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:44.449245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.852862+0000","last_change":"2024-04-24T23:27:19.987574+0000","last_active":"2024-04-24T23:27:26.852862+0000","last_peered":"2024-04-24T23:27:26.852862+0000","last_clean":"2024-04-24T23:27:26.852862+0000","last_became_active":"2024-04-24T23:27:19.987242+0000","last_became_peered":"2024-04-24T23:27:19.987242+0000","last_unstale":"2024-04-24T23:27:26.852862+0000","last_undegraded":"2024-04-24T23:27:26.852862+0000","last_fullsized":"2024-04-24T23:27:26.852862+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:39:10.777152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.216054+0000","last_change":"2024-04-24T23:27:19.980983+0000","last_active":"2024-04-24T23:27:26.216054+0000","last_peered":"2024-04-24T23:27:26.216054+0000","last_clean":"2024-04-24T23:27:26.216054+0000","last_became_active":"2024-04-24T23:27:19.980659+0000","last_became_peered":"2024-04-24T23:27:19.980659+0000","last_unstale":"2024-04-24T23:27:26.216054+0000","last_undegraded":"2024-04-24T23:27:26.216054+0000","last_fullsized":"2024-04-24T23:27:26.216054+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:02:52.647687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.888064+0000","last_change":"2024-04-24T23:27:19.988661+0000","last_active":"2024-04-24T23:27:26.888064+0000","last_peered":"2024-04-24T23:27:26.888064+0000","last_clean":"2024-04-24T23:27:26.888064+0000","last_became_active":"2024-04-24T23:27:19.988137+0000","last_became_peered":"2024-04-24T23:27:19.988137+0000","last_unstale":"2024-04-24T23:27:26.888064+0000","last_undegraded":"2024-04-24T23:27:26.888064+0000","last_fullsized":"2024-04-24T23:27:26.888064+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:47:59.736882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.283984+0000","last_change":"2024-04-24T23:27:19.985864+0000","last_active":"2024-04-24T23:27:26.283984+0000","last_peered":"2024-04-24T23:27:26.283984+0000","last_clean":"2024-04-24T23:27:26.283984+0000","last_became_active":"2024-04-24T23:27:19.984877+0000","last_became_peered":"2024-04-24T23:27:19.984877+0000","last_unstale":"2024-04-24T23:27:26.283984+0000","last_undegraded":"2024-04-24T23:27:26.283984+0000","last_fullsized":"2024-04-24T23:27:26.283984+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:14:43.458970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.852307+0000","last_change":"2024-04-24T23:27:19.988831+0000","last_active":"2024-04-24T23:27:26.852307+0000","last_peered":"2024-04-24T23:27:26.852307+0000","last_clean":"2024-04-24T23:27:26.852307+0000","last_became_active":"2024-04-24T23:27:19.988478+0000","last_became_peered":"2024-04-24T23:27:19.988478+0000","last_unstale":"2024-04-24T23:27:26.852307+0000","last_undegraded":"2024-04-24T23:27:26.852307+0000","last_fullsized":"2024-04-24T23:27:26.852307+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:18:37.488710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.216350+0000","last_change":"2024-04-24T23:27:19.983553+0000","last_active":"2024-04-24T23:27:26.216350+0000","last_peered":"2024-04-24T23:27:26.216350+0000","last_clean":"2024-04-24T23:27:26.216350+0000","last_became_active":"2024-04-24T23:27:19.981042+0000","last_became_peered":"2024-04-24T23:27:19.981042+0000","last_unstale":"2024-04-24T23:27:26.216350+0000","last_undegraded":"2024-04-24T23:27:26.216350+0000","last_fullsized":"2024-04-24T23:27:26.216350+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:09:42.608238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.851121+0000","last_change":"2024-04-24T23:27:19.985425+0000","last_active":"2024-04-24T23:27:26.851121+0000","last_peered":"2024-04-24T23:27:26.851121+0000","last_clean":"2024-04-24T23:27:26.851121+0000","last_became_active":"2024-04-24T23:27:19.985094+0000","last_became_peered":"2024-04-24T23:27:19.985094+0000","last_unstale":"2024-04-24T23:27:26.851121+0000","last_undegraded":"2024-04-24T23:27:26.851121+0000","last_fullsized":"2024-04-24T23:27:26.851121+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:07:15.612247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.852516+0000","last_change":"2024-04-24T23:27:19.988224+0000","last_active":"2024-04-24T23:27:26.852516+0000","last_peered":"2024-04-24T23:27:26.852516+0000","last_clean":"2024-04-24T23:27:26.852516+0000","last_became_active":"2024-04-24T23:27:19.987820+0000","last_became_peered":"2024-04-24T23:27:19.987820+0000","last_unstale":"2024-04-24T23:27:26.852516+0000","last_undegraded":"2024-04-24T23:27:26.852516+0000","last_fullsized":"2024-04-24T23:27:26.852516+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:27.653290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.215564+0000","last_change":"2024-04-24T23:27:19.989672+0000","last_active":"2024-04-24T23:27:26.215564+0000","last_peered":"2024-04-24T23:27:26.215564+0000","last_clean":"2024-04-24T23:27:26.215564+0000","last_became_active":"2024-04-24T23:27:19.989337+0000","last_became_peered":"2024-04-24T23:27:19.989337+0000","last_unstale":"2024-04-24T23:27:26.215564+0000","last_undegraded":"2024-04-24T23:27:26.215564+0000","last_fullsized":"2024-04-24T23:27:26.215564+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:07:13.399366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:27:26.887762+0000","last_change":"2024-04-24T23:27:19.987703+0000","last_active":"2024-04-24T23:27:26.887762+0000","last_peered":"2024-04-24T23:27:26.887762+0000","last_clean":"2024-04-24T23:27:26.887762+0000","last_became_active":"2024-04-24T23:27:19.987355+0000","last_became_peered":"2024-04-24T23:27:19.987355+0000","last_unstale":"2024-04-24T23:27:26.887762+0000","last_undegraded":"2024-04-24T23:27:26.887762+0000","last_fullsized":"2024-04-24T23:27:26.887762+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:03:15.812726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":12288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":280134,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":43760,"num_objects":50,"num_object_clones":0,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":0,"num_read":29,"num_read_kb":60,"num_write":124,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":33,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":286720,"data_stored":94703,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383699,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":978944,"data_stored":487204,"data_compressed":17667,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645336,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1105920,"data_stored":1119430,"data_compressed":53298,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808862,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":1003520,"data_stored":495632,"data_compressed":17667,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070499,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":995328,"data_stored":487360,"data_compressed":17667,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332135,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1036288,"data_stored":1078194,"data_compressed":53298,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495661,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":974848,"data_stored":486622,"data_compressed":17667,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574918,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36300,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708852,"statfs":{"total":95995035648,"available":95957864448,"internally_reserved":0,"allocated":987136,"data_stored":490464,"data_compressed":17667,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738442,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":486712,"data_compressed":17667,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411394,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32200,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712952,"statfs":{"total":95995035648,"available":95962062848,"internally_reserved":0,"allocated":1048576,"data_stored":528834,"data_compressed":17667,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182461,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":991232,"data_stored":486906,"data_compressed":17667,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920823,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":978944,"data_stored":486644,"data_compressed":17667,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757299,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1056768,"data_stored":1082953,"data_compressed":53298,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33993,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33993,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33993,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35631,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35631,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35631,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35631,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35631,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":42212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":42440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T23:27:49.015 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-24T23:27:49.015 INFO:tasks.cephadm:Setting up client nodes... 2024-04-24T23:27:49.015 DEBUG:teuthology.orchestra.run.smithi080:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ec7885aa-0290-11ef-bc93-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-24T23:27:49.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:48 smithi148 ceph-mon[32528]: from='client.14937 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:27:49.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:48 smithi148 ceph-mon[32528]: from='client.14943 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:27:49.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:48 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/476755511' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:27:49.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:48 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/3908477863' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:27:49.287 INFO:teuthology.orchestra.run.smithi080.stderr:Inferring config /var/lib/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:27:49.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:48 smithi080 ceph-mon[29152]: from='client.14937 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:27:49.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:48 smithi080 ceph-mon[29152]: from='client.14943 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:27:49.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:48 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/476755511' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:27:49.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:48 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/3908477863' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:27:49.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:48 smithi134 ceph-mon[32619]: from='client.14937 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:27:49.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:48 smithi134 ceph-mon[32619]: from='client.14943 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:27:49.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:48 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/476755511' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:27:49.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:48 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/3908477863' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:27:50.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:49 smithi148 ceph-mon[32528]: from='client.14961 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T23:27:50.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:49 smithi148 ceph-mon[32528]: pgmap v311: 129 pgs: 129 active+clean; 619 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 4.9 KiB/s wr, 8 op/s 2024-04-24T23:27:50.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:49 smithi148 ceph-mon[32528]: from='client.14967 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T23:27:50.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:49 smithi080 ceph-mon[29152]: from='client.14961 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T23:27:50.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:49 smithi080 ceph-mon[29152]: pgmap v311: 129 pgs: 129 active+clean; 619 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 4.9 KiB/s wr, 8 op/s 2024-04-24T23:27:50.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:49 smithi080 ceph-mon[29152]: from='client.14967 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T23:27:50.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:49 smithi134 ceph-mon[32619]: from='client.14961 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T23:27:50.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:49 smithi134 ceph-mon[32619]: pgmap v311: 129 pgs: 129 active+clean; 619 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 4.9 KiB/s wr, 8 op/s 2024-04-24T23:27:50.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:49 smithi134 ceph-mon[32619]: from='client.14967 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T23:27:51.093 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:51.091+0000 7ff2b7fd9700 1 -- 172.21.15.80:0/4255412195 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff2b01022c0 msgr2=0x7ff2b01026e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:51.093 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:51.091+0000 7ff2b7fd9700 1 --2- 172.21.15.80:0/4255412195 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff2b01022c0 0x7ff2b01026e0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac0099c0 tx=0x7ff2ac009cd0 comp rx=0 tx=0).stop 2024-04-24T23:27:51.093 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:51.091+0000 7ff2b7fd9700 1 -- 172.21.15.80:0/4255412195 shutdown_connections 2024-04-24T23:27:51.094 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:51.091+0000 7ff2b7fd9700 1 --2- 172.21.15.80:0/4255412195 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff2b0103e60 0x7ff2b010a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:51.094 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:51.091+0000 7ff2b7fd9700 1 --2- 172.21.15.80:0/4255412195 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff2b01022c0 0x7ff2b01026e0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:51.094 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:51.091+0000 7ff2b7fd9700 1 --2- 172.21.15.80:0/4255412195 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff2b01034c0 0x7ff2b0103920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:51.094 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:51.091+0000 7ff2b7fd9700 1 -- 172.21.15.80:0/4255412195 >> 172.21.15.80:0/4255412195 conn(0x7ff2b00fd860 msgr2=0x7ff2b00ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:51.094 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:51.091+0000 7ff2b7fd9700 1 -- 172.21.15.80:0/4255412195 shutdown_connections 2024-04-24T23:27:51.094 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:51.091+0000 7ff2b7fd9700 1 -- 172.21.15.80:0/4255412195 wait complete. 2024-04-24T23:27:51.094 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:51.092+0000 7ff2b7fd9700 1 Processor -- start 2024-04-24T23:27:51.094 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:51.092+0000 7ff2b7fd9700 1 -- start start 2024-04-24T23:27:51.094 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:51.092+0000 7ff2b7fd9700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff2b01022c0 0x7ff2b0198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:51.094 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:51.092+0000 7ff2b7fd9700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff2b01034c0 0x7ff2b0198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:51.094 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:51.092+0000 7ff2b7fd9700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff2b0103e60 0x7ff2b019fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:51.094 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:51.092+0000 7ff2b7fd9700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b010df80 con 0x7ff2b0103e60 2024-04-24T23:27:51.095 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:51.092+0000 7ff2b7fd9700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b010de00 con 0x7ff2b01022c0 2024-04-24T23:27:51.095 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:51.092+0000 7ff2b7fd9700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b010e100 con 0x7ff2b01034c0 2024-04-24T23:27:51.095 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:51.092+0000 7ff2b6576700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff2b0103e60 0x7ff2b019fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:51.095 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:51.092+0000 7ff2b6576700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff2b0103e60 0x7ff2b019fad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:37574/0 (socket says 172.21.15.80:37574) 2024-04-24T23:27:51.095 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:51.092+0000 7ff2b6576700 1 -- 172.21.15.80:0/2929146276 learned_addr learned my addr 172.21.15.80:0/2929146276 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:27:51.095 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:51.092+0000 7ff2b5574700 1 --2- 172.21.15.80:0/2929146276 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff2b01034c0 0x7ff2b0198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:51.095 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:51.093+0000 7ff2b5d75700 1 --2- 172.21.15.80:0/2929146276 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff2b01022c0 0x7ff2b0198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:51.095 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:51.093+0000 7ff2b6576700 1 -- 172.21.15.80:0/2929146276 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff2b01034c0 msgr2=0x7ff2b0198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:51.095 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:51.093+0000 7ff2b6576700 1 --2- 172.21.15.80:0/2929146276 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff2b01034c0 0x7ff2b0198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:51.095 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:51.093+0000 7ff2b6576700 1 -- 172.21.15.80:0/2929146276 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff2b01022c0 msgr2=0x7ff2b0198510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:51.095 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:51.093+0000 7ff2b6576700 1 --2- 172.21.15.80:0/2929146276 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff2b01022c0 0x7ff2b0198510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:51.095 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:51.093+0000 7ff2b6576700 1 -- 172.21.15.80:0/2929146276 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2b01a0170 con 0x7ff2b0103e60 2024-04-24T23:27:51.095 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:51.093+0000 7ff2b6576700 1 --2- 172.21.15.80:0/2929146276 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff2b0103e60 0x7ff2b019fad0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7ff2a800b9a0 tx=0x7ff2a800bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:51.095 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:51.093+0000 7ff2a6ffd700 1 -- 172.21.15.80:0/2929146276 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2a8019070 con 0x7ff2b0103e60 2024-04-24T23:27:51.096 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:51.093+0000 7ff2a6ffd700 1 -- 172.21.15.80:0/2929146276 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2a8014440 con 0x7ff2b0103e60 2024-04-24T23:27:51.096 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:51.093+0000 7ff2a6ffd700 1 -- 172.21.15.80:0/2929146276 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2a80075d0 con 0x7ff2b0103e60 2024-04-24T23:27:51.096 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:51.093+0000 7ff2b7fd9700 1 -- 172.21.15.80:0/2929146276 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2b01a0460 con 0x7ff2b0103e60 2024-04-24T23:27:51.096 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:51.093+0000 7ff2b7fd9700 1 -- 172.21.15.80:0/2929146276 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2b01a0a20 con 0x7ff2b0103e60 2024-04-24T23:27:51.097 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:51.094+0000 7ff2b7fd9700 1 -- 172.21.15.80:0/2929146276 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2b00507a0 con 0x7ff2b0103e60 2024-04-24T23:27:51.097 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:51.095+0000 7ff2a6ffd700 1 -- 172.21.15.80:0/2929146276 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7ff2a801c070 con 0x7ff2b0103e60 2024-04-24T23:27:51.097 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:51.095+0000 7ff2a6ffd700 1 --2- 172.21.15.80:0/2929146276 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7ff29c0a2d60 0x7ff29c0a5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:51.097 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:51.096+0000 7ff2b5d75700 1 --2- 172.21.15.80:0/2929146276 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7ff29c0a2d60 0x7ff29c0a5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:51.097 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:51.096+0000 7ff2a6ffd700 1 -- 172.21.15.80:0/2929146276 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7ff2a80ce700 con 0x7ff2b0103e60 2024-04-24T23:27:51.097 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:51.096+0000 7ff2b5d75700 1 --2- 172.21.15.80:0/2929146276 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7ff29c0a2d60 0x7ff29c0a5220 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac00b580 tx=0x7ff2ac005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:51.100 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:51.100+0000 7ff2a6ffd700 1 -- 172.21.15.80:0/2929146276 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff2a809b460 con 0x7ff2b0103e60 2024-04-24T23:27:51.375 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:51.376+0000 7ff2b7fd9700 1 -- 172.21.15.80:0/2929146276 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7ff2b002db30 con 0x7ff2b0103e60 2024-04-24T23:27:51.377 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:51.378+0000 7ff2a6ffd700 1 -- 172.21.15.80:0/2929146276 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7ff2a809ad60 con 0x7ff2b0103e60 2024-04-24T23:27:51.378 INFO:teuthology.orchestra.run.smithi080.stdout:[client.0] 2024-04-24T23:27:51.378 INFO:teuthology.orchestra.run.smithi080.stdout: key = AQB3lSlm/tByFhAAQPN3lacCY6YI0J7qtBoOdA== 2024-04-24T23:27:51.380 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:51.380+0000 7ff2b7fd9700 1 -- 172.21.15.80:0/2929146276 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7ff29c0a2d60 msgr2=0x7ff29c0a5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:51.380 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:51.380+0000 7ff2b7fd9700 1 --2- 172.21.15.80:0/2929146276 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7ff29c0a2d60 0x7ff29c0a5220 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac00b580 tx=0x7ff2ac005fb0 comp rx=0 tx=0).stop 2024-04-24T23:27:51.380 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:51.380+0000 7ff2b7fd9700 1 -- 172.21.15.80:0/2929146276 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff2b0103e60 msgr2=0x7ff2b019fad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:51.380 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:51.380+0000 7ff2b7fd9700 1 --2- 172.21.15.80:0/2929146276 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff2b0103e60 0x7ff2b019fad0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7ff2a800b9a0 tx=0x7ff2a800bd60 comp rx=0 tx=0).stop 2024-04-24T23:27:51.380 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:51.380+0000 7ff2b7fd9700 1 -- 172.21.15.80:0/2929146276 shutdown_connections 2024-04-24T23:27:51.380 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:51.380+0000 7ff2b7fd9700 1 --2- 172.21.15.80:0/2929146276 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7ff29c0a2d60 0x7ff29c0a5220 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:51.380 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:51.380+0000 7ff2b7fd9700 1 --2- 172.21.15.80:0/2929146276 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff2b0103e60 0x7ff2b019fad0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:51.381 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:51.380+0000 7ff2b7fd9700 1 --2- 172.21.15.80:0/2929146276 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff2b01022c0 0x7ff2b0198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:51.381 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:51.380+0000 7ff2b7fd9700 1 --2- 172.21.15.80:0/2929146276 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff2b01034c0 0x7ff2b0198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:51.381 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:51.380+0000 7ff2b7fd9700 1 -- 172.21.15.80:0/2929146276 >> 172.21.15.80:0/2929146276 conn(0x7ff2b00fd860 msgr2=0x7ff2b0106760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:51.381 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:51.380+0000 7ff2b7fd9700 1 -- 172.21.15.80:0/2929146276 shutdown_connections 2024-04-24T23:27:51.381 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:51.380+0000 7ff2b7fd9700 1 -- 172.21.15.80:0/2929146276 wait complete. 2024-04-24T23:27:52.170 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-24T23:27:52.171 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-04-24T23:27:52.171 DEBUG:teuthology.orchestra.run.smithi080:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-24T23:27:52.207 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-24T23:27:52.207 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-04-24T23:27:52.207 DEBUG:teuthology.orchestra.run.smithi080:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid ec7885aa-0290-11ef-bc93-c7b262605968 -- ceph mgr dump --format=json 2024-04-24T23:27:52.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:52 smithi148 ceph-mon[32528]: pgmap v312: 129 pgs: 129 active+clean; 621 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 5.7 KiB/s wr, 11 op/s 2024-04-24T23:27:52.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:52 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/2929146276' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-24T23:27:52.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:52 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/2929146276' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-24T23:27:52.266 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:52 smithi080 ceph-mon[29152]: pgmap v312: 129 pgs: 129 active+clean; 621 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 5.7 KiB/s wr, 11 op/s 2024-04-24T23:27:52.267 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:52 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/2929146276' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-24T23:27:52.267 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:52 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/2929146276' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-24T23:27:52.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:52 smithi134 ceph-mon[32619]: pgmap v312: 129 pgs: 129 active+clean; 621 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 5.7 KiB/s wr, 11 op/s 2024-04-24T23:27:52.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:52 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/2929146276' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-24T23:27:52.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:52 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/2929146276' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-24T23:27:52.488 INFO:teuthology.orchestra.run.smithi080.stderr:Inferring config /var/lib/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:27:54.038 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:54 smithi080 ceph-mon[29152]: pgmap v313: 129 pgs: 129 active+clean; 621 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 5.7 KiB/s wr, 11 op/s 2024-04-24T23:27:54.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:54 smithi148 ceph-mon[32528]: pgmap v313: 129 pgs: 129 active+clean; 621 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 5.7 KiB/s wr, 11 op/s 2024-04-24T23:27:54.383 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:54.367+0000 7f9cf6c57700 1 -- 172.21.15.80:0/3400304029 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9cf0101410 msgr2=0x7f9cf0101890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:54.383 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:54.367+0000 7f9cf6c57700 1 --2- 172.21.15.80:0/3400304029 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9cf0101410 0x7f9cf0101890 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc009990 tx=0x7f9cdc009ca0 comp rx=0 tx=0).stop 2024-04-24T23:27:54.384 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:54.368+0000 7f9cf6c57700 1 -- 172.21.15.80:0/3400304029 shutdown_connections 2024-04-24T23:27:54.384 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:54.368+0000 7f9cf6c57700 1 --2- 172.21.15.80:0/3400304029 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9cf0101dd0 0x7f9cf010dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:54.384 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:54.368+0000 7f9cf6c57700 1 --2- 172.21.15.80:0/3400304029 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9cf0107280 0x7f9cf0107660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:54.384 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:54.368+0000 7f9cf6c57700 1 --2- 172.21.15.80:0/3400304029 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9cf0101410 0x7f9cf0101890 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:54.384 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:54.368+0000 7f9cf6c57700 1 -- 172.21.15.80:0/3400304029 >> 172.21.15.80:0/3400304029 conn(0x7f9cf0078020 msgr2=0x7f9cf0078430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:54.384 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:54.368+0000 7f9cf6c57700 1 -- 172.21.15.80:0/3400304029 shutdown_connections 2024-04-24T23:27:54.384 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:54.368+0000 7f9cf6c57700 1 -- 172.21.15.80:0/3400304029 wait complete. 2024-04-24T23:27:54.384 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:54.368+0000 7f9cf6c57700 1 Processor -- start 2024-04-24T23:27:54.384 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:54.368+0000 7f9cf6c57700 1 -- start start 2024-04-24T23:27:54.384 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:54.368+0000 7f9cf6c57700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9cf0101410 0x7f9cf01a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:54.384 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:54.368+0000 7f9cf6c57700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9cf0101dd0 0x7f9cf01a5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:54.384 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:54.368+0000 7f9cf6c57700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9cf0107280 0x7f9cf019f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:54.384 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:54.368+0000 7f9cf6c57700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f9cf0111520 con 0x7f9cf0107280 2024-04-24T23:27:54.384 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:54.368+0000 7f9cf6c57700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9cf01113a0 con 0x7f9cf0101dd0 2024-04-24T23:27:54.385 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:54.368+0000 7f9cf6c57700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f9cf01116a0 con 0x7f9cf0101410 2024-04-24T23:27:54.385 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:54.369+0000 7f9cf51f4700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9cf0107280 0x7f9cf019f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:54.385 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:54.369+0000 7f9cf51f4700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9cf0107280 0x7f9cf019f200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:37096/0 (socket says 172.21.15.80:37096) 2024-04-24T23:27:54.385 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:54.369+0000 7f9cf51f4700 1 -- 172.21.15.80:0/2896465882 learned_addr learned my addr 172.21.15.80:0/2896465882 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:27:54.385 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:54.369+0000 7f9ceffff700 1 --2- 172.21.15.80:0/2896465882 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9cf0101dd0 0x7f9cf01a5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:54.385 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:54.369+0000 7f9cf49f3700 1 --2- 172.21.15.80:0/2896465882 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9cf0101410 0x7f9cf01a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:54.385 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:54.369+0000 7f9cf51f4700 1 -- 172.21.15.80:0/2896465882 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9cf0101410 msgr2=0x7f9cf01a4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:54.385 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:54.369+0000 7f9cf51f4700 1 --2- 172.21.15.80:0/2896465882 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9cf0101410 0x7f9cf01a4d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:54.385 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:54.369+0000 7f9cf51f4700 1 -- 172.21.15.80:0/2896465882 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9cf0101dd0 msgr2=0x7f9cf01a5280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:54.385 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:54.369+0000 7f9cf51f4700 1 --2- 172.21.15.80:0/2896465882 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9cf0101dd0 0x7f9cf01a5280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:54.385 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:54.369+0000 7f9cf51f4700 1 -- 172.21.15.80:0/2896465882 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cf019fb00 con 0x7f9cf0107280 2024-04-24T23:27:54.385 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:54.369+0000 7f9cf51f4700 1 --2- 172.21.15.80:0/2896465882 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9cf0107280 0x7f9cf019f200 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f9ce8007ee0 tx=0x7f9ce800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:54.385 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:54.369+0000 7f9cedffb700 1 -- 172.21.15.80:0/2896465882 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ce8004bb0 con 0x7f9cf0107280 2024-04-24T23:27:54.386 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:54.369+0000 7f9cedffb700 1 -- 172.21.15.80:0/2896465882 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ce8004d50 con 0x7f9cf0107280 2024-04-24T23:27:54.386 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:54.369+0000 7f9cf6c57700 1 -- 172.21.15.80:0/2896465882 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cf019fd90 con 0x7f9cf0107280 2024-04-24T23:27:54.386 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:54.369+0000 7f9cedffb700 1 -- 172.21.15.80:0/2896465882 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ce8005960 con 0x7f9cf0107280 2024-04-24T23:27:54.386 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:54.369+0000 7f9cf6c57700 1 -- 172.21.15.80:0/2896465882 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cf01abd00 con 0x7f9cf0107280 2024-04-24T23:27:54.386 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:54.370+0000 7f9cf6c57700 1 -- 172.21.15.80:0/2896465882 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9cf01028a0 con 0x7f9cf0107280 2024-04-24T23:27:54.386 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:54.371+0000 7f9cedffb700 1 -- 172.21.15.80:0/2896465882 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f9ce801a020 con 0x7f9cf0107280 2024-04-24T23:27:54.386 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:54.372+0000 7f9cedffb700 1 --2- 172.21.15.80:0/2896465882 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f9ce00a2cb0 0x7f9ce00a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:54.386 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:54.372+0000 7f9cf49f3700 1 --2- 172.21.15.80:0/2896465882 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f9ce00a2cb0 0x7f9ce00a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:54.386 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:54.372+0000 7f9cedffb700 1 -- 172.21.15.80:0/2896465882 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f9ce80ceb30 con 0x7f9cf0107280 2024-04-24T23:27:54.386 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:54.372+0000 7f9cf49f3700 1 --2- 172.21.15.80:0/2896465882 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f9ce00a2cb0 0x7f9ce00a5170 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9ce4005950 tx=0x7f9ce400a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:54.386 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:54.377+0000 7f9cedffb700 1 -- 172.21.15.80:0/2896465882 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f9ce8010070 con 0x7f9cf0107280 2024-04-24T23:27:54.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:54 smithi134 ceph-mon[32619]: pgmap v313: 129 pgs: 129 active+clean; 621 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 5.7 KiB/s wr, 11 op/s 2024-04-24T23:27:54.626 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:54.626+0000 7f9cf6c57700 1 -- 172.21.15.80:0/2896465882 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f9cf01abed0 con 0x7f9cf0107280 2024-04-24T23:27:54.631 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:54.631+0000 7f9cedffb700 1 -- 172.21.15.80:0/2896465882 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+263198 (secure 0 0 0) 0x7f9ce809b6f0 con 0x7f9cf0107280 2024-04-24T23:27:54.634 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-24T23:27:54.636 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:54.634+0000 7f9cf6c57700 1 -- 172.21.15.80:0/2896465882 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f9ce00a2cb0 msgr2=0x7f9ce00a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:54.636 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:54.634+0000 7f9cf6c57700 1 --2- 172.21.15.80:0/2896465882 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f9ce00a2cb0 0x7f9ce00a5170 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9ce4005950 tx=0x7f9ce400a300 comp rx=0 tx=0).stop 2024-04-24T23:27:54.636 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:54.634+0000 7f9cf6c57700 1 -- 172.21.15.80:0/2896465882 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9cf0107280 msgr2=0x7f9cf019f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:54.636 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:54.634+0000 7f9cf6c57700 1 --2- 172.21.15.80:0/2896465882 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9cf0107280 0x7f9cf019f200 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f9ce8007ee0 tx=0x7f9ce800d3b0 comp rx=0 tx=0).stop 2024-04-24T23:27:54.636 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:54.634+0000 7f9cf6c57700 1 -- 172.21.15.80:0/2896465882 shutdown_connections 2024-04-24T23:27:54.636 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:54.634+0000 7f9cf6c57700 1 --2- 172.21.15.80:0/2896465882 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f9ce00a2cb0 0x7f9ce00a5170 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:54.636 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:54.634+0000 7f9cf6c57700 1 --2- 172.21.15.80:0/2896465882 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9cf0107280 0x7f9cf019f200 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:54.637 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:54.634+0000 7f9cf6c57700 1 --2- 172.21.15.80:0/2896465882 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9cf0101410 0x7f9cf01a4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:54.637 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:54.634+0000 7f9cf6c57700 1 --2- 172.21.15.80:0/2896465882 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9cf0101dd0 0x7f9cf01a5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:54.637 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:54.634+0000 7f9cf6c57700 1 -- 172.21.15.80:0/2896465882 >> 172.21.15.80:0/2896465882 conn(0x7f9cf0078020 msgr2=0x7f9cf0100550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:54.637 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:54.634+0000 7f9cf6c57700 1 -- 172.21.15.80:0/2896465882 shutdown_connections 2024-04-24T23:27:54.637 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:54.634+0000 7f9cf6c57700 1 -- 172.21.15.80:0/2896465882 wait complete. 2024-04-24T23:27:55.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:55 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/2896465882' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-24T23:27:55.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:55 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/2896465882' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-24T23:27:55.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:55 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/2896465882' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-24T23:27:55.482 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":3830596922},{"type":"v1","addr":"172.21.15.80:6801","nonce":3830596922}]},"active_addr":"172.21.15.80:6801/3830596922","active_change":"2024-04-24T23:20:06.793760+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24118,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24125,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.80:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.80:0","nonce":990423954}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.80:0","nonce":1833070360}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.80:0","nonce":3400278491}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.80:0","nonce":63470302}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.80:0","nonce":4222159009}]}]} 2024-04-24T23:27:55.489 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-04-24T23:27:55.490 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-04-24T23:27:55.490 DEBUG:teuthology.orchestra.run.smithi080:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid ec7885aa-0290-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-24T23:27:55.737 INFO:teuthology.orchestra.run.smithi080.stderr:Inferring config /var/lib/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:27:56.126 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:56 smithi080 ceph-mon[29152]: pgmap v314: 129 pgs: 129 active+clean; 626 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.4 KiB/s wr, 14 op/s 2024-04-24T23:27:56.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:56 smithi134 ceph-mon[32619]: pgmap v314: 129 pgs: 129 active+clean; 626 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.4 KiB/s wr, 14 op/s 2024-04-24T23:27:56.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:56 smithi148 ceph-mon[32528]: pgmap v314: 129 pgs: 129 active+clean; 626 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.4 KiB/s wr, 14 op/s 2024-04-24T23:27:57.520 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:57.519+0000 7fb469fcb700 1 -- 172.21.15.80:0/2500128633 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb4641082b0 msgr2=0x7fb464108690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:57.520 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:57.519+0000 7fb469fcb700 1 --2- 172.21.15.80:0/2500128633 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb4641082b0 0x7fb464108690 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fb454009990 tx=0x7fb454009ca0 comp rx=0 tx=0).stop 2024-04-24T23:27:57.520 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:57.519+0000 7fb469fcb700 1 -- 172.21.15.80:0/2500128633 shutdown_connections 2024-04-24T23:27:57.520 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:57.519+0000 7fb469fcb700 1 --2- 172.21.15.80:0/2500128633 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb46406a8e0 0x7fb46410f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:57.520 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:57.519+0000 7fb469fcb700 1 --2- 172.21.15.80:0/2500128633 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb4641082b0 0x7fb464108690 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:57.520 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:57.519+0000 7fb469fcb700 1 --2- 172.21.15.80:0/2500128633 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb464069f40 0x7fb46406a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:57.520 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:57.519+0000 7fb469fcb700 1 -- 172.21.15.80:0/2500128633 >> 172.21.15.80:0/2500128633 conn(0x7fb4640780e0 msgr2=0x7fb4640784f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:57.520 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:57.519+0000 7fb469fcb700 1 -- 172.21.15.80:0/2500128633 shutdown_connections 2024-04-24T23:27:57.520 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:57.519+0000 7fb469fcb700 1 -- 172.21.15.80:0/2500128633 wait complete. 2024-04-24T23:27:57.522 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:57.520+0000 7fb469fcb700 1 Processor -- start 2024-04-24T23:27:57.522 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:57.520+0000 7fb469fcb700 1 -- start start 2024-04-24T23:27:57.522 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:57.521+0000 7fb469fcb700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb464069f40 0x7fb46419e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:57.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:57.521+0000 7fb469fcb700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb46406a8e0 0x7fb46419eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:57.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:57.521+0000 7fb469fcb700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb4641082b0 0x7fb464198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:57.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:57.521+0000 7fb469fcb700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fb4641124f0 con 0x7fb46406a8e0 2024-04-24T23:27:57.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:57.521+0000 7fb469fcb700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb464112370 con 0x7fb4641082b0 2024-04-24T23:27:57.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:57.521+0000 7fb469fcb700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb464112670 con 0x7fb464069f40 2024-04-24T23:27:57.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:57.521+0000 7fb45affd700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb46406a8e0 0x7fb46419eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:57.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:57.521+0000 7fb45affd700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb46406a8e0 0x7fb46419eb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:37114/0 (socket says 172.21.15.80:37114) 2024-04-24T23:27:57.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:57.521+0000 7fb45affd700 1 -- 172.21.15.80:0/4247198092 learned_addr learned my addr 172.21.15.80:0/4247198092 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:27:57.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:57.521+0000 7fb463fff700 1 --2- 172.21.15.80:0/4247198092 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb4641082b0 0x7fb464198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:57.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:57.521+0000 7fb4637fe700 1 --2- 172.21.15.80:0/4247198092 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb464069f40 0x7fb46419e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:57.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:57.521+0000 7fb45affd700 1 -- 172.21.15.80:0/4247198092 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb464069f40 msgr2=0x7fb46419e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:57.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:57.521+0000 7fb45affd700 1 --2- 172.21.15.80:0/4247198092 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb464069f40 0x7fb46419e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:57.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:57.521+0000 7fb45affd700 1 -- 172.21.15.80:0/4247198092 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb4641082b0 msgr2=0x7fb464198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:57.524 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:57.521+0000 7fb45affd700 1 --2- 172.21.15.80:0/4247198092 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb4641082b0 0x7fb464198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:57.524 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:57.521+0000 7fb45affd700 1 -- 172.21.15.80:0/4247198092 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb464198ff0 con 0x7fb46406a8e0 2024-04-24T23:27:57.524 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:57.521+0000 7fb463fff700 1 --2- 172.21.15.80:0/4247198092 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb4641082b0 0x7fb464198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:27:57.524 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:57.521+0000 7fb4637fe700 1 --2- 172.21.15.80:0/4247198092 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb464069f40 0x7fb46419e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:27:57.524 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:57.521+0000 7fb45affd700 1 --2- 172.21.15.80:0/4247198092 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb46406a8e0 0x7fb46419eb30 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fb44c00c9b0 tx=0x7fb44c00cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:57.524 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:57.521+0000 7fb4617fa700 1 -- 172.21.15.80:0/4247198092 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb44c007b00 con 0x7fb46406a8e0 2024-04-24T23:27:57.524 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:57.521+0000 7fb4617fa700 1 -- 172.21.15.80:0/4247198092 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb44c007ca0 con 0x7fb46406a8e0 2024-04-24T23:27:57.524 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:57.521+0000 7fb469fcb700 1 -- 172.21.15.80:0/4247198092 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4641992e0 con 0x7fb46406a8e0 2024-04-24T23:27:57.524 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:57.521+0000 7fb4617fa700 1 -- 172.21.15.80:0/4247198092 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb44c018780 con 0x7fb46406a8e0 2024-04-24T23:27:57.524 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:57.522+0000 7fb469fcb700 1 -- 172.21.15.80:0/4247198092 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4641a5570 con 0x7fb46406a8e0 2024-04-24T23:27:57.524 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:57.523+0000 7fb469fcb700 1 -- 172.21.15.80:0/4247198092 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb464103960 con 0x7fb46406a8e0 2024-04-24T23:27:57.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:57.524+0000 7fb4617fa700 1 -- 172.21.15.80:0/4247198092 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fb44c00f450 con 0x7fb46406a8e0 2024-04-24T23:27:57.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:57.524+0000 7fb4617fa700 1 --2- 172.21.15.80:0/4247198092 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fb4500a2cb0 0x7fb4500a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:57.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:57.524+0000 7fb4637fe700 1 --2- 172.21.15.80:0/4247198092 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fb4500a2cb0 0x7fb4500a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:57.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:57.524+0000 7fb4617fa700 1 -- 172.21.15.80:0/4247198092 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fb44c0cf480 con 0x7fb46406a8e0 2024-04-24T23:27:57.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:57.525+0000 7fb4637fe700 1 --2- 172.21.15.80:0/4247198092 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fb4500a2cb0 0x7fb4500a5170 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb45400b5c0 tx=0x7fb454005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:57.528 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:57.529+0000 7fb4617fa700 1 -- 172.21.15.80:0/4247198092 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fb44c014030 con 0x7fb46406a8e0 2024-04-24T23:27:57.726 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:57.726+0000 7fb469fcb700 1 -- 172.21.15.80:0/4247198092 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb464199bf0 con 0x7fb46406a8e0 2024-04-24T23:27:57.728 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:57.728+0000 7fb4617fa700 1 -- 172.21.15.80:0/4247198092 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24124 (secure 0 0 0) 0x7fb44c09c040 con 0x7fb46406a8e0 2024-04-24T23:27:57.728 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-24T23:27:57.729 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":90,"fsid":"ec7885aa-0290-11ef-bc93-c7b262605968","created":"2024-04-24T23:19:05.586407+0000","modified":"2024-04-24T23:27:26.212213+0000","last_up_change":"2024-04-24T23:26:25.938316+0000","last_in_change":"2024-04-24T23:26:04.786699+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:22:33.124428+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:27:17.575788+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:27:18.492762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T23:27:22.438503+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a525741d-b46c-4eef-aa64-3f057cac1497","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":2597718112},{"type":"v1","addr":"172.21.15.80:6803","nonce":2597718112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":2597718112},{"type":"v1","addr":"172.21.15.80:6805","nonce":2597718112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2597718112},{"type":"v1","addr":"172.21.15.80:6809","nonce":2597718112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":2597718112},{"type":"v1","addr":"172.21.15.80:6807","nonce":2597718112}]},"public_addr":"172.21.15.80:6803/2597718112","cluster_addr":"172.21.15.80:6805/2597718112","heartbeat_back_addr":"172.21.15.80:6809/2597718112","heartbeat_front_addr":"172.21.15.80:6807/2597718112","state":["exists","up"]},{"osd":1,"uuid":"2d0f0d67-6db4-40ac-91cc-58604ab6795a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":2395610201},{"type":"v1","addr":"172.21.15.134:6801","nonce":2395610201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":2395610201},{"type":"v1","addr":"172.21.15.134:6803","nonce":2395610201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":2395610201},{"type":"v1","addr":"172.21.15.134:6807","nonce":2395610201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":2395610201},{"type":"v1","addr":"172.21.15.134:6805","nonce":2395610201}]},"public_addr":"172.21.15.134:6801/2395610201","cluster_addr":"172.21.15.134:6803/2395610201","heartbeat_back_addr":"172.21.15.134:6807/2395610201","heartbeat_front_addr":"172.21.15.134:6805/2395610201","state":["exists","up"]},{"osd":2,"uuid":"781fb879-fedc-4c84-82fb-59e073bc4cf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1629152187},{"type":"v1","addr":"172.21.15.148:6801","nonce":1629152187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1629152187},{"type":"v1","addr":"172.21.15.148:6803","nonce":1629152187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1629152187},{"type":"v1","addr":"172.21.15.148:6807","nonce":1629152187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1629152187},{"type":"v1","addr":"172.21.15.148:6805","nonce":1629152187}]},"public_addr":"172.21.15.148:6801/1629152187","cluster_addr":"172.21.15.148:6803/1629152187","heartbeat_back_addr":"172.21.15.148:6807/1629152187","heartbeat_front_addr":"172.21.15.148:6805/1629152187","state":["exists","up"]},{"osd":3,"uuid":"cd982757-f2f6-4296-b3e7-4cd40fbd8d9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":3128298510},{"type":"v1","addr":"172.21.15.80:6811","nonce":3128298510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":3128298510},{"type":"v1","addr":"172.21.15.80:6813","nonce":3128298510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":3128298510},{"type":"v1","addr":"172.21.15.80:6817","nonce":3128298510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":3128298510},{"type":"v1","addr":"172.21.15.80:6815","nonce":3128298510}]},"public_addr":"172.21.15.80:6811/3128298510","cluster_addr":"172.21.15.80:6813/3128298510","heartbeat_back_addr":"172.21.15.80:6817/3128298510","heartbeat_front_addr":"172.21.15.80:6815/3128298510","state":["exists","up"]},{"osd":4,"uuid":"dfc55a79-a7e8-417a-b898-ec44171a54af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1917556060},{"type":"v1","addr":"172.21.15.134:6809","nonce":1917556060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1917556060},{"type":"v1","addr":"172.21.15.134:6811","nonce":1917556060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1917556060},{"type":"v1","addr":"172.21.15.134:6815","nonce":1917556060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1917556060},{"type":"v1","addr":"172.21.15.134:6813","nonce":1917556060}]},"public_addr":"172.21.15.134:6809/1917556060","cluster_addr":"172.21.15.134:6811/1917556060","heartbeat_back_addr":"172.21.15.134:6815/1917556060","heartbeat_front_addr":"172.21.15.134:6813/1917556060","state":["exists","up"]},{"osd":5,"uuid":"b5491337-0694-440c-b679-312f3d14cf65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":2715951281},{"type":"v1","addr":"172.21.15.148:6809","nonce":2715951281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":2715951281},{"type":"v1","addr":"172.21.15.148:6811","nonce":2715951281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":2715951281},{"type":"v1","addr":"172.21.15.148:6815","nonce":2715951281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":2715951281},{"type":"v1","addr":"172.21.15.148:6813","nonce":2715951281}]},"public_addr":"172.21.15.148:6809/2715951281","cluster_addr":"172.21.15.148:6811/2715951281","heartbeat_back_addr":"172.21.15.148:6815/2715951281","heartbeat_front_addr":"172.21.15.148:6813/2715951281","state":["exists","up"]},{"osd":6,"uuid":"ba4b5df7-d86e-47cd-ab67-cf763c4e4d6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":953261724},{"type":"v1","addr":"172.21.15.80:6819","nonce":953261724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":953261724},{"type":"v1","addr":"172.21.15.80:6821","nonce":953261724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":953261724},{"type":"v1","addr":"172.21.15.80:6825","nonce":953261724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":953261724},{"type":"v1","addr":"172.21.15.80:6823","nonce":953261724}]},"public_addr":"172.21.15.80:6819/953261724","cluster_addr":"172.21.15.80:6821/953261724","heartbeat_back_addr":"172.21.15.80:6825/953261724","heartbeat_front_addr":"172.21.15.80:6823/953261724","state":["exists","up"]},{"osd":7,"uuid":"464a503a-75ae-49e7-ab6e-a7f9f2a6a307","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":108354844},{"type":"v1","addr":"172.21.15.134:6817","nonce":108354844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":108354844},{"type":"v1","addr":"172.21.15.134:6819","nonce":108354844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":108354844},{"type":"v1","addr":"172.21.15.134:6823","nonce":108354844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":108354844},{"type":"v1","addr":"172.21.15.134:6821","nonce":108354844}]},"public_addr":"172.21.15.134:6817/108354844","cluster_addr":"172.21.15.134:6819/108354844","heartbeat_back_addr":"172.21.15.134:6823/108354844","heartbeat_front_addr":"172.21.15.134:6821/108354844","state":["exists","up"]},{"osd":8,"uuid":"ce544b2e-4540-47c0-9997-500c45351330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3559655329},{"type":"v1","addr":"172.21.15.148:6817","nonce":3559655329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":3559655329},{"type":"v1","addr":"172.21.15.148:6819","nonce":3559655329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":3559655329},{"type":"v1","addr":"172.21.15.148:6823","nonce":3559655329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3559655329},{"type":"v1","addr":"172.21.15.148:6821","nonce":3559655329}]},"public_addr":"172.21.15.148:6817/3559655329","cluster_addr":"172.21.15.148:6819/3559655329","heartbeat_back_addr":"172.21.15.148:6823/3559655329","heartbeat_front_addr":"172.21.15.148:6821/3559655329","state":["exists","up"]},{"osd":9,"uuid":"06de3732-19b9-496c-a464-55322efa6759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":4019913545},{"type":"v1","addr":"172.21.15.80:6827","nonce":4019913545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":4019913545},{"type":"v1","addr":"172.21.15.80:6829","nonce":4019913545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":4019913545},{"type":"v1","addr":"172.21.15.80:6833","nonce":4019913545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":4019913545},{"type":"v1","addr":"172.21.15.80:6831","nonce":4019913545}]},"public_addr":"172.21.15.80:6827/4019913545","cluster_addr":"172.21.15.80:6829/4019913545","heartbeat_back_addr":"172.21.15.80:6833/4019913545","heartbeat_front_addr":"172.21.15.80:6831/4019913545","state":["exists","up"]},{"osd":10,"uuid":"8d92f0d9-5c64-429a-b750-c33ae0e070c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":647343766},{"type":"v1","addr":"172.21.15.134:6825","nonce":647343766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":647343766},{"type":"v1","addr":"172.21.15.134:6827","nonce":647343766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":647343766},{"type":"v1","addr":"172.21.15.134:6831","nonce":647343766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":647343766},{"type":"v1","addr":"172.21.15.134:6829","nonce":647343766}]},"public_addr":"172.21.15.134:6825/647343766","cluster_addr":"172.21.15.134:6827/647343766","heartbeat_back_addr":"172.21.15.134:6831/647343766","heartbeat_front_addr":"172.21.15.134:6829/647343766","state":["exists","up"]},{"osd":11,"uuid":"4c003a41-f65b-479f-bc80-b27d680cf936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":4068024116},{"type":"v1","addr":"172.21.15.148:6825","nonce":4068024116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":4068024116},{"type":"v1","addr":"172.21.15.148:6827","nonce":4068024116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":4068024116},{"type":"v1","addr":"172.21.15.148:6831","nonce":4068024116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":4068024116},{"type":"v1","addr":"172.21.15.148:6829","nonce":4068024116}]},"public_addr":"172.21.15.148:6825/4068024116","cluster_addr":"172.21.15.148:6827/4068024116","heartbeat_back_addr":"172.21.15.148:6831/4068024116","heartbeat_front_addr":"172.21.15.148:6829/4068024116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:21:42.433631+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:22:06.145933+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:22:30.726143+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:22:55.729961+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:23:21.280913+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:23:46.609983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:24:12.585247+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:24:38.359287+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:25:04.491232+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:25:30.788089+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:25:56.891078+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:26:23.899113+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.80:0/2828378749":"2024-04-25T23:20:06.793609+0000","172.21.15.80:6801/233364790":"2024-04-25T23:20:06.793609+0000","172.21.15.80:6800/1614691410":"2024-04-25T23:19:40.490692+0000","172.21.15.80:0/1406187145":"2024-04-25T23:20:06.793609+0000","172.21.15.80:0/92953396":"2024-04-25T23:19:40.490692+0000","172.21.15.80:0/598129432":"2024-04-25T23:20:06.793609+0000","172.21.15.80:0/232193118":"2024-04-25T23:19:40.490692+0000","172.21.15.80:6800/233364790":"2024-04-25T23:20:06.793609+0000","172.21.15.80:6801/1614691410":"2024-04-25T23:19:40.490692+0000","172.21.15.80:0/2081562529":"2024-04-25T23:19:40.490692+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:27:57.730 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:57.730+0000 7fb469fcb700 1 -- 172.21.15.80:0/4247198092 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fb4500a2cb0 msgr2=0x7fb4500a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:57.730 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:57.730+0000 7fb469fcb700 1 --2- 172.21.15.80:0/4247198092 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fb4500a2cb0 0x7fb4500a5170 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb45400b5c0 tx=0x7fb454005fb0 comp rx=0 tx=0).stop 2024-04-24T23:27:57.730 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:57.730+0000 7fb469fcb700 1 -- 172.21.15.80:0/4247198092 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb46406a8e0 msgr2=0x7fb46419eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:57.731 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:57.730+0000 7fb469fcb700 1 --2- 172.21.15.80:0/4247198092 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb46406a8e0 0x7fb46419eb30 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fb44c00c9b0 tx=0x7fb44c00cd70 comp rx=0 tx=0).stop 2024-04-24T23:27:57.731 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:57.730+0000 7fb469fcb700 1 -- 172.21.15.80:0/4247198092 shutdown_connections 2024-04-24T23:27:57.731 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:57.730+0000 7fb469fcb700 1 --2- 172.21.15.80:0/4247198092 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb4641082b0 0x7fb464198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:57.731 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:57.730+0000 7fb469fcb700 1 --2- 172.21.15.80:0/4247198092 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb464069f40 0x7fb46419e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:57.731 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:57.730+0000 7fb469fcb700 1 --2- 172.21.15.80:0/4247198092 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fb4500a2cb0 0x7fb4500a5170 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:57.731 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:57.730+0000 7fb469fcb700 1 --2- 172.21.15.80:0/4247198092 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb46406a8e0 0x7fb46419eb30 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:57.731 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:57.730+0000 7fb469fcb700 1 -- 172.21.15.80:0/4247198092 >> 172.21.15.80:0/4247198092 conn(0x7fb4640780e0 msgr2=0x7fb4640ffbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:57.731 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:57.730+0000 7fb469fcb700 1 -- 172.21.15.80:0/4247198092 shutdown_connections 2024-04-24T23:27:57.731 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:27:57.730+0000 7fb469fcb700 1 -- 172.21.15.80:0/4247198092 wait complete. 2024-04-24T23:27:58.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:58 smithi080 ceph-mon[29152]: pgmap v315: 129 pgs: 129 active+clean; 626 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.4 KiB/s wr, 14 op/s 2024-04-24T23:27:58.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:27:58 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/4247198092' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:27:58.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:58 smithi134 ceph-mon[32619]: pgmap v315: 129 pgs: 129 active+clean; 626 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.4 KiB/s wr, 14 op/s 2024-04-24T23:27:58.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:27:58 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/4247198092' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:27:58.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:58 smithi148 ceph-mon[32528]: pgmap v315: 129 pgs: 129 active+clean; 626 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.4 KiB/s wr, 14 op/s 2024-04-24T23:27:58.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:27:58 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/4247198092' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:27:58.632 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-04-24T23:27:58.632 DEBUG:teuthology.orchestra.run.smithi080:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid ec7885aa-0290-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-24T23:27:58.874 INFO:teuthology.orchestra.run.smithi080.stderr:Inferring config /var/lib/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:28:00.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:28:00 smithi080 ceph-mon[29152]: pgmap v316: 129 pgs: 129 active+clean; 636 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 6.8 KiB/s wr, 14 op/s 2024-04-24T23:28:00.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:28:00 smithi134 ceph-mon[32619]: pgmap v316: 129 pgs: 129 active+clean; 636 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 6.8 KiB/s wr, 14 op/s 2024-04-24T23:28:00.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:28:00 smithi148 ceph-mon[32528]: pgmap v316: 129 pgs: 129 active+clean; 636 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 6.8 KiB/s wr, 14 op/s 2024-04-24T23:28:00.580 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:00.577+0000 7f30313f1700 1 -- 172.21.15.80:0/2560742837 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f302c1026c0 msgr2=0x7f302c10e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:00.580 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:00.577+0000 7f30313f1700 1 --2- 172.21.15.80:0/2560742837 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f302c1026c0 0x7f302c10e0d0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f302400b0d0 tx=0x7f302400b3e0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.580 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:00.577+0000 7f30313f1700 1 -- 172.21.15.80:0/2560742837 shutdown_connections 2024-04-24T23:28:00.580 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:00.577+0000 7f30313f1700 1 --2- 172.21.15.80:0/2560742837 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f302c1026c0 0x7f302c10e0d0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.580 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:00.577+0000 7f30313f1700 1 --2- 172.21.15.80:0/2560742837 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f302c107420 0x7f302c107800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.580 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:00.577+0000 7f30313f1700 1 --2- 172.21.15.80:0/2560742837 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f302c101d00 0x7f302c102180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.581 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:00.577+0000 7f30313f1700 1 -- 172.21.15.80:0/2560742837 >> 172.21.15.80:0/2560742837 conn(0x7f302c076450 msgr2=0x7f302c076860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:00.581 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:00.577+0000 7f30313f1700 1 -- 172.21.15.80:0/2560742837 shutdown_connections 2024-04-24T23:28:00.581 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:00.577+0000 7f30313f1700 1 -- 172.21.15.80:0/2560742837 wait complete. 2024-04-24T23:28:00.581 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:00.578+0000 7f30313f1700 1 Processor -- start 2024-04-24T23:28:00.581 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:00.578+0000 7f30313f1700 1 -- start start 2024-04-24T23:28:00.581 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:00.578+0000 7f30313f1700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f302c101d00 0x7f302c102e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:00.581 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:00.578+0000 7f30313f1700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f302c1026c0 0x7f302c1033c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:00.581 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:00.578+0000 7f30313f1700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f302c107420 0x7f302c103900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:00.581 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:00.578+0000 7f30313f1700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f302c111570 con 0x7f302c101d00 2024-04-24T23:28:00.581 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:00.578+0000 7f30313f1700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f302c1113f0 con 0x7f302c107420 2024-04-24T23:28:00.581 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:00.578+0000 7f30313f1700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f302c1116f0 con 0x7f302c1026c0 2024-04-24T23:28:00.581 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:00.578+0000 7f302affd700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f302c101d00 0x7f302c102e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:00.581 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:00.578+0000 7f302affd700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f302c101d00 0x7f302c102e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:37134/0 (socket says 172.21.15.80:37134) 2024-04-24T23:28:00.581 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:00.578+0000 7f302affd700 1 -- 172.21.15.80:0/259354176 learned_addr learned my addr 172.21.15.80:0/259354176 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:28:00.582 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:00.578+0000 7f3023fff700 1 --2- 172.21.15.80:0/259354176 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f302c1026c0 0x7f302c1033c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:00.582 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:00.578+0000 7f302b7fe700 1 --2- 172.21.15.80:0/259354176 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f302c107420 0x7f302c103900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:00.582 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:00.579+0000 7f302affd700 1 -- 172.21.15.80:0/259354176 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f302c1026c0 msgr2=0x7f302c1033c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:00.582 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:00.579+0000 7f302affd700 1 --2- 172.21.15.80:0/259354176 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f302c1026c0 0x7f302c1033c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.582 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:00.579+0000 7f302affd700 1 -- 172.21.15.80:0/259354176 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f302c107420 msgr2=0x7f302c103900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:00.582 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:00.579+0000 7f302affd700 1 --2- 172.21.15.80:0/259354176 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f302c107420 0x7f302c103900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.582 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:00.579+0000 7f302affd700 1 -- 172.21.15.80:0/259354176 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f302c1a6fa0 con 0x7f302c101d00 2024-04-24T23:28:00.582 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:00.579+0000 7f302affd700 1 --2- 172.21.15.80:0/259354176 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f302c101d00 0x7f302c102e80 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f301c00c9b0 tx=0x7f301c00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:00.582 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:00.579+0000 7f3028ff9700 1 -- 172.21.15.80:0/259354176 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f301c007b00 con 0x7f302c101d00 2024-04-24T23:28:00.582 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:00.579+0000 7f30313f1700 1 -- 172.21.15.80:0/259354176 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f302c1a7290 con 0x7f302c101d00 2024-04-24T23:28:00.582 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:00.579+0000 7f3028ff9700 1 -- 172.21.15.80:0/259354176 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f301c00ce50 con 0x7f302c101d00 2024-04-24T23:28:00.582 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:00.579+0000 7f3028ff9700 1 -- 172.21.15.80:0/259354176 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f301c018660 con 0x7f302c101d00 2024-04-24T23:28:00.582 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:00.579+0000 7f30313f1700 1 -- 172.21.15.80:0/259354176 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f302c1a77d0 con 0x7f302c101d00 2024-04-24T23:28:00.583 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:00.580+0000 7f30313f1700 1 -- 172.21.15.80:0/259354176 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f302c068f20 con 0x7f302c101d00 2024-04-24T23:28:00.583 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:00.582+0000 7f3028ff9700 1 -- 172.21.15.80:0/259354176 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f301c00f450 con 0x7f302c101d00 2024-04-24T23:28:00.584 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:00.582+0000 7f3028ff9700 1 --2- 172.21.15.80:0/259354176 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f30180a2cb0 0x7f30180a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:00.584 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:00.583+0000 7f3023fff700 1 --2- 172.21.15.80:0/259354176 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f30180a2cb0 0x7f30180a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:00.584 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:00.583+0000 7f3028ff9700 1 -- 172.21.15.80:0/259354176 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f301c0cf520 con 0x7f302c101d00 2024-04-24T23:28:00.584 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:00.583+0000 7f3023fff700 1 --2- 172.21.15.80:0/259354176 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f30180a2cb0 0x7f30180a5170 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f302c076a50 tx=0x7f301400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:00.587 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:00.586+0000 7f3028ff9700 1 -- 172.21.15.80:0/259354176 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f301c014030 con 0x7f302c101d00 2024-04-24T23:28:00.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:00.788+0000 7f30313f1700 1 -- 172.21.15.80:0/259354176 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f302c104360 con 0x7f302c101d00 2024-04-24T23:28:00.790 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:00.790+0000 7f3028ff9700 1 -- 172.21.15.80:0/259354176 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24124 (secure 0 0 0) 0x7f301c09c170 con 0x7f302c101d00 2024-04-24T23:28:00.791 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-24T23:28:00.791 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":90,"fsid":"ec7885aa-0290-11ef-bc93-c7b262605968","created":"2024-04-24T23:19:05.586407+0000","modified":"2024-04-24T23:27:26.212213+0000","last_up_change":"2024-04-24T23:26:25.938316+0000","last_in_change":"2024-04-24T23:26:04.786699+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:22:33.124428+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:27:17.575788+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:27:18.492762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T23:27:22.438503+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a525741d-b46c-4eef-aa64-3f057cac1497","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":2597718112},{"type":"v1","addr":"172.21.15.80:6803","nonce":2597718112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":2597718112},{"type":"v1","addr":"172.21.15.80:6805","nonce":2597718112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2597718112},{"type":"v1","addr":"172.21.15.80:6809","nonce":2597718112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":2597718112},{"type":"v1","addr":"172.21.15.80:6807","nonce":2597718112}]},"public_addr":"172.21.15.80:6803/2597718112","cluster_addr":"172.21.15.80:6805/2597718112","heartbeat_back_addr":"172.21.15.80:6809/2597718112","heartbeat_front_addr":"172.21.15.80:6807/2597718112","state":["exists","up"]},{"osd":1,"uuid":"2d0f0d67-6db4-40ac-91cc-58604ab6795a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":2395610201},{"type":"v1","addr":"172.21.15.134:6801","nonce":2395610201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":2395610201},{"type":"v1","addr":"172.21.15.134:6803","nonce":2395610201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":2395610201},{"type":"v1","addr":"172.21.15.134:6807","nonce":2395610201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":2395610201},{"type":"v1","addr":"172.21.15.134:6805","nonce":2395610201}]},"public_addr":"172.21.15.134:6801/2395610201","cluster_addr":"172.21.15.134:6803/2395610201","heartbeat_back_addr":"172.21.15.134:6807/2395610201","heartbeat_front_addr":"172.21.15.134:6805/2395610201","state":["exists","up"]},{"osd":2,"uuid":"781fb879-fedc-4c84-82fb-59e073bc4cf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1629152187},{"type":"v1","addr":"172.21.15.148:6801","nonce":1629152187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1629152187},{"type":"v1","addr":"172.21.15.148:6803","nonce":1629152187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1629152187},{"type":"v1","addr":"172.21.15.148:6807","nonce":1629152187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1629152187},{"type":"v1","addr":"172.21.15.148:6805","nonce":1629152187}]},"public_addr":"172.21.15.148:6801/1629152187","cluster_addr":"172.21.15.148:6803/1629152187","heartbeat_back_addr":"172.21.15.148:6807/1629152187","heartbeat_front_addr":"172.21.15.148:6805/1629152187","state":["exists","up"]},{"osd":3,"uuid":"cd982757-f2f6-4296-b3e7-4cd40fbd8d9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":3128298510},{"type":"v1","addr":"172.21.15.80:6811","nonce":3128298510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":3128298510},{"type":"v1","addr":"172.21.15.80:6813","nonce":3128298510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":3128298510},{"type":"v1","addr":"172.21.15.80:6817","nonce":3128298510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":3128298510},{"type":"v1","addr":"172.21.15.80:6815","nonce":3128298510}]},"public_addr":"172.21.15.80:6811/3128298510","cluster_addr":"172.21.15.80:6813/3128298510","heartbeat_back_addr":"172.21.15.80:6817/3128298510","heartbeat_front_addr":"172.21.15.80:6815/3128298510","state":["exists","up"]},{"osd":4,"uuid":"dfc55a79-a7e8-417a-b898-ec44171a54af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1917556060},{"type":"v1","addr":"172.21.15.134:6809","nonce":1917556060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1917556060},{"type":"v1","addr":"172.21.15.134:6811","nonce":1917556060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1917556060},{"type":"v1","addr":"172.21.15.134:6815","nonce":1917556060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1917556060},{"type":"v1","addr":"172.21.15.134:6813","nonce":1917556060}]},"public_addr":"172.21.15.134:6809/1917556060","cluster_addr":"172.21.15.134:6811/1917556060","heartbeat_back_addr":"172.21.15.134:6815/1917556060","heartbeat_front_addr":"172.21.15.134:6813/1917556060","state":["exists","up"]},{"osd":5,"uuid":"b5491337-0694-440c-b679-312f3d14cf65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":2715951281},{"type":"v1","addr":"172.21.15.148:6809","nonce":2715951281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":2715951281},{"type":"v1","addr":"172.21.15.148:6811","nonce":2715951281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":2715951281},{"type":"v1","addr":"172.21.15.148:6815","nonce":2715951281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":2715951281},{"type":"v1","addr":"172.21.15.148:6813","nonce":2715951281}]},"public_addr":"172.21.15.148:6809/2715951281","cluster_addr":"172.21.15.148:6811/2715951281","heartbeat_back_addr":"172.21.15.148:6815/2715951281","heartbeat_front_addr":"172.21.15.148:6813/2715951281","state":["exists","up"]},{"osd":6,"uuid":"ba4b5df7-d86e-47cd-ab67-cf763c4e4d6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":953261724},{"type":"v1","addr":"172.21.15.80:6819","nonce":953261724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":953261724},{"type":"v1","addr":"172.21.15.80:6821","nonce":953261724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":953261724},{"type":"v1","addr":"172.21.15.80:6825","nonce":953261724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":953261724},{"type":"v1","addr":"172.21.15.80:6823","nonce":953261724}]},"public_addr":"172.21.15.80:6819/953261724","cluster_addr":"172.21.15.80:6821/953261724","heartbeat_back_addr":"172.21.15.80:6825/953261724","heartbeat_front_addr":"172.21.15.80:6823/953261724","state":["exists","up"]},{"osd":7,"uuid":"464a503a-75ae-49e7-ab6e-a7f9f2a6a307","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":108354844},{"type":"v1","addr":"172.21.15.134:6817","nonce":108354844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":108354844},{"type":"v1","addr":"172.21.15.134:6819","nonce":108354844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":108354844},{"type":"v1","addr":"172.21.15.134:6823","nonce":108354844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":108354844},{"type":"v1","addr":"172.21.15.134:6821","nonce":108354844}]},"public_addr":"172.21.15.134:6817/108354844","cluster_addr":"172.21.15.134:6819/108354844","heartbeat_back_addr":"172.21.15.134:6823/108354844","heartbeat_front_addr":"172.21.15.134:6821/108354844","state":["exists","up"]},{"osd":8,"uuid":"ce544b2e-4540-47c0-9997-500c45351330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3559655329},{"type":"v1","addr":"172.21.15.148:6817","nonce":3559655329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":3559655329},{"type":"v1","addr":"172.21.15.148:6819","nonce":3559655329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":3559655329},{"type":"v1","addr":"172.21.15.148:6823","nonce":3559655329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3559655329},{"type":"v1","addr":"172.21.15.148:6821","nonce":3559655329}]},"public_addr":"172.21.15.148:6817/3559655329","cluster_addr":"172.21.15.148:6819/3559655329","heartbeat_back_addr":"172.21.15.148:6823/3559655329","heartbeat_front_addr":"172.21.15.148:6821/3559655329","state":["exists","up"]},{"osd":9,"uuid":"06de3732-19b9-496c-a464-55322efa6759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":4019913545},{"type":"v1","addr":"172.21.15.80:6827","nonce":4019913545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":4019913545},{"type":"v1","addr":"172.21.15.80:6829","nonce":4019913545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":4019913545},{"type":"v1","addr":"172.21.15.80:6833","nonce":4019913545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":4019913545},{"type":"v1","addr":"172.21.15.80:6831","nonce":4019913545}]},"public_addr":"172.21.15.80:6827/4019913545","cluster_addr":"172.21.15.80:6829/4019913545","heartbeat_back_addr":"172.21.15.80:6833/4019913545","heartbeat_front_addr":"172.21.15.80:6831/4019913545","state":["exists","up"]},{"osd":10,"uuid":"8d92f0d9-5c64-429a-b750-c33ae0e070c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":647343766},{"type":"v1","addr":"172.21.15.134:6825","nonce":647343766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":647343766},{"type":"v1","addr":"172.21.15.134:6827","nonce":647343766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":647343766},{"type":"v1","addr":"172.21.15.134:6831","nonce":647343766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":647343766},{"type":"v1","addr":"172.21.15.134:6829","nonce":647343766}]},"public_addr":"172.21.15.134:6825/647343766","cluster_addr":"172.21.15.134:6827/647343766","heartbeat_back_addr":"172.21.15.134:6831/647343766","heartbeat_front_addr":"172.21.15.134:6829/647343766","state":["exists","up"]},{"osd":11,"uuid":"4c003a41-f65b-479f-bc80-b27d680cf936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":4068024116},{"type":"v1","addr":"172.21.15.148:6825","nonce":4068024116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":4068024116},{"type":"v1","addr":"172.21.15.148:6827","nonce":4068024116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":4068024116},{"type":"v1","addr":"172.21.15.148:6831","nonce":4068024116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":4068024116},{"type":"v1","addr":"172.21.15.148:6829","nonce":4068024116}]},"public_addr":"172.21.15.148:6825/4068024116","cluster_addr":"172.21.15.148:6827/4068024116","heartbeat_back_addr":"172.21.15.148:6831/4068024116","heartbeat_front_addr":"172.21.15.148:6829/4068024116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:21:42.433631+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:22:06.145933+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:22:30.726143+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:22:55.729961+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:23:21.280913+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:23:46.609983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:24:12.585247+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:24:38.359287+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:25:04.491232+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:25:30.788089+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:25:56.891078+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:26:23.899113+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.80:0/2828378749":"2024-04-25T23:20:06.793609+0000","172.21.15.80:6801/233364790":"2024-04-25T23:20:06.793609+0000","172.21.15.80:6800/1614691410":"2024-04-25T23:19:40.490692+0000","172.21.15.80:0/1406187145":"2024-04-25T23:20:06.793609+0000","172.21.15.80:0/92953396":"2024-04-25T23:19:40.490692+0000","172.21.15.80:0/598129432":"2024-04-25T23:20:06.793609+0000","172.21.15.80:0/232193118":"2024-04-25T23:19:40.490692+0000","172.21.15.80:6800/233364790":"2024-04-25T23:20:06.793609+0000","172.21.15.80:6801/1614691410":"2024-04-25T23:19:40.490692+0000","172.21.15.80:0/2081562529":"2024-04-25T23:19:40.490692+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:28:00.793 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:00.792+0000 7f30313f1700 1 -- 172.21.15.80:0/259354176 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f30180a2cb0 msgr2=0x7f30180a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:00.793 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:00.792+0000 7f30313f1700 1 --2- 172.21.15.80:0/259354176 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f30180a2cb0 0x7f30180a5170 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f302c076a50 tx=0x7f301400b410 comp rx=0 tx=0).stop 2024-04-24T23:28:00.793 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:00.792+0000 7f30313f1700 1 -- 172.21.15.80:0/259354176 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f302c101d00 msgr2=0x7f302c102e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:00.793 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:00.792+0000 7f30313f1700 1 --2- 172.21.15.80:0/259354176 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f302c101d00 0x7f302c102e80 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f301c00c9b0 tx=0x7f301c00ccc0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.793 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:00.792+0000 7f30313f1700 1 -- 172.21.15.80:0/259354176 shutdown_connections 2024-04-24T23:28:00.793 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:00.792+0000 7f30313f1700 1 --2- 172.21.15.80:0/259354176 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f302c107420 0x7f302c103900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.793 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:00.792+0000 7f30313f1700 1 --2- 172.21.15.80:0/259354176 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f30180a2cb0 0x7f30180a5170 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.793 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:00.792+0000 7f30313f1700 1 --2- 172.21.15.80:0/259354176 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f302c101d00 0x7f302c102e80 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.793 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:00.792+0000 7f30313f1700 1 --2- 172.21.15.80:0/259354176 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f302c1026c0 0x7f302c1033c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:00.794 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:00.792+0000 7f30313f1700 1 -- 172.21.15.80:0/259354176 >> 172.21.15.80:0/259354176 conn(0x7f302c076450 msgr2=0x7f302c100cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:00.794 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:00.792+0000 7f30313f1700 1 -- 172.21.15.80:0/259354176 shutdown_connections 2024-04-24T23:28:00.794 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:00.792+0000 7f30313f1700 1 -- 172.21.15.80:0/259354176 wait complete. 2024-04-24T23:28:01.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:28:01 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/259354176' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:28:01.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:28:01 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/259354176' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:28:01.489 DEBUG:teuthology.orchestra.run.smithi080:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid ec7885aa-0290-11ef-bc93-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-04-24T23:28:01.490 DEBUG:teuthology.orchestra.run.smithi080:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid ec7885aa-0290-11ef-bc93-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-04-24T23:28:01.490 DEBUG:teuthology.orchestra.run.smithi080:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid ec7885aa-0290-11ef-bc93-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-04-24T23:28:01.491 DEBUG:teuthology.orchestra.run.smithi080:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid ec7885aa-0290-11ef-bc93-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-04-24T23:28:01.491 DEBUG:teuthology.orchestra.run.smithi080:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid ec7885aa-0290-11ef-bc93-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-04-24T23:28:01.491 DEBUG:teuthology.orchestra.run.smithi080:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid ec7885aa-0290-11ef-bc93-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-04-24T23:28:01.492 DEBUG:teuthology.orchestra.run.smithi080:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid ec7885aa-0290-11ef-bc93-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-04-24T23:28:01.492 DEBUG:teuthology.orchestra.run.smithi080:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid ec7885aa-0290-11ef-bc93-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-04-24T23:28:01.493 DEBUG:teuthology.orchestra.run.smithi080:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid ec7885aa-0290-11ef-bc93-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-04-24T23:28:01.493 DEBUG:teuthology.orchestra.run.smithi080:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid ec7885aa-0290-11ef-bc93-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-04-24T23:28:01.494 DEBUG:teuthology.orchestra.run.smithi080:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid ec7885aa-0290-11ef-bc93-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-04-24T23:28:01.494 DEBUG:teuthology.orchestra.run.smithi080:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid ec7885aa-0290-11ef-bc93-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-04-24T23:28:01.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:28:01 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/259354176' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:28:02.281 INFO:teuthology.orchestra.run.smithi080.stderr:Inferring config /var/lib/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:28:02.281 INFO:teuthology.orchestra.run.smithi080.stderr:Inferring config /var/lib/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:28:02.281 INFO:teuthology.orchestra.run.smithi080.stderr:Inferring config /var/lib/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:28:02.284 INFO:teuthology.orchestra.run.smithi080.stderr:Inferring config /var/lib/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:28:02.284 INFO:teuthology.orchestra.run.smithi080.stderr:Inferring config /var/lib/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:28:02.290 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:28:02 smithi080 ceph-mon[29152]: pgmap v317: 129 pgs: 129 active+clean; 636 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.8 KiB/s wr, 9 op/s 2024-04-24T23:28:02.290 INFO:teuthology.orchestra.run.smithi080.stderr:Inferring config /var/lib/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:28:02.291 INFO:teuthology.orchestra.run.smithi080.stderr:Inferring config /var/lib/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:28:02.291 INFO:teuthology.orchestra.run.smithi080.stderr:Inferring config /var/lib/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:28:02.294 INFO:teuthology.orchestra.run.smithi080.stderr:Inferring config /var/lib/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:28:02.297 INFO:teuthology.orchestra.run.smithi080.stderr:Inferring config /var/lib/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:28:02.297 INFO:teuthology.orchestra.run.smithi080.stderr:Inferring config /var/lib/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:28:02.303 INFO:teuthology.orchestra.run.smithi080.stderr:Inferring config /var/lib/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:28:02.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:28:02 smithi134 ceph-mon[32619]: pgmap v317: 129 pgs: 129 active+clean; 636 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.8 KiB/s wr, 9 op/s 2024-04-24T23:28:02.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:28:02 smithi148 ceph-mon[32528]: pgmap v317: 129 pgs: 129 active+clean; 636 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.8 KiB/s wr, 9 op/s 2024-04-24T23:28:04.281 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:28:04 smithi080 ceph-mon[29152]: pgmap v318: 129 pgs: 129 active+clean; 636 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.8 KiB/s wr, 6 op/s 2024-04-24T23:28:04.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:28:04 smithi134 ceph-mon[32619]: pgmap v318: 129 pgs: 129 active+clean; 636 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.8 KiB/s wr, 6 op/s 2024-04-24T23:28:04.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:28:04 smithi148 ceph-mon[32528]: pgmap v318: 129 pgs: 129 active+clean; 636 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.8 KiB/s wr, 6 op/s 2024-04-24T23:28:06.270 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:28:06 smithi080 ceph-mon[29152]: pgmap v319: 129 pgs: 129 active+clean; 636 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-04-24T23:28:06.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:28:06 smithi134 ceph-mon[32619]: pgmap v319: 129 pgs: 129 active+clean; 636 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-04-24T23:28:06.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:28:06 smithi148 ceph-mon[32528]: pgmap v319: 129 pgs: 129 active+clean; 636 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-04-24T23:28:07.121 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:28:07 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]: dispatch 2024-04-24T23:28:07.121 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:28:07 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 1]}]: dispatch 2024-04-24T23:28:07.121 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:28:07 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-04-24T23:28:07.121 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:28:07 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 11]}]: dispatch 2024-04-24T23:28:07.121 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:28:07 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]: dispatch 2024-04-24T23:28:07.121 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:28:07 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]: dispatch 2024-04-24T23:28:07.121 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:28:07 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]: dispatch 2024-04-24T23:28:07.121 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:28:07 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 7]}]: dispatch 2024-04-24T23:28:07.121 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:28:07 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]: dispatch 2024-04-24T23:28:07.122 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:28:07 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]: dispatch 2024-04-24T23:28:07.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:28:07 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]: dispatch 2024-04-24T23:28:07.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:28:07 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 1]}]: dispatch 2024-04-24T23:28:07.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:28:07 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-04-24T23:28:07.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:28:07 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 11]}]: dispatch 2024-04-24T23:28:07.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:28:07 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]: dispatch 2024-04-24T23:28:07.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:28:07 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]: dispatch 2024-04-24T23:28:07.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:28:07 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]: dispatch 2024-04-24T23:28:07.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:28:07 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 7]}]: dispatch 2024-04-24T23:28:07.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:28:07 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]: dispatch 2024-04-24T23:28:07.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:28:07 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]: dispatch 2024-04-24T23:28:07.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:28:07 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]: dispatch 2024-04-24T23:28:07.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:28:07 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 1]}]: dispatch 2024-04-24T23:28:07.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:28:07 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-04-24T23:28:07.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:28:07 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 11]}]: dispatch 2024-04-24T23:28:07.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:28:07 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]: dispatch 2024-04-24T23:28:07.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:28:07 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]: dispatch 2024-04-24T23:28:07.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:28:07 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]: dispatch 2024-04-24T23:28:07.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:28:07 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 7]}]: dispatch 2024-04-24T23:28:07.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:28:07 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]: dispatch 2024-04-24T23:28:07.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:28:07 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]: dispatch 2024-04-24T23:28:08.361 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:28:08 smithi080 ceph-mon[29152]: pgmap v320: 129 pgs: 129 active+clean; 636 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-04-24T23:28:08.361 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:28:08 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2024-04-24T23:28:08.361 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:28:08 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 1]}]': finished 2024-04-24T23:28:08.361 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:28:08 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-04-24T23:28:08.361 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:28:08 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 11]}]': finished 2024-04-24T23:28:08.361 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:28:08 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]': finished 2024-04-24T23:28:08.361 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:28:08 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]': finished 2024-04-24T23:28:08.362 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:28:08 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]': finished 2024-04-24T23:28:08.362 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:28:08 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 7]}]': finished 2024-04-24T23:28:08.362 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:28:08 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2024-04-24T23:28:08.362 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:28:08 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]': finished 2024-04-24T23:28:08.362 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:28:08 smithi080 ceph-mon[29152]: osdmap e91: 12 total, 12 up, 12 in 2024-04-24T23:28:08.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:28:08 smithi134 ceph-mon[32619]: pgmap v320: 129 pgs: 129 active+clean; 636 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-04-24T23:28:08.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:28:08 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2024-04-24T23:28:08.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:28:08 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 1]}]': finished 2024-04-24T23:28:08.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:28:08 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-04-24T23:28:08.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:28:08 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 11]}]': finished 2024-04-24T23:28:08.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:28:08 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]': finished 2024-04-24T23:28:08.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:28:08 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]': finished 2024-04-24T23:28:08.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:28:08 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]': finished 2024-04-24T23:28:08.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:28:08 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 7]}]': finished 2024-04-24T23:28:08.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:28:08 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2024-04-24T23:28:08.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:28:08 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]': finished 2024-04-24T23:28:08.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:28:08 smithi134 ceph-mon[32619]: osdmap e91: 12 total, 12 up, 12 in 2024-04-24T23:28:08.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:28:08 smithi148 ceph-mon[32528]: pgmap v320: 129 pgs: 129 active+clean; 636 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-04-24T23:28:08.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:28:08 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2024-04-24T23:28:08.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:28:08 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 1]}]': finished 2024-04-24T23:28:08.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:28:08 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-04-24T23:28:08.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:28:08 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 11]}]': finished 2024-04-24T23:28:08.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:28:08 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]': finished 2024-04-24T23:28:08.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:28:08 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]': finished 2024-04-24T23:28:08.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:28:08 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]': finished 2024-04-24T23:28:08.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:28:08 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 7]}]': finished 2024-04-24T23:28:08.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:28:08 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2024-04-24T23:28:08.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:28:08 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]': finished 2024-04-24T23:28:08.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:28:08 smithi148 ceph-mon[32528]: osdmap e91: 12 total, 12 up, 12 in 2024-04-24T23:28:09.202 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:28:09 smithi080 ceph-mon[29152]: osdmap e92: 12 total, 12 up, 12 in 2024-04-24T23:28:09.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:28:09 smithi134 ceph-mon[32619]: osdmap e92: 12 total, 12 up, 12 in 2024-04-24T23:28:09.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:28:09 smithi148 ceph-mon[32528]: osdmap e92: 12 total, 12 up, 12 in 2024-04-24T23:28:10.407 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:28:10 smithi148 ceph-mon[32528]: pgmap v323: 129 pgs: 2 peering, 127 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 383 B/s wr, 6 op/s 2024-04-24T23:28:10.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:28:10 smithi134 ceph-mon[32619]: pgmap v323: 129 pgs: 2 peering, 127 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 383 B/s wr, 6 op/s 2024-04-24T23:28:10.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:28:10 smithi080 ceph-mon[29152]: pgmap v323: 129 pgs: 2 peering, 127 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 383 B/s wr, 6 op/s 2024-04-24T23:28:10.839 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:10.838+0000 7fb1770b9700 1 -- 172.21.15.80:0/2381754545 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb1701071a0 msgr2=0x7fb170107580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:10.840 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:10.838+0000 7fb1770b9700 1 --2- 172.21.15.80:0/2381754545 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb1701071a0 0x7fb170107580 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fb164009990 tx=0x7fb164009ca0 comp rx=0 tx=0).stop 2024-04-24T23:28:10.840 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:10.838+0000 7fb1770b9700 1 -- 172.21.15.80:0/2381754545 shutdown_connections 2024-04-24T23:28:10.840 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:10.838+0000 7fb1770b9700 1 --2- 172.21.15.80:0/2381754545 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb170101af0 0x7fb17010dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:10.840 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:10.838+0000 7fb1770b9700 1 --2- 172.21.15.80:0/2381754545 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb1701071a0 0x7fb170107580 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:10.840 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:10.838+0000 7fb1770b9700 1 --2- 172.21.15.80:0/2381754545 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb170101130 0x7fb1701015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:10.840 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:10.838+0000 7fb1770b9700 1 -- 172.21.15.80:0/2381754545 >> 172.21.15.80:0/2381754545 conn(0x7fb1700780e0 msgr2=0x7fb1700784f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:10.840 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:10.838+0000 7fb1770b9700 1 -- 172.21.15.80:0/2381754545 shutdown_connections 2024-04-24T23:28:10.840 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:10.838+0000 7fb1770b9700 1 -- 172.21.15.80:0/2381754545 wait complete. 2024-04-24T23:28:10.842 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:10.839+0000 7fb1770b9700 1 Processor -- start 2024-04-24T23:28:10.842 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:10.839+0000 7fb1770b9700 1 -- start start 2024-04-24T23:28:10.842 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:10.839+0000 7fb1770b9700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb170101130 0x7fb170102e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:10.842 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:10.839+0000 7fb1770b9700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb170101af0 0x7fb170103350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:10.842 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:10.839+0000 7fb1770b9700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb1701071a0 0x7fb170103890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:10.842 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:10.839+0000 7fb1770b9700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fb170111490 con 0x7fb170101af0 2024-04-24T23:28:10.843 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:10.839+0000 7fb1770b9700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb170111310 con 0x7fb170101130 2024-04-24T23:28:10.843 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:10.839+0000 7fb1770b9700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb170111610 con 0x7fb1701071a0 2024-04-24T23:28:10.843 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:10.839+0000 7fb16ffff700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb170101af0 0x7fb170103350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:10.843 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:10.839+0000 7fb174e55700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb170101130 0x7fb170102e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:10.843 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:10.839+0000 7fb16ffff700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb170101af0 0x7fb170103350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:60928/0 (socket says 172.21.15.80:60928) 2024-04-24T23:28:10.843 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:10.839+0000 7fb16ffff700 1 -- 172.21.15.80:0/867921459 learned_addr learned my addr 172.21.15.80:0/867921459 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:28:10.843 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:10.840+0000 7fb175656700 1 --2- 172.21.15.80:0/867921459 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb1701071a0 0x7fb170103890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:10.843 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:10.840+0000 7fb16ffff700 1 -- 172.21.15.80:0/867921459 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb1701071a0 msgr2=0x7fb170103890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:10.843 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:10.840+0000 7fb16ffff700 1 --2- 172.21.15.80:0/867921459 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb1701071a0 0x7fb170103890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:10.843 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:10.840+0000 7fb16ffff700 1 -- 172.21.15.80:0/867921459 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb170101130 msgr2=0x7fb170102e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:10.843 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:10.840+0000 7fb16ffff700 1 --2- 172.21.15.80:0/867921459 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb170101130 0x7fb170102e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:10.843 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:10.840+0000 7fb16ffff700 1 -- 172.21.15.80:0/867921459 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1701a6f90 con 0x7fb170101af0 2024-04-24T23:28:10.844 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:10.840+0000 7fb175656700 1 --2- 172.21.15.80:0/867921459 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb1701071a0 0x7fb170103890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:28:10.844 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:10.840+0000 7fb174e55700 1 --2- 172.21.15.80:0/867921459 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb170101130 0x7fb170102e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:28:10.844 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:10.840+0000 7fb16ffff700 1 --2- 172.21.15.80:0/867921459 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb170101af0 0x7fb170103350 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fb15c00b530 tx=0x7fb15c00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:10.844 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:10.840+0000 7fb16dffb700 1 -- 172.21.15.80:0/867921459 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb15c015020 con 0x7fb170101af0 2024-04-24T23:28:10.844 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:10.840+0000 7fb1770b9700 1 -- 172.21.15.80:0/867921459 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1701a7280 con 0x7fb170101af0 2024-04-24T23:28:10.845 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:10.840+0000 7fb16dffb700 1 -- 172.21.15.80:0/867921459 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb15c011d10 con 0x7fb170101af0 2024-04-24T23:28:10.845 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:10.840+0000 7fb16dffb700 1 -- 172.21.15.80:0/867921459 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb15c00f2e0 con 0x7fb170101af0 2024-04-24T23:28:10.845 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:10.840+0000 7fb1770b9700 1 -- 172.21.15.80:0/867921459 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1701a77c0 con 0x7fb170101af0 2024-04-24T23:28:10.845 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:10.841+0000 7fb1770b9700 1 -- 172.21.15.80:0/867921459 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb170105220 con 0x7fb170101af0 2024-04-24T23:28:10.847 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:10.842+0000 7fb16dffb700 1 -- 172.21.15.80:0/867921459 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fb15c018020 con 0x7fb170101af0 2024-04-24T23:28:10.847 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:10.843+0000 7fb16dffb700 1 --2- 172.21.15.80:0/867921459 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fb1600a2cb0 0x7fb1600a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:10.847 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:10.843+0000 7fb174e55700 1 --2- 172.21.15.80:0/867921459 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fb1600a2cb0 0x7fb1600a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:10.847 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:10.843+0000 7fb16dffb700 1 -- 172.21.15.80:0/867921459 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fb15c0cf2f0 con 0x7fb170101af0 2024-04-24T23:28:10.847 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:10.843+0000 7fb16dffb700 1 --2- 172.21.15.80:0/867921459 >> [v2:172.21.15.148:6808/2715951281,v1:172.21.15.148:6809/2715951281] conn(0x7fb1600ade10 0x7fb1600b02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:10.847 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:10.843+0000 7fb16dffb700 1 -- 172.21.15.80:0/867921459 --> [v2:172.21.15.148:6808/2715951281,v1:172.21.15.148:6809/2715951281] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb15c013ec0 con 0x7fb1600ade10 2024-04-24T23:28:10.848 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:10.843+0000 7fb16dffb700 1 -- 172.21.15.80:0/867921459 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7fb15c0cf740 con 0x7fb170101af0 2024-04-24T23:28:10.848 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:10.843+0000 7fb174e55700 1 --2- 172.21.15.80:0/867921459 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fb1600a2cb0 0x7fb1600a5170 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb16400b5c0 tx=0x7fb164005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:10.848 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:10.844+0000 7fb175656700 1 --2- 172.21.15.80:0/867921459 >> [v2:172.21.15.148:6808/2715951281,v1:172.21.15.148:6809/2715951281] conn(0x7fb1600ade10 0x7fb1600b02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:10.848 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:10.844+0000 7fb175656700 1 --2- 172.21.15.80:0/867921459 >> [v2:172.21.15.148:6808/2715951281,v1:172.21.15.148:6809/2715951281] conn(0x7fb1600ade10 0x7fb1600b02b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:10.848 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:10.845+0000 7fb16dffb700 1 -- 172.21.15.80:0/867921459 <== osd.5 v2:172.21.15.148:6808/2715951281 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb15c013ec0 con 0x7fb1600ade10 2024-04-24T23:28:10.861 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:10.861+0000 7fb1770b9700 1 -- 172.21.15.80:0/867921459 --> [v2:172.21.15.148:6808/2715951281,v1:172.21.15.148:6809/2715951281] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb170102850 con 0x7fb1600ade10 2024-04-24T23:28:10.862 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:10.861+0000 7fb16dffb700 1 -- 172.21.15.80:0/867921459 <== osd.5 v2:172.21.15.148:6808/2715951281 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fb170102850 con 0x7fb1600ade10 2024-04-24T23:28:10.863 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:10.862+0000 7fb1770b9700 1 -- 172.21.15.80:0/867921459 >> [v2:172.21.15.148:6808/2715951281,v1:172.21.15.148:6809/2715951281] conn(0x7fb1600ade10 msgr2=0x7fb1600b02b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:10.863 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:10.862+0000 7fb1770b9700 1 --2- 172.21.15.80:0/867921459 >> [v2:172.21.15.148:6808/2715951281,v1:172.21.15.148:6809/2715951281] conn(0x7fb1600ade10 0x7fb1600b02b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:10.863 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:10.862+0000 7fb1770b9700 1 -- 172.21.15.80:0/867921459 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fb1600a2cb0 msgr2=0x7fb1600a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:10.863 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:10.862+0000 7fb1770b9700 1 --2- 172.21.15.80:0/867921459 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fb1600a2cb0 0x7fb1600a5170 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb16400b5c0 tx=0x7fb164005960 comp rx=0 tx=0).stop 2024-04-24T23:28:10.863 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:10.862+0000 7fb1770b9700 1 -- 172.21.15.80:0/867921459 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb170101af0 msgr2=0x7fb170103350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:10.863 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:10.862+0000 7fb1770b9700 1 --2- 172.21.15.80:0/867921459 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb170101af0 0x7fb170103350 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fb15c00b530 tx=0x7fb15c00b840 comp rx=0 tx=0).stop 2024-04-24T23:28:10.863 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:10.862+0000 7fb175656700 1 -- 172.21.15.80:0/867921459 reap_dead start 2024-04-24T23:28:10.863 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:10.862+0000 7fb1770b9700 1 -- 172.21.15.80:0/867921459 shutdown_connections 2024-04-24T23:28:10.863 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:10.862+0000 7fb1770b9700 1 -- 172.21.15.80:0/867921459 >> 172.21.15.80:0/867921459 conn(0x7fb1700780e0 msgr2=0x7fb170100460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:10.864 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:10.862+0000 7fb1770b9700 1 -- 172.21.15.80:0/867921459 shutdown_connections 2024-04-24T23:28:10.864 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:10.862+0000 7fb1770b9700 1 -- 172.21.15.80:0/867921459 wait complete. 2024-04-24T23:28:12.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:28:12 smithi080 ceph-mon[29152]: pgmap v324: 129 pgs: 2 peering, 127 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 383 B/s wr, 7 op/s; 0 B/s, 0 objects/s recovering 2024-04-24T23:28:12.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:28:12 smithi134 ceph-mon[32619]: pgmap v324: 129 pgs: 2 peering, 127 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 383 B/s wr, 7 op/s; 0 B/s, 0 objects/s recovering 2024-04-24T23:28:12.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:28:12 smithi148 ceph-mon[32528]: pgmap v324: 129 pgs: 2 peering, 127 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 383 B/s wr, 7 op/s; 0 B/s, 0 objects/s recovering 2024-04-24T23:28:13.678 INFO:teuthology.orchestra.run.smithi080.stdout:163208757304 2024-04-24T23:28:13.678 DEBUG:teuthology.orchestra.run.smithi080:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid ec7885aa-0290-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-04-24T23:28:13.786 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:11.048+0000 7f68a3727700 1 -- 172.21.15.80:0/2496279523 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f689c101b10 msgr2=0x7f689c10dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:13.786 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:11.048+0000 7f68a3727700 1 --2- 172.21.15.80:0/2496279523 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f689c101b10 0x7f689c10dff0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f689400b0a0 tx=0x7f689400b3b0 comp rx=0 tx=0).stop 2024-04-24T23:28:13.786 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:11.048+0000 7f68a3727700 1 -- 172.21.15.80:0/2496279523 shutdown_connections 2024-04-24T23:28:13.786 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:11.048+0000 7f68a3727700 1 --2- 172.21.15.80:0/2496279523 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f689c101b10 0x7f689c10dff0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:13.786 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:11.048+0000 7f68a3727700 1 --2- 172.21.15.80:0/2496279523 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f689c1071b0 0x7f689c107590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:13.786 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:11.048+0000 7f68a3727700 1 --2- 172.21.15.80:0/2496279523 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f689c101170 0x7f689c1015d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:13.786 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:11.048+0000 7f68a3727700 1 -- 172.21.15.80:0/2496279523 >> 172.21.15.80:0/2496279523 conn(0x7f689c078060 msgr2=0x7f689c078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:13.787 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:11.048+0000 7f68a3727700 1 -- 172.21.15.80:0/2496279523 shutdown_connections 2024-04-24T23:28:13.787 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:11.048+0000 7f68a3727700 1 -- 172.21.15.80:0/2496279523 wait complete. 2024-04-24T23:28:13.787 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:11.048+0000 7f68a3727700 1 Processor -- start 2024-04-24T23:28:13.787 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:11.048+0000 7f68a3727700 1 -- start start 2024-04-24T23:28:13.787 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:11.048+0000 7f68a3727700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f689c101170 0x7f689c102e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:13.787 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:11.048+0000 7f68a3727700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f689c101b10 0x7f689c103360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:13.787 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:11.048+0000 7f68a3727700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f689c1071b0 0x7f689c1038a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:13.787 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:11.049+0000 7f68a3727700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f689c111400 con 0x7f689c101b10 2024-04-24T23:28:13.787 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:11.049+0000 7f68a3727700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f689c111280 con 0x7f689c1071b0 2024-04-24T23:28:13.787 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:11.049+0000 7f68a3727700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f689c111580 con 0x7f689c101170 2024-04-24T23:28:13.787 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:11.049+0000 7f68a0cc2700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f689c101b10 0x7f689c103360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:13.787 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:11.049+0000 7f68a0cc2700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f689c101b10 0x7f689c103360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:60956/0 (socket says 172.21.15.80:60956) 2024-04-24T23:28:13.788 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:11.049+0000 7f68a0cc2700 1 -- 172.21.15.80:0/3605573142 learned_addr learned my addr 172.21.15.80:0/3605573142 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:28:13.788 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:11.049+0000 7f68a1cc4700 1 --2- 172.21.15.80:0/3605573142 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f689c1071b0 0x7f689c1038a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:13.788 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:11.049+0000 7f68a14c3700 1 --2- 172.21.15.80:0/3605573142 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f689c101170 0x7f689c102e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:13.788 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:11.049+0000 7f68a0cc2700 1 -- 172.21.15.80:0/3605573142 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f689c101170 msgr2=0x7f689c102e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:13.788 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:11.049+0000 7f68a0cc2700 1 --2- 172.21.15.80:0/3605573142 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f689c101170 0x7f689c102e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:13.788 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:11.049+0000 7f68a0cc2700 1 -- 172.21.15.80:0/3605573142 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f689c1071b0 msgr2=0x7f689c1038a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:13.788 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:11.049+0000 7f68a0cc2700 1 --2- 172.21.15.80:0/3605573142 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f689c1071b0 0x7f689c1038a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:13.788 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:11.049+0000 7f68a0cc2700 1 -- 172.21.15.80:0/3605573142 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f689c1a6ed0 con 0x7f689c101b10 2024-04-24T23:28:13.788 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:11.049+0000 7f68a0cc2700 1 --2- 172.21.15.80:0/3605573142 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f689c101b10 0x7f689c103360 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f688c00d950 tx=0x7f688c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:13.788 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:11.049+0000 7f68927fc700 1 -- 172.21.15.80:0/3605573142 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f688c016070 con 0x7f689c101b10 2024-04-24T23:28:13.788 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:11.049+0000 7f68a3727700 1 -- 172.21.15.80:0/3605573142 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f689c1a71c0 con 0x7f689c101b10 2024-04-24T23:28:13.788 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:11.049+0000 7f68927fc700 1 -- 172.21.15.80:0/3605573142 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f688c009e20 con 0x7f689c101b10 2024-04-24T23:28:13.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:11.049+0000 7f68927fc700 1 -- 172.21.15.80:0/3605573142 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f688c00f620 con 0x7f689c101b10 2024-04-24T23:28:13.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:11.049+0000 7f68a3727700 1 -- 172.21.15.80:0/3605573142 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f689c1a7750 con 0x7f689c101b10 2024-04-24T23:28:13.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:11.050+0000 7f68a3727700 1 -- 172.21.15.80:0/3605573142 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f689c1051f0 con 0x7f689c101b10 2024-04-24T23:28:13.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:11.051+0000 7f68927fc700 1 -- 172.21.15.80:0/3605573142 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f688c019070 con 0x7f689c101b10 2024-04-24T23:28:13.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:11.052+0000 7f68927fc700 1 --2- 172.21.15.80:0/3605573142 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f68880a2cb0 0x7f68880a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:13.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:11.052+0000 7f68a14c3700 1 --2- 172.21.15.80:0/3605573142 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f68880a2cb0 0x7f68880a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:13.790 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:11.052+0000 7f68927fc700 1 -- 172.21.15.80:0/3605573142 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f688c0cf620 con 0x7f689c101b10 2024-04-24T23:28:13.790 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:11.052+0000 7f68927fc700 1 --2- 172.21.15.80:0/3605573142 >> [v2:172.21.15.80:6826/4019913545,v1:172.21.15.80:6827/4019913545] conn(0x7f68880ade10 0x7f68880b02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:13.790 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:11.052+0000 7f68927fc700 1 -- 172.21.15.80:0/3605573142 --> [v2:172.21.15.80:6826/4019913545,v1:172.21.15.80:6827/4019913545] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f688c010e50 con 0x7f68880ade10 2024-04-24T23:28:13.790 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:11.052+0000 7f68927fc700 1 -- 172.21.15.80:0/3605573142 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f688c0cfa70 con 0x7f689c101b10 2024-04-24T23:28:13.790 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:11.052+0000 7f68a1cc4700 1 --2- 172.21.15.80:0/3605573142 >> [v2:172.21.15.80:6826/4019913545,v1:172.21.15.80:6827/4019913545] conn(0x7f68880ade10 0x7f68880b02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:13.790 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:11.052+0000 7f68a14c3700 1 --2- 172.21.15.80:0/3605573142 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f68880a2cb0 0x7f68880a5170 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f6898005950 tx=0x7f68980058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:13.790 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:11.053+0000 7f68a1cc4700 1 --2- 172.21.15.80:0/3605573142 >> [v2:172.21.15.80:6826/4019913545,v1:172.21.15.80:6827/4019913545] conn(0x7f68880ade10 0x7f68880b02b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:13.790 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:11.053+0000 7f68927fc700 1 -- 172.21.15.80:0/3605573142 <== osd.9 v2:172.21.15.80:6826/4019913545 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f688c010e50 con 0x7f68880ade10 2024-04-24T23:28:13.790 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:11.070+0000 7f68a3727700 1 -- 172.21.15.80:0/3605573142 --> [v2:172.21.15.80:6826/4019913545,v1:172.21.15.80:6827/4019913545] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f689c102860 con 0x7f68880ade10 2024-04-24T23:28:13.791 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:11.070+0000 7f68927fc700 1 -- 172.21.15.80:0/3605573142 <== osd.9 v2:172.21.15.80:6826/4019913545 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f689c102860 con 0x7f68880ade10 2024-04-24T23:28:13.791 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:11.070+0000 7f68a3727700 1 -- 172.21.15.80:0/3605573142 >> [v2:172.21.15.80:6826/4019913545,v1:172.21.15.80:6827/4019913545] conn(0x7f68880ade10 msgr2=0x7f68880b02b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:13.791 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:11.070+0000 7f68a3727700 1 --2- 172.21.15.80:0/3605573142 >> [v2:172.21.15.80:6826/4019913545,v1:172.21.15.80:6827/4019913545] conn(0x7f68880ade10 0x7f68880b02b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:13.791 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:11.071+0000 7f68a3727700 1 -- 172.21.15.80:0/3605573142 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f68880a2cb0 msgr2=0x7f68880a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:13.791 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:11.071+0000 7f68a3727700 1 --2- 172.21.15.80:0/3605573142 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f68880a2cb0 0x7f68880a5170 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f6898005950 tx=0x7f68980058e0 comp rx=0 tx=0).stop 2024-04-24T23:28:13.791 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:11.071+0000 7f68a3727700 1 -- 172.21.15.80:0/3605573142 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f689c101b10 msgr2=0x7f689c103360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:13.791 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:11.071+0000 7f68a3727700 1 --2- 172.21.15.80:0/3605573142 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f689c101b10 0x7f689c103360 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f688c00d950 tx=0x7f688c00dc60 comp rx=0 tx=0).stop 2024-04-24T23:28:13.791 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:11.071+0000 7f68a1cc4700 1 -- 172.21.15.80:0/3605573142 reap_dead start 2024-04-24T23:28:13.791 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:11.071+0000 7f68a3727700 1 -- 172.21.15.80:0/3605573142 shutdown_connections 2024-04-24T23:28:13.791 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:11.071+0000 7f68a3727700 1 -- 172.21.15.80:0/3605573142 >> 172.21.15.80:0/3605573142 conn(0x7f689c078060 msgr2=0x7f689c100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:13.792 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:11.071+0000 7f68a3727700 1 -- 172.21.15.80:0/3605573142 shutdown_connections 2024-04-24T23:28:13.792 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:11.071+0000 7f68a3727700 1 -- 172.21.15.80:0/3605573142 wait complete. 2024-04-24T23:28:14.415 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:28:14 smithi080 ceph-mon[29152]: pgmap v325: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 383 B/s wr, 4 op/s; 86 B/s, 1 keys/s, 1 objects/s recovering 2024-04-24T23:28:14.441 INFO:teuthology.orchestra.run.smithi080.stdout:287762808867 2024-04-24T23:28:14.442 DEBUG:teuthology.orchestra.run.smithi080:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid ec7885aa-0290-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-04-24T23:28:14.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:28:14 smithi134 ceph-mon[32619]: pgmap v325: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 383 B/s wr, 4 op/s; 86 B/s, 1 keys/s, 1 objects/s recovering 2024-04-24T23:28:14.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:28:14 smithi148 ceph-mon[32528]: pgmap v325: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 383 B/s wr, 4 op/s; 86 B/s, 1 keys/s, 1 objects/s recovering 2024-04-24T23:28:16.404 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:28:16 smithi080 ceph-mon[29152]: pgmap v326: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 383 B/s wr, 7 op/s; 86 B/s, 1 keys/s, 1 objects/s recovering 2024-04-24T23:28:16.428 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:16.403+0000 7fea6c321700 1 -- 172.21.15.80:0/337739342 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fea64107540 msgr2=0x7fea6410f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:16.428 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:16.403+0000 7fea6c321700 1 --2- 172.21.15.80:0/337739342 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fea64107540 0x7fea6410f110 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fea5c00b0d0 tx=0x7fea5c00b3e0 comp rx=0 tx=0).stop 2024-04-24T23:28:16.428 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:16.403+0000 7fea6c321700 1 -- 172.21.15.80:0/337739342 shutdown_connections 2024-04-24T23:28:16.428 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:16.403+0000 7fea6c321700 1 --2- 172.21.15.80:0/337739342 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fea64107540 0x7fea6410f110 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:16.428 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:16.403+0000 7fea6c321700 1 --2- 172.21.15.80:0/337739342 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea64102550 0x7fea64102930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:16.429 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:16.403+0000 7fea6c321700 1 --2- 172.21.15.80:0/337739342 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fea64102f00 0x7fea64106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:16.429 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:16.403+0000 7fea6c321700 1 -- 172.21.15.80:0/337739342 >> 172.21.15.80:0/337739342 conn(0x7fea640793f0 msgr2=0x7fea64079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:16.429 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:16.403+0000 7fea6c321700 1 -- 172.21.15.80:0/337739342 shutdown_connections 2024-04-24T23:28:16.429 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:16.403+0000 7fea6c321700 1 -- 172.21.15.80:0/337739342 wait complete. 2024-04-24T23:28:16.429 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:16.404+0000 7fea6c321700 1 Processor -- start 2024-04-24T23:28:16.429 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:16.404+0000 7fea6c321700 1 -- start start 2024-04-24T23:28:16.429 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:16.404+0000 7fea6c321700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fea64102550 0x7fea6419ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:16.429 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:16.404+0000 7fea6c321700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fea64102f00 0x7fea6419b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:16.429 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:16.404+0000 7fea6c321700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea64107540 0x7fea6419f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:16.429 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:16.404+0000 7fea6c321700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fea64112620 con 0x7fea64102550 2024-04-24T23:28:16.429 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:16.404+0000 7fea6c321700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fea641124a0 con 0x7fea64102f00 2024-04-24T23:28:16.429 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:16.404+0000 7fea6c321700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fea641127a0 con 0x7fea64107540 2024-04-24T23:28:16.430 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:16.404+0000 7fea6a0bd700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fea64102550 0x7fea6419ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:16.430 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:16.404+0000 7fea6a0bd700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fea64102550 0x7fea6419ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:43572/0 (socket says 172.21.15.80:43572) 2024-04-24T23:28:16.430 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:16.404+0000 7fea6a0bd700 1 -- 172.21.15.80:0/1233177886 learned_addr learned my addr 172.21.15.80:0/1233177886 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:28:16.430 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:16.404+0000 7fea698bc700 1 --2- 172.21.15.80:0/1233177886 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fea64102f00 0x7fea6419b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:16.430 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:16.404+0000 7fea6a8be700 1 --2- 172.21.15.80:0/1233177886 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea64107540 0x7fea6419f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:16.430 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:16.405+0000 7fea6a0bd700 1 -- 172.21.15.80:0/1233177886 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea64107540 msgr2=0x7fea6419f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:16.430 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:16.405+0000 7fea6a0bd700 1 --2- 172.21.15.80:0/1233177886 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea64107540 0x7fea6419f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:16.430 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:16.405+0000 7fea6a0bd700 1 -- 172.21.15.80:0/1233177886 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fea64102f00 msgr2=0x7fea6419b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:16.430 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:16.405+0000 7fea6a0bd700 1 --2- 172.21.15.80:0/1233177886 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fea64102f00 0x7fea6419b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:16.430 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:16.405+0000 7fea6a0bd700 1 -- 172.21.15.80:0/1233177886 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea6419fd40 con 0x7fea64102550 2024-04-24T23:28:16.431 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:16.405+0000 7fea6a0bd700 1 --2- 172.21.15.80:0/1233177886 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fea64102550 0x7fea6419ad30 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fea6000b530 tx=0x7fea6000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:16.431 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:16.405+0000 7fea5b7fe700 1 -- 172.21.15.80:0/1233177886 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea60015020 con 0x7fea64102550 2024-04-24T23:28:16.431 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:16.405+0000 7fea6c321700 1 -- 172.21.15.80:0/1233177886 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea641a0030 con 0x7fea64102550 2024-04-24T23:28:16.431 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:16.405+0000 7fea5b7fe700 1 -- 172.21.15.80:0/1233177886 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea60011d10 con 0x7fea64102550 2024-04-24T23:28:16.431 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:16.405+0000 7fea5b7fe700 1 -- 172.21.15.80:0/1233177886 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea6000f2e0 con 0x7fea64102550 2024-04-24T23:28:16.431 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:16.405+0000 7fea6c321700 1 -- 172.21.15.80:0/1233177886 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea641a78b0 con 0x7fea64102550 2024-04-24T23:28:16.431 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:16.406+0000 7fea6c321700 1 -- 172.21.15.80:0/1233177886 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fea640757c0 con 0x7fea64102550 2024-04-24T23:28:16.431 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:16.407+0000 7fea5b7fe700 1 -- 172.21.15.80:0/1233177886 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fea60018020 con 0x7fea64102550 2024-04-24T23:28:16.432 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:16.408+0000 7fea5b7fe700 1 --2- 172.21.15.80:0/1233177886 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fea500a2cb0 0x7fea500a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:16.432 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:16.408+0000 7fea698bc700 1 --2- 172.21.15.80:0/1233177886 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fea500a2cb0 0x7fea500a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:16.432 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:16.408+0000 7fea5b7fe700 1 -- 172.21.15.80:0/1233177886 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fea600cf090 con 0x7fea64102550 2024-04-24T23:28:16.432 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:16.408+0000 7fea5b7fe700 1 --2- 172.21.15.80:0/1233177886 >> [v2:172.21.15.148:6824/4068024116,v1:172.21.15.148:6825/4068024116] conn(0x7fea500ade10 0x7fea500b02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:16.432 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:16.408+0000 7fea5b7fe700 1 -- 172.21.15.80:0/1233177886 --> [v2:172.21.15.148:6824/4068024116,v1:172.21.15.148:6825/4068024116] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fea500b0980 con 0x7fea500ade10 2024-04-24T23:28:16.432 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:16.408+0000 7fea5b7fe700 1 -- 172.21.15.80:0/1233177886 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7fea600cf4e0 con 0x7fea64102550 2024-04-24T23:28:16.432 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:16.408+0000 7fea698bc700 1 --2- 172.21.15.80:0/1233177886 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fea500a2cb0 0x7fea500a5170 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fea64108df0 tx=0x7fea54009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:16.432 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:16.408+0000 7fea6a8be700 1 --2- 172.21.15.80:0/1233177886 >> [v2:172.21.15.148:6824/4068024116,v1:172.21.15.148:6825/4068024116] conn(0x7fea500ade10 0x7fea500b02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:16.432 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:16.409+0000 7fea6a8be700 1 --2- 172.21.15.80:0/1233177886 >> [v2:172.21.15.148:6824/4068024116,v1:172.21.15.148:6825/4068024116] conn(0x7fea500ade10 0x7fea500b02b0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:16.432 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:16.410+0000 7fea5b7fe700 1 -- 172.21.15.80:0/1233177886 <== osd.11 v2:172.21.15.148:6824/4068024116 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fea500b0980 con 0x7fea500ade10 2024-04-24T23:28:16.432 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:16.426+0000 7fea6c321700 1 -- 172.21.15.80:0/1233177886 --> [v2:172.21.15.148:6824/4068024116,v1:172.21.15.148:6825/4068024116] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fea64103940 con 0x7fea500ade10 2024-04-24T23:28:16.432 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:16.426+0000 7fea5b7fe700 1 -- 172.21.15.80:0/1233177886 <== osd.11 v2:172.21.15.148:6824/4068024116 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fea64103940 con 0x7fea500ade10 2024-04-24T23:28:16.433 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:16.426+0000 7fea6c321700 1 -- 172.21.15.80:0/1233177886 >> [v2:172.21.15.148:6824/4068024116,v1:172.21.15.148:6825/4068024116] conn(0x7fea500ade10 msgr2=0x7fea500b02b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:16.433 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:16.426+0000 7fea6c321700 1 --2- 172.21.15.80:0/1233177886 >> [v2:172.21.15.148:6824/4068024116,v1:172.21.15.148:6825/4068024116] conn(0x7fea500ade10 0x7fea500b02b0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:16.433 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:16.426+0000 7fea6c321700 1 -- 172.21.15.80:0/1233177886 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fea500a2cb0 msgr2=0x7fea500a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:16.433 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:16.426+0000 7fea6c321700 1 --2- 172.21.15.80:0/1233177886 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fea500a2cb0 0x7fea500a5170 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fea64108df0 tx=0x7fea54009190 comp rx=0 tx=0).stop 2024-04-24T23:28:16.433 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:16.426+0000 7fea6c321700 1 -- 172.21.15.80:0/1233177886 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fea64102550 msgr2=0x7fea6419ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:16.433 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:16.426+0000 7fea6c321700 1 --2- 172.21.15.80:0/1233177886 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fea64102550 0x7fea6419ad30 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fea6000b530 tx=0x7fea6000b840 comp rx=0 tx=0).stop 2024-04-24T23:28:16.433 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:16.426+0000 7fea6a8be700 1 -- 172.21.15.80:0/1233177886 reap_dead start 2024-04-24T23:28:16.433 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:16.426+0000 7fea6c321700 1 -- 172.21.15.80:0/1233177886 shutdown_connections 2024-04-24T23:28:16.433 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:16.426+0000 7fea6c321700 1 -- 172.21.15.80:0/1233177886 >> 172.21.15.80:0/1233177886 conn(0x7fea640793f0 msgr2=0x7fea6410ede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:16.434 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:16.427+0000 7fea6c321700 1 -- 172.21.15.80:0/1233177886 shutdown_connections 2024-04-24T23:28:16.434 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:16.427+0000 7fea6c321700 1 -- 172.21.15.80:0/1233177886 wait complete. 2024-04-24T23:28:16.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:28:16 smithi134 ceph-mon[32619]: pgmap v326: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 383 B/s wr, 7 op/s; 86 B/s, 1 keys/s, 1 objects/s recovering 2024-04-24T23:28:16.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:28:16 smithi148 ceph-mon[32528]: pgmap v326: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 383 B/s wr, 7 op/s; 86 B/s, 1 keys/s, 1 objects/s recovering 2024-04-24T23:28:17.891 INFO:teuthology.orchestra.run.smithi080.stdout:343597383705 2024-04-24T23:28:17.891 DEBUG:teuthology.orchestra.run.smithi080:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid ec7885aa-0290-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-04-24T23:28:18.159 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:28:18 smithi080 ceph-mon[29152]: pgmap v327: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 311 B/s wr, 6 op/s; 70 B/s, 1 keys/s, 1 objects/s recovering 2024-04-24T23:28:18.159 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:28:18 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:28:18.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:28:18 smithi134 ceph-mon[32619]: pgmap v327: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 311 B/s wr, 6 op/s; 70 B/s, 1 keys/s, 1 objects/s recovering 2024-04-24T23:28:18.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:28:18 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:28:18.469 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:18.465+0000 7f9400b58700 1 -- 172.21.15.80:0/1592007729 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f93fc101590 msgr2=0x7f93fc10d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:18.469 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:18.465+0000 7f9400b58700 1 --2- 172.21.15.80:0/1592007729 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f93fc101590 0x7f93fc10d7a0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f93ec00aaa0 tx=0x7f93ec00adb0 comp rx=0 tx=0).stop 2024-04-24T23:28:18.470 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:18.465+0000 7f9400b58700 1 -- 172.21.15.80:0/1592007729 shutdown_connections 2024-04-24T23:28:18.470 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:18.465+0000 7f9400b58700 1 --2- 172.21.15.80:0/1592007729 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f93fc101590 0x7f93fc10d7a0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:18.470 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:18.465+0000 7f9400b58700 1 --2- 172.21.15.80:0/1592007729 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f93fc106980 0x7f93fc106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:18.470 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:18.465+0000 7f9400b58700 1 --2- 172.21.15.80:0/1592007729 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f93fc100bd0 0x7f93fc101050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:18.470 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:18.465+0000 7f9400b58700 1 -- 172.21.15.80:0/1592007729 >> 172.21.15.80:0/1592007729 conn(0x7f93fc076450 msgr2=0x7f93fc076860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:18.470 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:18.465+0000 7f9400b58700 1 -- 172.21.15.80:0/1592007729 shutdown_connections 2024-04-24T23:28:18.470 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:18.465+0000 7f9400b58700 1 -- 172.21.15.80:0/1592007729 wait complete. 2024-04-24T23:28:18.470 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:18.466+0000 7f9400b58700 1 Processor -- start 2024-04-24T23:28:18.470 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:18.466+0000 7f9400b58700 1 -- start start 2024-04-24T23:28:18.470 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:18.467+0000 7f9400b58700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f93fc100bd0 0x7f93fc19e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:18.470 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:18.467+0000 7f9400b58700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f93fc101590 0x7f93fc19ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:18.470 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:18.467+0000 7f9400b58700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f93fc106980 0x7f93fc1987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:18.470 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:18.467+0000 7f9400b58700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f93fc110380 con 0x7f93fc101590 2024-04-24T23:28:18.471 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:18.467+0000 7f9400b58700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f93fc110200 con 0x7f93fc106980 2024-04-24T23:28:18.471 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:18.467+0000 7f9400b58700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f93fc110500 con 0x7f93fc100bd0 2024-04-24T23:28:18.471 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:18.467+0000 7f93f9d9b700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f93fc101590 0x7f93fc19ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:18.471 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:18.467+0000 7f93f9d9b700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f93fc101590 0x7f93fc19ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:43602/0 (socket says 172.21.15.80:43602) 2024-04-24T23:28:18.471 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:18.467+0000 7f93f9d9b700 1 -- 172.21.15.80:0/838547234 learned_addr learned my addr 172.21.15.80:0/838547234 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:28:18.471 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:18.467+0000 7f93fad9d700 1 --2- 172.21.15.80:0/838547234 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f93fc106980 0x7f93fc1987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:18.471 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:18.467+0000 7f93fa59c700 1 --2- 172.21.15.80:0/838547234 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f93fc100bd0 0x7f93fc19e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:18.471 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:18.467+0000 7f93f9d9b700 1 -- 172.21.15.80:0/838547234 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f93fc100bd0 msgr2=0x7f93fc19e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:18.471 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:18.467+0000 7f93f9d9b700 1 --2- 172.21.15.80:0/838547234 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f93fc100bd0 0x7f93fc19e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:18.471 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:18.467+0000 7f93f9d9b700 1 -- 172.21.15.80:0/838547234 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f93fc106980 msgr2=0x7f93fc1987b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:18.471 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:18.467+0000 7f93f9d9b700 1 --2- 172.21.15.80:0/838547234 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f93fc106980 0x7f93fc1987b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:18.471 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:18.467+0000 7f93f9d9b700 1 -- 172.21.15.80:0/838547234 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93fc199090 con 0x7f93fc101590 2024-04-24T23:28:18.472 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:18.467+0000 7f93fad9d700 1 --2- 172.21.15.80:0/838547234 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f93fc106980 0x7f93fc1987b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:28:18.472 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:18.467+0000 7f93f9d9b700 1 --2- 172.21.15.80:0/838547234 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f93fc101590 0x7f93fc19ebd0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f93e400e9c0 tx=0x7f93e400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:18.472 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:18.468+0000 7f93f37fe700 1 -- 172.21.15.80:0/838547234 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93e400cbd0 con 0x7f93fc101590 2024-04-24T23:28:18.472 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:18.468+0000 7f9400b58700 1 -- 172.21.15.80:0/838547234 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93fc199380 con 0x7f93fc101590 2024-04-24T23:28:18.472 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:18.468+0000 7f93f37fe700 1 -- 172.21.15.80:0/838547234 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93e400ee60 con 0x7f93fc101590 2024-04-24T23:28:18.472 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:18.468+0000 7f93f37fe700 1 -- 172.21.15.80:0/838547234 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93e4005630 con 0x7f93fc101590 2024-04-24T23:28:18.472 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:18.468+0000 7f9400b58700 1 -- 172.21.15.80:0/838547234 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93fc1a5610 con 0x7f93fc101590 2024-04-24T23:28:18.472 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:18.469+0000 7f9400b58700 1 -- 172.21.15.80:0/838547234 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f93fc062980 con 0x7f93fc101590 2024-04-24T23:28:18.473 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:18.470+0000 7f93f37fe700 1 -- 172.21.15.80:0/838547234 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f93e4004750 con 0x7f93fc101590 2024-04-24T23:28:18.473 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:18.470+0000 7f93f37fe700 1 --2- 172.21.15.80:0/838547234 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f93e80a2cb0 0x7f93e80a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:18.473 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:18.470+0000 7f93fa59c700 1 --2- 172.21.15.80:0/838547234 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f93e80a2cb0 0x7f93e80a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:18.474 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:18.471+0000 7f93f37fe700 1 -- 172.21.15.80:0/838547234 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f93e40cfbe0 con 0x7f93fc101590 2024-04-24T23:28:18.474 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:18.471+0000 7f93f37fe700 1 --2- 172.21.15.80:0/838547234 >> [v2:172.21.15.80:6810/3128298510,v1:172.21.15.80:6811/3128298510] conn(0x7f93e80ade10 0x7f93e80b02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:18.474 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:18.471+0000 7f93f37fe700 1 -- 172.21.15.80:0/838547234 --> [v2:172.21.15.80:6810/3128298510,v1:172.21.15.80:6811/3128298510] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f93e80b0980 con 0x7f93e80ade10 2024-04-24T23:28:18.474 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:18.471+0000 7f93f37fe700 1 -- 172.21.15.80:0/838547234 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f93e40d0030 con 0x7f93fc101590 2024-04-24T23:28:18.474 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:18.471+0000 7f93fad9d700 1 --2- 172.21.15.80:0/838547234 >> [v2:172.21.15.80:6810/3128298510,v1:172.21.15.80:6811/3128298510] conn(0x7f93e80ade10 0x7f93e80b02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:18.474 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:18.471+0000 7f93fa59c700 1 --2- 172.21.15.80:0/838547234 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f93e80a2cb0 0x7f93e80a5170 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f93f400f540 tx=0x7f93f400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:18.474 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:18.471+0000 7f93fad9d700 1 --2- 172.21.15.80:0/838547234 >> [v2:172.21.15.80:6810/3128298510,v1:172.21.15.80:6811/3128298510] conn(0x7f93e80ade10 0x7f93e80b02b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:18.474 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:18.472+0000 7f93f37fe700 1 -- 172.21.15.80:0/838547234 <== osd.3 v2:172.21.15.80:6810/3128298510 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f93e80b0980 con 0x7f93e80ade10 2024-04-24T23:28:18.490 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:18.489+0000 7f9400b58700 1 -- 172.21.15.80:0/838547234 --> [v2:172.21.15.80:6810/3128298510,v1:172.21.15.80:6811/3128298510] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f93fc062e00 con 0x7f93e80ade10 2024-04-24T23:28:18.490 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:18.489+0000 7f93f37fe700 1 -- 172.21.15.80:0/838547234 <== osd.3 v2:172.21.15.80:6810/3128298510 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f93fc062e00 con 0x7f93e80ade10 2024-04-24T23:28:18.491 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:18.489+0000 7f9400b58700 1 -- 172.21.15.80:0/838547234 >> [v2:172.21.15.80:6810/3128298510,v1:172.21.15.80:6811/3128298510] conn(0x7f93e80ade10 msgr2=0x7f93e80b02b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:18.491 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:18.489+0000 7f9400b58700 1 --2- 172.21.15.80:0/838547234 >> [v2:172.21.15.80:6810/3128298510,v1:172.21.15.80:6811/3128298510] conn(0x7f93e80ade10 0x7f93e80b02b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:18.491 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:18.489+0000 7f9400b58700 1 -- 172.21.15.80:0/838547234 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f93e80a2cb0 msgr2=0x7f93e80a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:18.491 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:18.489+0000 7f9400b58700 1 --2- 172.21.15.80:0/838547234 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f93e80a2cb0 0x7f93e80a5170 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f93f400f540 tx=0x7f93f400b040 comp rx=0 tx=0).stop 2024-04-24T23:28:18.491 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:18.489+0000 7f9400b58700 1 -- 172.21.15.80:0/838547234 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f93fc101590 msgr2=0x7f93fc19ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:18.491 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:18.489+0000 7f9400b58700 1 --2- 172.21.15.80:0/838547234 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f93fc101590 0x7f93fc19ebd0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f93e400e9c0 tx=0x7f93e400ecd0 comp rx=0 tx=0).stop 2024-04-24T23:28:18.491 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:18.489+0000 7f93fad9d700 1 -- 172.21.15.80:0/838547234 reap_dead start 2024-04-24T23:28:18.491 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:18.489+0000 7f9400b58700 1 -- 172.21.15.80:0/838547234 shutdown_connections 2024-04-24T23:28:18.492 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:18.490+0000 7f9400b58700 1 -- 172.21.15.80:0/838547234 >> 172.21.15.80:0/838547234 conn(0x7f93fc076450 msgr2=0x7f93fc0feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:18.492 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:18.490+0000 7f9400b58700 1 -- 172.21.15.80:0/838547234 shutdown_connections 2024-04-24T23:28:18.492 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:18.490+0000 7f9400b58700 1 -- 172.21.15.80:0/838547234 wait complete. 2024-04-24T23:28:18.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:28:18 smithi148 ceph-mon[32528]: pgmap v327: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 311 B/s wr, 6 op/s; 70 B/s, 1 keys/s, 1 objects/s recovering 2024-04-24T23:28:18.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:28:18 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:28:19.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:19.691+0000 7feeecb80700 1 -- 172.21.15.80:0/3750335191 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feee81082b0 msgr2=0x7feee8108690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:19.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:19.691+0000 7feeecb80700 1 --2- 172.21.15.80:0/3750335191 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feee81082b0 0x7feee8108690 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7feedc009990 tx=0x7feedc009ca0 comp rx=0 tx=0).stop 2024-04-24T23:28:19.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:19.691+0000 7feeecb80700 1 -- 172.21.15.80:0/3750335191 shutdown_connections 2024-04-24T23:28:19.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:19.691+0000 7feeecb80700 1 --2- 172.21.15.80:0/3750335191 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feee806a8e0 0x7feee810f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:19.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:19.691+0000 7feeecb80700 1 --2- 172.21.15.80:0/3750335191 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feee81082b0 0x7feee8108690 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:19.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:19.691+0000 7feeecb80700 1 --2- 172.21.15.80:0/3750335191 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feee8069f40 0x7feee806a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:19.696 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:19.691+0000 7feeecb80700 1 -- 172.21.15.80:0/3750335191 >> 172.21.15.80:0/3750335191 conn(0x7feee80780e0 msgr2=0x7feee80784f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:19.696 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:19.691+0000 7feeecb80700 1 -- 172.21.15.80:0/3750335191 shutdown_connections 2024-04-24T23:28:19.696 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:19.691+0000 7feeecb80700 1 -- 172.21.15.80:0/3750335191 wait complete. 2024-04-24T23:28:19.696 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:19.692+0000 7feeecb80700 1 Processor -- start 2024-04-24T23:28:19.696 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:19.692+0000 7feeecb80700 1 -- start start 2024-04-24T23:28:19.696 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:19.692+0000 7feeecb80700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feee8069f40 0x7feee8101200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:19.696 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:19.692+0000 7feeecb80700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feee806a8e0 0x7feee8101740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:19.696 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:19.692+0000 7feeecb80700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feee81082b0 0x7feee8101c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:19.696 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:19.692+0000 7feeecb80700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7feee8112510 con 0x7feee81082b0 2024-04-24T23:28:19.697 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:19.692+0000 7feeecb80700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7feee8112390 con 0x7feee8069f40 2024-04-24T23:28:19.697 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:19.692+0000 7feeecb80700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7feee8112690 con 0x7feee806a8e0 2024-04-24T23:28:19.697 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:19.692+0000 7feee6d9d700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feee81082b0 0x7feee8101c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:19.697 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:19.692+0000 7feee659c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feee8069f40 0x7feee8101200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:19.697 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:19.693+0000 7feee6d9d700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feee81082b0 0x7feee8101c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:43642/0 (socket says 172.21.15.80:43642) 2024-04-24T23:28:19.697 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:19.693+0000 7feee6d9d700 1 -- 172.21.15.80:0/2722284737 learned_addr learned my addr 172.21.15.80:0/2722284737 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:28:19.697 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:19.693+0000 7feee5d9b700 1 --2- 172.21.15.80:0/2722284737 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feee806a8e0 0x7feee8101740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:19.697 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:19.693+0000 7feee659c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feee8069f40 0x7feee8101200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.80:53304/0 (socket says 172.21.15.80:53304) 2024-04-24T23:28:19.697 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:19.693+0000 7feee6d9d700 1 -- 172.21.15.80:0/2722284737 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feee806a8e0 msgr2=0x7feee8101740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:19.697 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:19.693+0000 7feee6d9d700 1 --2- 172.21.15.80:0/2722284737 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feee806a8e0 0x7feee8101740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:19.697 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:19.693+0000 7feee6d9d700 1 -- 172.21.15.80:0/2722284737 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feee8069f40 msgr2=0x7feee8101200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:19.698 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:19.693+0000 7feee6d9d700 1 --2- 172.21.15.80:0/2722284737 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feee8069f40 0x7feee8101200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:19.698 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:19.693+0000 7feee6d9d700 1 -- 172.21.15.80:0/2722284737 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feee81a6f70 con 0x7feee81082b0 2024-04-24T23:28:19.698 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:19.693+0000 7feee659c700 1 --2- 172.21.15.80:0/2722284737 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feee8069f40 0x7feee8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:28:19.698 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:19.693+0000 7feee6d9d700 1 --2- 172.21.15.80:0/2722284737 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feee81082b0 0x7feee8101c80 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7feee0007ee0 tx=0x7feee000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:19.698 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:19.693+0000 7feed77fe700 1 -- 172.21.15.80:0/2722284737 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee0004bb0 con 0x7feee81082b0 2024-04-24T23:28:19.698 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:19.693+0000 7feeecb80700 1 -- 172.21.15.80:0/2722284737 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feee81a7260 con 0x7feee81082b0 2024-04-24T23:28:19.698 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:19.693+0000 7feed77fe700 1 -- 172.21.15.80:0/2722284737 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feee0004d50 con 0x7feee81082b0 2024-04-24T23:28:19.698 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:19.693+0000 7feed77fe700 1 -- 172.21.15.80:0/2722284737 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee0005960 con 0x7feee81082b0 2024-04-24T23:28:19.698 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:19.693+0000 7feeecb80700 1 -- 172.21.15.80:0/2722284737 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feee81a77a0 con 0x7feee81082b0 2024-04-24T23:28:19.698 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:19.694+0000 7feeecb80700 1 -- 172.21.15.80:0/2722284737 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7feee8062940 con 0x7feee81082b0 2024-04-24T23:28:19.700 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:19.695+0000 7feed77fe700 1 -- 172.21.15.80:0/2722284737 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7feee001a020 con 0x7feee81082b0 2024-04-24T23:28:19.700 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:19.696+0000 7feed77fe700 1 --2- 172.21.15.80:0/2722284737 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7feed00a2cb0 0x7feed00a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:19.700 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:19.696+0000 7feee659c700 1 --2- 172.21.15.80:0/2722284737 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7feed00a2cb0 0x7feed00a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:19.700 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:19.696+0000 7feed77fe700 1 -- 172.21.15.80:0/2722284737 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7feee00cf2a0 con 0x7feee81082b0 2024-04-24T23:28:19.700 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:19.696+0000 7feed77fe700 1 --2- 172.21.15.80:0/2722284737 >> [v2:172.21.15.134:6824/647343766,v1:172.21.15.134:6825/647343766] conn(0x7feed00ade10 0x7feed00b02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:19.700 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:19.696+0000 7feed77fe700 1 -- 172.21.15.80:0/2722284737 --> [v2:172.21.15.134:6824/647343766,v1:172.21.15.134:6825/647343766] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7feee001de50 con 0x7feed00ade10 2024-04-24T23:28:19.700 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:19.696+0000 7feed77fe700 1 -- 172.21.15.80:0/2722284737 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7feee00cf6f0 con 0x7feee81082b0 2024-04-24T23:28:19.700 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:19.696+0000 7feee659c700 1 --2- 172.21.15.80:0/2722284737 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7feed00a2cb0 0x7feed00a5170 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7feedc0098e0 tx=0x7feedc005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:19.700 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:19.696+0000 7feee5d9b700 1 --2- 172.21.15.80:0/2722284737 >> [v2:172.21.15.134:6824/647343766,v1:172.21.15.134:6825/647343766] conn(0x7feed00ade10 0x7feed00b02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:19.700 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:19.697+0000 7feee5d9b700 1 --2- 172.21.15.80:0/2722284737 >> [v2:172.21.15.134:6824/647343766,v1:172.21.15.134:6825/647343766] conn(0x7feed00ade10 0x7feed00b02b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:19.700 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:19.698+0000 7feed77fe700 1 -- 172.21.15.80:0/2722284737 <== osd.10 v2:172.21.15.134:6824/647343766 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7feee001de50 con 0x7feed00ade10 2024-04-24T23:28:19.715 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:19.714+0000 7feeecb80700 1 -- 172.21.15.80:0/2722284737 --> [v2:172.21.15.134:6824/647343766,v1:172.21.15.134:6825/647343766] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7feee8062d40 con 0x7feed00ade10 2024-04-24T23:28:19.715 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:19.714+0000 7feed77fe700 1 -- 172.21.15.80:0/2722284737 <== osd.10 v2:172.21.15.134:6824/647343766 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7feee8062d40 con 0x7feed00ade10 2024-04-24T23:28:19.716 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:19.715+0000 7feeecb80700 1 -- 172.21.15.80:0/2722284737 >> [v2:172.21.15.134:6824/647343766,v1:172.21.15.134:6825/647343766] conn(0x7feed00ade10 msgr2=0x7feed00b02b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:19.716 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:19.715+0000 7feeecb80700 1 --2- 172.21.15.80:0/2722284737 >> [v2:172.21.15.134:6824/647343766,v1:172.21.15.134:6825/647343766] conn(0x7feed00ade10 0x7feed00b02b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:19.716 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:19.715+0000 7feeecb80700 1 -- 172.21.15.80:0/2722284737 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7feed00a2cb0 msgr2=0x7feed00a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:19.716 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:19.715+0000 7feeecb80700 1 --2- 172.21.15.80:0/2722284737 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7feed00a2cb0 0x7feed00a5170 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7feedc0098e0 tx=0x7feedc005960 comp rx=0 tx=0).stop 2024-04-24T23:28:19.716 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:19.715+0000 7feeecb80700 1 -- 172.21.15.80:0/2722284737 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feee81082b0 msgr2=0x7feee8101c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:19.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:19.715+0000 7feeecb80700 1 --2- 172.21.15.80:0/2722284737 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feee81082b0 0x7feee8101c80 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7feee0007ee0 tx=0x7feee000d3b0 comp rx=0 tx=0).stop 2024-04-24T23:28:19.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:19.715+0000 7feee6d9d700 1 -- 172.21.15.80:0/2722284737 reap_dead start 2024-04-24T23:28:19.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:19.715+0000 7feeecb80700 1 -- 172.21.15.80:0/2722284737 shutdown_connections 2024-04-24T23:28:19.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:19.715+0000 7feeecb80700 1 -- 172.21.15.80:0/2722284737 >> 172.21.15.80:0/2722284737 conn(0x7feee80780e0 msgr2=0x7feee80fff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:19.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:19.715+0000 7feeecb80700 1 -- 172.21.15.80:0/2722284737 shutdown_connections 2024-04-24T23:28:19.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:19.715+0000 7feeecb80700 1 -- 172.21.15.80:0/2722284737 wait complete. 2024-04-24T23:28:20.103 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:28:20 smithi080 ceph-mon[29152]: pgmap v328: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 283 B/s wr, 7 op/s; 63 B/s, 0 keys/s, 1 objects/s recovering 2024-04-24T23:28:20.103 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:28:20 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:28:20.103 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:28:20 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:28:20.103 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:28:20 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:28:20.103 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:28:20 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:28:20.222 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:20.219+0000 7f69afe9f700 1 -- 172.21.15.80:0/1801736364 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f69a81071a0 msgr2=0x7f69a8107580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:20.222 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:20.219+0000 7f69afe9f700 1 --2- 172.21.15.80:0/1801736364 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f69a81071a0 0x7f69a8107580 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f6998009990 tx=0x7f6998009ca0 comp rx=0 tx=0).stop 2024-04-24T23:28:20.222 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:20.219+0000 7f69afe9f700 1 -- 172.21.15.80:0/1801736364 shutdown_connections 2024-04-24T23:28:20.222 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:20.219+0000 7f69afe9f700 1 --2- 172.21.15.80:0/1801736364 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f69a8101af0 0x7f69a810dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:20.222 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:20.219+0000 7f69afe9f700 1 --2- 172.21.15.80:0/1801736364 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f69a81071a0 0x7f69a8107580 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:20.222 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:20.219+0000 7f69afe9f700 1 --2- 172.21.15.80:0/1801736364 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f69a8101130 0x7f69a81015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:20.222 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:20.219+0000 7f69afe9f700 1 -- 172.21.15.80:0/1801736364 >> 172.21.15.80:0/1801736364 conn(0x7f69a80780e0 msgr2=0x7f69a80784f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:20.222 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:20.219+0000 7f69afe9f700 1 -- 172.21.15.80:0/1801736364 shutdown_connections 2024-04-24T23:28:20.222 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:20.219+0000 7f69afe9f700 1 -- 172.21.15.80:0/1801736364 wait complete. 2024-04-24T23:28:20.223 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:20.220+0000 7f69afe9f700 1 Processor -- start 2024-04-24T23:28:20.223 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:20.220+0000 7f69afe9f700 1 -- start start 2024-04-24T23:28:20.223 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:20.220+0000 7f69afe9f700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f69a8101130 0x7f69a819d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:20.223 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:20.220+0000 7f69afe9f700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f69a8101af0 0x7f69a819db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:20.223 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:20.220+0000 7f69afe9f700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f69a81071a0 0x7f69a8197720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:20.223 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:20.220+0000 7f69afe9f700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f69a8111440 con 0x7f69a8101130 2024-04-24T23:28:20.223 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:20.220+0000 7f69afe9f700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f69a81112c0 con 0x7f69a8101af0 2024-04-24T23:28:20.223 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:20.220+0000 7f69afe9f700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f69a81115c0 con 0x7f69a81071a0 2024-04-24T23:28:20.223 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:20.220+0000 7f69adc3b700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f69a8101130 0x7f69a819d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:20.223 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:20.220+0000 7f69adc3b700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f69a8101130 0x7f69a819d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:43662/0 (socket says 172.21.15.80:43662) 2024-04-24T23:28:20.223 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:20.220+0000 7f69adc3b700 1 -- 172.21.15.80:0/1162841807 learned_addr learned my addr 172.21.15.80:0/1162841807 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:28:20.223 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:20.220+0000 7f69ae43c700 1 --2- 172.21.15.80:0/1162841807 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f69a81071a0 0x7f69a8197720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:20.223 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:20.221+0000 7f69ad43a700 1 --2- 172.21.15.80:0/1162841807 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f69a8101af0 0x7f69a819db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:20.223 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:20.221+0000 7f69adc3b700 1 -- 172.21.15.80:0/1162841807 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f69a81071a0 msgr2=0x7f69a8197720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:20.224 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:20.221+0000 7f69adc3b700 1 --2- 172.21.15.80:0/1162841807 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f69a81071a0 0x7f69a8197720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:20.224 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:20.221+0000 7f69adc3b700 1 -- 172.21.15.80:0/1162841807 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f69a8101af0 msgr2=0x7f69a819db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:20.224 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:20.221+0000 7f69adc3b700 1 --2- 172.21.15.80:0/1162841807 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f69a8101af0 0x7f69a819db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:20.224 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:20.221+0000 7f69adc3b700 1 -- 172.21.15.80:0/1162841807 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69a8197f80 con 0x7f69a8101130 2024-04-24T23:28:20.224 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:20.221+0000 7f69ad43a700 1 --2- 172.21.15.80:0/1162841807 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f69a8101af0 0x7f69a819db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:28:20.224 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:20.221+0000 7f69adc3b700 1 --2- 172.21.15.80:0/1162841807 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f69a8101130 0x7f69a819d600 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f6998000c00 tx=0x7f699800bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:20.224 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:20.221+0000 7f699effd700 1 -- 172.21.15.80:0/1162841807 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6998027070 con 0x7f69a8101130 2024-04-24T23:28:20.224 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:20.221+0000 7f699effd700 1 -- 172.21.15.80:0/1162841807 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6998003d70 con 0x7f69a8101130 2024-04-24T23:28:20.224 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:20.221+0000 7f69afe9f700 1 -- 172.21.15.80:0/1162841807 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69a8198210 con 0x7f69a8101130 2024-04-24T23:28:20.225 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:20.221+0000 7f699effd700 1 -- 172.21.15.80:0/1162841807 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f699800f6c0 con 0x7f69a8101130 2024-04-24T23:28:20.226 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:20.221+0000 7f69afe9f700 1 -- 172.21.15.80:0/1162841807 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69a8069f10 con 0x7f69a8101130 2024-04-24T23:28:20.226 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:20.222+0000 7f69afe9f700 1 -- 172.21.15.80:0/1162841807 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f69a806a480 con 0x7f69a8101130 2024-04-24T23:28:20.226 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:20.223+0000 7f699effd700 1 -- 172.21.15.80:0/1162841807 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f699802a070 con 0x7f69a8101130 2024-04-24T23:28:20.226 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:20.224+0000 7f699effd700 1 --2- 172.21.15.80:0/1162841807 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f69940a2d30 0x7f69940a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:20.226 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:20.224+0000 7f699effd700 1 -- 172.21.15.80:0/1162841807 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f69980d0ca0 con 0x7f69a8101130 2024-04-24T23:28:20.227 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:20.224+0000 7f69ad43a700 1 --2- 172.21.15.80:0/1162841807 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f69940a2d30 0x7f69940a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:20.227 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:20.224+0000 7f699effd700 1 --2- 172.21.15.80:0/1162841807 >> [v2:172.21.15.134:6808/1917556060,v1:172.21.15.134:6809/1917556060] conn(0x7f69940ade90 0x7f69940b0330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:20.227 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:20.224+0000 7f699effd700 1 -- 172.21.15.80:0/1162841807 --> [v2:172.21.15.134:6808/1917556060,v1:172.21.15.134:6809/1917556060] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f699800b0b0 con 0x7f69940ade90 2024-04-24T23:28:20.227 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:20.224+0000 7f699effd700 1 -- 172.21.15.80:0/1162841807 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f69980d10f0 con 0x7f69a8101130 2024-04-24T23:28:20.227 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:20.224+0000 7f69ae43c700 1 --2- 172.21.15.80:0/1162841807 >> [v2:172.21.15.134:6808/1917556060,v1:172.21.15.134:6809/1917556060] conn(0x7f69940ade90 0x7f69940b0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:20.227 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:20.225+0000 7f69ad43a700 1 --2- 172.21.15.80:0/1162841807 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f69940a2d30 0x7f69940a51f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f69a4005fd0 tx=0x7f69a4005e20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:20.227 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:20.225+0000 7f69ae43c700 1 --2- 172.21.15.80:0/1162841807 >> [v2:172.21.15.134:6808/1917556060,v1:172.21.15.134:6809/1917556060] conn(0x7f69940ade90 0x7f69940b0330 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:20.227 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:20.226+0000 7f699effd700 1 -- 172.21.15.80:0/1162841807 <== osd.4 v2:172.21.15.134:6808/1917556060 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f699800b0b0 con 0x7f69940ade90 2024-04-24T23:28:20.244 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:20.243+0000 7f69afe9f700 1 -- 172.21.15.80:0/1162841807 --> [v2:172.21.15.134:6808/1917556060,v1:172.21.15.134:6809/1917556060] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f69a806a880 con 0x7f69940ade90 2024-04-24T23:28:20.244 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:20.244+0000 7f699effd700 1 -- 172.21.15.80:0/1162841807 <== osd.4 v2:172.21.15.134:6808/1917556060 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f69a806a880 con 0x7f69940ade90 2024-04-24T23:28:20.245 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:20.244+0000 7f69afe9f700 1 -- 172.21.15.80:0/1162841807 >> [v2:172.21.15.134:6808/1917556060,v1:172.21.15.134:6809/1917556060] conn(0x7f69940ade90 msgr2=0x7f69940b0330 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:20.245 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:20.244+0000 7f69afe9f700 1 --2- 172.21.15.80:0/1162841807 >> [v2:172.21.15.134:6808/1917556060,v1:172.21.15.134:6809/1917556060] conn(0x7f69940ade90 0x7f69940b0330 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:20.245 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:20.244+0000 7f69afe9f700 1 -- 172.21.15.80:0/1162841807 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f69940a2d30 msgr2=0x7f69940a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:20.246 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:20.244+0000 7f69afe9f700 1 --2- 172.21.15.80:0/1162841807 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f69940a2d30 0x7f69940a51f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f69a4005fd0 tx=0x7f69a4005e20 comp rx=0 tx=0).stop 2024-04-24T23:28:20.246 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:20.244+0000 7f69afe9f700 1 -- 172.21.15.80:0/1162841807 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f69a8101130 msgr2=0x7f69a819d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:20.246 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:20.244+0000 7f69afe9f700 1 --2- 172.21.15.80:0/1162841807 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f69a8101130 0x7f69a819d600 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f6998000c00 tx=0x7f699800bab0 comp rx=0 tx=0).stop 2024-04-24T23:28:20.246 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:20.244+0000 7f69ae43c700 1 -- 172.21.15.80:0/1162841807 reap_dead start 2024-04-24T23:28:20.246 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:20.244+0000 7f69afe9f700 1 -- 172.21.15.80:0/1162841807 shutdown_connections 2024-04-24T23:28:20.246 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:20.244+0000 7f69afe9f700 1 -- 172.21.15.80:0/1162841807 >> 172.21.15.80:0/1162841807 conn(0x7f69a80780e0 msgr2=0x7f69a80ffc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:20.246 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:20.244+0000 7f69afe9f700 1 -- 172.21.15.80:0/1162841807 shutdown_connections 2024-04-24T23:28:20.246 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:20.244+0000 7f69afe9f700 1 -- 172.21.15.80:0/1162841807 wait complete. 2024-04-24T23:28:20.311 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:19.585+0000 7f6a7a2b4700 1 -- 172.21.15.80:0/2557516198 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6a74101800 msgr2=0x7f6a74106f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:20.312 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:19.585+0000 7f6a7a2b4700 1 --2- 172.21.15.80:0/2557516198 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6a74101800 0x7f6a74106f40 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f6a680099c0 tx=0x7f6a68009cd0 comp rx=0 tx=0).stop 2024-04-24T23:28:20.312 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:19.585+0000 7f6a7a2b4700 1 -- 172.21.15.80:0/2557516198 shutdown_connections 2024-04-24T23:28:20.312 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:19.585+0000 7f6a7a2b4700 1 --2- 172.21.15.80:0/2557516198 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a74107670 0x7f6a7410f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:20.312 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:19.585+0000 7f6a7a2b4700 1 --2- 172.21.15.80:0/2557516198 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6a74100e50 0x7f6a74101230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:20.312 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:19.585+0000 7f6a7a2b4700 1 --2- 172.21.15.80:0/2557516198 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6a74101800 0x7f6a74106f40 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:20.312 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:19.585+0000 7f6a7a2b4700 1 -- 172.21.15.80:0/2557516198 >> 172.21.15.80:0/2557516198 conn(0x7f6a74076450 msgr2=0x7f6a74076860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:20.312 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:19.585+0000 7f6a7a2b4700 1 -- 172.21.15.80:0/2557516198 shutdown_connections 2024-04-24T23:28:20.312 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:19.585+0000 7f6a7a2b4700 1 -- 172.21.15.80:0/2557516198 wait complete. 2024-04-24T23:28:20.312 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:19.585+0000 7f6a7a2b4700 1 Processor -- start 2024-04-24T23:28:20.312 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:19.585+0000 7f6a7a2b4700 1 -- start start 2024-04-24T23:28:20.312 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:19.586+0000 7f6a7a2b4700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6a74100e50 0x7f6a7419e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:20.312 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:19.586+0000 7f6a7a2b4700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a74101800 0x7f6a7419eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:20.313 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:19.586+0000 7f6a7a2b4700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6a74107670 0x7f6a741987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:20.313 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:19.586+0000 7f6a7a2b4700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f6a74112560 con 0x7f6a74100e50 2024-04-24T23:28:20.313 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:19.586+0000 7f6a7a2b4700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6a741123e0 con 0x7f6a74107670 2024-04-24T23:28:20.313 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:19.586+0000 7f6a7a2b4700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f6a741126e0 con 0x7f6a74101800 2024-04-24T23:28:20.313 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:19.586+0000 7f6a73fff700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6a74100e50 0x7f6a7419e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:20.313 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:19.586+0000 7f6a73fff700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6a74100e50 0x7f6a7419e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:43620/0 (socket says 172.21.15.80:43620) 2024-04-24T23:28:20.313 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:19.586+0000 7f6a73fff700 1 -- 172.21.15.80:0/3788083456 learned_addr learned my addr 172.21.15.80:0/3788083456 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:28:20.313 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:19.586+0000 7f6a78851700 1 --2- 172.21.15.80:0/3788083456 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6a74107670 0x7f6a741987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:20.313 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:19.586+0000 7f6a737fe700 1 --2- 172.21.15.80:0/3788083456 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a74101800 0x7f6a7419eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:20.313 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:19.586+0000 7f6a73fff700 1 -- 172.21.15.80:0/3788083456 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a74101800 msgr2=0x7f6a7419eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:20.313 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:19.586+0000 7f6a73fff700 1 --2- 172.21.15.80:0/3788083456 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a74101800 0x7f6a7419eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:20.313 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:19.586+0000 7f6a73fff700 1 -- 172.21.15.80:0/3788083456 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6a74107670 msgr2=0x7f6a741987a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:20.314 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:19.586+0000 7f6a73fff700 1 --2- 172.21.15.80:0/3788083456 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6a74107670 0x7f6a741987a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:20.314 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:19.586+0000 7f6a73fff700 1 -- 172.21.15.80:0/3788083456 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a74198f80 con 0x7f6a74100e50 2024-04-24T23:28:20.314 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:19.586+0000 7f6a73fff700 1 --2- 172.21.15.80:0/3788083456 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6a74100e50 0x7f6a7419e620 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f6a6400e9a0 tx=0x7f6a6400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:20.314 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:19.586+0000 7f6a717fa700 1 -- 172.21.15.80:0/3788083456 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a6400ca10 con 0x7f6a74100e50 2024-04-24T23:28:20.314 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:19.586+0000 7f6a7a2b4700 1 -- 172.21.15.80:0/3788083456 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a74199270 con 0x7f6a74100e50 2024-04-24T23:28:20.314 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:19.586+0000 7f6a717fa700 1 -- 172.21.15.80:0/3788083456 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a6400cbb0 con 0x7f6a74100e50 2024-04-24T23:28:20.315 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:19.586+0000 7f6a717fa700 1 -- 172.21.15.80:0/3788083456 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a64018680 con 0x7f6a74100e50 2024-04-24T23:28:20.315 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:19.586+0000 7f6a7a2b4700 1 -- 172.21.15.80:0/3788083456 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a741a5480 con 0x7f6a74100e50 2024-04-24T23:28:20.315 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:19.587+0000 7f6a7a2b4700 1 -- 172.21.15.80:0/3788083456 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6a740629c0 con 0x7f6a74100e50 2024-04-24T23:28:20.315 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:19.589+0000 7f6a717fa700 1 -- 172.21.15.80:0/3788083456 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f6a64010430 con 0x7f6a74100e50 2024-04-24T23:28:20.315 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:19.589+0000 7f6a717fa700 1 --2- 172.21.15.80:0/3788083456 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f6a5c0a2cb0 0x7f6a5c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:20.315 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:19.589+0000 7f6a737fe700 1 --2- 172.21.15.80:0/3788083456 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f6a5c0a2cb0 0x7f6a5c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:20.315 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:19.589+0000 7f6a717fa700 1 -- 172.21.15.80:0/3788083456 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f6a640cff50 con 0x7f6a74100e50 2024-04-24T23:28:20.315 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:19.590+0000 7f6a717fa700 1 --2- 172.21.15.80:0/3788083456 >> [v2:172.21.15.148:6800/1629152187,v1:172.21.15.148:6801/1629152187] conn(0x7f6a5c0ade10 0x7f6a5c0b02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:20.315 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:19.590+0000 7f6a717fa700 1 -- 172.21.15.80:0/3788083456 --> [v2:172.21.15.148:6800/1629152187,v1:172.21.15.148:6801/1629152187] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6a6401bec0 con 0x7f6a5c0ade10 2024-04-24T23:28:20.315 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:19.590+0000 7f6a717fa700 1 -- 172.21.15.80:0/3788083456 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f6a640d03a0 con 0x7f6a74100e50 2024-04-24T23:28:20.315 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:19.590+0000 7f6a737fe700 1 --2- 172.21.15.80:0/3788083456 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f6a5c0a2cb0 0x7f6a5c0a5170 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f6a740ff4f0 tx=0x7f6a68005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:20.315 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:19.590+0000 7f6a78851700 1 --2- 172.21.15.80:0/3788083456 >> [v2:172.21.15.148:6800/1629152187,v1:172.21.15.148:6801/1629152187] conn(0x7f6a5c0ade10 0x7f6a5c0b02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:20.316 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:19.590+0000 7f6a78851700 1 --2- 172.21.15.80:0/3788083456 >> [v2:172.21.15.148:6800/1629152187,v1:172.21.15.148:6801/1629152187] conn(0x7f6a5c0ade10 0x7f6a5c0b02b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:20.316 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:19.591+0000 7f6a717fa700 1 -- 172.21.15.80:0/3788083456 <== osd.2 v2:172.21.15.148:6800/1629152187 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6a6401bec0 con 0x7f6a5c0ade10 2024-04-24T23:28:20.316 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:19.607+0000 7f6a7a2b4700 1 -- 172.21.15.80:0/3788083456 --> [v2:172.21.15.148:6800/1629152187,v1:172.21.15.148:6801/1629152187] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6a74062dc0 con 0x7f6a5c0ade10 2024-04-24T23:28:20.316 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:19.608+0000 7f6a717fa700 1 -- 172.21.15.80:0/3788083456 <== osd.2 v2:172.21.15.148:6800/1629152187 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f6a74062dc0 con 0x7f6a5c0ade10 2024-04-24T23:28:20.316 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:19.608+0000 7f6a7a2b4700 1 -- 172.21.15.80:0/3788083456 >> [v2:172.21.15.148:6800/1629152187,v1:172.21.15.148:6801/1629152187] conn(0x7f6a5c0ade10 msgr2=0x7f6a5c0b02b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:20.316 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:19.608+0000 7f6a7a2b4700 1 --2- 172.21.15.80:0/3788083456 >> [v2:172.21.15.148:6800/1629152187,v1:172.21.15.148:6801/1629152187] conn(0x7f6a5c0ade10 0x7f6a5c0b02b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:20.316 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:19.608+0000 7f6a7a2b4700 1 -- 172.21.15.80:0/3788083456 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f6a5c0a2cb0 msgr2=0x7f6a5c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:20.316 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:19.608+0000 7f6a7a2b4700 1 --2- 172.21.15.80:0/3788083456 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f6a5c0a2cb0 0x7f6a5c0a5170 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f6a740ff4f0 tx=0x7f6a68005fb0 comp rx=0 tx=0).stop 2024-04-24T23:28:20.316 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:19.608+0000 7f6a7a2b4700 1 -- 172.21.15.80:0/3788083456 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6a74100e50 msgr2=0x7f6a7419e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:20.316 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:19.608+0000 7f6a7a2b4700 1 --2- 172.21.15.80:0/3788083456 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6a74100e50 0x7f6a7419e620 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f6a6400e9a0 tx=0x7f6a6400ecb0 comp rx=0 tx=0).stop 2024-04-24T23:28:20.316 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:19.608+0000 7f6a78851700 1 -- 172.21.15.80:0/3788083456 reap_dead start 2024-04-24T23:28:20.316 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:19.609+0000 7f6a7a2b4700 1 -- 172.21.15.80:0/3788083456 shutdown_connections 2024-04-24T23:28:20.316 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:19.609+0000 7f6a7a2b4700 1 -- 172.21.15.80:0/3788083456 >> 172.21.15.80:0/3788083456 conn(0x7f6a74076450 msgr2=0x7f6a740ffd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:20.317 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:19.609+0000 7f6a7a2b4700 1 -- 172.21.15.80:0/3788083456 shutdown_connections 2024-04-24T23:28:20.317 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:19.609+0000 7f6a7a2b4700 1 -- 172.21.15.80:0/3788083456 wait complete. 2024-04-24T23:28:20.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:28:20 smithi134 ceph-mon[32619]: pgmap v328: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 283 B/s wr, 7 op/s; 63 B/s, 0 keys/s, 1 objects/s recovering 2024-04-24T23:28:20.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:28:20 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:28:20.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:28:20 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:28:20.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:28:20 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:28:20.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:28:20 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:28:20.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:28:20 smithi148 ceph-mon[32528]: pgmap v328: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 283 B/s wr, 7 op/s; 63 B/s, 0 keys/s, 1 objects/s recovering 2024-04-24T23:28:20.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:28:20 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:28:20.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:28:20 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:28:20.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:28:20 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:28:20.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:28:20 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:28:20.976 INFO:teuthology.orchestra.run.smithi080.stdout:77309411401 2024-04-24T23:28:20.977 DEBUG:teuthology.orchestra.run.smithi080:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid ec7885aa-0290-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-04-24T23:28:21.636 INFO:teuthology.orchestra.run.smithi080.stdout:309237645343 2024-04-24T23:28:21.636 DEBUG:teuthology.orchestra.run.smithi080:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid ec7885aa-0290-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-04-24T23:28:21.940 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:21.938+0000 7f12bf409700 1 -- 172.21.15.80:0/3312542965 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f12b8106d50 msgr2=0x7f12b81008f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:21.940 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:21.938+0000 7f12bf409700 1 --2- 172.21.15.80:0/3312542965 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f12b8106d50 0x7f12b81008f0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f12a8009990 tx=0x7f12a8009ca0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.940 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:21.938+0000 7f12bf409700 1 -- 172.21.15.80:0/3312542965 shutdown_connections 2024-04-24T23:28:21.941 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:21.938+0000 7f12bf409700 1 --2- 172.21.15.80:0/3312542965 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12b81071d0 0x7f12b8100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.941 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:21.938+0000 7f12bf409700 1 --2- 172.21.15.80:0/3312542965 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12b8069710 0x7f12b8106810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.941 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:21.938+0000 7f12bf409700 1 --2- 172.21.15.80:0/3312542965 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f12b8106d50 0x7f12b81008f0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.941 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:21.938+0000 7f12bf409700 1 -- 172.21.15.80:0/3312542965 >> 172.21.15.80:0/3312542965 conn(0x7f12b80777a0 msgr2=0x7f12b80feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:21.941 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:21.938+0000 7f12bf409700 1 -- 172.21.15.80:0/3312542965 shutdown_connections 2024-04-24T23:28:21.941 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:21.938+0000 7f12bf409700 1 -- 172.21.15.80:0/3312542965 wait complete. 2024-04-24T23:28:21.941 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:21.939+0000 7f12bf409700 1 Processor -- start 2024-04-24T23:28:21.941 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:21.939+0000 7f12bf409700 1 -- start start 2024-04-24T23:28:21.941 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:21.939+0000 7f12bf409700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12b8069710 0x7f12b81a0840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:21.942 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:21.939+0000 7f12bf409700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f12b8106d50 0x7f12b81a0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:21.942 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:21.939+0000 7f12bf409700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12b81071d0 0x7f12b819aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:21.942 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:21.939+0000 7f12bf409700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f12b810a4a0 con 0x7f12b8106d50 2024-04-24T23:28:21.942 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:21.939+0000 7f12bf409700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f12b810a320 con 0x7f12b81071d0 2024-04-24T23:28:21.942 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:21.939+0000 7f12bf409700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f12b810a620 con 0x7f12b8069710 2024-04-24T23:28:21.942 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:21.939+0000 7f12bc9a4700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f12b8106d50 0x7f12b81a0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:21.943 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:21.939+0000 7f12bc9a4700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f12b8106d50 0x7f12b81a0d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:43688/0 (socket says 172.21.15.80:43688) 2024-04-24T23:28:21.943 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:21.939+0000 7f12bc9a4700 1 -- 172.21.15.80:0/1606081887 learned_addr learned my addr 172.21.15.80:0/1606081887 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:28:21.943 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:21.939+0000 7f12bd1a5700 1 --2- 172.21.15.80:0/1606081887 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12b8069710 0x7f12b81a0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:21.943 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:21.939+0000 7f12bd9a6700 1 --2- 172.21.15.80:0/1606081887 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12b81071d0 0x7f12b819aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:21.943 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:21.939+0000 7f12bc9a4700 1 -- 172.21.15.80:0/1606081887 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12b8069710 msgr2=0x7f12b81a0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:21.943 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:21.939+0000 7f12bc9a4700 1 --2- 172.21.15.80:0/1606081887 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12b8069710 0x7f12b81a0840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.943 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:21.939+0000 7f12bc9a4700 1 -- 172.21.15.80:0/1606081887 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12b81071d0 msgr2=0x7f12b819aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:21.943 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:21.939+0000 7f12bc9a4700 1 --2- 172.21.15.80:0/1606081887 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12b81071d0 0x7f12b819aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.944 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:21.939+0000 7f12bc9a4700 1 -- 172.21.15.80:0/1606081887 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12b819b160 con 0x7f12b8106d50 2024-04-24T23:28:21.944 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:21.940+0000 7f12bc9a4700 1 --2- 172.21.15.80:0/1606081887 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f12b8106d50 0x7f12b81a0d80 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f12a8000c00 tx=0x7f12a800bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:21.945 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:21.940+0000 7f12ae7fc700 1 -- 172.21.15.80:0/1606081887 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a8027070 con 0x7f12b8106d50 2024-04-24T23:28:21.945 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:21.940+0000 7f12bf409700 1 -- 172.21.15.80:0/1606081887 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12b819b3f0 con 0x7f12b8106d50 2024-04-24T23:28:21.945 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:21.940+0000 7f12bf409700 1 -- 172.21.15.80:0/1606081887 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12b81a7710 con 0x7f12b8106d50 2024-04-24T23:28:21.945 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:21.940+0000 7f12ae7fc700 1 -- 172.21.15.80:0/1606081887 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12a8003d70 con 0x7f12b8106d50 2024-04-24T23:28:21.945 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:21.940+0000 7f12ae7fc700 1 -- 172.21.15.80:0/1606081887 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a800f460 con 0x7f12b8106d50 2024-04-24T23:28:21.945 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:21.941+0000 7f12bf409700 1 -- 172.21.15.80:0/1606081887 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f12b81039c0 con 0x7f12b8106d50 2024-04-24T23:28:21.945 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:21.943+0000 7f12ae7fc700 1 -- 172.21.15.80:0/1606081887 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f12a802a070 con 0x7f12b8106d50 2024-04-24T23:28:21.946 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:21.944+0000 7f12ae7fc700 1 --2- 172.21.15.80:0/1606081887 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f12a40a2d30 0x7f12a40a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:21.946 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:21.944+0000 7f12ae7fc700 1 -- 172.21.15.80:0/1606081887 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f12a80d0750 con 0x7f12b8106d50 2024-04-24T23:28:21.946 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:21.944+0000 7f12bd1a5700 1 --2- 172.21.15.80:0/1606081887 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f12a40a2d30 0x7f12a40a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:21.946 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:21.944+0000 7f12ae7fc700 1 --2- 172.21.15.80:0/1606081887 >> [v2:172.21.15.134:6800/2395610201,v1:172.21.15.134:6801/2395610201] conn(0x7f12a40ade90 0x7f12a40b0330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:21.946 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:21.944+0000 7f12ae7fc700 1 -- 172.21.15.80:0/1606081887 --> [v2:172.21.15.134:6800/2395610201,v1:172.21.15.134:6801/2395610201] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f12a80043d0 con 0x7f12a40ade90 2024-04-24T23:28:21.946 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:21.944+0000 7f12ae7fc700 1 -- 172.21.15.80:0/1606081887 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f12a80d0ba0 con 0x7f12b8106d50 2024-04-24T23:28:21.946 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:21.944+0000 7f12bd1a5700 1 --2- 172.21.15.80:0/1606081887 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f12a40a2d30 0x7f12a40a51f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f12b819bb40 tx=0x7f12b4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:21.946 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:21.945+0000 7f12bd9a6700 1 --2- 172.21.15.80:0/1606081887 >> [v2:172.21.15.134:6800/2395610201,v1:172.21.15.134:6801/2395610201] conn(0x7f12a40ade90 0x7f12a40b0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:21.947 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:21.945+0000 7f12bd9a6700 1 --2- 172.21.15.80:0/1606081887 >> [v2:172.21.15.134:6800/2395610201,v1:172.21.15.134:6801/2395610201] conn(0x7f12a40ade90 0x7f12a40b0330 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:21.947 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:21.946+0000 7f12ae7fc700 1 -- 172.21.15.80:0/1606081887 <== osd.1 v2:172.21.15.134:6800/2395610201 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f12a80043d0 con 0x7f12a40ade90 2024-04-24T23:28:21.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:21.963+0000 7f12bf409700 1 -- 172.21.15.80:0/1606081887 --> [v2:172.21.15.134:6800/2395610201,v1:172.21.15.134:6801/2395610201] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f12b8101fe0 con 0x7f12a40ade90 2024-04-24T23:28:21.965 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:21.963+0000 7f12ae7fc700 1 -- 172.21.15.80:0/1606081887 <== osd.1 v2:172.21.15.134:6800/2395610201 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f12b8101fe0 con 0x7f12a40ade90 2024-04-24T23:28:21.967 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:21.964+0000 7f12bf409700 1 -- 172.21.15.80:0/1606081887 >> [v2:172.21.15.134:6800/2395610201,v1:172.21.15.134:6801/2395610201] conn(0x7f12a40ade90 msgr2=0x7f12a40b0330 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:21.967 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:21.964+0000 7f12bf409700 1 --2- 172.21.15.80:0/1606081887 >> [v2:172.21.15.134:6800/2395610201,v1:172.21.15.134:6801/2395610201] conn(0x7f12a40ade90 0x7f12a40b0330 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.967 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:21.964+0000 7f12bf409700 1 -- 172.21.15.80:0/1606081887 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f12a40a2d30 msgr2=0x7f12a40a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:21.968 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:21.964+0000 7f12bf409700 1 --2- 172.21.15.80:0/1606081887 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f12a40a2d30 0x7f12a40a51f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f12b819bb40 tx=0x7f12b4009260 comp rx=0 tx=0).stop 2024-04-24T23:28:21.968 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:21.964+0000 7f12bf409700 1 -- 172.21.15.80:0/1606081887 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f12b8106d50 msgr2=0x7f12b81a0d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:21.968 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:21.964+0000 7f12bf409700 1 --2- 172.21.15.80:0/1606081887 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f12b8106d50 0x7f12b81a0d80 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f12a8000c00 tx=0x7f12a800bab0 comp rx=0 tx=0).stop 2024-04-24T23:28:21.968 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:21.964+0000 7f12bd9a6700 1 -- 172.21.15.80:0/1606081887 reap_dead start 2024-04-24T23:28:21.968 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:21.964+0000 7f12bf409700 1 -- 172.21.15.80:0/1606081887 shutdown_connections 2024-04-24T23:28:21.968 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:21.964+0000 7f12bf409700 1 -- 172.21.15.80:0/1606081887 >> 172.21.15.80:0/1606081887 conn(0x7f12b80777a0 msgr2=0x7f12b8102f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:21.968 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:21.964+0000 7f12bf409700 1 -- 172.21.15.80:0/1606081887 shutdown_connections 2024-04-24T23:28:21.968 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:21.964+0000 7f12bf409700 1 -- 172.21.15.80:0/1606081887 wait complete. 2024-04-24T23:28:22.262 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:28:22 smithi080 ceph-mon[29152]: pgmap v329: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s; 57 B/s, 0 keys/s, 1 objects/s recovering 2024-04-24T23:28:22.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:28:22 smithi134 ceph-mon[32619]: pgmap v329: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s; 57 B/s, 0 keys/s, 1 objects/s recovering 2024-04-24T23:28:22.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:28:22 smithi148 ceph-mon[32528]: pgmap v329: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s; 57 B/s, 0 keys/s, 1 objects/s recovering 2024-04-24T23:28:23.270 INFO:teuthology.orchestra.run.smithi080.stdout:107374182467 2024-04-24T23:28:23.271 DEBUG:teuthology.orchestra.run.smithi080:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid ec7885aa-0290-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-04-24T23:28:23.960 INFO:teuthology.orchestra.run.smithi080.stdout:55834574926 2024-04-24T23:28:23.960 DEBUG:teuthology.orchestra.run.smithi080:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid ec7885aa-0290-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-04-24T23:28:24.277 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:28:24 smithi080 ceph-mon[29152]: pgmap v330: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s; 57 B/s, 0 keys/s, 1 objects/s recovering 2024-04-24T23:28:24.281 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:24.278+0000 7ff9a86f9700 1 -- 172.21.15.80:0/4095676786 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff9a006ac00 msgr2=0x7ff9a006afe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:24.281 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:24.278+0000 7ff9a86f9700 1 --2- 172.21.15.80:0/4095676786 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff9a006ac00 0x7ff9a006afe0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7ff9900099c0 tx=0x7ff990009cd0 comp rx=0 tx=0).stop 2024-04-24T23:28:24.281 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:24.278+0000 7ff9a86f9700 1 -- 172.21.15.80:0/4095676786 shutdown_connections 2024-04-24T23:28:24.281 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:24.278+0000 7ff9a86f9700 1 --2- 172.21.15.80:0/4095676786 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff9a01075e0 0x7ff9a010f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:24.281 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:24.278+0000 7ff9a86f9700 1 --2- 172.21.15.80:0/4095676786 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff9a006ac00 0x7ff9a006afe0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:24.281 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:24.278+0000 7ff9a86f9700 1 --2- 172.21.15.80:0/4095676786 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9a006b5b0 0x7ff9a0106eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:24.281 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:24.278+0000 7ff9a86f9700 1 -- 172.21.15.80:0/4095676786 >> 172.21.15.80:0/4095676786 conn(0x7ff9a00771a0 msgr2=0x7ff9a00775b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:24.281 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:24.278+0000 7ff9a86f9700 1 -- 172.21.15.80:0/4095676786 shutdown_connections 2024-04-24T23:28:24.281 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:24.278+0000 7ff9a86f9700 1 -- 172.21.15.80:0/4095676786 wait complete. 2024-04-24T23:28:24.282 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:24.279+0000 7ff9a86f9700 1 Processor -- start 2024-04-24T23:28:24.282 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:24.279+0000 7ff9a86f9700 1 -- start start 2024-04-24T23:28:24.282 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:24.279+0000 7ff9a86f9700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff9a006ac00 0x7ff9a019e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:24.282 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:24.279+0000 7ff9a86f9700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9a006b5b0 0x7ff9a019ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:24.282 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:24.279+0000 7ff9a86f9700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff9a01075e0 0x7ff9a0198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:24.282 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:24.279+0000 7ff9a86f9700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a01125a0 con 0x7ff9a006ac00 2024-04-24T23:28:24.282 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:24.279+0000 7ff9a86f9700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a0112420 con 0x7ff9a006b5b0 2024-04-24T23:28:24.282 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:24.279+0000 7ff9a86f9700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a0112720 con 0x7ff9a01075e0 2024-04-24T23:28:24.282 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:24.279+0000 7ff9a6495700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff9a006ac00 0x7ff9a019e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:24.282 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:24.279+0000 7ff9a6495700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff9a006ac00 0x7ff9a019e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:48616/0 (socket says 172.21.15.80:48616) 2024-04-24T23:28:24.282 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:24.279+0000 7ff9a6495700 1 -- 172.21.15.80:0/2535296886 learned_addr learned my addr 172.21.15.80:0/2535296886 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:28:24.282 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:24.279+0000 7ff9a6c96700 1 --2- 172.21.15.80:0/2535296886 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff9a01075e0 0x7ff9a0198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:24.282 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:24.279+0000 7ff9a5c94700 1 --2- 172.21.15.80:0/2535296886 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9a006b5b0 0x7ff9a019ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:24.283 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:24.280+0000 7ff9a6495700 1 -- 172.21.15.80:0/2535296886 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff9a01075e0 msgr2=0x7ff9a0198860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:24.283 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:24.280+0000 7ff9a6495700 1 --2- 172.21.15.80:0/2535296886 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff9a01075e0 0x7ff9a0198860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:24.283 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:24.280+0000 7ff9a6495700 1 -- 172.21.15.80:0/2535296886 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9a006b5b0 msgr2=0x7ff9a019ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:24.283 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:24.280+0000 7ff9a6495700 1 --2- 172.21.15.80:0/2535296886 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9a006b5b0 0x7ff9a019ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:24.283 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:24.280+0000 7ff9a6495700 1 -- 172.21.15.80:0/2535296886 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9a0198fc0 con 0x7ff9a006ac00 2024-04-24T23:28:24.283 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:24.280+0000 7ff9a5c94700 1 --2- 172.21.15.80:0/2535296886 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9a006b5b0 0x7ff9a019ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:28:24.283 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:24.280+0000 7ff9a6495700 1 --2- 172.21.15.80:0/2535296886 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff9a006ac00 0x7ff9a019e6a0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7ff990006010 tx=0x7ff990009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:24.283 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:24.280+0000 7ff9977fe700 1 -- 172.21.15.80:0/2535296886 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff990027070 con 0x7ff9a006ac00 2024-04-24T23:28:24.283 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:24.280+0000 7ff9a86f9700 1 -- 172.21.15.80:0/2535296886 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9a0199250 con 0x7ff9a006ac00 2024-04-24T23:28:24.283 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:24.280+0000 7ff9977fe700 1 -- 172.21.15.80:0/2535296886 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff990004050 con 0x7ff9a006ac00 2024-04-24T23:28:24.283 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:24.280+0000 7ff9977fe700 1 -- 172.21.15.80:0/2535296886 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff990022e10 con 0x7ff9a006ac00 2024-04-24T23:28:24.283 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:24.280+0000 7ff9a86f9700 1 -- 172.21.15.80:0/2535296886 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9a01a5570 con 0x7ff9a006ac00 2024-04-24T23:28:24.285 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:24.281+0000 7ff9a86f9700 1 -- 172.21.15.80:0/2535296886 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff9a0062980 con 0x7ff9a006ac00 2024-04-24T23:28:24.285 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:24.282+0000 7ff9977fe700 1 -- 172.21.15.80:0/2535296886 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7ff99002a070 con 0x7ff9a006ac00 2024-04-24T23:28:24.285 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:24.283+0000 7ff9977fe700 1 --2- 172.21.15.80:0/2535296886 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7ff98c0a2cb0 0x7ff98c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:24.285 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:24.283+0000 7ff9a5c94700 1 --2- 172.21.15.80:0/2535296886 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7ff98c0a2cb0 0x7ff98c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:24.285 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:24.283+0000 7ff9977fe700 1 -- 172.21.15.80:0/2535296886 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7ff9900d0b00 con 0x7ff9a006ac00 2024-04-24T23:28:24.285 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:24.283+0000 7ff9977fe700 1 --2- 172.21.15.80:0/2535296886 >> [v2:172.21.15.134:6816/108354844,v1:172.21.15.134:6817/108354844] conn(0x7ff98c0ade10 0x7ff98c0b02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:24.285 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:24.283+0000 7ff9977fe700 1 -- 172.21.15.80:0/2535296886 --> [v2:172.21.15.134:6816/108354844,v1:172.21.15.134:6817/108354844] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff99000b0b0 con 0x7ff98c0ade10 2024-04-24T23:28:24.285 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:24.283+0000 7ff9977fe700 1 -- 172.21.15.80:0/2535296886 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7ff9900d0f50 con 0x7ff9a006ac00 2024-04-24T23:28:24.285 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:24.283+0000 7ff9a5c94700 1 --2- 172.21.15.80:0/2535296886 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7ff98c0a2cb0 0x7ff98c0a5170 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff9a0100d30 tx=0x7ff99c00b5f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:24.286 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:24.283+0000 7ff9a6c96700 1 --2- 172.21.15.80:0/2535296886 >> [v2:172.21.15.134:6816/108354844,v1:172.21.15.134:6817/108354844] conn(0x7ff98c0ade10 0x7ff98c0b02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:24.286 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:24.284+0000 7ff9a6c96700 1 --2- 172.21.15.80:0/2535296886 >> [v2:172.21.15.134:6816/108354844,v1:172.21.15.134:6817/108354844] conn(0x7ff98c0ade10 0x7ff98c0b02b0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:24.286 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:24.285+0000 7ff9977fe700 1 -- 172.21.15.80:0/2535296886 <== osd.7 v2:172.21.15.134:6816/108354844 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff99000b0b0 con 0x7ff98c0ade10 2024-04-24T23:28:24.301 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:24.300+0000 7ff9a86f9700 1 -- 172.21.15.80:0/2535296886 --> [v2:172.21.15.134:6816/108354844,v1:172.21.15.134:6817/108354844] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff9a0062d80 con 0x7ff98c0ade10 2024-04-24T23:28:24.302 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:24.301+0000 7ff9977fe700 1 -- 172.21.15.80:0/2535296886 <== osd.7 v2:172.21.15.134:6816/108354844 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7ff9a0062d80 con 0x7ff98c0ade10 2024-04-24T23:28:24.303 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:24.301+0000 7ff9a86f9700 1 -- 172.21.15.80:0/2535296886 >> [v2:172.21.15.134:6816/108354844,v1:172.21.15.134:6817/108354844] conn(0x7ff98c0ade10 msgr2=0x7ff98c0b02b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:24.303 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:24.301+0000 7ff9a86f9700 1 --2- 172.21.15.80:0/2535296886 >> [v2:172.21.15.134:6816/108354844,v1:172.21.15.134:6817/108354844] conn(0x7ff98c0ade10 0x7ff98c0b02b0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:24.303 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:24.301+0000 7ff9a86f9700 1 -- 172.21.15.80:0/2535296886 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7ff98c0a2cb0 msgr2=0x7ff98c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:24.303 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:24.301+0000 7ff9a86f9700 1 --2- 172.21.15.80:0/2535296886 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7ff98c0a2cb0 0x7ff98c0a5170 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff9a0100d30 tx=0x7ff99c00b5f0 comp rx=0 tx=0).stop 2024-04-24T23:28:24.303 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:24.301+0000 7ff9a86f9700 1 -- 172.21.15.80:0/2535296886 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff9a006ac00 msgr2=0x7ff9a019e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:24.303 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:24.301+0000 7ff9a86f9700 1 --2- 172.21.15.80:0/2535296886 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff9a006ac00 0x7ff9a019e6a0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7ff990006010 tx=0x7ff990009f80 comp rx=0 tx=0).stop 2024-04-24T23:28:24.303 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:24.301+0000 7ff9a6c96700 1 -- 172.21.15.80:0/2535296886 reap_dead start 2024-04-24T23:28:24.303 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:24.302+0000 7ff9a86f9700 1 -- 172.21.15.80:0/2535296886 shutdown_connections 2024-04-24T23:28:24.303 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:24.302+0000 7ff9a86f9700 1 -- 172.21.15.80:0/2535296886 >> 172.21.15.80:0/2535296886 conn(0x7ff9a00771a0 msgr2=0x7ff9a00ff470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:24.303 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:24.302+0000 7ff9a86f9700 1 -- 172.21.15.80:0/2535296886 shutdown_connections 2024-04-24T23:28:24.303 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:24.302+0000 7ff9a86f9700 1 -- 172.21.15.80:0/2535296886 wait complete. 2024-04-24T23:28:24.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:28:24 smithi134 ceph-mon[32619]: pgmap v330: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s; 57 B/s, 0 keys/s, 1 objects/s recovering 2024-04-24T23:28:24.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:28:24 smithi148 ceph-mon[32528]: pgmap v330: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s; 57 B/s, 0 keys/s, 1 objects/s recovering 2024-04-24T23:28:25.595 INFO:teuthology.orchestra.run.smithi080.stdout:219043332143 2024-04-24T23:28:25.595 DEBUG:teuthology.orchestra.run.smithi080:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid ec7885aa-0290-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-04-24T23:28:26.270 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:28:26 smithi080 ceph-mon[29152]: pgmap v331: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T23:28:26.297 INFO:teuthology.orchestra.run.smithi080.stdout:141733920831 2024-04-24T23:28:26.298 DEBUG:teuthology.orchestra.run.smithi080:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid ec7885aa-0290-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-04-24T23:28:26.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:28:26 smithi134 ceph-mon[32619]: pgmap v331: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T23:28:26.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:28:26 smithi148 ceph-mon[32528]: pgmap v331: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T23:28:26.743 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:26.740+0000 7ff4323a6700 1 -- 172.21.15.80:0/2513491509 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff42c1024e0 msgr2=0x7ff42c1028c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:26.743 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:26.740+0000 7ff4323a6700 1 --2- 172.21.15.80:0/2513491509 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff42c1024e0 0x7ff42c1028c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7ff420009990 tx=0x7ff420009ca0 comp rx=0 tx=0).stop 2024-04-24T23:28:26.743 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:26.740+0000 7ff4323a6700 1 -- 172.21.15.80:0/2513491509 shutdown_connections 2024-04-24T23:28:26.743 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:26.740+0000 7ff4323a6700 1 --2- 172.21.15.80:0/2513491509 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff42c107570 0x7ff42c10f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:26.743 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:26.740+0000 7ff4323a6700 1 --2- 172.21.15.80:0/2513491509 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff42c1024e0 0x7ff42c1028c0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:26.743 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:26.740+0000 7ff4323a6700 1 --2- 172.21.15.80:0/2513491509 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff42c102e90 0x7ff42c106e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:26.743 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:26.740+0000 7ff4323a6700 1 -- 172.21.15.80:0/2513491509 >> 172.21.15.80:0/2513491509 conn(0x7ff42c079390 msgr2=0x7ff42c0797a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:26.743 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:26.740+0000 7ff4323a6700 1 -- 172.21.15.80:0/2513491509 shutdown_connections 2024-04-24T23:28:26.743 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:26.740+0000 7ff4323a6700 1 -- 172.21.15.80:0/2513491509 wait complete. 2024-04-24T23:28:26.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:26.741+0000 7ff4323a6700 1 Processor -- start 2024-04-24T23:28:26.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:26.741+0000 7ff4323a6700 1 -- start start 2024-04-24T23:28:26.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:26.741+0000 7ff4323a6700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff42c1024e0 0x7ff42c19e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:26.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:26.741+0000 7ff4323a6700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff42c102e90 0x7ff42c19ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:26.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:26.741+0000 7ff4323a6700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff42c107570 0x7ff42c1987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:26.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:26.741+0000 7ff4323a6700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7ff42c112530 con 0x7ff42c107570 2024-04-24T23:28:26.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:26.741+0000 7ff4323a6700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff42c1123b0 con 0x7ff42c1024e0 2024-04-24T23:28:26.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:26.741+0000 7ff4323a6700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff42c1126b0 con 0x7ff42c102e90 2024-04-24T23:28:26.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:26.741+0000 7ff430943700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff42c107570 0x7ff42c1987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:26.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:26.741+0000 7ff430943700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff42c107570 0x7ff42c1987f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:48638/0 (socket says 172.21.15.80:48638) 2024-04-24T23:28:26.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:26.741+0000 7ff430943700 1 -- 172.21.15.80:0/2536612186 learned_addr learned my addr 172.21.15.80:0/2536612186 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:28:26.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:26.741+0000 7ff42b7fe700 1 --2- 172.21.15.80:0/2536612186 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff42c102e90 0x7ff42c19ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:26.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:26.741+0000 7ff42bfff700 1 --2- 172.21.15.80:0/2536612186 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff42c1024e0 0x7ff42c19e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:26.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:26.742+0000 7ff430943700 1 -- 172.21.15.80:0/2536612186 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff42c102e90 msgr2=0x7ff42c19ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:26.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:26.742+0000 7ff430943700 1 --2- 172.21.15.80:0/2536612186 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff42c102e90 0x7ff42c19ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:26.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:26.742+0000 7ff430943700 1 -- 172.21.15.80:0/2536612186 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff42c1024e0 msgr2=0x7ff42c19e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:26.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:26.742+0000 7ff430943700 1 --2- 172.21.15.80:0/2536612186 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff42c1024e0 0x7ff42c19e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:26.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:26.742+0000 7ff430943700 1 -- 172.21.15.80:0/2536612186 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff42c198fd0 con 0x7ff42c107570 2024-04-24T23:28:26.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:26.742+0000 7ff430943700 1 --2- 172.21.15.80:0/2536612186 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff42c107570 0x7ff42c1987f0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7ff424007ee0 tx=0x7ff42400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:26.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:26.742+0000 7ff4297fa700 1 -- 172.21.15.80:0/2536612186 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff424004bb0 con 0x7ff42c107570 2024-04-24T23:28:26.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:26.742+0000 7ff4297fa700 1 -- 172.21.15.80:0/2536612186 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff424004d50 con 0x7ff42c107570 2024-04-24T23:28:26.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:26.742+0000 7ff4297fa700 1 -- 172.21.15.80:0/2536612186 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff424005960 con 0x7ff42c107570 2024-04-24T23:28:26.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:26.742+0000 7ff4323a6700 1 -- 172.21.15.80:0/2536612186 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff42c1992c0 con 0x7ff42c107570 2024-04-24T23:28:26.746 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:26.742+0000 7ff4323a6700 1 -- 172.21.15.80:0/2536612186 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff42c1a54d0 con 0x7ff42c107570 2024-04-24T23:28:26.746 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:26.743+0000 7ff4323a6700 1 -- 172.21.15.80:0/2536612186 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff42c0629c0 con 0x7ff42c107570 2024-04-24T23:28:26.746 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:26.744+0000 7ff4297fa700 1 -- 172.21.15.80:0/2536612186 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7ff42401a020 con 0x7ff42c107570 2024-04-24T23:28:26.747 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:26.745+0000 7ff4297fa700 1 --2- 172.21.15.80:0/2536612186 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7ff4140a2cb0 0x7ff4140a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:26.747 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:26.745+0000 7ff42bfff700 1 --2- 172.21.15.80:0/2536612186 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7ff4140a2cb0 0x7ff4140a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:26.747 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:26.745+0000 7ff4297fa700 1 -- 172.21.15.80:0/2536612186 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7ff4240cf2b0 con 0x7ff42c107570 2024-04-24T23:28:26.747 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:26.745+0000 7ff4297fa700 1 --2- 172.21.15.80:0/2536612186 >> [v2:172.21.15.80:6818/953261724,v1:172.21.15.80:6819/953261724] conn(0x7ff4140ade10 0x7ff4140b02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:26.747 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:26.745+0000 7ff4297fa700 1 -- 172.21.15.80:0/2536612186 --> [v2:172.21.15.80:6818/953261724,v1:172.21.15.80:6819/953261724] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff424017e70 con 0x7ff4140ade10 2024-04-24T23:28:26.747 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:26.745+0000 7ff4297fa700 1 -- 172.21.15.80:0/2536612186 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7ff4240cf700 con 0x7ff42c107570 2024-04-24T23:28:26.747 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:26.745+0000 7ff42b7fe700 1 --2- 172.21.15.80:0/2536612186 >> [v2:172.21.15.80:6818/953261724,v1:172.21.15.80:6819/953261724] conn(0x7ff4140ade10 0x7ff4140b02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:26.747 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:26.745+0000 7ff42bfff700 1 --2- 172.21.15.80:0/2536612186 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7ff4140a2cb0 0x7ff4140a5170 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff42000b5c0 tx=0x7ff420005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:26.747 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:26.746+0000 7ff42b7fe700 1 --2- 172.21.15.80:0/2536612186 >> [v2:172.21.15.80:6818/953261724,v1:172.21.15.80:6819/953261724] conn(0x7ff4140ade10 0x7ff4140b02b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:26.748 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:26.746+0000 7ff4297fa700 1 -- 172.21.15.80:0/2536612186 <== osd.6 v2:172.21.15.80:6818/953261724 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff424017e70 con 0x7ff4140ade10 2024-04-24T23:28:26.764 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:26.762+0000 7ff4323a6700 1 -- 172.21.15.80:0/2536612186 --> [v2:172.21.15.80:6818/953261724,v1:172.21.15.80:6819/953261724] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff42c062e40 con 0x7ff4140ade10 2024-04-24T23:28:26.764 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:26.763+0000 7ff4297fa700 1 -- 172.21.15.80:0/2536612186 <== osd.6 v2:172.21.15.80:6818/953261724 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7ff42c062e40 con 0x7ff4140ade10 2024-04-24T23:28:26.765 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:26.763+0000 7ff4323a6700 1 -- 172.21.15.80:0/2536612186 >> [v2:172.21.15.80:6818/953261724,v1:172.21.15.80:6819/953261724] conn(0x7ff4140ade10 msgr2=0x7ff4140b02b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:26.765 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:26.763+0000 7ff4323a6700 1 --2- 172.21.15.80:0/2536612186 >> [v2:172.21.15.80:6818/953261724,v1:172.21.15.80:6819/953261724] conn(0x7ff4140ade10 0x7ff4140b02b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:26.765 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:26.763+0000 7ff4323a6700 1 -- 172.21.15.80:0/2536612186 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7ff4140a2cb0 msgr2=0x7ff4140a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:26.765 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:26.763+0000 7ff4323a6700 1 --2- 172.21.15.80:0/2536612186 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7ff4140a2cb0 0x7ff4140a5170 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff42000b5c0 tx=0x7ff420005960 comp rx=0 tx=0).stop 2024-04-24T23:28:26.765 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:26.763+0000 7ff4323a6700 1 -- 172.21.15.80:0/2536612186 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff42c107570 msgr2=0x7ff42c1987f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:26.765 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:26.763+0000 7ff4323a6700 1 --2- 172.21.15.80:0/2536612186 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff42c107570 0x7ff42c1987f0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7ff424007ee0 tx=0x7ff42400d3b0 comp rx=0 tx=0).stop 2024-04-24T23:28:26.765 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:26.763+0000 7ff430943700 1 -- 172.21.15.80:0/2536612186 reap_dead start 2024-04-24T23:28:26.765 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:26.763+0000 7ff4323a6700 1 -- 172.21.15.80:0/2536612186 shutdown_connections 2024-04-24T23:28:26.765 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:26.763+0000 7ff4323a6700 1 -- 172.21.15.80:0/2536612186 >> 172.21.15.80:0/2536612186 conn(0x7ff42c079390 msgr2=0x7ff42c101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:26.766 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:26.763+0000 7ff4323a6700 1 -- 172.21.15.80:0/2536612186 shutdown_connections 2024-04-24T23:28:26.766 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:26.763+0000 7ff4323a6700 1 -- 172.21.15.80:0/2536612186 wait complete. 2024-04-24T23:28:27.473 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:26.977+0000 7f85c91f2700 1 -- 172.21.15.80:0/953127811 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f85c406a8e0 msgr2=0x7f85c410f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:27.473 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:26.977+0000 7f85c91f2700 1 --2- 172.21.15.80:0/953127811 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f85c406a8e0 0x7f85c410f0f0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f85b4009990 tx=0x7f85b4009ca0 comp rx=0 tx=0).stop 2024-04-24T23:28:27.473 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:26.977+0000 7f85c91f2700 1 -- 172.21.15.80:0/953127811 shutdown_connections 2024-04-24T23:28:27.473 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:26.977+0000 7f85c91f2700 1 --2- 172.21.15.80:0/953127811 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f85c406a8e0 0x7f85c410f0f0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:27.473 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:26.977+0000 7f85c91f2700 1 --2- 172.21.15.80:0/953127811 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85c41082b0 0x7f85c4108690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:27.473 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:26.977+0000 7f85c91f2700 1 --2- 172.21.15.80:0/953127811 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f85c4069f40 0x7f85c406a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:27.473 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:26.977+0000 7f85c91f2700 1 -- 172.21.15.80:0/953127811 >> 172.21.15.80:0/953127811 conn(0x7f85c40780e0 msgr2=0x7f85c40784f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:27.473 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:26.977+0000 7f85c91f2700 1 -- 172.21.15.80:0/953127811 shutdown_connections 2024-04-24T23:28:27.473 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:26.977+0000 7f85c91f2700 1 -- 172.21.15.80:0/953127811 wait complete. 2024-04-24T23:28:27.473 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:26.978+0000 7f85c91f2700 1 Processor -- start 2024-04-24T23:28:27.473 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:26.978+0000 7f85c91f2700 1 -- start start 2024-04-24T23:28:27.473 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:26.978+0000 7f85c91f2700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85c4069f40 0x7f85c419e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:27.474 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:26.978+0000 7f85c91f2700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f85c406a8e0 0x7f85c419ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:27.474 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:26.978+0000 7f85c91f2700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f85c41082b0 0x7f85c41988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:27.474 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:26.978+0000 7f85c91f2700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f85c41125b0 con 0x7f85c406a8e0 2024-04-24T23:28:27.474 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:26.978+0000 7f85c91f2700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f85c4112430 con 0x7f85c4069f40 2024-04-24T23:28:27.474 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:26.978+0000 7f85c91f2700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f85c4112730 con 0x7f85c41082b0 2024-04-24T23:28:27.474 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:26.978+0000 7f85c259c700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f85c406a8e0 0x7f85c419ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:27.474 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:26.978+0000 7f85c259c700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f85c406a8e0 0x7f85c419ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:48650/0 (socket says 172.21.15.80:48650) 2024-04-24T23:28:27.474 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:26.978+0000 7f85c259c700 1 -- 172.21.15.80:0/662363055 learned_addr learned my addr 172.21.15.80:0/662363055 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:28:27.474 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:26.978+0000 7f85c2d9d700 1 --2- 172.21.15.80:0/662363055 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85c4069f40 0x7f85c419e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:27.474 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:26.978+0000 7f85c359e700 1 --2- 172.21.15.80:0/662363055 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f85c41082b0 0x7f85c41988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:27.474 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:26.978+0000 7f85c259c700 1 -- 172.21.15.80:0/662363055 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f85c41082b0 msgr2=0x7f85c41988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:27.474 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:26.979+0000 7f85c259c700 1 --2- 172.21.15.80:0/662363055 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f85c41082b0 0x7f85c41988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:27.474 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:26.979+0000 7f85c259c700 1 -- 172.21.15.80:0/662363055 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85c4069f40 msgr2=0x7f85c419e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:27.474 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:26.979+0000 7f85c259c700 1 --2- 172.21.15.80:0/662363055 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85c4069f40 0x7f85c419e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:27.475 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:26.979+0000 7f85c259c700 1 -- 172.21.15.80:0/662363055 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85c4199050 con 0x7f85c406a8e0 2024-04-24T23:28:27.475 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:26.979+0000 7f85c259c700 1 --2- 172.21.15.80:0/662363055 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f85c406a8e0 0x7f85c419ec20 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f85ac00b530 tx=0x7f85ac00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:27.475 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:26.979+0000 7f85bbfff700 1 -- 172.21.15.80:0/662363055 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85ac015020 con 0x7f85c406a8e0 2024-04-24T23:28:27.475 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:26.979+0000 7f85c91f2700 1 -- 172.21.15.80:0/662363055 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85c4199340 con 0x7f85c406a8e0 2024-04-24T23:28:27.475 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:26.979+0000 7f85bbfff700 1 -- 172.21.15.80:0/662363055 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85ac011d10 con 0x7f85c406a8e0 2024-04-24T23:28:27.475 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:26.979+0000 7f85c91f2700 1 -- 172.21.15.80:0/662363055 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85c41a55b0 con 0x7f85c406a8e0 2024-04-24T23:28:27.475 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:26.979+0000 7f85bbfff700 1 -- 172.21.15.80:0/662363055 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85ac01a400 con 0x7f85c406a8e0 2024-04-24T23:28:27.475 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:26.980+0000 7f85c91f2700 1 -- 172.21.15.80:0/662363055 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f85c4062940 con 0x7f85c406a8e0 2024-04-24T23:28:27.476 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:26.981+0000 7f85bbfff700 1 -- 172.21.15.80:0/662363055 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f85ac018020 con 0x7f85c406a8e0 2024-04-24T23:28:27.476 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:26.982+0000 7f85bbfff700 1 --2- 172.21.15.80:0/662363055 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f85b00a2cb0 0x7f85b00a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:27.476 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:26.982+0000 7f85c2d9d700 1 --2- 172.21.15.80:0/662363055 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f85b00a2cb0 0x7f85b00a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:27.476 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:26.982+0000 7f85bbfff700 1 -- 172.21.15.80:0/662363055 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f85ac0cf2c0 con 0x7f85c406a8e0 2024-04-24T23:28:27.476 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:26.982+0000 7f85bbfff700 1 --2- 172.21.15.80:0/662363055 >> [v2:172.21.15.80:6802/2597718112,v1:172.21.15.80:6803/2597718112] conn(0x7f85b00ade10 0x7f85b00b02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:27.476 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:26.982+0000 7f85bbfff700 1 -- 172.21.15.80:0/662363055 --> [v2:172.21.15.80:6802/2597718112,v1:172.21.15.80:6803/2597718112] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f85ac013ec0 con 0x7f85b00ade10 2024-04-24T23:28:27.476 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:26.982+0000 7f85bbfff700 1 -- 172.21.15.80:0/662363055 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f85ac0cf710 con 0x7f85c406a8e0 2024-04-24T23:28:27.476 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:26.982+0000 7f85c359e700 1 --2- 172.21.15.80:0/662363055 >> [v2:172.21.15.80:6802/2597718112,v1:172.21.15.80:6803/2597718112] conn(0x7f85b00ade10 0x7f85b00b02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:27.476 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:26.982+0000 7f85c2d9d700 1 --2- 172.21.15.80:0/662363055 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f85b00a2cb0 0x7f85b00a5170 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f85bc009550 tx=0x7f85bc0091b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:27.476 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:26.982+0000 7f85c359e700 1 --2- 172.21.15.80:0/662363055 >> [v2:172.21.15.80:6802/2597718112,v1:172.21.15.80:6803/2597718112] conn(0x7f85b00ade10 0x7f85b00b02b0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:27.476 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:26.983+0000 7f85bbfff700 1 -- 172.21.15.80:0/662363055 <== osd.0 v2:172.21.15.80:6802/2597718112 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f85ac013ec0 con 0x7f85b00ade10 2024-04-24T23:28:27.476 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:26.999+0000 7f85c91f2700 1 -- 172.21.15.80:0/662363055 --> [v2:172.21.15.80:6802/2597718112,v1:172.21.15.80:6803/2597718112] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f85c4062dc0 con 0x7f85b00ade10 2024-04-24T23:28:27.476 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:26.999+0000 7f85bbfff700 1 -- 172.21.15.80:0/662363055 <== osd.0 v2:172.21.15.80:6802/2597718112 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f85c4062dc0 con 0x7f85b00ade10 2024-04-24T23:28:27.476 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:26.999+0000 7f85c91f2700 1 -- 172.21.15.80:0/662363055 >> [v2:172.21.15.80:6802/2597718112,v1:172.21.15.80:6803/2597718112] conn(0x7f85b00ade10 msgr2=0x7f85b00b02b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:27.477 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:26.999+0000 7f85c91f2700 1 --2- 172.21.15.80:0/662363055 >> [v2:172.21.15.80:6802/2597718112,v1:172.21.15.80:6803/2597718112] conn(0x7f85b00ade10 0x7f85b00b02b0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:27.477 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:27.000+0000 7f85c91f2700 1 -- 172.21.15.80:0/662363055 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f85b00a2cb0 msgr2=0x7f85b00a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:27.477 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:27.000+0000 7f85c91f2700 1 --2- 172.21.15.80:0/662363055 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f85b00a2cb0 0x7f85b00a5170 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f85bc009550 tx=0x7f85bc0091b0 comp rx=0 tx=0).stop 2024-04-24T23:28:27.477 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:27.000+0000 7f85c91f2700 1 -- 172.21.15.80:0/662363055 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f85c406a8e0 msgr2=0x7f85c419ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:27.477 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:27.000+0000 7f85c91f2700 1 --2- 172.21.15.80:0/662363055 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f85c406a8e0 0x7f85c419ec20 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f85ac00b530 tx=0x7f85ac00b840 comp rx=0 tx=0).stop 2024-04-24T23:28:27.477 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:27.000+0000 7f85c359e700 1 -- 172.21.15.80:0/662363055 reap_dead start 2024-04-24T23:28:27.477 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:27.000+0000 7f85c91f2700 1 -- 172.21.15.80:0/662363055 shutdown_connections 2024-04-24T23:28:27.477 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:27.000+0000 7f85c91f2700 1 -- 172.21.15.80:0/662363055 >> 172.21.15.80:0/662363055 conn(0x7f85c40780e0 msgr2=0x7f85c40ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:27.477 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:27.000+0000 7f85c91f2700 1 -- 172.21.15.80:0/662363055 shutdown_connections 2024-04-24T23:28:27.477 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:27.000+0000 7f85c91f2700 1 -- 172.21.15.80:0/662363055 wait complete. 2024-04-24T23:28:27.733 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:27.723+0000 7f90674fa700 1 -- 172.21.15.80:0/218537044 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f90601022b0 msgr2=0x7f9060102710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:27.733 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:27.723+0000 7f90674fa700 1 --2- 172.21.15.80:0/218537044 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f90601022b0 0x7f9060102710 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f9050009990 tx=0x7f9050009ca0 comp rx=0 tx=0).stop 2024-04-24T23:28:27.733 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:27.723+0000 7f90674fa700 1 -- 172.21.15.80:0/218537044 shutdown_connections 2024-04-24T23:28:27.733 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:27.723+0000 7f90674fa700 1 --2- 172.21.15.80:0/218537044 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9060102c50 0x7f906010f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:27.733 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:27.723+0000 7f90674fa700 1 --2- 172.21.15.80:0/218537044 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90601082f0 0x7f90601086d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:27.733 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:27.723+0000 7f90674fa700 1 --2- 172.21.15.80:0/218537044 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f90601022b0 0x7f9060102710 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:27.734 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:27.723+0000 7f90674fa700 1 -- 172.21.15.80:0/218537044 >> 172.21.15.80:0/218537044 conn(0x7f90600793e0 msgr2=0x7f90600797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:27.734 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:27.723+0000 7f90674fa700 1 -- 172.21.15.80:0/218537044 shutdown_connections 2024-04-24T23:28:27.734 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:27.723+0000 7f90674fa700 1 -- 172.21.15.80:0/218537044 wait complete. 2024-04-24T23:28:27.734 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:27.724+0000 7f90674fa700 1 Processor -- start 2024-04-24T23:28:27.734 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:27.724+0000 7f90674fa700 1 -- start start 2024-04-24T23:28:27.734 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:27.724+0000 7f90674fa700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90601022b0 0x7f906019e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:27.734 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:27.724+0000 7f90674fa700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9060102c50 0x7f906019eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:27.734 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:27.724+0000 7f90674fa700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90601082f0 0x7f90601987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:27.734 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:27.724+0000 7f90674fa700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f9060112550 con 0x7f9060102c50 2024-04-24T23:28:27.734 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:27.724+0000 7f90674fa700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f90601123d0 con 0x7f90601022b0 2024-04-24T23:28:27.734 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:27.724+0000 7f90674fa700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f90601126d0 con 0x7f90601082f0 2024-04-24T23:28:27.734 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:27.724+0000 7f9064a95700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9060102c50 0x7f906019eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:27.734 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:27.724+0000 7f9064a95700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9060102c50 0x7f906019eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:48674/0 (socket says 172.21.15.80:48674) 2024-04-24T23:28:27.735 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:27.724+0000 7f9064a95700 1 -- 172.21.15.80:0/2167966829 learned_addr learned my addr 172.21.15.80:0/2167966829 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:28:27.735 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:27.724+0000 7f9065296700 1 --2- 172.21.15.80:0/2167966829 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90601022b0 0x7f906019e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:27.735 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:27.724+0000 7f9064a95700 1 -- 172.21.15.80:0/2167966829 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90601082f0 msgr2=0x7f90601987d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:27.735 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:27.724+0000 7f9064a95700 1 --2- 172.21.15.80:0/2167966829 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90601082f0 0x7f90601987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:27.735 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:27.724+0000 7f9064a95700 1 -- 172.21.15.80:0/2167966829 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90601022b0 msgr2=0x7f906019e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:27.735 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:27.724+0000 7f9064a95700 1 --2- 172.21.15.80:0/2167966829 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90601022b0 0x7f906019e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:27.735 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:27.724+0000 7f9064a95700 1 -- 172.21.15.80:0/2167966829 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9060198fb0 con 0x7f9060102c50 2024-04-24T23:28:27.735 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:27.724+0000 7f9065296700 1 --2- 172.21.15.80:0/2167966829 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90601022b0 0x7f906019e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:28:27.735 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:27.724+0000 7f9064a95700 1 --2- 172.21.15.80:0/2167966829 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9060102c50 0x7f906019eb90 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f9050000c00 tx=0x7f9050003730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:27.735 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:27.724+0000 7f90567fc700 1 -- 172.21.15.80:0/2167966829 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9050027070 con 0x7f9060102c50 2024-04-24T23:28:27.735 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:27.725+0000 7f90674fa700 1 -- 172.21.15.80:0/2167966829 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9060199240 con 0x7f9060102c50 2024-04-24T23:28:27.735 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:27.725+0000 7f90567fc700 1 -- 172.21.15.80:0/2167966829 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9050004190 con 0x7f9060102c50 2024-04-24T23:28:27.735 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:27.725+0000 7f90567fc700 1 -- 172.21.15.80:0/2167966829 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f905000f460 con 0x7f9060102c50 2024-04-24T23:28:27.735 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:27.725+0000 7f90674fa700 1 -- 172.21.15.80:0/2167966829 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90601a54b0 con 0x7f9060102c50 2024-04-24T23:28:27.736 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:27.725+0000 7f90674fa700 1 -- 172.21.15.80:0/2167966829 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f90600629c0 con 0x7f9060102c50 2024-04-24T23:28:27.736 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:27.727+0000 7f90567fc700 1 -- 172.21.15.80:0/2167966829 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f905002a070 con 0x7f9060102c50 2024-04-24T23:28:27.736 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:27.727+0000 7f90567fc700 1 --2- 172.21.15.80:0/2167966829 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f904c0a2cb0 0x7f904c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:27.736 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:27.727+0000 7f9065296700 1 --2- 172.21.15.80:0/2167966829 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f904c0a2cb0 0x7f904c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:27.736 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:27.727+0000 7f90567fc700 1 -- 172.21.15.80:0/2167966829 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f90500d0f50 con 0x7f9060102c50 2024-04-24T23:28:27.736 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:27.727+0000 7f90567fc700 1 --2- 172.21.15.80:0/2167966829 >> [v2:172.21.15.148:6816/3559655329,v1:172.21.15.148:6817/3559655329] conn(0x7f904c0ade10 0x7f904c0b02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:27.736 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:27.727+0000 7f90567fc700 1 -- 172.21.15.80:0/2167966829 --> [v2:172.21.15.148:6816/3559655329,v1:172.21.15.148:6817/3559655329] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9050024ec0 con 0x7f904c0ade10 2024-04-24T23:28:27.736 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:27.727+0000 7f90567fc700 1 -- 172.21.15.80:0/2167966829 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f90500d13a0 con 0x7f9060102c50 2024-04-24T23:28:27.736 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:27.728+0000 7f9065296700 1 --2- 172.21.15.80:0/2167966829 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f904c0a2cb0 0x7f904c0a5170 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f90580060b0 tx=0x7f9058007620 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:27.736 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:27.728+0000 7f9065a97700 1 --2- 172.21.15.80:0/2167966829 >> [v2:172.21.15.148:6816/3559655329,v1:172.21.15.148:6817/3559655329] conn(0x7f904c0ade10 0x7f904c0b02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:27.736 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:27.728+0000 7f9065a97700 1 --2- 172.21.15.80:0/2167966829 >> [v2:172.21.15.148:6816/3559655329,v1:172.21.15.148:6817/3559655329] conn(0x7f904c0ade10 0x7f904c0b02b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:27.736 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:27.729+0000 7f90567fc700 1 -- 172.21.15.80:0/2167966829 <== osd.8 v2:172.21.15.148:6816/3559655329 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9050024ec0 con 0x7f904c0ade10 2024-04-24T23:28:27.746 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:27.745+0000 7f90674fa700 1 -- 172.21.15.80:0/2167966829 --> [v2:172.21.15.148:6816/3559655329,v1:172.21.15.148:6817/3559655329] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9060062dc0 con 0x7f904c0ade10 2024-04-24T23:28:27.747 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:27.746+0000 7f90567fc700 1 -- 172.21.15.80:0/2167966829 <== osd.8 v2:172.21.15.148:6816/3559655329 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f9060062dc0 con 0x7f904c0ade10 2024-04-24T23:28:27.748 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:27.746+0000 7f90674fa700 1 -- 172.21.15.80:0/2167966829 >> [v2:172.21.15.148:6816/3559655329,v1:172.21.15.148:6817/3559655329] conn(0x7f904c0ade10 msgr2=0x7f904c0b02b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:27.748 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:27.746+0000 7f90674fa700 1 --2- 172.21.15.80:0/2167966829 >> [v2:172.21.15.148:6816/3559655329,v1:172.21.15.148:6817/3559655329] conn(0x7f904c0ade10 0x7f904c0b02b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:27.748 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:27.747+0000 7f90674fa700 1 -- 172.21.15.80:0/2167966829 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f904c0a2cb0 msgr2=0x7f904c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:27.748 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:27.747+0000 7f90674fa700 1 --2- 172.21.15.80:0/2167966829 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f904c0a2cb0 0x7f904c0a5170 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f90580060b0 tx=0x7f9058007620 comp rx=0 tx=0).stop 2024-04-24T23:28:27.748 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:27.747+0000 7f90674fa700 1 -- 172.21.15.80:0/2167966829 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9060102c50 msgr2=0x7f906019eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:27.748 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:27.747+0000 7f90674fa700 1 --2- 172.21.15.80:0/2167966829 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9060102c50 0x7f906019eb90 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f9050000c00 tx=0x7f9050003730 comp rx=0 tx=0).stop 2024-04-24T23:28:27.749 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:27.747+0000 7f9065a97700 1 -- 172.21.15.80:0/2167966829 reap_dead start 2024-04-24T23:28:27.749 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:27.747+0000 7f90674fa700 1 -- 172.21.15.80:0/2167966829 shutdown_connections 2024-04-24T23:28:27.749 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:27.747+0000 7f90674fa700 1 -- 172.21.15.80:0/2167966829 >> 172.21.15.80:0/2167966829 conn(0x7f90600793e0 msgr2=0x7f9060100ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:27.749 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:27.747+0000 7f90674fa700 1 -- 172.21.15.80:0/2167966829 shutdown_connections 2024-04-24T23:28:27.749 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:27.747+0000 7f90674fa700 1 -- 172.21.15.80:0/2167966829 wait complete. 2024-04-24T23:28:28.381 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:28:28 smithi080 ceph-mon[29152]: pgmap v332: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-24T23:28:28.407 INFO:teuthology.orchestra.run.smithi080.stdout:253403070508 2024-04-24T23:28:28.408 DEBUG:teuthology.orchestra.run.smithi080:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid ec7885aa-0290-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-04-24T23:28:28.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:28:28 smithi134 ceph-mon[32619]: pgmap v332: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-24T23:28:28.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:28:28 smithi148 ceph-mon[32528]: pgmap v332: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-24T23:28:28.849 INFO:teuthology.orchestra.run.smithi080.stdout:197568495670 2024-04-24T23:28:28.849 DEBUG:teuthology.orchestra.run.smithi080:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid ec7885aa-0290-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-04-24T23:28:29.689 INFO:teuthology.orchestra.run.smithi080.stdout:34359738451 2024-04-24T23:28:29.689 DEBUG:teuthology.orchestra.run.smithi080:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid ec7885aa-0290-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-04-24T23:28:29.894 INFO:teuthology.orchestra.run.smithi080.stderr:Inferring config /var/lib/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:28:29.894 INFO:teuthology.orchestra.run.smithi080.stderr:Inferring config /var/lib/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:28:29.894 INFO:teuthology.orchestra.run.smithi080.stderr:Inferring config /var/lib/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:28:29.899 INFO:teuthology.orchestra.run.smithi080.stderr:Inferring config /var/lib/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:28:29.899 INFO:teuthology.orchestra.run.smithi080.stderr:Inferring config /var/lib/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:28:29.905 INFO:teuthology.orchestra.run.smithi080.stderr:Inferring config /var/lib/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:28:29.906 INFO:teuthology.orchestra.run.smithi080.stderr:Inferring config /var/lib/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:28:29.906 INFO:teuthology.orchestra.run.smithi080.stderr:Inferring config /var/lib/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:28:29.908 INFO:teuthology.orchestra.run.smithi080.stderr:Inferring config /var/lib/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:28:29.912 INFO:teuthology.orchestra.run.smithi080.stderr:Inferring config /var/lib/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:28:29.915 INFO:teuthology.orchestra.run.smithi080.stderr:Inferring config /var/lib/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:28:30.311 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:28:30 smithi080 ceph-mon[29152]: pgmap v333: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-04-24T23:28:30.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:28:30 smithi134 ceph-mon[32619]: pgmap v333: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-04-24T23:28:30.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:28:30 smithi148 ceph-mon[32528]: pgmap v333: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-04-24T23:28:32.282 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:28:32 smithi080 ceph-mon[29152]: pgmap v334: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T23:28:32.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:28:32 smithi134 ceph-mon[32619]: pgmap v334: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T23:28:32.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:28:32 smithi148 ceph-mon[32528]: pgmap v334: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T23:28:34.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:28:34 smithi080 ceph-mon[29152]: pgmap v335: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-04-24T23:28:34.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:28:34 smithi134 ceph-mon[32619]: pgmap v335: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-04-24T23:28:34.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:28:34 smithi148 ceph-mon[32528]: pgmap v335: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-04-24T23:28:35.081 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:35.069+0000 7f7a33455700 1 -- 172.21.15.80:0/2167853181 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7a2c1071a0 msgr2=0x7f7a2c107580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:35.081 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:35.069+0000 7f7a33455700 1 --2- 172.21.15.80:0/2167853181 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7a2c1071a0 0x7f7a2c107580 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f7a28009990 tx=0x7f7a28009ca0 comp rx=0 tx=0).stop 2024-04-24T23:28:35.081 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:35.069+0000 7f7a33455700 1 -- 172.21.15.80:0/2167853181 shutdown_connections 2024-04-24T23:28:35.081 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:35.069+0000 7f7a33455700 1 --2- 172.21.15.80:0/2167853181 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7a2c101af0 0x7f7a2c10dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:35.081 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:35.069+0000 7f7a33455700 1 --2- 172.21.15.80:0/2167853181 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7a2c1071a0 0x7f7a2c107580 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:35.081 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:35.069+0000 7f7a33455700 1 --2- 172.21.15.80:0/2167853181 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7a2c101130 0x7f7a2c1015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:35.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:35.069+0000 7f7a33455700 1 -- 172.21.15.80:0/2167853181 >> 172.21.15.80:0/2167853181 conn(0x7f7a2c0780e0 msgr2=0x7f7a2c0784f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:35.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:35.069+0000 7f7a33455700 1 -- 172.21.15.80:0/2167853181 shutdown_connections 2024-04-24T23:28:35.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:35.069+0000 7f7a33455700 1 -- 172.21.15.80:0/2167853181 wait complete. 2024-04-24T23:28:35.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:35.070+0000 7f7a33455700 1 Processor -- start 2024-04-24T23:28:35.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:35.070+0000 7f7a33455700 1 -- start start 2024-04-24T23:28:35.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:35.070+0000 7f7a33455700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7a2c101130 0x7f7a2c19d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:35.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:35.070+0000 7f7a33455700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7a2c101af0 0x7f7a2c19dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:35.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:35.070+0000 7f7a33455700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7a2c1071a0 0x7f7a2c197770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:35.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:35.070+0000 7f7a33455700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f7a2c111410 con 0x7f7a2c1071a0 2024-04-24T23:28:35.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:35.070+0000 7f7a33455700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7a2c111290 con 0x7f7a2c101130 2024-04-24T23:28:35.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:35.070+0000 7f7a33455700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f7a2c111590 con 0x7f7a2c101af0 2024-04-24T23:28:35.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:35.070+0000 7f7a319f2700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7a2c1071a0 0x7f7a2c197770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:35.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:35.070+0000 7f7a319f2700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7a2c1071a0 0x7f7a2c197770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:51730/0 (socket says 172.21.15.80:51730) 2024-04-24T23:28:35.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:35.070+0000 7f7a319f2700 1 -- 172.21.15.80:0/73479854 learned_addr learned my addr 172.21.15.80:0/73479854 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:28:35.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:35.070+0000 7f7a311f1700 1 --2- 172.21.15.80:0/73479854 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7a2c101130 0x7f7a2c19d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:35.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:35.070+0000 7f7a309f0700 1 --2- 172.21.15.80:0/73479854 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7a2c101af0 0x7f7a2c19dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:35.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:35.071+0000 7f7a319f2700 1 -- 172.21.15.80:0/73479854 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7a2c101af0 msgr2=0x7f7a2c19dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:35.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:35.071+0000 7f7a319f2700 1 --2- 172.21.15.80:0/73479854 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7a2c101af0 0x7f7a2c19dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:35.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:35.071+0000 7f7a319f2700 1 -- 172.21.15.80:0/73479854 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7a2c101130 msgr2=0x7f7a2c19d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:35.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:35.071+0000 7f7a319f2700 1 --2- 172.21.15.80:0/73479854 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7a2c101130 0x7f7a2c19d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:35.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:35.071+0000 7f7a319f2700 1 -- 172.21.15.80:0/73479854 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a2c198050 con 0x7f7a2c1071a0 2024-04-24T23:28:35.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:35.071+0000 7f7a319f2700 1 --2- 172.21.15.80:0/73479854 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7a2c1071a0 0x7f7a2c197770 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f7a24007ee0 tx=0x7f7a2400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:35.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:35.071+0000 7f7a227fc700 1 -- 172.21.15.80:0/73479854 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a24004bb0 con 0x7f7a2c1071a0 2024-04-24T23:28:35.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:35.071+0000 7f7a227fc700 1 -- 172.21.15.80:0/73479854 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a24004d50 con 0x7f7a2c1071a0 2024-04-24T23:28:35.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:35.071+0000 7f7a33455700 1 -- 172.21.15.80:0/73479854 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a2c198340 con 0x7f7a2c1071a0 2024-04-24T23:28:35.084 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:35.071+0000 7f7a227fc700 1 -- 172.21.15.80:0/73479854 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a24005960 con 0x7f7a2c1071a0 2024-04-24T23:28:35.084 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:35.071+0000 7f7a33455700 1 -- 172.21.15.80:0/73479854 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a2c1a4620 con 0x7f7a2c1071a0 2024-04-24T23:28:35.084 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:35.072+0000 7f7a33455700 1 -- 172.21.15.80:0/73479854 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a2c102850 con 0x7f7a2c1071a0 2024-04-24T23:28:35.084 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:35.073+0000 7f7a227fc700 1 -- 172.21.15.80:0/73479854 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f7a2401a020 con 0x7f7a2c1071a0 2024-04-24T23:28:35.084 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:35.074+0000 7f7a227fc700 1 --2- 172.21.15.80:0/73479854 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f7a180a2cb0 0x7f7a180a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:35.084 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:35.074+0000 7f7a311f1700 1 --2- 172.21.15.80:0/73479854 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f7a180a2cb0 0x7f7a180a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:35.084 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:35.074+0000 7f7a227fc700 1 -- 172.21.15.80:0/73479854 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f7a240cec00 con 0x7f7a2c1071a0 2024-04-24T23:28:35.084 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:35.074+0000 7f7a311f1700 1 --2- 172.21.15.80:0/73479854 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f7a180a2cb0 0x7f7a180a5170 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f7a2800b5c0 tx=0x7f7a28005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:35.084 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:35.078+0000 7f7a227fc700 1 -- 172.21.15.80:0/73479854 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f7a24010070 con 0x7f7a2c1071a0 2024-04-24T23:28:35.282 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:35.281+0000 7f7a33455700 1 -- 172.21.15.80:0/73479854 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f7a2c02db30 con 0x7f7a2c1071a0 2024-04-24T23:28:35.282 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:35.282+0000 7f7a227fc700 1 -- 172.21.15.80:0/73479854 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f7a2409b6a0 con 0x7f7a2c1071a0 2024-04-24T23:28:35.282 INFO:teuthology.orchestra.run.smithi080.stdout:163208757309 2024-04-24T23:28:35.284 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:35.284+0000 7f7a33455700 1 -- 172.21.15.80:0/73479854 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f7a180a2cb0 msgr2=0x7f7a180a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:35.284 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:35.284+0000 7f7a33455700 1 --2- 172.21.15.80:0/73479854 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f7a180a2cb0 0x7f7a180a5170 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f7a2800b5c0 tx=0x7f7a28005960 comp rx=0 tx=0).stop 2024-04-24T23:28:35.284 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:35.284+0000 7f7a33455700 1 -- 172.21.15.80:0/73479854 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7a2c1071a0 msgr2=0x7f7a2c197770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:35.284 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:35.284+0000 7f7a33455700 1 --2- 172.21.15.80:0/73479854 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7a2c1071a0 0x7f7a2c197770 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f7a24007ee0 tx=0x7f7a2400d3b0 comp rx=0 tx=0).stop 2024-04-24T23:28:35.284 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:35.284+0000 7f7a33455700 1 -- 172.21.15.80:0/73479854 shutdown_connections 2024-04-24T23:28:35.284 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:35.284+0000 7f7a33455700 1 --2- 172.21.15.80:0/73479854 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f7a180a2cb0 0x7f7a180a5170 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:35.284 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:35.284+0000 7f7a33455700 1 --2- 172.21.15.80:0/73479854 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7a2c1071a0 0x7f7a2c197770 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:35.284 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:35.284+0000 7f7a33455700 1 --2- 172.21.15.80:0/73479854 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7a2c101130 0x7f7a2c19d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:35.284 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:35.284+0000 7f7a33455700 1 --2- 172.21.15.80:0/73479854 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7a2c101af0 0x7f7a2c19dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:35.284 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:35.284+0000 7f7a33455700 1 -- 172.21.15.80:0/73479854 >> 172.21.15.80:0/73479854 conn(0x7f7a2c0780e0 msgr2=0x7f7a2c0ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:35.284 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:35.284+0000 7f7a33455700 1 -- 172.21.15.80:0/73479854 shutdown_connections 2024-04-24T23:28:35.284 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:35.284+0000 7f7a33455700 1 -- 172.21.15.80:0/73479854 wait complete. 2024-04-24T23:28:36.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:28:36 smithi134 ceph-mon[32619]: pgmap v336: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-04-24T23:28:36.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:28:36 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/73479854' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-24T23:28:36.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:28:36 smithi148 ceph-mon[32528]: pgmap v336: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-04-24T23:28:36.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:28:36 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/73479854' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-24T23:28:36.514 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:28:36 smithi080 ceph-mon[29152]: pgmap v336: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-04-24T23:28:36.514 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:28:36 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/73479854' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-24T23:28:38.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:28:38 smithi134 ceph-mon[32619]: pgmap v337: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:28:38.502 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:28:38 smithi080 ceph-mon[29152]: pgmap v337: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:28:38.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:28:38 smithi148 ceph-mon[32528]: pgmap v337: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:28:39.353 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:28:39 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/1821490898' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-24T23:28:39.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:28:39 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/1821490898' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-24T23:28:39.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:28:39 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/1821490898' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-24T23:28:40.108 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:40.103+0000 7ffbc8108700 1 -- 172.21.15.80:0/1684022212 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffbc01011c0 msgr2=0x7ffbc010f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:40.108 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:40.103+0000 7ffbc8108700 1 --2- 172.21.15.80:0/1684022212 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffbc01011c0 0x7ffbc010f100 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7ffbb800b0a0 tx=0x7ffbb800b3b0 comp rx=0 tx=0).stop 2024-04-24T23:28:40.108 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:40.103+0000 7ffbc8108700 1 -- 172.21.15.80:0/1684022212 shutdown_connections 2024-04-24T23:28:40.108 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:40.103+0000 7ffbc8108700 1 --2- 172.21.15.80:0/1684022212 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffbc01011c0 0x7ffbc010f100 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:40.108 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:40.103+0000 7ffbc8108700 1 --2- 172.21.15.80:0/1684022212 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffbc01082c0 0x7ffbc01086a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:40.108 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:40.103+0000 7ffbc8108700 1 --2- 172.21.15.80:0/1684022212 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffbc0100820 0x7ffbc0100c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:40.108 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:40.103+0000 7ffbc8108700 1 -- 172.21.15.80:0/1684022212 >> 172.21.15.80:0/1684022212 conn(0x7ffbc00777a0 msgr2=0x7ffbc00feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:40.108 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:40.103+0000 7ffbc8108700 1 -- 172.21.15.80:0/1684022212 shutdown_connections 2024-04-24T23:28:40.108 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:40.103+0000 7ffbc8108700 1 -- 172.21.15.80:0/1684022212 wait complete. 2024-04-24T23:28:40.108 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:40.104+0000 7ffbc8108700 1 Processor -- start 2024-04-24T23:28:40.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:40.104+0000 7ffbc8108700 1 -- start start 2024-04-24T23:28:40.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:40.104+0000 7ffbc8108700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffbc0100820 0x7ffbc019e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:40.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:40.104+0000 7ffbc8108700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffbc01011c0 0x7ffbc019ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:40.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:40.104+0000 7ffbc8108700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffbc01082c0 0x7ffbc0198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:40.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:40.104+0000 7ffbc8108700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7ffbc0112510 con 0x7ffbc01082c0 2024-04-24T23:28:40.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:40.104+0000 7ffbc8108700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ffbc0112390 con 0x7ffbc0100820 2024-04-24T23:28:40.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:40.104+0000 7ffbc8108700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ffbc0112690 con 0x7ffbc01011c0 2024-04-24T23:28:40.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:40.104+0000 7ffbc66a5700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffbc01082c0 0x7ffbc0198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:40.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:40.104+0000 7ffbc66a5700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffbc01082c0 0x7ffbc0198770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:51760/0 (socket says 172.21.15.80:51760) 2024-04-24T23:28:40.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:40.104+0000 7ffbc66a5700 1 -- 172.21.15.80:0/1605829689 learned_addr learned my addr 172.21.15.80:0/1605829689 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:28:40.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:40.104+0000 7ffbc5ea4700 1 --2- 172.21.15.80:0/1605829689 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffbc0100820 0x7ffbc019e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:40.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:40.104+0000 7ffbc56a3700 1 --2- 172.21.15.80:0/1605829689 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffbc01011c0 0x7ffbc019ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:40.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:40.104+0000 7ffbc66a5700 1 -- 172.21.15.80:0/1605829689 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffbc01011c0 msgr2=0x7ffbc019ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:40.110 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:40.104+0000 7ffbc66a5700 1 --2- 172.21.15.80:0/1605829689 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffbc01011c0 0x7ffbc019ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:40.110 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:40.104+0000 7ffbc66a5700 1 -- 172.21.15.80:0/1605829689 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffbc0100820 msgr2=0x7ffbc019e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:40.110 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:40.104+0000 7ffbc66a5700 1 --2- 172.21.15.80:0/1605829689 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffbc0100820 0x7ffbc019e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:40.110 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:40.104+0000 7ffbc66a5700 1 -- 172.21.15.80:0/1605829689 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbc0199050 con 0x7ffbc01082c0 2024-04-24T23:28:40.110 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:40.105+0000 7ffbc66a5700 1 --2- 172.21.15.80:0/1605829689 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffbc01082c0 0x7ffbc0198770 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7ffbb80062a0 tx=0x7ffbb8004d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:40.110 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:40.105+0000 7ffbb6ffd700 1 -- 172.21.15.80:0/1605829689 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbb8021070 con 0x7ffbc01082c0 2024-04-24T23:28:40.110 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:40.105+0000 7ffbc8108700 1 -- 172.21.15.80:0/1605829689 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbc01992e0 con 0x7ffbc01082c0 2024-04-24T23:28:40.110 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:40.105+0000 7ffbb6ffd700 1 -- 172.21.15.80:0/1605829689 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffbb8003c00 con 0x7ffbc01082c0 2024-04-24T23:28:40.110 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:40.105+0000 7ffbb6ffd700 1 -- 172.21.15.80:0/1605829689 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbb8004880 con 0x7ffbc01082c0 2024-04-24T23:28:40.110 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:40.105+0000 7ffbc8108700 1 -- 172.21.15.80:0/1605829689 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbc01a5620 con 0x7ffbc01082c0 2024-04-24T23:28:40.110 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:40.106+0000 7ffbc8108700 1 -- 172.21.15.80:0/1605829689 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbc0103b40 con 0x7ffbc01082c0 2024-04-24T23:28:40.110 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:40.107+0000 7ffbb6ffd700 1 -- 172.21.15.80:0/1605829689 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7ffbb8003db0 con 0x7ffbc01082c0 2024-04-24T23:28:40.110 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:40.107+0000 7ffbb6ffd700 1 --2- 172.21.15.80:0/1605829689 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7ffbac0a2cb0 0x7ffbac0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:40.111 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:40.107+0000 7ffbc5ea4700 1 --2- 172.21.15.80:0/1605829689 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7ffbac0a2cb0 0x7ffbac0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:40.111 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:40.107+0000 7ffbb6ffd700 1 -- 172.21.15.80:0/1605829689 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7ffbb800f020 con 0x7ffbc01082c0 2024-04-24T23:28:40.111 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:40.108+0000 7ffbc5ea4700 1 --2- 172.21.15.80:0/1605829689 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7ffbac0a2cb0 0x7ffbac0a5170 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ffbbc007b60 tx=0x7ffbbc0073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:40.112 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:40.112+0000 7ffbb6ffd700 1 -- 172.21.15.80:0/1605829689 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ffbb802a020 con 0x7ffbc01082c0 2024-04-24T23:28:40.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:28:40 smithi080 ceph-mon[29152]: pgmap v338: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-04-24T23:28:40.318 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:40.318+0000 7ffbc8108700 1 -- 172.21.15.80:0/1605829689 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7ffbc0199510 con 0x7ffbc01082c0 2024-04-24T23:28:40.318 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:40.318+0000 7ffbb6ffd700 1 -- 172.21.15.80:0/1605829689 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ffbb809d540 con 0x7ffbc01082c0 2024-04-24T23:28:40.318 INFO:teuthology.orchestra.run.smithi080.stdout:77309411405 2024-04-24T23:28:40.320 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:40.320+0000 7ffbc8108700 1 -- 172.21.15.80:0/1605829689 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7ffbac0a2cb0 msgr2=0x7ffbac0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:40.320 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:40.320+0000 7ffbc8108700 1 --2- 172.21.15.80:0/1605829689 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7ffbac0a2cb0 0x7ffbac0a5170 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ffbbc007b60 tx=0x7ffbbc0073d0 comp rx=0 tx=0).stop 2024-04-24T23:28:40.320 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:40.320+0000 7ffbc8108700 1 -- 172.21.15.80:0/1605829689 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffbc01082c0 msgr2=0x7ffbc0198770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:40.320 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:40.320+0000 7ffbc8108700 1 --2- 172.21.15.80:0/1605829689 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffbc01082c0 0x7ffbc0198770 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7ffbb80062a0 tx=0x7ffbb8004d70 comp rx=0 tx=0).stop 2024-04-24T23:28:40.320 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:40.320+0000 7ffbc8108700 1 -- 172.21.15.80:0/1605829689 shutdown_connections 2024-04-24T23:28:40.320 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:40.320+0000 7ffbc8108700 1 --2- 172.21.15.80:0/1605829689 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7ffbac0a2cb0 0x7ffbac0a5170 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:40.320 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:40.320+0000 7ffbc8108700 1 --2- 172.21.15.80:0/1605829689 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffbc01082c0 0x7ffbc0198770 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:40.321 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:40.320+0000 7ffbc8108700 1 --2- 172.21.15.80:0/1605829689 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffbc0100820 0x7ffbc019e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:40.321 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:40.320+0000 7ffbc8108700 1 --2- 172.21.15.80:0/1605829689 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffbc01011c0 0x7ffbc019ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:40.321 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:40.320+0000 7ffbc8108700 1 -- 172.21.15.80:0/1605829689 >> 172.21.15.80:0/1605829689 conn(0x7ffbc00777a0 msgr2=0x7ffbc00feeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:40.321 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:40.320+0000 7ffbc8108700 1 -- 172.21.15.80:0/1605829689 shutdown_connections 2024-04-24T23:28:40.321 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:40.320+0000 7ffbc8108700 1 -- 172.21.15.80:0/1605829689 wait complete. 2024-04-24T23:28:40.407 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:28:40 smithi148 ceph-mon[32528]: pgmap v338: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-04-24T23:28:40.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:28:40 smithi134 ceph-mon[32619]: pgmap v338: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-04-24T23:28:40.637 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:38.612+0000 7f90ef788700 1 -- 172.21.15.80:0/3176330020 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f90e8102df0 msgr2=0x7f90e8103270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:40.637 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:38.612+0000 7f90ef788700 1 --2- 172.21.15.80:0/3176330020 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f90e8102df0 0x7f90e8103270 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f90d80099c0 tx=0x7f90d8009cd0 comp rx=0 tx=0).stop 2024-04-24T23:28:40.637 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:38.612+0000 7f90ef788700 1 -- 172.21.15.80:0/3176330020 shutdown_connections 2024-04-24T23:28:40.637 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:38.612+0000 7f90ef788700 1 --2- 172.21.15.80:0/3176330020 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90e81037b0 0x7f90e810e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:40.637 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:38.612+0000 7f90ef788700 1 --2- 172.21.15.80:0/3176330020 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90e8101510 0x7f90e81018f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:40.637 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:38.612+0000 7f90ef788700 1 --2- 172.21.15.80:0/3176330020 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f90e8102df0 0x7f90e8103270 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:40.637 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:38.612+0000 7f90ef788700 1 -- 172.21.15.80:0/3176330020 >> 172.21.15.80:0/3176330020 conn(0x7f90e80771d0 msgr2=0x7f90e80775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:40.637 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:38.612+0000 7f90ef788700 1 -- 172.21.15.80:0/3176330020 shutdown_connections 2024-04-24T23:28:40.637 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:38.612+0000 7f90ef788700 1 -- 172.21.15.80:0/3176330020 wait complete. 2024-04-24T23:28:40.638 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:38.613+0000 7f90ef788700 1 Processor -- start 2024-04-24T23:28:40.638 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:38.613+0000 7f90ef788700 1 -- start start 2024-04-24T23:28:40.638 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:38.613+0000 7f90ef788700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90e8101510 0x7f90e8199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:40.638 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:38.613+0000 7f90ef788700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90e8102df0 0x7f90e8199ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:40.638 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:38.613+0000 7f90ef788700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f90e81037b0 0x7f90e81a4a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:40.638 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:38.613+0000 7f90ef788700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f90e8111640 con 0x7f90e81037b0 2024-04-24T23:28:40.638 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:38.613+0000 7f90ef788700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f90e81114c0 con 0x7f90e8101510 2024-04-24T23:28:40.638 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:38.613+0000 7f90ef788700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f90e81117c0 con 0x7f90e8102df0 2024-04-24T23:28:40.638 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:38.613+0000 7f90edd25700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f90e81037b0 0x7f90e81a4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:40.638 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:38.613+0000 7f90edd25700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f90e81037b0 0x7f90e81a4a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:51740/0 (socket says 172.21.15.80:51740) 2024-04-24T23:28:40.638 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:38.613+0000 7f90edd25700 1 -- 172.21.15.80:0/1821490898 learned_addr learned my addr 172.21.15.80:0/1821490898 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:28:40.638 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:38.613+0000 7f90ed524700 1 --2- 172.21.15.80:0/1821490898 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90e8101510 0x7f90e8199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:40.638 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:38.614+0000 7f90ecd23700 1 --2- 172.21.15.80:0/1821490898 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90e8102df0 0x7f90e8199ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:40.638 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:38.614+0000 7f90edd25700 1 -- 172.21.15.80:0/1821490898 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90e8102df0 msgr2=0x7f90e8199ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:40.639 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:38.614+0000 7f90edd25700 1 --2- 172.21.15.80:0/1821490898 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90e8102df0 0x7f90e8199ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:40.639 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:38.614+0000 7f90edd25700 1 -- 172.21.15.80:0/1821490898 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90e8101510 msgr2=0x7f90e8199980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:40.639 INFO:teuthology.orchestra.run.smithi080.stdout:253403070510 2024-04-24T23:28:40.639 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:38.614+0000 7f90edd25700 1 --2- 172.21.15.80:0/1821490898 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90e8101510 0x7f90e8199980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:40.639 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:38.614+0000 7f90edd25700 1 -- 172.21.15.80:0/1821490898 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90e81a4f40 con 0x7f90e81037b0 2024-04-24T23:28:40.639 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:38.614+0000 7f90edd25700 1 --2- 172.21.15.80:0/1821490898 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f90e81037b0 0x7f90e81a4a00 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f90e000b970 tx=0x7f90e000bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:40.639 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:38.614+0000 7f90de7fc700 1 -- 172.21.15.80:0/1821490898 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e0019070 con 0x7f90e81037b0 2024-04-24T23:28:40.640 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:38.614+0000 7f90de7fc700 1 -- 172.21.15.80:0/1821490898 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90e000ce10 con 0x7f90e81037b0 2024-04-24T23:28:40.640 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:38.614+0000 7f90ef788700 1 -- 172.21.15.80:0/1821490898 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90e81a5170 con 0x7f90e81037b0 2024-04-24T23:28:40.640 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:38.614+0000 7f90de7fc700 1 -- 172.21.15.80:0/1821490898 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e0007370 con 0x7f90e81037b0 2024-04-24T23:28:40.640 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:38.614+0000 7f90ef788700 1 -- 172.21.15.80:0/1821490898 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90e81a5630 con 0x7f90e81037b0 2024-04-24T23:28:40.640 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:38.615+0000 7f90ef788700 1 -- 172.21.15.80:0/1821490898 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90e8068f20 con 0x7f90e81037b0 2024-04-24T23:28:40.640 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:38.616+0000 7f90de7fc700 1 -- 172.21.15.80:0/1821490898 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f90e001c070 con 0x7f90e81037b0 2024-04-24T23:28:40.640 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:38.617+0000 7f90de7fc700 1 --2- 172.21.15.80:0/1821490898 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f90d40a2cb0 0x7f90d40a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:40.640 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:38.617+0000 7f90de7fc700 1 -- 172.21.15.80:0/1821490898 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f90e00cec30 con 0x7f90e81037b0 2024-04-24T23:28:40.640 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:38.617+0000 7f90ed524700 1 --2- 172.21.15.80:0/1821490898 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f90d40a2cb0 0x7f90d40a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:40.640 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:38.617+0000 7f90ed524700 1 --2- 172.21.15.80:0/1821490898 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f90d40a2cb0 0x7f90d40a5170 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f90e4007be0 tx=0x7f90e40073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:40.640 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:38.621+0000 7f90de7fc700 1 -- 172.21.15.80:0/1821490898 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f90e009b7e0 con 0x7f90e81037b0 2024-04-24T23:28:40.640 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:38.837+0000 7f90ef788700 1 -- 172.21.15.80:0/1821490898 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f90e802db30 con 0x7f90e81037b0 2024-04-24T23:28:40.640 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:38.838+0000 7f90de7fc700 1 -- 172.21.15.80:0/1821490898 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f90e009b0e0 con 0x7f90e81037b0 2024-04-24T23:28:40.640 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:38.840+0000 7f90ef788700 1 -- 172.21.15.80:0/1821490898 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f90d40a2cb0 msgr2=0x7f90d40a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:40.641 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:38.840+0000 7f90ef788700 1 --2- 172.21.15.80:0/1821490898 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f90d40a2cb0 0x7f90d40a5170 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f90e4007be0 tx=0x7f90e40073d0 comp rx=0 tx=0).stop 2024-04-24T23:28:40.641 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:38.840+0000 7f90ef788700 1 -- 172.21.15.80:0/1821490898 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f90e81037b0 msgr2=0x7f90e81a4a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:40.641 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:38.840+0000 7f90ef788700 1 --2- 172.21.15.80:0/1821490898 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f90e81037b0 0x7f90e81a4a00 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f90e000b970 tx=0x7f90e000bc80 comp rx=0 tx=0).stop 2024-04-24T23:28:40.641 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:38.840+0000 7f90ef788700 1 -- 172.21.15.80:0/1821490898 shutdown_connections 2024-04-24T23:28:40.641 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:38.840+0000 7f90ef788700 1 --2- 172.21.15.80:0/1821490898 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f90d40a2cb0 0x7f90d40a5170 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:40.641 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:38.840+0000 7f90ef788700 1 --2- 172.21.15.80:0/1821490898 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f90e81037b0 0x7f90e81a4a00 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:40.641 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:38.840+0000 7f90ef788700 1 --2- 172.21.15.80:0/1821490898 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90e8101510 0x7f90e8199980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:40.641 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:38.840+0000 7f90ef788700 1 --2- 172.21.15.80:0/1821490898 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90e8102df0 0x7f90e8199ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:40.641 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:38.840+0000 7f90ef788700 1 -- 172.21.15.80:0/1821490898 >> 172.21.15.80:0/1821490898 conn(0x7f90e80771d0 msgr2=0x7f90e80ff1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:40.641 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:38.840+0000 7f90ef788700 1 -- 172.21.15.80:0/1821490898 shutdown_connections 2024-04-24T23:28:40.641 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:38.840+0000 7f90ef788700 1 -- 172.21.15.80:0/1821490898 wait complete. 2024-04-24T23:28:41.243 INFO:tasks.cephadm.ceph_manager.ceph:need seq 253403070508 got 253403070510 for osd.8 2024-04-24T23:28:41.244 DEBUG:teuthology.parallel:result is None 2024-04-24T23:28:41.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:28:41 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/1605829689' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-24T23:28:41.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:28:41 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/1605829689' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-24T23:28:41.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:28:41 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/1605829689' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-24T23:28:42.342 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:28:42 smithi080 ceph-mon[29152]: pgmap v339: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-24T23:28:42.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:28:42 smithi134 ceph-mon[32619]: pgmap v339: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-24T23:28:42.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:28:42 smithi148 ceph-mon[32528]: pgmap v339: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-24T23:28:43.134 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411401 got 77309411405 for osd.2 2024-04-24T23:28:43.134 DEBUG:teuthology.parallel:result is None 2024-04-24T23:28:43.792 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757304 got 163208757309 for osd.5 2024-04-24T23:28:43.792 DEBUG:teuthology.parallel:result is None 2024-04-24T23:28:44.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:28:44 smithi134 ceph-mon[32619]: pgmap v340: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:28:44.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:28:44 smithi148 ceph-mon[32528]: pgmap v340: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:28:44.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:28:44 smithi080 ceph-mon[29152]: pgmap v340: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:28:45.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:28:45 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/1467417812' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-24T23:28:45.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:28:45 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/1467417812' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-24T23:28:45.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:28:45 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/1467417812' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-24T23:28:45.606 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:43.985+0000 7fb7b2d1f700 1 -- 172.21.15.80:0/3377749460 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb7ac101af0 msgr2=0x7fb7ac10dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:45.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:43.985+0000 7fb7b2d1f700 1 --2- 172.21.15.80:0/3377749460 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb7ac101af0 0x7fb7ac10dfe0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fb7a400b0a0 tx=0x7fb7a400b3b0 comp rx=0 tx=0).stop 2024-04-24T23:28:45.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:43.985+0000 7fb7b2d1f700 1 -- 172.21.15.80:0/3377749460 shutdown_connections 2024-04-24T23:28:45.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:43.985+0000 7fb7b2d1f700 1 --2- 172.21.15.80:0/3377749460 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb7ac101af0 0x7fb7ac10dfe0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:45.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:43.985+0000 7fb7b2d1f700 1 --2- 172.21.15.80:0/3377749460 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7ac1071a0 0x7fb7ac107580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:45.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:43.985+0000 7fb7b2d1f700 1 --2- 172.21.15.80:0/3377749460 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb7ac101130 0x7fb7ac1015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:45.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:43.985+0000 7fb7b2d1f700 1 -- 172.21.15.80:0/3377749460 >> 172.21.15.80:0/3377749460 conn(0x7fb7ac0780e0 msgr2=0x7fb7ac0784f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:45.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:43.985+0000 7fb7b2d1f700 1 -- 172.21.15.80:0/3377749460 shutdown_connections 2024-04-24T23:28:45.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:43.985+0000 7fb7b2d1f700 1 -- 172.21.15.80:0/3377749460 wait complete. 2024-04-24T23:28:45.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:43.986+0000 7fb7b2d1f700 1 Processor -- start 2024-04-24T23:28:45.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:43.986+0000 7fb7b2d1f700 1 -- start start 2024-04-24T23:28:45.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:43.986+0000 7fb7b2d1f700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7ac101130 0x7fb7ac19e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:45.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:43.986+0000 7fb7b2d1f700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb7ac101af0 0x7fb7ac19eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:45.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:43.986+0000 7fb7b2d1f700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb7ac1071a0 0x7fb7ac198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:45.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:43.986+0000 7fb7b2d1f700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fb7ac111410 con 0x7fb7ac1071a0 2024-04-24T23:28:45.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:43.986+0000 7fb7b2d1f700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb7ac111290 con 0x7fb7ac101af0 2024-04-24T23:28:45.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:43.986+0000 7fb7b2d1f700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb7ac111590 con 0x7fb7ac101130 2024-04-24T23:28:45.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:43.986+0000 7fb7b12bc700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb7ac1071a0 0x7fb7ac198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:45.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:43.986+0000 7fb7b12bc700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb7ac1071a0 0x7fb7ac198780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:44700/0 (socket says 172.21.15.80:44700) 2024-04-24T23:28:45.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:43.986+0000 7fb7b12bc700 1 -- 172.21.15.80:0/1467417812 learned_addr learned my addr 172.21.15.80:0/1467417812 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:28:45.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:43.986+0000 7fb7b0abb700 1 --2- 172.21.15.80:0/1467417812 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7ac101130 0x7fb7ac19e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:45.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:43.986+0000 7fb7abfff700 1 --2- 172.21.15.80:0/1467417812 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb7ac101af0 0x7fb7ac19eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:45.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:43.986+0000 7fb7b12bc700 1 -- 172.21.15.80:0/1467417812 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7ac101130 msgr2=0x7fb7ac19e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:45.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:43.986+0000 7fb7b12bc700 1 --2- 172.21.15.80:0/1467417812 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7ac101130 0x7fb7ac19e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:45.609 INFO:teuthology.orchestra.run.smithi080.stdout:141733920835 2024-04-24T23:28:45.609 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:43.986+0000 7fb7b12bc700 1 -- 172.21.15.80:0/1467417812 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb7ac101af0 msgr2=0x7fb7ac19eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:45.609 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:43.986+0000 7fb7b12bc700 1 --2- 172.21.15.80:0/1467417812 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb7ac101af0 0x7fb7ac19eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:45.609 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:43.986+0000 7fb7b12bc700 1 -- 172.21.15.80:0/1467417812 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7ac198f60 con 0x7fb7ac1071a0 2024-04-24T23:28:45.609 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:43.987+0000 7fb7b12bc700 1 --2- 172.21.15.80:0/1467417812 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb7ac1071a0 0x7fb7ac198780 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fb7a40062a0 tx=0x7fb7a4004d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:45.609 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:43.987+0000 7fb7a9ffb700 1 -- 172.21.15.80:0/1467417812 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7a4021070 con 0x7fb7ac1071a0 2024-04-24T23:28:45.610 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:43.987+0000 7fb7b2d1f700 1 -- 172.21.15.80:0/1467417812 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7ac1991f0 con 0x7fb7ac1071a0 2024-04-24T23:28:45.610 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:43.987+0000 7fb7a9ffb700 1 -- 172.21.15.80:0/1467417812 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7a4007c00 con 0x7fb7ac1071a0 2024-04-24T23:28:45.610 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:43.987+0000 7fb7a9ffb700 1 -- 172.21.15.80:0/1467417812 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7a401d450 con 0x7fb7ac1071a0 2024-04-24T23:28:45.610 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:43.987+0000 7fb7b2d1f700 1 -- 172.21.15.80:0/1467417812 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7ac1a5460 con 0x7fb7ac1071a0 2024-04-24T23:28:45.610 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:43.988+0000 7fb7b2d1f700 1 -- 172.21.15.80:0/1467417812 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7ac102850 con 0x7fb7ac1071a0 2024-04-24T23:28:45.610 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:43.989+0000 7fb7a9ffb700 1 -- 172.21.15.80:0/1467417812 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fb7a4004000 con 0x7fb7ac1071a0 2024-04-24T23:28:45.610 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:43.989+0000 7fb7a9ffb700 1 --2- 172.21.15.80:0/1467417812 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fb79c0a2cb0 0x7fb79c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:45.610 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:43.990+0000 7fb7b0abb700 1 --2- 172.21.15.80:0/1467417812 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fb79c0a2cb0 0x7fb79c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:45.610 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:43.990+0000 7fb7a9ffb700 1 -- 172.21.15.80:0/1467417812 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fb7a400f020 con 0x7fb7ac1071a0 2024-04-24T23:28:45.610 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:43.990+0000 7fb7b0abb700 1 --2- 172.21.15.80:0/1467417812 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fb79c0a2cb0 0x7fb79c0a5170 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb7a0005950 tx=0x7fb7a000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:45.610 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:43.993+0000 7fb7a9ffb700 1 -- 172.21.15.80:0/1467417812 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fb7a402a020 con 0x7fb7ac1071a0 2024-04-24T23:28:45.610 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:44.196+0000 7fb7b2d1f700 1 -- 172.21.15.80:0/1467417812 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fb7ac1a56f0 con 0x7fb7ac1071a0 2024-04-24T23:28:45.610 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:44.196+0000 7fb7a9ffb700 1 -- 172.21.15.80:0/1467417812 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fb7a409d5a0 con 0x7fb7ac1071a0 2024-04-24T23:28:45.611 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:44.199+0000 7fb7b2d1f700 1 -- 172.21.15.80:0/1467417812 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fb79c0a2cb0 msgr2=0x7fb79c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:45.611 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:44.199+0000 7fb7b2d1f700 1 --2- 172.21.15.80:0/1467417812 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fb79c0a2cb0 0x7fb79c0a5170 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb7a0005950 tx=0x7fb7a000b410 comp rx=0 tx=0).stop 2024-04-24T23:28:45.611 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:44.199+0000 7fb7b2d1f700 1 -- 172.21.15.80:0/1467417812 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb7ac1071a0 msgr2=0x7fb7ac198780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:45.611 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:44.199+0000 7fb7b2d1f700 1 --2- 172.21.15.80:0/1467417812 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb7ac1071a0 0x7fb7ac198780 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fb7a40062a0 tx=0x7fb7a4004d70 comp rx=0 tx=0).stop 2024-04-24T23:28:45.611 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:44.199+0000 7fb7b2d1f700 1 -- 172.21.15.80:0/1467417812 shutdown_connections 2024-04-24T23:28:45.611 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:44.199+0000 7fb7b2d1f700 1 --2- 172.21.15.80:0/1467417812 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fb79c0a2cb0 0x7fb79c0a5170 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:45.611 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:44.199+0000 7fb7b2d1f700 1 --2- 172.21.15.80:0/1467417812 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb7ac1071a0 0x7fb7ac198780 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:45.611 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:44.199+0000 7fb7b2d1f700 1 --2- 172.21.15.80:0/1467417812 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7ac101130 0x7fb7ac19e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:45.611 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:44.199+0000 7fb7b2d1f700 1 --2- 172.21.15.80:0/1467417812 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb7ac101af0 0x7fb7ac19eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:45.611 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:44.199+0000 7fb7b2d1f700 1 -- 172.21.15.80:0/1467417812 >> 172.21.15.80:0/1467417812 conn(0x7fb7ac0780e0 msgr2=0x7fb7ac100360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:45.611 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:44.199+0000 7fb7b2d1f700 1 -- 172.21.15.80:0/1467417812 shutdown_connections 2024-04-24T23:28:45.611 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:44.199+0000 7fb7b2d1f700 1 -- 172.21.15.80:0/1467417812 wait complete. 2024-04-24T23:28:46.174 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:46.166+0000 7f70ab746700 1 -- 172.21.15.80:0/2486271915 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f70a4101af0 msgr2=0x7f70a410dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:46.174 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:46.166+0000 7f70ab746700 1 --2- 172.21.15.80:0/2486271915 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f70a4101af0 0x7f70a410dfe0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f709c00b0a0 tx=0x7f709c00b3b0 comp rx=0 tx=0).stop 2024-04-24T23:28:46.175 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:46.166+0000 7f70ab746700 1 -- 172.21.15.80:0/2486271915 shutdown_connections 2024-04-24T23:28:46.175 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:46.166+0000 7f70ab746700 1 --2- 172.21.15.80:0/2486271915 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f70a4101af0 0x7f70a410dfe0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:46.175 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:46.166+0000 7f70ab746700 1 --2- 172.21.15.80:0/2486271915 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f70a41071a0 0x7f70a4107580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:46.175 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:46.166+0000 7f70ab746700 1 --2- 172.21.15.80:0/2486271915 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f70a4101130 0x7f70a41015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:46.175 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:46.166+0000 7f70ab746700 1 -- 172.21.15.80:0/2486271915 >> 172.21.15.80:0/2486271915 conn(0x7f70a40780e0 msgr2=0x7f70a40784f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:46.175 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:46.166+0000 7f70ab746700 1 -- 172.21.15.80:0/2486271915 shutdown_connections 2024-04-24T23:28:46.176 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:46.166+0000 7f70ab746700 1 -- 172.21.15.80:0/2486271915 wait complete. 2024-04-24T23:28:46.176 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:46.167+0000 7f70ab746700 1 Processor -- start 2024-04-24T23:28:46.176 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:46.167+0000 7f70ab746700 1 -- start start 2024-04-24T23:28:46.176 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:46.167+0000 7f70ab746700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f70a4101130 0x7f70a4197600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:46.176 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:46.167+0000 7f70ab746700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f70a4101af0 0x7f70a4197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:46.176 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:46.167+0000 7f70ab746700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f70a41071a0 0x7f70a419c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:46.176 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:46.167+0000 7f70ab746700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f70a4111430 con 0x7f70a41071a0 2024-04-24T23:28:46.176 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:46.167+0000 7f70ab746700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f70a41112b0 con 0x7f70a4101af0 2024-04-24T23:28:46.176 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:46.167+0000 7f70ab746700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f70a41115b0 con 0x7f70a4101130 2024-04-24T23:28:46.176 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:46.167+0000 7f70a9ce3700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f70a41071a0 0x7f70a419c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:46.176 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:46.167+0000 7f70a9ce3700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f70a41071a0 0x7f70a419c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:44712/0 (socket says 172.21.15.80:44712) 2024-04-24T23:28:46.177 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:46.167+0000 7f70a9ce3700 1 -- 172.21.15.80:0/1856296330 learned_addr learned my addr 172.21.15.80:0/1856296330 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:28:46.177 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:46.167+0000 7f70a94e2700 1 --2- 172.21.15.80:0/1856296330 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f70a4101130 0x7f70a4197600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:46.177 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:46.167+0000 7f70a8ce1700 1 --2- 172.21.15.80:0/1856296330 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f70a4101af0 0x7f70a4197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:46.177 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:46.167+0000 7f70a9ce3700 1 -- 172.21.15.80:0/1856296330 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f70a4101130 msgr2=0x7f70a4197600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:46.177 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:46.167+0000 7f70a9ce3700 1 --2- 172.21.15.80:0/1856296330 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f70a4101130 0x7f70a4197600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:46.177 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:46.167+0000 7f70a9ce3700 1 -- 172.21.15.80:0/1856296330 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f70a4101af0 msgr2=0x7f70a4197b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:46.177 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:46.167+0000 7f70a9ce3700 1 --2- 172.21.15.80:0/1856296330 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f70a4101af0 0x7f70a4197b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:46.177 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:46.167+0000 7f70a9ce3700 1 -- 172.21.15.80:0/1856296330 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70a419cb90 con 0x7f70a41071a0 2024-04-24T23:28:46.177 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:46.167+0000 7f70a8ce1700 1 --2- 172.21.15.80:0/1856296330 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f70a4101af0 0x7f70a4197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:28:46.178 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:46.168+0000 7f70a9ce3700 1 --2- 172.21.15.80:0/1856296330 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f70a41071a0 0x7f70a419c650 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f709c0062a0 tx=0x7f709c004d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:46.178 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:46.168+0000 7f709a7fc700 1 -- 172.21.15.80:0/1856296330 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f709c021070 con 0x7f70a41071a0 2024-04-24T23:28:46.178 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:46.168+0000 7f709a7fc700 1 -- 172.21.15.80:0/1856296330 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f709c007c00 con 0x7f70a41071a0 2024-04-24T23:28:46.178 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:46.168+0000 7f70ab746700 1 -- 172.21.15.80:0/1856296330 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70a419ce20 con 0x7f70a41071a0 2024-04-24T23:28:46.178 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:46.168+0000 7f709a7fc700 1 -- 172.21.15.80:0/1856296330 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f709c01d450 con 0x7f70a41071a0 2024-04-24T23:28:46.178 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:46.168+0000 7f70ab746700 1 -- 172.21.15.80:0/1856296330 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70a419d360 con 0x7f70a41071a0 2024-04-24T23:28:46.178 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:46.169+0000 7f70ab746700 1 -- 172.21.15.80:0/1856296330 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70a4102850 con 0x7f70a41071a0 2024-04-24T23:28:46.178 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:46.170+0000 7f709a7fc700 1 -- 172.21.15.80:0/1856296330 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f709c004000 con 0x7f70a41071a0 2024-04-24T23:28:46.178 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:46.171+0000 7f709a7fc700 1 --2- 172.21.15.80:0/1856296330 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f70900a2cb0 0x7f70900a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:46.178 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:46.171+0000 7f70a94e2700 1 --2- 172.21.15.80:0/1856296330 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f70900a2cb0 0x7f70900a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:46.178 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:46.171+0000 7f709a7fc700 1 -- 172.21.15.80:0/1856296330 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f709c00f020 con 0x7f70a41071a0 2024-04-24T23:28:46.179 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:46.172+0000 7f70a94e2700 1 --2- 172.21.15.80:0/1856296330 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f70900a2cb0 0x7f70900a5170 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f70a00045b0 tx=0x7f70a000b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:46.179 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:46.175+0000 7f709a7fc700 1 -- 172.21.15.80:0/1856296330 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f709c02a020 con 0x7f70a41071a0 2024-04-24T23:28:46.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:28:46 smithi080 ceph-mon[29152]: pgmap v341: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T23:28:46.377 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:46.375+0000 7f70ab746700 1 -- 172.21.15.80:0/1856296330 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f70a402db30 con 0x7f70a41071a0 2024-04-24T23:28:46.377 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:46.375+0000 7f709a7fc700 1 -- 172.21.15.80:0/1856296330 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f709c09d590 con 0x7f70a41071a0 2024-04-24T23:28:46.377 INFO:teuthology.orchestra.run.smithi080.stdout:309237645348 2024-04-24T23:28:46.379 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:46.378+0000 7f70ab746700 1 -- 172.21.15.80:0/1856296330 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f70900a2cb0 msgr2=0x7f70900a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:46.379 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:46.378+0000 7f70ab746700 1 --2- 172.21.15.80:0/1856296330 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f70900a2cb0 0x7f70900a5170 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f70a00045b0 tx=0x7f70a000b3c0 comp rx=0 tx=0).stop 2024-04-24T23:28:46.379 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:46.378+0000 7f70ab746700 1 -- 172.21.15.80:0/1856296330 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f70a41071a0 msgr2=0x7f70a419c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:46.379 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:46.378+0000 7f70ab746700 1 --2- 172.21.15.80:0/1856296330 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f70a41071a0 0x7f70a419c650 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f709c0062a0 tx=0x7f709c004d70 comp rx=0 tx=0).stop 2024-04-24T23:28:46.379 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:46.378+0000 7f70ab746700 1 -- 172.21.15.80:0/1856296330 shutdown_connections 2024-04-24T23:28:46.379 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:46.378+0000 7f70ab746700 1 --2- 172.21.15.80:0/1856296330 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f70900a2cb0 0x7f70900a5170 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:46.379 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:46.378+0000 7f70ab746700 1 --2- 172.21.15.80:0/1856296330 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f70a41071a0 0x7f70a419c650 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:46.379 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:46.378+0000 7f70ab746700 1 --2- 172.21.15.80:0/1856296330 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f70a4101130 0x7f70a4197600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:46.379 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:46.378+0000 7f70ab746700 1 --2- 172.21.15.80:0/1856296330 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f70a4101af0 0x7f70a4197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:46.380 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:46.378+0000 7f70ab746700 1 -- 172.21.15.80:0/1856296330 >> 172.21.15.80:0/1856296330 conn(0x7f70a40780e0 msgr2=0x7f70a40ffbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:46.380 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:46.378+0000 7f70ab746700 1 -- 172.21.15.80:0/1856296330 shutdown_connections 2024-04-24T23:28:46.380 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:46.378+0000 7f70ab746700 1 -- 172.21.15.80:0/1856296330 wait complete. 2024-04-24T23:28:46.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:28:46 smithi134 ceph-mon[32619]: pgmap v341: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T23:28:46.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:28:46 smithi148 ceph-mon[32528]: pgmap v341: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T23:28:46.608 INFO:teuthology.orchestra.run.smithi080.stderr:Inferring config /var/lib/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:28:46.634 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920831 got 141733920835 for osd.4 2024-04-24T23:28:46.634 DEBUG:teuthology.parallel:result is None 2024-04-24T23:28:47.224 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:47.223+0000 7f27e2718700 1 -- 172.21.15.80:0/4273617806 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f27dc1064c0 msgr2=0x7f27dc10e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:47.224 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:47.223+0000 7f27e2718700 1 --2- 172.21.15.80:0/4273617806 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f27dc1064c0 0x7f27dc10e060 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f27d400b0d0 tx=0x7f27d400b3e0 comp rx=0 tx=0).stop 2024-04-24T23:28:47.224 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:47.223+0000 7f27e2718700 1 -- 172.21.15.80:0/4273617806 shutdown_connections 2024-04-24T23:28:47.224 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:47.223+0000 7f27e2718700 1 --2- 172.21.15.80:0/4273617806 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f27dc1064c0 0x7f27dc10e060 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:47.224 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:47.223+0000 7f27e2718700 1 --2- 172.21.15.80:0/4273617806 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f27dc101770 0x7f27dc101b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:47.224 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:47.223+0000 7f27e2718700 1 --2- 172.21.15.80:0/4273617806 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27dc102120 0x7f27dc105d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:47.224 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:47.223+0000 7f27e2718700 1 -- 172.21.15.80:0/4273617806 >> 172.21.15.80:0/4273617806 conn(0x7f27dc0780e0 msgr2=0x7f27dc0784f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:47.224 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:47.223+0000 7f27e2718700 1 -- 172.21.15.80:0/4273617806 shutdown_connections 2024-04-24T23:28:47.225 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:47.223+0000 7f27e2718700 1 -- 172.21.15.80:0/4273617806 wait complete. 2024-04-24T23:28:47.225 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:47.223+0000 7f27e2718700 1 Processor -- start 2024-04-24T23:28:47.225 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:47.223+0000 7f27e2718700 1 -- start start 2024-04-24T23:28:47.227 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:47.224+0000 7f27e2718700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f27dc101770 0x7f27dc19e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:47.227 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:47.224+0000 7f27e2718700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27dc102120 0x7f27dc19ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:47.227 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:47.224+0000 7f27e2718700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f27dc1064c0 0x7f27dc198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:47.227 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:47.224+0000 7f27e2718700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f27dc111500 con 0x7f27dc101770 2024-04-24T23:28:47.227 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:47.224+0000 7f27e2718700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f27dc111380 con 0x7f27dc1064c0 2024-04-24T23:28:47.227 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:47.224+0000 7f27e2718700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f27dc111680 con 0x7f27dc102120 2024-04-24T23:28:47.227 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:47.224+0000 7f27dbfff700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f27dc101770 0x7f27dc19e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:47.227 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:47.224+0000 7f27dbfff700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f27dc101770 0x7f27dc19e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:44734/0 (socket says 172.21.15.80:44734) 2024-04-24T23:28:47.227 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:47.224+0000 7f27dbfff700 1 -- 172.21.15.80:0/2472560814 learned_addr learned my addr 172.21.15.80:0/2472560814 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:28:47.227 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:47.224+0000 7f27db7fe700 1 --2- 172.21.15.80:0/2472560814 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27dc102120 0x7f27dc19ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:47.227 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:47.224+0000 7f27e0cb5700 1 --2- 172.21.15.80:0/2472560814 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f27dc1064c0 0x7f27dc198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:47.227 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:47.224+0000 7f27dbfff700 1 -- 172.21.15.80:0/2472560814 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27dc102120 msgr2=0x7f27dc19ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:47.228 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:47.224+0000 7f27dbfff700 1 --2- 172.21.15.80:0/2472560814 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27dc102120 0x7f27dc19ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:47.228 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:47.224+0000 7f27dbfff700 1 -- 172.21.15.80:0/2472560814 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f27dc1064c0 msgr2=0x7f27dc198850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:47.228 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:47.224+0000 7f27dbfff700 1 --2- 172.21.15.80:0/2472560814 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f27dc1064c0 0x7f27dc198850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:47.228 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:47.224+0000 7f27dbfff700 1 -- 172.21.15.80:0/2472560814 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27dc199030 con 0x7f27dc101770 2024-04-24T23:28:47.228 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:47.224+0000 7f27dbfff700 1 --2- 172.21.15.80:0/2472560814 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f27dc101770 0x7f27dc19e6d0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f27cc00b530 tx=0x7f27cc00b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:47.228 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:47.224+0000 7f27d97fa700 1 -- 172.21.15.80:0/2472560814 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27cc015020 con 0x7f27dc101770 2024-04-24T23:28:47.228 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:47.225+0000 7f27e2718700 1 -- 172.21.15.80:0/2472560814 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27dc199320 con 0x7f27dc101770 2024-04-24T23:28:47.228 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:47.225+0000 7f27d97fa700 1 -- 172.21.15.80:0/2472560814 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27cc011d10 con 0x7f27dc101770 2024-04-24T23:28:47.228 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:47.225+0000 7f27e2718700 1 -- 172.21.15.80:0/2472560814 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27dc1a5530 con 0x7f27dc101770 2024-04-24T23:28:47.228 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:47.225+0000 7f27d97fa700 1 -- 172.21.15.80:0/2472560814 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27cc010e50 con 0x7f27dc101770 2024-04-24T23:28:47.229 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:47.225+0000 7f27e2718700 1 -- 172.21.15.80:0/2472560814 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27dc06a1b0 con 0x7f27dc101770 2024-04-24T23:28:47.229 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:47.227+0000 7f27d97fa700 1 -- 172.21.15.80:0/2472560814 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f27cc018020 con 0x7f27dc101770 2024-04-24T23:28:47.230 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:47.227+0000 7f27d97fa700 1 --2- 172.21.15.80:0/2472560814 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f27c40a2e50 0x7f27c40a5310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:47.230 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:47.228+0000 7f27db7fe700 1 --2- 172.21.15.80:0/2472560814 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f27c40a2e50 0x7f27c40a5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:47.230 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:47.228+0000 7f27d97fa700 1 -- 172.21.15.80:0/2472560814 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f27cc0cf1f0 con 0x7f27dc101770 2024-04-24T23:28:47.230 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:47.228+0000 7f27db7fe700 1 --2- 172.21.15.80:0/2472560814 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f27c40a2e50 0x7f27c40a5310 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f27d0005fd0 tx=0x7f27d000a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:47.232 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:47.231+0000 7f27d97fa700 1 -- 172.21.15.80:0/2472560814 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f27cc01f020 con 0x7f27dc101770 2024-04-24T23:28:47.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:28:47 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/1856296330' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-24T23:28:47.433 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:47.432+0000 7f27e2718700 1 -- 172.21.15.80:0/2472560814 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f27dc199d80 con 0x7f27dc101770 2024-04-24T23:28:47.433 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:47.433+0000 7f27d97fa700 1 -- 172.21.15.80:0/2472560814 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f27cc09bda0 con 0x7f27dc101770 2024-04-24T23:28:47.434 INFO:teuthology.orchestra.run.smithi080.stdout:55834574931 2024-04-24T23:28:47.436 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:47.435+0000 7f27e2718700 1 -- 172.21.15.80:0/2472560814 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f27c40a2e50 msgr2=0x7f27c40a5310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:47.436 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:47.435+0000 7f27e2718700 1 --2- 172.21.15.80:0/2472560814 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f27c40a2e50 0x7f27c40a5310 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f27d0005fd0 tx=0x7f27d000a560 comp rx=0 tx=0).stop 2024-04-24T23:28:47.436 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:47.435+0000 7f27e2718700 1 -- 172.21.15.80:0/2472560814 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f27dc101770 msgr2=0x7f27dc19e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:47.436 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:47.435+0000 7f27e2718700 1 --2- 172.21.15.80:0/2472560814 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f27dc101770 0x7f27dc19e6d0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f27cc00b530 tx=0x7f27cc00b8f0 comp rx=0 tx=0).stop 2024-04-24T23:28:47.436 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:47.435+0000 7f27e2718700 1 -- 172.21.15.80:0/2472560814 shutdown_connections 2024-04-24T23:28:47.436 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:47.435+0000 7f27e2718700 1 --2- 172.21.15.80:0/2472560814 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f27dc1064c0 0x7f27dc198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:47.436 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:47.435+0000 7f27e2718700 1 --2- 172.21.15.80:0/2472560814 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f27c40a2e50 0x7f27c40a5310 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:47.436 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:47.435+0000 7f27e2718700 1 --2- 172.21.15.80:0/2472560814 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f27dc101770 0x7f27dc19e6d0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:47.436 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:47.435+0000 7f27e2718700 1 --2- 172.21.15.80:0/2472560814 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27dc102120 0x7f27dc19ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:47.436 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:47.435+0000 7f27e2718700 1 -- 172.21.15.80:0/2472560814 >> 172.21.15.80:0/2472560814 conn(0x7f27dc0780e0 msgr2=0x7f27dc06ab80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:47.436 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:47.435+0000 7f27e2718700 1 -- 172.21.15.80:0/2472560814 shutdown_connections 2024-04-24T23:28:47.436 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:47.435+0000 7f27e2718700 1 -- 172.21.15.80:0/2472560814 wait complete. 2024-04-24T23:28:47.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:28:47 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/1856296330' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-24T23:28:47.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:28:47 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/1856296330' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-24T23:28:48.203 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:28:48 smithi080 ceph-mon[29152]: pgmap v342: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:28:48.203 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:28:48 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/2472560814' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-24T23:28:48.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:28:48 smithi134 ceph-mon[32619]: pgmap v342: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:28:48.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:28:48 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/2472560814' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-24T23:28:48.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:28:48 smithi148 ceph-mon[32528]: pgmap v342: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:28:48.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:28:48 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/2472560814' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-24T23:28:48.761 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:48.744+0000 7eff75640700 1 -- 172.21.15.80:0/365134719 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7eff70102f00 msgr2=0x7eff70106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:48.762 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:48.744+0000 7eff75640700 1 --2- 172.21.15.80:0/365134719 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7eff70102f00 0x7eff70106e60 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7eff640099c0 tx=0x7eff64009cd0 comp rx=0 tx=0).stop 2024-04-24T23:28:48.762 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:48.744+0000 7eff75640700 1 -- 172.21.15.80:0/365134719 shutdown_connections 2024-04-24T23:28:48.762 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:48.744+0000 7eff75640700 1 --2- 172.21.15.80:0/365134719 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff70107540 0x7eff7010f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:48.762 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:48.744+0000 7eff75640700 1 --2- 172.21.15.80:0/365134719 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff70102550 0x7eff70102930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:48.762 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:48.744+0000 7eff75640700 1 --2- 172.21.15.80:0/365134719 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7eff70102f00 0x7eff70106e60 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:48.762 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:48.744+0000 7eff75640700 1 -- 172.21.15.80:0/365134719 >> 172.21.15.80:0/365134719 conn(0x7eff700793f0 msgr2=0x7eff70079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:48.762 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:48.744+0000 7eff75640700 1 -- 172.21.15.80:0/365134719 shutdown_connections 2024-04-24T23:28:48.762 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:48.744+0000 7eff75640700 1 -- 172.21.15.80:0/365134719 wait complete. 2024-04-24T23:28:48.762 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:48.745+0000 7eff75640700 1 Processor -- start 2024-04-24T23:28:48.762 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:48.745+0000 7eff75640700 1 -- start start 2024-04-24T23:28:48.762 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:48.745+0000 7eff75640700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7eff70102550 0x7eff70074530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:48.762 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:48.745+0000 7eff75640700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff70102f00 0x7eff70074a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:48.763 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:48.745+0000 7eff75640700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff70107540 0x7eff70074fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:48.763 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:48.745+0000 7eff75640700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7eff70112600 con 0x7eff70102550 2024-04-24T23:28:48.763 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:48.745+0000 7eff75640700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7eff70112480 con 0x7eff70102f00 2024-04-24T23:28:48.763 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:48.745+0000 7eff75640700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7eff70112780 con 0x7eff70107540 2024-04-24T23:28:48.763 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:48.745+0000 7eff6effd700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7eff70102550 0x7eff70074530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:48.763 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:48.745+0000 7eff6effd700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7eff70102550 0x7eff70074530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:44744/0 (socket says 172.21.15.80:44744) 2024-04-24T23:28:48.763 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:48.745+0000 7eff6effd700 1 -- 172.21.15.80:0/465061873 learned_addr learned my addr 172.21.15.80:0/465061873 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:28:48.763 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:48.746+0000 7eff6f7fe700 1 --2- 172.21.15.80:0/465061873 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff70107540 0x7eff70074fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:48.763 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:48.746+0000 7eff6e7fc700 1 --2- 172.21.15.80:0/465061873 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff70102f00 0x7eff70074a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:48.763 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:48.746+0000 7eff6effd700 1 -- 172.21.15.80:0/465061873 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff70107540 msgr2=0x7eff70074fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:48.763 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:48.746+0000 7eff6effd700 1 --2- 172.21.15.80:0/465061873 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff70107540 0x7eff70074fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:48.763 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:48.746+0000 7eff6effd700 1 -- 172.21.15.80:0/465061873 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff70102f00 msgr2=0x7eff70074a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:48.763 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:48.746+0000 7eff6effd700 1 --2- 172.21.15.80:0/465061873 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff70102f00 0x7eff70074a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:48.763 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:48.746+0000 7eff6effd700 1 -- 172.21.15.80:0/465061873 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff701a7120 con 0x7eff70102550 2024-04-24T23:28:48.764 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:48.746+0000 7eff6effd700 1 --2- 172.21.15.80:0/465061873 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7eff70102550 0x7eff70074530 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7eff6000b530 tx=0x7eff6000b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:48.764 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:48.746+0000 7eff57fff700 1 -- 172.21.15.80:0/465061873 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff60015020 con 0x7eff70102550 2024-04-24T23:28:48.764 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:48.746+0000 7eff57fff700 1 -- 172.21.15.80:0/465061873 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff60011d10 con 0x7eff70102550 2024-04-24T23:28:48.764 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:48.746+0000 7eff75640700 1 -- 172.21.15.80:0/465061873 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff701a7410 con 0x7eff70102550 2024-04-24T23:28:48.764 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:48.746+0000 7eff57fff700 1 -- 172.21.15.80:0/465061873 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff60010e50 con 0x7eff70102550 2024-04-24T23:28:48.764 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:48.746+0000 7eff75640700 1 -- 172.21.15.80:0/465061873 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff701a7920 con 0x7eff70102550 2024-04-24T23:28:48.764 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:48.747+0000 7eff75640700 1 -- 172.21.15.80:0/465061873 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff70103940 con 0x7eff70102550 2024-04-24T23:28:48.764 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:48.748+0000 7eff57fff700 1 -- 172.21.15.80:0/465061873 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7eff60018020 con 0x7eff70102550 2024-04-24T23:28:48.764 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:48.749+0000 7eff57fff700 1 --2- 172.21.15.80:0/465061873 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7eff580a2e50 0x7eff580a5310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:48.764 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:48.749+0000 7eff6e7fc700 1 --2- 172.21.15.80:0/465061873 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7eff580a2e50 0x7eff580a5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:48.764 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:48.749+0000 7eff57fff700 1 -- 172.21.15.80:0/465061873 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7eff600cec80 con 0x7eff70102550 2024-04-24T23:28:48.764 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:48.749+0000 7eff6e7fc700 1 --2- 172.21.15.80:0/465061873 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7eff580a2e50 0x7eff580a5310 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7eff64000c00 tx=0x7eff64005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:48.764 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:48.754+0000 7eff57fff700 1 -- 172.21.15.80:0/465061873 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7eff6001f020 con 0x7eff70102550 2024-04-24T23:28:48.957 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:48.956+0000 7eff75640700 1 -- 172.21.15.80:0/465061873 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7eff70075a60 con 0x7eff70102550 2024-04-24T23:28:48.957 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:48.956+0000 7eff57fff700 1 -- 172.21.15.80:0/465061873 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7eff6009b830 con 0x7eff70102550 2024-04-24T23:28:48.957 INFO:teuthology.orchestra.run.smithi080.stdout:287762808874 2024-04-24T23:28:48.960 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:48.959+0000 7eff75640700 1 -- 172.21.15.80:0/465061873 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7eff580a2e50 msgr2=0x7eff580a5310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:48.960 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:48.959+0000 7eff75640700 1 --2- 172.21.15.80:0/465061873 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7eff580a2e50 0x7eff580a5310 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7eff64000c00 tx=0x7eff64005fd0 comp rx=0 tx=0).stop 2024-04-24T23:28:48.960 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:48.959+0000 7eff75640700 1 -- 172.21.15.80:0/465061873 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7eff70102550 msgr2=0x7eff70074530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:48.960 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:48.959+0000 7eff75640700 1 --2- 172.21.15.80:0/465061873 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7eff70102550 0x7eff70074530 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7eff6000b530 tx=0x7eff6000b8f0 comp rx=0 tx=0).stop 2024-04-24T23:28:48.960 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:48.959+0000 7eff75640700 1 -- 172.21.15.80:0/465061873 shutdown_connections 2024-04-24T23:28:48.960 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:48.959+0000 7eff75640700 1 --2- 172.21.15.80:0/465061873 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff70107540 0x7eff70074fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:48.960 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:48.959+0000 7eff75640700 1 --2- 172.21.15.80:0/465061873 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7eff580a2e50 0x7eff580a5310 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:48.961 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:48.959+0000 7eff75640700 1 --2- 172.21.15.80:0/465061873 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7eff70102550 0x7eff70074530 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:48.961 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:48.959+0000 7eff75640700 1 --2- 172.21.15.80:0/465061873 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff70102f00 0x7eff70074a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:48.961 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:48.959+0000 7eff75640700 1 -- 172.21.15.80:0/465061873 >> 172.21.15.80:0/465061873 conn(0x7eff700793f0 msgr2=0x7eff701005d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:48.961 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:48.959+0000 7eff75640700 1 -- 172.21.15.80:0/465061873 shutdown_connections 2024-04-24T23:28:48.961 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:48.959+0000 7eff75640700 1 -- 172.21.15.80:0/465061873 wait complete. 2024-04-24T23:28:49.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:28:49 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/465061873' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-24T23:28:49.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:28:49 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/465061873' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-24T23:28:49.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:28:49 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/465061873' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-24T23:28:50.068 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:50.057+0000 7f1a3c6b3700 1 -- 172.21.15.80:0/2720608775 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1a341071a0 msgr2=0x7f1a34107580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:50.068 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:50.057+0000 7f1a3c6b3700 1 --2- 172.21.15.80:0/2720608775 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1a341071a0 0x7f1a34107580 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f1a30009990 tx=0x7f1a30009ca0 comp rx=0 tx=0).stop 2024-04-24T23:28:50.068 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:50.057+0000 7f1a3c6b3700 1 -- 172.21.15.80:0/2720608775 shutdown_connections 2024-04-24T23:28:50.068 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:50.057+0000 7f1a3c6b3700 1 --2- 172.21.15.80:0/2720608775 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1a34101af0 0x7f1a3410dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:50.068 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:50.057+0000 7f1a3c6b3700 1 --2- 172.21.15.80:0/2720608775 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1a341071a0 0x7f1a34107580 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:50.068 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:50.057+0000 7f1a3c6b3700 1 --2- 172.21.15.80:0/2720608775 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a34101130 0x7f1a341015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:50.068 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:50.057+0000 7f1a3c6b3700 1 -- 172.21.15.80:0/2720608775 >> 172.21.15.80:0/2720608775 conn(0x7f1a340780e0 msgr2=0x7f1a340784f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:50.068 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:50.058+0000 7f1a3c6b3700 1 -- 172.21.15.80:0/2720608775 shutdown_connections 2024-04-24T23:28:50.069 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:50.058+0000 7f1a3c6b3700 1 -- 172.21.15.80:0/2720608775 wait complete. 2024-04-24T23:28:50.069 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:50.058+0000 7f1a3c6b3700 1 Processor -- start 2024-04-24T23:28:50.069 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:50.058+0000 7f1a3c6b3700 1 -- start start 2024-04-24T23:28:50.069 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:50.058+0000 7f1a3c6b3700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1a34101130 0x7f1a341998c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:50.069 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:50.058+0000 7f1a3c6b3700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1a34101af0 0x7f1a34199e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:50.069 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:50.058+0000 7f1a3c6b3700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a341071a0 0x7f1a341a4940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:50.069 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:50.058+0000 7f1a3c6b3700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f1a34111440 con 0x7f1a34101af0 2024-04-24T23:28:50.069 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:50.059+0000 7f1a3c6b3700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1a341112c0 con 0x7f1a34101130 2024-04-24T23:28:50.069 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:50.059+0000 7f1a3c6b3700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1a341115c0 con 0x7f1a341071a0 2024-04-24T23:28:50.069 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:50.059+0000 7f1a39c4e700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1a34101af0 0x7f1a34199e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:50.069 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:50.059+0000 7f1a39c4e700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1a34101af0 0x7f1a34199e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:44788/0 (socket says 172.21.15.80:44788) 2024-04-24T23:28:50.069 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:50.059+0000 7f1a39c4e700 1 -- 172.21.15.80:0/873349850 learned_addr learned my addr 172.21.15.80:0/873349850 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:28:50.069 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:50.059+0000 7f1a3ac50700 1 --2- 172.21.15.80:0/873349850 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a341071a0 0x7f1a341a4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:50.069 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:50.059+0000 7f1a3a44f700 1 --2- 172.21.15.80:0/873349850 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1a34101130 0x7f1a341998c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:50.070 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:50.059+0000 7f1a39c4e700 1 -- 172.21.15.80:0/873349850 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a341071a0 msgr2=0x7f1a341a4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:50.070 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:50.059+0000 7f1a39c4e700 1 --2- 172.21.15.80:0/873349850 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a341071a0 0x7f1a341a4940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:50.070 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:50.059+0000 7f1a39c4e700 1 -- 172.21.15.80:0/873349850 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1a34101130 msgr2=0x7f1a341998c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:50.070 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:50.059+0000 7f1a39c4e700 1 --2- 172.21.15.80:0/873349850 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1a34101130 0x7f1a341998c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:50.070 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:50.059+0000 7f1a39c4e700 1 -- 172.21.15.80:0/873349850 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a341a4e80 con 0x7f1a34101af0 2024-04-24T23:28:50.070 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:50.059+0000 7f1a3a44f700 1 --2- 172.21.15.80:0/873349850 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1a34101130 0x7f1a341998c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:28:50.070 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:50.059+0000 7f1a3ac50700 1 --2- 172.21.15.80:0/873349850 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a341071a0 0x7f1a341a4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:28:50.070 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:50.059+0000 7f1a39c4e700 1 --2- 172.21.15.80:0/873349850 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1a34101af0 0x7f1a34199e00 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f1a2400d980 tx=0x7f1a2400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:50.070 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:50.059+0000 7f1a2b7fe700 1 -- 172.21.15.80:0/873349850 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a24017070 con 0x7f1a34101af0 2024-04-24T23:28:50.070 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:50.059+0000 7f1a3c6b3700 1 -- 172.21.15.80:0/873349850 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a34197710 con 0x7f1a34101af0 2024-04-24T23:28:50.070 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:50.059+0000 7f1a2b7fe700 1 -- 172.21.15.80:0/873349850 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a24005020 con 0x7f1a34101af0 2024-04-24T23:28:50.070 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:50.059+0000 7f1a3c6b3700 1 -- 172.21.15.80:0/873349850 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a34197c50 con 0x7f1a34101af0 2024-04-24T23:28:50.070 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:50.059+0000 7f1a2b7fe700 1 -- 172.21.15.80:0/873349850 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a2400fbf0 con 0x7f1a34101af0 2024-04-24T23:28:50.070 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:50.060+0000 7f1a3c6b3700 1 -- 172.21.15.80:0/873349850 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a34068f20 con 0x7f1a34101af0 2024-04-24T23:28:50.071 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:50.062+0000 7f1a2b7fe700 1 -- 172.21.15.80:0/873349850 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f1a240051d0 con 0x7f1a34101af0 2024-04-24T23:28:50.071 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:50.063+0000 7f1a2b7fe700 1 --2- 172.21.15.80:0/873349850 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f1a200a2cb0 0x7f1a200a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:50.071 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:50.063+0000 7f1a2b7fe700 1 -- 172.21.15.80:0/873349850 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f1a240cf740 con 0x7f1a34101af0 2024-04-24T23:28:50.071 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:50.063+0000 7f1a3a44f700 1 --2- 172.21.15.80:0/873349850 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f1a200a2cb0 0x7f1a200a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:50.071 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:50.063+0000 7f1a3a44f700 1 --2- 172.21.15.80:0/873349850 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f1a200a2cb0 0x7f1a200a5170 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f1a3000b2e0 tx=0x7f1a30005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:50.071 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:50.067+0000 7f1a2b7fe700 1 -- 172.21.15.80:0/873349850 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f1a24020020 con 0x7f1a34101af0 2024-04-24T23:28:50.278 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:50.277+0000 7f1a3c6b3700 1 -- 172.21.15.80:0/873349850 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f1a3402db30 con 0x7f1a34101af0 2024-04-24T23:28:50.278 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:50.277+0000 7f1a2b7fe700 1 -- 172.21.15.80:0/873349850 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f1a2409c1e0 con 0x7f1a34101af0 2024-04-24T23:28:50.278 INFO:teuthology.orchestra.run.smithi080.stdout:107374182474 2024-04-24T23:28:50.280 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:50.279+0000 7f1a3c6b3700 1 -- 172.21.15.80:0/873349850 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f1a200a2cb0 msgr2=0x7f1a200a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:50.280 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:50.279+0000 7f1a3c6b3700 1 --2- 172.21.15.80:0/873349850 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f1a200a2cb0 0x7f1a200a5170 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f1a3000b2e0 tx=0x7f1a30005960 comp rx=0 tx=0).stop 2024-04-24T23:28:50.280 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:50.279+0000 7f1a3c6b3700 1 -- 172.21.15.80:0/873349850 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1a34101af0 msgr2=0x7f1a34199e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:50.280 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:50.279+0000 7f1a3c6b3700 1 --2- 172.21.15.80:0/873349850 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1a34101af0 0x7f1a34199e00 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f1a2400d980 tx=0x7f1a2400dc90 comp rx=0 tx=0).stop 2024-04-24T23:28:50.280 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:50.279+0000 7f1a3c6b3700 1 -- 172.21.15.80:0/873349850 shutdown_connections 2024-04-24T23:28:50.281 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:50.279+0000 7f1a3c6b3700 1 --2- 172.21.15.80:0/873349850 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a341071a0 0x7f1a341a4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:50.281 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:50.279+0000 7f1a3c6b3700 1 --2- 172.21.15.80:0/873349850 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1a34101130 0x7f1a341998c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:50.281 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:50.279+0000 7f1a3c6b3700 1 --2- 172.21.15.80:0/873349850 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f1a200a2cb0 0x7f1a200a5170 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:50.281 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:50.279+0000 7f1a3c6b3700 1 --2- 172.21.15.80:0/873349850 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1a34101af0 0x7f1a34199e00 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:50.281 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:50.279+0000 7f1a3c6b3700 1 -- 172.21.15.80:0/873349850 >> 172.21.15.80:0/873349850 conn(0x7f1a340780e0 msgr2=0x7f1a340ffef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:50.281 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:50.279+0000 7f1a3c6b3700 1 -- 172.21.15.80:0/873349850 shutdown_connections 2024-04-24T23:28:50.281 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:50.279+0000 7f1a3c6b3700 1 -- 172.21.15.80:0/873349850 wait complete. 2024-04-24T23:28:50.287 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:50.286+0000 7f0c7e8a8700 1 -- 172.21.15.80:0/499709837 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0c78101b20 msgr2=0x7f0c7810e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:50.287 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:50.286+0000 7f0c7e8a8700 1 --2- 172.21.15.80:0/499709837 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0c78101b20 0x7f0c7810e010 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f0c7000b0a0 tx=0x7f0c7000b3b0 comp rx=0 tx=0).stop 2024-04-24T23:28:50.287 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:50.286+0000 7f0c7e8a8700 1 -- 172.21.15.80:0/499709837 shutdown_connections 2024-04-24T23:28:50.287 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:50.286+0000 7f0c7e8a8700 1 --2- 172.21.15.80:0/499709837 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0c78101b20 0x7f0c7810e010 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:50.287 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:50.286+0000 7f0c7e8a8700 1 --2- 172.21.15.80:0/499709837 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c781071d0 0x7f0c781075b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:50.287 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:50.286+0000 7f0c7e8a8700 1 --2- 172.21.15.80:0/499709837 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c78101160 0x7f0c781015e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:50.287 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:50.286+0000 7f0c7e8a8700 1 -- 172.21.15.80:0/499709837 >> 172.21.15.80:0/499709837 conn(0x7f0c78078170 msgr2=0x7f0c78078580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:50.287 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:50.286+0000 7f0c7e8a8700 1 -- 172.21.15.80:0/499709837 shutdown_connections 2024-04-24T23:28:50.288 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:50.286+0000 7f0c7e8a8700 1 -- 172.21.15.80:0/499709837 wait complete. 2024-04-24T23:28:50.290 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:50.287+0000 7f0c7e8a8700 1 Processor -- start 2024-04-24T23:28:50.290 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:50.287+0000 7f0c7e8a8700 1 -- start start 2024-04-24T23:28:50.290 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:50.287+0000 7f0c7e8a8700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c78101160 0x7f0c781080b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:50.290 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:50.287+0000 7f0c7e8a8700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0c78101b20 0x7f0c781085f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:50.290 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:50.287+0000 7f0c7e8a8700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c781071d0 0x7f0c78108b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:50.290 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:50.287+0000 7f0c7e8a8700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f0c78111470 con 0x7f0c78101b20 2024-04-24T23:28:50.290 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:50.287+0000 7f0c7e8a8700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0c781112f0 con 0x7f0c781071d0 2024-04-24T23:28:50.290 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:50.287+0000 7f0c7e8a8700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f0c781115f0 con 0x7f0c78101160 2024-04-24T23:28:50.291 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:50.287+0000 7f0c777fe700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0c78101b20 0x7f0c781085f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:50.291 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:50.287+0000 7f0c777fe700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0c78101b20 0x7f0c781085f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:44820/0 (socket says 172.21.15.80:44820) 2024-04-24T23:28:50.291 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:50.287+0000 7f0c777fe700 1 -- 172.21.15.80:0/2408685675 learned_addr learned my addr 172.21.15.80:0/2408685675 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:28:50.291 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:50.287+0000 7f0c77fff700 1 --2- 172.21.15.80:0/2408685675 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c78101160 0x7f0c781080b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:50.291 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:50.287+0000 7f0c7ce45700 1 --2- 172.21.15.80:0/2408685675 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c781071d0 0x7f0c78108b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:50.291 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:50.288+0000 7f0c777fe700 1 -- 172.21.15.80:0/2408685675 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c78101160 msgr2=0x7f0c781080b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:50.291 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:50.288+0000 7f0c777fe700 1 --2- 172.21.15.80:0/2408685675 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c78101160 0x7f0c781080b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:50.291 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:50.288+0000 7f0c777fe700 1 -- 172.21.15.80:0/2408685675 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c781071d0 msgr2=0x7f0c78108b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:50.291 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:50.288+0000 7f0c777fe700 1 --2- 172.21.15.80:0/2408685675 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c781071d0 0x7f0c78108b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:50.291 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:50.288+0000 7f0c777fe700 1 -- 172.21.15.80:0/2408685675 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c781a6ef0 con 0x7f0c78101b20 2024-04-24T23:28:50.291 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:50.288+0000 7f0c777fe700 1 --2- 172.21.15.80:0/2408685675 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0c78101b20 0x7f0c781085f0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f0c6800d980 tx=0x7f0c6800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:50.291 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:50.288+0000 7f0c757fa700 1 -- 172.21.15.80:0/2408685675 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c68017070 con 0x7f0c78101b20 2024-04-24T23:28:50.291 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:50.288+0000 7f0c7e8a8700 1 -- 172.21.15.80:0/2408685675 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c781a71e0 con 0x7f0c78101b20 2024-04-24T23:28:50.291 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:50.288+0000 7f0c757fa700 1 -- 172.21.15.80:0/2408685675 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c68005020 con 0x7f0c78101b20 2024-04-24T23:28:50.291 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:50.288+0000 7f0c757fa700 1 -- 172.21.15.80:0/2408685675 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c6800fbf0 con 0x7f0c78101b20 2024-04-24T23:28:50.292 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:50.288+0000 7f0c7e8a8700 1 -- 172.21.15.80:0/2408685675 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c781a7720 con 0x7f0c78101b20 2024-04-24T23:28:50.292 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:50.289+0000 7f0c7e8a8700 1 -- 172.21.15.80:0/2408685675 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c78102880 con 0x7f0c78101b20 2024-04-24T23:28:50.293 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:50.291+0000 7f0c757fa700 1 -- 172.21.15.80:0/2408685675 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f0c680051d0 con 0x7f0c78101b20 2024-04-24T23:28:50.293 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:50.291+0000 7f0c757fa700 1 --2- 172.21.15.80:0/2408685675 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f0c600a2cb0 0x7f0c600a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:50.293 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:50.292+0000 7f0c77fff700 1 --2- 172.21.15.80:0/2408685675 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f0c600a2cb0 0x7f0c600a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:50.293 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:50.292+0000 7f0c757fa700 1 -- 172.21.15.80:0/2408685675 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f0c680cf6f0 con 0x7f0c78101b20 2024-04-24T23:28:50.293 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:50.292+0000 7f0c77fff700 1 --2- 172.21.15.80:0/2408685675 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f0c600a2cb0 0x7f0c600a5170 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c005950 tx=0x7f0c6c00a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:50.296 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:50.296+0000 7f0c757fa700 1 -- 172.21.15.80:0/2408685675 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f0c68020020 con 0x7f0c78101b20 2024-04-24T23:28:50.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:28:50 smithi134 ceph-mon[32619]: pgmap v343: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T23:28:50.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:28:50 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/3221753615' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-24T23:28:50.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:28:50 smithi148 ceph-mon[32528]: pgmap v343: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T23:28:50.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:28:50 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/3221753615' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-24T23:28:50.510 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:50.510+0000 7f0c7e8a8700 1 -- 172.21.15.80:0/2408685675 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f0c781a7a40 con 0x7f0c78101b20 2024-04-24T23:28:50.511 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:28:50 smithi080 ceph-mon[29152]: pgmap v343: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T23:28:50.511 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:28:50 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/3221753615' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-24T23:28:50.511 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:50.510+0000 7f0c757fa700 1 -- 172.21.15.80:0/2408685675 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f0c6809c190 con 0x7f0c78101b20 2024-04-24T23:28:50.511 INFO:teuthology.orchestra.run.smithi080.stdout:197568495674 2024-04-24T23:28:50.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:50.513+0000 7f0c7e8a8700 1 -- 172.21.15.80:0/2408685675 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f0c600a2cb0 msgr2=0x7f0c600a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:50.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:50.513+0000 7f0c7e8a8700 1 --2- 172.21.15.80:0/2408685675 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f0c600a2cb0 0x7f0c600a5170 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c005950 tx=0x7f0c6c00a300 comp rx=0 tx=0).stop 2024-04-24T23:28:50.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:50.513+0000 7f0c7e8a8700 1 -- 172.21.15.80:0/2408685675 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0c78101b20 msgr2=0x7f0c781085f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:50.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:50.513+0000 7f0c7e8a8700 1 --2- 172.21.15.80:0/2408685675 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0c78101b20 0x7f0c781085f0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f0c6800d980 tx=0x7f0c6800dc90 comp rx=0 tx=0).stop 2024-04-24T23:28:50.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:50.514+0000 7f0c7e8a8700 1 -- 172.21.15.80:0/2408685675 shutdown_connections 2024-04-24T23:28:50.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:50.514+0000 7f0c7e8a8700 1 --2- 172.21.15.80:0/2408685675 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c781071d0 0x7f0c78108b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:50.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:50.514+0000 7f0c7e8a8700 1 --2- 172.21.15.80:0/2408685675 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c78101160 0x7f0c781080b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:50.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:50.514+0000 7f0c7e8a8700 1 --2- 172.21.15.80:0/2408685675 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f0c600a2cb0 0x7f0c600a5170 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:50.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:50.514+0000 7f0c7e8a8700 1 --2- 172.21.15.80:0/2408685675 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0c78101b20 0x7f0c781085f0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:50.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:50.514+0000 7f0c7e8a8700 1 -- 172.21.15.80:0/2408685675 >> 172.21.15.80:0/2408685675 conn(0x7f0c78078170 msgr2=0x7f0c780ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:50.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:50.514+0000 7f0c7e8a8700 1 -- 172.21.15.80:0/2408685675 shutdown_connections 2024-04-24T23:28:50.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:50.514+0000 7f0c7e8a8700 1 -- 172.21.15.80:0/2408685675 wait complete. 2024-04-24T23:28:51.042 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574926 got 55834574931 for osd.1 2024-04-24T23:28:51.043 DEBUG:teuthology.parallel:result is None 2024-04-24T23:28:51.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:28:51 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/873349850' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-24T23:28:51.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:28:51 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/2408685675' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-24T23:28:51.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:28:51 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/873349850' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-24T23:28:51.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:28:51 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/2408685675' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-24T23:28:51.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:28:51 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/873349850' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-24T23:28:51.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:28:51 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/2408685675' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-24T23:28:51.873 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:49.838+0000 7f1c1b714700 1 -- 172.21.15.80:0/2960223777 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1c1406ac00 msgr2=0x7f1c1406afe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:51.873 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:49.838+0000 7f1c1b714700 1 --2- 172.21.15.80:0/2960223777 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1c1406ac00 0x7f1c1406afe0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f1c040099c0 tx=0x7f1c04009cd0 comp rx=0 tx=0).stop 2024-04-24T23:28:51.873 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:49.838+0000 7f1c1b714700 1 -- 172.21.15.80:0/2960223777 shutdown_connections 2024-04-24T23:28:51.874 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:49.838+0000 7f1c1b714700 1 --2- 172.21.15.80:0/2960223777 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1c14107610 0x7f1c1410f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:51.874 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:49.838+0000 7f1c1b714700 1 --2- 172.21.15.80:0/2960223777 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1c1406ac00 0x7f1c1406afe0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:51.874 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:49.838+0000 7f1c1b714700 1 --2- 172.21.15.80:0/2960223777 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1c1406b5b0 0x7f1c14106ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:51.874 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:49.838+0000 7f1c1b714700 1 -- 172.21.15.80:0/2960223777 >> 172.21.15.80:0/2960223777 conn(0x7f1c14077180 msgr2=0x7f1c14077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:51.874 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:49.838+0000 7f1c1b714700 1 -- 172.21.15.80:0/2960223777 shutdown_connections 2024-04-24T23:28:51.874 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:49.838+0000 7f1c1b714700 1 -- 172.21.15.80:0/2960223777 wait complete. 2024-04-24T23:28:51.874 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:49.838+0000 7f1c1b714700 1 Processor -- start 2024-04-24T23:28:51.874 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:49.838+0000 7f1c1b714700 1 -- start start 2024-04-24T23:28:51.874 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:49.839+0000 7f1c1b714700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1c1406ac00 0x7f1c1419e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:51.874 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:49.839+0000 7f1c1b714700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1c1406b5b0 0x7f1c1419ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:51.874 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:49.839+0000 7f1c1b714700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1c14107610 0x7f1c14198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:51.874 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:49.839+0000 7f1c1b714700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f1c14112680 con 0x7f1c14107610 2024-04-24T23:28:51.874 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:49.839+0000 7f1c1b714700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1c14112500 con 0x7f1c1406ac00 2024-04-24T23:28:51.874 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:49.839+0000 7f1c1b714700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1c14112800 con 0x7f1c1406b5b0 2024-04-24T23:28:51.875 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:49.839+0000 7f1c19cb1700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1c14107610 0x7f1c14198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:51.875 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:49.839+0000 7f1c19cb1700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1c14107610 0x7f1c14198a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:44770/0 (socket says 172.21.15.80:44770) 2024-04-24T23:28:51.875 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:49.839+0000 7f1c19cb1700 1 -- 172.21.15.80:0/3221753615 learned_addr learned my addr 172.21.15.80:0/3221753615 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:28:51.875 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:49.839+0000 7f1c194b0700 1 --2- 172.21.15.80:0/3221753615 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1c1406ac00 0x7f1c1419e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:51.875 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:49.839+0000 7f1c18caf700 1 --2- 172.21.15.80:0/3221753615 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1c1406b5b0 0x7f1c1419ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:51.875 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:49.839+0000 7f1c19cb1700 1 -- 172.21.15.80:0/3221753615 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1c1406b5b0 msgr2=0x7f1c1419ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:51.875 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:49.839+0000 7f1c19cb1700 1 --2- 172.21.15.80:0/3221753615 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1c1406b5b0 0x7f1c1419ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:51.875 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:49.839+0000 7f1c19cb1700 1 -- 172.21.15.80:0/3221753615 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1c1406ac00 msgr2=0x7f1c1419e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:51.875 INFO:teuthology.orchestra.run.smithi080.stdout:219043332148 2024-04-24T23:28:51.876 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:49.839+0000 7f1c19cb1700 1 --2- 172.21.15.80:0/3221753615 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1c1406ac00 0x7f1c1419e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:51.876 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:49.839+0000 7f1c19cb1700 1 -- 172.21.15.80:0/3221753615 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c14199160 con 0x7f1c14107610 2024-04-24T23:28:51.876 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:49.839+0000 7f1c19cb1700 1 --2- 172.21.15.80:0/3221753615 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1c14107610 0x7f1c14198a00 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c00b970 tx=0x7f1c0c00bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:51.876 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:49.839+0000 7f1c0a7fc700 1 -- 172.21.15.80:0/3221753615 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c0c019070 con 0x7f1c14107610 2024-04-24T23:28:51.876 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:49.839+0000 7f1c0a7fc700 1 -- 172.21.15.80:0/3221753615 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c0c014440 con 0x7f1c14107610 2024-04-24T23:28:51.876 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:49.839+0000 7f1c1b714700 1 -- 172.21.15.80:0/3221753615 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c14199450 con 0x7f1c14107610 2024-04-24T23:28:51.876 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:49.839+0000 7f1c0a7fc700 1 -- 172.21.15.80:0/3221753615 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c0c0075d0 con 0x7f1c14107610 2024-04-24T23:28:51.876 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:49.839+0000 7f1c1b714700 1 -- 172.21.15.80:0/3221753615 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c141a5710 con 0x7f1c14107610 2024-04-24T23:28:51.876 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:49.840+0000 7f1c1b714700 1 -- 172.21.15.80:0/3221753615 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c1410a7e0 con 0x7f1c14107610 2024-04-24T23:28:51.876 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:49.841+0000 7f1c0a7fc700 1 -- 172.21.15.80:0/3221753615 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f1c0c01c070 con 0x7f1c14107610 2024-04-24T23:28:51.876 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:49.842+0000 7f1c0a7fc700 1 --2- 172.21.15.80:0/3221753615 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f1c000a2cb0 0x7f1c000a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:51.876 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:49.842+0000 7f1c194b0700 1 --2- 172.21.15.80:0/3221753615 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f1c000a2cb0 0x7f1c000a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:51.877 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:49.842+0000 7f1c0a7fc700 1 -- 172.21.15.80:0/3221753615 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f1c0c0ced50 con 0x7f1c14107610 2024-04-24T23:28:51.877 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:49.842+0000 7f1c194b0700 1 --2- 172.21.15.80:0/3221753615 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f1c000a2cb0 0x7f1c000a5170 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f1c04009990 tx=0x7f1c04009f10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:51.877 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:49.845+0000 7f1c0a7fc700 1 -- 172.21.15.80:0/3221753615 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f1c0c09b900 con 0x7f1c14107610 2024-04-24T23:28:51.877 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:50.056+0000 7f1c1b714700 1 -- 172.21.15.80:0/3221753615 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f1c14199f30 con 0x7f1c14107610 2024-04-24T23:28:51.877 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:50.056+0000 7f1c0a7fc700 1 -- 172.21.15.80:0/3221753615 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f1c0c09b200 con 0x7f1c14107610 2024-04-24T23:28:51.877 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:50.058+0000 7f1c1b714700 1 -- 172.21.15.80:0/3221753615 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f1c000a2cb0 msgr2=0x7f1c000a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:51.877 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:50.058+0000 7f1c1b714700 1 --2- 172.21.15.80:0/3221753615 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f1c000a2cb0 0x7f1c000a5170 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f1c04009990 tx=0x7f1c04009f10 comp rx=0 tx=0).stop 2024-04-24T23:28:51.877 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:50.058+0000 7f1c1b714700 1 -- 172.21.15.80:0/3221753615 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1c14107610 msgr2=0x7f1c14198a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:51.877 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:50.058+0000 7f1c1b714700 1 --2- 172.21.15.80:0/3221753615 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1c14107610 0x7f1c14198a00 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c00b970 tx=0x7f1c0c00bd30 comp rx=0 tx=0).stop 2024-04-24T23:28:51.877 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:50.058+0000 7f1c1b714700 1 -- 172.21.15.80:0/3221753615 shutdown_connections 2024-04-24T23:28:51.877 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:50.058+0000 7f1c1b714700 1 --2- 172.21.15.80:0/3221753615 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f1c000a2cb0 0x7f1c000a5170 secure :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f1c04009990 tx=0x7f1c04009f10 comp rx=0 tx=0).stop 2024-04-24T23:28:51.877 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:50.058+0000 7f1c1b714700 1 --2- 172.21.15.80:0/3221753615 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1c14107610 0x7f1c14198a00 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:51.878 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:50.058+0000 7f1c1b714700 1 --2- 172.21.15.80:0/3221753615 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1c1406ac00 0x7f1c1419e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:51.878 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:50.058+0000 7f1c1b714700 1 --2- 172.21.15.80:0/3221753615 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1c1406b5b0 0x7f1c1419ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:51.878 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:50.059+0000 7f1c1b714700 1 -- 172.21.15.80:0/3221753615 >> 172.21.15.80:0/3221753615 conn(0x7f1c14077180 msgr2=0x7f1c140ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:51.878 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:50.059+0000 7f1c1b714700 1 -- 172.21.15.80:0/3221753615 shutdown_connections 2024-04-24T23:28:51.878 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:50.059+0000 7f1c1b714700 1 -- 172.21.15.80:0/3221753615 wait complete. 2024-04-24T23:28:52.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:28:52 smithi134 ceph-mon[32619]: pgmap v344: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T23:28:52.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:28:52 smithi148 ceph-mon[32528]: pgmap v344: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T23:28:52.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:28:52 smithi080 ceph-mon[29152]: pgmap v344: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T23:28:52.613 INFO:tasks.cephadm.ceph_manager.ceph:need seq 309237645343 got 309237645348 for osd.10 2024-04-24T23:28:52.613 DEBUG:teuthology.parallel:result is None 2024-04-24T23:28:52.747 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182467 got 107374182474 for osd.3 2024-04-24T23:28:52.747 DEBUG:teuthology.parallel:result is None 2024-04-24T23:28:53.174 INFO:tasks.cephadm.ceph_manager.ceph:need seq 287762808867 got 287762808874 for osd.9 2024-04-24T23:28:53.174 DEBUG:teuthology.parallel:result is None 2024-04-24T23:28:53.618 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495670 got 197568495674 for osd.6 2024-04-24T23:28:53.618 DEBUG:teuthology.parallel:result is None 2024-04-24T23:28:54.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:28:54 smithi134 ceph-mon[32619]: pgmap v345: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:28:54.489 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:28:54 smithi080 ceph-mon[29152]: pgmap v345: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:28:54.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:28:54 smithi148 ceph-mon[32528]: pgmap v345: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:28:54.526 INFO:tasks.cephadm.ceph_manager.ceph:need seq 219043332143 got 219043332148 for osd.7 2024-04-24T23:28:54.526 DEBUG:teuthology.parallel:result is None 2024-04-24T23:28:55.980 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:55.979+0000 7fd8a1f88700 1 -- 172.21.15.80:0/3848460928 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd89c1022c0 msgr2=0x7fd89c102740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:55.981 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:55.979+0000 7fd8a1f88700 1 --2- 172.21.15.80:0/3848460928 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd89c1022c0 0x7fd89c102740 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fd890009990 tx=0x7fd890009ca0 comp rx=0 tx=0).stop 2024-04-24T23:28:55.981 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:55.979+0000 7fd8a1f88700 1 -- 172.21.15.80:0/3848460928 shutdown_connections 2024-04-24T23:28:55.981 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:55.979+0000 7fd8a1f88700 1 --2- 172.21.15.80:0/3848460928 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd89c102c80 0x7fd89c10f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:55.981 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:55.979+0000 7fd8a1f88700 1 --2- 172.21.15.80:0/3848460928 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd89c108360 0x7fd89c108740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:55.981 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:55.979+0000 7fd8a1f88700 1 --2- 172.21.15.80:0/3848460928 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd89c1022c0 0x7fd89c102740 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:55.981 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:55.979+0000 7fd8a1f88700 1 -- 172.21.15.80:0/3848460928 >> 172.21.15.80:0/3848460928 conn(0x7fd89c079420 msgr2=0x7fd89c079830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:55.982 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:55.980+0000 7fd8a1f88700 1 -- 172.21.15.80:0/3848460928 shutdown_connections 2024-04-24T23:28:55.982 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:55.981+0000 7fd8a1f88700 1 -- 172.21.15.80:0/3848460928 wait complete. 2024-04-24T23:28:55.982 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:55.982+0000 7fd8a1f88700 1 Processor -- start 2024-04-24T23:28:55.983 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:55.982+0000 7fd8a1f88700 1 -- start start 2024-04-24T23:28:55.983 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:55.983+0000 7fd8a1f88700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd89c1022c0 0x7fd89c1a0920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:55.984 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:55.983+0000 7fd8a1f88700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd89c102c80 0x7fd89c1a0e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:55.984 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:55.983+0000 7fd8a1f88700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd89c108360 0x7fd89c19aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:55.984 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:55.983+0000 7fd8a1f88700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fd89c1125a0 con 0x7fd89c102c80 2024-04-24T23:28:55.984 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:55.983+0000 7fd8a1f88700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd89c112420 con 0x7fd89c108360 2024-04-24T23:28:55.984 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:55.983+0000 7fd8a1f88700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd89c112720 con 0x7fd89c1022c0 2024-04-24T23:28:55.984 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:55.983+0000 7fd89b7fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd89c1022c0 0x7fd89c1a0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:55.984 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:55.983+0000 7fd89b7fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd89c1022c0 0x7fd89c1a0920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.80:42410/0 (socket says 172.21.15.80:42410) 2024-04-24T23:28:55.984 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:55.983+0000 7fd89b7fe700 1 -- 172.21.15.80:0/359057797 learned_addr learned my addr 172.21.15.80:0/359057797 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:28:55.984 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:55.983+0000 7fd89affd700 1 --2- 172.21.15.80:0/359057797 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd89c102c80 0x7fd89c1a0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:55.984 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:55.983+0000 7fd89bfff700 1 --2- 172.21.15.80:0/359057797 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd89c108360 0x7fd89c19aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:55.985 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:55.983+0000 7fd89b7fe700 1 -- 172.21.15.80:0/359057797 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd89c108360 msgr2=0x7fd89c19aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:55.985 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:55.983+0000 7fd89b7fe700 1 --2- 172.21.15.80:0/359057797 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd89c108360 0x7fd89c19aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:55.985 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:55.983+0000 7fd89b7fe700 1 -- 172.21.15.80:0/359057797 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd89c102c80 msgr2=0x7fd89c1a0e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:55.985 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:55.983+0000 7fd89b7fe700 1 --2- 172.21.15.80:0/359057797 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd89c102c80 0x7fd89c1a0e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:55.985 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:55.983+0000 7fd89b7fe700 1 -- 172.21.15.80:0/359057797 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd89c19b240 con 0x7fd89c1022c0 2024-04-24T23:28:55.985 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:55.983+0000 7fd89affd700 1 --2- 172.21.15.80:0/359057797 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd89c102c80 0x7fd89c1a0e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:28:55.986 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:55.984+0000 7fd89b7fe700 1 --2- 172.21.15.80:0/359057797 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd89c1022c0 0x7fd89c1a0920 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd88c00d950 tx=0x7fd88c00dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:55.986 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:55.984+0000 7fd898ff9700 1 -- 172.21.15.80:0/359057797 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd88c016070 con 0x7fd89c1022c0 2024-04-24T23:28:55.986 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:55.984+0000 7fd8a1f88700 1 -- 172.21.15.80:0/359057797 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd89c19b530 con 0x7fd89c1022c0 2024-04-24T23:28:55.986 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:55.984+0000 7fd8a1f88700 1 -- 172.21.15.80:0/359057797 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd89c1a77f0 con 0x7fd89c1022c0 2024-04-24T23:28:55.986 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:55.984+0000 7fd898ff9700 1 -- 172.21.15.80:0/359057797 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd88c00fd70 con 0x7fd89c1022c0 2024-04-24T23:28:55.986 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:55.984+0000 7fd898ff9700 1 -- 172.21.15.80:0/359057797 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd88c020060 con 0x7fd89c1022c0 2024-04-24T23:28:55.986 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:55.986+0000 7fd8a1f88700 1 -- 172.21.15.80:0/359057797 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd888005330 con 0x7fd89c1022c0 2024-04-24T23:28:55.987 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:55.987+0000 7fd898ff9700 1 -- 172.21.15.80:0/359057797 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fd88c019070 con 0x7fd89c1022c0 2024-04-24T23:28:55.988 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:55.988+0000 7fd898ff9700 1 --2- 172.21.15.80:0/359057797 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fd8840a2d30 0x7fd8840a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:55.989 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:55.988+0000 7fd898ff9700 1 -- 172.21.15.80:0/359057797 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fd88c0cef00 con 0x7fd89c1022c0 2024-04-24T23:28:55.989 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:55.988+0000 7fd89affd700 1 --2- 172.21.15.80:0/359057797 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fd8840a2d30 0x7fd8840a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:55.989 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:55.988+0000 7fd89affd700 1 --2- 172.21.15.80:0/359057797 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fd8840a2d30 0x7fd8840a51f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fd89000b2e0 tx=0x7fd89001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:55.993 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:55.993+0000 7fd898ff9700 1 -- 172.21.15.80:0/359057797 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fd88c09b940 con 0x7fd89c1022c0 2024-04-24T23:28:56.207 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:56.205+0000 7fd8a1f88700 1 -- 172.21.15.80:0/359057797 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7fd8880058f0 con 0x7fd89c1022c0 2024-04-24T23:28:56.207 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:56.206+0000 7fd898ff9700 1 -- 172.21.15.80:0/359057797 <== mon.1 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fd88c09b240 con 0x7fd89c1022c0 2024-04-24T23:28:56.207 INFO:teuthology.orchestra.run.smithi080.stdout:343597383713 2024-04-24T23:28:56.210 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:56.209+0000 7fd8a1f88700 1 -- 172.21.15.80:0/359057797 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fd8840a2d30 msgr2=0x7fd8840a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:56.210 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:56.209+0000 7fd8a1f88700 1 --2- 172.21.15.80:0/359057797 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fd8840a2d30 0x7fd8840a51f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fd89000b2e0 tx=0x7fd89001a040 comp rx=0 tx=0).stop 2024-04-24T23:28:56.210 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:56.209+0000 7fd8a1f88700 1 -- 172.21.15.80:0/359057797 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd89c1022c0 msgr2=0x7fd89c1a0920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:56.210 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:56.209+0000 7fd8a1f88700 1 --2- 172.21.15.80:0/359057797 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd89c1022c0 0x7fd89c1a0920 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd88c00d950 tx=0x7fd88c00dc60 comp rx=0 tx=0).stop 2024-04-24T23:28:56.211 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:56.209+0000 7fd8a1f88700 1 -- 172.21.15.80:0/359057797 shutdown_connections 2024-04-24T23:28:56.211 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:56.209+0000 7fd8a1f88700 1 --2- 172.21.15.80:0/359057797 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd89c108360 0x7fd89c19aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:56.211 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:56.209+0000 7fd8a1f88700 1 --2- 172.21.15.80:0/359057797 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd89c1022c0 0x7fd89c1a0920 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:56.212 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:56.209+0000 7fd8a1f88700 1 --2- 172.21.15.80:0/359057797 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fd8840a2d30 0x7fd8840a51f0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:56.212 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:56.209+0000 7fd8a1f88700 1 --2- 172.21.15.80:0/359057797 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd89c102c80 0x7fd89c1a0e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:56.212 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:56.209+0000 7fd8a1f88700 1 -- 172.21.15.80:0/359057797 >> 172.21.15.80:0/359057797 conn(0x7fd89c079420 msgr2=0x7fd89c100ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:56.212 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:56.210+0000 7fd8a1f88700 1 -- 172.21.15.80:0/359057797 shutdown_connections 2024-04-24T23:28:56.212 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:56.210+0000 7fd8a1f88700 1 -- 172.21.15.80:0/359057797 wait complete. 2024-04-24T23:28:56.235 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:28:56 smithi080 ceph-mon[29152]: pgmap v346: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T23:28:56.235 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:28:56 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:28:56.235 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:28:56 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:28:56.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:28:56 smithi134 ceph-mon[32619]: pgmap v346: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T23:28:56.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:28:56 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:28:56.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:28:56 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:28:56.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:28:56 smithi148 ceph-mon[32528]: pgmap v346: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T23:28:56.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:28:56 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:28:56.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:28:56 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:28:56.645 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:56.644+0000 7f3500367700 1 -- 172.21.15.80:0/235908334 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f34f8101af0 msgr2=0x7f34f810dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:56.646 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:56.644+0000 7f3500367700 1 --2- 172.21.15.80:0/235908334 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f34f8101af0 0x7f34f810dfe0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f34f000b0a0 tx=0x7f34f000b3b0 comp rx=0 tx=0).stop 2024-04-24T23:28:56.646 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:56.644+0000 7f3500367700 1 -- 172.21.15.80:0/235908334 shutdown_connections 2024-04-24T23:28:56.646 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:56.644+0000 7f3500367700 1 --2- 172.21.15.80:0/235908334 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f34f8101af0 0x7f34f810dfe0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:56.646 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:56.644+0000 7f3500367700 1 --2- 172.21.15.80:0/235908334 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f34f81071a0 0x7f34f8107580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:56.646 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:56.644+0000 7f3500367700 1 --2- 172.21.15.80:0/235908334 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34f8101130 0x7f34f81015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:56.646 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:56.644+0000 7f3500367700 1 -- 172.21.15.80:0/235908334 >> 172.21.15.80:0/235908334 conn(0x7f34f80780e0 msgr2=0x7f34f80784f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:56.646 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:56.644+0000 7f3500367700 1 -- 172.21.15.80:0/235908334 shutdown_connections 2024-04-24T23:28:56.646 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:56.644+0000 7f3500367700 1 -- 172.21.15.80:0/235908334 wait complete. 2024-04-24T23:28:56.646 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:56.645+0000 7f3500367700 1 Processor -- start 2024-04-24T23:28:56.646 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:56.645+0000 7f3500367700 1 -- start start 2024-04-24T23:28:56.648 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:56.645+0000 7f3500367700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f34f8101130 0x7f34f819e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:56.648 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:56.645+0000 7f3500367700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34f8101af0 0x7f34f819ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:56.648 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:56.645+0000 7f3500367700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f34f81071a0 0x7f34f8198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:56.649 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:56.645+0000 7f3500367700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f34f81114e0 con 0x7f34f8101130 2024-04-24T23:28:56.649 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:56.645+0000 7f3500367700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f34f8111360 con 0x7f34f81071a0 2024-04-24T23:28:56.649 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:56.645+0000 7f3500367700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f34f8111660 con 0x7f34f8101af0 2024-04-24T23:28:56.649 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:56.645+0000 7f34fe103700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f34f8101130 0x7f34f819e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:56.649 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:56.645+0000 7f34fe103700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f34f8101130 0x7f34f819e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53062/0 (socket says 172.21.15.80:53062) 2024-04-24T23:28:56.649 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:56.645+0000 7f34fe103700 1 -- 172.21.15.80:0/1272990030 learned_addr learned my addr 172.21.15.80:0/1272990030 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:28:56.649 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:56.646+0000 7f34fe904700 1 --2- 172.21.15.80:0/1272990030 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f34f81071a0 0x7f34f8198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:56.649 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:56.646+0000 7f34fd902700 1 --2- 172.21.15.80:0/1272990030 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34f8101af0 0x7f34f819ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:56.649 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:56.646+0000 7f34fe103700 1 -- 172.21.15.80:0/1272990030 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34f8101af0 msgr2=0x7f34f819ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:56.649 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:56.646+0000 7f34fe103700 1 --2- 172.21.15.80:0/1272990030 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34f8101af0 0x7f34f819ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:56.649 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:56.646+0000 7f34fe103700 1 -- 172.21.15.80:0/1272990030 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f34f81071a0 msgr2=0x7f34f8198890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:56.649 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:56.646+0000 7f34fe103700 1 --2- 172.21.15.80:0/1272990030 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f34f81071a0 0x7f34f8198890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:56.649 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:56.646+0000 7f34fe103700 1 -- 172.21.15.80:0/1272990030 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34f8199070 con 0x7f34f8101130 2024-04-24T23:28:56.649 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:56.646+0000 7f34fe904700 1 --2- 172.21.15.80:0/1272990030 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f34f81071a0 0x7f34f8198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:28:56.650 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:56.646+0000 7f34fe103700 1 --2- 172.21.15.80:0/1272990030 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f34f8101130 0x7f34f819e6c0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f34e800c9b0 tx=0x7f34e800cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:56.650 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:56.646+0000 7f34ef7fe700 1 -- 172.21.15.80:0/1272990030 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34e8007b00 con 0x7f34f8101130 2024-04-24T23:28:56.650 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:56.646+0000 7f34ef7fe700 1 -- 172.21.15.80:0/1272990030 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34e8007ca0 con 0x7f34f8101130 2024-04-24T23:28:56.650 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:56.646+0000 7f3500367700 1 -- 172.21.15.80:0/1272990030 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34f8199360 con 0x7f34f8101130 2024-04-24T23:28:56.650 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:56.646+0000 7f34ef7fe700 1 -- 172.21.15.80:0/1272990030 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34e8018780 con 0x7f34f8101130 2024-04-24T23:28:56.650 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:56.646+0000 7f3500367700 1 -- 172.21.15.80:0/1272990030 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34f81a5520 con 0x7f34f8101130 2024-04-24T23:28:56.650 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:56.647+0000 7f3500367700 1 -- 172.21.15.80:0/1272990030 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34f8102850 con 0x7f34f8101130 2024-04-24T23:28:56.651 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:56.648+0000 7f34ef7fe700 1 -- 172.21.15.80:0/1272990030 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f34e800f450 con 0x7f34f8101130 2024-04-24T23:28:56.651 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:56.649+0000 7f34ef7fe700 1 --2- 172.21.15.80:0/1272990030 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f34e40a2cb0 0x7f34e40a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:56.651 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:56.649+0000 7f34fd902700 1 --2- 172.21.15.80:0/1272990030 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f34e40a2cb0 0x7f34e40a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:56.651 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:56.649+0000 7f34ef7fe700 1 -- 172.21.15.80:0/1272990030 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f34e80cf850 con 0x7f34f8101130 2024-04-24T23:28:56.651 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:56.649+0000 7f34fd902700 1 --2- 172.21.15.80:0/1272990030 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f34e40a2cb0 0x7f34e40a5170 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f34f4006fd0 tx=0x7f34f4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:56.653 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:56.653+0000 7f34ef7fe700 1 -- 172.21.15.80:0/1272990030 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f34e8014030 con 0x7f34f8101130 2024-04-24T23:28:56.861 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:56.860+0000 7f3500367700 1 -- 172.21.15.80:0/1272990030 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f34f81a5810 con 0x7f34f8101130 2024-04-24T23:28:56.861 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:56.860+0000 7f34ef7fe700 1 -- 172.21.15.80:0/1272990030 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f34e800de20 con 0x7f34f8101130 2024-04-24T23:28:56.861 INFO:teuthology.orchestra.run.smithi080.stdout:34359738457 2024-04-24T23:28:56.863 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:56.862+0000 7f3500367700 1 -- 172.21.15.80:0/1272990030 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f34e40a2cb0 msgr2=0x7f34e40a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:56.863 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:56.862+0000 7f3500367700 1 --2- 172.21.15.80:0/1272990030 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f34e40a2cb0 0x7f34e40a5170 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f34f4006fd0 tx=0x7f34f4008040 comp rx=0 tx=0).stop 2024-04-24T23:28:56.863 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:56.862+0000 7f3500367700 1 -- 172.21.15.80:0/1272990030 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f34f8101130 msgr2=0x7f34f819e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:56.863 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:56.862+0000 7f3500367700 1 --2- 172.21.15.80:0/1272990030 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f34f8101130 0x7f34f819e6c0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f34e800c9b0 tx=0x7f34e800cd70 comp rx=0 tx=0).stop 2024-04-24T23:28:56.863 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:56.862+0000 7f3500367700 1 -- 172.21.15.80:0/1272990030 shutdown_connections 2024-04-24T23:28:56.863 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:56.862+0000 7f3500367700 1 --2- 172.21.15.80:0/1272990030 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f34f81071a0 0x7f34f8198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:56.863 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:56.862+0000 7f3500367700 1 --2- 172.21.15.80:0/1272990030 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f34e40a2cb0 0x7f34e40a5170 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:56.863 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:56.862+0000 7f3500367700 1 --2- 172.21.15.80:0/1272990030 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f34f8101130 0x7f34f819e6c0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:56.863 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:56.862+0000 7f3500367700 1 --2- 172.21.15.80:0/1272990030 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34f8101af0 0x7f34f819ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:56.864 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:56.862+0000 7f3500367700 1 -- 172.21.15.80:0/1272990030 >> 172.21.15.80:0/1272990030 conn(0x7f34f80780e0 msgr2=0x7f34f8100280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:56.864 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:56.862+0000 7f3500367700 1 -- 172.21.15.80:0/1272990030 shutdown_connections 2024-04-24T23:28:56.864 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:56.862+0000 7f3500367700 1 -- 172.21.15.80:0/1272990030 wait complete. 2024-04-24T23:28:57.129 INFO:tasks.cephadm.ceph_manager.ceph:need seq 343597383705 got 343597383713 for osd.11 2024-04-24T23:28:57.129 DEBUG:teuthology.parallel:result is None 2024-04-24T23:28:57.391 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:28:57 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:28:57.391 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:28:57 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:28:57.391 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:28:57 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:28:57.391 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:28:57 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/359057797' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-24T23:28:57.391 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:28:57 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/1272990030' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-24T23:28:57.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:28:57 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:28:57.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:28:57 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:28:57.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:28:57 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:28:57.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:28:57 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/359057797' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-24T23:28:57.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:28:57 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/1272990030' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-24T23:28:57.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:28:57 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:28:57.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:28:57 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:28:57.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:28:57 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:28:57.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:28:57 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/359057797' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-24T23:28:57.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:28:57 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/1272990030' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-24T23:28:57.775 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738451 got 34359738457 for osd.0 2024-04-24T23:28:57.775 DEBUG:teuthology.parallel:result is None 2024-04-24T23:28:57.775 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-24T23:28:57.776 DEBUG:teuthology.orchestra.run.smithi080:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid ec7885aa-0290-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-24T23:28:58.016 INFO:teuthology.orchestra.run.smithi080.stderr:Inferring config /var/lib/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:28:58.352 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:28:58 smithi080 ceph-mon[29152]: pgmap v347: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:28:58.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:28:58 smithi134 ceph-mon[32619]: pgmap v347: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:28:58.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:28:58 smithi148 ceph-mon[32528]: pgmap v347: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:28:59.914 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:59.869+0000 7fb36535b700 1 -- 172.21.15.80:0/1875542399 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb360101af0 msgr2=0x7fb36010dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:59.914 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:59.870+0000 7fb36535b700 1 --2- 172.21.15.80:0/1875542399 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb360101af0 0x7fb36010dfe0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fb35800b0a0 tx=0x7fb35800b3b0 comp rx=0 tx=0).stop 2024-04-24T23:28:59.914 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:59.870+0000 7fb36535b700 1 -- 172.21.15.80:0/1875542399 shutdown_connections 2024-04-24T23:28:59.914 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:59.870+0000 7fb36535b700 1 --2- 172.21.15.80:0/1875542399 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb360101af0 0x7fb36010dfe0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:59.914 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:59.870+0000 7fb36535b700 1 --2- 172.21.15.80:0/1875542399 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3601071a0 0x7fb360107580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:59.914 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:59.870+0000 7fb36535b700 1 --2- 172.21.15.80:0/1875542399 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb360101130 0x7fb3601015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:59.914 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:59.870+0000 7fb36535b700 1 -- 172.21.15.80:0/1875542399 >> 172.21.15.80:0/1875542399 conn(0x7fb3600780e0 msgr2=0x7fb3600784f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:59.914 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:59.870+0000 7fb36535b700 1 -- 172.21.15.80:0/1875542399 shutdown_connections 2024-04-24T23:28:59.914 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:59.870+0000 7fb36535b700 1 -- 172.21.15.80:0/1875542399 wait complete. 2024-04-24T23:28:59.915 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:59.870+0000 7fb36535b700 1 Processor -- start 2024-04-24T23:28:59.915 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:59.870+0000 7fb36535b700 1 -- start start 2024-04-24T23:28:59.915 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:59.871+0000 7fb36535b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb360101130 0x7fb36019e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:59.915 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:59.871+0000 7fb36535b700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb360101af0 0x7fb36019eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:59.915 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:59.871+0000 7fb36535b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3601071a0 0x7fb3601987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:59.915 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:59.871+0000 7fb36535b700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fb360111490 con 0x7fb360101af0 2024-04-24T23:28:59.915 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:59.871+0000 7fb36535b700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb360111310 con 0x7fb360101130 2024-04-24T23:28:59.915 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:59.871+0000 7fb36535b700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb360111610 con 0x7fb3601071a0 2024-04-24T23:28:59.915 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:59.871+0000 7fb35e7fc700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb360101af0 0x7fb36019eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:59.915 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:59.871+0000 7fb35e7fc700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb360101af0 0x7fb36019eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53070/0 (socket says 172.21.15.80:53070) 2024-04-24T23:28:59.915 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:59.871+0000 7fb35e7fc700 1 -- 172.21.15.80:0/4050997412 learned_addr learned my addr 172.21.15.80:0/4050997412 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:28:59.915 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:59.871+0000 7fb35effd700 1 --2- 172.21.15.80:0/4050997412 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb360101130 0x7fb36019e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:59.915 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:59.871+0000 7fb35f7fe700 1 --2- 172.21.15.80:0/4050997412 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3601071a0 0x7fb3601987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:59.916 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:59.871+0000 7fb35e7fc700 1 -- 172.21.15.80:0/4050997412 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3601071a0 msgr2=0x7fb3601987e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:59.916 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:59.871+0000 7fb35e7fc700 1 --2- 172.21.15.80:0/4050997412 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3601071a0 0x7fb3601987e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:59.916 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:59.871+0000 7fb35e7fc700 1 -- 172.21.15.80:0/4050997412 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb360101130 msgr2=0x7fb36019e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:59.916 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:59.871+0000 7fb35e7fc700 1 --2- 172.21.15.80:0/4050997412 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb360101130 0x7fb36019e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:59.916 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:59.871+0000 7fb35e7fc700 1 -- 172.21.15.80:0/4050997412 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb360198fc0 con 0x7fb360101af0 2024-04-24T23:28:59.916 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:59.871+0000 7fb35e7fc700 1 --2- 172.21.15.80:0/4050997412 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb360101af0 0x7fb36019eb50 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fb35000c9b0 tx=0x7fb35000ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:59.916 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:59.871+0000 7fb347fff700 1 -- 172.21.15.80:0/4050997412 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb350007b00 con 0x7fb360101af0 2024-04-24T23:28:59.916 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:59.871+0000 7fb36535b700 1 -- 172.21.15.80:0/4050997412 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3601992b0 con 0x7fb360101af0 2024-04-24T23:28:59.916 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:59.871+0000 7fb347fff700 1 -- 172.21.15.80:0/4050997412 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb35000ce50 con 0x7fb360101af0 2024-04-24T23:28:59.916 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:59.871+0000 7fb347fff700 1 -- 172.21.15.80:0/4050997412 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb350018660 con 0x7fb360101af0 2024-04-24T23:28:59.916 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:59.871+0000 7fb36535b700 1 -- 172.21.15.80:0/4050997412 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3601a5470 con 0x7fb360101af0 2024-04-24T23:28:59.916 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:59.872+0000 7fb36535b700 1 -- 172.21.15.80:0/4050997412 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb360102850 con 0x7fb360101af0 2024-04-24T23:28:59.916 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:59.873+0000 7fb347fff700 1 -- 172.21.15.80:0/4050997412 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fb35000f450 con 0x7fb360101af0 2024-04-24T23:28:59.916 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:59.874+0000 7fb347fff700 1 --2- 172.21.15.80:0/4050997412 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fb3480a2cb0 0x7fb3480a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:59.916 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:59.874+0000 7fb35effd700 1 --2- 172.21.15.80:0/4050997412 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fb3480a2cb0 0x7fb3480a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:59.917 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:59.874+0000 7fb347fff700 1 -- 172.21.15.80:0/4050997412 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fb3500cf6d0 con 0x7fb360101af0 2024-04-24T23:28:59.917 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:59.874+0000 7fb35effd700 1 --2- 172.21.15.80:0/4050997412 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fb3480a2cb0 0x7fb3480a5170 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb354009a10 tx=0x7fb354009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:59.917 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:28:59.878+0000 7fb347fff700 1 -- 172.21.15.80:0/4050997412 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fb350014030 con 0x7fb360101af0 2024-04-24T23:29:00.072 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:00.072+0000 7fb36535b700 1 -- 172.21.15.80:0/4050997412 --> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb360104f50 con 0x7fb3480a2cb0 2024-04-24T23:29:00.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:00.081+0000 7fb347fff700 1 -- 172.21.15.80:0/4050997412 <== mgr.14152 v2:172.21.15.80:6800/3830596922 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340639 (secure 0 0 0) 0x7fb360104f50 con 0x7fb3480a2cb0 2024-04-24T23:29:00.087 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-24T23:29:00.089 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:00.084+0000 7fb36535b700 1 -- 172.21.15.80:0/4050997412 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fb3480a2cb0 msgr2=0x7fb3480a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:00.089 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:00.084+0000 7fb36535b700 1 --2- 172.21.15.80:0/4050997412 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fb3480a2cb0 0x7fb3480a5170 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb354009a10 tx=0x7fb354009190 comp rx=0 tx=0).stop 2024-04-24T23:29:00.089 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:00.084+0000 7fb36535b700 1 -- 172.21.15.80:0/4050997412 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb360101af0 msgr2=0x7fb36019eb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:00.089 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:00.084+0000 7fb36535b700 1 --2- 172.21.15.80:0/4050997412 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb360101af0 0x7fb36019eb50 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fb35000c9b0 tx=0x7fb35000ccc0 comp rx=0 tx=0).stop 2024-04-24T23:29:00.089 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:00.085+0000 7fb36535b700 1 -- 172.21.15.80:0/4050997412 shutdown_connections 2024-04-24T23:29:00.090 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:00.085+0000 7fb36535b700 1 --2- 172.21.15.80:0/4050997412 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3601071a0 0x7fb3601987e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:00.090 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:00.085+0000 7fb36535b700 1 --2- 172.21.15.80:0/4050997412 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb360101130 0x7fb36019e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:00.090 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:00.085+0000 7fb36535b700 1 --2- 172.21.15.80:0/4050997412 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fb3480a2cb0 0x7fb3480a5170 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:00.090 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:00.085+0000 7fb36535b700 1 --2- 172.21.15.80:0/4050997412 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb360101af0 0x7fb36019eb50 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:00.090 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:00.085+0000 7fb36535b700 1 -- 172.21.15.80:0/4050997412 >> 172.21.15.80:0/4050997412 conn(0x7fb3600780e0 msgr2=0x7fb36006a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:00.090 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:00.085+0000 7fb36535b700 1 -- 172.21.15.80:0/4050997412 shutdown_connections 2024-04-24T23:29:00.090 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:00.085+0000 7fb36535b700 1 -- 172.21.15.80:0/4050997412 wait complete. 2024-04-24T23:29:00.090 INFO:teuthology.orchestra.run.smithi080.stderr:dumped all 2024-04-24T23:29:00.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:29:00 smithi134 ceph-mon[32619]: pgmap v348: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T23:29:00.458 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:29:00 smithi080 ceph-mon[29152]: pgmap v348: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T23:29:00.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:29:00 smithi148 ceph-mon[32528]: pgmap v348: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T23:29:00.824 INFO:teuthology.orchestra.run.smithi080.stdout:{"pg_ready":true,"pg_map":{"version":348,"stamp":"2024-04-24T23:28:58.906671+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":653865,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":582,"num_read_kb":405,"num_write":342,"num_write_kb":3051,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":1181430,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":277,"ondisk_log_size":277,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":365692,"kb_used_data":12296,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124576132,"statfs":{"total":1151940427776,"available":1151565959168,"internally_reserved":0,"allocated":12591104,"data_stored":8060285,"data_compressed":390669,"data_compressed_allocated":528384,"data_compressed_original":2555904,"omap_allocated":19141,"internal_metadata":361674043},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":80,"num_read_kb":40,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003322"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.053673+0000","last_change":"2024-04-24T23:27:19.978807+0000","last_active":"2024-04-24T23:28:08.053673+0000","last_peered":"2024-04-24T23:28:08.053673+0000","last_clean":"2024-04-24T23:28:08.053673+0000","last_became_active":"2024-04-24T23:27:19.978436+0000","last_became_peered":"2024-04-24T23:27:19.978436+0000","last_unstale":"2024-04-24T23:28:08.053673+0000","last_undegraded":"2024-04-24T23:28:08.053673+0000","last_fullsized":"2024-04-24T23:28:08.053673+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:00:13.126518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.049915+0000","last_change":"2024-04-24T23:27:19.980327+0000","last_active":"2024-04-24T23:28:08.049915+0000","last_peered":"2024-04-24T23:28:08.049915+0000","last_clean":"2024-04-24T23:28:08.049915+0000","last_became_active":"2024-04-24T23:27:19.979912+0000","last_became_peered":"2024-04-24T23:27:19.979912+0000","last_unstale":"2024-04-24T23:28:08.049915+0000","last_undegraded":"2024-04-24T23:28:08.049915+0000","last_fullsized":"2024-04-24T23:28:08.049915+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:25:41.657205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.050043+0000","last_change":"2024-04-24T23:27:19.976451+0000","last_active":"2024-04-24T23:28:08.050043+0000","last_peered":"2024-04-24T23:28:08.050043+0000","last_clean":"2024-04-24T23:28:08.050043+0000","last_became_active":"2024-04-24T23:27:19.975976+0000","last_became_peered":"2024-04-24T23:27:19.975976+0000","last_unstale":"2024-04-24T23:28:08.050043+0000","last_undegraded":"2024-04-24T23:28:08.050043+0000","last_fullsized":"2024-04-24T23:28:08.050043+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:14:51.605217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.055317+0000","last_change":"2024-04-24T23:27:19.991245+0000","last_active":"2024-04-24T23:28:08.055317+0000","last_peered":"2024-04-24T23:28:08.055317+0000","last_clean":"2024-04-24T23:28:08.055317+0000","last_became_active":"2024-04-24T23:27:19.990448+0000","last_became_peered":"2024-04-24T23:27:19.990448+0000","last_unstale":"2024-04-24T23:28:08.055317+0000","last_undegraded":"2024-04-24T23:28:08.055317+0000","last_fullsized":"2024-04-24T23:28:08.055317+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:03:21.013005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.047937+0000","last_change":"2024-04-24T23:27:19.977664+0000","last_active":"2024-04-24T23:28:08.047937+0000","last_peered":"2024-04-24T23:28:08.047937+0000","last_clean":"2024-04-24T23:28:08.047937+0000","last_became_active":"2024-04-24T23:27:19.977300+0000","last_became_peered":"2024-04-24T23:27:19.977300+0000","last_unstale":"2024-04-24T23:28:08.047937+0000","last_undegraded":"2024-04-24T23:28:08.047937+0000","last_fullsized":"2024-04-24T23:28:08.047937+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:19:27.229712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.107750+0000","last_change":"2024-04-24T23:27:19.989994+0000","last_active":"2024-04-24T23:28:08.107750+0000","last_peered":"2024-04-24T23:28:08.107750+0000","last_clean":"2024-04-24T23:28:08.107750+0000","last_became_active":"2024-04-24T23:27:19.989639+0000","last_became_peered":"2024-04-24T23:27:19.989639+0000","last_unstale":"2024-04-24T23:28:08.107750+0000","last_undegraded":"2024-04-24T23:28:08.107750+0000","last_fullsized":"2024-04-24T23:28:08.107750+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:20:43.931086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.052802+0000","last_change":"2024-04-24T23:27:19.992886+0000","last_active":"2024-04-24T23:28:08.052802+0000","last_peered":"2024-04-24T23:28:08.052802+0000","last_clean":"2024-04-24T23:28:08.052802+0000","last_became_active":"2024-04-24T23:27:19.992438+0000","last_became_peered":"2024-04-24T23:27:19.992438+0000","last_unstale":"2024-04-24T23:28:08.052802+0000","last_undegraded":"2024-04-24T23:28:08.052802+0000","last_fullsized":"2024-04-24T23:28:08.052802+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:02:34.110984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.051900+0000","last_change":"2024-04-24T23:27:19.981592+0000","last_active":"2024-04-24T23:28:08.051900+0000","last_peered":"2024-04-24T23:28:08.051900+0000","last_clean":"2024-04-24T23:28:08.051900+0000","last_became_active":"2024-04-24T23:27:19.981248+0000","last_became_peered":"2024-04-24T23:27:19.981248+0000","last_unstale":"2024-04-24T23:28:08.051900+0000","last_undegraded":"2024-04-24T23:28:08.051900+0000","last_fullsized":"2024-04-24T23:28:08.051900+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:53:31.191318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.055503+0000","last_change":"2024-04-24T23:27:19.975995+0000","last_active":"2024-04-24T23:28:08.055503+0000","last_peered":"2024-04-24T23:28:08.055503+0000","last_clean":"2024-04-24T23:28:08.055503+0000","last_became_active":"2024-04-24T23:27:19.975614+0000","last_became_peered":"2024-04-24T23:27:19.975614+0000","last_unstale":"2024-04-24T23:28:08.055503+0000","last_undegraded":"2024-04-24T23:28:08.055503+0000","last_fullsized":"2024-04-24T23:28:08.055503+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:33:22.439998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.108161+0000","last_change":"2024-04-24T23:27:19.991265+0000","last_active":"2024-04-24T23:28:08.108161+0000","last_peered":"2024-04-24T23:28:08.108161+0000","last_clean":"2024-04-24T23:28:08.108161+0000","last_became_active":"2024-04-24T23:27:19.990928+0000","last_became_peered":"2024-04-24T23:27:19.990928+0000","last_unstale":"2024-04-24T23:28:08.108161+0000","last_undegraded":"2024-04-24T23:28:08.108161+0000","last_fullsized":"2024-04-24T23:28:08.108161+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:20:41.244669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.053316+0000","last_change":"2024-04-24T23:27:19.985182+0000","last_active":"2024-04-24T23:28:08.053316+0000","last_peered":"2024-04-24T23:28:08.053316+0000","last_clean":"2024-04-24T23:28:08.053316+0000","last_became_active":"2024-04-24T23:27:19.984600+0000","last_became_peered":"2024-04-24T23:27:19.984600+0000","last_unstale":"2024-04-24T23:28:08.053316+0000","last_undegraded":"2024-04-24T23:28:08.053316+0000","last_fullsized":"2024-04-24T23:28:08.053316+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:16:09.919736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.055198+0000","last_change":"2024-04-24T23:27:19.994271+0000","last_active":"2024-04-24T23:28:08.055198+0000","last_peered":"2024-04-24T23:28:08.055198+0000","last_clean":"2024-04-24T23:28:08.055198+0000","last_became_active":"2024-04-24T23:27:19.993926+0000","last_became_peered":"2024-04-24T23:27:19.993926+0000","last_unstale":"2024-04-24T23:28:08.055198+0000","last_undegraded":"2024-04-24T23:28:08.055198+0000","last_fullsized":"2024-04-24T23:28:08.055198+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:49:07.500370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.050492+0000","last_change":"2024-04-24T23:27:19.984230+0000","last_active":"2024-04-24T23:28:08.050492+0000","last_peered":"2024-04-24T23:28:08.050492+0000","last_clean":"2024-04-24T23:28:08.050492+0000","last_became_active":"2024-04-24T23:27:19.983902+0000","last_became_peered":"2024-04-24T23:27:19.983902+0000","last_unstale":"2024-04-24T23:28:08.050492+0000","last_undegraded":"2024-04-24T23:28:08.050492+0000","last_fullsized":"2024-04-24T23:28:08.050492+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:12:40.985092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.049599+0000","last_change":"2024-04-24T23:27:19.979822+0000","last_active":"2024-04-24T23:28:08.049599+0000","last_peered":"2024-04-24T23:28:08.049599+0000","last_clean":"2024-04-24T23:28:08.049599+0000","last_became_active":"2024-04-24T23:27:19.979452+0000","last_became_peered":"2024-04-24T23:27:19.979452+0000","last_unstale":"2024-04-24T23:28:08.049599+0000","last_undegraded":"2024-04-24T23:28:08.049599+0000","last_fullsized":"2024-04-24T23:28:08.049599+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:08:53.945221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.053470+0000","last_change":"2024-04-24T23:28:08.053470+0000","last_active":"2024-04-24T23:28:08.053470+0000","last_peered":"2024-04-24T23:28:08.053470+0000","last_clean":"2024-04-24T23:28:08.053470+0000","last_became_active":"2024-04-24T23:28:08.053064+0000","last_became_peered":"2024-04-24T23:28:08.053064+0000","last_unstale":"2024-04-24T23:28:08.053470+0000","last_undegraded":"2024-04-24T23:28:08.053470+0000","last_fullsized":"2024-04-24T23:28:08.053470+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:13:34.489239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.054396+0000","last_change":"2024-04-24T23:27:19.984346+0000","last_active":"2024-04-24T23:28:08.054396+0000","last_peered":"2024-04-24T23:28:08.054396+0000","last_clean":"2024-04-24T23:28:08.054396+0000","last_became_active":"2024-04-24T23:27:19.983999+0000","last_became_peered":"2024-04-24T23:27:19.983999+0000","last_unstale":"2024-04-24T23:28:08.054396+0000","last_undegraded":"2024-04-24T23:28:08.054396+0000","last_fullsized":"2024-04-24T23:28:08.054396+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:15:35.329454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.052937+0000","last_change":"2024-04-24T23:27:19.979939+0000","last_active":"2024-04-24T23:28:08.052937+0000","last_peered":"2024-04-24T23:28:08.052937+0000","last_clean":"2024-04-24T23:28:08.052937+0000","last_became_active":"2024-04-24T23:27:19.979600+0000","last_became_peered":"2024-04-24T23:27:19.979600+0000","last_unstale":"2024-04-24T23:28:08.052937+0000","last_undegraded":"2024-04-24T23:28:08.052937+0000","last_fullsized":"2024-04-24T23:28:08.052937+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:37:52.250143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.047230+0000","last_change":"2024-04-24T23:27:19.984797+0000","last_active":"2024-04-24T23:28:08.047230+0000","last_peered":"2024-04-24T23:28:08.047230+0000","last_clean":"2024-04-24T23:28:08.047230+0000","last_became_active":"2024-04-24T23:27:19.983881+0000","last_became_peered":"2024-04-24T23:27:19.983881+0000","last_unstale":"2024-04-24T23:28:08.047230+0000","last_undegraded":"2024-04-24T23:28:08.047230+0000","last_fullsized":"2024-04-24T23:28:08.047230+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:17:24.630821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.052732+0000","last_change":"2024-04-24T23:27:19.977558+0000","last_active":"2024-04-24T23:28:08.052732+0000","last_peered":"2024-04-24T23:28:08.052732+0000","last_clean":"2024-04-24T23:28:08.052732+0000","last_became_active":"2024-04-24T23:27:19.977220+0000","last_became_peered":"2024-04-24T23:27:19.977220+0000","last_unstale":"2024-04-24T23:28:08.052732+0000","last_undegraded":"2024-04-24T23:28:08.052732+0000","last_fullsized":"2024-04-24T23:28:08.052732+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:38:49.615455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.050760+0000","last_change":"2024-04-24T23:27:24.234304+0000","last_active":"2024-04-24T23:28:08.050760+0000","last_peered":"2024-04-24T23:28:08.050760+0000","last_clean":"2024-04-24T23:28:08.050760+0000","last_became_active":"2024-04-24T23:27:24.233865+0000","last_became_peered":"2024-04-24T23:27:24.233865+0000","last_unstale":"2024-04-24T23:28:08.050760+0000","last_undegraded":"2024-04-24T23:28:08.050760+0000","last_fullsized":"2024-04-24T23:28:08.050760+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:46:20.853060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'9","reported_seq":55,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.061595+0000","last_change":"2024-04-24T23:28:08.061595+0000","last_active":"2024-04-24T23:28:08.061595+0000","last_peered":"2024-04-24T23:28:08.061595+0000","last_clean":"2024-04-24T23:28:08.061595+0000","last_became_active":"2024-04-24T23:28:08.053973+0000","last_became_peered":"2024-04-24T23:28:08.053973+0000","last_unstale":"2024-04-24T23:28:08.061595+0000","last_undegraded":"2024-04-24T23:28:08.061595+0000","last_fullsized":"2024-04-24T23:28:08.061595+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:57:55.667666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,8],"acting":[9,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.110963+0000","last_change":"2024-04-24T23:28:08.110963+0000","last_active":"2024-04-24T23:28:08.110963+0000","last_peered":"2024-04-24T23:28:08.110963+0000","last_clean":"2024-04-24T23:28:08.110963+0000","last_became_active":"2024-04-24T23:28:08.110633+0000","last_became_peered":"2024-04-24T23:28:08.110633+0000","last_unstale":"2024-04-24T23:28:08.110963+0000","last_undegraded":"2024-04-24T23:28:08.110963+0000","last_fullsized":"2024-04-24T23:28:08.110963+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:43:21.694477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.054431+0000","last_change":"2024-04-24T23:27:24.234134+0000","last_active":"2024-04-24T23:28:08.054431+0000","last_peered":"2024-04-24T23:28:08.054431+0000","last_clean":"2024-04-24T23:28:08.054431+0000","last_became_active":"2024-04-24T23:27:24.233650+0000","last_became_peered":"2024-04-24T23:27:24.233650+0000","last_unstale":"2024-04-24T23:28:08.054431+0000","last_undegraded":"2024-04-24T23:28:08.054431+0000","last_fullsized":"2024-04-24T23:28:08.054431+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:04:54.191186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.052785+0000","last_change":"2024-04-24T23:27:18.969572+0000","last_active":"2024-04-24T23:28:08.052785+0000","last_peered":"2024-04-24T23:28:08.052785+0000","last_clean":"2024-04-24T23:28:08.052785+0000","last_became_active":"2024-04-24T23:27:18.968897+0000","last_became_peered":"2024-04-24T23:27:18.968897+0000","last_unstale":"2024-04-24T23:28:08.052785+0000","last_undegraded":"2024-04-24T23:28:08.052785+0000","last_fullsized":"2024-04-24T23:28:08.052785+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:42:51.380386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.050009+0000","last_change":"2024-04-24T23:27:19.977290+0000","last_active":"2024-04-24T23:28:08.050009+0000","last_peered":"2024-04-24T23:28:08.050009+0000","last_clean":"2024-04-24T23:28:08.050009+0000","last_became_active":"2024-04-24T23:27:19.976802+0000","last_became_peered":"2024-04-24T23:27:19.976802+0000","last_unstale":"2024-04-24T23:28:08.050009+0000","last_undegraded":"2024-04-24T23:28:08.050009+0000","last_fullsized":"2024-04-24T23:28:08.050009+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:55:15.827279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.054314+0000","last_change":"2024-04-24T23:27:24.235706+0000","last_active":"2024-04-24T23:28:08.054314+0000","last_peered":"2024-04-24T23:28:08.054314+0000","last_clean":"2024-04-24T23:28:08.054314+0000","last_became_active":"2024-04-24T23:27:24.235367+0000","last_became_peered":"2024-04-24T23:27:24.235367+0000","last_unstale":"2024-04-24T23:28:08.054314+0000","last_undegraded":"2024-04-24T23:28:08.054314+0000","last_fullsized":"2024-04-24T23:28:08.054314+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:17.444845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.061873+0000","last_change":"2024-04-24T23:28:08.061398+0000","last_active":"2024-04-24T23:28:08.061873+0000","last_peered":"2024-04-24T23:28:08.061873+0000","last_clean":"2024-04-24T23:28:08.061873+0000","last_became_active":"2024-04-24T23:28:08.056810+0000","last_became_peered":"2024-04-24T23:28:08.056810+0000","last_unstale":"2024-04-24T23:28:08.061873+0000","last_undegraded":"2024-04-24T23:28:08.061873+0000","last_fullsized":"2024-04-24T23:28:08.061873+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:10:56.126730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.054033+0000","last_change":"2024-04-24T23:27:19.975222+0000","last_active":"2024-04-24T23:28:08.054033+0000","last_peered":"2024-04-24T23:28:08.054033+0000","last_clean":"2024-04-24T23:28:08.054033+0000","last_became_active":"2024-04-24T23:27:19.974881+0000","last_became_peered":"2024-04-24T23:27:19.974881+0000","last_unstale":"2024-04-24T23:28:08.054033+0000","last_undegraded":"2024-04-24T23:28:08.054033+0000","last_fullsized":"2024-04-24T23:28:08.054033+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:16:17.884748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.054784+0000","last_change":"2024-04-24T23:27:24.232445+0000","last_active":"2024-04-24T23:28:08.054784+0000","last_peered":"2024-04-24T23:28:08.054784+0000","last_clean":"2024-04-24T23:28:08.054784+0000","last_became_active":"2024-04-24T23:27:24.231754+0000","last_became_peered":"2024-04-24T23:27:24.231754+0000","last_unstale":"2024-04-24T23:28:08.054784+0000","last_undegraded":"2024-04-24T23:28:08.054784+0000","last_fullsized":"2024-04-24T23:28:08.054784+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:45:43.630508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.048112+0000","last_change":"2024-04-24T23:27:18.966558+0000","last_active":"2024-04-24T23:28:08.048112+0000","last_peered":"2024-04-24T23:28:08.048112+0000","last_clean":"2024-04-24T23:28:08.048112+0000","last_became_active":"2024-04-24T23:27:18.966212+0000","last_became_peered":"2024-04-24T23:27:18.966212+0000","last_unstale":"2024-04-24T23:28:08.048112+0000","last_undegraded":"2024-04-24T23:28:08.048112+0000","last_fullsized":"2024-04-24T23:28:08.048112+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:24:17.262587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.108560+0000","last_change":"2024-04-24T23:27:19.976811+0000","last_active":"2024-04-24T23:28:08.108560+0000","last_peered":"2024-04-24T23:28:08.108560+0000","last_clean":"2024-04-24T23:28:08.108560+0000","last_became_active":"2024-04-24T23:27:19.976456+0000","last_became_peered":"2024-04-24T23:27:19.976456+0000","last_unstale":"2024-04-24T23:28:08.108560+0000","last_undegraded":"2024-04-24T23:28:08.108560+0000","last_fullsized":"2024-04-24T23:28:08.108560+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:43:13.755833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.052676+0000","last_change":"2024-04-24T23:27:24.232136+0000","last_active":"2024-04-24T23:28:08.052676+0000","last_peered":"2024-04-24T23:28:08.052676+0000","last_clean":"2024-04-24T23:28:08.052676+0000","last_became_active":"2024-04-24T23:27:24.231459+0000","last_became_peered":"2024-04-24T23:27:24.231459+0000","last_unstale":"2024-04-24T23:28:08.052676+0000","last_undegraded":"2024-04-24T23:28:08.052676+0000","last_fullsized":"2024-04-24T23:28:08.052676+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:51:51.114787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'6","reported_seq":111,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:57.092359+0000","last_change":"2024-04-24T23:27:18.975333+0000","last_active":"2024-04-24T23:28:57.092359+0000","last_peered":"2024-04-24T23:28:57.092359+0000","last_clean":"2024-04-24T23:28:57.092359+0000","last_became_active":"2024-04-24T23:27:18.974869+0000","last_became_peered":"2024-04-24T23:27:18.974869+0000","last_unstale":"2024-04-24T23:28:57.092359+0000","last_undegraded":"2024-04-24T23:28:57.092359+0000","last_fullsized":"2024-04-24T23:28:57.092359+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:30:26.758596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.048206+0000","last_change":"2024-04-24T23:27:19.974989+0000","last_active":"2024-04-24T23:28:08.048206+0000","last_peered":"2024-04-24T23:28:08.048206+0000","last_clean":"2024-04-24T23:28:08.048206+0000","last_became_active":"2024-04-24T23:27:19.974312+0000","last_became_peered":"2024-04-24T23:27:19.974312+0000","last_unstale":"2024-04-24T23:28:08.048206+0000","last_undegraded":"2024-04-24T23:28:08.048206+0000","last_fullsized":"2024-04-24T23:28:08.048206+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:16:39.685430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.052142+0000","last_change":"2024-04-24T23:27:24.229404+0000","last_active":"2024-04-24T23:28:08.052142+0000","last_peered":"2024-04-24T23:28:08.052142+0000","last_clean":"2024-04-24T23:28:08.052142+0000","last_became_active":"2024-04-24T23:27:24.228905+0000","last_became_peered":"2024-04-24T23:27:24.228905+0000","last_unstale":"2024-04-24T23:28:08.052142+0000","last_undegraded":"2024-04-24T23:28:08.052142+0000","last_fullsized":"2024-04-24T23:28:08.052142+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:59:25.069173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.107739+0000","last_change":"2024-04-24T23:27:18.971313+0000","last_active":"2024-04-24T23:28:08.107739+0000","last_peered":"2024-04-24T23:28:08.107739+0000","last_clean":"2024-04-24T23:28:08.107739+0000","last_became_active":"2024-04-24T23:27:18.970888+0000","last_became_peered":"2024-04-24T23:27:18.970888+0000","last_unstale":"2024-04-24T23:28:08.107739+0000","last_undegraded":"2024-04-24T23:28:08.107739+0000","last_fullsized":"2024-04-24T23:28:08.107739+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:34:20.118693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.108592+0000","last_change":"2024-04-24T23:27:19.978939+0000","last_active":"2024-04-24T23:28:08.108592+0000","last_peered":"2024-04-24T23:28:08.108592+0000","last_clean":"2024-04-24T23:28:08.108592+0000","last_became_active":"2024-04-24T23:27:19.978556+0000","last_became_peered":"2024-04-24T23:27:19.978556+0000","last_unstale":"2024-04-24T23:28:08.108592+0000","last_undegraded":"2024-04-24T23:28:08.108592+0000","last_fullsized":"2024-04-24T23:28:08.108592+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:28:43.118561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.052805+0000","last_change":"2024-04-24T23:27:24.232384+0000","last_active":"2024-04-24T23:28:08.052805+0000","last_peered":"2024-04-24T23:28:08.052805+0000","last_clean":"2024-04-24T23:28:08.052805+0000","last_became_active":"2024-04-24T23:27:24.231497+0000","last_became_peered":"2024-04-24T23:27:24.231497+0000","last_unstale":"2024-04-24T23:28:08.052805+0000","last_undegraded":"2024-04-24T23:28:08.052805+0000","last_fullsized":"2024-04-24T23:28:08.052805+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:26:54.716821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.049740+0000","last_change":"2024-04-24T23:27:18.968949+0000","last_active":"2024-04-24T23:28:08.049740+0000","last_peered":"2024-04-24T23:28:08.049740+0000","last_clean":"2024-04-24T23:28:08.049740+0000","last_became_active":"2024-04-24T23:27:18.968612+0000","last_became_peered":"2024-04-24T23:27:18.968612+0000","last_unstale":"2024-04-24T23:28:08.049740+0000","last_undegraded":"2024-04-24T23:28:08.049740+0000","last_fullsized":"2024-04-24T23:28:08.049740+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:03:45.908080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.052112+0000","last_change":"2024-04-24T23:27:19.979670+0000","last_active":"2024-04-24T23:28:08.052112+0000","last_peered":"2024-04-24T23:28:08.052112+0000","last_clean":"2024-04-24T23:28:08.052112+0000","last_became_active":"2024-04-24T23:27:19.979277+0000","last_became_peered":"2024-04-24T23:27:19.979277+0000","last_unstale":"2024-04-24T23:28:08.052112+0000","last_undegraded":"2024-04-24T23:28:08.052112+0000","last_fullsized":"2024-04-24T23:28:08.052112+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:05:54.958747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.049717+0000","last_change":"2024-04-24T23:27:24.237085+0000","last_active":"2024-04-24T23:28:08.049717+0000","last_peered":"2024-04-24T23:28:08.049717+0000","last_clean":"2024-04-24T23:28:08.049717+0000","last_became_active":"2024-04-24T23:27:24.236671+0000","last_became_peered":"2024-04-24T23:27:24.236671+0000","last_unstale":"2024-04-24T23:28:08.049717+0000","last_undegraded":"2024-04-24T23:28:08.049717+0000","last_fullsized":"2024-04-24T23:28:08.049717+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:48:38.482374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.108123+0000","last_change":"2024-04-24T23:27:18.974146+0000","last_active":"2024-04-24T23:28:08.108123+0000","last_peered":"2024-04-24T23:28:08.108123+0000","last_clean":"2024-04-24T23:28:08.108123+0000","last_became_active":"2024-04-24T23:27:18.973814+0000","last_became_peered":"2024-04-24T23:27:18.973814+0000","last_unstale":"2024-04-24T23:28:08.108123+0000","last_undegraded":"2024-04-24T23:28:08.108123+0000","last_fullsized":"2024-04-24T23:28:08.108123+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:22:52.409448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.047011+0000","last_change":"2024-04-24T23:27:19.984325+0000","last_active":"2024-04-24T23:28:08.047011+0000","last_peered":"2024-04-24T23:28:08.047011+0000","last_clean":"2024-04-24T23:28:08.047011+0000","last_became_active":"2024-04-24T23:27:19.983918+0000","last_became_peered":"2024-04-24T23:27:19.983918+0000","last_unstale":"2024-04-24T23:28:08.047011+0000","last_undegraded":"2024-04-24T23:28:08.047011+0000","last_fullsized":"2024-04-24T23:28:08.047011+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:24:07.640983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.052334+0000","last_change":"2024-04-24T23:27:24.223561+0000","last_active":"2024-04-24T23:28:08.052334+0000","last_peered":"2024-04-24T23:28:08.052334+0000","last_clean":"2024-04-24T23:28:08.052334+0000","last_became_active":"2024-04-24T23:27:24.223127+0000","last_became_peered":"2024-04-24T23:27:24.223127+0000","last_unstale":"2024-04-24T23:28:08.052334+0000","last_undegraded":"2024-04-24T23:28:08.052334+0000","last_fullsized":"2024-04-24T23:28:08.052334+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:31:28.079633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"90'3","reported_seq":105,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:57.091588+0000","last_change":"2024-04-24T23:27:18.971374+0000","last_active":"2024-04-24T23:28:57.091588+0000","last_peered":"2024-04-24T23:28:57.091588+0000","last_clean":"2024-04-24T23:28:57.091588+0000","last_became_active":"2024-04-24T23:27:18.970818+0000","last_became_peered":"2024-04-24T23:27:18.970818+0000","last_unstale":"2024-04-24T23:28:57.091588+0000","last_undegraded":"2024-04-24T23:28:57.091588+0000","last_fullsized":"2024-04-24T23:28:57.091588+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:12:17.547968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":73,"num_read_kb":73,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.054624+0000","last_change":"2024-04-24T23:27:19.980275+0000","last_active":"2024-04-24T23:28:08.054624+0000","last_peered":"2024-04-24T23:28:08.054624+0000","last_clean":"2024-04-24T23:28:08.054624+0000","last_became_active":"2024-04-24T23:27:19.979925+0000","last_became_peered":"2024-04-24T23:27:19.979925+0000","last_unstale":"2024-04-24T23:28:08.054624+0000","last_undegraded":"2024-04-24T23:28:08.054624+0000","last_fullsized":"2024-04-24T23:28:08.054624+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:21:10.681169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.046914+0000","last_change":"2024-04-24T23:27:24.231389+0000","last_active":"2024-04-24T23:28:08.046914+0000","last_peered":"2024-04-24T23:28:08.046914+0000","last_clean":"2024-04-24T23:28:08.046914+0000","last_became_active":"2024-04-24T23:27:24.230973+0000","last_became_peered":"2024-04-24T23:27:24.230973+0000","last_unstale":"2024-04-24T23:28:08.046914+0000","last_undegraded":"2024-04-24T23:28:08.046914+0000","last_fullsized":"2024-04-24T23:28:08.046914+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:35.943122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.052820+0000","last_change":"2024-04-24T23:27:18.977009+0000","last_active":"2024-04-24T23:28:08.052820+0000","last_peered":"2024-04-24T23:28:08.052820+0000","last_clean":"2024-04-24T23:28:08.052820+0000","last_became_active":"2024-04-24T23:27:18.976667+0000","last_became_peered":"2024-04-24T23:27:18.976667+0000","last_unstale":"2024-04-24T23:28:08.052820+0000","last_undegraded":"2024-04-24T23:28:08.052820+0000","last_fullsized":"2024-04-24T23:28:08.052820+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:19:47.817961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.050104+0000","last_change":"2024-04-24T23:27:19.988589+0000","last_active":"2024-04-24T23:28:08.050104+0000","last_peered":"2024-04-24T23:28:08.050104+0000","last_clean":"2024-04-24T23:28:08.050104+0000","last_became_active":"2024-04-24T23:27:19.988261+0000","last_became_peered":"2024-04-24T23:27:19.988261+0000","last_unstale":"2024-04-24T23:28:08.050104+0000","last_undegraded":"2024-04-24T23:28:08.050104+0000","last_fullsized":"2024-04-24T23:28:08.050104+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:44:15.809776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.055011+0000","last_change":"2024-04-24T23:27:24.231256+0000","last_active":"2024-04-24T23:28:08.055011+0000","last_peered":"2024-04-24T23:28:08.055011+0000","last_clean":"2024-04-24T23:28:08.055011+0000","last_became_active":"2024-04-24T23:27:24.230545+0000","last_became_peered":"2024-04-24T23:27:24.230545+0000","last_unstale":"2024-04-24T23:28:08.055011+0000","last_undegraded":"2024-04-24T23:28:08.055011+0000","last_fullsized":"2024-04-24T23:28:08.055011+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:43:40.460473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.108465+0000","last_change":"2024-04-24T23:27:18.968805+0000","last_active":"2024-04-24T23:28:08.108465+0000","last_peered":"2024-04-24T23:28:08.108465+0000","last_clean":"2024-04-24T23:28:08.108465+0000","last_became_active":"2024-04-24T23:27:18.968423+0000","last_became_peered":"2024-04-24T23:27:18.968423+0000","last_unstale":"2024-04-24T23:28:08.108465+0000","last_undegraded":"2024-04-24T23:28:08.108465+0000","last_fullsized":"2024-04-24T23:28:08.108465+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:14:04.077773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.056901+0000","last_change":"2024-04-24T23:28:08.056901+0000","last_active":"2024-04-24T23:28:08.056901+0000","last_peered":"2024-04-24T23:28:08.056901+0000","last_clean":"2024-04-24T23:28:08.056901+0000","last_became_active":"2024-04-24T23:28:08.056422+0000","last_became_peered":"2024-04-24T23:28:08.056422+0000","last_unstale":"2024-04-24T23:28:08.056901+0000","last_undegraded":"2024-04-24T23:28:08.056901+0000","last_fullsized":"2024-04-24T23:28:08.056901+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:10:01.716479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.046944+0000","last_change":"2024-04-24T23:27:24.225481+0000","last_active":"2024-04-24T23:28:08.046944+0000","last_peered":"2024-04-24T23:28:08.046944+0000","last_clean":"2024-04-24T23:28:08.046944+0000","last_became_active":"2024-04-24T23:27:24.224363+0000","last_became_peered":"2024-04-24T23:27:24.224363+0000","last_unstale":"2024-04-24T23:28:08.046944+0000","last_undegraded":"2024-04-24T23:28:08.046944+0000","last_fullsized":"2024-04-24T23:28:08.046944+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:36:17.715596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"90'30","reported_seq":212,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:58.102734+0000","last_change":"2024-04-24T23:27:18.968264+0000","last_active":"2024-04-24T23:28:58.102734+0000","last_peered":"2024-04-24T23:28:58.102734+0000","last_clean":"2024-04-24T23:28:58.102734+0000","last_became_active":"2024-04-24T23:27:18.967856+0000","last_became_peered":"2024-04-24T23:27:18.967856+0000","last_unstale":"2024-04-24T23:28:58.102734+0000","last_undegraded":"2024-04-24T23:28:58.102734+0000","last_fullsized":"2024-04-24T23:28:58.102734+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:48:44.347254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58871,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":153,"num_read_kb":134,"num_write":31,"num_write_kb":71,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.048638+0000","last_change":"2024-04-24T23:27:19.984719+0000","last_active":"2024-04-24T23:28:08.048638+0000","last_peered":"2024-04-24T23:28:08.048638+0000","last_clean":"2024-04-24T23:28:08.048638+0000","last_became_active":"2024-04-24T23:27:19.984386+0000","last_became_peered":"2024-04-24T23:27:19.984386+0000","last_unstale":"2024-04-24T23:28:08.048638+0000","last_undegraded":"2024-04-24T23:28:08.048638+0000","last_fullsized":"2024-04-24T23:28:08.048638+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:02:13.739759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.108842+0000","last_change":"2024-04-24T23:27:24.230795+0000","last_active":"2024-04-24T23:28:08.108842+0000","last_peered":"2024-04-24T23:28:08.108842+0000","last_clean":"2024-04-24T23:28:08.108842+0000","last_became_active":"2024-04-24T23:27:24.230360+0000","last_became_peered":"2024-04-24T23:27:24.230360+0000","last_unstale":"2024-04-24T23:28:08.108842+0000","last_undegraded":"2024-04-24T23:28:08.108842+0000","last_fullsized":"2024-04-24T23:28:08.108842+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:51:24.875668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.051758+0000","last_change":"2024-04-24T23:27:18.964964+0000","last_active":"2024-04-24T23:28:08.051758+0000","last_peered":"2024-04-24T23:28:08.051758+0000","last_clean":"2024-04-24T23:28:08.051758+0000","last_became_active":"2024-04-24T23:27:18.964632+0000","last_became_peered":"2024-04-24T23:27:18.964632+0000","last_unstale":"2024-04-24T23:28:08.051758+0000","last_undegraded":"2024-04-24T23:28:08.051758+0000","last_fullsized":"2024-04-24T23:28:08.051758+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:52:23.144629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.049966+0000","last_change":"2024-04-24T23:27:19.987258+0000","last_active":"2024-04-24T23:28:08.049966+0000","last_peered":"2024-04-24T23:28:08.049966+0000","last_clean":"2024-04-24T23:28:08.049966+0000","last_became_active":"2024-04-24T23:27:19.986864+0000","last_became_peered":"2024-04-24T23:27:19.986864+0000","last_unstale":"2024-04-24T23:28:08.049966+0000","last_undegraded":"2024-04-24T23:28:08.049966+0000","last_fullsized":"2024-04-24T23:28:08.049966+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:57:40.503506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.051117+0000","last_change":"2024-04-24T23:27:24.223758+0000","last_active":"2024-04-24T23:28:08.051117+0000","last_peered":"2024-04-24T23:28:08.051117+0000","last_clean":"2024-04-24T23:28:08.051117+0000","last_became_active":"2024-04-24T23:27:24.223343+0000","last_became_peered":"2024-04-24T23:27:24.223343+0000","last_unstale":"2024-04-24T23:28:08.051117+0000","last_undegraded":"2024-04-24T23:28:08.051117+0000","last_fullsized":"2024-04-24T23:28:08.051117+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:41:52.620132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.108091+0000","last_change":"2024-04-24T23:27:18.974229+0000","last_active":"2024-04-24T23:28:08.108091+0000","last_peered":"2024-04-24T23:28:08.108091+0000","last_clean":"2024-04-24T23:28:08.108091+0000","last_became_active":"2024-04-24T23:27:18.973893+0000","last_became_peered":"2024-04-24T23:27:18.973893+0000","last_unstale":"2024-04-24T23:28:08.108091+0000","last_undegraded":"2024-04-24T23:28:08.108091+0000","last_fullsized":"2024-04-24T23:28:08.108091+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:41:45.699709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.053223+0000","last_change":"2024-04-24T23:27:19.986910+0000","last_active":"2024-04-24T23:28:08.053223+0000","last_peered":"2024-04-24T23:28:08.053223+0000","last_clean":"2024-04-24T23:28:08.053223+0000","last_became_active":"2024-04-24T23:27:19.986556+0000","last_became_peered":"2024-04-24T23:27:19.986556+0000","last_unstale":"2024-04-24T23:28:08.053223+0000","last_undegraded":"2024-04-24T23:28:08.053223+0000","last_fullsized":"2024-04-24T23:28:08.053223+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:06:28.506556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.047275+0000","last_change":"2024-04-24T23:27:24.233115+0000","last_active":"2024-04-24T23:28:08.047275+0000","last_peered":"2024-04-24T23:28:08.047275+0000","last_clean":"2024-04-24T23:28:08.047275+0000","last_became_active":"2024-04-24T23:27:24.232714+0000","last_became_peered":"2024-04-24T23:27:24.232714+0000","last_unstale":"2024-04-24T23:28:08.047275+0000","last_undegraded":"2024-04-24T23:28:08.047275+0000","last_fullsized":"2024-04-24T23:28:08.047275+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:04.365515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"90'4","reported_seq":104,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:52.084680+0000","last_change":"2024-04-24T23:27:18.966426+0000","last_active":"2024-04-24T23:28:52.084680+0000","last_peered":"2024-04-24T23:28:52.084680+0000","last_clean":"2024-04-24T23:28:52.084680+0000","last_became_active":"2024-04-24T23:27:18.966070+0000","last_became_peered":"2024-04-24T23:27:18.966070+0000","last_unstale":"2024-04-24T23:28:52.084680+0000","last_undegraded":"2024-04-24T23:28:52.084680+0000","last_fullsized":"2024-04-24T23:28:52.084680+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:56:21.679639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":72,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.052719+0000","last_change":"2024-04-24T23:27:19.991829+0000","last_active":"2024-04-24T23:28:08.052719+0000","last_peered":"2024-04-24T23:28:08.052719+0000","last_clean":"2024-04-24T23:28:08.052719+0000","last_became_active":"2024-04-24T23:27:19.991492+0000","last_became_peered":"2024-04-24T23:27:19.991492+0000","last_unstale":"2024-04-24T23:28:08.052719+0000","last_undegraded":"2024-04-24T23:28:08.052719+0000","last_fullsized":"2024-04-24T23:28:08.052719+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:23:45.195217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.050757+0000","last_change":"2024-04-24T23:27:24.229794+0000","last_active":"2024-04-24T23:28:08.050757+0000","last_peered":"2024-04-24T23:28:08.050757+0000","last_clean":"2024-04-24T23:28:08.050757+0000","last_became_active":"2024-04-24T23:27:24.229410+0000","last_became_peered":"2024-04-24T23:27:24.229410+0000","last_unstale":"2024-04-24T23:28:08.050757+0000","last_undegraded":"2024-04-24T23:28:08.050757+0000","last_fullsized":"2024-04-24T23:28:08.050757+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:00:53.188307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":69,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.068516+0000","last_change":"2024-04-24T23:28:08.068516+0000","last_active":"2024-04-24T23:28:08.068516+0000","last_peered":"2024-04-24T23:28:08.068516+0000","last_clean":"2024-04-24T23:28:08.068516+0000","last_became_active":"2024-04-24T23:28:08.056740+0000","last_became_peered":"2024-04-24T23:28:08.056740+0000","last_unstale":"2024-04-24T23:28:08.068516+0000","last_undegraded":"2024-04-24T23:28:08.068516+0000","last_fullsized":"2024-04-24T23:28:08.068516+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:47:35.568377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,1],"acting":[8,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.108769+0000","last_change":"2024-04-24T23:27:19.982608+0000","last_active":"2024-04-24T23:28:08.108769+0000","last_peered":"2024-04-24T23:28:08.108769+0000","last_clean":"2024-04-24T23:28:08.108769+0000","last_became_active":"2024-04-24T23:27:19.982270+0000","last_became_peered":"2024-04-24T23:27:19.982270+0000","last_unstale":"2024-04-24T23:28:08.108769+0000","last_undegraded":"2024-04-24T23:28:08.108769+0000","last_fullsized":"2024-04-24T23:28:08.108769+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:01:10.525164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.107822+0000","last_change":"2024-04-24T23:27:24.236143+0000","last_active":"2024-04-24T23:28:08.107822+0000","last_peered":"2024-04-24T23:28:08.107822+0000","last_clean":"2024-04-24T23:28:08.107822+0000","last_became_active":"2024-04-24T23:27:24.235724+0000","last_became_peered":"2024-04-24T23:27:24.235724+0000","last_unstale":"2024-04-24T23:28:08.107822+0000","last_undegraded":"2024-04-24T23:28:08.107822+0000","last_fullsized":"2024-04-24T23:28:08.107822+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:55:41.317044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.048339+0000","last_change":"2024-04-24T23:27:18.971312+0000","last_active":"2024-04-24T23:28:08.048339+0000","last_peered":"2024-04-24T23:28:08.048339+0000","last_clean":"2024-04-24T23:28:08.048339+0000","last_became_active":"2024-04-24T23:27:18.970929+0000","last_became_peered":"2024-04-24T23:27:18.970929+0000","last_unstale":"2024-04-24T23:28:08.048339+0000","last_undegraded":"2024-04-24T23:28:08.048339+0000","last_fullsized":"2024-04-24T23:28:08.048339+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:54:20.318916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.056700+0000","last_change":"2024-04-24T23:28:08.056700+0000","last_active":"2024-04-24T23:28:08.056700+0000","last_peered":"2024-04-24T23:28:08.056700+0000","last_clean":"2024-04-24T23:28:08.056700+0000","last_became_active":"2024-04-24T23:28:08.056271+0000","last_became_peered":"2024-04-24T23:28:08.056271+0000","last_unstale":"2024-04-24T23:28:08.056700+0000","last_undegraded":"2024-04-24T23:28:08.056700+0000","last_fullsized":"2024-04-24T23:28:08.056700+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:03:34.808028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.048141+0000","last_change":"2024-04-24T23:27:24.718948+0000","last_active":"2024-04-24T23:28:08.048141+0000","last_peered":"2024-04-24T23:28:08.048141+0000","last_clean":"2024-04-24T23:28:08.048141+0000","last_became_active":"2024-04-24T23:27:24.718424+0000","last_became_peered":"2024-04-24T23:27:24.718424+0000","last_unstale":"2024-04-24T23:28:08.048141+0000","last_undegraded":"2024-04-24T23:28:08.048141+0000","last_fullsized":"2024-04-24T23:28:08.048141+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:22:01.984976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.047178+0000","last_change":"2024-04-24T23:27:18.957073+0000","last_active":"2024-04-24T23:28:08.047178+0000","last_peered":"2024-04-24T23:28:08.047178+0000","last_clean":"2024-04-24T23:28:08.047178+0000","last_became_active":"2024-04-24T23:27:18.956713+0000","last_became_peered":"2024-04-24T23:27:18.956713+0000","last_unstale":"2024-04-24T23:28:08.047178+0000","last_undegraded":"2024-04-24T23:28:08.047178+0000","last_fullsized":"2024-04-24T23:28:08.047178+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:16:21.015902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.108021+0000","last_change":"2024-04-24T23:27:19.985058+0000","last_active":"2024-04-24T23:28:08.108021+0000","last_peered":"2024-04-24T23:28:08.108021+0000","last_clean":"2024-04-24T23:28:08.108021+0000","last_became_active":"2024-04-24T23:27:19.984675+0000","last_became_peered":"2024-04-24T23:27:19.984675+0000","last_unstale":"2024-04-24T23:28:08.108021+0000","last_undegraded":"2024-04-24T23:28:08.108021+0000","last_fullsized":"2024-04-24T23:28:08.108021+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:22:35.252839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.049921+0000","last_change":"2024-04-24T23:27:24.236117+0000","last_active":"2024-04-24T23:28:08.049921+0000","last_peered":"2024-04-24T23:28:08.049921+0000","last_clean":"2024-04-24T23:28:08.049921+0000","last_became_active":"2024-04-24T23:27:24.235720+0000","last_became_peered":"2024-04-24T23:27:24.235720+0000","last_unstale":"2024-04-24T23:28:08.049921+0000","last_undegraded":"2024-04-24T23:28:08.049921+0000","last_fullsized":"2024-04-24T23:28:08.049921+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:46:40.604988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.052828+0000","last_change":"2024-04-24T23:27:18.973690+0000","last_active":"2024-04-24T23:28:08.052828+0000","last_peered":"2024-04-24T23:28:08.052828+0000","last_clean":"2024-04-24T23:28:08.052828+0000","last_became_active":"2024-04-24T23:27:18.973182+0000","last_became_peered":"2024-04-24T23:27:18.973182+0000","last_unstale":"2024-04-24T23:28:08.052828+0000","last_undegraded":"2024-04-24T23:28:08.052828+0000","last_fullsized":"2024-04-24T23:28:08.052828+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:00:44.809880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.107706+0000","last_change":"2024-04-24T23:27:19.991252+0000","last_active":"2024-04-24T23:28:08.107706+0000","last_peered":"2024-04-24T23:28:08.107706+0000","last_clean":"2024-04-24T23:28:08.107706+0000","last_became_active":"2024-04-24T23:27:19.990892+0000","last_became_peered":"2024-04-24T23:27:19.990892+0000","last_unstale":"2024-04-24T23:28:08.107706+0000","last_undegraded":"2024-04-24T23:28:08.107706+0000","last_fullsized":"2024-04-24T23:28:08.107706+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:29:37.835521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.050746+0000","last_change":"2024-04-24T23:27:24.718929+0000","last_active":"2024-04-24T23:28:08.050746+0000","last_peered":"2024-04-24T23:28:08.050746+0000","last_clean":"2024-04-24T23:28:08.050746+0000","last_became_active":"2024-04-24T23:27:24.718486+0000","last_became_peered":"2024-04-24T23:27:24.718486+0000","last_unstale":"2024-04-24T23:28:08.050746+0000","last_undegraded":"2024-04-24T23:28:08.050746+0000","last_fullsized":"2024-04-24T23:28:08.050746+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:31:13.359829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.108257+0000","last_change":"2024-04-24T23:27:19.983181+0000","last_active":"2024-04-24T23:28:08.108257+0000","last_peered":"2024-04-24T23:28:08.108257+0000","last_clean":"2024-04-24T23:28:08.108257+0000","last_became_active":"2024-04-24T23:27:19.981764+0000","last_became_peered":"2024-04-24T23:27:19.981764+0000","last_unstale":"2024-04-24T23:28:08.108257+0000","last_undegraded":"2024-04-24T23:28:08.108257+0000","last_fullsized":"2024-04-24T23:28:08.108257+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:01:20.653572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'4","reported_seq":102,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:53.085340+0000","last_change":"2024-04-24T23:27:18.976351+0000","last_active":"2024-04-24T23:28:53.085340+0000","last_peered":"2024-04-24T23:28:53.085340+0000","last_clean":"2024-04-24T23:28:53.085340+0000","last_became_active":"2024-04-24T23:27:18.975552+0000","last_became_peered":"2024-04-24T23:27:18.975552+0000","last_unstale":"2024-04-24T23:28:53.085340+0000","last_undegraded":"2024-04-24T23:28:53.085340+0000","last_fullsized":"2024-04-24T23:28:53.085340+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:26:29.551439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":69,"num_read_kb":69,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.052960+0000","last_change":"2024-04-24T23:27:24.229528+0000","last_active":"2024-04-24T23:28:08.052960+0000","last_peered":"2024-04-24T23:28:08.052960+0000","last_clean":"2024-04-24T23:28:08.052960+0000","last_became_active":"2024-04-24T23:27:24.229099+0000","last_became_peered":"2024-04-24T23:27:24.229099+0000","last_unstale":"2024-04-24T23:28:08.052960+0000","last_undegraded":"2024-04-24T23:28:08.052960+0000","last_fullsized":"2024-04-24T23:28:08.052960+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:51:51.421912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.052330+0000","last_change":"2024-04-24T23:27:19.980094+0000","last_active":"2024-04-24T23:28:08.052330+0000","last_peered":"2024-04-24T23:28:08.052330+0000","last_clean":"2024-04-24T23:28:08.052330+0000","last_became_active":"2024-04-24T23:27:19.979710+0000","last_became_peered":"2024-04-24T23:27:19.979710+0000","last_unstale":"2024-04-24T23:28:08.052330+0000","last_undegraded":"2024-04-24T23:28:08.052330+0000","last_fullsized":"2024-04-24T23:28:08.052330+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:50:37.920235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.054516+0000","last_change":"2024-04-24T23:27:18.963224+0000","last_active":"2024-04-24T23:28:08.054516+0000","last_peered":"2024-04-24T23:28:08.054516+0000","last_clean":"2024-04-24T23:28:08.054516+0000","last_became_active":"2024-04-24T23:27:18.962794+0000","last_became_peered":"2024-04-24T23:27:18.962794+0000","last_unstale":"2024-04-24T23:28:08.054516+0000","last_undegraded":"2024-04-24T23:28:08.054516+0000","last_fullsized":"2024-04-24T23:28:08.054516+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:05:36.869062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":354,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.052634+0000","last_change":"2024-04-24T23:26:02.307599+0000","last_active":"2024-04-24T23:28:08.052634+0000","last_peered":"2024-04-24T23:28:08.052634+0000","last_clean":"2024-04-24T23:28:08.052634+0000","last_became_active":"2024-04-24T23:26:02.307186+0000","last_became_peered":"2024-04-24T23:26:02.307186+0000","last_unstale":"2024-04-24T23:28:08.052634+0000","last_undegraded":"2024-04-24T23:28:08.052634+0000","last_fullsized":"2024-04-24T23:28:08.052634+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:22:33.824274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:22:33.824274+0000","last_clean_scrub_stamp":"2024-04-24T23:22:33.824274+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:19:26.789955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.108408+0000","last_change":"2024-04-24T23:27:24.719315+0000","last_active":"2024-04-24T23:28:08.108408+0000","last_peered":"2024-04-24T23:28:08.108408+0000","last_clean":"2024-04-24T23:28:08.108408+0000","last_became_active":"2024-04-24T23:27:24.718868+0000","last_became_peered":"2024-04-24T23:27:24.718868+0000","last_unstale":"2024-04-24T23:28:08.108408+0000","last_undegraded":"2024-04-24T23:28:08.108408+0000","last_fullsized":"2024-04-24T23:28:08.108408+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:43:32.122873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.049529+0000","last_change":"2024-04-24T23:27:19.983711+0000","last_active":"2024-04-24T23:28:08.049529+0000","last_peered":"2024-04-24T23:28:08.049529+0000","last_clean":"2024-04-24T23:28:08.049529+0000","last_became_active":"2024-04-24T23:27:19.983302+0000","last_became_peered":"2024-04-24T23:27:19.983302+0000","last_unstale":"2024-04-24T23:28:08.049529+0000","last_undegraded":"2024-04-24T23:28:08.049529+0000","last_fullsized":"2024-04-24T23:28:08.049529+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:44:49.251679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.052711+0000","last_change":"2024-04-24T23:27:18.976766+0000","last_active":"2024-04-24T23:28:08.052711+0000","last_peered":"2024-04-24T23:28:08.052711+0000","last_clean":"2024-04-24T23:28:08.052711+0000","last_became_active":"2024-04-24T23:27:18.976420+0000","last_became_peered":"2024-04-24T23:27:18.976420+0000","last_unstale":"2024-04-24T23:28:08.052711+0000","last_undegraded":"2024-04-24T23:28:08.052711+0000","last_fullsized":"2024-04-24T23:28:08.052711+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:59:34.262774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.050964+0000","last_change":"2024-04-24T23:27:24.226979+0000","last_active":"2024-04-24T23:28:08.050964+0000","last_peered":"2024-04-24T23:28:08.050964+0000","last_clean":"2024-04-24T23:28:08.050964+0000","last_became_active":"2024-04-24T23:27:24.226596+0000","last_became_peered":"2024-04-24T23:27:24.226596+0000","last_unstale":"2024-04-24T23:28:08.050964+0000","last_undegraded":"2024-04-24T23:28:08.050964+0000","last_fullsized":"2024-04-24T23:28:08.050964+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:50:48.585067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.048677+0000","last_change":"2024-04-24T23:27:18.970811+0000","last_active":"2024-04-24T23:28:08.048677+0000","last_peered":"2024-04-24T23:28:08.048677+0000","last_clean":"2024-04-24T23:28:08.048677+0000","last_became_active":"2024-04-24T23:27:18.970292+0000","last_became_peered":"2024-04-24T23:27:18.970292+0000","last_unstale":"2024-04-24T23:28:08.048677+0000","last_undegraded":"2024-04-24T23:28:08.048677+0000","last_fullsized":"2024-04-24T23:28:08.048677+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:06:37.556527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.049751+0000","last_change":"2024-04-24T23:27:19.975455+0000","last_active":"2024-04-24T23:28:08.049751+0000","last_peered":"2024-04-24T23:28:08.049751+0000","last_clean":"2024-04-24T23:28:08.049751+0000","last_became_active":"2024-04-24T23:27:19.973504+0000","last_became_peered":"2024-04-24T23:27:19.973504+0000","last_unstale":"2024-04-24T23:28:08.049751+0000","last_undegraded":"2024-04-24T23:28:08.049751+0000","last_fullsized":"2024-04-24T23:28:08.049751+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:50:34.450370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.052606+0000","last_change":"2024-04-24T23:27:24.719253+0000","last_active":"2024-04-24T23:28:08.052606+0000","last_peered":"2024-04-24T23:28:08.052606+0000","last_clean":"2024-04-24T23:28:08.052606+0000","last_became_active":"2024-04-24T23:27:24.718357+0000","last_became_peered":"2024-04-24T23:27:24.718357+0000","last_unstale":"2024-04-24T23:28:08.052606+0000","last_undegraded":"2024-04-24T23:28:08.052606+0000","last_fullsized":"2024-04-24T23:28:08.052606+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:31:03.798037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'8","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.047387+0000","last_change":"2024-04-24T23:27:18.974310+0000","last_active":"2024-04-24T23:28:08.047387+0000","last_peered":"2024-04-24T23:28:08.047387+0000","last_clean":"2024-04-24T23:28:08.047387+0000","last_became_active":"2024-04-24T23:27:18.973956+0000","last_became_peered":"2024-04-24T23:27:18.973956+0000","last_unstale":"2024-04-24T23:28:08.047387+0000","last_undegraded":"2024-04-24T23:28:08.047387+0000","last_fullsized":"2024-04-24T23:28:08.047387+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:44:27.233155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.052463+0000","last_change":"2024-04-24T23:27:19.985763+0000","last_active":"2024-04-24T23:28:08.052463+0000","last_peered":"2024-04-24T23:28:08.052463+0000","last_clean":"2024-04-24T23:28:08.052463+0000","last_became_active":"2024-04-24T23:27:19.985413+0000","last_became_peered":"2024-04-24T23:27:19.985413+0000","last_unstale":"2024-04-24T23:28:08.052463+0000","last_undegraded":"2024-04-24T23:28:08.052463+0000","last_fullsized":"2024-04-24T23:28:08.052463+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:04:54.910407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.050347+0000","last_change":"2024-04-24T23:27:24.225615+0000","last_active":"2024-04-24T23:28:08.050347+0000","last_peered":"2024-04-24T23:28:08.050347+0000","last_clean":"2024-04-24T23:28:08.050347+0000","last_became_active":"2024-04-24T23:27:24.225029+0000","last_became_peered":"2024-04-24T23:27:24.225029+0000","last_unstale":"2024-04-24T23:28:08.050347+0000","last_undegraded":"2024-04-24T23:28:08.050347+0000","last_fullsized":"2024-04-24T23:28:08.050347+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:56:02.956669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.055506+0000","last_change":"2024-04-24T23:27:18.963235+0000","last_active":"2024-04-24T23:28:08.055506+0000","last_peered":"2024-04-24T23:28:08.055506+0000","last_clean":"2024-04-24T23:28:08.055506+0000","last_became_active":"2024-04-24T23:27:18.962835+0000","last_became_peered":"2024-04-24T23:27:18.962835+0000","last_unstale":"2024-04-24T23:28:08.055506+0000","last_undegraded":"2024-04-24T23:28:08.055506+0000","last_fullsized":"2024-04-24T23:28:08.055506+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:34:29.376856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.049827+0000","last_change":"2024-04-24T23:27:19.987386+0000","last_active":"2024-04-24T23:28:08.049827+0000","last_peered":"2024-04-24T23:28:08.049827+0000","last_clean":"2024-04-24T23:28:08.049827+0000","last_became_active":"2024-04-24T23:27:19.987003+0000","last_became_peered":"2024-04-24T23:27:19.987003+0000","last_unstale":"2024-04-24T23:28:08.049827+0000","last_undegraded":"2024-04-24T23:28:08.049827+0000","last_fullsized":"2024-04-24T23:28:08.049827+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:20:25.635732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.055096+0000","last_change":"2024-04-24T23:27:24.719291+0000","last_active":"2024-04-24T23:28:08.055096+0000","last_peered":"2024-04-24T23:28:08.055096+0000","last_clean":"2024-04-24T23:28:08.055096+0000","last_became_active":"2024-04-24T23:27:24.718796+0000","last_became_peered":"2024-04-24T23:27:24.718796+0000","last_unstale":"2024-04-24T23:28:08.055096+0000","last_undegraded":"2024-04-24T23:28:08.055096+0000","last_fullsized":"2024-04-24T23:28:08.055096+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:20:25.135902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.054510+0000","last_change":"2024-04-24T23:27:18.967856+0000","last_active":"2024-04-24T23:28:08.054510+0000","last_peered":"2024-04-24T23:28:08.054510+0000","last_clean":"2024-04-24T23:28:08.054510+0000","last_became_active":"2024-04-24T23:27:18.967497+0000","last_became_peered":"2024-04-24T23:27:18.967497+0000","last_unstale":"2024-04-24T23:28:08.054510+0000","last_undegraded":"2024-04-24T23:28:08.054510+0000","last_fullsized":"2024-04-24T23:28:08.054510+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:21:29.841791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.047878+0000","last_change":"2024-04-24T23:27:19.981302+0000","last_active":"2024-04-24T23:28:08.047878+0000","last_peered":"2024-04-24T23:28:08.047878+0000","last_clean":"2024-04-24T23:28:08.047878+0000","last_became_active":"2024-04-24T23:27:19.980938+0000","last_became_peered":"2024-04-24T23:27:19.980938+0000","last_unstale":"2024-04-24T23:28:08.047878+0000","last_undegraded":"2024-04-24T23:28:08.047878+0000","last_fullsized":"2024-04-24T23:28:08.047878+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:24:53.939882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.047345+0000","last_change":"2024-04-24T23:27:24.224626+0000","last_active":"2024-04-24T23:28:08.047345+0000","last_peered":"2024-04-24T23:28:08.047345+0000","last_clean":"2024-04-24T23:28:08.047345+0000","last_became_active":"2024-04-24T23:27:24.224175+0000","last_became_peered":"2024-04-24T23:27:24.224175+0000","last_unstale":"2024-04-24T23:28:08.047345+0000","last_undegraded":"2024-04-24T23:28:08.047345+0000","last_fullsized":"2024-04-24T23:28:08.047345+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:54:44.155637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"90'3","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.052252+0000","last_change":"2024-04-24T23:27:18.975662+0000","last_active":"2024-04-24T23:28:08.052252+0000","last_peered":"2024-04-24T23:28:08.052252+0000","last_clean":"2024-04-24T23:28:08.052252+0000","last_became_active":"2024-04-24T23:27:18.975326+0000","last_became_peered":"2024-04-24T23:27:18.975326+0000","last_unstale":"2024-04-24T23:28:08.052252+0000","last_undegraded":"2024-04-24T23:28:08.052252+0000","last_fullsized":"2024-04-24T23:28:08.052252+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:39.440866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.050612+0000","last_change":"2024-04-24T23:27:19.978818+0000","last_active":"2024-04-24T23:28:08.050612+0000","last_peered":"2024-04-24T23:28:08.050612+0000","last_clean":"2024-04-24T23:28:08.050612+0000","last_became_active":"2024-04-24T23:27:19.978107+0000","last_became_peered":"2024-04-24T23:27:19.978107+0000","last_unstale":"2024-04-24T23:28:08.050612+0000","last_undegraded":"2024-04-24T23:28:08.050612+0000","last_fullsized":"2024-04-24T23:28:08.050612+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:22:25.797996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.054570+0000","last_change":"2024-04-24T23:27:24.234504+0000","last_active":"2024-04-24T23:28:08.054570+0000","last_peered":"2024-04-24T23:28:08.054570+0000","last_clean":"2024-04-24T23:28:08.054570+0000","last_became_active":"2024-04-24T23:27:24.234019+0000","last_became_peered":"2024-04-24T23:27:24.234019+0000","last_unstale":"2024-04-24T23:28:08.054570+0000","last_undegraded":"2024-04-24T23:28:08.054570+0000","last_fullsized":"2024-04-24T23:28:08.054570+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:06:19.101515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'1","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.050627+0000","last_change":"2024-04-24T23:27:18.967165+0000","last_active":"2024-04-24T23:28:08.050627+0000","last_peered":"2024-04-24T23:28:08.050627+0000","last_clean":"2024-04-24T23:28:08.050627+0000","last_became_active":"2024-04-24T23:27:18.966628+0000","last_became_peered":"2024-04-24T23:27:18.966628+0000","last_unstale":"2024-04-24T23:28:08.050627+0000","last_undegraded":"2024-04-24T23:28:08.050627+0000","last_fullsized":"2024-04-24T23:28:08.050627+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:12:20.520777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.052544+0000","last_change":"2024-04-24T23:27:19.988069+0000","last_active":"2024-04-24T23:28:08.052544+0000","last_peered":"2024-04-24T23:28:08.052544+0000","last_clean":"2024-04-24T23:28:08.052544+0000","last_became_active":"2024-04-24T23:27:19.987660+0000","last_became_peered":"2024-04-24T23:27:19.987660+0000","last_unstale":"2024-04-24T23:28:08.052544+0000","last_undegraded":"2024-04-24T23:28:08.052544+0000","last_fullsized":"2024-04-24T23:28:08.052544+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:21:03.505313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.054624+0000","last_change":"2024-04-24T23:27:24.719041+0000","last_active":"2024-04-24T23:28:08.054624+0000","last_peered":"2024-04-24T23:28:08.054624+0000","last_clean":"2024-04-24T23:28:08.054624+0000","last_became_active":"2024-04-24T23:27:24.718523+0000","last_became_peered":"2024-04-24T23:27:24.718523+0000","last_unstale":"2024-04-24T23:28:08.054624+0000","last_undegraded":"2024-04-24T23:28:08.054624+0000","last_fullsized":"2024-04-24T23:28:08.054624+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:27:57.396800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.048114+0000","last_change":"2024-04-24T23:27:18.975034+0000","last_active":"2024-04-24T23:28:08.048114+0000","last_peered":"2024-04-24T23:28:08.048114+0000","last_clean":"2024-04-24T23:28:08.048114+0000","last_became_active":"2024-04-24T23:27:18.973764+0000","last_became_peered":"2024-04-24T23:27:18.973764+0000","last_unstale":"2024-04-24T23:28:08.048114+0000","last_undegraded":"2024-04-24T23:28:08.048114+0000","last_fullsized":"2024-04-24T23:28:08.048114+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:43:23.451904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.053061+0000","last_change":"2024-04-24T23:27:19.984948+0000","last_active":"2024-04-24T23:28:08.053061+0000","last_peered":"2024-04-24T23:28:08.053061+0000","last_clean":"2024-04-24T23:28:08.053061+0000","last_became_active":"2024-04-24T23:27:19.984484+0000","last_became_peered":"2024-04-24T23:27:19.984484+0000","last_unstale":"2024-04-24T23:28:08.053061+0000","last_undegraded":"2024-04-24T23:28:08.053061+0000","last_fullsized":"2024-04-24T23:28:08.053061+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:17:43.335983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.053809+0000","last_change":"2024-04-24T23:27:24.231307+0000","last_active":"2024-04-24T23:28:08.053809+0000","last_peered":"2024-04-24T23:28:08.053809+0000","last_clean":"2024-04-24T23:28:08.053809+0000","last_became_active":"2024-04-24T23:27:24.230848+0000","last_became_peered":"2024-04-24T23:27:24.230848+0000","last_unstale":"2024-04-24T23:28:08.053809+0000","last_undegraded":"2024-04-24T23:28:08.053809+0000","last_fullsized":"2024-04-24T23:28:08.053809+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:28.586230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.052444+0000","last_change":"2024-04-24T23:27:18.964232+0000","last_active":"2024-04-24T23:28:08.052444+0000","last_peered":"2024-04-24T23:28:08.052444+0000","last_clean":"2024-04-24T23:28:08.052444+0000","last_became_active":"2024-04-24T23:27:18.963855+0000","last_became_peered":"2024-04-24T23:27:18.963855+0000","last_unstale":"2024-04-24T23:28:08.052444+0000","last_undegraded":"2024-04-24T23:28:08.052444+0000","last_fullsized":"2024-04-24T23:28:08.052444+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:02:21.000326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.053233+0000","last_change":"2024-04-24T23:27:19.984354+0000","last_active":"2024-04-24T23:28:08.053233+0000","last_peered":"2024-04-24T23:28:08.053233+0000","last_clean":"2024-04-24T23:28:08.053233+0000","last_became_active":"2024-04-24T23:27:19.983972+0000","last_became_peered":"2024-04-24T23:27:19.983972+0000","last_unstale":"2024-04-24T23:28:08.053233+0000","last_undegraded":"2024-04-24T23:28:08.053233+0000","last_fullsized":"2024-04-24T23:28:08.053233+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:28:28.390283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.050141+0000","last_change":"2024-04-24T23:27:24.238404+0000","last_active":"2024-04-24T23:28:08.050141+0000","last_peered":"2024-04-24T23:28:08.050141+0000","last_clean":"2024-04-24T23:28:08.050141+0000","last_became_active":"2024-04-24T23:27:24.237976+0000","last_became_peered":"2024-04-24T23:27:24.237976+0000","last_unstale":"2024-04-24T23:28:08.050141+0000","last_undegraded":"2024-04-24T23:28:08.050141+0000","last_fullsized":"2024-04-24T23:28:08.050141+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:19:31.476789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.108574+0000","last_change":"2024-04-24T23:27:18.971259+0000","last_active":"2024-04-24T23:28:08.108574+0000","last_peered":"2024-04-24T23:28:08.108574+0000","last_clean":"2024-04-24T23:28:08.108574+0000","last_became_active":"2024-04-24T23:27:18.970856+0000","last_became_peered":"2024-04-24T23:27:18.970856+0000","last_unstale":"2024-04-24T23:28:08.108574+0000","last_undegraded":"2024-04-24T23:28:08.108574+0000","last_fullsized":"2024-04-24T23:28:08.108574+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:54:58.642528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.053032+0000","last_change":"2024-04-24T23:27:19.986320+0000","last_active":"2024-04-24T23:28:08.053032+0000","last_peered":"2024-04-24T23:28:08.053032+0000","last_clean":"2024-04-24T23:28:08.053032+0000","last_became_active":"2024-04-24T23:27:19.985951+0000","last_became_peered":"2024-04-24T23:27:19.985951+0000","last_unstale":"2024-04-24T23:28:08.053032+0000","last_undegraded":"2024-04-24T23:28:08.053032+0000","last_fullsized":"2024-04-24T23:28:08.053032+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:05:35.427735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.107485+0000","last_change":"2024-04-24T23:27:24.228149+0000","last_active":"2024-04-24T23:28:08.107485+0000","last_peered":"2024-04-24T23:28:08.107485+0000","last_clean":"2024-04-24T23:28:08.107485+0000","last_became_active":"2024-04-24T23:27:24.227683+0000","last_became_peered":"2024-04-24T23:27:24.227683+0000","last_unstale":"2024-04-24T23:28:08.107485+0000","last_undegraded":"2024-04-24T23:28:08.107485+0000","last_fullsized":"2024-04-24T23:28:08.107485+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:55:50.408834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.059374+0000","last_change":"2024-04-24T23:28:08.059374+0000","last_active":"2024-04-24T23:28:08.059374+0000","last_peered":"2024-04-24T23:28:08.059374+0000","last_clean":"2024-04-24T23:28:08.059374+0000","last_became_active":"2024-04-24T23:28:08.056104+0000","last_became_peered":"2024-04-24T23:28:08.056104+0000","last_unstale":"2024-04-24T23:28:08.059374+0000","last_undegraded":"2024-04-24T23:28:08.059374+0000","last_fullsized":"2024-04-24T23:28:08.059374+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:07:17.185948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.052905+0000","last_change":"2024-04-24T23:27:19.990935+0000","last_active":"2024-04-24T23:28:08.052905+0000","last_peered":"2024-04-24T23:28:08.052905+0000","last_clean":"2024-04-24T23:28:08.052905+0000","last_became_active":"2024-04-24T23:27:19.990524+0000","last_became_peered":"2024-04-24T23:27:19.990524+0000","last_unstale":"2024-04-24T23:28:08.052905+0000","last_undegraded":"2024-04-24T23:28:08.052905+0000","last_fullsized":"2024-04-24T23:28:08.052905+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:54:02.463248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.050721+0000","last_change":"2024-04-24T23:27:19.979142+0000","last_active":"2024-04-24T23:28:08.050721+0000","last_peered":"2024-04-24T23:28:08.050721+0000","last_clean":"2024-04-24T23:28:08.050721+0000","last_became_active":"2024-04-24T23:27:19.978799+0000","last_became_peered":"2024-04-24T23:27:19.978799+0000","last_unstale":"2024-04-24T23:28:08.050721+0000","last_undegraded":"2024-04-24T23:28:08.050721+0000","last_fullsized":"2024-04-24T23:28:08.050721+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:07:19.046313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.109820+0000","last_change":"2024-04-24T23:28:08.109820+0000","last_active":"2024-04-24T23:28:08.109820+0000","last_peered":"2024-04-24T23:28:08.109820+0000","last_clean":"2024-04-24T23:28:08.109820+0000","last_became_active":"2024-04-24T23:28:08.109500+0000","last_became_peered":"2024-04-24T23:28:08.109500+0000","last_unstale":"2024-04-24T23:28:08.109820+0000","last_undegraded":"2024-04-24T23:28:08.109820+0000","last_fullsized":"2024-04-24T23:28:08.109820+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:34:32.814256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.050447+0000","last_change":"2024-04-24T23:27:19.980008+0000","last_active":"2024-04-24T23:28:08.050447+0000","last_peered":"2024-04-24T23:28:08.050447+0000","last_clean":"2024-04-24T23:28:08.050447+0000","last_became_active":"2024-04-24T23:27:19.979424+0000","last_became_peered":"2024-04-24T23:27:19.979424+0000","last_unstale":"2024-04-24T23:28:08.050447+0000","last_undegraded":"2024-04-24T23:28:08.050447+0000","last_fullsized":"2024-04-24T23:28:08.050447+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:44.449245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.050318+0000","last_change":"2024-04-24T23:27:19.987574+0000","last_active":"2024-04-24T23:28:08.050318+0000","last_peered":"2024-04-24T23:28:08.050318+0000","last_clean":"2024-04-24T23:28:08.050318+0000","last_became_active":"2024-04-24T23:27:19.987242+0000","last_became_peered":"2024-04-24T23:27:19.987242+0000","last_unstale":"2024-04-24T23:28:08.050318+0000","last_undegraded":"2024-04-24T23:28:08.050318+0000","last_fullsized":"2024-04-24T23:28:08.050318+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:39:10.777152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.049647+0000","last_change":"2024-04-24T23:27:19.980983+0000","last_active":"2024-04-24T23:28:08.049647+0000","last_peered":"2024-04-24T23:28:08.049647+0000","last_clean":"2024-04-24T23:28:08.049647+0000","last_became_active":"2024-04-24T23:27:19.980659+0000","last_became_peered":"2024-04-24T23:27:19.980659+0000","last_unstale":"2024-04-24T23:28:08.049647+0000","last_undegraded":"2024-04-24T23:28:08.049647+0000","last_fullsized":"2024-04-24T23:28:08.049647+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:02:52.647687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.055819+0000","last_change":"2024-04-24T23:28:08.055819+0000","last_active":"2024-04-24T23:28:08.055819+0000","last_peered":"2024-04-24T23:28:08.055819+0000","last_clean":"2024-04-24T23:28:08.055819+0000","last_became_active":"2024-04-24T23:28:08.055386+0000","last_became_peered":"2024-04-24T23:28:08.055386+0000","last_unstale":"2024-04-24T23:28:08.055819+0000","last_undegraded":"2024-04-24T23:28:08.055819+0000","last_fullsized":"2024-04-24T23:28:08.055819+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:59:30.488812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.052507+0000","last_change":"2024-04-24T23:27:19.985864+0000","last_active":"2024-04-24T23:28:08.052507+0000","last_peered":"2024-04-24T23:28:08.052507+0000","last_clean":"2024-04-24T23:28:08.052507+0000","last_became_active":"2024-04-24T23:27:19.984877+0000","last_became_peered":"2024-04-24T23:27:19.984877+0000","last_unstale":"2024-04-24T23:28:08.052507+0000","last_undegraded":"2024-04-24T23:28:08.052507+0000","last_fullsized":"2024-04-24T23:28:08.052507+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:14:43.458970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.052622+0000","last_change":"2024-04-24T23:27:19.988831+0000","last_active":"2024-04-24T23:28:08.052622+0000","last_peered":"2024-04-24T23:28:08.052622+0000","last_clean":"2024-04-24T23:28:08.052622+0000","last_became_active":"2024-04-24T23:27:19.988478+0000","last_became_peered":"2024-04-24T23:27:19.988478+0000","last_unstale":"2024-04-24T23:28:08.052622+0000","last_undegraded":"2024-04-24T23:28:08.052622+0000","last_fullsized":"2024-04-24T23:28:08.052622+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:18:37.488710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.108145+0000","last_change":"2024-04-24T23:27:19.983553+0000","last_active":"2024-04-24T23:28:08.108145+0000","last_peered":"2024-04-24T23:28:08.108145+0000","last_clean":"2024-04-24T23:28:08.108145+0000","last_became_active":"2024-04-24T23:27:19.981042+0000","last_became_peered":"2024-04-24T23:27:19.981042+0000","last_unstale":"2024-04-24T23:28:08.108145+0000","last_undegraded":"2024-04-24T23:28:08.108145+0000","last_fullsized":"2024-04-24T23:28:08.108145+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:09:42.608238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.052172+0000","last_change":"2024-04-24T23:27:19.985425+0000","last_active":"2024-04-24T23:28:08.052172+0000","last_peered":"2024-04-24T23:28:08.052172+0000","last_clean":"2024-04-24T23:28:08.052172+0000","last_became_active":"2024-04-24T23:27:19.985094+0000","last_became_peered":"2024-04-24T23:27:19.985094+0000","last_unstale":"2024-04-24T23:28:08.052172+0000","last_undegraded":"2024-04-24T23:28:08.052172+0000","last_fullsized":"2024-04-24T23:28:08.052172+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:07:15.612247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.054093+0000","last_change":"2024-04-24T23:27:19.988224+0000","last_active":"2024-04-24T23:28:08.054093+0000","last_peered":"2024-04-24T23:28:08.054093+0000","last_clean":"2024-04-24T23:28:08.054093+0000","last_became_active":"2024-04-24T23:27:19.987820+0000","last_became_peered":"2024-04-24T23:27:19.987820+0000","last_unstale":"2024-04-24T23:28:08.054093+0000","last_undegraded":"2024-04-24T23:28:08.054093+0000","last_fullsized":"2024-04-24T23:28:08.054093+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:27.653290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.050499+0000","last_change":"2024-04-24T23:27:19.989672+0000","last_active":"2024-04-24T23:28:08.050499+0000","last_peered":"2024-04-24T23:28:08.050499+0000","last_clean":"2024-04-24T23:28:08.050499+0000","last_became_active":"2024-04-24T23:27:19.989337+0000","last_became_peered":"2024-04-24T23:27:19.989337+0000","last_unstale":"2024-04-24T23:28:08.050499+0000","last_undegraded":"2024-04-24T23:28:08.050499+0000","last_fullsized":"2024-04-24T23:28:08.050499+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:07:13.399366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.051316+0000","last_change":"2024-04-24T23:27:19.987703+0000","last_active":"2024-04-24T23:28:08.051316+0000","last_peered":"2024-04-24T23:28:08.051316+0000","last_clean":"2024-04-24T23:28:08.051316+0000","last_became_active":"2024-04-24T23:27:19.987355+0000","last_became_peered":"2024-04-24T23:27:19.987355+0000","last_unstale":"2024-04-24T23:28:08.051316+0000","last_undegraded":"2024-04-24T23:28:08.051316+0000","last_fullsized":"2024-04-24T23:28:08.051316+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:03:15.812726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":280134,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":63263,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":456,"num_read_kb":296,"num_write":159,"num_write_kb":111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":694,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":189789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":138,"ondisk_log_size":138,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383714,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":1011712,"data_stored":510812,"data_compressed":23648,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645351,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28148,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717004,"statfs":{"total":95995035648,"available":95966212096,"internally_reserved":0,"allocated":1150976,"data_stored":1156760,"data_compressed":59279,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808877,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1028096,"data_stored":514562,"data_compressed":23648,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070514,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1024000,"data_stored":510386,"data_compressed":23648,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332150,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1069056,"data_stored":1106104,"data_compressed":59279,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495676,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32164,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712988,"statfs":{"total":95995035648,"available":95962099712,"internally_reserved":0,"allocated":1003520,"data_stored":505642,"data_compressed":23648,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574933,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36396,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708756,"statfs":{"total":95995035648,"available":95957766144,"internally_reserved":0,"allocated":1077248,"data_stored":564535,"data_compressed":23648,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738457,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1028096,"data_stored":507030,"data_compressed":23648,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411409,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32244,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712908,"statfs":{"total":95995035648,"available":95962017792,"internally_reserved":0,"allocated":1085440,"data_stored":566052,"data_compressed":23648,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182476,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32176,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712976,"statfs":{"total":95995035648,"available":95962087424,"internally_reserved":0,"allocated":1015808,"data_stored":505836,"data_compressed":23648,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920838,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1015808,"data_stored":510683,"data_compressed":23648,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757314,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1081344,"data_stored":1101883,"data_compressed":59279,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33993,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33993,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33993,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35631,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35631,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35631,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35631,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35631,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":58983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":60500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":60840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T23:29:00.829 DEBUG:teuthology.orchestra.run.smithi080:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid ec7885aa-0290-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-24T23:29:01.072 INFO:teuthology.orchestra.run.smithi080.stderr:Inferring config /var/lib/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:29:01.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:29:01 smithi134 ceph-mon[32619]: from='client.15141 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T23:29:01.464 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:29:01 smithi080 ceph-mon[29152]: from='client.15141 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T23:29:01.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:29:01 smithi148 ceph-mon[32528]: from='client.15141 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T23:29:02.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:29:02 smithi134 ceph-mon[32619]: pgmap v349: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T23:29:02.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:29:02 smithi148 ceph-mon[32528]: pgmap v349: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T23:29:02.536 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:29:02 smithi080 ceph-mon[29152]: pgmap v349: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T23:29:02.926 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:02.925+0000 7fb417e06700 1 -- 172.21.15.80:0/1662633435 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb410101800 msgr2=0x7fb410106f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:02.926 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:02.925+0000 7fb417e06700 1 --2- 172.21.15.80:0/1662633435 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb410101800 0x7fb410106f40 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fb4000099c0 tx=0x7fb400009cd0 comp rx=0 tx=0).stop 2024-04-24T23:29:02.926 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:02.925+0000 7fb417e06700 1 -- 172.21.15.80:0/1662633435 shutdown_connections 2024-04-24T23:29:02.926 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:02.925+0000 7fb417e06700 1 --2- 172.21.15.80:0/1662633435 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb410107670 0x7fb41010f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:02.926 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:02.925+0000 7fb417e06700 1 --2- 172.21.15.80:0/1662633435 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb410100e50 0x7fb410101230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:02.926 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:02.925+0000 7fb417e06700 1 --2- 172.21.15.80:0/1662633435 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb410101800 0x7fb410106f40 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:02.926 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:02.925+0000 7fb417e06700 1 -- 172.21.15.80:0/1662633435 >> 172.21.15.80:0/1662633435 conn(0x7fb410076450 msgr2=0x7fb410076860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:02.926 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:02.925+0000 7fb417e06700 1 -- 172.21.15.80:0/1662633435 shutdown_connections 2024-04-24T23:29:02.926 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:02.925+0000 7fb417e06700 1 -- 172.21.15.80:0/1662633435 wait complete. 2024-04-24T23:29:02.928 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:02.925+0000 7fb417e06700 1 Processor -- start 2024-04-24T23:29:02.928 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:02.925+0000 7fb417e06700 1 -- start start 2024-04-24T23:29:02.929 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:02.926+0000 7fb417e06700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb410100e50 0x7fb410101dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:02.929 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:02.926+0000 7fb417e06700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb410101800 0x7fb410102310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:02.929 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:02.926+0000 7fb417e06700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb410107670 0x7fb410102850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:02.929 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:02.926+0000 7fb417e06700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fb410112600 con 0x7fb410101800 2024-04-24T23:29:02.929 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:02.926+0000 7fb417e06700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb410112480 con 0x7fb410107670 2024-04-24T23:29:02.929 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:02.926+0000 7fb417e06700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb410112780 con 0x7fb410100e50 2024-04-24T23:29:02.929 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:02.926+0000 7fb4153a1700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb410101800 0x7fb410102310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:02.929 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:02.926+0000 7fb4153a1700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb410101800 0x7fb410102310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:57180/0 (socket says 172.21.15.80:57180) 2024-04-24T23:29:02.929 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:02.926+0000 7fb4153a1700 1 -- 172.21.15.80:0/3794464725 learned_addr learned my addr 172.21.15.80:0/3794464725 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:29:02.929 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:02.926+0000 7fb4163a3700 1 --2- 172.21.15.80:0/3794464725 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb410107670 0x7fb410102850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:02.929 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:02.926+0000 7fb415ba2700 1 --2- 172.21.15.80:0/3794464725 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb410100e50 0x7fb410101dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:02.929 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:02.926+0000 7fb4153a1700 1 -- 172.21.15.80:0/3794464725 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb410100e50 msgr2=0x7fb410101dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:02.929 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:02.926+0000 7fb4153a1700 1 --2- 172.21.15.80:0/3794464725 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb410100e50 0x7fb410101dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:02.929 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:02.926+0000 7fb4153a1700 1 -- 172.21.15.80:0/3794464725 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb410107670 msgr2=0x7fb410102850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:02.930 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:02.926+0000 7fb4153a1700 1 --2- 172.21.15.80:0/3794464725 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb410107670 0x7fb410102850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:02.930 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:02.926+0000 7fb4153a1700 1 -- 172.21.15.80:0/3794464725 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4101a6ee0 con 0x7fb410101800 2024-04-24T23:29:02.930 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:02.926+0000 7fb415ba2700 1 --2- 172.21.15.80:0/3794464725 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb410100e50 0x7fb410101dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:29:02.930 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:02.926+0000 7fb4153a1700 1 --2- 172.21.15.80:0/3794464725 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb410101800 0x7fb410102310 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fb400006010 tx=0x7fb400009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:02.930 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:02.926+0000 7fb406ffd700 1 -- 172.21.15.80:0/3794464725 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb400027070 con 0x7fb410101800 2024-04-24T23:29:02.930 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:02.926+0000 7fb406ffd700 1 -- 172.21.15.80:0/3794464725 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb400004050 con 0x7fb410101800 2024-04-24T23:29:02.930 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:02.926+0000 7fb417e06700 1 -- 172.21.15.80:0/3794464725 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4101a7170 con 0x7fb410101800 2024-04-24T23:29:02.930 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:02.927+0000 7fb406ffd700 1 -- 172.21.15.80:0/3794464725 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb400022e10 con 0x7fb410101800 2024-04-24T23:29:02.930 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:02.927+0000 7fb417e06700 1 -- 172.21.15.80:0/3794464725 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4101a75c0 con 0x7fb410101800 2024-04-24T23:29:02.930 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:02.928+0000 7fb417e06700 1 -- 172.21.15.80:0/3794464725 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb41010a820 con 0x7fb410101800 2024-04-24T23:29:02.931 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:02.929+0000 7fb406ffd700 1 -- 172.21.15.80:0/3794464725 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fb40002a070 con 0x7fb410101800 2024-04-24T23:29:02.931 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:02.929+0000 7fb406ffd700 1 --2- 172.21.15.80:0/3794464725 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fb3fc0a2cb0 0x7fb3fc0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:02.931 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:02.929+0000 7fb415ba2700 1 --2- 172.21.15.80:0/3794464725 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fb3fc0a2cb0 0x7fb3fc0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:02.931 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:02.929+0000 7fb406ffd700 1 -- 172.21.15.80:0/3794464725 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fb4000d0750 con 0x7fb410101800 2024-04-24T23:29:02.932 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:02.930+0000 7fb415ba2700 1 --2- 172.21.15.80:0/3794464725 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fb3fc0a2cb0 0x7fb3fc0a5170 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb40c005fd0 tx=0x7fb40c005ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:02.933 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:02.933+0000 7fb406ffd700 1 -- 172.21.15.80:0/3794464725 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fb400031070 con 0x7fb410101800 2024-04-24T23:29:03.129 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:03.128+0000 7fb417e06700 1 -- 172.21.15.80:0/3794464725 --> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb41002dc10 con 0x7fb3fc0a2cb0 2024-04-24T23:29:03.138 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:03.137+0000 7fb406ffd700 1 -- 172.21.15.80:0/3794464725 <== mgr.14152 v2:172.21.15.80:6800/3830596922 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340639 (secure 0 0 0) 0x7fb41002dc10 con 0x7fb3fc0a2cb0 2024-04-24T23:29:03.143 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-24T23:29:03.145 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:03.141+0000 7fb417e06700 1 -- 172.21.15.80:0/3794464725 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fb3fc0a2cb0 msgr2=0x7fb3fc0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:03.145 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:03.141+0000 7fb417e06700 1 --2- 172.21.15.80:0/3794464725 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fb3fc0a2cb0 0x7fb3fc0a5170 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb40c005fd0 tx=0x7fb40c005ee0 comp rx=0 tx=0).stop 2024-04-24T23:29:03.145 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:03.141+0000 7fb417e06700 1 -- 172.21.15.80:0/3794464725 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb410101800 msgr2=0x7fb410102310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:03.145 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:03.141+0000 7fb417e06700 1 --2- 172.21.15.80:0/3794464725 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb410101800 0x7fb410102310 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fb400006010 tx=0x7fb400009f80 comp rx=0 tx=0).stop 2024-04-24T23:29:03.145 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:03.141+0000 7fb417e06700 1 -- 172.21.15.80:0/3794464725 shutdown_connections 2024-04-24T23:29:03.145 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:03.141+0000 7fb417e06700 1 --2- 172.21.15.80:0/3794464725 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb410107670 0x7fb410102850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:03.145 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:03.141+0000 7fb417e06700 1 --2- 172.21.15.80:0/3794464725 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb410100e50 0x7fb410101dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:03.145 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:03.141+0000 7fb417e06700 1 --2- 172.21.15.80:0/3794464725 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fb3fc0a2cb0 0x7fb3fc0a5170 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:03.145 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:03.141+0000 7fb417e06700 1 --2- 172.21.15.80:0/3794464725 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb410101800 0x7fb410102310 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:03.146 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:03.141+0000 7fb417e06700 1 -- 172.21.15.80:0/3794464725 >> 172.21.15.80:0/3794464725 conn(0x7fb410076450 msgr2=0x7fb4100feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:03.146 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:03.141+0000 7fb417e06700 1 -- 172.21.15.80:0/3794464725 shutdown_connections 2024-04-24T23:29:03.146 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:03.141+0000 7fb417e06700 1 -- 172.21.15.80:0/3794464725 wait complete. 2024-04-24T23:29:03.146 INFO:teuthology.orchestra.run.smithi080.stderr:dumped all 2024-04-24T23:29:03.882 INFO:teuthology.orchestra.run.smithi080.stdout:{"pg_ready":true,"pg_map":{"version":350,"stamp":"2024-04-24T23:29:02.907720+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":653865,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":592,"num_read_kb":410,"num_write":342,"num_write_kb":3051,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":1181430,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":277,"ondisk_log_size":277,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":365692,"kb_used_data":12296,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124576132,"statfs":{"total":1151940427776,"available":1151565959168,"internally_reserved":0,"allocated":12591104,"data_stored":8060285,"data_compressed":390669,"data_compressed_allocated":528384,"data_compressed_original":2555904,"omap_allocated":19141,"internal_metadata":361674043},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":60,"num_read_kb":30,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003396"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.053673+0000","last_change":"2024-04-24T23:27:19.978807+0000","last_active":"2024-04-24T23:28:08.053673+0000","last_peered":"2024-04-24T23:28:08.053673+0000","last_clean":"2024-04-24T23:28:08.053673+0000","last_became_active":"2024-04-24T23:27:19.978436+0000","last_became_peered":"2024-04-24T23:27:19.978436+0000","last_unstale":"2024-04-24T23:28:08.053673+0000","last_undegraded":"2024-04-24T23:28:08.053673+0000","last_fullsized":"2024-04-24T23:28:08.053673+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:00:13.126518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.049915+0000","last_change":"2024-04-24T23:27:19.980327+0000","last_active":"2024-04-24T23:28:08.049915+0000","last_peered":"2024-04-24T23:28:08.049915+0000","last_clean":"2024-04-24T23:28:08.049915+0000","last_became_active":"2024-04-24T23:27:19.979912+0000","last_became_peered":"2024-04-24T23:27:19.979912+0000","last_unstale":"2024-04-24T23:28:08.049915+0000","last_undegraded":"2024-04-24T23:28:08.049915+0000","last_fullsized":"2024-04-24T23:28:08.049915+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:25:41.657205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.050043+0000","last_change":"2024-04-24T23:27:19.976451+0000","last_active":"2024-04-24T23:28:08.050043+0000","last_peered":"2024-04-24T23:28:08.050043+0000","last_clean":"2024-04-24T23:28:08.050043+0000","last_became_active":"2024-04-24T23:27:19.975976+0000","last_became_peered":"2024-04-24T23:27:19.975976+0000","last_unstale":"2024-04-24T23:28:08.050043+0000","last_undegraded":"2024-04-24T23:28:08.050043+0000","last_fullsized":"2024-04-24T23:28:08.050043+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:14:51.605217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.055317+0000","last_change":"2024-04-24T23:27:19.991245+0000","last_active":"2024-04-24T23:28:08.055317+0000","last_peered":"2024-04-24T23:28:08.055317+0000","last_clean":"2024-04-24T23:28:08.055317+0000","last_became_active":"2024-04-24T23:27:19.990448+0000","last_became_peered":"2024-04-24T23:27:19.990448+0000","last_unstale":"2024-04-24T23:28:08.055317+0000","last_undegraded":"2024-04-24T23:28:08.055317+0000","last_fullsized":"2024-04-24T23:28:08.055317+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:03:21.013005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.047937+0000","last_change":"2024-04-24T23:27:19.977664+0000","last_active":"2024-04-24T23:28:08.047937+0000","last_peered":"2024-04-24T23:28:08.047937+0000","last_clean":"2024-04-24T23:28:08.047937+0000","last_became_active":"2024-04-24T23:27:19.977300+0000","last_became_peered":"2024-04-24T23:27:19.977300+0000","last_unstale":"2024-04-24T23:28:08.047937+0000","last_undegraded":"2024-04-24T23:28:08.047937+0000","last_fullsized":"2024-04-24T23:28:08.047937+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:19:27.229712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.107750+0000","last_change":"2024-04-24T23:27:19.989994+0000","last_active":"2024-04-24T23:28:08.107750+0000","last_peered":"2024-04-24T23:28:08.107750+0000","last_clean":"2024-04-24T23:28:08.107750+0000","last_became_active":"2024-04-24T23:27:19.989639+0000","last_became_peered":"2024-04-24T23:27:19.989639+0000","last_unstale":"2024-04-24T23:28:08.107750+0000","last_undegraded":"2024-04-24T23:28:08.107750+0000","last_fullsized":"2024-04-24T23:28:08.107750+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:20:43.931086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.052802+0000","last_change":"2024-04-24T23:27:19.992886+0000","last_active":"2024-04-24T23:28:08.052802+0000","last_peered":"2024-04-24T23:28:08.052802+0000","last_clean":"2024-04-24T23:28:08.052802+0000","last_became_active":"2024-04-24T23:27:19.992438+0000","last_became_peered":"2024-04-24T23:27:19.992438+0000","last_unstale":"2024-04-24T23:28:08.052802+0000","last_undegraded":"2024-04-24T23:28:08.052802+0000","last_fullsized":"2024-04-24T23:28:08.052802+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:02:34.110984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.051900+0000","last_change":"2024-04-24T23:27:19.981592+0000","last_active":"2024-04-24T23:28:08.051900+0000","last_peered":"2024-04-24T23:28:08.051900+0000","last_clean":"2024-04-24T23:28:08.051900+0000","last_became_active":"2024-04-24T23:27:19.981248+0000","last_became_peered":"2024-04-24T23:27:19.981248+0000","last_unstale":"2024-04-24T23:28:08.051900+0000","last_undegraded":"2024-04-24T23:28:08.051900+0000","last_fullsized":"2024-04-24T23:28:08.051900+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:53:31.191318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.055503+0000","last_change":"2024-04-24T23:27:19.975995+0000","last_active":"2024-04-24T23:28:08.055503+0000","last_peered":"2024-04-24T23:28:08.055503+0000","last_clean":"2024-04-24T23:28:08.055503+0000","last_became_active":"2024-04-24T23:27:19.975614+0000","last_became_peered":"2024-04-24T23:27:19.975614+0000","last_unstale":"2024-04-24T23:28:08.055503+0000","last_undegraded":"2024-04-24T23:28:08.055503+0000","last_fullsized":"2024-04-24T23:28:08.055503+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:33:22.439998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.108161+0000","last_change":"2024-04-24T23:27:19.991265+0000","last_active":"2024-04-24T23:28:08.108161+0000","last_peered":"2024-04-24T23:28:08.108161+0000","last_clean":"2024-04-24T23:28:08.108161+0000","last_became_active":"2024-04-24T23:27:19.990928+0000","last_became_peered":"2024-04-24T23:27:19.990928+0000","last_unstale":"2024-04-24T23:28:08.108161+0000","last_undegraded":"2024-04-24T23:28:08.108161+0000","last_fullsized":"2024-04-24T23:28:08.108161+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:20:41.244669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.053316+0000","last_change":"2024-04-24T23:27:19.985182+0000","last_active":"2024-04-24T23:28:08.053316+0000","last_peered":"2024-04-24T23:28:08.053316+0000","last_clean":"2024-04-24T23:28:08.053316+0000","last_became_active":"2024-04-24T23:27:19.984600+0000","last_became_peered":"2024-04-24T23:27:19.984600+0000","last_unstale":"2024-04-24T23:28:08.053316+0000","last_undegraded":"2024-04-24T23:28:08.053316+0000","last_fullsized":"2024-04-24T23:28:08.053316+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:16:09.919736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.055198+0000","last_change":"2024-04-24T23:27:19.994271+0000","last_active":"2024-04-24T23:28:08.055198+0000","last_peered":"2024-04-24T23:28:08.055198+0000","last_clean":"2024-04-24T23:28:08.055198+0000","last_became_active":"2024-04-24T23:27:19.993926+0000","last_became_peered":"2024-04-24T23:27:19.993926+0000","last_unstale":"2024-04-24T23:28:08.055198+0000","last_undegraded":"2024-04-24T23:28:08.055198+0000","last_fullsized":"2024-04-24T23:28:08.055198+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:49:07.500370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.050492+0000","last_change":"2024-04-24T23:27:19.984230+0000","last_active":"2024-04-24T23:28:08.050492+0000","last_peered":"2024-04-24T23:28:08.050492+0000","last_clean":"2024-04-24T23:28:08.050492+0000","last_became_active":"2024-04-24T23:27:19.983902+0000","last_became_peered":"2024-04-24T23:27:19.983902+0000","last_unstale":"2024-04-24T23:28:08.050492+0000","last_undegraded":"2024-04-24T23:28:08.050492+0000","last_fullsized":"2024-04-24T23:28:08.050492+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:12:40.985092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.049599+0000","last_change":"2024-04-24T23:27:19.979822+0000","last_active":"2024-04-24T23:28:08.049599+0000","last_peered":"2024-04-24T23:28:08.049599+0000","last_clean":"2024-04-24T23:28:08.049599+0000","last_became_active":"2024-04-24T23:27:19.979452+0000","last_became_peered":"2024-04-24T23:27:19.979452+0000","last_unstale":"2024-04-24T23:28:08.049599+0000","last_undegraded":"2024-04-24T23:28:08.049599+0000","last_fullsized":"2024-04-24T23:28:08.049599+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:08:53.945221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.053470+0000","last_change":"2024-04-24T23:28:08.053470+0000","last_active":"2024-04-24T23:28:08.053470+0000","last_peered":"2024-04-24T23:28:08.053470+0000","last_clean":"2024-04-24T23:28:08.053470+0000","last_became_active":"2024-04-24T23:28:08.053064+0000","last_became_peered":"2024-04-24T23:28:08.053064+0000","last_unstale":"2024-04-24T23:28:08.053470+0000","last_undegraded":"2024-04-24T23:28:08.053470+0000","last_fullsized":"2024-04-24T23:28:08.053470+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:13:34.489239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.054396+0000","last_change":"2024-04-24T23:27:19.984346+0000","last_active":"2024-04-24T23:28:08.054396+0000","last_peered":"2024-04-24T23:28:08.054396+0000","last_clean":"2024-04-24T23:28:08.054396+0000","last_became_active":"2024-04-24T23:27:19.983999+0000","last_became_peered":"2024-04-24T23:27:19.983999+0000","last_unstale":"2024-04-24T23:28:08.054396+0000","last_undegraded":"2024-04-24T23:28:08.054396+0000","last_fullsized":"2024-04-24T23:28:08.054396+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:15:35.329454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.052937+0000","last_change":"2024-04-24T23:27:19.979939+0000","last_active":"2024-04-24T23:28:08.052937+0000","last_peered":"2024-04-24T23:28:08.052937+0000","last_clean":"2024-04-24T23:28:08.052937+0000","last_became_active":"2024-04-24T23:27:19.979600+0000","last_became_peered":"2024-04-24T23:27:19.979600+0000","last_unstale":"2024-04-24T23:28:08.052937+0000","last_undegraded":"2024-04-24T23:28:08.052937+0000","last_fullsized":"2024-04-24T23:28:08.052937+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:37:52.250143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.047230+0000","last_change":"2024-04-24T23:27:19.984797+0000","last_active":"2024-04-24T23:28:08.047230+0000","last_peered":"2024-04-24T23:28:08.047230+0000","last_clean":"2024-04-24T23:28:08.047230+0000","last_became_active":"2024-04-24T23:27:19.983881+0000","last_became_peered":"2024-04-24T23:27:19.983881+0000","last_unstale":"2024-04-24T23:28:08.047230+0000","last_undegraded":"2024-04-24T23:28:08.047230+0000","last_fullsized":"2024-04-24T23:28:08.047230+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:17:24.630821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.052732+0000","last_change":"2024-04-24T23:27:19.977558+0000","last_active":"2024-04-24T23:28:08.052732+0000","last_peered":"2024-04-24T23:28:08.052732+0000","last_clean":"2024-04-24T23:28:08.052732+0000","last_became_active":"2024-04-24T23:27:19.977220+0000","last_became_peered":"2024-04-24T23:27:19.977220+0000","last_unstale":"2024-04-24T23:28:08.052732+0000","last_undegraded":"2024-04-24T23:28:08.052732+0000","last_fullsized":"2024-04-24T23:28:08.052732+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:38:49.615455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.050760+0000","last_change":"2024-04-24T23:27:24.234304+0000","last_active":"2024-04-24T23:28:08.050760+0000","last_peered":"2024-04-24T23:28:08.050760+0000","last_clean":"2024-04-24T23:28:08.050760+0000","last_became_active":"2024-04-24T23:27:24.233865+0000","last_became_peered":"2024-04-24T23:27:24.233865+0000","last_unstale":"2024-04-24T23:28:08.050760+0000","last_undegraded":"2024-04-24T23:28:08.050760+0000","last_fullsized":"2024-04-24T23:28:08.050760+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:46:20.853060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'9","reported_seq":55,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.061595+0000","last_change":"2024-04-24T23:28:08.061595+0000","last_active":"2024-04-24T23:28:08.061595+0000","last_peered":"2024-04-24T23:28:08.061595+0000","last_clean":"2024-04-24T23:28:08.061595+0000","last_became_active":"2024-04-24T23:28:08.053973+0000","last_became_peered":"2024-04-24T23:28:08.053973+0000","last_unstale":"2024-04-24T23:28:08.061595+0000","last_undegraded":"2024-04-24T23:28:08.061595+0000","last_fullsized":"2024-04-24T23:28:08.061595+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:57:55.667666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,8],"acting":[9,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.110963+0000","last_change":"2024-04-24T23:28:08.110963+0000","last_active":"2024-04-24T23:28:08.110963+0000","last_peered":"2024-04-24T23:28:08.110963+0000","last_clean":"2024-04-24T23:28:08.110963+0000","last_became_active":"2024-04-24T23:28:08.110633+0000","last_became_peered":"2024-04-24T23:28:08.110633+0000","last_unstale":"2024-04-24T23:28:08.110963+0000","last_undegraded":"2024-04-24T23:28:08.110963+0000","last_fullsized":"2024-04-24T23:28:08.110963+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:43:21.694477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.054431+0000","last_change":"2024-04-24T23:27:24.234134+0000","last_active":"2024-04-24T23:28:08.054431+0000","last_peered":"2024-04-24T23:28:08.054431+0000","last_clean":"2024-04-24T23:28:08.054431+0000","last_became_active":"2024-04-24T23:27:24.233650+0000","last_became_peered":"2024-04-24T23:27:24.233650+0000","last_unstale":"2024-04-24T23:28:08.054431+0000","last_undegraded":"2024-04-24T23:28:08.054431+0000","last_fullsized":"2024-04-24T23:28:08.054431+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:04:54.191186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.052785+0000","last_change":"2024-04-24T23:27:18.969572+0000","last_active":"2024-04-24T23:28:08.052785+0000","last_peered":"2024-04-24T23:28:08.052785+0000","last_clean":"2024-04-24T23:28:08.052785+0000","last_became_active":"2024-04-24T23:27:18.968897+0000","last_became_peered":"2024-04-24T23:27:18.968897+0000","last_unstale":"2024-04-24T23:28:08.052785+0000","last_undegraded":"2024-04-24T23:28:08.052785+0000","last_fullsized":"2024-04-24T23:28:08.052785+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:42:51.380386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.050009+0000","last_change":"2024-04-24T23:27:19.977290+0000","last_active":"2024-04-24T23:28:08.050009+0000","last_peered":"2024-04-24T23:28:08.050009+0000","last_clean":"2024-04-24T23:28:08.050009+0000","last_became_active":"2024-04-24T23:27:19.976802+0000","last_became_peered":"2024-04-24T23:27:19.976802+0000","last_unstale":"2024-04-24T23:28:08.050009+0000","last_undegraded":"2024-04-24T23:28:08.050009+0000","last_fullsized":"2024-04-24T23:28:08.050009+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:55:15.827279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.054314+0000","last_change":"2024-04-24T23:27:24.235706+0000","last_active":"2024-04-24T23:28:08.054314+0000","last_peered":"2024-04-24T23:28:08.054314+0000","last_clean":"2024-04-24T23:28:08.054314+0000","last_became_active":"2024-04-24T23:27:24.235367+0000","last_became_peered":"2024-04-24T23:27:24.235367+0000","last_unstale":"2024-04-24T23:28:08.054314+0000","last_undegraded":"2024-04-24T23:28:08.054314+0000","last_fullsized":"2024-04-24T23:28:08.054314+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:17.444845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.061873+0000","last_change":"2024-04-24T23:28:08.061398+0000","last_active":"2024-04-24T23:28:08.061873+0000","last_peered":"2024-04-24T23:28:08.061873+0000","last_clean":"2024-04-24T23:28:08.061873+0000","last_became_active":"2024-04-24T23:28:08.056810+0000","last_became_peered":"2024-04-24T23:28:08.056810+0000","last_unstale":"2024-04-24T23:28:08.061873+0000","last_undegraded":"2024-04-24T23:28:08.061873+0000","last_fullsized":"2024-04-24T23:28:08.061873+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:10:56.126730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.054033+0000","last_change":"2024-04-24T23:27:19.975222+0000","last_active":"2024-04-24T23:28:08.054033+0000","last_peered":"2024-04-24T23:28:08.054033+0000","last_clean":"2024-04-24T23:28:08.054033+0000","last_became_active":"2024-04-24T23:27:19.974881+0000","last_became_peered":"2024-04-24T23:27:19.974881+0000","last_unstale":"2024-04-24T23:28:08.054033+0000","last_undegraded":"2024-04-24T23:28:08.054033+0000","last_fullsized":"2024-04-24T23:28:08.054033+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:16:17.884748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.054784+0000","last_change":"2024-04-24T23:27:24.232445+0000","last_active":"2024-04-24T23:28:08.054784+0000","last_peered":"2024-04-24T23:28:08.054784+0000","last_clean":"2024-04-24T23:28:08.054784+0000","last_became_active":"2024-04-24T23:27:24.231754+0000","last_became_peered":"2024-04-24T23:27:24.231754+0000","last_unstale":"2024-04-24T23:28:08.054784+0000","last_undegraded":"2024-04-24T23:28:08.054784+0000","last_fullsized":"2024-04-24T23:28:08.054784+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:45:43.630508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.048112+0000","last_change":"2024-04-24T23:27:18.966558+0000","last_active":"2024-04-24T23:28:08.048112+0000","last_peered":"2024-04-24T23:28:08.048112+0000","last_clean":"2024-04-24T23:28:08.048112+0000","last_became_active":"2024-04-24T23:27:18.966212+0000","last_became_peered":"2024-04-24T23:27:18.966212+0000","last_unstale":"2024-04-24T23:28:08.048112+0000","last_undegraded":"2024-04-24T23:28:08.048112+0000","last_fullsized":"2024-04-24T23:28:08.048112+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:24:17.262587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.108560+0000","last_change":"2024-04-24T23:27:19.976811+0000","last_active":"2024-04-24T23:28:08.108560+0000","last_peered":"2024-04-24T23:28:08.108560+0000","last_clean":"2024-04-24T23:28:08.108560+0000","last_became_active":"2024-04-24T23:27:19.976456+0000","last_became_peered":"2024-04-24T23:27:19.976456+0000","last_unstale":"2024-04-24T23:28:08.108560+0000","last_undegraded":"2024-04-24T23:28:08.108560+0000","last_fullsized":"2024-04-24T23:28:08.108560+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:43:13.755833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.052676+0000","last_change":"2024-04-24T23:27:24.232136+0000","last_active":"2024-04-24T23:28:08.052676+0000","last_peered":"2024-04-24T23:28:08.052676+0000","last_clean":"2024-04-24T23:28:08.052676+0000","last_became_active":"2024-04-24T23:27:24.231459+0000","last_became_peered":"2024-04-24T23:27:24.231459+0000","last_unstale":"2024-04-24T23:28:08.052676+0000","last_undegraded":"2024-04-24T23:28:08.052676+0000","last_fullsized":"2024-04-24T23:28:08.052676+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:51:51.114787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'6","reported_seq":111,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:57.092359+0000","last_change":"2024-04-24T23:27:18.975333+0000","last_active":"2024-04-24T23:28:57.092359+0000","last_peered":"2024-04-24T23:28:57.092359+0000","last_clean":"2024-04-24T23:28:57.092359+0000","last_became_active":"2024-04-24T23:27:18.974869+0000","last_became_peered":"2024-04-24T23:27:18.974869+0000","last_unstale":"2024-04-24T23:28:57.092359+0000","last_undegraded":"2024-04-24T23:28:57.092359+0000","last_fullsized":"2024-04-24T23:28:57.092359+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:30:26.758596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.048206+0000","last_change":"2024-04-24T23:27:19.974989+0000","last_active":"2024-04-24T23:28:08.048206+0000","last_peered":"2024-04-24T23:28:08.048206+0000","last_clean":"2024-04-24T23:28:08.048206+0000","last_became_active":"2024-04-24T23:27:19.974312+0000","last_became_peered":"2024-04-24T23:27:19.974312+0000","last_unstale":"2024-04-24T23:28:08.048206+0000","last_undegraded":"2024-04-24T23:28:08.048206+0000","last_fullsized":"2024-04-24T23:28:08.048206+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:16:39.685430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.052142+0000","last_change":"2024-04-24T23:27:24.229404+0000","last_active":"2024-04-24T23:28:08.052142+0000","last_peered":"2024-04-24T23:28:08.052142+0000","last_clean":"2024-04-24T23:28:08.052142+0000","last_became_active":"2024-04-24T23:27:24.228905+0000","last_became_peered":"2024-04-24T23:27:24.228905+0000","last_unstale":"2024-04-24T23:28:08.052142+0000","last_undegraded":"2024-04-24T23:28:08.052142+0000","last_fullsized":"2024-04-24T23:28:08.052142+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:59:25.069173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.107739+0000","last_change":"2024-04-24T23:27:18.971313+0000","last_active":"2024-04-24T23:28:08.107739+0000","last_peered":"2024-04-24T23:28:08.107739+0000","last_clean":"2024-04-24T23:28:08.107739+0000","last_became_active":"2024-04-24T23:27:18.970888+0000","last_became_peered":"2024-04-24T23:27:18.970888+0000","last_unstale":"2024-04-24T23:28:08.107739+0000","last_undegraded":"2024-04-24T23:28:08.107739+0000","last_fullsized":"2024-04-24T23:28:08.107739+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:34:20.118693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.108592+0000","last_change":"2024-04-24T23:27:19.978939+0000","last_active":"2024-04-24T23:28:08.108592+0000","last_peered":"2024-04-24T23:28:08.108592+0000","last_clean":"2024-04-24T23:28:08.108592+0000","last_became_active":"2024-04-24T23:27:19.978556+0000","last_became_peered":"2024-04-24T23:27:19.978556+0000","last_unstale":"2024-04-24T23:28:08.108592+0000","last_undegraded":"2024-04-24T23:28:08.108592+0000","last_fullsized":"2024-04-24T23:28:08.108592+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:28:43.118561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.052805+0000","last_change":"2024-04-24T23:27:24.232384+0000","last_active":"2024-04-24T23:28:08.052805+0000","last_peered":"2024-04-24T23:28:08.052805+0000","last_clean":"2024-04-24T23:28:08.052805+0000","last_became_active":"2024-04-24T23:27:24.231497+0000","last_became_peered":"2024-04-24T23:27:24.231497+0000","last_unstale":"2024-04-24T23:28:08.052805+0000","last_undegraded":"2024-04-24T23:28:08.052805+0000","last_fullsized":"2024-04-24T23:28:08.052805+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:26:54.716821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.049740+0000","last_change":"2024-04-24T23:27:18.968949+0000","last_active":"2024-04-24T23:28:08.049740+0000","last_peered":"2024-04-24T23:28:08.049740+0000","last_clean":"2024-04-24T23:28:08.049740+0000","last_became_active":"2024-04-24T23:27:18.968612+0000","last_became_peered":"2024-04-24T23:27:18.968612+0000","last_unstale":"2024-04-24T23:28:08.049740+0000","last_undegraded":"2024-04-24T23:28:08.049740+0000","last_fullsized":"2024-04-24T23:28:08.049740+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:03:45.908080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.052112+0000","last_change":"2024-04-24T23:27:19.979670+0000","last_active":"2024-04-24T23:28:08.052112+0000","last_peered":"2024-04-24T23:28:08.052112+0000","last_clean":"2024-04-24T23:28:08.052112+0000","last_became_active":"2024-04-24T23:27:19.979277+0000","last_became_peered":"2024-04-24T23:27:19.979277+0000","last_unstale":"2024-04-24T23:28:08.052112+0000","last_undegraded":"2024-04-24T23:28:08.052112+0000","last_fullsized":"2024-04-24T23:28:08.052112+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:05:54.958747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.049717+0000","last_change":"2024-04-24T23:27:24.237085+0000","last_active":"2024-04-24T23:28:08.049717+0000","last_peered":"2024-04-24T23:28:08.049717+0000","last_clean":"2024-04-24T23:28:08.049717+0000","last_became_active":"2024-04-24T23:27:24.236671+0000","last_became_peered":"2024-04-24T23:27:24.236671+0000","last_unstale":"2024-04-24T23:28:08.049717+0000","last_undegraded":"2024-04-24T23:28:08.049717+0000","last_fullsized":"2024-04-24T23:28:08.049717+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:48:38.482374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.108123+0000","last_change":"2024-04-24T23:27:18.974146+0000","last_active":"2024-04-24T23:28:08.108123+0000","last_peered":"2024-04-24T23:28:08.108123+0000","last_clean":"2024-04-24T23:28:08.108123+0000","last_became_active":"2024-04-24T23:27:18.973814+0000","last_became_peered":"2024-04-24T23:27:18.973814+0000","last_unstale":"2024-04-24T23:28:08.108123+0000","last_undegraded":"2024-04-24T23:28:08.108123+0000","last_fullsized":"2024-04-24T23:28:08.108123+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:22:52.409448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.047011+0000","last_change":"2024-04-24T23:27:19.984325+0000","last_active":"2024-04-24T23:28:08.047011+0000","last_peered":"2024-04-24T23:28:08.047011+0000","last_clean":"2024-04-24T23:28:08.047011+0000","last_became_active":"2024-04-24T23:27:19.983918+0000","last_became_peered":"2024-04-24T23:27:19.983918+0000","last_unstale":"2024-04-24T23:28:08.047011+0000","last_undegraded":"2024-04-24T23:28:08.047011+0000","last_fullsized":"2024-04-24T23:28:08.047011+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:24:07.640983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.052334+0000","last_change":"2024-04-24T23:27:24.223561+0000","last_active":"2024-04-24T23:28:08.052334+0000","last_peered":"2024-04-24T23:28:08.052334+0000","last_clean":"2024-04-24T23:28:08.052334+0000","last_became_active":"2024-04-24T23:27:24.223127+0000","last_became_peered":"2024-04-24T23:27:24.223127+0000","last_unstale":"2024-04-24T23:28:08.052334+0000","last_undegraded":"2024-04-24T23:28:08.052334+0000","last_fullsized":"2024-04-24T23:28:08.052334+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:31:28.079633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"90'3","reported_seq":105,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:57.091588+0000","last_change":"2024-04-24T23:27:18.971374+0000","last_active":"2024-04-24T23:28:57.091588+0000","last_peered":"2024-04-24T23:28:57.091588+0000","last_clean":"2024-04-24T23:28:57.091588+0000","last_became_active":"2024-04-24T23:27:18.970818+0000","last_became_peered":"2024-04-24T23:27:18.970818+0000","last_unstale":"2024-04-24T23:28:57.091588+0000","last_undegraded":"2024-04-24T23:28:57.091588+0000","last_fullsized":"2024-04-24T23:28:57.091588+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:12:17.547968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":73,"num_read_kb":73,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.054624+0000","last_change":"2024-04-24T23:27:19.980275+0000","last_active":"2024-04-24T23:28:08.054624+0000","last_peered":"2024-04-24T23:28:08.054624+0000","last_clean":"2024-04-24T23:28:08.054624+0000","last_became_active":"2024-04-24T23:27:19.979925+0000","last_became_peered":"2024-04-24T23:27:19.979925+0000","last_unstale":"2024-04-24T23:28:08.054624+0000","last_undegraded":"2024-04-24T23:28:08.054624+0000","last_fullsized":"2024-04-24T23:28:08.054624+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:21:10.681169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.046914+0000","last_change":"2024-04-24T23:27:24.231389+0000","last_active":"2024-04-24T23:28:08.046914+0000","last_peered":"2024-04-24T23:28:08.046914+0000","last_clean":"2024-04-24T23:28:08.046914+0000","last_became_active":"2024-04-24T23:27:24.230973+0000","last_became_peered":"2024-04-24T23:27:24.230973+0000","last_unstale":"2024-04-24T23:28:08.046914+0000","last_undegraded":"2024-04-24T23:28:08.046914+0000","last_fullsized":"2024-04-24T23:28:08.046914+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:35.943122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.052820+0000","last_change":"2024-04-24T23:27:18.977009+0000","last_active":"2024-04-24T23:28:08.052820+0000","last_peered":"2024-04-24T23:28:08.052820+0000","last_clean":"2024-04-24T23:28:08.052820+0000","last_became_active":"2024-04-24T23:27:18.976667+0000","last_became_peered":"2024-04-24T23:27:18.976667+0000","last_unstale":"2024-04-24T23:28:08.052820+0000","last_undegraded":"2024-04-24T23:28:08.052820+0000","last_fullsized":"2024-04-24T23:28:08.052820+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:19:47.817961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.050104+0000","last_change":"2024-04-24T23:27:19.988589+0000","last_active":"2024-04-24T23:28:08.050104+0000","last_peered":"2024-04-24T23:28:08.050104+0000","last_clean":"2024-04-24T23:28:08.050104+0000","last_became_active":"2024-04-24T23:27:19.988261+0000","last_became_peered":"2024-04-24T23:27:19.988261+0000","last_unstale":"2024-04-24T23:28:08.050104+0000","last_undegraded":"2024-04-24T23:28:08.050104+0000","last_fullsized":"2024-04-24T23:28:08.050104+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:44:15.809776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.055011+0000","last_change":"2024-04-24T23:27:24.231256+0000","last_active":"2024-04-24T23:28:08.055011+0000","last_peered":"2024-04-24T23:28:08.055011+0000","last_clean":"2024-04-24T23:28:08.055011+0000","last_became_active":"2024-04-24T23:27:24.230545+0000","last_became_peered":"2024-04-24T23:27:24.230545+0000","last_unstale":"2024-04-24T23:28:08.055011+0000","last_undegraded":"2024-04-24T23:28:08.055011+0000","last_fullsized":"2024-04-24T23:28:08.055011+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:43:40.460473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.108465+0000","last_change":"2024-04-24T23:27:18.968805+0000","last_active":"2024-04-24T23:28:08.108465+0000","last_peered":"2024-04-24T23:28:08.108465+0000","last_clean":"2024-04-24T23:28:08.108465+0000","last_became_active":"2024-04-24T23:27:18.968423+0000","last_became_peered":"2024-04-24T23:27:18.968423+0000","last_unstale":"2024-04-24T23:28:08.108465+0000","last_undegraded":"2024-04-24T23:28:08.108465+0000","last_fullsized":"2024-04-24T23:28:08.108465+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:14:04.077773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.056901+0000","last_change":"2024-04-24T23:28:08.056901+0000","last_active":"2024-04-24T23:28:08.056901+0000","last_peered":"2024-04-24T23:28:08.056901+0000","last_clean":"2024-04-24T23:28:08.056901+0000","last_became_active":"2024-04-24T23:28:08.056422+0000","last_became_peered":"2024-04-24T23:28:08.056422+0000","last_unstale":"2024-04-24T23:28:08.056901+0000","last_undegraded":"2024-04-24T23:28:08.056901+0000","last_fullsized":"2024-04-24T23:28:08.056901+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:10:01.716479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.046944+0000","last_change":"2024-04-24T23:27:24.225481+0000","last_active":"2024-04-24T23:28:08.046944+0000","last_peered":"2024-04-24T23:28:08.046944+0000","last_clean":"2024-04-24T23:28:08.046944+0000","last_became_active":"2024-04-24T23:27:24.224363+0000","last_became_peered":"2024-04-24T23:27:24.224363+0000","last_unstale":"2024-04-24T23:28:08.046944+0000","last_undegraded":"2024-04-24T23:28:08.046944+0000","last_fullsized":"2024-04-24T23:28:08.046944+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:36:17.715596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"90'30","reported_seq":212,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:58.102734+0000","last_change":"2024-04-24T23:27:18.968264+0000","last_active":"2024-04-24T23:28:58.102734+0000","last_peered":"2024-04-24T23:28:58.102734+0000","last_clean":"2024-04-24T23:28:58.102734+0000","last_became_active":"2024-04-24T23:27:18.967856+0000","last_became_peered":"2024-04-24T23:27:18.967856+0000","last_unstale":"2024-04-24T23:28:58.102734+0000","last_undegraded":"2024-04-24T23:28:58.102734+0000","last_fullsized":"2024-04-24T23:28:58.102734+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:48:44.347254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58871,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":153,"num_read_kb":134,"num_write":31,"num_write_kb":71,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.048638+0000","last_change":"2024-04-24T23:27:19.984719+0000","last_active":"2024-04-24T23:28:08.048638+0000","last_peered":"2024-04-24T23:28:08.048638+0000","last_clean":"2024-04-24T23:28:08.048638+0000","last_became_active":"2024-04-24T23:27:19.984386+0000","last_became_peered":"2024-04-24T23:27:19.984386+0000","last_unstale":"2024-04-24T23:28:08.048638+0000","last_undegraded":"2024-04-24T23:28:08.048638+0000","last_fullsized":"2024-04-24T23:28:08.048638+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:02:13.739759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.108842+0000","last_change":"2024-04-24T23:27:24.230795+0000","last_active":"2024-04-24T23:28:08.108842+0000","last_peered":"2024-04-24T23:28:08.108842+0000","last_clean":"2024-04-24T23:28:08.108842+0000","last_became_active":"2024-04-24T23:27:24.230360+0000","last_became_peered":"2024-04-24T23:27:24.230360+0000","last_unstale":"2024-04-24T23:28:08.108842+0000","last_undegraded":"2024-04-24T23:28:08.108842+0000","last_fullsized":"2024-04-24T23:28:08.108842+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:51:24.875668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.051758+0000","last_change":"2024-04-24T23:27:18.964964+0000","last_active":"2024-04-24T23:28:08.051758+0000","last_peered":"2024-04-24T23:28:08.051758+0000","last_clean":"2024-04-24T23:28:08.051758+0000","last_became_active":"2024-04-24T23:27:18.964632+0000","last_became_peered":"2024-04-24T23:27:18.964632+0000","last_unstale":"2024-04-24T23:28:08.051758+0000","last_undegraded":"2024-04-24T23:28:08.051758+0000","last_fullsized":"2024-04-24T23:28:08.051758+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:52:23.144629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.049966+0000","last_change":"2024-04-24T23:27:19.987258+0000","last_active":"2024-04-24T23:28:08.049966+0000","last_peered":"2024-04-24T23:28:08.049966+0000","last_clean":"2024-04-24T23:28:08.049966+0000","last_became_active":"2024-04-24T23:27:19.986864+0000","last_became_peered":"2024-04-24T23:27:19.986864+0000","last_unstale":"2024-04-24T23:28:08.049966+0000","last_undegraded":"2024-04-24T23:28:08.049966+0000","last_fullsized":"2024-04-24T23:28:08.049966+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:57:40.503506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.051117+0000","last_change":"2024-04-24T23:27:24.223758+0000","last_active":"2024-04-24T23:28:08.051117+0000","last_peered":"2024-04-24T23:28:08.051117+0000","last_clean":"2024-04-24T23:28:08.051117+0000","last_became_active":"2024-04-24T23:27:24.223343+0000","last_became_peered":"2024-04-24T23:27:24.223343+0000","last_unstale":"2024-04-24T23:28:08.051117+0000","last_undegraded":"2024-04-24T23:28:08.051117+0000","last_fullsized":"2024-04-24T23:28:08.051117+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:41:52.620132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.108091+0000","last_change":"2024-04-24T23:27:18.974229+0000","last_active":"2024-04-24T23:28:08.108091+0000","last_peered":"2024-04-24T23:28:08.108091+0000","last_clean":"2024-04-24T23:28:08.108091+0000","last_became_active":"2024-04-24T23:27:18.973893+0000","last_became_peered":"2024-04-24T23:27:18.973893+0000","last_unstale":"2024-04-24T23:28:08.108091+0000","last_undegraded":"2024-04-24T23:28:08.108091+0000","last_fullsized":"2024-04-24T23:28:08.108091+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:41:45.699709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.053223+0000","last_change":"2024-04-24T23:27:19.986910+0000","last_active":"2024-04-24T23:28:08.053223+0000","last_peered":"2024-04-24T23:28:08.053223+0000","last_clean":"2024-04-24T23:28:08.053223+0000","last_became_active":"2024-04-24T23:27:19.986556+0000","last_became_peered":"2024-04-24T23:27:19.986556+0000","last_unstale":"2024-04-24T23:28:08.053223+0000","last_undegraded":"2024-04-24T23:28:08.053223+0000","last_fullsized":"2024-04-24T23:28:08.053223+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:06:28.506556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.047275+0000","last_change":"2024-04-24T23:27:24.233115+0000","last_active":"2024-04-24T23:28:08.047275+0000","last_peered":"2024-04-24T23:28:08.047275+0000","last_clean":"2024-04-24T23:28:08.047275+0000","last_became_active":"2024-04-24T23:27:24.232714+0000","last_became_peered":"2024-04-24T23:27:24.232714+0000","last_unstale":"2024-04-24T23:28:08.047275+0000","last_undegraded":"2024-04-24T23:28:08.047275+0000","last_fullsized":"2024-04-24T23:28:08.047275+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:04.365515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"90'4","reported_seq":109,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:57.091529+0000","last_change":"2024-04-24T23:27:18.966426+0000","last_active":"2024-04-24T23:28:57.091529+0000","last_peered":"2024-04-24T23:28:57.091529+0000","last_clean":"2024-04-24T23:28:57.091529+0000","last_became_active":"2024-04-24T23:27:18.966070+0000","last_became_peered":"2024-04-24T23:27:18.966070+0000","last_unstale":"2024-04-24T23:28:57.091529+0000","last_undegraded":"2024-04-24T23:28:57.091529+0000","last_fullsized":"2024-04-24T23:28:57.091529+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:56:21.679639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":77,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.052719+0000","last_change":"2024-04-24T23:27:19.991829+0000","last_active":"2024-04-24T23:28:08.052719+0000","last_peered":"2024-04-24T23:28:08.052719+0000","last_clean":"2024-04-24T23:28:08.052719+0000","last_became_active":"2024-04-24T23:27:19.991492+0000","last_became_peered":"2024-04-24T23:27:19.991492+0000","last_unstale":"2024-04-24T23:28:08.052719+0000","last_undegraded":"2024-04-24T23:28:08.052719+0000","last_fullsized":"2024-04-24T23:28:08.052719+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:23:45.195217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.050757+0000","last_change":"2024-04-24T23:27:24.229794+0000","last_active":"2024-04-24T23:28:08.050757+0000","last_peered":"2024-04-24T23:28:08.050757+0000","last_clean":"2024-04-24T23:28:08.050757+0000","last_became_active":"2024-04-24T23:27:24.229410+0000","last_became_peered":"2024-04-24T23:27:24.229410+0000","last_unstale":"2024-04-24T23:28:08.050757+0000","last_undegraded":"2024-04-24T23:28:08.050757+0000","last_fullsized":"2024-04-24T23:28:08.050757+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:00:53.188307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":69,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.068516+0000","last_change":"2024-04-24T23:28:08.068516+0000","last_active":"2024-04-24T23:28:08.068516+0000","last_peered":"2024-04-24T23:28:08.068516+0000","last_clean":"2024-04-24T23:28:08.068516+0000","last_became_active":"2024-04-24T23:28:08.056740+0000","last_became_peered":"2024-04-24T23:28:08.056740+0000","last_unstale":"2024-04-24T23:28:08.068516+0000","last_undegraded":"2024-04-24T23:28:08.068516+0000","last_fullsized":"2024-04-24T23:28:08.068516+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:47:35.568377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,1],"acting":[8,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.108769+0000","last_change":"2024-04-24T23:27:19.982608+0000","last_active":"2024-04-24T23:28:08.108769+0000","last_peered":"2024-04-24T23:28:08.108769+0000","last_clean":"2024-04-24T23:28:08.108769+0000","last_became_active":"2024-04-24T23:27:19.982270+0000","last_became_peered":"2024-04-24T23:27:19.982270+0000","last_unstale":"2024-04-24T23:28:08.108769+0000","last_undegraded":"2024-04-24T23:28:08.108769+0000","last_fullsized":"2024-04-24T23:28:08.108769+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:01:10.525164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.107822+0000","last_change":"2024-04-24T23:27:24.236143+0000","last_active":"2024-04-24T23:28:08.107822+0000","last_peered":"2024-04-24T23:28:08.107822+0000","last_clean":"2024-04-24T23:28:08.107822+0000","last_became_active":"2024-04-24T23:27:24.235724+0000","last_became_peered":"2024-04-24T23:27:24.235724+0000","last_unstale":"2024-04-24T23:28:08.107822+0000","last_undegraded":"2024-04-24T23:28:08.107822+0000","last_fullsized":"2024-04-24T23:28:08.107822+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:55:41.317044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.048339+0000","last_change":"2024-04-24T23:27:18.971312+0000","last_active":"2024-04-24T23:28:08.048339+0000","last_peered":"2024-04-24T23:28:08.048339+0000","last_clean":"2024-04-24T23:28:08.048339+0000","last_became_active":"2024-04-24T23:27:18.970929+0000","last_became_peered":"2024-04-24T23:27:18.970929+0000","last_unstale":"2024-04-24T23:28:08.048339+0000","last_undegraded":"2024-04-24T23:28:08.048339+0000","last_fullsized":"2024-04-24T23:28:08.048339+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:54:20.318916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.056700+0000","last_change":"2024-04-24T23:28:08.056700+0000","last_active":"2024-04-24T23:28:08.056700+0000","last_peered":"2024-04-24T23:28:08.056700+0000","last_clean":"2024-04-24T23:28:08.056700+0000","last_became_active":"2024-04-24T23:28:08.056271+0000","last_became_peered":"2024-04-24T23:28:08.056271+0000","last_unstale":"2024-04-24T23:28:08.056700+0000","last_undegraded":"2024-04-24T23:28:08.056700+0000","last_fullsized":"2024-04-24T23:28:08.056700+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:03:34.808028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.048141+0000","last_change":"2024-04-24T23:27:24.718948+0000","last_active":"2024-04-24T23:28:08.048141+0000","last_peered":"2024-04-24T23:28:08.048141+0000","last_clean":"2024-04-24T23:28:08.048141+0000","last_became_active":"2024-04-24T23:27:24.718424+0000","last_became_peered":"2024-04-24T23:27:24.718424+0000","last_unstale":"2024-04-24T23:28:08.048141+0000","last_undegraded":"2024-04-24T23:28:08.048141+0000","last_fullsized":"2024-04-24T23:28:08.048141+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:22:01.984976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.047178+0000","last_change":"2024-04-24T23:27:18.957073+0000","last_active":"2024-04-24T23:28:08.047178+0000","last_peered":"2024-04-24T23:28:08.047178+0000","last_clean":"2024-04-24T23:28:08.047178+0000","last_became_active":"2024-04-24T23:27:18.956713+0000","last_became_peered":"2024-04-24T23:27:18.956713+0000","last_unstale":"2024-04-24T23:28:08.047178+0000","last_undegraded":"2024-04-24T23:28:08.047178+0000","last_fullsized":"2024-04-24T23:28:08.047178+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:16:21.015902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.108021+0000","last_change":"2024-04-24T23:27:19.985058+0000","last_active":"2024-04-24T23:28:08.108021+0000","last_peered":"2024-04-24T23:28:08.108021+0000","last_clean":"2024-04-24T23:28:08.108021+0000","last_became_active":"2024-04-24T23:27:19.984675+0000","last_became_peered":"2024-04-24T23:27:19.984675+0000","last_unstale":"2024-04-24T23:28:08.108021+0000","last_undegraded":"2024-04-24T23:28:08.108021+0000","last_fullsized":"2024-04-24T23:28:08.108021+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:22:35.252839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.049921+0000","last_change":"2024-04-24T23:27:24.236117+0000","last_active":"2024-04-24T23:28:08.049921+0000","last_peered":"2024-04-24T23:28:08.049921+0000","last_clean":"2024-04-24T23:28:08.049921+0000","last_became_active":"2024-04-24T23:27:24.235720+0000","last_became_peered":"2024-04-24T23:27:24.235720+0000","last_unstale":"2024-04-24T23:28:08.049921+0000","last_undegraded":"2024-04-24T23:28:08.049921+0000","last_fullsized":"2024-04-24T23:28:08.049921+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:46:40.604988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.052828+0000","last_change":"2024-04-24T23:27:18.973690+0000","last_active":"2024-04-24T23:28:08.052828+0000","last_peered":"2024-04-24T23:28:08.052828+0000","last_clean":"2024-04-24T23:28:08.052828+0000","last_became_active":"2024-04-24T23:27:18.973182+0000","last_became_peered":"2024-04-24T23:27:18.973182+0000","last_unstale":"2024-04-24T23:28:08.052828+0000","last_undegraded":"2024-04-24T23:28:08.052828+0000","last_fullsized":"2024-04-24T23:28:08.052828+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:00:44.809880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.107706+0000","last_change":"2024-04-24T23:27:19.991252+0000","last_active":"2024-04-24T23:28:08.107706+0000","last_peered":"2024-04-24T23:28:08.107706+0000","last_clean":"2024-04-24T23:28:08.107706+0000","last_became_active":"2024-04-24T23:27:19.990892+0000","last_became_peered":"2024-04-24T23:27:19.990892+0000","last_unstale":"2024-04-24T23:28:08.107706+0000","last_undegraded":"2024-04-24T23:28:08.107706+0000","last_fullsized":"2024-04-24T23:28:08.107706+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:29:37.835521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.050746+0000","last_change":"2024-04-24T23:27:24.718929+0000","last_active":"2024-04-24T23:28:08.050746+0000","last_peered":"2024-04-24T23:28:08.050746+0000","last_clean":"2024-04-24T23:28:08.050746+0000","last_became_active":"2024-04-24T23:27:24.718486+0000","last_became_peered":"2024-04-24T23:27:24.718486+0000","last_unstale":"2024-04-24T23:28:08.050746+0000","last_undegraded":"2024-04-24T23:28:08.050746+0000","last_fullsized":"2024-04-24T23:28:08.050746+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:31:13.359829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.108257+0000","last_change":"2024-04-24T23:27:19.983181+0000","last_active":"2024-04-24T23:28:08.108257+0000","last_peered":"2024-04-24T23:28:08.108257+0000","last_clean":"2024-04-24T23:28:08.108257+0000","last_became_active":"2024-04-24T23:27:19.981764+0000","last_became_peered":"2024-04-24T23:27:19.981764+0000","last_unstale":"2024-04-24T23:28:08.108257+0000","last_undegraded":"2024-04-24T23:28:08.108257+0000","last_fullsized":"2024-04-24T23:28:08.108257+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:01:20.653572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'4","reported_seq":107,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:58.092275+0000","last_change":"2024-04-24T23:27:18.976351+0000","last_active":"2024-04-24T23:28:58.092275+0000","last_peered":"2024-04-24T23:28:58.092275+0000","last_clean":"2024-04-24T23:28:58.092275+0000","last_became_active":"2024-04-24T23:27:18.975552+0000","last_became_peered":"2024-04-24T23:27:18.975552+0000","last_unstale":"2024-04-24T23:28:58.092275+0000","last_undegraded":"2024-04-24T23:28:58.092275+0000","last_fullsized":"2024-04-24T23:28:58.092275+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:26:29.551439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":74,"num_read_kb":74,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.052960+0000","last_change":"2024-04-24T23:27:24.229528+0000","last_active":"2024-04-24T23:28:08.052960+0000","last_peered":"2024-04-24T23:28:08.052960+0000","last_clean":"2024-04-24T23:28:08.052960+0000","last_became_active":"2024-04-24T23:27:24.229099+0000","last_became_peered":"2024-04-24T23:27:24.229099+0000","last_unstale":"2024-04-24T23:28:08.052960+0000","last_undegraded":"2024-04-24T23:28:08.052960+0000","last_fullsized":"2024-04-24T23:28:08.052960+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:51:51.421912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.052330+0000","last_change":"2024-04-24T23:27:19.980094+0000","last_active":"2024-04-24T23:28:08.052330+0000","last_peered":"2024-04-24T23:28:08.052330+0000","last_clean":"2024-04-24T23:28:08.052330+0000","last_became_active":"2024-04-24T23:27:19.979710+0000","last_became_peered":"2024-04-24T23:27:19.979710+0000","last_unstale":"2024-04-24T23:28:08.052330+0000","last_undegraded":"2024-04-24T23:28:08.052330+0000","last_fullsized":"2024-04-24T23:28:08.052330+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:50:37.920235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.054516+0000","last_change":"2024-04-24T23:27:18.963224+0000","last_active":"2024-04-24T23:28:08.054516+0000","last_peered":"2024-04-24T23:28:08.054516+0000","last_clean":"2024-04-24T23:28:08.054516+0000","last_became_active":"2024-04-24T23:27:18.962794+0000","last_became_peered":"2024-04-24T23:27:18.962794+0000","last_unstale":"2024-04-24T23:28:08.054516+0000","last_undegraded":"2024-04-24T23:28:08.054516+0000","last_fullsized":"2024-04-24T23:28:08.054516+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:05:36.869062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":354,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.052634+0000","last_change":"2024-04-24T23:26:02.307599+0000","last_active":"2024-04-24T23:28:08.052634+0000","last_peered":"2024-04-24T23:28:08.052634+0000","last_clean":"2024-04-24T23:28:08.052634+0000","last_became_active":"2024-04-24T23:26:02.307186+0000","last_became_peered":"2024-04-24T23:26:02.307186+0000","last_unstale":"2024-04-24T23:28:08.052634+0000","last_undegraded":"2024-04-24T23:28:08.052634+0000","last_fullsized":"2024-04-24T23:28:08.052634+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:22:33.824274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:22:33.824274+0000","last_clean_scrub_stamp":"2024-04-24T23:22:33.824274+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:19:26.789955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.108408+0000","last_change":"2024-04-24T23:27:24.719315+0000","last_active":"2024-04-24T23:28:08.108408+0000","last_peered":"2024-04-24T23:28:08.108408+0000","last_clean":"2024-04-24T23:28:08.108408+0000","last_became_active":"2024-04-24T23:27:24.718868+0000","last_became_peered":"2024-04-24T23:27:24.718868+0000","last_unstale":"2024-04-24T23:28:08.108408+0000","last_undegraded":"2024-04-24T23:28:08.108408+0000","last_fullsized":"2024-04-24T23:28:08.108408+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:43:32.122873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.049529+0000","last_change":"2024-04-24T23:27:19.983711+0000","last_active":"2024-04-24T23:28:08.049529+0000","last_peered":"2024-04-24T23:28:08.049529+0000","last_clean":"2024-04-24T23:28:08.049529+0000","last_became_active":"2024-04-24T23:27:19.983302+0000","last_became_peered":"2024-04-24T23:27:19.983302+0000","last_unstale":"2024-04-24T23:28:08.049529+0000","last_undegraded":"2024-04-24T23:28:08.049529+0000","last_fullsized":"2024-04-24T23:28:08.049529+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:44:49.251679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.052711+0000","last_change":"2024-04-24T23:27:18.976766+0000","last_active":"2024-04-24T23:28:08.052711+0000","last_peered":"2024-04-24T23:28:08.052711+0000","last_clean":"2024-04-24T23:28:08.052711+0000","last_became_active":"2024-04-24T23:27:18.976420+0000","last_became_peered":"2024-04-24T23:27:18.976420+0000","last_unstale":"2024-04-24T23:28:08.052711+0000","last_undegraded":"2024-04-24T23:28:08.052711+0000","last_fullsized":"2024-04-24T23:28:08.052711+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:59:34.262774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.050964+0000","last_change":"2024-04-24T23:27:24.226979+0000","last_active":"2024-04-24T23:28:08.050964+0000","last_peered":"2024-04-24T23:28:08.050964+0000","last_clean":"2024-04-24T23:28:08.050964+0000","last_became_active":"2024-04-24T23:27:24.226596+0000","last_became_peered":"2024-04-24T23:27:24.226596+0000","last_unstale":"2024-04-24T23:28:08.050964+0000","last_undegraded":"2024-04-24T23:28:08.050964+0000","last_fullsized":"2024-04-24T23:28:08.050964+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:50:48.585067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.048677+0000","last_change":"2024-04-24T23:27:18.970811+0000","last_active":"2024-04-24T23:28:08.048677+0000","last_peered":"2024-04-24T23:28:08.048677+0000","last_clean":"2024-04-24T23:28:08.048677+0000","last_became_active":"2024-04-24T23:27:18.970292+0000","last_became_peered":"2024-04-24T23:27:18.970292+0000","last_unstale":"2024-04-24T23:28:08.048677+0000","last_undegraded":"2024-04-24T23:28:08.048677+0000","last_fullsized":"2024-04-24T23:28:08.048677+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:06:37.556527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.049751+0000","last_change":"2024-04-24T23:27:19.975455+0000","last_active":"2024-04-24T23:28:08.049751+0000","last_peered":"2024-04-24T23:28:08.049751+0000","last_clean":"2024-04-24T23:28:08.049751+0000","last_became_active":"2024-04-24T23:27:19.973504+0000","last_became_peered":"2024-04-24T23:27:19.973504+0000","last_unstale":"2024-04-24T23:28:08.049751+0000","last_undegraded":"2024-04-24T23:28:08.049751+0000","last_fullsized":"2024-04-24T23:28:08.049751+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:50:34.450370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.052606+0000","last_change":"2024-04-24T23:27:24.719253+0000","last_active":"2024-04-24T23:28:08.052606+0000","last_peered":"2024-04-24T23:28:08.052606+0000","last_clean":"2024-04-24T23:28:08.052606+0000","last_became_active":"2024-04-24T23:27:24.718357+0000","last_became_peered":"2024-04-24T23:27:24.718357+0000","last_unstale":"2024-04-24T23:28:08.052606+0000","last_undegraded":"2024-04-24T23:28:08.052606+0000","last_fullsized":"2024-04-24T23:28:08.052606+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:31:03.798037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'8","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.047387+0000","last_change":"2024-04-24T23:27:18.974310+0000","last_active":"2024-04-24T23:28:08.047387+0000","last_peered":"2024-04-24T23:28:08.047387+0000","last_clean":"2024-04-24T23:28:08.047387+0000","last_became_active":"2024-04-24T23:27:18.973956+0000","last_became_peered":"2024-04-24T23:27:18.973956+0000","last_unstale":"2024-04-24T23:28:08.047387+0000","last_undegraded":"2024-04-24T23:28:08.047387+0000","last_fullsized":"2024-04-24T23:28:08.047387+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:44:27.233155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.052463+0000","last_change":"2024-04-24T23:27:19.985763+0000","last_active":"2024-04-24T23:28:08.052463+0000","last_peered":"2024-04-24T23:28:08.052463+0000","last_clean":"2024-04-24T23:28:08.052463+0000","last_became_active":"2024-04-24T23:27:19.985413+0000","last_became_peered":"2024-04-24T23:27:19.985413+0000","last_unstale":"2024-04-24T23:28:08.052463+0000","last_undegraded":"2024-04-24T23:28:08.052463+0000","last_fullsized":"2024-04-24T23:28:08.052463+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:04:54.910407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.050347+0000","last_change":"2024-04-24T23:27:24.225615+0000","last_active":"2024-04-24T23:28:08.050347+0000","last_peered":"2024-04-24T23:28:08.050347+0000","last_clean":"2024-04-24T23:28:08.050347+0000","last_became_active":"2024-04-24T23:27:24.225029+0000","last_became_peered":"2024-04-24T23:27:24.225029+0000","last_unstale":"2024-04-24T23:28:08.050347+0000","last_undegraded":"2024-04-24T23:28:08.050347+0000","last_fullsized":"2024-04-24T23:28:08.050347+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:56:02.956669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.055506+0000","last_change":"2024-04-24T23:27:18.963235+0000","last_active":"2024-04-24T23:28:08.055506+0000","last_peered":"2024-04-24T23:28:08.055506+0000","last_clean":"2024-04-24T23:28:08.055506+0000","last_became_active":"2024-04-24T23:27:18.962835+0000","last_became_peered":"2024-04-24T23:27:18.962835+0000","last_unstale":"2024-04-24T23:28:08.055506+0000","last_undegraded":"2024-04-24T23:28:08.055506+0000","last_fullsized":"2024-04-24T23:28:08.055506+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:34:29.376856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.049827+0000","last_change":"2024-04-24T23:27:19.987386+0000","last_active":"2024-04-24T23:28:08.049827+0000","last_peered":"2024-04-24T23:28:08.049827+0000","last_clean":"2024-04-24T23:28:08.049827+0000","last_became_active":"2024-04-24T23:27:19.987003+0000","last_became_peered":"2024-04-24T23:27:19.987003+0000","last_unstale":"2024-04-24T23:28:08.049827+0000","last_undegraded":"2024-04-24T23:28:08.049827+0000","last_fullsized":"2024-04-24T23:28:08.049827+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:20:25.635732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.055096+0000","last_change":"2024-04-24T23:27:24.719291+0000","last_active":"2024-04-24T23:28:08.055096+0000","last_peered":"2024-04-24T23:28:08.055096+0000","last_clean":"2024-04-24T23:28:08.055096+0000","last_became_active":"2024-04-24T23:27:24.718796+0000","last_became_peered":"2024-04-24T23:27:24.718796+0000","last_unstale":"2024-04-24T23:28:08.055096+0000","last_undegraded":"2024-04-24T23:28:08.055096+0000","last_fullsized":"2024-04-24T23:28:08.055096+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:20:25.135902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.054510+0000","last_change":"2024-04-24T23:27:18.967856+0000","last_active":"2024-04-24T23:28:08.054510+0000","last_peered":"2024-04-24T23:28:08.054510+0000","last_clean":"2024-04-24T23:28:08.054510+0000","last_became_active":"2024-04-24T23:27:18.967497+0000","last_became_peered":"2024-04-24T23:27:18.967497+0000","last_unstale":"2024-04-24T23:28:08.054510+0000","last_undegraded":"2024-04-24T23:28:08.054510+0000","last_fullsized":"2024-04-24T23:28:08.054510+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:21:29.841791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.047878+0000","last_change":"2024-04-24T23:27:19.981302+0000","last_active":"2024-04-24T23:28:08.047878+0000","last_peered":"2024-04-24T23:28:08.047878+0000","last_clean":"2024-04-24T23:28:08.047878+0000","last_became_active":"2024-04-24T23:27:19.980938+0000","last_became_peered":"2024-04-24T23:27:19.980938+0000","last_unstale":"2024-04-24T23:28:08.047878+0000","last_undegraded":"2024-04-24T23:28:08.047878+0000","last_fullsized":"2024-04-24T23:28:08.047878+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:24:53.939882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.047345+0000","last_change":"2024-04-24T23:27:24.224626+0000","last_active":"2024-04-24T23:28:08.047345+0000","last_peered":"2024-04-24T23:28:08.047345+0000","last_clean":"2024-04-24T23:28:08.047345+0000","last_became_active":"2024-04-24T23:27:24.224175+0000","last_became_peered":"2024-04-24T23:27:24.224175+0000","last_unstale":"2024-04-24T23:28:08.047345+0000","last_undegraded":"2024-04-24T23:28:08.047345+0000","last_fullsized":"2024-04-24T23:28:08.047345+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:54:44.155637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"90'3","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.052252+0000","last_change":"2024-04-24T23:27:18.975662+0000","last_active":"2024-04-24T23:28:08.052252+0000","last_peered":"2024-04-24T23:28:08.052252+0000","last_clean":"2024-04-24T23:28:08.052252+0000","last_became_active":"2024-04-24T23:27:18.975326+0000","last_became_peered":"2024-04-24T23:27:18.975326+0000","last_unstale":"2024-04-24T23:28:08.052252+0000","last_undegraded":"2024-04-24T23:28:08.052252+0000","last_fullsized":"2024-04-24T23:28:08.052252+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:39.440866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.050612+0000","last_change":"2024-04-24T23:27:19.978818+0000","last_active":"2024-04-24T23:28:08.050612+0000","last_peered":"2024-04-24T23:28:08.050612+0000","last_clean":"2024-04-24T23:28:08.050612+0000","last_became_active":"2024-04-24T23:27:19.978107+0000","last_became_peered":"2024-04-24T23:27:19.978107+0000","last_unstale":"2024-04-24T23:28:08.050612+0000","last_undegraded":"2024-04-24T23:28:08.050612+0000","last_fullsized":"2024-04-24T23:28:08.050612+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:22:25.797996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.054570+0000","last_change":"2024-04-24T23:27:24.234504+0000","last_active":"2024-04-24T23:28:08.054570+0000","last_peered":"2024-04-24T23:28:08.054570+0000","last_clean":"2024-04-24T23:28:08.054570+0000","last_became_active":"2024-04-24T23:27:24.234019+0000","last_became_peered":"2024-04-24T23:27:24.234019+0000","last_unstale":"2024-04-24T23:28:08.054570+0000","last_undegraded":"2024-04-24T23:28:08.054570+0000","last_fullsized":"2024-04-24T23:28:08.054570+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:06:19.101515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'1","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.050627+0000","last_change":"2024-04-24T23:27:18.967165+0000","last_active":"2024-04-24T23:28:08.050627+0000","last_peered":"2024-04-24T23:28:08.050627+0000","last_clean":"2024-04-24T23:28:08.050627+0000","last_became_active":"2024-04-24T23:27:18.966628+0000","last_became_peered":"2024-04-24T23:27:18.966628+0000","last_unstale":"2024-04-24T23:28:08.050627+0000","last_undegraded":"2024-04-24T23:28:08.050627+0000","last_fullsized":"2024-04-24T23:28:08.050627+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:12:20.520777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.052544+0000","last_change":"2024-04-24T23:27:19.988069+0000","last_active":"2024-04-24T23:28:08.052544+0000","last_peered":"2024-04-24T23:28:08.052544+0000","last_clean":"2024-04-24T23:28:08.052544+0000","last_became_active":"2024-04-24T23:27:19.987660+0000","last_became_peered":"2024-04-24T23:27:19.987660+0000","last_unstale":"2024-04-24T23:28:08.052544+0000","last_undegraded":"2024-04-24T23:28:08.052544+0000","last_fullsized":"2024-04-24T23:28:08.052544+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:21:03.505313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.054624+0000","last_change":"2024-04-24T23:27:24.719041+0000","last_active":"2024-04-24T23:28:08.054624+0000","last_peered":"2024-04-24T23:28:08.054624+0000","last_clean":"2024-04-24T23:28:08.054624+0000","last_became_active":"2024-04-24T23:27:24.718523+0000","last_became_peered":"2024-04-24T23:27:24.718523+0000","last_unstale":"2024-04-24T23:28:08.054624+0000","last_undegraded":"2024-04-24T23:28:08.054624+0000","last_fullsized":"2024-04-24T23:28:08.054624+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:27:57.396800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.048114+0000","last_change":"2024-04-24T23:27:18.975034+0000","last_active":"2024-04-24T23:28:08.048114+0000","last_peered":"2024-04-24T23:28:08.048114+0000","last_clean":"2024-04-24T23:28:08.048114+0000","last_became_active":"2024-04-24T23:27:18.973764+0000","last_became_peered":"2024-04-24T23:27:18.973764+0000","last_unstale":"2024-04-24T23:28:08.048114+0000","last_undegraded":"2024-04-24T23:28:08.048114+0000","last_fullsized":"2024-04-24T23:28:08.048114+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:43:23.451904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.053061+0000","last_change":"2024-04-24T23:27:19.984948+0000","last_active":"2024-04-24T23:28:08.053061+0000","last_peered":"2024-04-24T23:28:08.053061+0000","last_clean":"2024-04-24T23:28:08.053061+0000","last_became_active":"2024-04-24T23:27:19.984484+0000","last_became_peered":"2024-04-24T23:27:19.984484+0000","last_unstale":"2024-04-24T23:28:08.053061+0000","last_undegraded":"2024-04-24T23:28:08.053061+0000","last_fullsized":"2024-04-24T23:28:08.053061+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:17:43.335983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.053809+0000","last_change":"2024-04-24T23:27:24.231307+0000","last_active":"2024-04-24T23:28:08.053809+0000","last_peered":"2024-04-24T23:28:08.053809+0000","last_clean":"2024-04-24T23:28:08.053809+0000","last_became_active":"2024-04-24T23:27:24.230848+0000","last_became_peered":"2024-04-24T23:27:24.230848+0000","last_unstale":"2024-04-24T23:28:08.053809+0000","last_undegraded":"2024-04-24T23:28:08.053809+0000","last_fullsized":"2024-04-24T23:28:08.053809+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:28.586230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.052444+0000","last_change":"2024-04-24T23:27:18.964232+0000","last_active":"2024-04-24T23:28:08.052444+0000","last_peered":"2024-04-24T23:28:08.052444+0000","last_clean":"2024-04-24T23:28:08.052444+0000","last_became_active":"2024-04-24T23:27:18.963855+0000","last_became_peered":"2024-04-24T23:27:18.963855+0000","last_unstale":"2024-04-24T23:28:08.052444+0000","last_undegraded":"2024-04-24T23:28:08.052444+0000","last_fullsized":"2024-04-24T23:28:08.052444+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:02:21.000326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.053233+0000","last_change":"2024-04-24T23:27:19.984354+0000","last_active":"2024-04-24T23:28:08.053233+0000","last_peered":"2024-04-24T23:28:08.053233+0000","last_clean":"2024-04-24T23:28:08.053233+0000","last_became_active":"2024-04-24T23:27:19.983972+0000","last_became_peered":"2024-04-24T23:27:19.983972+0000","last_unstale":"2024-04-24T23:28:08.053233+0000","last_undegraded":"2024-04-24T23:28:08.053233+0000","last_fullsized":"2024-04-24T23:28:08.053233+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:28:28.390283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.050141+0000","last_change":"2024-04-24T23:27:24.238404+0000","last_active":"2024-04-24T23:28:08.050141+0000","last_peered":"2024-04-24T23:28:08.050141+0000","last_clean":"2024-04-24T23:28:08.050141+0000","last_became_active":"2024-04-24T23:27:24.237976+0000","last_became_peered":"2024-04-24T23:27:24.237976+0000","last_unstale":"2024-04-24T23:28:08.050141+0000","last_undegraded":"2024-04-24T23:28:08.050141+0000","last_fullsized":"2024-04-24T23:28:08.050141+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:19:31.476789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.108574+0000","last_change":"2024-04-24T23:27:18.971259+0000","last_active":"2024-04-24T23:28:08.108574+0000","last_peered":"2024-04-24T23:28:08.108574+0000","last_clean":"2024-04-24T23:28:08.108574+0000","last_became_active":"2024-04-24T23:27:18.970856+0000","last_became_peered":"2024-04-24T23:27:18.970856+0000","last_unstale":"2024-04-24T23:28:08.108574+0000","last_undegraded":"2024-04-24T23:28:08.108574+0000","last_fullsized":"2024-04-24T23:28:08.108574+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:54:58.642528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.053032+0000","last_change":"2024-04-24T23:27:19.986320+0000","last_active":"2024-04-24T23:28:08.053032+0000","last_peered":"2024-04-24T23:28:08.053032+0000","last_clean":"2024-04-24T23:28:08.053032+0000","last_became_active":"2024-04-24T23:27:19.985951+0000","last_became_peered":"2024-04-24T23:27:19.985951+0000","last_unstale":"2024-04-24T23:28:08.053032+0000","last_undegraded":"2024-04-24T23:28:08.053032+0000","last_fullsized":"2024-04-24T23:28:08.053032+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:05:35.427735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.107485+0000","last_change":"2024-04-24T23:27:24.228149+0000","last_active":"2024-04-24T23:28:08.107485+0000","last_peered":"2024-04-24T23:28:08.107485+0000","last_clean":"2024-04-24T23:28:08.107485+0000","last_became_active":"2024-04-24T23:27:24.227683+0000","last_became_peered":"2024-04-24T23:27:24.227683+0000","last_unstale":"2024-04-24T23:28:08.107485+0000","last_undegraded":"2024-04-24T23:28:08.107485+0000","last_fullsized":"2024-04-24T23:28:08.107485+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:55:50.408834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.059374+0000","last_change":"2024-04-24T23:28:08.059374+0000","last_active":"2024-04-24T23:28:08.059374+0000","last_peered":"2024-04-24T23:28:08.059374+0000","last_clean":"2024-04-24T23:28:08.059374+0000","last_became_active":"2024-04-24T23:28:08.056104+0000","last_became_peered":"2024-04-24T23:28:08.056104+0000","last_unstale":"2024-04-24T23:28:08.059374+0000","last_undegraded":"2024-04-24T23:28:08.059374+0000","last_fullsized":"2024-04-24T23:28:08.059374+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:07:17.185948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.052905+0000","last_change":"2024-04-24T23:27:19.990935+0000","last_active":"2024-04-24T23:28:08.052905+0000","last_peered":"2024-04-24T23:28:08.052905+0000","last_clean":"2024-04-24T23:28:08.052905+0000","last_became_active":"2024-04-24T23:27:19.990524+0000","last_became_peered":"2024-04-24T23:27:19.990524+0000","last_unstale":"2024-04-24T23:28:08.052905+0000","last_undegraded":"2024-04-24T23:28:08.052905+0000","last_fullsized":"2024-04-24T23:28:08.052905+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:54:02.463248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.050721+0000","last_change":"2024-04-24T23:27:19.979142+0000","last_active":"2024-04-24T23:28:08.050721+0000","last_peered":"2024-04-24T23:28:08.050721+0000","last_clean":"2024-04-24T23:28:08.050721+0000","last_became_active":"2024-04-24T23:27:19.978799+0000","last_became_peered":"2024-04-24T23:27:19.978799+0000","last_unstale":"2024-04-24T23:28:08.050721+0000","last_undegraded":"2024-04-24T23:28:08.050721+0000","last_fullsized":"2024-04-24T23:28:08.050721+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:07:19.046313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.109820+0000","last_change":"2024-04-24T23:28:08.109820+0000","last_active":"2024-04-24T23:28:08.109820+0000","last_peered":"2024-04-24T23:28:08.109820+0000","last_clean":"2024-04-24T23:28:08.109820+0000","last_became_active":"2024-04-24T23:28:08.109500+0000","last_became_peered":"2024-04-24T23:28:08.109500+0000","last_unstale":"2024-04-24T23:28:08.109820+0000","last_undegraded":"2024-04-24T23:28:08.109820+0000","last_fullsized":"2024-04-24T23:28:08.109820+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:34:32.814256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.050447+0000","last_change":"2024-04-24T23:27:19.980008+0000","last_active":"2024-04-24T23:28:08.050447+0000","last_peered":"2024-04-24T23:28:08.050447+0000","last_clean":"2024-04-24T23:28:08.050447+0000","last_became_active":"2024-04-24T23:27:19.979424+0000","last_became_peered":"2024-04-24T23:27:19.979424+0000","last_unstale":"2024-04-24T23:28:08.050447+0000","last_undegraded":"2024-04-24T23:28:08.050447+0000","last_fullsized":"2024-04-24T23:28:08.050447+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:44.449245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.050318+0000","last_change":"2024-04-24T23:27:19.987574+0000","last_active":"2024-04-24T23:28:08.050318+0000","last_peered":"2024-04-24T23:28:08.050318+0000","last_clean":"2024-04-24T23:28:08.050318+0000","last_became_active":"2024-04-24T23:27:19.987242+0000","last_became_peered":"2024-04-24T23:27:19.987242+0000","last_unstale":"2024-04-24T23:28:08.050318+0000","last_undegraded":"2024-04-24T23:28:08.050318+0000","last_fullsized":"2024-04-24T23:28:08.050318+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:39:10.777152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.049647+0000","last_change":"2024-04-24T23:27:19.980983+0000","last_active":"2024-04-24T23:28:08.049647+0000","last_peered":"2024-04-24T23:28:08.049647+0000","last_clean":"2024-04-24T23:28:08.049647+0000","last_became_active":"2024-04-24T23:27:19.980659+0000","last_became_peered":"2024-04-24T23:27:19.980659+0000","last_unstale":"2024-04-24T23:28:08.049647+0000","last_undegraded":"2024-04-24T23:28:08.049647+0000","last_fullsized":"2024-04-24T23:28:08.049647+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:02:52.647687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.055819+0000","last_change":"2024-04-24T23:28:08.055819+0000","last_active":"2024-04-24T23:28:08.055819+0000","last_peered":"2024-04-24T23:28:08.055819+0000","last_clean":"2024-04-24T23:28:08.055819+0000","last_became_active":"2024-04-24T23:28:08.055386+0000","last_became_peered":"2024-04-24T23:28:08.055386+0000","last_unstale":"2024-04-24T23:28:08.055819+0000","last_undegraded":"2024-04-24T23:28:08.055819+0000","last_fullsized":"2024-04-24T23:28:08.055819+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:59:30.488812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.052507+0000","last_change":"2024-04-24T23:27:19.985864+0000","last_active":"2024-04-24T23:28:08.052507+0000","last_peered":"2024-04-24T23:28:08.052507+0000","last_clean":"2024-04-24T23:28:08.052507+0000","last_became_active":"2024-04-24T23:27:19.984877+0000","last_became_peered":"2024-04-24T23:27:19.984877+0000","last_unstale":"2024-04-24T23:28:08.052507+0000","last_undegraded":"2024-04-24T23:28:08.052507+0000","last_fullsized":"2024-04-24T23:28:08.052507+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:14:43.458970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.052622+0000","last_change":"2024-04-24T23:27:19.988831+0000","last_active":"2024-04-24T23:28:08.052622+0000","last_peered":"2024-04-24T23:28:08.052622+0000","last_clean":"2024-04-24T23:28:08.052622+0000","last_became_active":"2024-04-24T23:27:19.988478+0000","last_became_peered":"2024-04-24T23:27:19.988478+0000","last_unstale":"2024-04-24T23:28:08.052622+0000","last_undegraded":"2024-04-24T23:28:08.052622+0000","last_fullsized":"2024-04-24T23:28:08.052622+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:18:37.488710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.108145+0000","last_change":"2024-04-24T23:27:19.983553+0000","last_active":"2024-04-24T23:28:08.108145+0000","last_peered":"2024-04-24T23:28:08.108145+0000","last_clean":"2024-04-24T23:28:08.108145+0000","last_became_active":"2024-04-24T23:27:19.981042+0000","last_became_peered":"2024-04-24T23:27:19.981042+0000","last_unstale":"2024-04-24T23:28:08.108145+0000","last_undegraded":"2024-04-24T23:28:08.108145+0000","last_fullsized":"2024-04-24T23:28:08.108145+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:09:42.608238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.052172+0000","last_change":"2024-04-24T23:27:19.985425+0000","last_active":"2024-04-24T23:28:08.052172+0000","last_peered":"2024-04-24T23:28:08.052172+0000","last_clean":"2024-04-24T23:28:08.052172+0000","last_became_active":"2024-04-24T23:27:19.985094+0000","last_became_peered":"2024-04-24T23:27:19.985094+0000","last_unstale":"2024-04-24T23:28:08.052172+0000","last_undegraded":"2024-04-24T23:28:08.052172+0000","last_fullsized":"2024-04-24T23:28:08.052172+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:07:15.612247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.054093+0000","last_change":"2024-04-24T23:27:19.988224+0000","last_active":"2024-04-24T23:28:08.054093+0000","last_peered":"2024-04-24T23:28:08.054093+0000","last_clean":"2024-04-24T23:28:08.054093+0000","last_became_active":"2024-04-24T23:27:19.987820+0000","last_became_peered":"2024-04-24T23:27:19.987820+0000","last_unstale":"2024-04-24T23:28:08.054093+0000","last_undegraded":"2024-04-24T23:28:08.054093+0000","last_fullsized":"2024-04-24T23:28:08.054093+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:27.653290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.050499+0000","last_change":"2024-04-24T23:27:19.989672+0000","last_active":"2024-04-24T23:28:08.050499+0000","last_peered":"2024-04-24T23:28:08.050499+0000","last_clean":"2024-04-24T23:28:08.050499+0000","last_became_active":"2024-04-24T23:27:19.989337+0000","last_became_peered":"2024-04-24T23:27:19.989337+0000","last_unstale":"2024-04-24T23:28:08.050499+0000","last_undegraded":"2024-04-24T23:28:08.050499+0000","last_fullsized":"2024-04-24T23:28:08.050499+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:07:13.399366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:28:08.051316+0000","last_change":"2024-04-24T23:27:19.987703+0000","last_active":"2024-04-24T23:28:08.051316+0000","last_peered":"2024-04-24T23:28:08.051316+0000","last_clean":"2024-04-24T23:28:08.051316+0000","last_became_active":"2024-04-24T23:27:19.987355+0000","last_became_peered":"2024-04-24T23:27:19.987355+0000","last_unstale":"2024-04-24T23:28:08.051316+0000","last_undegraded":"2024-04-24T23:28:08.051316+0000","last_fullsized":"2024-04-24T23:28:08.051316+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:03:15.812726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":280134,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":63263,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":466,"num_read_kb":301,"num_write":159,"num_write_kb":111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":694,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":189789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":138,"ondisk_log_size":138,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383715,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":1011712,"data_stored":510812,"data_compressed":23648,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645351,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28148,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717004,"statfs":{"total":95995035648,"available":95966212096,"internally_reserved":0,"allocated":1150976,"data_stored":1156760,"data_compressed":59279,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808877,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1028096,"data_stored":514562,"data_compressed":23648,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070515,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1024000,"data_stored":510386,"data_compressed":23648,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332151,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1069056,"data_stored":1106104,"data_compressed":59279,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495677,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32164,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712988,"statfs":{"total":95995035648,"available":95962099712,"internally_reserved":0,"allocated":1003520,"data_stored":505642,"data_compressed":23648,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574934,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36396,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708756,"statfs":{"total":95995035648,"available":95957766144,"internally_reserved":0,"allocated":1077248,"data_stored":564535,"data_compressed":23648,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738458,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1028096,"data_stored":507030,"data_compressed":23648,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411409,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32244,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712908,"statfs":{"total":95995035648,"available":95962017792,"internally_reserved":0,"allocated":1085440,"data_stored":566052,"data_compressed":23648,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182476,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32176,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712976,"statfs":{"total":95995035648,"available":95962087424,"internally_reserved":0,"allocated":1015808,"data_stored":505836,"data_compressed":23648,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920839,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1015808,"data_stored":510683,"data_compressed":23648,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757314,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1081344,"data_stored":1101883,"data_compressed":59279,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33993,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33993,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33993,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35631,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35631,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35631,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35631,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35631,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":58983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":60500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":60840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T23:29:03.888 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-24T23:29:03.888 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-24T23:29:03.888 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-04-24T23:29:03.888 DEBUG:teuthology.orchestra.run.smithi080:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid ec7885aa-0290-11ef-bc93-c7b262605968 -- ceph health --format=json 2024-04-24T23:29:04.131 INFO:teuthology.orchestra.run.smithi080.stderr:Inferring config /var/lib/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:29:04.466 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:29:04 smithi080 ceph-mon[29152]: pgmap v350: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:29:04.467 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:29:04 smithi080 ceph-mon[29152]: from='client.15147 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T23:29:04.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:29:04 smithi148 ceph-mon[32528]: pgmap v350: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:29:04.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:29:04 smithi148 ceph-mon[32528]: from='client.15147 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T23:29:04.697 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:29:04 smithi134 ceph-mon[32619]: pgmap v350: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:29:04.697 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:29:04 smithi134 ceph-mon[32619]: from='client.15147 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T23:29:05.919 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:05.916+0000 7fcc97790700 1 -- 172.21.15.80:0/2756206301 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcc90106980 msgr2=0x7fcc90106d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:05.919 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:05.916+0000 7fcc97790700 1 --2- 172.21.15.80:0/2756206301 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcc90106980 0x7fcc90106d60 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fcc8800b0a0 tx=0x7fcc8800b3b0 comp rx=0 tx=0).stop 2024-04-24T23:29:05.919 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:05.916+0000 7fcc97790700 1 -- 172.21.15.80:0/2756206301 shutdown_connections 2024-04-24T23:29:05.920 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:05.916+0000 7fcc97790700 1 --2- 172.21.15.80:0/2756206301 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc90101590 0x7fcc9010d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:05.920 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:05.916+0000 7fcc97790700 1 --2- 172.21.15.80:0/2756206301 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcc90106980 0x7fcc90106d60 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:05.920 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:05.916+0000 7fcc97790700 1 --2- 172.21.15.80:0/2756206301 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc90100bd0 0x7fcc90101050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:05.920 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:05.916+0000 7fcc97790700 1 -- 172.21.15.80:0/2756206301 >> 172.21.15.80:0/2756206301 conn(0x7fcc90076450 msgr2=0x7fcc90076860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:05.920 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:05.916+0000 7fcc97790700 1 -- 172.21.15.80:0/2756206301 shutdown_connections 2024-04-24T23:29:05.920 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:05.916+0000 7fcc97790700 1 -- 172.21.15.80:0/2756206301 wait complete. 2024-04-24T23:29:05.920 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:05.916+0000 7fcc97790700 1 Processor -- start 2024-04-24T23:29:05.920 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:05.916+0000 7fcc97790700 1 -- start start 2024-04-24T23:29:05.920 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:05.917+0000 7fcc97790700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcc90100bd0 0x7fcc9010f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:05.920 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:05.917+0000 7fcc97790700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc90101590 0x7fcc9010f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:05.920 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:05.917+0000 7fcc97790700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc90106980 0x7fcc9010fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:05.920 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:05.917+0000 7fcc97790700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fcc901125e0 con 0x7fcc90100bd0 2024-04-24T23:29:05.920 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:05.917+0000 7fcc97790700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcc90112460 con 0x7fcc90101590 2024-04-24T23:29:05.921 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:05.917+0000 7fcc97790700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fcc90112760 con 0x7fcc90106980 2024-04-24T23:29:05.921 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:05.917+0000 7fcc9552c700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcc90100bd0 0x7fcc9010f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:05.921 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:05.917+0000 7fcc9552c700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcc90100bd0 0x7fcc9010f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:57198/0 (socket says 172.21.15.80:57198) 2024-04-24T23:29:05.921 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:05.917+0000 7fcc9552c700 1 -- 172.21.15.80:0/1651427607 learned_addr learned my addr 172.21.15.80:0/1651427607 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:29:05.921 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:05.917+0000 7fcc94d2b700 1 --2- 172.21.15.80:0/1651427607 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc90101590 0x7fcc9010f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:05.921 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:05.917+0000 7fcc9552c700 1 -- 172.21.15.80:0/1651427607 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc90106980 msgr2=0x7fcc9010fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:05.921 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:05.917+0000 7fcc95d2d700 1 --2- 172.21.15.80:0/1651427607 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc90106980 0x7fcc9010fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:05.921 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:05.917+0000 7fcc9552c700 1 --2- 172.21.15.80:0/1651427607 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc90106980 0x7fcc9010fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:05.921 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:05.917+0000 7fcc9552c700 1 -- 172.21.15.80:0/1651427607 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc90101590 msgr2=0x7fcc9010f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:05.921 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:05.917+0000 7fcc9552c700 1 --2- 172.21.15.80:0/1651427607 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc90101590 0x7fcc9010f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:05.921 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:05.917+0000 7fcc9552c700 1 -- 172.21.15.80:0/1651427607 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc901a6eb0 con 0x7fcc90100bd0 2024-04-24T23:29:05.921 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:05.917+0000 7fcc94d2b700 1 --2- 172.21.15.80:0/1651427607 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc90101590 0x7fcc9010f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:29:05.921 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:05.917+0000 7fcc95d2d700 1 --2- 172.21.15.80:0/1651427607 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc90106980 0x7fcc9010fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:29:05.921 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:05.917+0000 7fcc9552c700 1 --2- 172.21.15.80:0/1651427607 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcc90100bd0 0x7fcc9010f040 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fcc88000f80 tx=0x7fcc8800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:05.922 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:05.917+0000 7fcc8e7fc700 1 -- 172.21.15.80:0/1651427607 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc88021070 con 0x7fcc90100bd0 2024-04-24T23:29:05.922 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:05.917+0000 7fcc97790700 1 -- 172.21.15.80:0/1651427607 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc901a7140 con 0x7fcc90100bd0 2024-04-24T23:29:05.922 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:05.917+0000 7fcc8e7fc700 1 -- 172.21.15.80:0/1651427607 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc88007db0 con 0x7fcc90100bd0 2024-04-24T23:29:05.922 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:05.917+0000 7fcc8e7fc700 1 -- 172.21.15.80:0/1651427607 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc88005550 con 0x7fcc90100bd0 2024-04-24T23:29:05.922 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:05.918+0000 7fcc97790700 1 -- 172.21.15.80:0/1651427607 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc901a7640 con 0x7fcc90100bd0 2024-04-24T23:29:05.922 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:05.918+0000 7fcc97790700 1 -- 172.21.15.80:0/1651427607 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc90108e60 con 0x7fcc90100bd0 2024-04-24T23:29:05.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:05.919+0000 7fcc8e7fc700 1 -- 172.21.15.80:0/1651427607 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fcc88024070 con 0x7fcc90100bd0 2024-04-24T23:29:05.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:05.920+0000 7fcc8e7fc700 1 --2- 172.21.15.80:0/1651427607 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fcc780a2cb0 0x7fcc780a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:05.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:05.920+0000 7fcc94d2b700 1 --2- 172.21.15.80:0/1651427607 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fcc780a2cb0 0x7fcc780a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:05.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:05.920+0000 7fcc8e7fc700 1 -- 172.21.15.80:0/1651427607 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fcc8800f020 con 0x7fcc90100bd0 2024-04-24T23:29:05.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:05.921+0000 7fcc94d2b700 1 --2- 172.21.15.80:0/1651427607 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fcc780a2cb0 0x7fcc780a5170 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fcc900769e0 tx=0x7fcc84008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:05.925 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:05.925+0000 7fcc8e7fc700 1 -- 172.21.15.80:0/1651427607 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fcc8802b020 con 0x7fcc90100bd0 2024-04-24T23:29:06.164 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:06.163+0000 7fcc97790700 1 -- 172.21.15.80:0/1651427607 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fcc90110520 con 0x7fcc90100bd0 2024-04-24T23:29:06.165 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:06.164+0000 7fcc8e7fc700 1 -- 172.21.15.80:0/1651427607 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fcc8809d810 con 0x7fcc90100bd0 2024-04-24T23:29:06.165 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-24T23:29:06.165 INFO:teuthology.orchestra.run.smithi080.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-24T23:29:06.167 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:06.166+0000 7fcc97790700 1 -- 172.21.15.80:0/1651427607 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fcc780a2cb0 msgr2=0x7fcc780a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:06.167 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:06.166+0000 7fcc97790700 1 --2- 172.21.15.80:0/1651427607 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fcc780a2cb0 0x7fcc780a5170 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fcc900769e0 tx=0x7fcc84008040 comp rx=0 tx=0).stop 2024-04-24T23:29:06.167 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:06.166+0000 7fcc97790700 1 -- 172.21.15.80:0/1651427607 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcc90100bd0 msgr2=0x7fcc9010f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:06.167 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:06.166+0000 7fcc97790700 1 --2- 172.21.15.80:0/1651427607 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcc90100bd0 0x7fcc9010f040 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fcc88000f80 tx=0x7fcc8800bf10 comp rx=0 tx=0).stop 2024-04-24T23:29:06.167 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:06.166+0000 7fcc97790700 1 -- 172.21.15.80:0/1651427607 shutdown_connections 2024-04-24T23:29:06.167 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:06.166+0000 7fcc97790700 1 --2- 172.21.15.80:0/1651427607 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc90106980 0x7fcc9010fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:06.167 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:06.166+0000 7fcc97790700 1 --2- 172.21.15.80:0/1651427607 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fcc780a2cb0 0x7fcc780a5170 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:06.167 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:06.166+0000 7fcc97790700 1 --2- 172.21.15.80:0/1651427607 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcc90100bd0 0x7fcc9010f040 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:06.167 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:06.166+0000 7fcc97790700 1 --2- 172.21.15.80:0/1651427607 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc90101590 0x7fcc9010f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:06.167 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:06.166+0000 7fcc97790700 1 -- 172.21.15.80:0/1651427607 >> 172.21.15.80:0/1651427607 conn(0x7fcc90076450 msgr2=0x7fcc900feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:06.167 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:06.166+0000 7fcc97790700 1 -- 172.21.15.80:0/1651427607 shutdown_connections 2024-04-24T23:29:06.168 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:06.166+0000 7fcc97790700 1 -- 172.21.15.80:0/1651427607 wait complete. 2024-04-24T23:29:06.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:29:06 smithi148 ceph-mon[32528]: pgmap v351: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T23:29:06.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:29:06 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/1651427607' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-24T23:29:06.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:29:06 smithi080 ceph-mon[29152]: pgmap v351: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T23:29:06.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:29:06 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/1651427607' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-24T23:29:06.697 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:29:06 smithi134 ceph-mon[32619]: pgmap v351: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T23:29:06.697 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:29:06 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/1651427607' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-24T23:29:07.070 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-04-24T23:29:07.071 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:29:07.176 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.173+0000 7f77a0237640 1 -- 172.21.15.80:0/2189864845 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7798100680 msgr2=0x7f7798100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:07.177 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.173+0000 7f77a0237640 1 --2- 172.21.15.80:0/2189864845 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7798100680 0x7f7798100aa0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f778c009900 tx=0x7f778c02a410 comp rx=0 tx=0).stop 2024-04-24T23:29:07.177 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.173+0000 7f77a0237640 1 -- 172.21.15.80:0/2189864845 shutdown_connections 2024-04-24T23:29:07.177 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.173+0000 7f77a0237640 1 --2- 172.21.15.80:0/2189864845 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7798102230 0x7f7798108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:07.177 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.173+0000 7f77a0237640 1 --2- 172.21.15.80:0/2189864845 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7798101870 0x7f7798101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:07.177 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.173+0000 7f77a0237640 1 --2- 172.21.15.80:0/2189864845 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7798100680 0x7f7798100aa0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:07.177 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.173+0000 7f77a0237640 1 -- 172.21.15.80:0/2189864845 >> 172.21.15.80:0/2189864845 conn(0x7f77980fbbf0 msgr2=0x7f77980fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:07.177 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.173+0000 7f77a0237640 1 -- 172.21.15.80:0/2189864845 shutdown_connections 2024-04-24T23:29:07.177 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.173+0000 7f77a0237640 1 -- 172.21.15.80:0/2189864845 wait complete. 2024-04-24T23:29:07.177 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.173+0000 7f77a0237640 1 Processor -- start 2024-04-24T23:29:07.177 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.173+0000 7f77a0237640 1 -- start start 2024-04-24T23:29:07.178 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.173+0000 7f77a0237640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7798100680 0x7f7798197be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:07.178 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.174+0000 7f77a0237640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7798101870 0x7f7798198120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:07.178 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.174+0000 7f77a0237640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7798102230 0x7f779819f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:07.178 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.174+0000 7f77a0237640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f779810d4e0 con 0x7f7798102230 2024-04-24T23:29:07.178 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.174+0000 7f77a0237640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f779810d360 con 0x7f7798101870 2024-04-24T23:29:07.178 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.174+0000 7f77a0237640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f779810d660 con 0x7f7798100680 2024-04-24T23:29:07.178 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.174+0000 7f779e7d4640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7798102230 0x7f779819f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:07.178 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.174+0000 7f779e7d4640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7798102230 0x7f779819f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:57216/0 (socket says 172.21.15.80:57216) 2024-04-24T23:29:07.178 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.174+0000 7f779e7d4640 1 -- 172.21.15.80:0/2558355233 learned_addr learned my addr 172.21.15.80:0/2558355233 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:29:07.178 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.174+0000 7f779d7d2640 1 --2- 172.21.15.80:0/2558355233 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7798101870 0x7f7798198120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:07.178 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.174+0000 7f779dfd3640 1 --2- 172.21.15.80:0/2558355233 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7798100680 0x7f7798197be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:07.178 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.174+0000 7f779e7d4640 1 -- 172.21.15.80:0/2558355233 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7798100680 msgr2=0x7f7798197be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:07.178 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.174+0000 7f779e7d4640 1 --2- 172.21.15.80:0/2558355233 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7798100680 0x7f7798197be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:07.178 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.174+0000 7f779e7d4640 1 -- 172.21.15.80:0/2558355233 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7798101870 msgr2=0x7f7798198120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:07.179 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.174+0000 7f779e7d4640 1 --2- 172.21.15.80:0/2558355233 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7798101870 0x7f7798198120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:07.179 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.174+0000 7f779e7d4640 1 -- 172.21.15.80:0/2558355233 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f779819f800 con 0x7f7798102230 2024-04-24T23:29:07.179 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.174+0000 7f779e7d4640 1 --2- 172.21.15.80:0/2558355233 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7798102230 0x7f779819f190 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f779000b970 tx=0x7f779000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:07.179 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.174+0000 7f778affd640 1 -- 172.21.15.80:0/2558355233 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7790019070 con 0x7f7798102230 2024-04-24T23:29:07.179 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.174+0000 7f778affd640 1 -- 172.21.15.80:0/2558355233 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f779000cb40 con 0x7f7798102230 2024-04-24T23:29:07.179 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.174+0000 7f77a0237640 1 -- 172.21.15.80:0/2558355233 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f779819faf0 con 0x7f7798102230 2024-04-24T23:29:07.179 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.174+0000 7f778affd640 1 -- 172.21.15.80:0/2558355233 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7790007650 con 0x7f7798102230 2024-04-24T23:29:07.179 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.174+0000 7f77a0237640 1 -- 172.21.15.80:0/2558355233 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77981a0070 con 0x7f7798102230 2024-04-24T23:29:07.179 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.175+0000 7f77a0237640 1 -- 172.21.15.80:0/2558355233 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7798107a70 con 0x7f7798102230 2024-04-24T23:29:07.180 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.177+0000 7f778affd640 1 -- 172.21.15.80:0/2558355233 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f779001c070 con 0x7f7798102230 2024-04-24T23:29:07.180 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.177+0000 7f778affd640 1 --2- 172.21.15.80:0/2558355233 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f77840a2c20 0x7f77840a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:07.180 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.177+0000 7f779dfd3640 1 --2- 172.21.15.80:0/2558355233 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f77840a2c20 0x7f77840a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:07.180 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.177+0000 7f778affd640 1 -- 172.21.15.80:0/2558355233 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f77900ceab0 con 0x7f7798102230 2024-04-24T23:29:07.180 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.178+0000 7f779dfd3640 1 --2- 172.21.15.80:0/2558355233 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f77840a2c20 0x7f77840a50e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f778c006010 tx=0x7f778c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:07.182 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.182+0000 7f778affd640 1 -- 172.21.15.80:0/2558355233 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f779009b660 con 0x7f7798102230 2024-04-24T23:29:07.220 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.219+0000 7f778affd640 1 -- 172.21.15.80:0/2558355233 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f779000ccf0 con 0x7f7798102230 2024-04-24T23:29:07.366 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.366+0000 7f77a0237640 1 -- 172.21.15.80:0/2558355233 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f779802d340 con 0x7f7798102230 2024-04-24T23:29:07.368 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.367+0000 7f778affd640 1 -- 172.21.15.80:0/2558355233 <== mon.0 v2:172.21.15.80:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+24839 (secure 0 0 0) 0x7f77900985b0 con 0x7f7798102230 2024-04-24T23:29:07.368 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-24T23:29:07.369 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":93,"fsid":"ec7885aa-0290-11ef-bc93-c7b262605968","created":"2024-04-24T23:19:05.586407+0000","modified":"2024-04-24T23:29:07.200752+0000","last_up_change":"2024-04-24T23:26:25.938316+0000","last_in_change":"2024-04-24T23:26:04.786699+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:22:33.124428+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:27:17.575788+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:27:18.492762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T23:27:22.438503+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a525741d-b46c-4eef-aa64-3f057cac1497","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":2597718112},{"type":"v1","addr":"172.21.15.80:6803","nonce":2597718112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":2597718112},{"type":"v1","addr":"172.21.15.80:6805","nonce":2597718112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2597718112},{"type":"v1","addr":"172.21.15.80:6809","nonce":2597718112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":2597718112},{"type":"v1","addr":"172.21.15.80:6807","nonce":2597718112}]},"public_addr":"172.21.15.80:6803/2597718112","cluster_addr":"172.21.15.80:6805/2597718112","heartbeat_back_addr":"172.21.15.80:6809/2597718112","heartbeat_front_addr":"172.21.15.80:6807/2597718112","state":["exists","up"]},{"osd":1,"uuid":"2d0f0d67-6db4-40ac-91cc-58604ab6795a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":2395610201},{"type":"v1","addr":"172.21.15.134:6801","nonce":2395610201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":2395610201},{"type":"v1","addr":"172.21.15.134:6803","nonce":2395610201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":2395610201},{"type":"v1","addr":"172.21.15.134:6807","nonce":2395610201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":2395610201},{"type":"v1","addr":"172.21.15.134:6805","nonce":2395610201}]},"public_addr":"172.21.15.134:6801/2395610201","cluster_addr":"172.21.15.134:6803/2395610201","heartbeat_back_addr":"172.21.15.134:6807/2395610201","heartbeat_front_addr":"172.21.15.134:6805/2395610201","state":["exists","up"]},{"osd":2,"uuid":"781fb879-fedc-4c84-82fb-59e073bc4cf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1629152187},{"type":"v1","addr":"172.21.15.148:6801","nonce":1629152187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1629152187},{"type":"v1","addr":"172.21.15.148:6803","nonce":1629152187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1629152187},{"type":"v1","addr":"172.21.15.148:6807","nonce":1629152187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1629152187},{"type":"v1","addr":"172.21.15.148:6805","nonce":1629152187}]},"public_addr":"172.21.15.148:6801/1629152187","cluster_addr":"172.21.15.148:6803/1629152187","heartbeat_back_addr":"172.21.15.148:6807/1629152187","heartbeat_front_addr":"172.21.15.148:6805/1629152187","state":["exists","up"]},{"osd":3,"uuid":"cd982757-f2f6-4296-b3e7-4cd40fbd8d9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":3128298510},{"type":"v1","addr":"172.21.15.80:6811","nonce":3128298510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":3128298510},{"type":"v1","addr":"172.21.15.80:6813","nonce":3128298510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":3128298510},{"type":"v1","addr":"172.21.15.80:6817","nonce":3128298510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":3128298510},{"type":"v1","addr":"172.21.15.80:6815","nonce":3128298510}]},"public_addr":"172.21.15.80:6811/3128298510","cluster_addr":"172.21.15.80:6813/3128298510","heartbeat_back_addr":"172.21.15.80:6817/3128298510","heartbeat_front_addr":"172.21.15.80:6815/3128298510","state":["exists","up"]},{"osd":4,"uuid":"dfc55a79-a7e8-417a-b898-ec44171a54af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1917556060},{"type":"v1","addr":"172.21.15.134:6809","nonce":1917556060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1917556060},{"type":"v1","addr":"172.21.15.134:6811","nonce":1917556060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1917556060},{"type":"v1","addr":"172.21.15.134:6815","nonce":1917556060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1917556060},{"type":"v1","addr":"172.21.15.134:6813","nonce":1917556060}]},"public_addr":"172.21.15.134:6809/1917556060","cluster_addr":"172.21.15.134:6811/1917556060","heartbeat_back_addr":"172.21.15.134:6815/1917556060","heartbeat_front_addr":"172.21.15.134:6813/1917556060","state":["exists","up"]},{"osd":5,"uuid":"b5491337-0694-440c-b679-312f3d14cf65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":2715951281},{"type":"v1","addr":"172.21.15.148:6809","nonce":2715951281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":2715951281},{"type":"v1","addr":"172.21.15.148:6811","nonce":2715951281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":2715951281},{"type":"v1","addr":"172.21.15.148:6815","nonce":2715951281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":2715951281},{"type":"v1","addr":"172.21.15.148:6813","nonce":2715951281}]},"public_addr":"172.21.15.148:6809/2715951281","cluster_addr":"172.21.15.148:6811/2715951281","heartbeat_back_addr":"172.21.15.148:6815/2715951281","heartbeat_front_addr":"172.21.15.148:6813/2715951281","state":["exists","up"]},{"osd":6,"uuid":"ba4b5df7-d86e-47cd-ab67-cf763c4e4d6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":953261724},{"type":"v1","addr":"172.21.15.80:6819","nonce":953261724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":953261724},{"type":"v1","addr":"172.21.15.80:6821","nonce":953261724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":953261724},{"type":"v1","addr":"172.21.15.80:6825","nonce":953261724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":953261724},{"type":"v1","addr":"172.21.15.80:6823","nonce":953261724}]},"public_addr":"172.21.15.80:6819/953261724","cluster_addr":"172.21.15.80:6821/953261724","heartbeat_back_addr":"172.21.15.80:6825/953261724","heartbeat_front_addr":"172.21.15.80:6823/953261724","state":["exists","up"]},{"osd":7,"uuid":"464a503a-75ae-49e7-ab6e-a7f9f2a6a307","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":108354844},{"type":"v1","addr":"172.21.15.134:6817","nonce":108354844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":108354844},{"type":"v1","addr":"172.21.15.134:6819","nonce":108354844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":108354844},{"type":"v1","addr":"172.21.15.134:6823","nonce":108354844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":108354844},{"type":"v1","addr":"172.21.15.134:6821","nonce":108354844}]},"public_addr":"172.21.15.134:6817/108354844","cluster_addr":"172.21.15.134:6819/108354844","heartbeat_back_addr":"172.21.15.134:6823/108354844","heartbeat_front_addr":"172.21.15.134:6821/108354844","state":["exists","up"]},{"osd":8,"uuid":"ce544b2e-4540-47c0-9997-500c45351330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3559655329},{"type":"v1","addr":"172.21.15.148:6817","nonce":3559655329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":3559655329},{"type":"v1","addr":"172.21.15.148:6819","nonce":3559655329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":3559655329},{"type":"v1","addr":"172.21.15.148:6823","nonce":3559655329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3559655329},{"type":"v1","addr":"172.21.15.148:6821","nonce":3559655329}]},"public_addr":"172.21.15.148:6817/3559655329","cluster_addr":"172.21.15.148:6819/3559655329","heartbeat_back_addr":"172.21.15.148:6823/3559655329","heartbeat_front_addr":"172.21.15.148:6821/3559655329","state":["exists","up"]},{"osd":9,"uuid":"06de3732-19b9-496c-a464-55322efa6759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":4019913545},{"type":"v1","addr":"172.21.15.80:6827","nonce":4019913545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":4019913545},{"type":"v1","addr":"172.21.15.80:6829","nonce":4019913545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":4019913545},{"type":"v1","addr":"172.21.15.80:6833","nonce":4019913545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":4019913545},{"type":"v1","addr":"172.21.15.80:6831","nonce":4019913545}]},"public_addr":"172.21.15.80:6827/4019913545","cluster_addr":"172.21.15.80:6829/4019913545","heartbeat_back_addr":"172.21.15.80:6833/4019913545","heartbeat_front_addr":"172.21.15.80:6831/4019913545","state":["exists","up"]},{"osd":10,"uuid":"8d92f0d9-5c64-429a-b750-c33ae0e070c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":647343766},{"type":"v1","addr":"172.21.15.134:6825","nonce":647343766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":647343766},{"type":"v1","addr":"172.21.15.134:6827","nonce":647343766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":647343766},{"type":"v1","addr":"172.21.15.134:6831","nonce":647343766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":647343766},{"type":"v1","addr":"172.21.15.134:6829","nonce":647343766}]},"public_addr":"172.21.15.134:6825/647343766","cluster_addr":"172.21.15.134:6827/647343766","heartbeat_back_addr":"172.21.15.134:6831/647343766","heartbeat_front_addr":"172.21.15.134:6829/647343766","state":["exists","up"]},{"osd":11,"uuid":"4c003a41-f65b-479f-bc80-b27d680cf936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":4068024116},{"type":"v1","addr":"172.21.15.148:6825","nonce":4068024116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":4068024116},{"type":"v1","addr":"172.21.15.148:6827","nonce":4068024116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":4068024116},{"type":"v1","addr":"172.21.15.148:6831","nonce":4068024116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":4068024116},{"type":"v1","addr":"172.21.15.148:6829","nonce":4068024116}]},"public_addr":"172.21.15.148:6825/4068024116","cluster_addr":"172.21.15.148:6827/4068024116","heartbeat_back_addr":"172.21.15.148:6831/4068024116","heartbeat_front_addr":"172.21.15.148:6829/4068024116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:21:42.433631+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:22:06.145933+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:22:30.726143+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:22:55.729961+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:23:21.280913+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:23:46.609983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:24:12.585247+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:24:38.359287+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:25:04.491232+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:25:30.788089+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:25:56.891078+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:26:23.899113+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":7}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"4.a","mappings":[{"from":5,"to":8}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.80:0/2828378749":"2024-04-25T23:20:06.793609+0000","172.21.15.80:6801/233364790":"2024-04-25T23:20:06.793609+0000","172.21.15.80:6800/1614691410":"2024-04-25T23:19:40.490692+0000","172.21.15.80:0/1406187145":"2024-04-25T23:20:06.793609+0000","172.21.15.80:0/92953396":"2024-04-25T23:19:40.490692+0000","172.21.15.80:0/598129432":"2024-04-25T23:20:06.793609+0000","172.21.15.80:0/232193118":"2024-04-25T23:19:40.490692+0000","172.21.15.80:6800/233364790":"2024-04-25T23:20:06.793609+0000","172.21.15.80:6801/1614691410":"2024-04-25T23:19:40.490692+0000","172.21.15.80:0/2081562529":"2024-04-25T23:19:40.490692+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:29:07.370 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.369+0000 7f77a0237640 1 -- 172.21.15.80:0/2558355233 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f77840a2c20 msgr2=0x7f77840a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:07.370 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.369+0000 7f77a0237640 1 --2- 172.21.15.80:0/2558355233 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f77840a2c20 0x7f77840a50e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f778c006010 tx=0x7f778c000bc0 comp rx=0 tx=0).stop 2024-04-24T23:29:07.370 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.369+0000 7f77a0237640 1 -- 172.21.15.80:0/2558355233 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7798102230 msgr2=0x7f779819f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:07.371 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.369+0000 7f77a0237640 1 --2- 172.21.15.80:0/2558355233 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7798102230 0x7f779819f190 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f779000b970 tx=0x7f779000be30 comp rx=0 tx=0).stop 2024-04-24T23:29:07.371 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.369+0000 7f77a0237640 1 -- 172.21.15.80:0/2558355233 shutdown_connections 2024-04-24T23:29:07.371 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.369+0000 7f77a0237640 1 --2- 172.21.15.80:0/2558355233 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f77840a2c20 0x7f77840a50e0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:07.371 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.369+0000 7f77a0237640 1 --2- 172.21.15.80:0/2558355233 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7798102230 0x7f779819f190 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:07.371 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.369+0000 7f77a0237640 1 --2- 172.21.15.80:0/2558355233 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7798101870 0x7f7798198120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:07.371 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.369+0000 7f77a0237640 1 --2- 172.21.15.80:0/2558355233 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7798100680 0x7f7798197be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:07.371 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.369+0000 7f77a0237640 1 -- 172.21.15.80:0/2558355233 >> 172.21.15.80:0/2558355233 conn(0x7f77980fbbf0 msgr2=0x7f7798104c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:07.371 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.369+0000 7f77a0237640 1 -- 172.21.15.80:0/2558355233 shutdown_connections 2024-04-24T23:29:07.371 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.370+0000 7f77a0237640 1 -- 172.21.15.80:0/2558355233 wait complete. 2024-04-24T23:29:07.380 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:29:07 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [5, 8]}]: dispatch 2024-04-24T23:29:07.381 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:29:07 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]: dispatch 2024-04-24T23:29:07.381 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:29:07 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]: dispatch 2024-04-24T23:29:07.381 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:29:07 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]: dispatch 2024-04-24T23:29:07.382 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:22:33.124428+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T23:27:17.575788+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T23:27:18.492762+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-24T23:27:22.438503+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-24T23:29:07.382 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:29:07.486 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.483+0000 7f0462a72640 1 -- 172.21.15.80:0/313666082 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f045c101920 msgr2=0x7f045c108240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:07.487 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.483+0000 7f0462a72640 1 --2- 172.21.15.80:0/313666082 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f045c101920 0x7f045c108240 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f0454009ca0 tx=0x7f045402a8c0 comp rx=0 tx=0).stop 2024-04-24T23:29:07.487 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.483+0000 7f0462a72640 1 -- 172.21.15.80:0/313666082 shutdown_connections 2024-04-24T23:29:07.487 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.483+0000 7f0462a72640 1 --2- 172.21.15.80:0/313666082 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f045c101920 0x7f045c108240 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:07.487 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.483+0000 7f0462a72640 1 --2- 172.21.15.80:0/313666082 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f045c100f60 0x7f045c1013e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:07.487 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.483+0000 7f0462a72640 1 --2- 172.21.15.80:0/313666082 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f045c0ffe10 0x7f045c100230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:07.487 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.483+0000 7f0462a72640 1 -- 172.21.15.80:0/313666082 >> 172.21.15.80:0/313666082 conn(0x7f045c0fb380 msgr2=0x7f045c0fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:07.487 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.483+0000 7f0462a72640 1 -- 172.21.15.80:0/313666082 shutdown_connections 2024-04-24T23:29:07.487 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.483+0000 7f0462a72640 1 -- 172.21.15.80:0/313666082 wait complete. 2024-04-24T23:29:07.487 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.483+0000 7f0462a72640 1 Processor -- start 2024-04-24T23:29:07.487 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.483+0000 7f0462a72640 1 -- start start 2024-04-24T23:29:07.487 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.483+0000 7f0462a72640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f045c0ffe10 0x7f045c103e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:07.487 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.483+0000 7f0462a72640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f045c100f60 0x7f045c1024a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:07.487 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.483+0000 7f0462a72640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f045c101920 0x7f045c1029e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:07.488 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.483+0000 7f0462a72640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f045c10c3e0 con 0x7f045c100f60 2024-04-24T23:29:07.488 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.483+0000 7f0462a72640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f045c10c260 con 0x7f045c101920 2024-04-24T23:29:07.488 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.483+0000 7f0462a72640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f045c10c560 con 0x7f045c0ffe10 2024-04-24T23:29:07.488 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.484+0000 7f045bfff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f045c100f60 0x7f045c1024a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:07.488 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.484+0000 7f045bfff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f045c100f60 0x7f045c1024a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:57244/0 (socket says 172.21.15.80:57244) 2024-04-24T23:29:07.488 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.484+0000 7f045bfff640 1 -- 172.21.15.80:0/420173203 learned_addr learned my addr 172.21.15.80:0/420173203 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:29:07.488 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.484+0000 7f046100f640 1 --2- 172.21.15.80:0/420173203 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f045c101920 0x7f045c1029e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:07.488 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.484+0000 7f046080e640 1 --2- 172.21.15.80:0/420173203 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f045c0ffe10 0x7f045c103e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:07.488 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.484+0000 7f045bfff640 1 -- 172.21.15.80:0/420173203 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f045c0ffe10 msgr2=0x7f045c103e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:07.488 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.484+0000 7f045bfff640 1 --2- 172.21.15.80:0/420173203 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f045c0ffe10 0x7f045c103e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:07.488 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.484+0000 7f045bfff640 1 -- 172.21.15.80:0/420173203 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f045c101920 msgr2=0x7f045c1029e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:07.488 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.484+0000 7f045bfff640 1 --2- 172.21.15.80:0/420173203 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f045c101920 0x7f045c1029e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:07.488 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.484+0000 7f045bfff640 1 -- 172.21.15.80:0/420173203 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f045c103270 con 0x7f045c100f60 2024-04-24T23:29:07.488 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.484+0000 7f046100f640 1 --2- 172.21.15.80:0/420173203 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f045c101920 0x7f045c1029e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:29:07.488 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.484+0000 7f045bfff640 1 --2- 172.21.15.80:0/420173203 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f045c100f60 0x7f045c1024a0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f044800b450 tx=0x7f044800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:07.489 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.484+0000 7f0459ffb640 1 -- 172.21.15.80:0/420173203 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0448015020 con 0x7f045c100f60 2024-04-24T23:29:07.489 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.484+0000 7f0459ffb640 1 -- 172.21.15.80:0/420173203 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0448011d70 con 0x7f045c100f60 2024-04-24T23:29:07.489 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.484+0000 7f0462a72640 1 -- 172.21.15.80:0/420173203 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f045c1a1c10 con 0x7f045c100f60 2024-04-24T23:29:07.489 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.484+0000 7f0459ffb640 1 -- 172.21.15.80:0/420173203 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f044800f340 con 0x7f045c100f60 2024-04-24T23:29:07.489 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.484+0000 7f0462a72640 1 -- 172.21.15.80:0/420173203 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f045c1a2160 con 0x7f045c100f60 2024-04-24T23:29:07.489 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.485+0000 7f0462a72640 1 -- 172.21.15.80:0/420173203 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f045c1a2330 con 0x7f045c100f60 2024-04-24T23:29:07.490 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.486+0000 7f0459ffb640 1 -- 172.21.15.80:0/420173203 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0448018020 con 0x7f045c100f60 2024-04-24T23:29:07.490 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.487+0000 7f0459ffb640 1 --2- 172.21.15.80:0/420173203 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f044c0a2b90 0x7f044c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:07.490 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.487+0000 7f046080e640 1 --2- 172.21.15.80:0/420173203 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f044c0a2b90 0x7f044c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:07.490 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.487+0000 7f0459ffb640 1 -- 172.21.15.80:0/420173203 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f04480cefb0 con 0x7f045c100f60 2024-04-24T23:29:07.490 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.487+0000 7f046080e640 1 --2- 172.21.15.80:0/420173203 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f044c0a2b90 0x7f044c0a5050 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f0450005fd0 tx=0x7f0450005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:07.492 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.491+0000 7f0459ffb640 1 -- 172.21.15.80:0/420173203 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f044801f020 con 0x7f045c100f60 2024-04-24T23:29:07.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:29:07 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [5, 8]}]: dispatch 2024-04-24T23:29:07.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:29:07 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]: dispatch 2024-04-24T23:29:07.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:29:07 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]: dispatch 2024-04-24T23:29:07.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:29:07 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]: dispatch 2024-04-24T23:29:07.668 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.666+0000 7f0462a72640 1 -- 172.21.15.80:0/420173203 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f045c1a2500 con 0x7f045c100f60 2024-04-24T23:29:07.668 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.667+0000 7f0459ffb640 1 -- 172.21.15.80:0/420173203 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) ==== 93+0+10 (secure 0 0 0) 0x7f044800d040 con 0x7f045c100f60 2024-04-24T23:29:07.669 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 1 2024-04-24T23:29:07.671 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.669+0000 7f0462a72640 1 -- 172.21.15.80:0/420173203 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f044c0a2b90 msgr2=0x7f044c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:07.671 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.669+0000 7f0462a72640 1 --2- 172.21.15.80:0/420173203 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f044c0a2b90 0x7f044c0a5050 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f0450005fd0 tx=0x7f0450005850 comp rx=0 tx=0).stop 2024-04-24T23:29:07.671 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.669+0000 7f0462a72640 1 -- 172.21.15.80:0/420173203 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f045c100f60 msgr2=0x7f045c1024a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:07.671 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.669+0000 7f0462a72640 1 --2- 172.21.15.80:0/420173203 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f045c100f60 0x7f045c1024a0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f044800b450 tx=0x7f044800b910 comp rx=0 tx=0).stop 2024-04-24T23:29:07.671 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.669+0000 7f0462a72640 1 -- 172.21.15.80:0/420173203 shutdown_connections 2024-04-24T23:29:07.672 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.669+0000 7f0462a72640 1 --2- 172.21.15.80:0/420173203 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f044c0a2b90 0x7f044c0a5050 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:07.672 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.669+0000 7f0462a72640 1 --2- 172.21.15.80:0/420173203 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f045c101920 0x7f045c1029e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:07.672 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.669+0000 7f0462a72640 1 --2- 172.21.15.80:0/420173203 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f045c100f60 0x7f045c1024a0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:07.672 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.669+0000 7f0462a72640 1 --2- 172.21.15.80:0/420173203 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f045c0ffe10 0x7f045c103e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:07.672 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.669+0000 7f0462a72640 1 -- 172.21.15.80:0/420173203 >> 172.21.15.80:0/420173203 conn(0x7f045c0fb380 msgr2=0x7f045c108900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:07.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.669+0000 7f0462a72640 1 -- 172.21.15.80:0/420173203 shutdown_connections 2024-04-24T23:29:07.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.669+0000 7f0462a72640 1 -- 172.21.15.80:0/420173203 wait complete. 2024-04-24T23:29:07.681 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T23:29:07.697 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:29:07 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [5, 8]}]: dispatch 2024-04-24T23:29:07.697 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:29:07 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]: dispatch 2024-04-24T23:29:07.697 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:29:07 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]: dispatch 2024-04-24T23:29:07.697 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:29:07 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]: dispatch 2024-04-24T23:29:07.783 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.781+0000 7f1e30916640 1 -- 172.21.15.80:0/2358695981 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1e2c102230 msgr2=0x7f1e2c108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:07.784 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.781+0000 7f1e30916640 1 --2- 172.21.15.80:0/2358695981 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1e2c102230 0x7f1e2c108ac0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c009900 tx=0x7f1e1c02a410 comp rx=0 tx=0).stop 2024-04-24T23:29:07.784 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.781+0000 7f1e30916640 1 -- 172.21.15.80:0/2358695981 shutdown_connections 2024-04-24T23:29:07.784 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.781+0000 7f1e30916640 1 --2- 172.21.15.80:0/2358695981 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1e2c102230 0x7f1e2c108ac0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:07.784 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.781+0000 7f1e30916640 1 --2- 172.21.15.80:0/2358695981 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1e2c101870 0x7f1e2c101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:07.784 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.781+0000 7f1e30916640 1 --2- 172.21.15.80:0/2358695981 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e2c100680 0x7f1e2c100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:07.784 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.781+0000 7f1e30916640 1 -- 172.21.15.80:0/2358695981 >> 172.21.15.80:0/2358695981 conn(0x7f1e2c0fbbf0 msgr2=0x7f1e2c0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:07.784 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.781+0000 7f1e30916640 1 -- 172.21.15.80:0/2358695981 shutdown_connections 2024-04-24T23:29:07.784 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.781+0000 7f1e30916640 1 -- 172.21.15.80:0/2358695981 wait complete. 2024-04-24T23:29:07.784 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.781+0000 7f1e30916640 1 Processor -- start 2024-04-24T23:29:07.784 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.781+0000 7f1e30916640 1 -- start start 2024-04-24T23:29:07.784 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.782+0000 7f1e30916640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e2c100680 0x7f1e2c06a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:07.784 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.782+0000 7f1e30916640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1e2c101870 0x7f1e2c19c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:07.785 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.782+0000 7f1e30916640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1e2c102230 0x7f1e2c19edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:07.785 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.782+0000 7f1e30916640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f1e2c10c300 con 0x7f1e2c102230 2024-04-24T23:29:07.785 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.782+0000 7f1e30916640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1e2c10c180 con 0x7f1e2c100680 2024-04-24T23:29:07.785 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.782+0000 7f1e30916640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1e2c10c480 con 0x7f1e2c101870 2024-04-24T23:29:07.785 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.782+0000 7f1e2ad9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1e2c102230 0x7f1e2c19edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:07.785 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.782+0000 7f1e2ad9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1e2c102230 0x7f1e2c19edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:57264/0 (socket says 172.21.15.80:57264) 2024-04-24T23:29:07.785 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.782+0000 7f1e2ad9d640 1 -- 172.21.15.80:0/2469612733 learned_addr learned my addr 172.21.15.80:0/2469612733 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:29:07.785 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.782+0000 7f1e29d9b640 1 --2- 172.21.15.80:0/2469612733 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1e2c101870 0x7f1e2c19c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:07.785 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.782+0000 7f1e2a59c640 1 --2- 172.21.15.80:0/2469612733 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e2c100680 0x7f1e2c06a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:07.785 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.782+0000 7f1e2ad9d640 1 -- 172.21.15.80:0/2469612733 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1e2c101870 msgr2=0x7f1e2c19c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:07.785 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.782+0000 7f1e2ad9d640 1 --2- 172.21.15.80:0/2469612733 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1e2c101870 0x7f1e2c19c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:07.785 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.782+0000 7f1e2ad9d640 1 -- 172.21.15.80:0/2469612733 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e2c100680 msgr2=0x7f1e2c06a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:07.785 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.782+0000 7f1e2ad9d640 1 --2- 172.21.15.80:0/2469612733 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e2c100680 0x7f1e2c06a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:07.785 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.782+0000 7f1e2ad9d640 1 -- 172.21.15.80:0/2469612733 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e2c19f670 con 0x7f1e2c102230 2024-04-24T23:29:07.785 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.782+0000 7f1e2ad9d640 1 --2- 172.21.15.80:0/2469612733 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1e2c102230 0x7f1e2c19edb0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c0314e0 tx=0x7f1e1c004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:07.786 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.782+0000 7f1e1b7fe640 1 -- 172.21.15.80:0/2469612733 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e1c03f030 con 0x7f1e2c102230 2024-04-24T23:29:07.786 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.782+0000 7f1e1b7fe640 1 -- 172.21.15.80:0/2469612733 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e1c003ce0 con 0x7f1e2c102230 2024-04-24T23:29:07.786 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.782+0000 7f1e30916640 1 -- 172.21.15.80:0/2469612733 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e2c19f900 con 0x7f1e2c102230 2024-04-24T23:29:07.786 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.782+0000 7f1e1b7fe640 1 -- 172.21.15.80:0/2469612733 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e1c035070 con 0x7f1e2c102230 2024-04-24T23:29:07.786 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.782+0000 7f1e30916640 1 -- 172.21.15.80:0/2469612733 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e2c19fdf0 con 0x7f1e2c102230 2024-04-24T23:29:07.787 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.783+0000 7f1e30916640 1 -- 172.21.15.80:0/2469612733 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e2c1079a0 con 0x7f1e2c102230 2024-04-24T23:29:07.787 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.784+0000 7f1e1b7fe640 1 -- 172.21.15.80:0/2469612733 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1e1c042030 con 0x7f1e2c102230 2024-04-24T23:29:07.787 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.785+0000 7f1e1b7fe640 1 --2- 172.21.15.80:0/2469612733 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f1e140a2b90 0x7f1e140a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:07.787 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.785+0000 7f1e2a59c640 1 --2- 172.21.15.80:0/2469612733 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f1e140a2b90 0x7f1e140a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:07.787 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.785+0000 7f1e1b7fe640 1 -- 172.21.15.80:0/2469612733 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f1e1c0efd10 con 0x7f1e2c102230 2024-04-24T23:29:07.787 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.785+0000 7f1e2a59c640 1 --2- 172.21.15.80:0/2469612733 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f1e140a2b90 0x7f1e140a5050 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f1e2c1016d0 tx=0x7f1e2400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:07.790 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.789+0000 7f1e1b7fe640 1 -- 172.21.15.80:0/2469612733 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f1e1c0bc250 con 0x7f1e2c102230 2024-04-24T23:29:07.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.963+0000 7f1e30916640 1 -- 172.21.15.80:0/2469612733 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f1e2c062340 con 0x7f1e2c102230 2024-04-24T23:29:07.965 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.964+0000 7f1e1b7fe640 1 -- 172.21.15.80:0/2469612733 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) ==== 104+0+11 (secure 0 0 0) 0x7f1e1c0c0550 con 0x7f1e2c102230 2024-04-24T23:29:07.965 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 32 2024-04-24T23:29:07.966 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.966+0000 7f1e30916640 1 -- 172.21.15.80:0/2469612733 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f1e140a2b90 msgr2=0x7f1e140a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:07.967 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.966+0000 7f1e30916640 1 --2- 172.21.15.80:0/2469612733 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f1e140a2b90 0x7f1e140a5050 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f1e2c1016d0 tx=0x7f1e2400b390 comp rx=0 tx=0).stop 2024-04-24T23:29:07.967 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.966+0000 7f1e30916640 1 -- 172.21.15.80:0/2469612733 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1e2c102230 msgr2=0x7f1e2c19edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:07.967 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.966+0000 7f1e30916640 1 --2- 172.21.15.80:0/2469612733 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1e2c102230 0x7f1e2c19edb0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c0314e0 tx=0x7f1e1c004360 comp rx=0 tx=0).stop 2024-04-24T23:29:07.967 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.966+0000 7f1e30916640 1 -- 172.21.15.80:0/2469612733 shutdown_connections 2024-04-24T23:29:07.967 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.966+0000 7f1e30916640 1 --2- 172.21.15.80:0/2469612733 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f1e140a2b90 0x7f1e140a5050 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:07.967 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.966+0000 7f1e30916640 1 --2- 172.21.15.80:0/2469612733 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1e2c102230 0x7f1e2c19edb0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:07.967 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.966+0000 7f1e30916640 1 --2- 172.21.15.80:0/2469612733 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1e2c101870 0x7f1e2c19c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:07.967 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.966+0000 7f1e30916640 1 --2- 172.21.15.80:0/2469612733 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e2c100680 0x7f1e2c06a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:07.967 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.966+0000 7f1e30916640 1 -- 172.21.15.80:0/2469612733 >> 172.21.15.80:0/2469612733 conn(0x7f1e2c0fbbf0 msgr2=0x7f1e2c104b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:07.967 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.966+0000 7f1e30916640 1 -- 172.21.15.80:0/2469612733 shutdown_connections 2024-04-24T23:29:07.967 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:07.966+0000 7f1e30916640 1 -- 172.21.15.80:0/2469612733 wait complete. 2024-04-24T23:29:07.978 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T23:29:08.099 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.096+0000 7f088ca7d640 1 -- 172.21.15.80:0/4106046942 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0888102990 msgr2=0x7f0888102e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:08.099 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.096+0000 7f088ca7d640 1 --2- 172.21.15.80:0/4106046942 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0888102990 0x7f0888102e10 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f0878009900 tx=0x7f087802a410 comp rx=0 tx=0).stop 2024-04-24T23:29:08.099 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.096+0000 7f088ca7d640 1 -- 172.21.15.80:0/4106046942 shutdown_connections 2024-04-24T23:29:08.099 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.096+0000 7f088ca7d640 1 --2- 172.21.15.80:0/4106046942 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0888103350 0x7f0888109be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:08.099 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.096+0000 7f088ca7d640 1 --2- 172.21.15.80:0/4106046942 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0888102990 0x7f0888102e10 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:08.099 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.096+0000 7f088ca7d640 1 --2- 172.21.15.80:0/4106046942 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f08881017a0 0x7f0888101bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:08.100 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.096+0000 7f088ca7d640 1 -- 172.21.15.80:0/4106046942 >> 172.21.15.80:0/4106046942 conn(0x7f08880fcd10 msgr2=0x7f08880ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:08.100 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.096+0000 7f088ca7d640 1 -- 172.21.15.80:0/4106046942 shutdown_connections 2024-04-24T23:29:08.100 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.096+0000 7f088ca7d640 1 -- 172.21.15.80:0/4106046942 wait complete. 2024-04-24T23:29:08.100 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.097+0000 7f088ca7d640 1 Processor -- start 2024-04-24T23:29:08.100 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.097+0000 7f088ca7d640 1 -- start start 2024-04-24T23:29:08.100 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.097+0000 7f088ca7d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f08881017a0 0x7f08880754f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:08.100 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.097+0000 7f088ca7d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0888102990 0x7f0888073b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:08.100 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.097+0000 7f088ca7d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0888103350 0x7f0888074070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:08.100 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.097+0000 7f088ca7d640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f088810d470 con 0x7f0888103350 2024-04-24T23:29:08.100 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.097+0000 7f088ca7d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f088810d2f0 con 0x7f0888102990 2024-04-24T23:29:08.100 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.097+0000 7f088ca7d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f088810d5f0 con 0x7f08881017a0 2024-04-24T23:29:08.100 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.097+0000 7f0886d9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0888103350 0x7f0888074070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:08.100 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.097+0000 7f0886d9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0888103350 0x7f0888074070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:57282/0 (socket says 172.21.15.80:57282) 2024-04-24T23:29:08.100 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.097+0000 7f0886d9d640 1 -- 172.21.15.80:0/965829966 learned_addr learned my addr 172.21.15.80:0/965829966 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:29:08.101 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.097+0000 7f088659c640 1 --2- 172.21.15.80:0/965829966 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f08881017a0 0x7f08880754f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:08.101 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.097+0000 7f0885d9b640 1 --2- 172.21.15.80:0/965829966 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0888102990 0x7f0888073b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:08.101 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.097+0000 7f0886d9d640 1 -- 172.21.15.80:0/965829966 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f08881017a0 msgr2=0x7f08880754f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:08.101 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.097+0000 7f0886d9d640 1 --2- 172.21.15.80:0/965829966 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f08881017a0 0x7f08880754f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:08.101 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.097+0000 7f0886d9d640 1 -- 172.21.15.80:0/965829966 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0888102990 msgr2=0x7f0888073b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:08.101 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.097+0000 7f0886d9d640 1 --2- 172.21.15.80:0/965829966 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0888102990 0x7f0888073b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:08.101 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.097+0000 7f0886d9d640 1 -- 172.21.15.80:0/965829966 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0888074900 con 0x7f0888103350 2024-04-24T23:29:08.101 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.097+0000 7f0885d9b640 1 --2- 172.21.15.80:0/965829966 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0888102990 0x7f0888073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:29:08.101 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.098+0000 7f0886d9d640 1 --2- 172.21.15.80:0/965829966 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0888103350 0x7f0888074070 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f088000eb10 tx=0x7f088000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:08.101 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.098+0000 7f08777fe640 1 -- 172.21.15.80:0/965829966 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0880004b20 con 0x7f0888103350 2024-04-24T23:29:08.101 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.098+0000 7f088ca7d640 1 -- 172.21.15.80:0/965829966 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08881a1c00 con 0x7f0888103350 2024-04-24T23:29:08.101 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.098+0000 7f08777fe640 1 -- 172.21.15.80:0/965829966 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0880004cc0 con 0x7f0888103350 2024-04-24T23:29:08.101 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.098+0000 7f08777fe640 1 -- 172.21.15.80:0/965829966 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0880010760 con 0x7f0888103350 2024-04-24T23:29:08.101 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.098+0000 7f088ca7d640 1 -- 172.21.15.80:0/965829966 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08881a21a0 con 0x7f0888103350 2024-04-24T23:29:08.102 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.099+0000 7f088ca7d640 1 -- 172.21.15.80:0/965829966 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0888108b90 con 0x7f0888103350 2024-04-24T23:29:08.102 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.101+0000 7f08777fe640 1 -- 172.21.15.80:0/965829966 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0880005000 con 0x7f0888103350 2024-04-24T23:29:08.103 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.101+0000 7f08777fe640 1 --2- 172.21.15.80:0/965829966 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f08700a2b90 0x7f08700a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:08.103 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.101+0000 7f08777fe640 1 -- 172.21.15.80:0/965829966 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7f08800cff00 con 0x7f0888103350 2024-04-24T23:29:08.103 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.101+0000 7f088659c640 1 --2- 172.21.15.80:0/965829966 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f08700a2b90 0x7f08700a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:08.103 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.102+0000 7f088659c640 1 --2- 172.21.15.80:0/965829966 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f08700a2b90 0x7f08700a5050 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f08881027f0 tx=0x7f087c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:08.105 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.104+0000 7f08777fe640 1 -- 172.21.15.80:0/965829966 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f088009ca90 con 0x7f0888103350 2024-04-24T23:29:08.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:29:08 smithi080 ceph-mon[29152]: pgmap v352: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:29:08.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:29:08 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [5, 8]}]': finished 2024-04-24T23:29:08.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:29:08 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]': finished 2024-04-24T23:29:08.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:29:08 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]': finished 2024-04-24T23:29:08.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:29:08 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]': finished 2024-04-24T23:29:08.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:29:08 smithi080 ceph-mon[29152]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-04-24T23:29:08.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:29:08 smithi080 ceph-mon[29152]: osdmap e93: 12 total, 12 up, 12 in 2024-04-24T23:29:08.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:29:08 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/2558355233' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:29:08.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:29:08 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/420173203' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:29:08.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:29:08 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/2469612733' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T23:29:08.293 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.293+0000 7f088ca7d640 1 -- 172.21.15.80:0/965829966 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f088802d170 con 0x7f0888103350 2024-04-24T23:29:08.294 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.293+0000 7f08777fe640 1 -- 172.21.15.80:0/965829966 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) ==== 100+0+11 (secure 0 0 0) 0x7f088009c390 con 0x7f0888103350 2024-04-24T23:29:08.294 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 64 2024-04-24T23:29:08.296 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.295+0000 7f088ca7d640 1 -- 172.21.15.80:0/965829966 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f08700a2b90 msgr2=0x7f08700a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:08.296 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.295+0000 7f088ca7d640 1 --2- 172.21.15.80:0/965829966 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f08700a2b90 0x7f08700a5050 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f08881027f0 tx=0x7f087c005f90 comp rx=0 tx=0).stop 2024-04-24T23:29:08.296 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.295+0000 7f088ca7d640 1 -- 172.21.15.80:0/965829966 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0888103350 msgr2=0x7f0888074070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:08.297 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.295+0000 7f088ca7d640 1 --2- 172.21.15.80:0/965829966 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0888103350 0x7f0888074070 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f088000eb10 tx=0x7f088000c2d0 comp rx=0 tx=0).stop 2024-04-24T23:29:08.297 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.295+0000 7f088ca7d640 1 -- 172.21.15.80:0/965829966 shutdown_connections 2024-04-24T23:29:08.297 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.295+0000 7f088ca7d640 1 --2- 172.21.15.80:0/965829966 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f08700a2b90 0x7f08700a5050 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:08.297 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.295+0000 7f088ca7d640 1 --2- 172.21.15.80:0/965829966 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0888103350 0x7f0888074070 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:08.297 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.295+0000 7f088ca7d640 1 --2- 172.21.15.80:0/965829966 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0888102990 0x7f0888073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:08.297 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.295+0000 7f088ca7d640 1 --2- 172.21.15.80:0/965829966 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f08881017a0 0x7f08880754f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:08.297 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.295+0000 7f088ca7d640 1 -- 172.21.15.80:0/965829966 >> 172.21.15.80:0/965829966 conn(0x7f08880fcd10 msgr2=0x7f0888105d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:08.297 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.295+0000 7f088ca7d640 1 -- 172.21.15.80:0/965829966 shutdown_connections 2024-04-24T23:29:08.297 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.295+0000 7f088ca7d640 1 -- 172.21.15.80:0/965829966 wait complete. 2024-04-24T23:29:08.308 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-24T23:29:08.412 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.409+0000 7f5574acd640 1 -- 172.21.15.80:0/3655165597 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5570101870 msgr2=0x7f5570101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:08.413 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.409+0000 7f5574acd640 1 --2- 172.21.15.80:0/3655165597 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5570101870 0x7f5570101cf0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f5558009900 tx=0x7f555802a410 comp rx=0 tx=0).stop 2024-04-24T23:29:08.413 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.409+0000 7f5574acd640 1 -- 172.21.15.80:0/3655165597 shutdown_connections 2024-04-24T23:29:08.413 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.409+0000 7f5574acd640 1 --2- 172.21.15.80:0/3655165597 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5570102230 0x7f5570108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:08.413 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.409+0000 7f5574acd640 1 --2- 172.21.15.80:0/3655165597 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5570101870 0x7f5570101cf0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:08.413 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.409+0000 7f5574acd640 1 --2- 172.21.15.80:0/3655165597 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5570100680 0x7f5570100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:08.413 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.409+0000 7f5574acd640 1 -- 172.21.15.80:0/3655165597 >> 172.21.15.80:0/3655165597 conn(0x7f55700fbbf0 msgr2=0x7f55700fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:08.414 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.409+0000 7f5574acd640 1 -- 172.21.15.80:0/3655165597 shutdown_connections 2024-04-24T23:29:08.414 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.409+0000 7f5574acd640 1 -- 172.21.15.80:0/3655165597 wait complete. 2024-04-24T23:29:08.414 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.410+0000 7f5574acd640 1 Processor -- start 2024-04-24T23:29:08.414 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.410+0000 7f5574acd640 1 -- start start 2024-04-24T23:29:08.414 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.410+0000 7f5574acd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5570100680 0x7f5570069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:08.414 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.410+0000 7f5574acd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5570101870 0x7f557019c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:08.414 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.410+0000 7f5574acd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5570102230 0x7f557019eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:08.414 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.410+0000 7f5574acd640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f557010c370 con 0x7f5570102230 2024-04-24T23:29:08.415 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.410+0000 7f5574acd640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f557010c1f0 con 0x7f5570101870 2024-04-24T23:29:08.415 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.410+0000 7f5574acd640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f557010c4f0 con 0x7f5570100680 2024-04-24T23:29:08.415 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.410+0000 7f556ed9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5570102230 0x7f557019eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:08.415 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.410+0000 7f556ed9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5570102230 0x7f557019eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:57304/0 (socket says 172.21.15.80:57304) 2024-04-24T23:29:08.415 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.410+0000 7f556ed9d640 1 -- 172.21.15.80:0/1138428572 learned_addr learned my addr 172.21.15.80:0/1138428572 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:29:08.415 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.410+0000 7f5565d9b640 1 --2- 172.21.15.80:0/1138428572 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5570101870 0x7f557019c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:08.415 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.410+0000 7f556e59c640 1 --2- 172.21.15.80:0/1138428572 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5570100680 0x7f5570069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:08.415 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.410+0000 7f556ed9d640 1 -- 172.21.15.80:0/1138428572 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5570100680 msgr2=0x7f5570069460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:08.415 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.410+0000 7f556ed9d640 1 --2- 172.21.15.80:0/1138428572 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5570100680 0x7f5570069460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:08.415 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.410+0000 7f556ed9d640 1 -- 172.21.15.80:0/1138428572 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5570101870 msgr2=0x7f557019c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:08.415 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.410+0000 7f556ed9d640 1 --2- 172.21.15.80:0/1138428572 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5570101870 0x7f557019c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:08.415 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.410+0000 7f556ed9d640 1 -- 172.21.15.80:0/1138428572 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f557019f770 con 0x7f5570102230 2024-04-24T23:29:08.415 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.410+0000 7f556e59c640 1 --2- 172.21.15.80:0/1138428572 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5570100680 0x7f5570069460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:29:08.415 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.411+0000 7f556ed9d640 1 --2- 172.21.15.80:0/1138428572 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5570102230 0x7f557019eeb0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f556800eaa0 tx=0x7f556800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:08.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.411+0000 7f5567fff640 1 -- 172.21.15.80:0/1138428572 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f556800cdb0 con 0x7f5570102230 2024-04-24T23:29:08.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.411+0000 7f5567fff640 1 -- 172.21.15.80:0/1138428572 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5568005020 con 0x7f5570102230 2024-04-24T23:29:08.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.411+0000 7f5574acd640 1 -- 172.21.15.80:0/1138428572 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f557019fa60 con 0x7f5570102230 2024-04-24T23:29:08.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.411+0000 7f5567fff640 1 -- 172.21.15.80:0/1138428572 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55680116b0 con 0x7f5570102230 2024-04-24T23:29:08.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.411+0000 7f5574acd640 1 -- 172.21.15.80:0/1138428572 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f557019ffb0 con 0x7f5570102230 2024-04-24T23:29:08.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.412+0000 7f5574acd640 1 -- 172.21.15.80:0/1138428572 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5570107a20 con 0x7f5570102230 2024-04-24T23:29:08.417 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.413+0000 7f5567fff640 1 -- 172.21.15.80:0/1138428572 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5568005560 con 0x7f5570102230 2024-04-24T23:29:08.417 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.413+0000 7f5567fff640 1 --2- 172.21.15.80:0/1138428572 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f555c0a2b90 0x7f555c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:08.417 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.414+0000 7f556e59c640 1 --2- 172.21.15.80:0/1138428572 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f555c0a2b90 0x7f555c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:08.417 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.414+0000 7f5567fff640 1 -- 172.21.15.80:0/1138428572 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f55680cf710 con 0x7f5570102230 2024-04-24T23:29:08.417 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.414+0000 7f556e59c640 1 --2- 172.21.15.80:0/1138428572 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f555c0a2b90 0x7f555c0a5050 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f5560006fd0 tx=0x7f5560008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:08.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.418+0000 7f5567fff640 1 -- 172.21.15.80:0/1138428572 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f5568016030 con 0x7f5570102230 2024-04-24T23:29:08.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:29:08 smithi148 ceph-mon[32528]: pgmap v352: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:29:08.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:29:08 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [5, 8]}]': finished 2024-04-24T23:29:08.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:29:08 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]': finished 2024-04-24T23:29:08.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:29:08 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]': finished 2024-04-24T23:29:08.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:29:08 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]': finished 2024-04-24T23:29:08.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:29:08 smithi148 ceph-mon[32528]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-04-24T23:29:08.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:29:08 smithi148 ceph-mon[32528]: osdmap e93: 12 total, 12 up, 12 in 2024-04-24T23:29:08.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:29:08 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/2558355233' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:29:08.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:29:08 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/420173203' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:29:08.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:29:08 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/2469612733' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T23:29:08.593 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.592+0000 7f5574acd640 1 -- 172.21.15.80:0/1138428572 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f5570062340 con 0x7f5570102230 2024-04-24T23:29:08.593 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.593+0000 7f5567fff640 1 -- 172.21.15.80:0/1138428572 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) ==== 103+0+11 (secure 0 0 0) 0x7f556809c2a0 con 0x7f5570102230 2024-04-24T23:29:08.594 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 32 2024-04-24T23:29:08.595 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.594+0000 7f5574acd640 1 -- 172.21.15.80:0/1138428572 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f555c0a2b90 msgr2=0x7f555c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:08.596 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.594+0000 7f5574acd640 1 --2- 172.21.15.80:0/1138428572 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f555c0a2b90 0x7f555c0a5050 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f5560006fd0 tx=0x7f5560008040 comp rx=0 tx=0).stop 2024-04-24T23:29:08.596 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.594+0000 7f5574acd640 1 -- 172.21.15.80:0/1138428572 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5570102230 msgr2=0x7f557019eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:08.596 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.594+0000 7f5574acd640 1 --2- 172.21.15.80:0/1138428572 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5570102230 0x7f557019eeb0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f556800eaa0 tx=0x7f556800ef60 comp rx=0 tx=0).stop 2024-04-24T23:29:08.596 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.594+0000 7f5574acd640 1 -- 172.21.15.80:0/1138428572 shutdown_connections 2024-04-24T23:29:08.596 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.594+0000 7f5574acd640 1 --2- 172.21.15.80:0/1138428572 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f555c0a2b90 0x7f555c0a5050 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:08.596 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.594+0000 7f5574acd640 1 --2- 172.21.15.80:0/1138428572 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5570102230 0x7f557019eeb0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:08.596 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.594+0000 7f5574acd640 1 --2- 172.21.15.80:0/1138428572 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5570101870 0x7f557019c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:08.596 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.594+0000 7f5574acd640 1 --2- 172.21.15.80:0/1138428572 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5570100680 0x7f5570069460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:08.596 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.594+0000 7f5574acd640 1 -- 172.21.15.80:0/1138428572 >> 172.21.15.80:0/1138428572 conn(0x7f55700fbbf0 msgr2=0x7f5570104bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:08.596 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.595+0000 7f5574acd640 1 -- 172.21.15.80:0/1138428572 shutdown_connections 2024-04-24T23:29:08.597 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.595+0000 7f5574acd640 1 -- 172.21.15.80:0/1138428572 wait complete. 2024-04-24T23:29:08.607 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:29:08.697 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:29:08 smithi134 ceph-mon[32619]: pgmap v352: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:29:08.697 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:29:08 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [5, 8]}]': finished 2024-04-24T23:29:08.697 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:29:08 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]': finished 2024-04-24T23:29:08.698 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:29:08 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]': finished 2024-04-24T23:29:08.698 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:29:08 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]': finished 2024-04-24T23:29:08.698 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:29:08 smithi134 ceph-mon[32619]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-04-24T23:29:08.698 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:29:08 smithi134 ceph-mon[32619]: osdmap e93: 12 total, 12 up, 12 in 2024-04-24T23:29:08.698 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:29:08 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/2558355233' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:29:08.698 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:29:08 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/420173203' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:29:08.698 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:29:08 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/2469612733' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T23:29:08.708 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.705+0000 7fdd11f16640 1 -- 172.21.15.80:0/2757011385 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdd0c102230 msgr2=0x7fdd0c108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:08.708 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.705+0000 7fdd11f16640 1 --2- 172.21.15.80:0/2757011385 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdd0c102230 0x7fdd0c108ac0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fdd04009ca0 tx=0x7fdd0402a8c0 comp rx=0 tx=0).stop 2024-04-24T23:29:08.708 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.705+0000 7fdd11f16640 1 -- 172.21.15.80:0/2757011385 shutdown_connections 2024-04-24T23:29:08.708 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.705+0000 7fdd11f16640 1 --2- 172.21.15.80:0/2757011385 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdd0c102230 0x7fdd0c108ac0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:08.708 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.705+0000 7fdd11f16640 1 --2- 172.21.15.80:0/2757011385 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdd0c101870 0x7fdd0c101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:08.708 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.705+0000 7fdd11f16640 1 --2- 172.21.15.80:0/2757011385 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd0c100680 0x7fdd0c100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:08.708 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.705+0000 7fdd11f16640 1 -- 172.21.15.80:0/2757011385 >> 172.21.15.80:0/2757011385 conn(0x7fdd0c0fbbf0 msgr2=0x7fdd0c0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:08.708 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.705+0000 7fdd11f16640 1 -- 172.21.15.80:0/2757011385 shutdown_connections 2024-04-24T23:29:08.708 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.705+0000 7fdd11f16640 1 -- 172.21.15.80:0/2757011385 wait complete. 2024-04-24T23:29:08.709 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.705+0000 7fdd11f16640 1 Processor -- start 2024-04-24T23:29:08.709 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.705+0000 7fdd11f16640 1 -- start start 2024-04-24T23:29:08.709 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.705+0000 7fdd11f16640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdd0c100680 0x7fdd0c197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:08.709 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.705+0000 7fdd11f16640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdd0c101870 0x7fdd0c197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:08.709 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.705+0000 7fdd11f16640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd0c102230 0x7fdd0c19f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:08.709 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.705+0000 7fdd11f16640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fdd0c10c300 con 0x7fdd0c100680 2024-04-24T23:29:08.709 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.705+0000 7fdd11f16640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fdd0c10c180 con 0x7fdd0c102230 2024-04-24T23:29:08.709 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.705+0000 7fdd11f16640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fdd0c10c480 con 0x7fdd0c101870 2024-04-24T23:29:08.709 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.705+0000 7fdd0b7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdd0c100680 0x7fdd0c197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:08.709 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.705+0000 7fdd0b7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdd0c100680 0x7fdd0c197a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:57314/0 (socket says 172.21.15.80:57314) 2024-04-24T23:29:08.709 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.705+0000 7fdd0b7fe640 1 -- 172.21.15.80:0/3369536162 learned_addr learned my addr 172.21.15.80:0/3369536162 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:29:08.709 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.706+0000 7fdd0bfff640 1 --2- 172.21.15.80:0/3369536162 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd0c102230 0x7fdd0c19f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:08.710 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.706+0000 7fdd03fff640 1 --2- 172.21.15.80:0/3369536162 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdd0c101870 0x7fdd0c197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:08.710 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.706+0000 7fdd0b7fe640 1 -- 172.21.15.80:0/3369536162 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdd0c101870 msgr2=0x7fdd0c197fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:08.710 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.706+0000 7fdd0b7fe640 1 --2- 172.21.15.80:0/3369536162 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdd0c101870 0x7fdd0c197fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:08.710 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.706+0000 7fdd0b7fe640 1 -- 172.21.15.80:0/3369536162 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd0c102230 msgr2=0x7fdd0c19f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:08.710 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.706+0000 7fdd0b7fe640 1 --2- 172.21.15.80:0/3369536162 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd0c102230 0x7fdd0c19f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:08.710 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.706+0000 7fdd0b7fe640 1 -- 172.21.15.80:0/3369536162 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd0c19f6b0 con 0x7fdd0c100680 2024-04-24T23:29:08.710 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.706+0000 7fdd0b7fe640 1 --2- 172.21.15.80:0/3369536162 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdd0c100680 0x7fdd0c197a90 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fdcfc00e9c0 tx=0x7fdcfc00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:08.710 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.706+0000 7fdd097fa640 1 -- 172.21.15.80:0/3369536162 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcfc00cd80 con 0x7fdd0c100680 2024-04-24T23:29:08.710 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.706+0000 7fdd097fa640 1 -- 172.21.15.80:0/3369536162 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdcfc0035f0 con 0x7fdd0c100680 2024-04-24T23:29:08.710 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.706+0000 7fdd11f16640 1 -- 172.21.15.80:0/3369536162 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd0c19f9a0 con 0x7fdd0c100680 2024-04-24T23:29:08.710 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.706+0000 7fdd097fa640 1 -- 172.21.15.80:0/3369536162 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcfc010690 con 0x7fdd0c100680 2024-04-24T23:29:08.710 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.706+0000 7fdd11f16640 1 -- 172.21.15.80:0/3369536162 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd0c19fef0 con 0x7fdd0c100680 2024-04-24T23:29:08.710 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.707+0000 7fdd11f16640 1 -- 172.21.15.80:0/3369536162 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd0c1a0120 con 0x7fdd0c100680 2024-04-24T23:29:08.711 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.708+0000 7fdd097fa640 1 -- 172.21.15.80:0/3369536162 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fdcfc0037a0 con 0x7fdd0c100680 2024-04-24T23:29:08.712 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.709+0000 7fdd097fa640 1 --2- 172.21.15.80:0/3369536162 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fdcf80a2b90 0x7fdcf80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:08.712 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.709+0000 7fdd03fff640 1 --2- 172.21.15.80:0/3369536162 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fdcf80a2b90 0x7fdcf80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:08.712 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.709+0000 7fdd097fa640 1 -- 172.21.15.80:0/3369536162 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fdcfc0cf8a0 con 0x7fdd0c100680 2024-04-24T23:29:08.712 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.709+0000 7fdd03fff640 1 --2- 172.21.15.80:0/3369536162 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fdcf80a2b90 0x7fdcf80a5050 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fdcf4005fd0 tx=0x7fdcf4005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:08.714 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.713+0000 7fdd097fa640 1 -- 172.21.15.80:0/3369536162 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fdcfc09c430 con 0x7fdd0c100680 2024-04-24T23:29:08.926 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.925+0000 7fdd11f16640 1 -- 172.21.15.80:0/3369536162 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdd0c1a02f0 con 0x7fdd0c100680 2024-04-24T23:29:08.927 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.926+0000 7fdd097fa640 1 -- 172.21.15.80:0/3369536162 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11872 (secure 0 0 0) 0x7fdcfc09bd30 con 0x7fdd0c100680 2024-04-24T23:29:08.927 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-24T23:29:08.927 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/891591085","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":891591085},{"type":"v1","addr":"172.21.15.80:6835","nonce":891591085}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6837/259529026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6836","nonce":259529026},{"type":"v1","addr":"172.21.15.80:6837","nonce":259529026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14646,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6839/4194284854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6838","nonce":4194284854},{"type":"v1","addr":"172.21.15.80:6839","nonce":4194284854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14658,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6841/3145730958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6840","nonce":3145730958},{"type":"v1","addr":"172.21.15.80:6841","nonce":3145730958}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24395,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/3736784385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3736784385},{"type":"v1","addr":"172.21.15.134:6833","nonce":3736784385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/124777462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":124777462},{"type":"v1","addr":"172.21.15.148:6833","nonce":124777462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T23:27:19.300566+0000","modified":"2024-04-24T23:27:45.981839+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24496,"mds_1":24488,"mds_2":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/1399801120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1399801120},{"type":"v1","addr":"172.21.15.134:6835","nonce":1399801120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/430512704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":430512704},{"type":"v1","addr":"172.21.15.134:6837","nonce":430512704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/3002319310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3002319310},{"type":"v1","addr":"172.21.15.148:6835","nonce":3002319310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.148:6837/856528403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":856528403},{"type":"v1","addr":"172.21.15.148:6837","nonce":856528403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6839/1743815021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":1743815021},{"type":"v1","addr":"172.21.15.134:6839","nonce":1743815021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.148:6839/2952757594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":2952757594},{"type":"v1","addr":"172.21.15.148:6839","nonce":2952757594}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24496,"qdb_cluster":[24472,24488,24496]},"id":1}]} 2024-04-24T23:29:08.927 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 24 2024-04-24T23:29:08.929 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.928+0000 7fdd11f16640 1 -- 172.21.15.80:0/3369536162 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fdcf80a2b90 msgr2=0x7fdcf80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:08.929 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.928+0000 7fdd11f16640 1 --2- 172.21.15.80:0/3369536162 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fdcf80a2b90 0x7fdcf80a5050 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fdcf4005fd0 tx=0x7fdcf4005850 comp rx=0 tx=0).stop 2024-04-24T23:29:08.929 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.928+0000 7fdd11f16640 1 -- 172.21.15.80:0/3369536162 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdd0c100680 msgr2=0x7fdd0c197a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:08.929 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.928+0000 7fdd11f16640 1 --2- 172.21.15.80:0/3369536162 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdd0c100680 0x7fdd0c197a90 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fdcfc00e9c0 tx=0x7fdcfc00ee80 comp rx=0 tx=0).stop 2024-04-24T23:29:08.929 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.928+0000 7fdd11f16640 1 -- 172.21.15.80:0/3369536162 shutdown_connections 2024-04-24T23:29:08.930 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.928+0000 7fdd11f16640 1 --2- 172.21.15.80:0/3369536162 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fdcf80a2b90 0x7fdcf80a5050 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:08.930 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.928+0000 7fdd11f16640 1 --2- 172.21.15.80:0/3369536162 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd0c102230 0x7fdd0c19f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:08.930 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.928+0000 7fdd11f16640 1 --2- 172.21.15.80:0/3369536162 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdd0c101870 0x7fdd0c197fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:08.930 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.928+0000 7fdd11f16640 1 --2- 172.21.15.80:0/3369536162 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdd0c100680 0x7fdd0c197a90 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:08.930 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.928+0000 7fdd11f16640 1 -- 172.21.15.80:0/3369536162 >> 172.21.15.80:0/3369536162 conn(0x7fdd0c0fbbf0 msgr2=0x7fdd0c0fd660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:08.930 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.928+0000 7fdd11f16640 1 -- 172.21.15.80:0/3369536162 shutdown_connections 2024-04-24T23:29:08.930 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:08.928+0000 7fdd11f16640 1 -- 172.21.15.80:0/3369536162 wait complete. 2024-04-24T23:29:08.941 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:29:09.040 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.037+0000 7f993e4bd640 1 -- 172.21.15.80:0/2114358964 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9938101870 msgr2=0x7f9938101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:09.040 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.037+0000 7f993e4bd640 1 --2- 172.21.15.80:0/2114358964 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9938101870 0x7f9938101cf0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f992c009900 tx=0x7f992c02a410 comp rx=0 tx=0).stop 2024-04-24T23:29:09.041 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.037+0000 7f993e4bd640 1 -- 172.21.15.80:0/2114358964 shutdown_connections 2024-04-24T23:29:09.041 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.037+0000 7f993e4bd640 1 --2- 172.21.15.80:0/2114358964 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9938102230 0x7f9938108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:09.041 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.037+0000 7f993e4bd640 1 --2- 172.21.15.80:0/2114358964 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9938101870 0x7f9938101cf0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:09.041 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.037+0000 7f993e4bd640 1 --2- 172.21.15.80:0/2114358964 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9938100680 0x7f9938100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:09.041 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.037+0000 7f993e4bd640 1 -- 172.21.15.80:0/2114358964 >> 172.21.15.80:0/2114358964 conn(0x7f99380fbbf0 msgr2=0x7f99380fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:09.041 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.037+0000 7f993e4bd640 1 -- 172.21.15.80:0/2114358964 shutdown_connections 2024-04-24T23:29:09.041 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.037+0000 7f993e4bd640 1 -- 172.21.15.80:0/2114358964 wait complete. 2024-04-24T23:29:09.041 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.038+0000 7f993e4bd640 1 Processor -- start 2024-04-24T23:29:09.041 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.038+0000 7f993e4bd640 1 -- start start 2024-04-24T23:29:09.041 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.038+0000 7f993e4bd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9938100680 0x7f9938197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:09.041 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.038+0000 7f993e4bd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9938101870 0x7f9938197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:09.041 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.038+0000 7f993e4bd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9938102230 0x7f993819f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:09.041 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.038+0000 7f993e4bd640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f993810c2e0 con 0x7f9938102230 2024-04-24T23:29:09.042 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.038+0000 7f993e4bd640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f993810c160 con 0x7f9938100680 2024-04-24T23:29:09.042 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.038+0000 7f993e4bd640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f993810c460 con 0x7f9938101870 2024-04-24T23:29:09.042 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.038+0000 7f993ca5a640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9938102230 0x7f993819f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:09.042 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.038+0000 7f993ca5a640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9938102230 0x7f993819f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:57330/0 (socket says 172.21.15.80:57330) 2024-04-24T23:29:09.042 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.038+0000 7f993ca5a640 1 -- 172.21.15.80:0/4214137511 learned_addr learned my addr 172.21.15.80:0/4214137511 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:29:09.042 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.038+0000 7f99377fe640 1 --2- 172.21.15.80:0/4214137511 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9938101870 0x7f9938197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:09.042 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.039+0000 7f9937fff640 1 --2- 172.21.15.80:0/4214137511 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9938100680 0x7f9938197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:09.042 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.039+0000 7f993ca5a640 1 -- 172.21.15.80:0/4214137511 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9938101870 msgr2=0x7f9938197fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:09.042 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.039+0000 7f993ca5a640 1 --2- 172.21.15.80:0/4214137511 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9938101870 0x7f9938197fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:09.042 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.039+0000 7f993ca5a640 1 -- 172.21.15.80:0/4214137511 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9938100680 msgr2=0x7f9938197aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:09.042 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.039+0000 7f993ca5a640 1 --2- 172.21.15.80:0/4214137511 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9938100680 0x7f9938197aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:09.042 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.039+0000 7f993ca5a640 1 -- 172.21.15.80:0/4214137511 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f993819f6c0 con 0x7f9938102230 2024-04-24T23:29:09.042 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.039+0000 7f9937fff640 1 --2- 172.21.15.80:0/4214137511 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9938100680 0x7f9938197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:29:09.042 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.039+0000 7f993ca5a640 1 --2- 172.21.15.80:0/4214137511 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9938102230 0x7f993819f050 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f993000cad0 tx=0x7f993000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:09.042 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.039+0000 7f99357fa640 1 -- 172.21.15.80:0/4214137511 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f993000ae20 con 0x7f9938102230 2024-04-24T23:29:09.043 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.039+0000 7f993e4bd640 1 -- 172.21.15.80:0/4214137511 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f993819f9b0 con 0x7f9938102230 2024-04-24T23:29:09.043 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.039+0000 7f99357fa640 1 -- 172.21.15.80:0/4214137511 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9930005020 con 0x7f9938102230 2024-04-24T23:29:09.043 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.039+0000 7f99357fa640 1 -- 172.21.15.80:0/4214137511 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f993000f6f0 con 0x7f9938102230 2024-04-24T23:29:09.043 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.039+0000 7f993e4bd640 1 -- 172.21.15.80:0/4214137511 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f993819fed0 con 0x7f9938102230 2024-04-24T23:29:09.044 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.040+0000 7f993e4bd640 1 -- 172.21.15.80:0/4214137511 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99381a00a0 con 0x7f9938102230 2024-04-24T23:29:09.044 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.041+0000 7f99357fa640 1 -- 172.21.15.80:0/4214137511 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f99300051d0 con 0x7f9938102230 2024-04-24T23:29:09.044 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.042+0000 7f99357fa640 1 --2- 172.21.15.80:0/4214137511 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f99200a2b90 0x7f99200a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:09.044 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.042+0000 7f9937fff640 1 --2- 172.21.15.80:0/4214137511 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f99200a2b90 0x7f99200a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:09.044 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.042+0000 7f99357fa640 1 -- 172.21.15.80:0/4214137511 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f99300cf4d0 con 0x7f9938102230 2024-04-24T23:29:09.044 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.042+0000 7f9937fff640 1 --2- 172.21.15.80:0/4214137511 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f99200a2b90 0x7f99200a5050 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f99280058c0 tx=0x7f9928005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:09.047 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.047+0000 7f99357fa640 1 -- 172.21.15.80:0/4214137511 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f9930014030 con 0x7f9938102230 2024-04-24T23:29:09.229 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.228+0000 7f993e4bd640 1 -- 172.21.15.80:0/4214137511 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f99381a0270 con 0x7f9938102230 2024-04-24T23:29:09.231 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.230+0000 7f99357fa640 1 -- 172.21.15.80:0/4214137511 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24839 (secure 0 0 0) 0x7f993009c060 con 0x7f9938102230 2024-04-24T23:29:09.231 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-24T23:29:09.231 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":94,"fsid":"ec7885aa-0290-11ef-bc93-c7b262605968","created":"2024-04-24T23:19:05.586407+0000","modified":"2024-04-24T23:29:08.209684+0000","last_up_change":"2024-04-24T23:26:25.938316+0000","last_in_change":"2024-04-24T23:26:04.786699+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:22:33.124428+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:27:17.575788+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:27:18.492762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T23:27:22.438503+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a525741d-b46c-4eef-aa64-3f057cac1497","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":2597718112},{"type":"v1","addr":"172.21.15.80:6803","nonce":2597718112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":2597718112},{"type":"v1","addr":"172.21.15.80:6805","nonce":2597718112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2597718112},{"type":"v1","addr":"172.21.15.80:6809","nonce":2597718112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":2597718112},{"type":"v1","addr":"172.21.15.80:6807","nonce":2597718112}]},"public_addr":"172.21.15.80:6803/2597718112","cluster_addr":"172.21.15.80:6805/2597718112","heartbeat_back_addr":"172.21.15.80:6809/2597718112","heartbeat_front_addr":"172.21.15.80:6807/2597718112","state":["exists","up"]},{"osd":1,"uuid":"2d0f0d67-6db4-40ac-91cc-58604ab6795a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":2395610201},{"type":"v1","addr":"172.21.15.134:6801","nonce":2395610201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":2395610201},{"type":"v1","addr":"172.21.15.134:6803","nonce":2395610201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":2395610201},{"type":"v1","addr":"172.21.15.134:6807","nonce":2395610201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":2395610201},{"type":"v1","addr":"172.21.15.134:6805","nonce":2395610201}]},"public_addr":"172.21.15.134:6801/2395610201","cluster_addr":"172.21.15.134:6803/2395610201","heartbeat_back_addr":"172.21.15.134:6807/2395610201","heartbeat_front_addr":"172.21.15.134:6805/2395610201","state":["exists","up"]},{"osd":2,"uuid":"781fb879-fedc-4c84-82fb-59e073bc4cf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1629152187},{"type":"v1","addr":"172.21.15.148:6801","nonce":1629152187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1629152187},{"type":"v1","addr":"172.21.15.148:6803","nonce":1629152187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1629152187},{"type":"v1","addr":"172.21.15.148:6807","nonce":1629152187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1629152187},{"type":"v1","addr":"172.21.15.148:6805","nonce":1629152187}]},"public_addr":"172.21.15.148:6801/1629152187","cluster_addr":"172.21.15.148:6803/1629152187","heartbeat_back_addr":"172.21.15.148:6807/1629152187","heartbeat_front_addr":"172.21.15.148:6805/1629152187","state":["exists","up"]},{"osd":3,"uuid":"cd982757-f2f6-4296-b3e7-4cd40fbd8d9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":3128298510},{"type":"v1","addr":"172.21.15.80:6811","nonce":3128298510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":3128298510},{"type":"v1","addr":"172.21.15.80:6813","nonce":3128298510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":3128298510},{"type":"v1","addr":"172.21.15.80:6817","nonce":3128298510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":3128298510},{"type":"v1","addr":"172.21.15.80:6815","nonce":3128298510}]},"public_addr":"172.21.15.80:6811/3128298510","cluster_addr":"172.21.15.80:6813/3128298510","heartbeat_back_addr":"172.21.15.80:6817/3128298510","heartbeat_front_addr":"172.21.15.80:6815/3128298510","state":["exists","up"]},{"osd":4,"uuid":"dfc55a79-a7e8-417a-b898-ec44171a54af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1917556060},{"type":"v1","addr":"172.21.15.134:6809","nonce":1917556060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1917556060},{"type":"v1","addr":"172.21.15.134:6811","nonce":1917556060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1917556060},{"type":"v1","addr":"172.21.15.134:6815","nonce":1917556060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1917556060},{"type":"v1","addr":"172.21.15.134:6813","nonce":1917556060}]},"public_addr":"172.21.15.134:6809/1917556060","cluster_addr":"172.21.15.134:6811/1917556060","heartbeat_back_addr":"172.21.15.134:6815/1917556060","heartbeat_front_addr":"172.21.15.134:6813/1917556060","state":["exists","up"]},{"osd":5,"uuid":"b5491337-0694-440c-b679-312f3d14cf65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":2715951281},{"type":"v1","addr":"172.21.15.148:6809","nonce":2715951281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":2715951281},{"type":"v1","addr":"172.21.15.148:6811","nonce":2715951281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":2715951281},{"type":"v1","addr":"172.21.15.148:6815","nonce":2715951281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":2715951281},{"type":"v1","addr":"172.21.15.148:6813","nonce":2715951281}]},"public_addr":"172.21.15.148:6809/2715951281","cluster_addr":"172.21.15.148:6811/2715951281","heartbeat_back_addr":"172.21.15.148:6815/2715951281","heartbeat_front_addr":"172.21.15.148:6813/2715951281","state":["exists","up"]},{"osd":6,"uuid":"ba4b5df7-d86e-47cd-ab67-cf763c4e4d6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":953261724},{"type":"v1","addr":"172.21.15.80:6819","nonce":953261724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":953261724},{"type":"v1","addr":"172.21.15.80:6821","nonce":953261724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":953261724},{"type":"v1","addr":"172.21.15.80:6825","nonce":953261724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":953261724},{"type":"v1","addr":"172.21.15.80:6823","nonce":953261724}]},"public_addr":"172.21.15.80:6819/953261724","cluster_addr":"172.21.15.80:6821/953261724","heartbeat_back_addr":"172.21.15.80:6825/953261724","heartbeat_front_addr":"172.21.15.80:6823/953261724","state":["exists","up"]},{"osd":7,"uuid":"464a503a-75ae-49e7-ab6e-a7f9f2a6a307","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":108354844},{"type":"v1","addr":"172.21.15.134:6817","nonce":108354844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":108354844},{"type":"v1","addr":"172.21.15.134:6819","nonce":108354844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":108354844},{"type":"v1","addr":"172.21.15.134:6823","nonce":108354844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":108354844},{"type":"v1","addr":"172.21.15.134:6821","nonce":108354844}]},"public_addr":"172.21.15.134:6817/108354844","cluster_addr":"172.21.15.134:6819/108354844","heartbeat_back_addr":"172.21.15.134:6823/108354844","heartbeat_front_addr":"172.21.15.134:6821/108354844","state":["exists","up"]},{"osd":8,"uuid":"ce544b2e-4540-47c0-9997-500c45351330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3559655329},{"type":"v1","addr":"172.21.15.148:6817","nonce":3559655329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":3559655329},{"type":"v1","addr":"172.21.15.148:6819","nonce":3559655329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":3559655329},{"type":"v1","addr":"172.21.15.148:6823","nonce":3559655329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3559655329},{"type":"v1","addr":"172.21.15.148:6821","nonce":3559655329}]},"public_addr":"172.21.15.148:6817/3559655329","cluster_addr":"172.21.15.148:6819/3559655329","heartbeat_back_addr":"172.21.15.148:6823/3559655329","heartbeat_front_addr":"172.21.15.148:6821/3559655329","state":["exists","up"]},{"osd":9,"uuid":"06de3732-19b9-496c-a464-55322efa6759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":4019913545},{"type":"v1","addr":"172.21.15.80:6827","nonce":4019913545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":4019913545},{"type":"v1","addr":"172.21.15.80:6829","nonce":4019913545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":4019913545},{"type":"v1","addr":"172.21.15.80:6833","nonce":4019913545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":4019913545},{"type":"v1","addr":"172.21.15.80:6831","nonce":4019913545}]},"public_addr":"172.21.15.80:6827/4019913545","cluster_addr":"172.21.15.80:6829/4019913545","heartbeat_back_addr":"172.21.15.80:6833/4019913545","heartbeat_front_addr":"172.21.15.80:6831/4019913545","state":["exists","up"]},{"osd":10,"uuid":"8d92f0d9-5c64-429a-b750-c33ae0e070c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":647343766},{"type":"v1","addr":"172.21.15.134:6825","nonce":647343766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":647343766},{"type":"v1","addr":"172.21.15.134:6827","nonce":647343766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":647343766},{"type":"v1","addr":"172.21.15.134:6831","nonce":647343766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":647343766},{"type":"v1","addr":"172.21.15.134:6829","nonce":647343766}]},"public_addr":"172.21.15.134:6825/647343766","cluster_addr":"172.21.15.134:6827/647343766","heartbeat_back_addr":"172.21.15.134:6831/647343766","heartbeat_front_addr":"172.21.15.134:6829/647343766","state":["exists","up"]},{"osd":11,"uuid":"4c003a41-f65b-479f-bc80-b27d680cf936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":4068024116},{"type":"v1","addr":"172.21.15.148:6825","nonce":4068024116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":4068024116},{"type":"v1","addr":"172.21.15.148:6827","nonce":4068024116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":4068024116},{"type":"v1","addr":"172.21.15.148:6831","nonce":4068024116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":4068024116},{"type":"v1","addr":"172.21.15.148:6829","nonce":4068024116}]},"public_addr":"172.21.15.148:6825/4068024116","cluster_addr":"172.21.15.148:6827/4068024116","heartbeat_back_addr":"172.21.15.148:6831/4068024116","heartbeat_front_addr":"172.21.15.148:6829/4068024116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:21:42.433631+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:22:06.145933+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:22:30.726143+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:22:55.729961+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:23:21.280913+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:23:46.609983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:24:12.585247+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:24:38.359287+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:25:04.491232+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:25:30.788089+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:25:56.891078+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:26:23.899113+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":7}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"4.a","mappings":[{"from":5,"to":8}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.80:0/2828378749":"2024-04-25T23:20:06.793609+0000","172.21.15.80:6801/233364790":"2024-04-25T23:20:06.793609+0000","172.21.15.80:6800/1614691410":"2024-04-25T23:19:40.490692+0000","172.21.15.80:0/1406187145":"2024-04-25T23:20:06.793609+0000","172.21.15.80:0/92953396":"2024-04-25T23:19:40.490692+0000","172.21.15.80:0/598129432":"2024-04-25T23:20:06.793609+0000","172.21.15.80:0/232193118":"2024-04-25T23:19:40.490692+0000","172.21.15.80:6800/233364790":"2024-04-25T23:20:06.793609+0000","172.21.15.80:6801/1614691410":"2024-04-25T23:19:40.490692+0000","172.21.15.80:0/2081562529":"2024-04-25T23:19:40.490692+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:29:09.233 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.232+0000 7f993e4bd640 1 -- 172.21.15.80:0/4214137511 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f99200a2b90 msgr2=0x7f99200a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:09.233 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.232+0000 7f993e4bd640 1 --2- 172.21.15.80:0/4214137511 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f99200a2b90 0x7f99200a5050 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f99280058c0 tx=0x7f9928005850 comp rx=0 tx=0).stop 2024-04-24T23:29:09.233 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.232+0000 7f993e4bd640 1 -- 172.21.15.80:0/4214137511 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9938102230 msgr2=0x7f993819f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:09.233 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.232+0000 7f993e4bd640 1 --2- 172.21.15.80:0/4214137511 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9938102230 0x7f993819f050 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f993000cad0 tx=0x7f993000cf90 comp rx=0 tx=0).stop 2024-04-24T23:29:09.233 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.232+0000 7f993e4bd640 1 -- 172.21.15.80:0/4214137511 shutdown_connections 2024-04-24T23:29:09.233 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.232+0000 7f993e4bd640 1 --2- 172.21.15.80:0/4214137511 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f99200a2b90 0x7f99200a5050 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:09.233 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.232+0000 7f993e4bd640 1 --2- 172.21.15.80:0/4214137511 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9938102230 0x7f993819f050 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:09.233 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.232+0000 7f993e4bd640 1 --2- 172.21.15.80:0/4214137511 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9938101870 0x7f9938197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:09.234 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.232+0000 7f993e4bd640 1 --2- 172.21.15.80:0/4214137511 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9938100680 0x7f9938197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:09.234 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.232+0000 7f993e4bd640 1 -- 172.21.15.80:0/4214137511 >> 172.21.15.80:0/4214137511 conn(0x7f99380fbbf0 msgr2=0x7f99380688d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:09.234 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.232+0000 7f993e4bd640 1 -- 172.21.15.80:0/4214137511 shutdown_connections 2024-04-24T23:29:09.234 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.232+0000 7f993e4bd640 1 -- 172.21.15.80:0/4214137511 wait complete. 2024-04-24T23:29:09.244 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:22:33.124428+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T23:27:17.575788+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T23:27:18.492762+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-24T23:27:22.438503+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-24T23:29:09.245 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:29:09.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.344+0000 7f36d6b14640 1 -- 172.21.15.80:0/530996804 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f36d0102230 msgr2=0x7f36d0108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:09.348 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.344+0000 7f36d6b14640 1 --2- 172.21.15.80:0/530996804 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f36d0102230 0x7f36d0108ac0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f36c8009ca0 tx=0x7f36c802a8c0 comp rx=0 tx=0).stop 2024-04-24T23:29:09.348 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.344+0000 7f36d6b14640 1 -- 172.21.15.80:0/530996804 shutdown_connections 2024-04-24T23:29:09.348 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.344+0000 7f36d6b14640 1 --2- 172.21.15.80:0/530996804 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f36d0102230 0x7f36d0108ac0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:09.348 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.344+0000 7f36d6b14640 1 --2- 172.21.15.80:0/530996804 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36d0101870 0x7f36d0101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:09.348 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.344+0000 7f36d6b14640 1 --2- 172.21.15.80:0/530996804 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f36d0100680 0x7f36d0100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:09.348 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.344+0000 7f36d6b14640 1 -- 172.21.15.80:0/530996804 >> 172.21.15.80:0/530996804 conn(0x7f36d00fbbf0 msgr2=0x7f36d00fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:09.348 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.344+0000 7f36d6b14640 1 -- 172.21.15.80:0/530996804 shutdown_connections 2024-04-24T23:29:09.348 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.344+0000 7f36d6b14640 1 -- 172.21.15.80:0/530996804 wait complete. 2024-04-24T23:29:09.348 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.345+0000 7f36d6b14640 1 Processor -- start 2024-04-24T23:29:09.348 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.345+0000 7f36d6b14640 1 -- start start 2024-04-24T23:29:09.348 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.345+0000 7f36d6b14640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f36d0100680 0x7f36d0197c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:09.348 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.345+0000 7f36d6b14640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f36d0101870 0x7f36d0198160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:09.349 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.345+0000 7f36d6b14640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36d0102230 0x7f36d019f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:09.349 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.345+0000 7f36d6b14640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f36d010d500 con 0x7f36d0100680 2024-04-24T23:29:09.349 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.345+0000 7f36d6b14640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f36d010d380 con 0x7f36d0101870 2024-04-24T23:29:09.349 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.345+0000 7f36d6b14640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f36d010d680 con 0x7f36d0102230 2024-04-24T23:29:09.349 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.345+0000 7f36d48b0640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f36d0100680 0x7f36d0197c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:09.349 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.345+0000 7f36d48b0640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f36d0100680 0x7f36d0197c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:57344/0 (socket says 172.21.15.80:57344) 2024-04-24T23:29:09.349 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.345+0000 7f36d48b0640 1 -- 172.21.15.80:0/422354391 learned_addr learned my addr 172.21.15.80:0/422354391 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:29:09.349 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.345+0000 7f36cffff640 1 --2- 172.21.15.80:0/422354391 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f36d0101870 0x7f36d0198160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:09.349 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.345+0000 7f36d50b1640 1 --2- 172.21.15.80:0/422354391 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36d0102230 0x7f36d019f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:09.349 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.346+0000 7f36d48b0640 1 -- 172.21.15.80:0/422354391 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36d0102230 msgr2=0x7f36d019f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:09.349 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.346+0000 7f36d48b0640 1 --2- 172.21.15.80:0/422354391 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36d0102230 0x7f36d019f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:09.349 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.346+0000 7f36d48b0640 1 -- 172.21.15.80:0/422354391 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f36d0101870 msgr2=0x7f36d0198160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:09.349 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.346+0000 7f36d48b0640 1 --2- 172.21.15.80:0/422354391 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f36d0101870 0x7f36d0198160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:09.349 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.346+0000 7f36d48b0640 1 -- 172.21.15.80:0/422354391 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36d019f830 con 0x7f36d0100680 2024-04-24T23:29:09.350 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.346+0000 7f36cffff640 1 --2- 172.21.15.80:0/422354391 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f36d0101870 0x7f36d0198160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:29:09.350 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.346+0000 7f36d48b0640 1 --2- 172.21.15.80:0/422354391 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f36d0100680 0x7f36d0197c20 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f36c400b450 tx=0x7f36c400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:09.350 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.346+0000 7f36cdffb640 1 -- 172.21.15.80:0/422354391 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36c4015020 con 0x7f36d0100680 2024-04-24T23:29:09.350 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.346+0000 7f36d6b14640 1 -- 172.21.15.80:0/422354391 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36d019fb20 con 0x7f36d0100680 2024-04-24T23:29:09.350 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.346+0000 7f36cdffb640 1 -- 172.21.15.80:0/422354391 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36c4011d70 con 0x7f36d0100680 2024-04-24T23:29:09.350 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.346+0000 7f36cdffb640 1 -- 172.21.15.80:0/422354391 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36c400f340 con 0x7f36d0100680 2024-04-24T23:29:09.350 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.346+0000 7f36d6b14640 1 -- 172.21.15.80:0/422354391 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36d01a0070 con 0x7f36d0100680 2024-04-24T23:29:09.350 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.347+0000 7f36d6b14640 1 -- 172.21.15.80:0/422354391 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36d0107a70 con 0x7f36d0100680 2024-04-24T23:29:09.351 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.348+0000 7f36cdffb640 1 -- 172.21.15.80:0/422354391 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f36c4018020 con 0x7f36d0100680 2024-04-24T23:29:09.351 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.348+0000 7f36cdffb640 1 --2- 172.21.15.80:0/422354391 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f36c00a2b90 0x7f36c00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:09.351 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.349+0000 7f36cffff640 1 --2- 172.21.15.80:0/422354391 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f36c00a2b90 0x7f36c00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:09.351 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.349+0000 7f36cdffb640 1 -- 172.21.15.80:0/422354391 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f36c40cf000 con 0x7f36d0100680 2024-04-24T23:29:09.351 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.349+0000 7f36cffff640 1 --2- 172.21.15.80:0/422354391 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f36c00a2b90 0x7f36c00a5050 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f36bc009930 tx=0x7f36bc008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:09.353 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.353+0000 7f36cdffb640 1 -- 172.21.15.80:0/422354391 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f36c401f020 con 0x7f36d0100680 2024-04-24T23:29:09.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:29:09 smithi148 ceph-mon[32528]: osdmap e94: 12 total, 12 up, 12 in 2024-04-24T23:29:09.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:29:09 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/965829966' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T23:29:09.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:29:09 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/1138428572' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T23:29:09.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:29:09 smithi148 ceph-mon[32528]: pgmap v355: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 639 B/s wr, 7 op/s 2024-04-24T23:29:09.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:29:09 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/3369536162' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:29:09.531 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.530+0000 7f36d6b14640 1 -- 172.21.15.80:0/422354391 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f36d002d170 con 0x7f36d0100680 2024-04-24T23:29:09.531 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.530+0000 7f36cdffb640 1 -- 172.21.15.80:0/422354391 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7f36c400d040 con 0x7f36d0100680 2024-04-24T23:29:09.531 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 1 2024-04-24T23:29:09.534 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.532+0000 7f36d6b14640 1 -- 172.21.15.80:0/422354391 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f36c00a2b90 msgr2=0x7f36c00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:09.534 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.532+0000 7f36d6b14640 1 --2- 172.21.15.80:0/422354391 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f36c00a2b90 0x7f36c00a5050 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f36bc009930 tx=0x7f36bc008040 comp rx=0 tx=0).stop 2024-04-24T23:29:09.534 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.532+0000 7f36d6b14640 1 -- 172.21.15.80:0/422354391 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f36d0100680 msgr2=0x7f36d0197c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:09.534 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.532+0000 7f36d6b14640 1 --2- 172.21.15.80:0/422354391 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f36d0100680 0x7f36d0197c20 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f36c400b450 tx=0x7f36c400b910 comp rx=0 tx=0).stop 2024-04-24T23:29:09.534 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.532+0000 7f36d6b14640 1 -- 172.21.15.80:0/422354391 shutdown_connections 2024-04-24T23:29:09.534 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.532+0000 7f36d6b14640 1 --2- 172.21.15.80:0/422354391 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f36c00a2b90 0x7f36c00a5050 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:09.534 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.532+0000 7f36d6b14640 1 --2- 172.21.15.80:0/422354391 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36d0102230 0x7f36d019f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:09.534 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.532+0000 7f36d6b14640 1 --2- 172.21.15.80:0/422354391 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f36d0101870 0x7f36d0198160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:09.534 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.532+0000 7f36d6b14640 1 --2- 172.21.15.80:0/422354391 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f36d0100680 0x7f36d0197c20 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:09.534 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.532+0000 7f36d6b14640 1 -- 172.21.15.80:0/422354391 >> 172.21.15.80:0/422354391 conn(0x7f36d00fbbf0 msgr2=0x7f36d0104c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:09.535 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.532+0000 7f36d6b14640 1 -- 172.21.15.80:0/422354391 shutdown_connections 2024-04-24T23:29:09.535 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.532+0000 7f36d6b14640 1 -- 172.21.15.80:0/422354391 wait complete. 2024-04-24T23:29:09.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:29:09 smithi080 ceph-mon[29152]: osdmap e94: 12 total, 12 up, 12 in 2024-04-24T23:29:09.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:29:09 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/965829966' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T23:29:09.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:29:09 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/1138428572' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T23:29:09.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:29:09 smithi080 ceph-mon[29152]: pgmap v355: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 639 B/s wr, 7 op/s 2024-04-24T23:29:09.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:29:09 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/3369536162' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:29:09.544 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T23:29:09.643 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.642+0000 7fa64ceee640 1 -- 172.21.15.80:0/2792614262 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa648102980 msgr2=0x7fa648102e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:09.643 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.642+0000 7fa64ceee640 1 --2- 172.21.15.80:0/2792614262 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa648102980 0x7fa648102e00 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fa63c009900 tx=0x7fa63c02a410 comp rx=0 tx=0).stop 2024-04-24T23:29:09.644 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.642+0000 7fa64ceee640 1 -- 172.21.15.80:0/2792614262 shutdown_connections 2024-04-24T23:29:09.644 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.642+0000 7fa64ceee640 1 --2- 172.21.15.80:0/2792614262 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa648103340 0x7fa648109bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:09.644 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.642+0000 7fa64ceee640 1 --2- 172.21.15.80:0/2792614262 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa648102980 0x7fa648102e00 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:09.644 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.642+0000 7fa64ceee640 1 --2- 172.21.15.80:0/2792614262 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa648101790 0x7fa648101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:09.644 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.642+0000 7fa64ceee640 1 -- 172.21.15.80:0/2792614262 >> 172.21.15.80:0/2792614262 conn(0x7fa6480fcd20 msgr2=0x7fa6480ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:09.644 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.642+0000 7fa64ceee640 1 -- 172.21.15.80:0/2792614262 shutdown_connections 2024-04-24T23:29:09.644 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.642+0000 7fa64ceee640 1 -- 172.21.15.80:0/2792614262 wait complete. 2024-04-24T23:29:09.646 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.643+0000 7fa64ceee640 1 Processor -- start 2024-04-24T23:29:09.647 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.643+0000 7fa64ceee640 1 -- start start 2024-04-24T23:29:09.647 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.643+0000 7fa64ceee640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa648101790 0x7fa648197a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:09.647 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.643+0000 7fa64ceee640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa648102980 0x7fa648197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:09.647 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.643+0000 7fa64ceee640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa648103340 0x7fa64819efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:09.647 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.643+0000 7fa64ceee640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fa64810d3f0 con 0x7fa648103340 2024-04-24T23:29:09.647 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.643+0000 7fa64ceee640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa64810d270 con 0x7fa648101790 2024-04-24T23:29:09.647 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.643+0000 7fa64ceee640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa64810d570 con 0x7fa648102980 2024-04-24T23:29:09.647 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.643+0000 7fa646d9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa648103340 0x7fa64819efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:09.647 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.643+0000 7fa646d9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa648103340 0x7fa64819efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:57370/0 (socket says 172.21.15.80:57370) 2024-04-24T23:29:09.647 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.643+0000 7fa646d9d640 1 -- 172.21.15.80:0/4065823044 learned_addr learned my addr 172.21.15.80:0/4065823044 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:29:09.647 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.643+0000 7fa645d9b640 1 --2- 172.21.15.80:0/4065823044 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa648102980 0x7fa648197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:09.647 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.644+0000 7fa64659c640 1 --2- 172.21.15.80:0/4065823044 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa648101790 0x7fa648197a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:09.647 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.644+0000 7fa646d9d640 1 -- 172.21.15.80:0/4065823044 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa648102980 msgr2=0x7fa648197f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:09.648 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.644+0000 7fa646d9d640 1 --2- 172.21.15.80:0/4065823044 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa648102980 0x7fa648197f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:09.648 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.644+0000 7fa646d9d640 1 -- 172.21.15.80:0/4065823044 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa648101790 msgr2=0x7fa648197a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:09.648 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.644+0000 7fa646d9d640 1 --2- 172.21.15.80:0/4065823044 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa648101790 0x7fa648197a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:09.648 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.644+0000 7fa646d9d640 1 -- 172.21.15.80:0/4065823044 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa64819f630 con 0x7fa648103340 2024-04-24T23:29:09.648 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.644+0000 7fa64659c640 1 --2- 172.21.15.80:0/4065823044 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa648101790 0x7fa648197a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:29:09.648 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.644+0000 7fa646d9d640 1 --2- 172.21.15.80:0/4065823044 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa648103340 0x7fa64819efc0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fa64000d750 tx=0x7fa64000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:09.648 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.644+0000 7fa6377fe640 1 -- 172.21.15.80:0/4065823044 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa640017070 con 0x7fa648103340 2024-04-24T23:29:09.648 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.644+0000 7fa64ceee640 1 -- 172.21.15.80:0/4065823044 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa64819f920 con 0x7fa648103340 2024-04-24T23:29:09.648 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.644+0000 7fa6377fe640 1 -- 172.21.15.80:0/4065823044 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa640005500 con 0x7fa648103340 2024-04-24T23:29:09.648 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.644+0000 7fa6377fe640 1 -- 172.21.15.80:0/4065823044 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa640004d90 con 0x7fa648103340 2024-04-24T23:29:09.648 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.644+0000 7fa64ceee640 1 -- 172.21.15.80:0/4065823044 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa64819fe40 con 0x7fa648103340 2024-04-24T23:29:09.648 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.645+0000 7fa64ceee640 1 -- 172.21.15.80:0/4065823044 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa648108ab0 con 0x7fa648103340 2024-04-24T23:29:09.648 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.646+0000 7fa6377fe640 1 -- 172.21.15.80:0/4065823044 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa640004f30 con 0x7fa648103340 2024-04-24T23:29:09.649 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.647+0000 7fa6377fe640 1 --2- 172.21.15.80:0/4065823044 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fa6300a2b90 0x7fa6300a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:09.649 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.647+0000 7fa6377fe640 1 -- 172.21.15.80:0/4065823044 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fa6400cf200 con 0x7fa648103340 2024-04-24T23:29:09.649 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.647+0000 7fa64659c640 1 --2- 172.21.15.80:0/4065823044 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fa6300a2b90 0x7fa6300a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:09.649 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.647+0000 7fa64659c640 1 --2- 172.21.15.80:0/4065823044 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fa6300a2b90 0x7fa6300a5050 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa6481027e0 tx=0x7fa638007870 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:09.651 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.650+0000 7fa6377fe640 1 -- 172.21.15.80:0/4065823044 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa64009bd90 con 0x7fa648103340 2024-04-24T23:29:09.697 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:29:09 smithi134 ceph-mon[32619]: osdmap e94: 12 total, 12 up, 12 in 2024-04-24T23:29:09.697 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:29:09 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/965829966' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T23:29:09.697 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:29:09 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/1138428572' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T23:29:09.697 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:29:09 smithi134 ceph-mon[32619]: pgmap v355: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 639 B/s wr, 7 op/s 2024-04-24T23:29:09.697 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:29:09 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/3369536162' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:29:09.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.825+0000 7fa64ceee640 1 -- 172.21.15.80:0/4065823044 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fa64802d300 con 0x7fa648103340 2024-04-24T23:29:09.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.825+0000 7fa6377fe640 1 -- 172.21.15.80:0/4065823044 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) ==== 104+0+11 (secure 0 0 0) 0x7fa64009b690 con 0x7fa648103340 2024-04-24T23:29:09.827 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 32 2024-04-24T23:29:09.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.828+0000 7fa64ceee640 1 -- 172.21.15.80:0/4065823044 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fa6300a2b90 msgr2=0x7fa6300a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:09.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.828+0000 7fa64ceee640 1 --2- 172.21.15.80:0/4065823044 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fa6300a2b90 0x7fa6300a5050 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa6481027e0 tx=0x7fa638007870 comp rx=0 tx=0).stop 2024-04-24T23:29:09.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.828+0000 7fa64ceee640 1 -- 172.21.15.80:0/4065823044 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa648103340 msgr2=0x7fa64819efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:09.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.828+0000 7fa64ceee640 1 --2- 172.21.15.80:0/4065823044 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa648103340 0x7fa64819efc0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fa64000d750 tx=0x7fa64000dc10 comp rx=0 tx=0).stop 2024-04-24T23:29:09.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.828+0000 7fa64ceee640 1 -- 172.21.15.80:0/4065823044 shutdown_connections 2024-04-24T23:29:09.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.828+0000 7fa64ceee640 1 --2- 172.21.15.80:0/4065823044 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fa6300a2b90 0x7fa6300a5050 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:09.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.828+0000 7fa64ceee640 1 --2- 172.21.15.80:0/4065823044 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa648103340 0x7fa64819efc0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:09.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.828+0000 7fa64ceee640 1 --2- 172.21.15.80:0/4065823044 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa648102980 0x7fa648197f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:09.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.828+0000 7fa64ceee640 1 --2- 172.21.15.80:0/4065823044 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa648101790 0x7fa648197a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:09.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.828+0000 7fa64ceee640 1 -- 172.21.15.80:0/4065823044 >> 172.21.15.80:0/4065823044 conn(0x7fa6480fcd20 msgr2=0x7fa648105c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:09.830 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.828+0000 7fa64ceee640 1 -- 172.21.15.80:0/4065823044 shutdown_connections 2024-04-24T23:29:09.830 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.828+0000 7fa64ceee640 1 -- 172.21.15.80:0/4065823044 wait complete. 2024-04-24T23:29:09.841 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T23:29:09.948 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.942+0000 7f4ebe4e7640 1 -- 172.21.15.80:0/1714028297 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4eb8102230 msgr2=0x7f4eb8108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:09.949 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.942+0000 7f4ebe4e7640 1 --2- 172.21.15.80:0/1714028297 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4eb8102230 0x7f4eb8108ac0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f4eb0009ca0 tx=0x7f4eb002a8c0 comp rx=0 tx=0).stop 2024-04-24T23:29:09.949 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.942+0000 7f4ebe4e7640 1 -- 172.21.15.80:0/1714028297 shutdown_connections 2024-04-24T23:29:09.949 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.942+0000 7f4ebe4e7640 1 --2- 172.21.15.80:0/1714028297 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4eb8102230 0x7f4eb8108ac0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:09.949 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.942+0000 7f4ebe4e7640 1 --2- 172.21.15.80:0/1714028297 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4eb8101870 0x7f4eb8101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:09.950 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.942+0000 7f4ebe4e7640 1 --2- 172.21.15.80:0/1714028297 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4eb8100680 0x7f4eb8100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:09.950 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.942+0000 7f4ebe4e7640 1 -- 172.21.15.80:0/1714028297 >> 172.21.15.80:0/1714028297 conn(0x7f4eb80fbbf0 msgr2=0x7f4eb80fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:09.950 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.942+0000 7f4ebe4e7640 1 -- 172.21.15.80:0/1714028297 shutdown_connections 2024-04-24T23:29:09.950 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.942+0000 7f4ebe4e7640 1 -- 172.21.15.80:0/1714028297 wait complete. 2024-04-24T23:29:09.950 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.943+0000 7f4ebe4e7640 1 Processor -- start 2024-04-24T23:29:09.950 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.943+0000 7f4ebe4e7640 1 -- start start 2024-04-24T23:29:09.950 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.943+0000 7f4ebe4e7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4eb8100680 0x7f4eb8197a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:09.950 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.943+0000 7f4ebe4e7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4eb8101870 0x7f4eb8197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:09.950 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.943+0000 7f4ebe4e7640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4eb8102230 0x7f4eb819efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:09.950 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.943+0000 7f4ebe4e7640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb810c300 con 0x7f4eb8102230 2024-04-24T23:29:09.950 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.943+0000 7f4ebe4e7640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb810c180 con 0x7f4eb8101870 2024-04-24T23:29:09.950 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.943+0000 7f4ebe4e7640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb810c480 con 0x7f4eb8100680 2024-04-24T23:29:09.950 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.943+0000 7f4ebca84640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4eb8102230 0x7f4eb819efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:09.951 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.943+0000 7f4ebca84640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4eb8102230 0x7f4eb819efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:57374/0 (socket says 172.21.15.80:57374) 2024-04-24T23:29:09.951 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.943+0000 7f4ebca84640 1 -- 172.21.15.80:0/2827917826 learned_addr learned my addr 172.21.15.80:0/2827917826 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:29:09.951 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.943+0000 7f4eb7fff640 1 --2- 172.21.15.80:0/2827917826 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4eb8100680 0x7f4eb8197a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:09.951 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.943+0000 7f4eb77fe640 1 --2- 172.21.15.80:0/2827917826 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4eb8101870 0x7f4eb8197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:09.951 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.944+0000 7f4ebca84640 1 -- 172.21.15.80:0/2827917826 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4eb8100680 msgr2=0x7f4eb8197a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:09.951 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.944+0000 7f4ebca84640 1 --2- 172.21.15.80:0/2827917826 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4eb8100680 0x7f4eb8197a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:09.951 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.944+0000 7f4ebca84640 1 -- 172.21.15.80:0/2827917826 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4eb8101870 msgr2=0x7f4eb8197f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:09.951 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.944+0000 7f4ebca84640 1 --2- 172.21.15.80:0/2827917826 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4eb8101870 0x7f4eb8197f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:09.951 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.944+0000 7f4ebca84640 1 -- 172.21.15.80:0/2827917826 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4eb819f620 con 0x7f4eb8102230 2024-04-24T23:29:09.951 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.944+0000 7f4ebca84640 1 --2- 172.21.15.80:0/2827917826 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4eb8102230 0x7f4eb819efb0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f4eb0005e30 tx=0x7f4eb0031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:09.952 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.944+0000 7f4eb57fa640 1 -- 172.21.15.80:0/2827917826 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb0003c30 con 0x7f4eb8102230 2024-04-24T23:29:09.952 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.944+0000 7f4ebe4e7640 1 -- 172.21.15.80:0/2827917826 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4eb819f8b0 con 0x7f4eb8102230 2024-04-24T23:29:09.952 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.944+0000 7f4eb57fa640 1 -- 172.21.15.80:0/2827917826 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4eb0003dd0 con 0x7f4eb8102230 2024-04-24T23:29:09.952 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.944+0000 7f4eb57fa640 1 -- 172.21.15.80:0/2827917826 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb002fde0 con 0x7f4eb8102230 2024-04-24T23:29:09.952 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.944+0000 7f4ebe4e7640 1 -- 172.21.15.80:0/2827917826 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4eb819fe00 con 0x7f4eb8102230 2024-04-24T23:29:09.952 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.945+0000 7f4ebe4e7640 1 -- 172.21.15.80:0/2827917826 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4eb81a0030 con 0x7f4eb8102230 2024-04-24T23:29:09.952 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.946+0000 7f4eb57fa640 1 -- 172.21.15.80:0/2827917826 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4eb0004050 con 0x7f4eb8102230 2024-04-24T23:29:09.952 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.946+0000 7f4eb57fa640 1 --2- 172.21.15.80:0/2827917826 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f4ea00a2b90 0x7f4ea00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:09.952 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.947+0000 7f4eb57fa640 1 -- 172.21.15.80:0/2827917826 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f4eb00f0930 con 0x7f4eb8102230 2024-04-24T23:29:09.952 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.947+0000 7f4eb7fff640 1 --2- 172.21.15.80:0/2827917826 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f4ea00a2b90 0x7f4ea00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:09.952 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.947+0000 7f4eb7fff640 1 --2- 172.21.15.80:0/2827917826 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f4ea00a2b90 0x7f4ea00a5050 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f4ea8005fd0 tx=0x7f4ea800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:09.953 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:09.951+0000 7f4eb57fa640 1 -- 172.21.15.80:0/2827917826 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f4eb00bd5e0 con 0x7f4eb8102230 2024-04-24T23:29:10.126 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.125+0000 7f4ebe4e7640 1 -- 172.21.15.80:0/2827917826 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f4eb81a0200 con 0x7f4eb8102230 2024-04-24T23:29:10.126 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.125+0000 7f4eb57fa640 1 -- 172.21.15.80:0/2827917826 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) ==== 100+0+11 (secure 0 0 0) 0x7f4eb00bcee0 con 0x7f4eb8102230 2024-04-24T23:29:10.126 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 64 2024-04-24T23:29:10.128 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.127+0000 7f4ebe4e7640 1 -- 172.21.15.80:0/2827917826 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f4ea00a2b90 msgr2=0x7f4ea00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:10.128 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.127+0000 7f4ebe4e7640 1 --2- 172.21.15.80:0/2827917826 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f4ea00a2b90 0x7f4ea00a5050 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f4ea8005fd0 tx=0x7f4ea800b390 comp rx=0 tx=0).stop 2024-04-24T23:29:10.128 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.127+0000 7f4ebe4e7640 1 -- 172.21.15.80:0/2827917826 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4eb8102230 msgr2=0x7f4eb819efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:10.129 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.127+0000 7f4ebe4e7640 1 --2- 172.21.15.80:0/2827917826 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4eb8102230 0x7f4eb819efb0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f4eb0005e30 tx=0x7f4eb0031ee0 comp rx=0 tx=0).stop 2024-04-24T23:29:10.129 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.127+0000 7f4ebe4e7640 1 -- 172.21.15.80:0/2827917826 shutdown_connections 2024-04-24T23:29:10.129 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.127+0000 7f4ebe4e7640 1 --2- 172.21.15.80:0/2827917826 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f4ea00a2b90 0x7f4ea00a5050 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:10.129 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.127+0000 7f4ebe4e7640 1 --2- 172.21.15.80:0/2827917826 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4eb8102230 0x7f4eb819efb0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:10.129 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.127+0000 7f4ebe4e7640 1 --2- 172.21.15.80:0/2827917826 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4eb8101870 0x7f4eb8197f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:10.129 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.127+0000 7f4ebe4e7640 1 --2- 172.21.15.80:0/2827917826 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4eb8100680 0x7f4eb8197a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:10.129 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.127+0000 7f4ebe4e7640 1 -- 172.21.15.80:0/2827917826 >> 172.21.15.80:0/2827917826 conn(0x7f4eb80fbbf0 msgr2=0x7f4eb80fd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:10.129 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.127+0000 7f4ebe4e7640 1 -- 172.21.15.80:0/2827917826 shutdown_connections 2024-04-24T23:29:10.129 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.127+0000 7f4ebe4e7640 1 -- 172.21.15.80:0/2827917826 wait complete. 2024-04-24T23:29:10.140 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-24T23:29:10.244 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.242+0000 7f5f9d1f9640 1 -- 172.21.15.80:0/622832287 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5f98100680 msgr2=0x7f5f98100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:10.244 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.242+0000 7f5f9d1f9640 1 --2- 172.21.15.80:0/622832287 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5f98100680 0x7f5f98100aa0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f5f88009ca0 tx=0x7f5f8802a8c0 comp rx=0 tx=0).stop 2024-04-24T23:29:10.244 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.242+0000 7f5f9d1f9640 1 -- 172.21.15.80:0/622832287 shutdown_connections 2024-04-24T23:29:10.244 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.242+0000 7f5f9d1f9640 1 --2- 172.21.15.80:0/622832287 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5f98102230 0x7f5f98108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:10.244 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.242+0000 7f5f9d1f9640 1 --2- 172.21.15.80:0/622832287 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f98101870 0x7f5f98101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:10.244 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.242+0000 7f5f9d1f9640 1 --2- 172.21.15.80:0/622832287 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5f98100680 0x7f5f98100aa0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:10.244 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.242+0000 7f5f9d1f9640 1 -- 172.21.15.80:0/622832287 >> 172.21.15.80:0/622832287 conn(0x7f5f980fbbf0 msgr2=0x7f5f980fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:10.244 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.242+0000 7f5f9d1f9640 1 -- 172.21.15.80:0/622832287 shutdown_connections 2024-04-24T23:29:10.244 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.242+0000 7f5f9d1f9640 1 -- 172.21.15.80:0/622832287 wait complete. 2024-04-24T23:29:10.244 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.243+0000 7f5f9d1f9640 1 Processor -- start 2024-04-24T23:29:10.245 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.243+0000 7f5f9d1f9640 1 -- start start 2024-04-24T23:29:10.246 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.243+0000 7f5f9d1f9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5f98100680 0x7f5f98197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:10.247 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.243+0000 7f5f9d1f9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f98101870 0x7f5f98198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:10.247 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.243+0000 7f5f9d1f9640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5f98102230 0x7f5f9819f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:10.247 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.243+0000 7f5f9d1f9640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f5f9810c300 con 0x7f5f98102230 2024-04-24T23:29:10.247 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.243+0000 7f5f9d1f9640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5f9810c180 con 0x7f5f98100680 2024-04-24T23:29:10.247 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.243+0000 7f5f9d1f9640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5f9810c480 con 0x7f5f98101870 2024-04-24T23:29:10.247 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.243+0000 7f5f9759e640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5f98102230 0x7f5f9819f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:10.247 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.243+0000 7f5f9759e640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5f98102230 0x7f5f9819f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:57388/0 (socket says 172.21.15.80:57388) 2024-04-24T23:29:10.247 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.244+0000 7f5f9759e640 1 -- 172.21.15.80:0/3345832306 learned_addr learned my addr 172.21.15.80:0/3345832306 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:29:10.247 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.244+0000 7f5f96d9d640 1 --2- 172.21.15.80:0/3345832306 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5f98100680 0x7f5f98197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:10.247 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.244+0000 7f5f9659c640 1 --2- 172.21.15.80:0/3345832306 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f98101870 0x7f5f98198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:10.247 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.244+0000 7f5f9759e640 1 -- 172.21.15.80:0/3345832306 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f98101870 msgr2=0x7f5f98198010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:10.247 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.244+0000 7f5f9759e640 1 --2- 172.21.15.80:0/3345832306 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f98101870 0x7f5f98198010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:10.247 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.244+0000 7f5f9759e640 1 -- 172.21.15.80:0/3345832306 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5f98100680 msgr2=0x7f5f98197ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:10.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.244+0000 7f5f9759e640 1 --2- 172.21.15.80:0/3345832306 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5f98100680 0x7f5f98197ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:10.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.244+0000 7f5f9759e640 1 -- 172.21.15.80:0/3345832306 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f9819f6f0 con 0x7f5f98102230 2024-04-24T23:29:10.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.244+0000 7f5f9759e640 1 --2- 172.21.15.80:0/3345832306 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5f98102230 0x7f5f9819f080 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f5f9000d750 tx=0x7f5f9000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:10.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.244+0000 7f5f7ffff640 1 -- 172.21.15.80:0/3345832306 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f90017070 con 0x7f5f98102230 2024-04-24T23:29:10.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.244+0000 7f5f9d1f9640 1 -- 172.21.15.80:0/3345832306 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f9819f9e0 con 0x7f5f98102230 2024-04-24T23:29:10.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.244+0000 7f5f7ffff640 1 -- 172.21.15.80:0/3345832306 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f90005500 con 0x7f5f98102230 2024-04-24T23:29:10.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.244+0000 7f5f7ffff640 1 -- 172.21.15.80:0/3345832306 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f90004d90 con 0x7f5f98102230 2024-04-24T23:29:10.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.244+0000 7f5f9d1f9640 1 -- 172.21.15.80:0/3345832306 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f9819fee0 con 0x7f5f98102230 2024-04-24T23:29:10.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.245+0000 7f5f9d1f9640 1 -- 172.21.15.80:0/3345832306 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f981a00b0 con 0x7f5f98102230 2024-04-24T23:29:10.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.246+0000 7f5f7ffff640 1 -- 172.21.15.80:0/3345832306 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5f90004f30 con 0x7f5f98102230 2024-04-24T23:29:10.249 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.247+0000 7f5f7ffff640 1 --2- 172.21.15.80:0/3345832306 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f5f800a2b90 0x7f5f800a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:10.249 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.247+0000 7f5f96d9d640 1 --2- 172.21.15.80:0/3345832306 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f5f800a2b90 0x7f5f800a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:10.249 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.247+0000 7f5f7ffff640 1 -- 172.21.15.80:0/3345832306 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f5f900ceee0 con 0x7f5f98102230 2024-04-24T23:29:10.249 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.247+0000 7f5f96d9d640 1 --2- 172.21.15.80:0/3345832306 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f5f800a2b90 0x7f5f800a5050 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f5f88006010 tx=0x7f5f88009360 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:10.251 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.251+0000 7f5f7ffff640 1 -- 172.21.15.80:0/3345832306 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f5f9009ba70 con 0x7f5f98102230 2024-04-24T23:29:10.427 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.426+0000 7f5f9d1f9640 1 -- 172.21.15.80:0/3345832306 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f5f981a0280 con 0x7f5f98102230 2024-04-24T23:29:10.428 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.427+0000 7f5f7ffff640 1 -- 172.21.15.80:0/3345832306 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) ==== 103+0+11 (secure 0 0 0) 0x7f5f9009b370 con 0x7f5f98102230 2024-04-24T23:29:10.428 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 32 2024-04-24T23:29:10.430 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.429+0000 7f5f9d1f9640 1 -- 172.21.15.80:0/3345832306 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f5f800a2b90 msgr2=0x7f5f800a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:10.430 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.429+0000 7f5f9d1f9640 1 --2- 172.21.15.80:0/3345832306 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f5f800a2b90 0x7f5f800a5050 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f5f88006010 tx=0x7f5f88009360 comp rx=0 tx=0).stop 2024-04-24T23:29:10.430 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.429+0000 7f5f9d1f9640 1 -- 172.21.15.80:0/3345832306 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5f98102230 msgr2=0x7f5f9819f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:10.430 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.429+0000 7f5f9d1f9640 1 --2- 172.21.15.80:0/3345832306 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5f98102230 0x7f5f9819f080 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f5f9000d750 tx=0x7f5f9000dc10 comp rx=0 tx=0).stop 2024-04-24T23:29:10.430 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.429+0000 7f5f9d1f9640 1 -- 172.21.15.80:0/3345832306 shutdown_connections 2024-04-24T23:29:10.430 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.429+0000 7f5f9d1f9640 1 --2- 172.21.15.80:0/3345832306 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f5f800a2b90 0x7f5f800a5050 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:10.430 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.429+0000 7f5f9d1f9640 1 --2- 172.21.15.80:0/3345832306 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5f98102230 0x7f5f9819f080 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:10.430 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.429+0000 7f5f9d1f9640 1 --2- 172.21.15.80:0/3345832306 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f98101870 0x7f5f98198010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:10.430 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.429+0000 7f5f9d1f9640 1 --2- 172.21.15.80:0/3345832306 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5f98100680 0x7f5f98197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:10.430 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.429+0000 7f5f9d1f9640 1 -- 172.21.15.80:0/3345832306 >> 172.21.15.80:0/3345832306 conn(0x7f5f980fbbf0 msgr2=0x7f5f980fd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:10.430 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.429+0000 7f5f9d1f9640 1 -- 172.21.15.80:0/3345832306 shutdown_connections 2024-04-24T23:29:10.431 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.429+0000 7f5f9d1f9640 1 -- 172.21.15.80:0/3345832306 wait complete. 2024-04-24T23:29:10.440 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:29:10 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/4214137511' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:29:10.440 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:29:10 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/422354391' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:29:10.440 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:29:10 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/4065823044' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T23:29:10.440 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:29:10 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/2827917826' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T23:29:10.441 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:29:10.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:29:10 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/4214137511' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:29:10.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:29:10 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/422354391' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:29:10.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:29:10 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/4065823044' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T23:29:10.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:29:10 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/2827917826' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T23:29:10.542 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.539+0000 7effd2990640 1 -- 172.21.15.80:0/528007192 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7effcc103340 msgr2=0x7effcc109bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:10.542 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.539+0000 7effd2990640 1 --2- 172.21.15.80:0/528007192 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7effcc103340 0x7effcc109bd0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7effc4009ca0 tx=0x7effc402a8c0 comp rx=0 tx=0).stop 2024-04-24T23:29:10.542 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.539+0000 7effd2990640 1 -- 172.21.15.80:0/528007192 shutdown_connections 2024-04-24T23:29:10.542 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.539+0000 7effd2990640 1 --2- 172.21.15.80:0/528007192 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7effcc103340 0x7effcc109bd0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:10.543 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.539+0000 7effd2990640 1 --2- 172.21.15.80:0/528007192 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7effcc102980 0x7effcc102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:10.543 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.539+0000 7effd2990640 1 --2- 172.21.15.80:0/528007192 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7effcc101790 0x7effcc101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:10.543 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.539+0000 7effd2990640 1 -- 172.21.15.80:0/528007192 >> 172.21.15.80:0/528007192 conn(0x7effcc0fcd20 msgr2=0x7effcc0ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:10.543 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.539+0000 7effd2990640 1 -- 172.21.15.80:0/528007192 shutdown_connections 2024-04-24T23:29:10.543 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.539+0000 7effd2990640 1 -- 172.21.15.80:0/528007192 wait complete. 2024-04-24T23:29:10.543 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.539+0000 7effd2990640 1 Processor -- start 2024-04-24T23:29:10.543 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.539+0000 7effd2990640 1 -- start start 2024-04-24T23:29:10.543 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.540+0000 7effd2990640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7effcc101790 0x7effcc0754f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:10.544 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.540+0000 7effd2990640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7effcc102980 0x7effcc073b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:10.544 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.540+0000 7effd2990640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7effcc103340 0x7effcc074070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:10.544 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.540+0000 7effd2990640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7effcc10d460 con 0x7effcc102980 2024-04-24T23:29:10.544 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.540+0000 7effd2990640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7effcc10d2e0 con 0x7effcc101790 2024-04-24T23:29:10.544 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.540+0000 7effd2990640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7effcc10d5e0 con 0x7effcc103340 2024-04-24T23:29:10.544 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.540+0000 7effcb7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7effcc102980 0x7effcc073b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:10.544 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.540+0000 7effcb7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7effcc102980 0x7effcc073b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:57414/0 (socket says 172.21.15.80:57414) 2024-04-24T23:29:10.544 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.540+0000 7effcb7fe640 1 -- 172.21.15.80:0/4077691008 learned_addr learned my addr 172.21.15.80:0/4077691008 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:29:10.545 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.540+0000 7effd0f2d640 1 --2- 172.21.15.80:0/4077691008 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7effcc103340 0x7effcc074070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:10.545 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.540+0000 7effcbfff640 1 --2- 172.21.15.80:0/4077691008 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7effcc101790 0x7effcc0754f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:10.545 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.540+0000 7effcb7fe640 1 -- 172.21.15.80:0/4077691008 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7effcc103340 msgr2=0x7effcc074070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:10.545 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.540+0000 7effcb7fe640 1 --2- 172.21.15.80:0/4077691008 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7effcc103340 0x7effcc074070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:10.545 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.540+0000 7effcb7fe640 1 -- 172.21.15.80:0/4077691008 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7effcc101790 msgr2=0x7effcc0754f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:10.545 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.540+0000 7effcb7fe640 1 --2- 172.21.15.80:0/4077691008 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7effcc101790 0x7effcc0754f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:10.545 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.540+0000 7effcb7fe640 1 -- 172.21.15.80:0/4077691008 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effcc074900 con 0x7effcc102980 2024-04-24T23:29:10.545 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.540+0000 7effd0f2d640 1 --2- 172.21.15.80:0/4077691008 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7effcc103340 0x7effcc074070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:29:10.545 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.540+0000 7effcbfff640 1 --2- 172.21.15.80:0/4077691008 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7effcc101790 0x7effcc0754f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:29:10.545 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.540+0000 7effcb7fe640 1 --2- 172.21.15.80:0/4077691008 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7effcc102980 0x7effcc073b30 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7effc000c890 tx=0x7effc000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:10.545 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.540+0000 7effc97fa640 1 -- 172.21.15.80:0/4077691008 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effc0004220 con 0x7effcc102980 2024-04-24T23:29:10.545 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.541+0000 7effc97fa640 1 -- 172.21.15.80:0/4077691008 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7effc00043c0 con 0x7effcc102980 2024-04-24T23:29:10.545 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.541+0000 7effd2990640 1 -- 172.21.15.80:0/4077691008 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effcc1a1b90 con 0x7effcc102980 2024-04-24T23:29:10.545 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.541+0000 7effc97fa640 1 -- 172.21.15.80:0/4077691008 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effc00049e0 con 0x7effcc102980 2024-04-24T23:29:10.546 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.541+0000 7effd2990640 1 -- 172.21.15.80:0/4077691008 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effcc1a20e0 con 0x7effcc102980 2024-04-24T23:29:10.546 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.541+0000 7effd2990640 1 -- 172.21.15.80:0/4077691008 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effcc108a50 con 0x7effcc102980 2024-04-24T23:29:10.546 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.542+0000 7effc97fa640 1 -- 172.21.15.80:0/4077691008 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7effc00079b0 con 0x7effcc102980 2024-04-24T23:29:10.547 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.543+0000 7effc97fa640 1 --2- 172.21.15.80:0/4077691008 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7effb40a2b90 0x7effb40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:10.547 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.543+0000 7effcbfff640 1 --2- 172.21.15.80:0/4077691008 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7effb40a2b90 0x7effb40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:10.547 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.543+0000 7effc97fa640 1 -- 172.21.15.80:0/4077691008 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7effc00cf5b0 con 0x7effcc102980 2024-04-24T23:29:10.547 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.544+0000 7effcbfff640 1 --2- 172.21.15.80:0/4077691008 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7effb40a2b90 0x7effb40a5050 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7effcc1027e0 tx=0x7effbc005e70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:10.548 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.548+0000 7effc97fa640 1 -- 172.21.15.80:0/4077691008 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7effc0014030 con 0x7effcc102980 2024-04-24T23:29:10.698 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:29:10 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/4214137511' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:29:10.698 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:29:10 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/422354391' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:29:10.698 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:29:10 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/4065823044' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T23:29:10.698 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:29:10 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/2827917826' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T23:29:10.759 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.758+0000 7effd2990640 1 -- 172.21.15.80:0/4077691008 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7effcc02d170 con 0x7effcc102980 2024-04-24T23:29:10.760 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.759+0000 7effc97fa640 1 -- 172.21.15.80:0/4077691008 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11872 (secure 0 0 0) 0x7effc009c140 con 0x7effcc102980 2024-04-24T23:29:10.760 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-24T23:29:10.760 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/891591085","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":891591085},{"type":"v1","addr":"172.21.15.80:6835","nonce":891591085}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6837/259529026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6836","nonce":259529026},{"type":"v1","addr":"172.21.15.80:6837","nonce":259529026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14646,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6839/4194284854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6838","nonce":4194284854},{"type":"v1","addr":"172.21.15.80:6839","nonce":4194284854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14658,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6841/3145730958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6840","nonce":3145730958},{"type":"v1","addr":"172.21.15.80:6841","nonce":3145730958}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24395,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/3736784385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3736784385},{"type":"v1","addr":"172.21.15.134:6833","nonce":3736784385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/124777462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":124777462},{"type":"v1","addr":"172.21.15.148:6833","nonce":124777462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T23:27:19.300566+0000","modified":"2024-04-24T23:27:45.981839+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24496,"mds_1":24488,"mds_2":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/1399801120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1399801120},{"type":"v1","addr":"172.21.15.134:6835","nonce":1399801120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/430512704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":430512704},{"type":"v1","addr":"172.21.15.134:6837","nonce":430512704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/3002319310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3002319310},{"type":"v1","addr":"172.21.15.148:6835","nonce":3002319310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.148:6837/856528403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":856528403},{"type":"v1","addr":"172.21.15.148:6837","nonce":856528403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6839/1743815021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":1743815021},{"type":"v1","addr":"172.21.15.134:6839","nonce":1743815021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.148:6839/2952757594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":2952757594},{"type":"v1","addr":"172.21.15.148:6839","nonce":2952757594}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24496,"qdb_cluster":[24472,24488,24496]},"id":1}]} 2024-04-24T23:29:10.760 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 24 2024-04-24T23:29:10.762 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.761+0000 7effd2990640 1 -- 172.21.15.80:0/4077691008 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7effb40a2b90 msgr2=0x7effb40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:10.762 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.761+0000 7effd2990640 1 --2- 172.21.15.80:0/4077691008 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7effb40a2b90 0x7effb40a5050 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7effcc1027e0 tx=0x7effbc005e70 comp rx=0 tx=0).stop 2024-04-24T23:29:10.762 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.761+0000 7effd2990640 1 -- 172.21.15.80:0/4077691008 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7effcc102980 msgr2=0x7effcc073b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:10.762 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.761+0000 7effd2990640 1 --2- 172.21.15.80:0/4077691008 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7effcc102980 0x7effcc073b30 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7effc000c890 tx=0x7effc000cd50 comp rx=0 tx=0).stop 2024-04-24T23:29:10.762 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.761+0000 7effd2990640 1 -- 172.21.15.80:0/4077691008 shutdown_connections 2024-04-24T23:29:10.762 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.761+0000 7effd2990640 1 --2- 172.21.15.80:0/4077691008 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7effb40a2b90 0x7effb40a5050 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:10.762 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.761+0000 7effd2990640 1 --2- 172.21.15.80:0/4077691008 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7effcc103340 0x7effcc074070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:10.762 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.761+0000 7effd2990640 1 --2- 172.21.15.80:0/4077691008 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7effcc102980 0x7effcc073b30 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:10.763 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.761+0000 7effd2990640 1 --2- 172.21.15.80:0/4077691008 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7effcc101790 0x7effcc0754f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:10.763 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.761+0000 7effd2990640 1 -- 172.21.15.80:0/4077691008 >> 172.21.15.80:0/4077691008 conn(0x7effcc0fcd20 msgr2=0x7effcc105c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:10.763 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.761+0000 7effd2990640 1 -- 172.21.15.80:0/4077691008 shutdown_connections 2024-04-24T23:29:10.763 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.761+0000 7effd2990640 1 -- 172.21.15.80:0/4077691008 wait complete. 2024-04-24T23:29:10.773 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:29:10.873 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.870+0000 7f2d65c64640 1 -- 172.21.15.80:0/3867276270 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2d60100690 msgr2=0x7f2d60100ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:10.873 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.870+0000 7f2d65c64640 1 --2- 172.21.15.80:0/3867276270 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2d60100690 0x7f2d60100ab0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f2d54009900 tx=0x7f2d5402a410 comp rx=0 tx=0).stop 2024-04-24T23:29:10.873 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.870+0000 7f2d65c64640 1 -- 172.21.15.80:0/3867276270 shutdown_connections 2024-04-24T23:29:10.873 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.870+0000 7f2d65c64640 1 --2- 172.21.15.80:0/3867276270 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2d60102240 0x7f2d60108ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:10.873 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.870+0000 7f2d65c64640 1 --2- 172.21.15.80:0/3867276270 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d60101880 0x7f2d60101d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:10.874 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.870+0000 7f2d65c64640 1 --2- 172.21.15.80:0/3867276270 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2d60100690 0x7f2d60100ab0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:10.874 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.870+0000 7f2d65c64640 1 -- 172.21.15.80:0/3867276270 >> 172.21.15.80:0/3867276270 conn(0x7f2d600fbc20 msgr2=0x7f2d600fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:10.874 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.870+0000 7f2d65c64640 1 -- 172.21.15.80:0/3867276270 shutdown_connections 2024-04-24T23:29:10.874 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.870+0000 7f2d65c64640 1 -- 172.21.15.80:0/3867276270 wait complete. 2024-04-24T23:29:10.874 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.870+0000 7f2d65c64640 1 Processor -- start 2024-04-24T23:29:10.874 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.871+0000 7f2d65c64640 1 -- start start 2024-04-24T23:29:10.874 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.871+0000 7f2d65c64640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2d60100690 0x7f2d60196b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:10.874 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.871+0000 7f2d65c64640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d60101880 0x7f2d60197070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:10.874 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.871+0000 7f2d65c64640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2d60102240 0x7f2d6019e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:10.874 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.871+0000 7f2d65c64640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f2d6010c430 con 0x7f2d60100690 2024-04-24T23:29:10.874 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.871+0000 7f2d65c64640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2d6010c2b0 con 0x7f2d60101880 2024-04-24T23:29:10.874 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.871+0000 7f2d65c64640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f2d6010c5b0 con 0x7f2d60102240 2024-04-24T23:29:10.874 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.871+0000 7f2d5f7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2d60100690 0x7f2d60196b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:10.875 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.871+0000 7f2d5f7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2d60100690 0x7f2d60196b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:57420/0 (socket says 172.21.15.80:57420) 2024-04-24T23:29:10.875 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.871+0000 7f2d5f7fe640 1 -- 172.21.15.80:0/1597170235 learned_addr learned my addr 172.21.15.80:0/1597170235 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:29:10.875 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.871+0000 7f2d5ffff640 1 --2- 172.21.15.80:0/1597170235 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2d60102240 0x7f2d6019e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:10.875 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.871+0000 7f2d5f7fe640 1 -- 172.21.15.80:0/1597170235 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2d60102240 msgr2=0x7f2d6019e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:10.875 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.871+0000 7f2d5f7fe640 1 --2- 172.21.15.80:0/1597170235 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2d60102240 0x7f2d6019e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:10.875 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.871+0000 7f2d5f7fe640 1 -- 172.21.15.80:0/1597170235 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d60101880 msgr2=0x7f2d60197070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:10.875 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.871+0000 7f2d5f7fe640 1 --2- 172.21.15.80:0/1597170235 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d60101880 0x7f2d60197070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:10.875 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.871+0000 7f2d5f7fe640 1 -- 172.21.15.80:0/1597170235 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d6019e750 con 0x7f2d60100690 2024-04-24T23:29:10.875 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.871+0000 7f2d5ffff640 1 --2- 172.21.15.80:0/1597170235 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2d60102240 0x7f2d6019e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:29:10.875 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.871+0000 7f2d5f7fe640 1 --2- 172.21.15.80:0/1597170235 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2d60100690 0x7f2d60196b30 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f2d5402aa00 tx=0x7f2d54032da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:10.875 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.871+0000 7f2d5cff9640 1 -- 172.21.15.80:0/1597170235 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d5403f030 con 0x7f2d60100690 2024-04-24T23:29:10.875 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.871+0000 7f2d5cff9640 1 -- 172.21.15.80:0/1597170235 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d54003ce0 con 0x7f2d60100690 2024-04-24T23:29:10.875 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.871+0000 7f2d65c64640 1 -- 172.21.15.80:0/1597170235 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d6019e9e0 con 0x7f2d60100690 2024-04-24T23:29:10.875 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.871+0000 7f2d5cff9640 1 -- 172.21.15.80:0/1597170235 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d54035070 con 0x7f2d60100690 2024-04-24T23:29:10.876 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.871+0000 7f2d65c64640 1 -- 172.21.15.80:0/1597170235 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d6019efa0 con 0x7f2d60100690 2024-04-24T23:29:10.876 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.872+0000 7f2d65c64640 1 -- 172.21.15.80:0/1597170235 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d60107a50 con 0x7f2d60100690 2024-04-24T23:29:10.876 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.873+0000 7f2d5cff9640 1 -- 172.21.15.80:0/1597170235 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2d54042030 con 0x7f2d60100690 2024-04-24T23:29:10.876 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.874+0000 7f2d5cff9640 1 --2- 172.21.15.80:0/1597170235 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f2d480a2c10 0x7f2d480a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:10.877 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.874+0000 7f2d5effd640 1 --2- 172.21.15.80:0/1597170235 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f2d480a2c10 0x7f2d480a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:10.877 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.874+0000 7f2d5cff9640 1 -- 172.21.15.80:0/1597170235 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f2d540efb90 con 0x7f2d60100690 2024-04-24T23:29:10.877 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.875+0000 7f2d5effd640 1 --2- 172.21.15.80:0/1597170235 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f2d480a2c10 0x7f2d480a50d0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f2d60198040 tx=0x7f2d50006c90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:10.879 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:10.879+0000 7f2d5cff9640 1 -- 172.21.15.80:0/1597170235 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f2d540bc720 con 0x7f2d60100690 2024-04-24T23:29:11.059 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.058+0000 7f2d65c64640 1 -- 172.21.15.80:0/1597170235 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2d60062370 con 0x7f2d60100690 2024-04-24T23:29:11.061 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.060+0000 7f2d5cff9640 1 -- 172.21.15.80:0/1597170235 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24839 (secure 0 0 0) 0x7f2d540bc020 con 0x7f2d60100690 2024-04-24T23:29:11.061 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-24T23:29:11.061 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":94,"fsid":"ec7885aa-0290-11ef-bc93-c7b262605968","created":"2024-04-24T23:19:05.586407+0000","modified":"2024-04-24T23:29:08.209684+0000","last_up_change":"2024-04-24T23:26:25.938316+0000","last_in_change":"2024-04-24T23:26:04.786699+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:22:33.124428+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:27:17.575788+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:27:18.492762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T23:27:22.438503+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a525741d-b46c-4eef-aa64-3f057cac1497","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":2597718112},{"type":"v1","addr":"172.21.15.80:6803","nonce":2597718112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":2597718112},{"type":"v1","addr":"172.21.15.80:6805","nonce":2597718112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2597718112},{"type":"v1","addr":"172.21.15.80:6809","nonce":2597718112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":2597718112},{"type":"v1","addr":"172.21.15.80:6807","nonce":2597718112}]},"public_addr":"172.21.15.80:6803/2597718112","cluster_addr":"172.21.15.80:6805/2597718112","heartbeat_back_addr":"172.21.15.80:6809/2597718112","heartbeat_front_addr":"172.21.15.80:6807/2597718112","state":["exists","up"]},{"osd":1,"uuid":"2d0f0d67-6db4-40ac-91cc-58604ab6795a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":2395610201},{"type":"v1","addr":"172.21.15.134:6801","nonce":2395610201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":2395610201},{"type":"v1","addr":"172.21.15.134:6803","nonce":2395610201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":2395610201},{"type":"v1","addr":"172.21.15.134:6807","nonce":2395610201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":2395610201},{"type":"v1","addr":"172.21.15.134:6805","nonce":2395610201}]},"public_addr":"172.21.15.134:6801/2395610201","cluster_addr":"172.21.15.134:6803/2395610201","heartbeat_back_addr":"172.21.15.134:6807/2395610201","heartbeat_front_addr":"172.21.15.134:6805/2395610201","state":["exists","up"]},{"osd":2,"uuid":"781fb879-fedc-4c84-82fb-59e073bc4cf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1629152187},{"type":"v1","addr":"172.21.15.148:6801","nonce":1629152187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1629152187},{"type":"v1","addr":"172.21.15.148:6803","nonce":1629152187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1629152187},{"type":"v1","addr":"172.21.15.148:6807","nonce":1629152187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1629152187},{"type":"v1","addr":"172.21.15.148:6805","nonce":1629152187}]},"public_addr":"172.21.15.148:6801/1629152187","cluster_addr":"172.21.15.148:6803/1629152187","heartbeat_back_addr":"172.21.15.148:6807/1629152187","heartbeat_front_addr":"172.21.15.148:6805/1629152187","state":["exists","up"]},{"osd":3,"uuid":"cd982757-f2f6-4296-b3e7-4cd40fbd8d9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":3128298510},{"type":"v1","addr":"172.21.15.80:6811","nonce":3128298510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":3128298510},{"type":"v1","addr":"172.21.15.80:6813","nonce":3128298510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":3128298510},{"type":"v1","addr":"172.21.15.80:6817","nonce":3128298510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":3128298510},{"type":"v1","addr":"172.21.15.80:6815","nonce":3128298510}]},"public_addr":"172.21.15.80:6811/3128298510","cluster_addr":"172.21.15.80:6813/3128298510","heartbeat_back_addr":"172.21.15.80:6817/3128298510","heartbeat_front_addr":"172.21.15.80:6815/3128298510","state":["exists","up"]},{"osd":4,"uuid":"dfc55a79-a7e8-417a-b898-ec44171a54af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1917556060},{"type":"v1","addr":"172.21.15.134:6809","nonce":1917556060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1917556060},{"type":"v1","addr":"172.21.15.134:6811","nonce":1917556060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1917556060},{"type":"v1","addr":"172.21.15.134:6815","nonce":1917556060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1917556060},{"type":"v1","addr":"172.21.15.134:6813","nonce":1917556060}]},"public_addr":"172.21.15.134:6809/1917556060","cluster_addr":"172.21.15.134:6811/1917556060","heartbeat_back_addr":"172.21.15.134:6815/1917556060","heartbeat_front_addr":"172.21.15.134:6813/1917556060","state":["exists","up"]},{"osd":5,"uuid":"b5491337-0694-440c-b679-312f3d14cf65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":2715951281},{"type":"v1","addr":"172.21.15.148:6809","nonce":2715951281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":2715951281},{"type":"v1","addr":"172.21.15.148:6811","nonce":2715951281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":2715951281},{"type":"v1","addr":"172.21.15.148:6815","nonce":2715951281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":2715951281},{"type":"v1","addr":"172.21.15.148:6813","nonce":2715951281}]},"public_addr":"172.21.15.148:6809/2715951281","cluster_addr":"172.21.15.148:6811/2715951281","heartbeat_back_addr":"172.21.15.148:6815/2715951281","heartbeat_front_addr":"172.21.15.148:6813/2715951281","state":["exists","up"]},{"osd":6,"uuid":"ba4b5df7-d86e-47cd-ab67-cf763c4e4d6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":953261724},{"type":"v1","addr":"172.21.15.80:6819","nonce":953261724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":953261724},{"type":"v1","addr":"172.21.15.80:6821","nonce":953261724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":953261724},{"type":"v1","addr":"172.21.15.80:6825","nonce":953261724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":953261724},{"type":"v1","addr":"172.21.15.80:6823","nonce":953261724}]},"public_addr":"172.21.15.80:6819/953261724","cluster_addr":"172.21.15.80:6821/953261724","heartbeat_back_addr":"172.21.15.80:6825/953261724","heartbeat_front_addr":"172.21.15.80:6823/953261724","state":["exists","up"]},{"osd":7,"uuid":"464a503a-75ae-49e7-ab6e-a7f9f2a6a307","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":108354844},{"type":"v1","addr":"172.21.15.134:6817","nonce":108354844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":108354844},{"type":"v1","addr":"172.21.15.134:6819","nonce":108354844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":108354844},{"type":"v1","addr":"172.21.15.134:6823","nonce":108354844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":108354844},{"type":"v1","addr":"172.21.15.134:6821","nonce":108354844}]},"public_addr":"172.21.15.134:6817/108354844","cluster_addr":"172.21.15.134:6819/108354844","heartbeat_back_addr":"172.21.15.134:6823/108354844","heartbeat_front_addr":"172.21.15.134:6821/108354844","state":["exists","up"]},{"osd":8,"uuid":"ce544b2e-4540-47c0-9997-500c45351330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3559655329},{"type":"v1","addr":"172.21.15.148:6817","nonce":3559655329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":3559655329},{"type":"v1","addr":"172.21.15.148:6819","nonce":3559655329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":3559655329},{"type":"v1","addr":"172.21.15.148:6823","nonce":3559655329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3559655329},{"type":"v1","addr":"172.21.15.148:6821","nonce":3559655329}]},"public_addr":"172.21.15.148:6817/3559655329","cluster_addr":"172.21.15.148:6819/3559655329","heartbeat_back_addr":"172.21.15.148:6823/3559655329","heartbeat_front_addr":"172.21.15.148:6821/3559655329","state":["exists","up"]},{"osd":9,"uuid":"06de3732-19b9-496c-a464-55322efa6759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":4019913545},{"type":"v1","addr":"172.21.15.80:6827","nonce":4019913545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":4019913545},{"type":"v1","addr":"172.21.15.80:6829","nonce":4019913545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":4019913545},{"type":"v1","addr":"172.21.15.80:6833","nonce":4019913545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":4019913545},{"type":"v1","addr":"172.21.15.80:6831","nonce":4019913545}]},"public_addr":"172.21.15.80:6827/4019913545","cluster_addr":"172.21.15.80:6829/4019913545","heartbeat_back_addr":"172.21.15.80:6833/4019913545","heartbeat_front_addr":"172.21.15.80:6831/4019913545","state":["exists","up"]},{"osd":10,"uuid":"8d92f0d9-5c64-429a-b750-c33ae0e070c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":647343766},{"type":"v1","addr":"172.21.15.134:6825","nonce":647343766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":647343766},{"type":"v1","addr":"172.21.15.134:6827","nonce":647343766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":647343766},{"type":"v1","addr":"172.21.15.134:6831","nonce":647343766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":647343766},{"type":"v1","addr":"172.21.15.134:6829","nonce":647343766}]},"public_addr":"172.21.15.134:6825/647343766","cluster_addr":"172.21.15.134:6827/647343766","heartbeat_back_addr":"172.21.15.134:6831/647343766","heartbeat_front_addr":"172.21.15.134:6829/647343766","state":["exists","up"]},{"osd":11,"uuid":"4c003a41-f65b-479f-bc80-b27d680cf936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":4068024116},{"type":"v1","addr":"172.21.15.148:6825","nonce":4068024116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":4068024116},{"type":"v1","addr":"172.21.15.148:6827","nonce":4068024116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":4068024116},{"type":"v1","addr":"172.21.15.148:6831","nonce":4068024116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":4068024116},{"type":"v1","addr":"172.21.15.148:6829","nonce":4068024116}]},"public_addr":"172.21.15.148:6825/4068024116","cluster_addr":"172.21.15.148:6827/4068024116","heartbeat_back_addr":"172.21.15.148:6831/4068024116","heartbeat_front_addr":"172.21.15.148:6829/4068024116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:21:42.433631+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:22:06.145933+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:22:30.726143+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:22:55.729961+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:23:21.280913+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:23:46.609983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:24:12.585247+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:24:38.359287+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:25:04.491232+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:25:30.788089+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:25:56.891078+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:26:23.899113+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":7}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"4.a","mappings":[{"from":5,"to":8}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.80:0/2828378749":"2024-04-25T23:20:06.793609+0000","172.21.15.80:6801/233364790":"2024-04-25T23:20:06.793609+0000","172.21.15.80:6800/1614691410":"2024-04-25T23:19:40.490692+0000","172.21.15.80:0/1406187145":"2024-04-25T23:20:06.793609+0000","172.21.15.80:0/92953396":"2024-04-25T23:19:40.490692+0000","172.21.15.80:0/598129432":"2024-04-25T23:20:06.793609+0000","172.21.15.80:0/232193118":"2024-04-25T23:19:40.490692+0000","172.21.15.80:6800/233364790":"2024-04-25T23:20:06.793609+0000","172.21.15.80:6801/1614691410":"2024-04-25T23:19:40.490692+0000","172.21.15.80:0/2081562529":"2024-04-25T23:19:40.490692+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:29:11.063 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.062+0000 7f2d65c64640 1 -- 172.21.15.80:0/1597170235 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f2d480a2c10 msgr2=0x7f2d480a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:11.063 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.062+0000 7f2d65c64640 1 --2- 172.21.15.80:0/1597170235 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f2d480a2c10 0x7f2d480a50d0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f2d60198040 tx=0x7f2d50006c90 comp rx=0 tx=0).stop 2024-04-24T23:29:11.063 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.062+0000 7f2d65c64640 1 -- 172.21.15.80:0/1597170235 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2d60100690 msgr2=0x7f2d60196b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:11.063 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.062+0000 7f2d65c64640 1 --2- 172.21.15.80:0/1597170235 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2d60100690 0x7f2d60196b30 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f2d5402aa00 tx=0x7f2d54032da0 comp rx=0 tx=0).stop 2024-04-24T23:29:11.063 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.062+0000 7f2d65c64640 1 -- 172.21.15.80:0/1597170235 shutdown_connections 2024-04-24T23:29:11.063 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.062+0000 7f2d65c64640 1 --2- 172.21.15.80:0/1597170235 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f2d480a2c10 0x7f2d480a50d0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:11.063 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.062+0000 7f2d65c64640 1 --2- 172.21.15.80:0/1597170235 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2d60102240 0x7f2d6019e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:11.064 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.062+0000 7f2d65c64640 1 --2- 172.21.15.80:0/1597170235 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d60101880 0x7f2d60197070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:11.064 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.062+0000 7f2d65c64640 1 --2- 172.21.15.80:0/1597170235 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2d60100690 0x7f2d60196b30 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:11.064 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.062+0000 7f2d65c64640 1 -- 172.21.15.80:0/1597170235 >> 172.21.15.80:0/1597170235 conn(0x7f2d600fbc20 msgr2=0x7f2d60104c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:11.064 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.062+0000 7f2d65c64640 1 -- 172.21.15.80:0/1597170235 shutdown_connections 2024-04-24T23:29:11.064 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.062+0000 7f2d65c64640 1 -- 172.21.15.80:0/1597170235 wait complete. 2024-04-24T23:29:11.075 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:29:11.175 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.172+0000 7f09aae15640 1 -- 172.21.15.80:0/2162071318 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f09a4101880 msgr2=0x7f09a4101d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:11.175 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.172+0000 7f09aae15640 1 --2- 172.21.15.80:0/2162071318 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f09a4101880 0x7f09a4101d00 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f0990009900 tx=0x7f099002a410 comp rx=0 tx=0).stop 2024-04-24T23:29:11.175 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.172+0000 7f09aae15640 1 -- 172.21.15.80:0/2162071318 shutdown_connections 2024-04-24T23:29:11.175 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.172+0000 7f09aae15640 1 --2- 172.21.15.80:0/2162071318 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09a4102240 0x7f09a4108ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:11.175 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.172+0000 7f09aae15640 1 --2- 172.21.15.80:0/2162071318 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f09a4101880 0x7f09a4101d00 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:11.175 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.172+0000 7f09aae15640 1 --2- 172.21.15.80:0/2162071318 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09a4100690 0x7f09a4100ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:11.175 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.172+0000 7f09aae15640 1 -- 172.21.15.80:0/2162071318 >> 172.21.15.80:0/2162071318 conn(0x7f09a40fbc20 msgr2=0x7f09a40fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:11.175 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.172+0000 7f09aae15640 1 -- 172.21.15.80:0/2162071318 shutdown_connections 2024-04-24T23:29:11.176 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.172+0000 7f09aae15640 1 -- 172.21.15.80:0/2162071318 wait complete. 2024-04-24T23:29:11.176 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.172+0000 7f09aae15640 1 Processor -- start 2024-04-24T23:29:11.176 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.172+0000 7f09aae15640 1 -- start start 2024-04-24T23:29:11.176 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.172+0000 7f09aae15640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f09a4100690 0x7f09a4197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:11.176 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.172+0000 7f09aae15640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09a4101880 0x7f09a4198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:11.176 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.172+0000 7f09aae15640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09a4102240 0x7f09a419f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:11.176 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.172+0000 7f09aae15640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f09a410c3c0 con 0x7f09a4100690 2024-04-24T23:29:11.176 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.172+0000 7f09aae15640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f09a410c240 con 0x7f09a4101880 2024-04-24T23:29:11.176 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.172+0000 7f09aae15640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f09a410c540 con 0x7f09a4102240 2024-04-24T23:29:11.176 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.172+0000 7f09a8bb1640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f09a4100690 0x7f09a4197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:11.176 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.173+0000 7f09a8bb1640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f09a4100690 0x7f09a4197b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:57436/0 (socket says 172.21.15.80:57436) 2024-04-24T23:29:11.176 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.173+0000 7f09a8bb1640 1 -- 172.21.15.80:0/574571358 learned_addr learned my addr 172.21.15.80:0/574571358 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:29:11.176 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.173+0000 7f09a3fff640 1 --2- 172.21.15.80:0/574571358 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09a4101880 0x7f09a4198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:11.176 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.173+0000 7f09a93b2640 1 --2- 172.21.15.80:0/574571358 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09a4102240 0x7f09a419f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:11.176 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.173+0000 7f09a8bb1640 1 -- 172.21.15.80:0/574571358 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09a4102240 msgr2=0x7f09a419f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:11.177 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.173+0000 7f09a8bb1640 1 --2- 172.21.15.80:0/574571358 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09a4102240 0x7f09a419f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:11.177 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.173+0000 7f09a8bb1640 1 -- 172.21.15.80:0/574571358 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09a4101880 msgr2=0x7f09a4198060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:11.177 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.173+0000 7f09a8bb1640 1 --2- 172.21.15.80:0/574571358 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09a4101880 0x7f09a4198060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:11.177 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.173+0000 7f09a8bb1640 1 -- 172.21.15.80:0/574571358 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09a419f740 con 0x7f09a4100690 2024-04-24T23:29:11.177 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.173+0000 7f09a93b2640 1 --2- 172.21.15.80:0/574571358 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09a4102240 0x7f09a419f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:29:11.177 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.173+0000 7f09a8bb1640 1 --2- 172.21.15.80:0/574571358 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f09a4100690 0x7f09a4197b20 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f099800d770 tx=0x7f099800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:11.177 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.173+0000 7f09a1ffb640 1 -- 172.21.15.80:0/574571358 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0998017070 con 0x7f09a4100690 2024-04-24T23:29:11.177 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.173+0000 7f09a1ffb640 1 -- 172.21.15.80:0/574571358 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0998009c30 con 0x7f09a4100690 2024-04-24T23:29:11.177 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.173+0000 7f09aae15640 1 -- 172.21.15.80:0/574571358 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09a419fa30 con 0x7f09a4100690 2024-04-24T23:29:11.177 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.173+0000 7f09a1ffb640 1 -- 172.21.15.80:0/574571358 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0998003dd0 con 0x7f09a4100690 2024-04-24T23:29:11.177 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.173+0000 7f09aae15640 1 -- 172.21.15.80:0/574571358 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09a419ffd0 con 0x7f09a4100690 2024-04-24T23:29:11.177 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.174+0000 7f09aae15640 1 -- 172.21.15.80:0/574571358 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09a41a01a0 con 0x7f09a4100690 2024-04-24T23:29:11.178 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.175+0000 7f09a1ffb640 1 -- 172.21.15.80:0/574571358 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0998003fb0 con 0x7f09a4100690 2024-04-24T23:29:11.178 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.176+0000 7f09a1ffb640 1 --2- 172.21.15.80:0/574571358 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f09940a2b90 0x7f09940a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:11.178 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.176+0000 7f09a3fff640 1 --2- 172.21.15.80:0/574571358 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f09940a2b90 0x7f09940a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:11.178 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.176+0000 7f09a1ffb640 1 -- 172.21.15.80:0/574571358 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f09980ceda0 con 0x7f09a4100690 2024-04-24T23:29:11.178 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.176+0000 7f09a3fff640 1 --2- 172.21.15.80:0/574571358 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f09940a2b90 0x7f09940a5050 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f0990031440 tx=0x7f09900315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:11.181 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.181+0000 7f09a1ffb640 1 -- 172.21.15.80:0/574571358 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f099809b930 con 0x7f09a4100690 2024-04-24T23:29:11.395 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.394+0000 7f09aae15640 1 -- 172.21.15.80:0/574571358 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f09a41a08f0 con 0x7f09a4100690 2024-04-24T23:29:11.395 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.395+0000 7f09a1ffb640 1 -- 172.21.15.80:0/574571358 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11872 (secure 0 0 0) 0x7f0998009de0 con 0x7f09a4100690 2024-04-24T23:29:11.396 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-24T23:29:11.396 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/891591085","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":891591085},{"type":"v1","addr":"172.21.15.80:6835","nonce":891591085}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6837/259529026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6836","nonce":259529026},{"type":"v1","addr":"172.21.15.80:6837","nonce":259529026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14646,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6839/4194284854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6838","nonce":4194284854},{"type":"v1","addr":"172.21.15.80:6839","nonce":4194284854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14658,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6841/3145730958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6840","nonce":3145730958},{"type":"v1","addr":"172.21.15.80:6841","nonce":3145730958}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24395,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/3736784385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3736784385},{"type":"v1","addr":"172.21.15.134:6833","nonce":3736784385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/124777462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":124777462},{"type":"v1","addr":"172.21.15.148:6833","nonce":124777462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T23:27:19.300566+0000","modified":"2024-04-24T23:27:45.981839+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24496,"mds_1":24488,"mds_2":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/1399801120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1399801120},{"type":"v1","addr":"172.21.15.134:6835","nonce":1399801120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/430512704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":430512704},{"type":"v1","addr":"172.21.15.134:6837","nonce":430512704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/3002319310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3002319310},{"type":"v1","addr":"172.21.15.148:6835","nonce":3002319310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.148:6837/856528403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":856528403},{"type":"v1","addr":"172.21.15.148:6837","nonce":856528403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6839/1743815021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":1743815021},{"type":"v1","addr":"172.21.15.134:6839","nonce":1743815021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.148:6839/2952757594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":2952757594},{"type":"v1","addr":"172.21.15.148:6839","nonce":2952757594}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24496,"qdb_cluster":[24472,24488,24496]},"id":1}]} 2024-04-24T23:29:11.396 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 24 2024-04-24T23:29:11.398 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.397+0000 7f09aae15640 1 -- 172.21.15.80:0/574571358 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f09940a2b90 msgr2=0x7f09940a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:11.398 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.397+0000 7f09aae15640 1 --2- 172.21.15.80:0/574571358 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f09940a2b90 0x7f09940a5050 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f0990031440 tx=0x7f09900315b0 comp rx=0 tx=0).stop 2024-04-24T23:29:11.398 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.397+0000 7f09aae15640 1 -- 172.21.15.80:0/574571358 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f09a4100690 msgr2=0x7f09a4197b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:11.398 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.397+0000 7f09aae15640 1 --2- 172.21.15.80:0/574571358 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f09a4100690 0x7f09a4197b20 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f099800d770 tx=0x7f099800dc30 comp rx=0 tx=0).stop 2024-04-24T23:29:11.398 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.397+0000 7f09aae15640 1 -- 172.21.15.80:0/574571358 shutdown_connections 2024-04-24T23:29:11.398 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.397+0000 7f09aae15640 1 --2- 172.21.15.80:0/574571358 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f09940a2b90 0x7f09940a5050 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:11.398 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.397+0000 7f09aae15640 1 --2- 172.21.15.80:0/574571358 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09a4102240 0x7f09a419f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:11.398 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.397+0000 7f09aae15640 1 --2- 172.21.15.80:0/574571358 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09a4101880 0x7f09a4198060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:11.398 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.397+0000 7f09aae15640 1 --2- 172.21.15.80:0/574571358 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f09a4100690 0x7f09a4197b20 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:11.398 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.397+0000 7f09aae15640 1 -- 172.21.15.80:0/574571358 >> 172.21.15.80:0/574571358 conn(0x7f09a40fbc20 msgr2=0x7f09a40fd650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:11.399 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.397+0000 7f09aae15640 1 -- 172.21.15.80:0/574571358 shutdown_connections 2024-04-24T23:29:11.399 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.397+0000 7f09aae15640 1 -- 172.21.15.80:0/574571358 wait complete. 2024-04-24T23:29:11.408 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:29:11 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/3345832306' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T23:29:11.408 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:29:11 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/4077691008' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:29:11.408 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:29:11 smithi080 ceph-mon[29152]: pgmap v356: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 895 B/s wr, 8 op/s; 14 B/s, 0 objects/s recovering 2024-04-24T23:29:11.408 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:29:11 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/1597170235' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:29:11.409 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-24T23:27:19.300566+0000', 'modified': '2024-04-24T23:27:45.981839+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24496, 'mds_1': 24488, 'mds_2': 24472}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24428': {'gid': 24428, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.134:6835/1399801120', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6834', 'nonce': 1399801120}, {'type': 'v1', 'addr': '172.21.15.134:6835', 'nonce': 1399801120}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24458': {'gid': 24458, 'name': 'h', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.134:6837/430512704', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6836', 'nonce': 430512704}, {'type': 'v1', 'addr': '172.21.15.134:6837', 'nonce': 430512704}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24460': {'gid': 24460, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.148:6835/3002319310', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6834', 'nonce': 3002319310}, {'type': 'v1', 'addr': '172.21.15.148:6835', 'nonce': 3002319310}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24472': {'gid': 24472, 'name': 'c', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.148:6837/856528403', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6836', 'nonce': 856528403}, {'type': 'v1', 'addr': '172.21.15.148:6837', 'nonce': 856528403}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24488': {'gid': 24488, 'name': 'b', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.134:6839/1743815021', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6838', 'nonce': 1743815021}, {'type': 'v1', 'addr': '172.21.15.134:6839', 'nonce': 1743815021}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24496': {'gid': 24496, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.148:6839/2952757594', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6838', 'nonce': 2952757594}, {'type': 'v1', 'addr': '172.21.15.148:6839', 'nonce': 2952757594}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24496, 'qdb_cluster': [24472, 24488, 24496]} 2024-04-24T23:29:11.409 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-04-24T23:29:11.410 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-24T23:29:11.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:29:11 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/3345832306' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T23:29:11.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:29:11 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/4077691008' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:29:11.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:29:11 smithi148 ceph-mon[32528]: pgmap v356: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 895 B/s wr, 8 op/s; 14 B/s, 0 objects/s recovering 2024-04-24T23:29:11.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:29:11 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/1597170235' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:29:11.510 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.507+0000 7f2c70f22640 1 -- 172.21.15.80:0/2641862606 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2c6c101870 msgr2=0x7f2c6c101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:11.510 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.507+0000 7f2c70f22640 1 --2- 172.21.15.80:0/2641862606 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2c6c101870 0x7f2c6c101cf0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f2c54009900 tx=0x7f2c5402a410 comp rx=0 tx=0).stop 2024-04-24T23:29:11.510 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.507+0000 7f2c70f22640 1 -- 172.21.15.80:0/2641862606 shutdown_connections 2024-04-24T23:29:11.511 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.507+0000 7f2c70f22640 1 --2- 172.21.15.80:0/2641862606 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2c6c102230 0x7f2c6c108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:11.511 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.507+0000 7f2c70f22640 1 --2- 172.21.15.80:0/2641862606 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2c6c101870 0x7f2c6c101cf0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:11.511 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.507+0000 7f2c70f22640 1 --2- 172.21.15.80:0/2641862606 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2c6c100680 0x7f2c6c100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:11.511 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.507+0000 7f2c70f22640 1 -- 172.21.15.80:0/2641862606 >> 172.21.15.80:0/2641862606 conn(0x7f2c6c0fbbf0 msgr2=0x7f2c6c0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:11.511 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.507+0000 7f2c70f22640 1 -- 172.21.15.80:0/2641862606 shutdown_connections 2024-04-24T23:29:11.511 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.507+0000 7f2c70f22640 1 -- 172.21.15.80:0/2641862606 wait complete. 2024-04-24T23:29:11.511 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.508+0000 7f2c70f22640 1 Processor -- start 2024-04-24T23:29:11.511 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.508+0000 7f2c70f22640 1 -- start start 2024-04-24T23:29:11.511 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.508+0000 7f2c70f22640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2c6c100680 0x7f2c6c197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:11.513 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.508+0000 7f2c70f22640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2c6c101870 0x7f2c6c197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:11.514 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.508+0000 7f2c70f22640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2c6c102230 0x7f2c6c19f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:11.514 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.508+0000 7f2c70f22640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f2c6c10c2e0 con 0x7f2c6c100680 2024-04-24T23:29:11.514 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.508+0000 7f2c70f22640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2c6c10c160 con 0x7f2c6c102230 2024-04-24T23:29:11.514 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.508+0000 7f2c70f22640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f2c6c10c460 con 0x7f2c6c101870 2024-04-24T23:29:11.514 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.508+0000 7f2c6a59c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2c6c100680 0x7f2c6c197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:11.514 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.508+0000 7f2c6a59c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2c6c100680 0x7f2c6c197a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:57454/0 (socket says 172.21.15.80:57454) 2024-04-24T23:29:11.514 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.508+0000 7f2c6a59c640 1 -- 172.21.15.80:0/670880050 learned_addr learned my addr 172.21.15.80:0/670880050 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:29:11.514 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.508+0000 7f2c63fff640 1 --2- 172.21.15.80:0/670880050 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2c6c101870 0x7f2c6c197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:11.514 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.509+0000 7f2c6ad9d640 1 --2- 172.21.15.80:0/670880050 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2c6c102230 0x7f2c6c19f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:11.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.509+0000 7f2c6a59c640 1 -- 172.21.15.80:0/670880050 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2c6c101870 msgr2=0x7f2c6c197f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:11.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.509+0000 7f2c6a59c640 1 --2- 172.21.15.80:0/670880050 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2c6c101870 0x7f2c6c197f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:11.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.509+0000 7f2c6a59c640 1 -- 172.21.15.80:0/670880050 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2c6c102230 msgr2=0x7f2c6c19f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:11.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.509+0000 7f2c6a59c640 1 --2- 172.21.15.80:0/670880050 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2c6c102230 0x7f2c6c19f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:11.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.509+0000 7f2c6a59c640 1 -- 172.21.15.80:0/670880050 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c6c19f670 con 0x7f2c6c100680 2024-04-24T23:29:11.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.509+0000 7f2c6ad9d640 1 --2- 172.21.15.80:0/670880050 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2c6c102230 0x7f2c6c19f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:29:11.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.509+0000 7f2c6a59c640 1 --2- 172.21.15.80:0/670880050 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2c6c100680 0x7f2c6c197a50 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c00c970 tx=0x7f2c5c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:11.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.509+0000 7f2c637fe640 1 -- 172.21.15.80:0/670880050 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c5c007c10 con 0x7f2c6c100680 2024-04-24T23:29:11.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.509+0000 7f2c637fe640 1 -- 172.21.15.80:0/670880050 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c5c007db0 con 0x7f2c6c100680 2024-04-24T23:29:11.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.509+0000 7f2c70f22640 1 -- 172.21.15.80:0/670880050 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c6c19f960 con 0x7f2c6c100680 2024-04-24T23:29:11.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.509+0000 7f2c637fe640 1 -- 172.21.15.80:0/670880050 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c5c00f940 con 0x7f2c6c100680 2024-04-24T23:29:11.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.509+0000 7f2c70f22640 1 -- 172.21.15.80:0/670880050 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c6c19fe80 con 0x7f2c6c100680 2024-04-24T23:29:11.522 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.511+0000 7f2c637fe640 1 -- 172.21.15.80:0/670880050 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2c5c00fae0 con 0x7f2c6c100680 2024-04-24T23:29:11.522 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.511+0000 7f2c73c35740 1 Processor -- start 2024-04-24T23:29:11.522 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.512+0000 7f2c73c35740 1 -- start start 2024-04-24T23:29:11.522 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.512+0000 7f2c73c35740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c62d407440 0x55c62d407860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:11.522 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.512+0000 7f2c73c35740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c62d407e30 0x55c62d40c930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:11.522 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.512+0000 7f2c73c35740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c62d40cfa0 0x55c62d40f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:11.522 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.512+0000 7f2c73c35740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x55c62d36af00 con 0x55c62d407440 2024-04-24T23:29:11.522 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.512+0000 7f2c73c35740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c62d313750 con 0x55c62d40cfa0 2024-04-24T23:29:11.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.512+0000 7f2c73c35740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55c62d366ec0 con 0x55c62d407e30 2024-04-24T23:29:11.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.512+0000 7f2c6ad9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c62d407440 0x55c62d407860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:11.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.512+0000 7f2c6ad9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c62d407440 0x55c62d407860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:57460/0 (socket says 172.21.15.80:57460) 2024-04-24T23:29:11.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.512+0000 7f2c6ad9d640 1 -- 172.21.15.80:0/1306797964 learned_addr learned my addr 172.21.15.80:0/1306797964 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:29:11.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.512+0000 7f2c637fe640 1 --2- 172.21.15.80:0/670880050 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f2c580a2b90 0x7f2c580a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:11.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.512+0000 7f2c63fff640 1 --2- 172.21.15.80:0/1306797964 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c62d40cfa0 0x55c62d40f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:11.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.512+0000 7f2c6a59c640 1 --2- 172.21.15.80:0/1306797964 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c62d407e30 0x55c62d40c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:11.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.512+0000 7f2c6ad9d640 1 -- 172.21.15.80:0/1306797964 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c62d407e30 msgr2=0x55c62d40c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:11.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.512+0000 7f2c6ad9d640 1 --2- 172.21.15.80:0/1306797964 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c62d407e30 0x55c62d40c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:11.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.512+0000 7f2c6ad9d640 1 -- 172.21.15.80:0/1306797964 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c62d40cfa0 msgr2=0x55c62d40f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:11.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.512+0000 7f2c6ad9d640 1 --2- 172.21.15.80:0/1306797964 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c62d40cfa0 0x55c62d40f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:11.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.512+0000 7f2c6ad9d640 1 -- 172.21.15.80:0/1306797964 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c62d07f2d0 con 0x55c62d407440 2024-04-24T23:29:11.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.512+0000 7f2c637fe640 1 -- 172.21.15.80:0/670880050 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f2c5c0cf860 con 0x7f2c6c100680 2024-04-24T23:29:11.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.512+0000 7f2c6ad9d640 1 --2- 172.21.15.80:0/670880050 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f2c580a2b90 0x7f2c580a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:11.524 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.512+0000 7f2c6ad9d640 1 --2- 172.21.15.80:0/1306797964 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c62d407440 0x55c62d407860 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f2c64008f50 tx=0x7f2c6400d7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c8c1b3f7273cafa server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:11.524 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.513+0000 7f2c60ff9640 1 -- 172.21.15.80:0/1306797964 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c6401a8e0 con 0x55c62d407440 2024-04-24T23:29:11.524 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.513+0000 7f2c60ff9640 1 -- 172.21.15.80:0/1306797964 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c64019410 con 0x55c62d407440 2024-04-24T23:29:11.524 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.513+0000 7f2c60ff9640 1 -- 172.21.15.80:0/1306797964 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c6400e610 con 0x55c62d407440 2024-04-24T23:29:11.524 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.513+0000 7f2c73c35740 1 -- 172.21.15.80:0/1306797964 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c62d407440 msgr2=0x55c62d407860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:11.524 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.513+0000 7f2c73c35740 1 --2- 172.21.15.80:0/1306797964 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c62d407440 0x55c62d407860 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f2c64008f50 tx=0x7f2c6400d7d0 comp rx=0 tx=0).stop 2024-04-24T23:29:11.524 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.513+0000 7f2c6ad9d640 1 --2- 172.21.15.80:0/670880050 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f2c580a2b90 0x7f2c580a5050 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f2c6401e000 tx=0x7f2c6401f000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:11.524 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.513+0000 7f2c73c35740 1 -- 172.21.15.80:0/1306797964 shutdown_connections 2024-04-24T23:29:11.524 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.513+0000 7f2c73c35740 1 --2- 172.21.15.80:0/1306797964 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c62d40cfa0 0x55c62d40f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:11.524 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.513+0000 7f2c73c35740 1 --2- 172.21.15.80:0/1306797964 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c62d407e30 0x55c62d40c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:11.524 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.513+0000 7f2c73c35740 1 --2- 172.21.15.80:0/1306797964 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c62d407440 0x55c62d407860 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:11.524 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.513+0000 7f2c73c35740 1 -- 172.21.15.80:0/1306797964 >> 172.21.15.80:0/1306797964 conn(0x55c62d402ab0 msgr2=0x55c62d402eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:11.524 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.513+0000 7f2c73c35740 1 -- 172.21.15.80:0/1306797964 shutdown_connections 2024-04-24T23:29:11.524 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.514+0000 7f2c73c35740 1 -- 172.21.15.80:0/1306797964 wait complete. 2024-04-24T23:29:11.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.514+0000 7f2c73c35740 1 -- start start 2024-04-24T23:29:11.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.514+0000 7f2c73c35740 1 Processor -- start 2024-04-24T23:29:11.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.514+0000 7f2c73c35740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c62d407440 0x55c62d411800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:11.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.514+0000 7f2c73c35740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c62d407e30 0x55c62d411d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:11.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.514+0000 7f2c73c35740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c62d40cfa0 0x55c62d418d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:11.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.514+0000 7f2c73c35740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x55c62d40ff50 con 0x55c62d40cfa0 2024-04-24T23:29:11.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.514+0000 7f2c73c35740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c62d326cf0 con 0x55c62d407e30 2024-04-24T23:29:11.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.514+0000 7f2c73c35740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55c62d4100d0 con 0x55c62d407440 2024-04-24T23:29:11.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.514+0000 7f2c73c35740 10 client.0 _ll_register_callbacks cb 0x55c62d3266e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:29:11.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.514+0000 7f2c6ad9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c62d40cfa0 0x55c62d418d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:11.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.514+0000 7f2c63fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c62d407e30 0x55c62d411d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:11.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.514+0000 7f2c63fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c62d407e30 0x55c62d411d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.80:53090/0 (socket says 172.21.15.80:53090) 2024-04-24T23:29:11.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.514+0000 7f2c6ad9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c62d40cfa0 0x55c62d418d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:57462/0 (socket says 172.21.15.80:57462) 2024-04-24T23:29:11.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.514+0000 7f2c63fff640 1 -- 172.21.15.80:0/2406948614 learned_addr learned my addr 172.21.15.80:0/2406948614 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:29:11.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.515+0000 7f2c6a59c640 1 --2- 172.21.15.80:0/2406948614 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c62d407440 0x55c62d411800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:11.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.515+0000 7f2c6ad9d640 1 -- 172.21.15.80:0/2406948614 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c62d407440 msgr2=0x55c62d411800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:11.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.515+0000 7f2c6ad9d640 1 --2- 172.21.15.80:0/2406948614 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c62d407440 0x55c62d411800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:11.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.515+0000 7f2c6ad9d640 1 -- 172.21.15.80:0/2406948614 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c62d407e30 msgr2=0x55c62d411d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:11.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.515+0000 7f2c6ad9d640 1 --2- 172.21.15.80:0/2406948614 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c62d407e30 0x55c62d411d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:11.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.515+0000 7f2c6ad9d640 1 -- 172.21.15.80:0/2406948614 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c62d1496b0 con 0x55c62d40cfa0 2024-04-24T23:29:11.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.515+0000 7f2c63fff640 1 --2- 172.21.15.80:0/2406948614 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c62d407e30 0x55c62d411d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:29:11.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.515+0000 7f2c6ad9d640 1 --2- 172.21.15.80:0/2406948614 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c62d40cfa0 0x55c62d418d80 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f2c64008250 tx=0x7f2c640197f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf2da8a6c08358a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:11.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.515+0000 7f2c51ffb640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-04-24T23:29:11.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.515+0000 7f2c51ffb640 1 -- 172.21.15.80:0/2406948614 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c64036060 con 0x55c62d40cfa0 2024-04-24T23:29:11.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.515+0000 7f2c51ffb640 1 -- 172.21.15.80:0/2406948614 <== mon.0 v2:172.21.15.80:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f2c6400eb30 con 0x55c62d40cfa0 2024-04-24T23:29:11.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.515+0000 7f2c73c35740 1 -- 172.21.15.80:0/2406948614 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c64019410 con 0x55c62d40cfa0 2024-04-24T23:29:11.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.515+0000 7f2c51ffb640 1 -- 172.21.15.80:0/2406948614 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f2c6403d050 con 0x55c62d40cfa0 2024-04-24T23:29:11.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.515+0000 7f2c73c35740 10 client.15249 fetch_fsmap learned FSMap version 24 2024-04-24T23:29:11.527 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.515+0000 7f2c73c35740 1 -- 172.21.15.80:0/2406948614 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55c62d41bd60 con 0x55c62d40cfa0 2024-04-24T23:29:11.527 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.515+0000 7f2c51ffb640 1 -- 172.21.15.80:0/2406948614 <== mon.0 v2:172.21.15.80:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c64019410 con 0x55c62d40cfa0 2024-04-24T23:29:11.527 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.516+0000 7f2c51ffb640 1 -- 172.21.15.80:0/2406948614 <== mon.0 v2:172.21.15.80:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f2c64030030 con 0x55c62d40cfa0 2024-04-24T23:29:11.527 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.516+0000 7f2c73c35740 10 client.15249 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T23:29:11.527 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.516+0000 7f2c73c35740 10 client.15249 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-24T23:29:11.527 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.516+0000 7f2c73c35740 20 client.15249 populate_metadata read hostname 'smithi080' 2024-04-24T23:29:11.527 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.516+0000 7f2c73c35740 1 --2- 172.21.15.80:0/2406948614 >> [v2:172.21.15.148:6836/856528403,v1:172.21.15.148:6837/856528403] conn(0x55c62d41e1f0 0x55c62d41f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:11.527 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.516+0000 7f2c73c35740 4 client.15249 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:29:11.527 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.516+0000 7f2c73c35740 1 -- 172.21.15.80:0/2406948614 --> [v2:172.21.15.148:6836/856528403,v1:172.21.15.148:6837/856528403] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c62d378440 con 0x55c62d41e1f0 2024-04-24T23:29:11.527 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.516+0000 7f2c6a59c640 1 --2- 172.21.15.80:0/2406948614 >> [v2:172.21.15.148:6836/856528403,v1:172.21.15.148:6837/856528403] conn(0x55c62d41e1f0 0x55c62d41f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:11.528 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.516+0000 7f2c51ffb640 10 client.15249 ms_handle_connect on v2:172.21.15.148:6836/856528403 2024-04-24T23:29:11.528 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.516+0000 7f2c6a59c640 1 --2- 172.21.15.80:0/2406948614 >> [v2:172.21.15.148:6836/856528403,v1:172.21.15.148:6837/856528403] conn(0x55c62d41e1f0 0x55c62d41f660 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5938b979d1527aa server_cookie=a4054fe9217009c9 in_seq=0 out_seq=0 2024-04-24T23:29:11.528 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.517+0000 7f2c51ffb640 1 -- 172.21.15.80:0/2406948614 <== mds.2 v2:172.21.15.148:6836/856528403 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c62d378440 con 0x55c62d41e1f0 2024-04-24T23:29:11.528 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.517+0000 7f2c51ffb640 10 client.15249 handle_command_reply: tid=0 2024-04-24T23:29:11.528 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.517+0000 7f2c73c35740 1 client.15249 shutdown 2024-04-24T23:29:11.528 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.517+0000 7f2c73c35740 1 -- 172.21.15.80:0/2406948614 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c62d40cfa0 msgr2=0x55c62d418d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:11.528 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.517+0000 7f2c73c35740 1 --2- 172.21.15.80:0/2406948614 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c62d40cfa0 0x55c62d418d80 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f2c64008250 tx=0x7f2c640197f0 comp rx=0 tx=0).stop 2024-04-24T23:29:11.528 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.517+0000 7f2c73c35740 1 -- 172.21.15.80:0/2406948614 shutdown_connections 2024-04-24T23:29:11.529 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.517+0000 7f2c73c35740 1 --2- 172.21.15.80:0/2406948614 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c62d40cfa0 0x55c62d418d80 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:11.529 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.517+0000 7f2c73c35740 1 --2- 172.21.15.80:0/2406948614 >> [v2:172.21.15.148:6836/856528403,v1:172.21.15.148:6837/856528403] conn(0x55c62d41e1f0 0x55c62d41f660 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:11.529 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.517+0000 7f2c73c35740 1 --2- 172.21.15.80:0/2406948614 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c62d407e30 0x55c62d411d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:11.529 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.517+0000 7f2c73c35740 1 --2- 172.21.15.80:0/2406948614 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c62d407440 0x55c62d411800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:11.529 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.517+0000 7f2c51ffb640 0 client.15249 ms_handle_reset on v2:172.21.15.148:6836/856528403 2024-04-24T23:29:11.529 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.517+0000 7f2c73c35740 1 -- 172.21.15.80:0/2406948614 >> 172.21.15.80:0/2406948614 conn(0x55c62d402ab0 msgr2=0x55c62d401ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:11.529 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.517+0000 7f2c73c35740 1 -- 172.21.15.80:0/2406948614 shutdown_connections 2024-04-24T23:29:11.529 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.517+0000 7f2c73c35740 1 -- 172.21.15.80:0/2406948614 wait complete. 2024-04-24T23:29:11.529 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.517+0000 7f2c73c35740 20 client.15249 trim_cache size 0 max 16384 2024-04-24T23:29:11.542 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.535+0000 7f2c73c35740 1 Processor -- start 2024-04-24T23:29:11.542 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.535+0000 7f2c73c35740 1 -- start start 2024-04-24T23:29:11.542 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.535+0000 7f2c73c35740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c62d407440 0x55c62d41d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:11.542 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.535+0000 7f2c73c35740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c62d407e30 0x55c62d4091b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:11.542 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.535+0000 7f2c73c35740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c62d40cfa0 0x55c62d4096f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:11.543 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.535+0000 7f2c73c35740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x55c62d41d9a0 con 0x55c62d407e30 2024-04-24T23:29:11.543 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.535+0000 7f2c73c35740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c62d41e070 con 0x55c62d407440 2024-04-24T23:29:11.543 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.535+0000 7f2c73c35740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55c62d41e740 con 0x55c62d40cfa0 2024-04-24T23:29:11.543 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.535+0000 7f2c63fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c62d407e30 0x55c62d4091b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:11.543 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.535+0000 7f2c63fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c62d407e30 0x55c62d4091b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:57464/0 (socket says 172.21.15.80:57464) 2024-04-24T23:29:11.543 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.535+0000 7f2c63fff640 1 -- 172.21.15.80:0/976518224 learned_addr learned my addr 172.21.15.80:0/976518224 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:29:11.543 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.536+0000 7f2c6a59c640 1 --2- 172.21.15.80:0/976518224 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c62d407440 0x55c62d41d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:11.543 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.536+0000 7f2c6ad9d640 1 --2- 172.21.15.80:0/976518224 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c62d40cfa0 0x55c62d4096f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:11.543 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.536+0000 7f2c63fff640 1 -- 172.21.15.80:0/976518224 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c62d40cfa0 msgr2=0x55c62d4096f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:11.543 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.536+0000 7f2c63fff640 1 --2- 172.21.15.80:0/976518224 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c62d40cfa0 0x55c62d4096f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:11.544 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.536+0000 7f2c63fff640 1 -- 172.21.15.80:0/976518224 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c62d407440 msgr2=0x55c62d41d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:11.544 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.536+0000 7f2c63fff640 1 --2- 172.21.15.80:0/976518224 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c62d407440 0x55c62d41d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:11.544 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.536+0000 7f2c63fff640 1 -- 172.21.15.80:0/976518224 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c62d3fcc10 con 0x55c62d407e30 2024-04-24T23:29:11.544 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.536+0000 7f2c63fff640 1 --2- 172.21.15.80:0/976518224 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c62d407e30 0x55c62d4091b0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f2c54005ab0 tx=0x7f2c54005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82ae2e4ce80cd933 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:11.544 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.536+0000 7f2c53fff640 1 -- 172.21.15.80:0/976518224 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c54035040 con 0x55c62d407e30 2024-04-24T23:29:11.544 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.536+0000 7f2c53fff640 1 -- 172.21.15.80:0/976518224 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c54005020 con 0x55c62d407e30 2024-04-24T23:29:11.544 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.536+0000 7f2c53fff640 1 -- 172.21.15.80:0/976518224 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c54004b90 con 0x55c62d407e30 2024-04-24T23:29:11.544 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.537+0000 7f2c73c35740 1 -- 172.21.15.80:0/976518224 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c62d407e30 msgr2=0x55c62d4091b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:11.544 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.537+0000 7f2c73c35740 1 --2- 172.21.15.80:0/976518224 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c62d407e30 0x55c62d4091b0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f2c54005ab0 tx=0x7f2c54005c00 comp rx=0 tx=0).stop 2024-04-24T23:29:11.544 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.537+0000 7f2c73c35740 1 -- 172.21.15.80:0/976518224 shutdown_connections 2024-04-24T23:29:11.544 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.537+0000 7f2c73c35740 1 --2- 172.21.15.80:0/976518224 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c62d40cfa0 0x55c62d4096f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:11.544 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.537+0000 7f2c73c35740 1 --2- 172.21.15.80:0/976518224 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c62d407e30 0x55c62d4091b0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:11.544 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.537+0000 7f2c73c35740 1 --2- 172.21.15.80:0/976518224 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c62d407440 0x55c62d41d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:11.544 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.537+0000 7f2c73c35740 1 -- 172.21.15.80:0/976518224 >> 172.21.15.80:0/976518224 conn(0x55c62d402ab0 msgr2=0x55c62d3fd690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:11.545 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.537+0000 7f2c73c35740 1 -- 172.21.15.80:0/976518224 shutdown_connections 2024-04-24T23:29:11.545 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.537+0000 7f2c73c35740 1 -- 172.21.15.80:0/976518224 wait complete. 2024-04-24T23:29:11.545 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.537+0000 7f2c73c35740 1 -- start start 2024-04-24T23:29:11.545 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.537+0000 7f2c73c35740 1 Processor -- start 2024-04-24T23:29:11.545 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.537+0000 7f2c73c35740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c62d407440 0x55c62d413810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:11.545 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.537+0000 7f2c73c35740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c62d407e30 0x55c62d413d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:11.545 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.537+0000 7f2c73c35740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c62d40cfa0 0x55c62d416d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:11.545 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.537+0000 7f2c73c35740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x55c62d41f4e0 con 0x55c62d407e30 2024-04-24T23:29:11.545 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.537+0000 7f2c73c35740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c62d41ee10 con 0x55c62d407440 2024-04-24T23:29:11.545 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.537+0000 7f2c73c35740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55c62d41fbb0 con 0x55c62d40cfa0 2024-04-24T23:29:11.546 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.537+0000 7f2c73c35740 10 client.0 _ll_register_callbacks cb 0x7f2c64008d80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:29:11.546 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.537+0000 7f2c63fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c62d407e30 0x55c62d413d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:11.546 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.537+0000 7f2c63fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c62d407e30 0x55c62d413d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:57466/0 (socket says 172.21.15.80:57466) 2024-04-24T23:29:11.546 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.537+0000 7f2c63fff640 1 -- 172.21.15.80:0/3739219358 learned_addr learned my addr 172.21.15.80:0/3739219358 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:29:11.546 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.537+0000 7f2c6ad9d640 1 --2- 172.21.15.80:0/3739219358 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c62d40cfa0 0x55c62d416d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:11.546 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.537+0000 7f2c63fff640 1 -- 172.21.15.80:0/3739219358 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c62d40cfa0 msgr2=0x55c62d416d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:11.546 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.537+0000 7f2c63fff640 1 --2- 172.21.15.80:0/3739219358 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c62d40cfa0 0x55c62d416d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:11.546 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.538+0000 7f2c63fff640 1 -- 172.21.15.80:0/3739219358 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c62d407440 msgr2=0x55c62d413810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:11.546 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.538+0000 7f2c63fff640 1 --2- 172.21.15.80:0/3739219358 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c62d407440 0x55c62d413810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:11.547 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.538+0000 7f2c63fff640 1 -- 172.21.15.80:0/3739219358 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c62d417310 con 0x55c62d407e30 2024-04-24T23:29:11.547 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.538+0000 7f2c63fff640 1 --2- 172.21.15.80:0/3739219358 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c62d407e30 0x55c62d413d50 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f2c54031610 tx=0x7f2c5402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e016827e4b4f3fe server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:11.547 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.538+0000 7f2c51ffb640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-04-24T23:29:11.547 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.538+0000 7f2c51ffb640 1 -- 172.21.15.80:0/3739219358 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c54053060 con 0x55c62d407e30 2024-04-24T23:29:11.547 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.538+0000 7f2c51ffb640 1 -- 172.21.15.80:0/3739219358 <== mon.0 v2:172.21.15.80:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f2c5404b070 con 0x55c62d407e30 2024-04-24T23:29:11.547 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.538+0000 7f2c73c35740 1 -- 172.21.15.80:0/3739219358 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c54005020 con 0x55c62d407e30 2024-04-24T23:29:11.547 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.538+0000 7f2c51ffb640 1 -- 172.21.15.80:0/3739219358 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f2c5405a050 con 0x55c62d407e30 2024-04-24T23:29:11.547 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.538+0000 7f2c73c35740 10 client.15255 fetch_fsmap learned FSMap version 24 2024-04-24T23:29:11.547 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.538+0000 7f2c73c35740 1 -- 172.21.15.80:0/3739219358 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55c62d418610 con 0x55c62d407e30 2024-04-24T23:29:11.547 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.538+0000 7f2c51ffb640 1 -- 172.21.15.80:0/3739219358 <== mon.0 v2:172.21.15.80:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c54005020 con 0x55c62d407e30 2024-04-24T23:29:11.547 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.538+0000 7f2c51ffb640 1 -- 172.21.15.80:0/3739219358 <== mon.0 v2:172.21.15.80:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f2c54046410 con 0x55c62d407e30 2024-04-24T23:29:11.547 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.538+0000 7f2c73c35740 10 client.15255 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T23:29:11.547 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.538+0000 7f2c73c35740 10 client.15255 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-24T23:29:11.548 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.538+0000 7f2c73c35740 20 client.15255 populate_metadata read hostname 'smithi080' 2024-04-24T23:29:11.548 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.538+0000 7f2c73c35740 1 --2- 172.21.15.80:0/3739219358 >> [v2:172.21.15.148:6836/856528403,v1:172.21.15.148:6837/856528403] conn(0x55c62d488820 0x55c62d484c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:11.548 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.538+0000 7f2c73c35740 4 client.15255 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T23:29:11.548 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.538+0000 7f2c73c35740 1 -- 172.21.15.80:0/3739219358 --> [v2:172.21.15.148:6836/856528403,v1:172.21.15.148:6837/856528403] -- command(tid 0: {"prefix": "status"}) -- 0x55c62d372790 con 0x55c62d488820 2024-04-24T23:29:11.548 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.539+0000 7f2c6a59c640 1 --2- 172.21.15.80:0/3739219358 >> [v2:172.21.15.148:6836/856528403,v1:172.21.15.148:6837/856528403] conn(0x55c62d488820 0x55c62d484c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:11.548 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.539+0000 7f2c6a59c640 1 --2- 172.21.15.80:0/3739219358 >> [v2:172.21.15.148:6836/856528403,v1:172.21.15.148:6837/856528403] conn(0x55c62d488820 0x55c62d484c60 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a9ba1cfe4ca516b5 server_cookie=588e54ea6e99ecb8 in_seq=0 out_seq=0 2024-04-24T23:29:11.548 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.539+0000 7f2c51ffb640 10 client.15255 ms_handle_connect on v2:172.21.15.148:6836/856528403 2024-04-24T23:29:11.548 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.539+0000 7f2c51ffb640 1 -- 172.21.15.80:0/3739219358 <== mds.2 v2:172.21.15.148:6836/856528403 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55c62d372790 con 0x55c62d488820 2024-04-24T23:29:11.548 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.539+0000 7f2c51ffb640 10 client.15255 handle_command_reply: tid=0 2024-04-24T23:29:11.548 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.539+0000 7f2c73c35740 1 client.15255 shutdown 2024-04-24T23:29:11.548 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.539+0000 7f2c73c35740 1 -- 172.21.15.80:0/3739219358 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c62d407e30 msgr2=0x55c62d413d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:11.549 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.539+0000 7f2c73c35740 1 --2- 172.21.15.80:0/3739219358 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c62d407e30 0x55c62d413d50 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f2c54031610 tx=0x7f2c5402a410 comp rx=0 tx=0).stop 2024-04-24T23:29:11.549 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.539+0000 7f2c73c35740 1 -- 172.21.15.80:0/3739219358 shutdown_connections 2024-04-24T23:29:11.549 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.539+0000 7f2c73c35740 1 --2- 172.21.15.80:0/3739219358 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c62d40cfa0 0x55c62d416d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:11.549 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.539+0000 7f2c73c35740 1 --2- 172.21.15.80:0/3739219358 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c62d407e30 0x55c62d413d50 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:11.550 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.539+0000 7f2c73c35740 1 --2- 172.21.15.80:0/3739219358 >> [v2:172.21.15.148:6836/856528403,v1:172.21.15.148:6837/856528403] conn(0x55c62d488820 0x55c62d484c60 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:11.550 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.539+0000 7f2c73c35740 1 --2- 172.21.15.80:0/3739219358 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c62d407440 0x55c62d413810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:11.550 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.539+0000 7f2c51ffb640 0 client.15255 ms_handle_reset on v2:172.21.15.148:6836/856528403 2024-04-24T23:29:11.550 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.539+0000 7f2c73c35740 1 -- 172.21.15.80:0/3739219358 >> 172.21.15.80:0/3739219358 conn(0x55c62d402ab0 msgr2=0x55c62d41d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:11.550 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.539+0000 7f2c73c35740 1 -- 172.21.15.80:0/3739219358 shutdown_connections 2024-04-24T23:29:11.550 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.539+0000 7f2c73c35740 1 -- 172.21.15.80:0/3739219358 wait complete. 2024-04-24T23:29:11.550 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.540+0000 7f2c73c35740 20 client.15255 trim_cache size 0 max 16384 2024-04-24T23:29:11.551 INFO:teuthology.orchestra.run.smithi080.stdout:{ 2024-04-24T23:29:11.551 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_fsid": "ec7885aa-0290-11ef-bc93-c7b262605968", 2024-04-24T23:29:11.551 INFO:teuthology.orchestra.run.smithi080.stdout: "whoami": 2, 2024-04-24T23:29:11.551 INFO:teuthology.orchestra.run.smithi080.stdout: "id": 24472, 2024-04-24T23:29:11.551 INFO:teuthology.orchestra.run.smithi080.stdout: "want_state": "up:active", 2024-04-24T23:29:11.551 INFO:teuthology.orchestra.run.smithi080.stdout: "state": "up:active", 2024-04-24T23:29:11.551 INFO:teuthology.orchestra.run.smithi080.stdout: "fs_name": "cephfs", 2024-04-24T23:29:11.551 INFO:teuthology.orchestra.run.smithi080.stdout: "rank_uptime": 86.562501103000002, 2024-04-24T23:29:11.551 INFO:teuthology.orchestra.run.smithi080.stdout: "mdsmap_epoch": 24, 2024-04-24T23:29:11.551 INFO:teuthology.orchestra.run.smithi080.stdout: "osdmap_epoch": 94, 2024-04-24T23:29:11.551 INFO:teuthology.orchestra.run.smithi080.stdout: "osdmap_epoch_barrier": 90, 2024-04-24T23:29:11.552 INFO:teuthology.orchestra.run.smithi080.stdout: "uptime": 132.77579998900001 2024-04-24T23:29:11.552 INFO:teuthology.orchestra.run.smithi080.stdout:} 2024-04-24T23:29:11.554 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.540+0000 7f2c70f22640 1 -- 172.21.15.80:0/670880050 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f2c580a2b90 msgr2=0x7f2c580a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:11.554 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.540+0000 7f2c70f22640 1 --2- 172.21.15.80:0/670880050 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f2c580a2b90 0x7f2c580a5050 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f2c6401e000 tx=0x7f2c6401f000 comp rx=0 tx=0).stop 2024-04-24T23:29:11.554 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.540+0000 7f2c70f22640 1 -- 172.21.15.80:0/670880050 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2c6c100680 msgr2=0x7f2c6c197a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:11.554 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.540+0000 7f2c70f22640 1 --2- 172.21.15.80:0/670880050 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2c6c100680 0x7f2c6c197a50 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c00c970 tx=0x7f2c5c00ce30 comp rx=0 tx=0).stop 2024-04-24T23:29:11.554 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.540+0000 7f2c70f22640 1 -- 172.21.15.80:0/670880050 shutdown_connections 2024-04-24T23:29:11.554 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.540+0000 7f2c70f22640 1 --2- 172.21.15.80:0/670880050 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f2c580a2b90 0x7f2c580a5050 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:11.554 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.540+0000 7f2c70f22640 1 --2- 172.21.15.80:0/670880050 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2c6c102230 0x7f2c6c19f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:11.554 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.540+0000 7f2c70f22640 1 --2- 172.21.15.80:0/670880050 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2c6c101870 0x7f2c6c197f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:11.554 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.540+0000 7f2c70f22640 1 --2- 172.21.15.80:0/670880050 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2c6c100680 0x7f2c6c197a50 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:11.554 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.540+0000 7f2c70f22640 1 -- 172.21.15.80:0/670880050 >> 172.21.15.80:0/670880050 conn(0x7f2c6c0fbbf0 msgr2=0x7f2c6c0fc270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:11.554 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.540+0000 7f2c70f22640 1 -- 172.21.15.80:0/670880050 shutdown_connections 2024-04-24T23:29:11.554 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.540+0000 7f2c70f22640 1 -- 172.21.15.80:0/670880050 wait complete. 2024-04-24T23:29:11.555 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-24T23:29:11.666 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.653+0000 7f0e3944d640 1 -- 172.21.15.80:0/3341103727 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0e34100680 msgr2=0x7f0e34100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:11.666 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.653+0000 7f0e3944d640 1 --2- 172.21.15.80:0/3341103727 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0e34100680 0x7f0e34100aa0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f0e24009ca0 tx=0x7f0e2402a8c0 comp rx=0 tx=0).stop 2024-04-24T23:29:11.666 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.653+0000 7f0e3944d640 1 -- 172.21.15.80:0/3341103727 shutdown_connections 2024-04-24T23:29:11.666 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.653+0000 7f0e3944d640 1 --2- 172.21.15.80:0/3341103727 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e34102230 0x7f0e34108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:11.666 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.653+0000 7f0e3944d640 1 --2- 172.21.15.80:0/3341103727 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0e34101870 0x7f0e34101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:11.666 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.653+0000 7f0e3944d640 1 --2- 172.21.15.80:0/3341103727 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0e34100680 0x7f0e34100aa0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:11.666 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.653+0000 7f0e3944d640 1 -- 172.21.15.80:0/3341103727 >> 172.21.15.80:0/3341103727 conn(0x7f0e340fbbf0 msgr2=0x7f0e340fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:11.667 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.653+0000 7f0e3944d640 1 -- 172.21.15.80:0/3341103727 shutdown_connections 2024-04-24T23:29:11.667 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.653+0000 7f0e3944d640 1 -- 172.21.15.80:0/3341103727 wait complete. 2024-04-24T23:29:11.667 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.654+0000 7f0e3944d640 1 Processor -- start 2024-04-24T23:29:11.667 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.654+0000 7f0e3944d640 1 -- start start 2024-04-24T23:29:11.667 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.654+0000 7f0e3944d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0e34100680 0x7f0e34197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:11.667 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.654+0000 7f0e3944d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e34101870 0x7f0e34197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:11.667 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.654+0000 7f0e3944d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0e34102230 0x7f0e3419f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:11.668 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.654+0000 7f0e3944d640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f0e3410c2d0 con 0x7f0e34100680 2024-04-24T23:29:11.668 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.654+0000 7f0e3944d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0e3410c150 con 0x7f0e34102230 2024-04-24T23:29:11.668 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.654+0000 7f0e3944d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f0e3410c450 con 0x7f0e34101870 2024-04-24T23:29:11.668 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.654+0000 7f0e32ffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0e34100680 0x7f0e34197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:11.668 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.654+0000 7f0e32ffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0e34100680 0x7f0e34197a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:57494/0 (socket says 172.21.15.80:57494) 2024-04-24T23:29:11.668 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.654+0000 7f0e32ffd640 1 -- 172.21.15.80:0/1021682405 learned_addr learned my addr 172.21.15.80:0/1021682405 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:29:11.668 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.654+0000 7f0e327fc640 1 --2- 172.21.15.80:0/1021682405 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e34101870 0x7f0e34197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:11.668 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.654+0000 7f0e32ffd640 1 -- 172.21.15.80:0/1021682405 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e34101870 msgr2=0x7f0e34197fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:11.668 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.654+0000 7f0e337fe640 1 --2- 172.21.15.80:0/1021682405 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0e34102230 0x7f0e3419f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:11.668 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.654+0000 7f0e32ffd640 1 --2- 172.21.15.80:0/1021682405 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e34101870 0x7f0e34197fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:11.668 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.654+0000 7f0e32ffd640 1 -- 172.21.15.80:0/1021682405 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0e34102230 msgr2=0x7f0e3419f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:11.668 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.654+0000 7f0e32ffd640 1 --2- 172.21.15.80:0/1021682405 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0e34102230 0x7f0e3419f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:11.668 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.654+0000 7f0e32ffd640 1 -- 172.21.15.80:0/1021682405 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e3419f6b0 con 0x7f0e34100680 2024-04-24T23:29:11.669 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.654+0000 7f0e327fc640 1 --2- 172.21.15.80:0/1021682405 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e34101870 0x7f0e34197fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:29:11.669 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.654+0000 7f0e337fe640 1 --2- 172.21.15.80:0/1021682405 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0e34102230 0x7f0e3419f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:29:11.669 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.655+0000 7f0e32ffd640 1 --2- 172.21.15.80:0/1021682405 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0e34100680 0x7f0e34197a90 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f0e24004af0 tx=0x7f0e24031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:11.669 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.655+0000 7f0e1bfff640 1 -- 172.21.15.80:0/1021682405 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e2402fb60 con 0x7f0e34100680 2024-04-24T23:29:11.669 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.655+0000 7f0e1bfff640 1 -- 172.21.15.80:0/1021682405 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e2402fd00 con 0x7f0e34100680 2024-04-24T23:29:11.669 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.655+0000 7f0e3944d640 1 -- 172.21.15.80:0/1021682405 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e3419f940 con 0x7f0e34100680 2024-04-24T23:29:11.669 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.655+0000 7f0e1bfff640 1 -- 172.21.15.80:0/1021682405 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e24046920 con 0x7f0e34100680 2024-04-24T23:29:11.669 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.655+0000 7f0e3944d640 1 -- 172.21.15.80:0/1021682405 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e3419fe90 con 0x7f0e34100680 2024-04-24T23:29:11.669 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.657+0000 7f0e1bfff640 1 -- 172.21.15.80:0/1021682405 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0e24030c30 con 0x7f0e34100680 2024-04-24T23:29:11.669 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.657+0000 7f0e3bdfd740 1 Processor -- start 2024-04-24T23:29:11.670 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.657+0000 7f0e3bdfd740 1 -- start start 2024-04-24T23:29:11.670 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.657+0000 7f0e3bdfd740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fda28d7850 0x55fda28d7c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:11.670 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.657+0000 7f0e3bdfd740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55fda28d8240 0x55fda28dcd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:11.670 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.657+0000 7f0e3bdfd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fda28dd3b0 0x55fda28df840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:11.670 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.657+0000 7f0e3bdfd740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x55fda2836f00 con 0x55fda28d8240 2024-04-24T23:29:11.670 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.657+0000 7f0e3bdfd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55fda27df750 con 0x55fda28dd3b0 2024-04-24T23:29:11.670 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.657+0000 7f0e3bdfd740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55fda2832ec0 con 0x55fda28d7850 2024-04-24T23:29:11.670 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.657+0000 7f0e1bfff640 1 --2- 172.21.15.80:0/1021682405 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f0e1c0a2b90 0x7f0e1c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:11.670 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.657+0000 7f0e32ffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55fda28d8240 0x55fda28dcd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:11.670 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.657+0000 7f0e32ffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55fda28d8240 0x55fda28dcd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:57500/0 (socket says 172.21.15.80:57500) 2024-04-24T23:29:11.671 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.657+0000 7f0e32ffd640 1 -- 172.21.15.80:0/276370997 learned_addr learned my addr 172.21.15.80:0/276370997 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:29:11.671 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.658+0000 7f0e337fe640 1 --2- 172.21.15.80:0/1021682405 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f0e1c0a2b90 0x7f0e1c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:11.671 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.658+0000 7f0e327fc640 1 --2- 172.21.15.80:0/276370997 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fda28dd3b0 0x55fda28df840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:11.671 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.658+0000 7f0e337fe640 1 --2- 172.21.15.80:0/276370997 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fda28d7850 0x55fda28d7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:11.671 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.658+0000 7f0e1bfff640 1 -- 172.21.15.80:0/1021682405 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f0e240f0b80 con 0x7f0e34100680 2024-04-24T23:29:11.671 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.658+0000 7f0e32ffd640 1 -- 172.21.15.80:0/276370997 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fda28d7850 msgr2=0x55fda28d7c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:11.671 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.658+0000 7f0e32ffd640 1 --2- 172.21.15.80:0/276370997 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fda28d7850 0x55fda28d7c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:11.671 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.658+0000 7f0e32ffd640 1 -- 172.21.15.80:0/276370997 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fda28dd3b0 msgr2=0x55fda28df840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:11.671 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.658+0000 7f0e32ffd640 1 --2- 172.21.15.80:0/276370997 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fda28dd3b0 0x55fda28df840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:11.672 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.658+0000 7f0e32ffd640 1 -- 172.21.15.80:0/276370997 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fda254b2d0 con 0x55fda28d8240 2024-04-24T23:29:11.672 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.658+0000 7f0e337fe640 1 --2- 172.21.15.80:0/1021682405 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f0e1c0a2b90 0x7f0e1c0a5050 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c00afd0 tx=0x7f0e2c00a9b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:11.672 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.658+0000 7f0e32ffd640 1 --2- 172.21.15.80:0/276370997 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55fda28d8240 0x55fda28dcd40 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f0e2402f540 tx=0x7f0e2402a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c0d5c1311fb06fa server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:11.672 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.658+0000 7f0e197fa640 1 -- 172.21.15.80:0/276370997 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e2402feb0 con 0x55fda28d8240 2024-04-24T23:29:11.672 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.658+0000 7f0e197fa640 1 -- 172.21.15.80:0/276370997 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e24036020 con 0x55fda28d8240 2024-04-24T23:29:11.672 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.658+0000 7f0e197fa640 1 -- 172.21.15.80:0/276370997 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e240bb3a0 con 0x55fda28d8240 2024-04-24T23:29:11.672 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.659+0000 7f0e3bdfd740 1 -- 172.21.15.80:0/276370997 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55fda28d8240 msgr2=0x55fda28dcd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:11.672 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.659+0000 7f0e3bdfd740 1 --2- 172.21.15.80:0/276370997 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55fda28d8240 0x55fda28dcd40 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f0e2402f540 tx=0x7f0e2402a8c0 comp rx=0 tx=0).stop 2024-04-24T23:29:11.672 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.659+0000 7f0e3bdfd740 1 -- 172.21.15.80:0/276370997 shutdown_connections 2024-04-24T23:29:11.672 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.659+0000 7f0e3bdfd740 1 --2- 172.21.15.80:0/276370997 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fda28dd3b0 0x55fda28df840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:11.672 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.659+0000 7f0e3bdfd740 1 --2- 172.21.15.80:0/276370997 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55fda28d8240 0x55fda28dcd40 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:11.672 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.659+0000 7f0e3bdfd740 1 --2- 172.21.15.80:0/276370997 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fda28d7850 0x55fda28d7c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:11.672 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.659+0000 7f0e3bdfd740 1 -- 172.21.15.80:0/276370997 >> 172.21.15.80:0/276370997 conn(0x55fda28ceab0 msgr2=0x55fda28ceeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:11.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.659+0000 7f0e3bdfd740 1 -- 172.21.15.80:0/276370997 shutdown_connections 2024-04-24T23:29:11.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.659+0000 7f0e3bdfd740 1 -- 172.21.15.80:0/276370997 wait complete. 2024-04-24T23:29:11.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.659+0000 7f0e3bdfd740 1 -- start start 2024-04-24T23:29:11.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.659+0000 7f0e3bdfd740 1 Processor -- start 2024-04-24T23:29:11.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.659+0000 7f0e3bdfd740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55fda28d7850 0x55fda28e1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:11.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.659+0000 7f0e3bdfd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fda28d8240 0x55fda28e1d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:11.674 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.659+0000 7f0e3bdfd740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fda28dd3b0 0x55fda28e8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:11.674 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.659+0000 7f0e3bdfd740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x55fda2836f00 con 0x55fda28d7850 2024-04-24T23:29:11.674 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.659+0000 7f0e3bdfd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55fda27f2cf0 con 0x55fda28d8240 2024-04-24T23:29:11.674 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.659+0000 7f0e3bdfd740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55fda28e0120 con 0x55fda28dd3b0 2024-04-24T23:29:11.674 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.659+0000 7f0e32ffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55fda28d7850 0x55fda28e1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:11.674 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.659+0000 7f0e3bdfd740 10 client.0 _ll_register_callbacks cb 0x55fda27f26e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:29:11.674 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.659+0000 7f0e32ffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55fda28d7850 0x55fda28e1830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:57516/0 (socket says 172.21.15.80:57516) 2024-04-24T23:29:11.674 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.659+0000 7f0e32ffd640 1 -- 172.21.15.80:0/3698690372 learned_addr learned my addr 172.21.15.80:0/3698690372 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:29:11.675 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.660+0000 7f0e337fe640 1 --2- 172.21.15.80:0/3698690372 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fda28dd3b0 0x55fda28e8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:11.675 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.660+0000 7f0e327fc640 1 --2- 172.21.15.80:0/3698690372 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fda28d8240 0x55fda28e1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:11.675 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.660+0000 7f0e32ffd640 1 -- 172.21.15.80:0/3698690372 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fda28dd3b0 msgr2=0x55fda28e8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:11.675 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.660+0000 7f0e32ffd640 1 --2- 172.21.15.80:0/3698690372 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fda28dd3b0 0x55fda28e8db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:11.675 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.660+0000 7f0e32ffd640 1 -- 172.21.15.80:0/3698690372 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fda28d8240 msgr2=0x55fda28e1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:11.675 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.660+0000 7f0e32ffd640 1 --2- 172.21.15.80:0/3698690372 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fda28d8240 0x55fda28e1d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:11.675 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.660+0000 7f0e32ffd640 1 -- 172.21.15.80:0/3698690372 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fda26156b0 con 0x55fda28d7850 2024-04-24T23:29:11.675 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.660+0000 7f0e32ffd640 1 --2- 172.21.15.80:0/3698690372 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55fda28d7850 0x55fda28e1830 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f0e240ee620 tx=0x7f0e2402ff30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a149dd07eb776d4c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:11.675 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.660+0000 7f0e16ff5640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-04-24T23:29:11.675 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.660+0000 7f0e16ff5640 1 -- 172.21.15.80:0/3698690372 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e240455e0 con 0x55fda28d7850 2024-04-24T23:29:11.675 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.660+0000 7f0e16ff5640 1 -- 172.21.15.80:0/3698690372 <== mon.0 v2:172.21.15.80:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f0e240bc460 con 0x55fda28d7850 2024-04-24T23:29:11.675 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.660+0000 7f0e3bdfd740 1 -- 172.21.15.80:0/3698690372 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e24036020 con 0x55fda28d7850 2024-04-24T23:29:11.676 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.660+0000 7f0e16ff5640 1 -- 172.21.15.80:0/3698690372 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f0e240c13c0 con 0x55fda28d7850 2024-04-24T23:29:11.676 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.660+0000 7f0e3bdfd740 10 client.15267 fetch_fsmap learned FSMap version 24 2024-04-24T23:29:11.676 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.660+0000 7f0e3bdfd740 1 -- 172.21.15.80:0/3698690372 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55fda28ebd90 con 0x55fda28d7850 2024-04-24T23:29:11.676 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.660+0000 7f0e16ff5640 1 -- 172.21.15.80:0/3698690372 <== mon.0 v2:172.21.15.80:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e24036020 con 0x55fda28d7850 2024-04-24T23:29:11.676 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.660+0000 7f0e16ff5640 1 -- 172.21.15.80:0/3698690372 <== mon.0 v2:172.21.15.80:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f0e2404e030 con 0x55fda28d7850 2024-04-24T23:29:11.676 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.660+0000 7f0e3bdfd740 10 client.15267 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T23:29:11.676 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.661+0000 7f0e3bdfd740 10 client.15267 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-24T23:29:11.676 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.661+0000 7f0e3bdfd740 20 client.15267 populate_metadata read hostname 'smithi080' 2024-04-24T23:29:11.676 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.661+0000 7f0e3bdfd740 1 --2- 172.21.15.80:0/3698690372 >> [v2:172.21.15.134:6838/1743815021,v1:172.21.15.134:6839/1743815021] conn(0x55fda28ee310 0x55fda28ef780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:11.676 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.661+0000 7f0e3bdfd740 4 client.15267 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:29:11.676 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.661+0000 7f0e3bdfd740 1 -- 172.21.15.80:0/3698690372 --> [v2:172.21.15.134:6838/1743815021,v1:172.21.15.134:6839/1743815021] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fda2844440 con 0x55fda28ee310 2024-04-24T23:29:11.676 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.661+0000 7f0e327fc640 1 --2- 172.21.15.80:0/3698690372 >> [v2:172.21.15.134:6838/1743815021,v1:172.21.15.134:6839/1743815021] conn(0x55fda28ee310 0x55fda28ef780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:11.676 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.661+0000 7f0e327fc640 1 --2- 172.21.15.80:0/3698690372 >> [v2:172.21.15.134:6838/1743815021,v1:172.21.15.134:6839/1743815021] conn(0x55fda28ee310 0x55fda28ef780 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=46bae6fd963bcced server_cookie=4f879e1063177a72 in_seq=0 out_seq=0 2024-04-24T23:29:11.676 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.661+0000 7f0e16ff5640 10 client.15267 ms_handle_connect on v2:172.21.15.134:6838/1743815021 2024-04-24T23:29:11.676 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.662+0000 7f0e16ff5640 1 -- 172.21.15.80:0/3698690372 <== mds.1 v2:172.21.15.134:6838/1743815021 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fda2844440 con 0x55fda28ee310 2024-04-24T23:29:11.677 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.662+0000 7f0e16ff5640 10 client.15267 handle_command_reply: tid=0 2024-04-24T23:29:11.677 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.662+0000 7f0e3bdfd740 1 client.15267 shutdown 2024-04-24T23:29:11.677 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.662+0000 7f0e3bdfd740 1 -- 172.21.15.80:0/3698690372 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55fda28d7850 msgr2=0x55fda28e1830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:11.677 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.662+0000 7f0e3bdfd740 1 --2- 172.21.15.80:0/3698690372 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55fda28d7850 0x55fda28e1830 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f0e240ee620 tx=0x7f0e2402ff30 comp rx=0 tx=0).stop 2024-04-24T23:29:11.677 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.662+0000 7f0e3bdfd740 1 -- 172.21.15.80:0/3698690372 shutdown_connections 2024-04-24T23:29:11.677 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.662+0000 7f0e3bdfd740 1 --2- 172.21.15.80:0/3698690372 >> [v2:172.21.15.134:6838/1743815021,v1:172.21.15.134:6839/1743815021] conn(0x55fda28ee310 0x55fda28ef780 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:11.677 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.662+0000 7f0e3bdfd740 1 --2- 172.21.15.80:0/3698690372 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fda28dd3b0 0x55fda28e8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:11.677 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.662+0000 7f0e3bdfd740 1 --2- 172.21.15.80:0/3698690372 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fda28d8240 0x55fda28e1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:11.677 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.662+0000 7f0e16ff5640 0 client.15267 ms_handle_reset on v2:172.21.15.134:6838/1743815021 2024-04-24T23:29:11.677 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.662+0000 7f0e3bdfd740 1 --2- 172.21.15.80:0/3698690372 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55fda28d7850 0x55fda28e1830 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:11.677 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.662+0000 7f0e3bdfd740 1 -- 172.21.15.80:0/3698690372 >> 172.21.15.80:0/3698690372 conn(0x55fda28ceab0 msgr2=0x55fda28cdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:11.677 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.662+0000 7f0e3bdfd740 1 -- 172.21.15.80:0/3698690372 shutdown_connections 2024-04-24T23:29:11.677 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.662+0000 7f0e3bdfd740 1 -- 172.21.15.80:0/3698690372 wait complete. 2024-04-24T23:29:11.677 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.662+0000 7f0e3bdfd740 20 client.15267 trim_cache size 0 max 16384 2024-04-24T23:29:11.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.680+0000 7f0e3bdfd740 1 Processor -- start 2024-04-24T23:29:11.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.680+0000 7f0e3bdfd740 1 -- start start 2024-04-24T23:29:11.688 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.680+0000 7f0e3bdfd740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55fda28d7850 0x55fda28e1b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:11.688 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.680+0000 7f0e3bdfd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fda28d8240 0x55fda28e8d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:11.688 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.680+0000 7f0e3bdfd740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fda28dd3b0 0x55fda28cd250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:11.688 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.680+0000 7f0e3bdfd740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x55fda28edac0 con 0x55fda28d7850 2024-04-24T23:29:11.688 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.680+0000 7f0e3bdfd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55fda28ee190 con 0x55fda28d8240 2024-04-24T23:29:11.688 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.680+0000 7f0e3bdfd740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55fda28ee860 con 0x55fda28dd3b0 2024-04-24T23:29:11.688 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.681+0000 7f0e32ffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55fda28d7850 0x55fda28e1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:11.688 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.681+0000 7f0e32ffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55fda28d7850 0x55fda28e1b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:57522/0 (socket says 172.21.15.80:57522) 2024-04-24T23:29:11.688 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.681+0000 7f0e32ffd640 1 -- 172.21.15.80:0/407774354 learned_addr learned my addr 172.21.15.80:0/407774354 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:29:11.688 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.681+0000 7f0e327fc640 1 --2- 172.21.15.80:0/407774354 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fda28d8240 0x55fda28e8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:11.689 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.681+0000 7f0e32ffd640 1 -- 172.21.15.80:0/407774354 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fda28dd3b0 msgr2=0x55fda28cd250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:11.689 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.681+0000 7f0e32ffd640 1 --2- 172.21.15.80:0/407774354 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fda28dd3b0 0x55fda28cd250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:11.689 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.681+0000 7f0e32ffd640 1 -- 172.21.15.80:0/407774354 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fda28d8240 msgr2=0x55fda28e8d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:11.689 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.681+0000 7f0e32ffd640 1 --2- 172.21.15.80:0/407774354 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fda28d8240 0x55fda28e8d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:11.689 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.681+0000 7f0e32ffd640 1 -- 172.21.15.80:0/407774354 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fda28c8c10 con 0x55fda28d7850 2024-04-24T23:29:11.689 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.681+0000 7f0e327fc640 1 --2- 172.21.15.80:0/407774354 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fda28d8240 0x55fda28e8d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:29:11.689 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.681+0000 7f0e32ffd640 1 --2- 172.21.15.80:0/407774354 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55fda28d7850 0x55fda28e1b50 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f0e240b99e0 tx=0x7f0e240ee300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb8647736b649606 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:11.689 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.681+0000 7f0e18ff9640 1 -- 172.21.15.80:0/407774354 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e240bd710 con 0x55fda28d7850 2024-04-24T23:29:11.689 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.681+0000 7f0e18ff9640 1 -- 172.21.15.80:0/407774354 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e24036030 con 0x55fda28d7850 2024-04-24T23:29:11.689 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.681+0000 7f0e18ff9640 1 -- 172.21.15.80:0/407774354 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e240b7b80 con 0x55fda28d7850 2024-04-24T23:29:11.689 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.682+0000 7f0e3bdfd740 1 -- 172.21.15.80:0/407774354 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55fda28d7850 msgr2=0x55fda28e1b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:11.689 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.682+0000 7f0e3bdfd740 1 --2- 172.21.15.80:0/407774354 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55fda28d7850 0x55fda28e1b50 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f0e240b99e0 tx=0x7f0e240ee300 comp rx=0 tx=0).stop 2024-04-24T23:29:11.689 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.682+0000 7f0e3bdfd740 1 -- 172.21.15.80:0/407774354 shutdown_connections 2024-04-24T23:29:11.689 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.682+0000 7f0e3bdfd740 1 --2- 172.21.15.80:0/407774354 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fda28dd3b0 0x55fda28cd250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:11.690 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.682+0000 7f0e3bdfd740 1 --2- 172.21.15.80:0/407774354 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fda28d8240 0x55fda28e8d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:11.690 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.682+0000 7f0e3bdfd740 1 --2- 172.21.15.80:0/407774354 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55fda28d7850 0x55fda28e1b50 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:11.690 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.682+0000 7f0e3bdfd740 1 -- 172.21.15.80:0/407774354 >> 172.21.15.80:0/407774354 conn(0x55fda28ceab0 msgr2=0x55fda28e1440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:11.690 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.682+0000 7f0e3bdfd740 1 -- 172.21.15.80:0/407774354 shutdown_connections 2024-04-24T23:29:11.690 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.682+0000 7f0e3bdfd740 1 -- 172.21.15.80:0/407774354 wait complete. 2024-04-24T23:29:11.690 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.682+0000 7f0e3bdfd740 1 -- start start 2024-04-24T23:29:11.690 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.682+0000 7f0e3bdfd740 1 Processor -- start 2024-04-24T23:29:11.690 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.682+0000 7f0e3bdfd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fda28d7850 0x55fda28e8b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:11.690 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.682+0000 7f0e3bdfd740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fda28d8240 0x55fda28e55d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:11.690 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.682+0000 7f0e3bdfd740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55fda28dd3b0 0x55fda28e5b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:11.690 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.682+0000 7f0e3bdfd740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x55fda28ef600 con 0x55fda28dd3b0 2024-04-24T23:29:11.690 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.682+0000 7f0e3bdfd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55fda28eef30 con 0x55fda28d7850 2024-04-24T23:29:11.690 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.682+0000 7f0e3bdfd740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55fda28efcd0 con 0x55fda28d8240 2024-04-24T23:29:11.690 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.682+0000 7f0e3bdfd740 10 client.0 _ll_register_callbacks cb 0x7f0e24005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:29:11.691 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.682+0000 7f0e337fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55fda28dd3b0 0x55fda28e5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:11.691 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.682+0000 7f0e337fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55fda28dd3b0 0x55fda28e5b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:57530/0 (socket says 172.21.15.80:57530) 2024-04-24T23:29:11.691 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.682+0000 7f0e337fe640 1 -- 172.21.15.80:0/1247662911 learned_addr learned my addr 172.21.15.80:0/1247662911 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:29:11.691 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.683+0000 7f0e327fc640 1 --2- 172.21.15.80:0/1247662911 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fda28d8240 0x55fda28e55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:11.691 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.683+0000 7f0e32ffd640 1 --2- 172.21.15.80:0/1247662911 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fda28d7850 0x55fda28e8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:11.691 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.683+0000 7f0e337fe640 1 -- 172.21.15.80:0/1247662911 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fda28d8240 msgr2=0x55fda28e55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:11.691 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.683+0000 7f0e337fe640 1 --2- 172.21.15.80:0/1247662911 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fda28d8240 0x55fda28e55d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:11.691 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.683+0000 7f0e337fe640 1 -- 172.21.15.80:0/1247662911 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fda28d7850 msgr2=0x55fda28e8b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:11.691 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.683+0000 7f0e337fe640 1 --2- 172.21.15.80:0/1247662911 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fda28d7850 0x55fda28e8b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:11.691 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.683+0000 7f0e337fe640 1 -- 172.21.15.80:0/1247662911 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fda29542c0 con 0x55fda28dd3b0 2024-04-24T23:29:11.691 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.683+0000 7f0e327fc640 1 --2- 172.21.15.80:0/1247662911 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fda28d8240 0x55fda28e55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:29:11.691 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.683+0000 7f0e32ffd640 1 --2- 172.21.15.80:0/1247662911 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fda28d7850 0x55fda28e8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:29:11.691 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.683+0000 7f0e337fe640 1 --2- 172.21.15.80:0/1247662911 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55fda28dd3b0 0x55fda28e5b10 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c00fbe0 tx=0x7f0e2c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5f088c90d891c81 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:11.692 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.683+0000 7f0e16ff5640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-04-24T23:29:11.692 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.683+0000 7f0e16ff5640 1 -- 172.21.15.80:0/1247662911 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e2c01fd00 con 0x55fda28dd3b0 2024-04-24T23:29:11.692 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.683+0000 7f0e16ff5640 1 -- 172.21.15.80:0/1247662911 <== mon.0 v2:172.21.15.80:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f0e2c0293f0 con 0x55fda28dd3b0 2024-04-24T23:29:11.692 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.683+0000 7f0e3bdfd740 1 -- 172.21.15.80:0/1247662911 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e24036030 con 0x55fda28dd3b0 2024-04-24T23:29:11.692 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.683+0000 7f0e16ff5640 1 -- 172.21.15.80:0/1247662911 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f0e2c005020 con 0x55fda28dd3b0 2024-04-24T23:29:11.692 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.683+0000 7f0e3bdfd740 10 client.15273 fetch_fsmap learned FSMap version 24 2024-04-24T23:29:11.692 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.683+0000 7f0e3bdfd740 1 -- 172.21.15.80:0/1247662911 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55fda29554d0 con 0x55fda28dd3b0 2024-04-24T23:29:11.692 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.683+0000 7f0e16ff5640 1 -- 172.21.15.80:0/1247662911 <== mon.0 v2:172.21.15.80:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e24036030 con 0x55fda28dd3b0 2024-04-24T23:29:11.692 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.683+0000 7f0e16ff5640 1 -- 172.21.15.80:0/1247662911 <== mon.0 v2:172.21.15.80:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f0e2c0295f0 con 0x55fda28dd3b0 2024-04-24T23:29:11.692 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.683+0000 7f0e3bdfd740 10 client.15273 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T23:29:11.692 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.683+0000 7f0e3bdfd740 10 client.15273 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-24T23:29:11.692 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.683+0000 7f0e3bdfd740 20 client.15273 populate_metadata read hostname 'smithi080' 2024-04-24T23:29:11.692 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.684+0000 7f0e3bdfd740 1 --2- 172.21.15.80:0/1247662911 >> [v2:172.21.15.134:6838/1743815021,v1:172.21.15.134:6839/1743815021] conn(0x55fda295a150 0x55fda2958e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:11.692 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.684+0000 7f0e3bdfd740 4 client.15273 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T23:29:11.693 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.684+0000 7f0e3bdfd740 1 -- 172.21.15.80:0/1247662911 --> [v2:172.21.15.134:6838/1743815021,v1:172.21.15.134:6839/1743815021] -- command(tid 0: {"prefix": "status"}) -- 0x55fda283e790 con 0x55fda295a150 2024-04-24T23:29:11.693 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.684+0000 7f0e32ffd640 1 --2- 172.21.15.80:0/1247662911 >> [v2:172.21.15.134:6838/1743815021,v1:172.21.15.134:6839/1743815021] conn(0x55fda295a150 0x55fda2958e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:11.693 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.684+0000 7f0e32ffd640 1 --2- 172.21.15.80:0/1247662911 >> [v2:172.21.15.134:6838/1743815021,v1:172.21.15.134:6839/1743815021] conn(0x55fda295a150 0x55fda2958e60 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a26484c96103f0dd server_cookie=f37ac17e328375f4 in_seq=0 out_seq=0 2024-04-24T23:29:11.693 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.684+0000 7f0e16ff5640 10 client.15273 ms_handle_connect on v2:172.21.15.134:6838/1743815021 2024-04-24T23:29:11.693 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.684+0000 7f0e16ff5640 1 -- 172.21.15.80:0/1247662911 <== mds.1 v2:172.21.15.134:6838/1743815021 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x55fda283e790 con 0x55fda295a150 2024-04-24T23:29:11.693 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.684+0000 7f0e16ff5640 10 client.15273 handle_command_reply: tid=0 2024-04-24T23:29:11.693 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.684+0000 7f0e3bdfd740 1 client.15273 shutdown 2024-04-24T23:29:11.693 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.684+0000 7f0e3bdfd740 1 -- 172.21.15.80:0/1247662911 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55fda28dd3b0 msgr2=0x55fda28e5b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:11.693 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.684+0000 7f0e3bdfd740 1 --2- 172.21.15.80:0/1247662911 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55fda28dd3b0 0x55fda28e5b10 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c00fbe0 tx=0x7f0e2c0046c0 comp rx=0 tx=0).stop 2024-04-24T23:29:11.693 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.684+0000 7f0e3bdfd740 1 -- 172.21.15.80:0/1247662911 shutdown_connections 2024-04-24T23:29:11.693 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.685+0000 7f0e3bdfd740 1 --2- 172.21.15.80:0/1247662911 >> [v2:172.21.15.134:6838/1743815021,v1:172.21.15.134:6839/1743815021] conn(0x55fda295a150 0x55fda2958e60 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:11.693 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.685+0000 7f0e3bdfd740 1 --2- 172.21.15.80:0/1247662911 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55fda28dd3b0 0x55fda28e5b10 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:11.693 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.685+0000 7f0e3bdfd740 1 --2- 172.21.15.80:0/1247662911 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fda28d8240 0x55fda28e55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:11.693 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.685+0000 7f0e16ff5640 0 client.15273 ms_handle_reset on v2:172.21.15.134:6838/1743815021 2024-04-24T23:29:11.694 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.685+0000 7f0e3bdfd740 1 --2- 172.21.15.80:0/1247662911 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fda28d7850 0x55fda28e8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:11.694 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.685+0000 7f0e3bdfd740 1 -- 172.21.15.80:0/1247662911 >> 172.21.15.80:0/1247662911 conn(0x55fda28ceab0 msgr2=0x55fda28e1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:11.694 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.685+0000 7f0e3bdfd740 1 -- 172.21.15.80:0/1247662911 shutdown_connections 2024-04-24T23:29:11.694 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.685+0000 7f0e3bdfd740 1 -- 172.21.15.80:0/1247662911 wait complete. 2024-04-24T23:29:11.694 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.685+0000 7f0e3bdfd740 20 client.15273 trim_cache size 0 max 16384 2024-04-24T23:29:11.694 INFO:teuthology.orchestra.run.smithi080.stdout:{ 2024-04-24T23:29:11.694 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_fsid": "ec7885aa-0290-11ef-bc93-c7b262605968", 2024-04-24T23:29:11.694 INFO:teuthology.orchestra.run.smithi080.stdout: "whoami": 1, 2024-04-24T23:29:11.694 INFO:teuthology.orchestra.run.smithi080.stdout: "id": 24488, 2024-04-24T23:29:11.695 INFO:teuthology.orchestra.run.smithi080.stdout: "want_state": "up:active", 2024-04-24T23:29:11.695 INFO:teuthology.orchestra.run.smithi080.stdout: "state": "up:active", 2024-04-24T23:29:11.695 INFO:teuthology.orchestra.run.smithi080.stdout: "fs_name": "cephfs", 2024-04-24T23:29:11.695 INFO:teuthology.orchestra.run.smithi080.stdout: "rank_uptime": 87.721616234999999, 2024-04-24T23:29:11.695 INFO:teuthology.orchestra.run.smithi080.stdout: "mdsmap_epoch": 24, 2024-04-24T23:29:11.695 INFO:teuthology.orchestra.run.smithi080.stdout: "osdmap_epoch": 94, 2024-04-24T23:29:11.695 INFO:teuthology.orchestra.run.smithi080.stdout: "osdmap_epoch_barrier": 90, 2024-04-24T23:29:11.695 INFO:teuthology.orchestra.run.smithi080.stdout: "uptime": 118.48041336 2024-04-24T23:29:11.695 INFO:teuthology.orchestra.run.smithi080.stdout:} 2024-04-24T23:29:11.696 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.685+0000 7f0e3944d640 1 -- 172.21.15.80:0/1021682405 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f0e1c0a2b90 msgr2=0x7f0e1c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:11.696 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.685+0000 7f0e3944d640 1 --2- 172.21.15.80:0/1021682405 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f0e1c0a2b90 0x7f0e1c0a5050 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c00afd0 tx=0x7f0e2c00a9b0 comp rx=0 tx=0).stop 2024-04-24T23:29:11.696 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.685+0000 7f0e3944d640 1 -- 172.21.15.80:0/1021682405 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0e34100680 msgr2=0x7f0e34197a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:11.696 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.685+0000 7f0e3944d640 1 --2- 172.21.15.80:0/1021682405 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0e34100680 0x7f0e34197a90 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f0e24004af0 tx=0x7f0e24031ee0 comp rx=0 tx=0).stop 2024-04-24T23:29:11.696 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.685+0000 7f0e3944d640 1 -- 172.21.15.80:0/1021682405 shutdown_connections 2024-04-24T23:29:11.696 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.685+0000 7f0e3944d640 1 --2- 172.21.15.80:0/1021682405 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f0e1c0a2b90 0x7f0e1c0a5050 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:11.696 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.685+0000 7f0e3944d640 1 --2- 172.21.15.80:0/1021682405 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0e34102230 0x7f0e3419f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:11.696 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.685+0000 7f0e3944d640 1 --2- 172.21.15.80:0/1021682405 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e34101870 0x7f0e34197fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:11.697 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.685+0000 7f0e3944d640 1 --2- 172.21.15.80:0/1021682405 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0e34100680 0x7f0e34197a90 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:11.697 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.685+0000 7f0e3944d640 1 -- 172.21.15.80:0/1021682405 >> 172.21.15.80:0/1021682405 conn(0x7f0e340fbbf0 msgr2=0x7f0e340fd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:11.697 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.685+0000 7f0e3944d640 1 -- 172.21.15.80:0/1021682405 shutdown_connections 2024-04-24T23:29:11.697 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.685+0000 7f0e3944d640 1 -- 172.21.15.80:0/1021682405 wait complete. 2024-04-24T23:29:11.697 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:29:11 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/3345832306' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T23:29:11.698 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:29:11 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/4077691008' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:29:11.698 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:29:11 smithi134 ceph-mon[32619]: pgmap v356: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 895 B/s wr, 8 op/s; 14 B/s, 0 objects/s recovering 2024-04-24T23:29:11.698 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:29:11 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/1597170235' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:29:11.698 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-24T23:29:11.800 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.797+0000 7f86e2e38640 1 -- 172.21.15.80:0/1245051817 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f86dc101880 msgr2=0x7f86dc101d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:11.800 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.797+0000 7f86e2e38640 1 --2- 172.21.15.80:0/1245051817 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f86dc101880 0x7f86dc101d00 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f86c8009900 tx=0x7f86c802a410 comp rx=0 tx=0).stop 2024-04-24T23:29:11.800 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.797+0000 7f86e2e38640 1 -- 172.21.15.80:0/1245051817 shutdown_connections 2024-04-24T23:29:11.800 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.797+0000 7f86e2e38640 1 --2- 172.21.15.80:0/1245051817 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f86dc102240 0x7f86dc108ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:11.800 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.797+0000 7f86e2e38640 1 --2- 172.21.15.80:0/1245051817 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f86dc101880 0x7f86dc101d00 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:11.800 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.797+0000 7f86e2e38640 1 --2- 172.21.15.80:0/1245051817 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86dc100690 0x7f86dc100ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:11.800 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.797+0000 7f86e2e38640 1 -- 172.21.15.80:0/1245051817 >> 172.21.15.80:0/1245051817 conn(0x7f86dc0fbc00 msgr2=0x7f86dc0fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:11.800 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.797+0000 7f86e2e38640 1 -- 172.21.15.80:0/1245051817 shutdown_connections 2024-04-24T23:29:11.800 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.797+0000 7f86e2e38640 1 -- 172.21.15.80:0/1245051817 wait complete. 2024-04-24T23:29:11.800 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.797+0000 7f86e2e38640 1 Processor -- start 2024-04-24T23:29:11.801 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.797+0000 7f86e2e38640 1 -- start start 2024-04-24T23:29:11.801 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.797+0000 7f86e2e38640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86dc100690 0x7f86dc196a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:11.801 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.797+0000 7f86e2e38640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f86dc101880 0x7f86dc196f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:11.801 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.797+0000 7f86e2e38640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f86dc102240 0x7f86dc19dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:11.801 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.797+0000 7f86e2e38640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f86dc10c310 con 0x7f86dc101880 2024-04-24T23:29:11.801 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.797+0000 7f86e2e38640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f86dc10c190 con 0x7f86dc100690 2024-04-24T23:29:11.801 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.797+0000 7f86e2e38640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f86dc10c490 con 0x7f86dc102240 2024-04-24T23:29:11.801 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.798+0000 7f86dbfff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f86dc101880 0x7f86dc196f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:11.801 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.798+0000 7f86dbfff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f86dc101880 0x7f86dc196f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:57548/0 (socket says 172.21.15.80:57548) 2024-04-24T23:29:11.801 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.798+0000 7f86dbfff640 1 -- 172.21.15.80:0/2195024662 learned_addr learned my addr 172.21.15.80:0/2195024662 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:29:11.801 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.798+0000 7f86e0bd4640 1 --2- 172.21.15.80:0/2195024662 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86dc100690 0x7f86dc196a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:11.801 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.798+0000 7f86e13d5640 1 --2- 172.21.15.80:0/2195024662 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f86dc102240 0x7f86dc19dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:11.801 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.798+0000 7f86dbfff640 1 -- 172.21.15.80:0/2195024662 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f86dc102240 msgr2=0x7f86dc19dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:11.801 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.798+0000 7f86dbfff640 1 --2- 172.21.15.80:0/2195024662 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f86dc102240 0x7f86dc19dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:11.801 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.798+0000 7f86dbfff640 1 -- 172.21.15.80:0/2195024662 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86dc100690 msgr2=0x7f86dc196a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:11.802 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.798+0000 7f86dbfff640 1 --2- 172.21.15.80:0/2195024662 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86dc100690 0x7f86dc196a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:11.802 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.798+0000 7f86dbfff640 1 -- 172.21.15.80:0/2195024662 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86dc069b10 con 0x7f86dc101880 2024-04-24T23:29:11.802 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.798+0000 7f86dbfff640 1 --2- 172.21.15.80:0/2195024662 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f86dc101880 0x7f86dc196f50 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f86c8005940 tx=0x7f86c8004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:11.802 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.798+0000 7f86d9ffb640 1 -- 172.21.15.80:0/2195024662 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c803f030 con 0x7f86dc101880 2024-04-24T23:29:11.802 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.798+0000 7f86e2e38640 1 -- 172.21.15.80:0/2195024662 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86dc069da0 con 0x7f86dc101880 2024-04-24T23:29:11.802 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.798+0000 7f86d9ffb640 1 -- 172.21.15.80:0/2195024662 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86c8003ce0 con 0x7f86dc101880 2024-04-24T23:29:11.802 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.799+0000 7f86e2e38640 1 -- 172.21.15.80:0/2195024662 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86dc06a260 con 0x7f86dc101880 2024-04-24T23:29:11.802 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.799+0000 7f86d9ffb640 1 -- 172.21.15.80:0/2195024662 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c8049060 con 0x7f86dc101880 2024-04-24T23:29:11.809 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.801+0000 7f86d9ffb640 1 -- 172.21.15.80:0/2195024662 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f86c8042030 con 0x7f86dc101880 2024-04-24T23:29:11.809 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.801+0000 7f86e5c26740 1 Processor -- start 2024-04-24T23:29:11.809 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.801+0000 7f86e5c26740 1 -- start start 2024-04-24T23:29:11.809 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.802+0000 7f86e5c26740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1fd728440 0x55d1fd728860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:11.809 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.802+0000 7f86e5c26740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d1fd728e30 0x55d1fd72d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:11.810 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.802+0000 7f86e5c26740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d1fd72dfa0 0x55d1fd730430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:11.810 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.802+0000 7f86e5c26740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x55d1fd68bf00 con 0x55d1fd728e30 2024-04-24T23:29:11.810 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.802+0000 7f86e5c26740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d1fd634750 con 0x55d1fd728440 2024-04-24T23:29:11.810 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.802+0000 7f86e5c26740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55d1fd687ec0 con 0x55d1fd72dfa0 2024-04-24T23:29:11.810 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.802+0000 7f86e0bd4640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d1fd728e30 0x55d1fd72d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:11.810 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.802+0000 7f86e0bd4640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d1fd728e30 0x55d1fd72d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:57560/0 (socket says 172.21.15.80:57560) 2024-04-24T23:29:11.810 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.802+0000 7f86e0bd4640 1 -- 172.21.15.80:0/1413078543 learned_addr learned my addr 172.21.15.80:0/1413078543 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:29:11.810 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.802+0000 7f86e13d5640 1 --2- 172.21.15.80:0/1413078543 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1fd728440 0x55d1fd728860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:11.810 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.802+0000 7f86dbfff640 1 --2- 172.21.15.80:0/1413078543 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d1fd72dfa0 0x55d1fd730430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:11.810 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.802+0000 7f86d9ffb640 1 --2- 172.21.15.80:0/2195024662 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f86cc0a2b90 0x7f86cc0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:11.810 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.802+0000 7f86e0bd4640 1 -- 172.21.15.80:0/1413078543 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d1fd72dfa0 msgr2=0x55d1fd730430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:11.810 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.802+0000 7f86e0bd4640 1 --2- 172.21.15.80:0/1413078543 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d1fd72dfa0 0x55d1fd730430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:11.810 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.802+0000 7f86e0bd4640 1 -- 172.21.15.80:0/1413078543 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1fd728440 msgr2=0x55d1fd728860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:11.810 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.802+0000 7f86e0bd4640 1 --2- 172.21.15.80:0/1413078543 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1fd728440 0x55d1fd728860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:11.811 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.802+0000 7f86e0bd4640 1 -- 172.21.15.80:0/1413078543 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1fd3a02d0 con 0x55d1fd728e30 2024-04-24T23:29:11.811 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.802+0000 7f86e13d5640 1 --2- 172.21.15.80:0/2195024662 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f86cc0a2b90 0x7f86cc0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:11.811 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.802+0000 7f86d9ffb640 1 -- 172.21.15.80:0/2195024662 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f86c80efd50 con 0x7f86dc101880 2024-04-24T23:29:11.811 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.802+0000 7f86e0bd4640 1 --2- 172.21.15.80:0/1413078543 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d1fd728e30 0x55d1fd72d930 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f86d000bcb0 tx=0x7f86d00107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10a3ea0d3026a17a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:11.811 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.802+0000 7f86c6ffd640 1 -- 172.21.15.80:0/1413078543 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d0004020 con 0x55d1fd728e30 2024-04-24T23:29:11.811 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.802+0000 7f86c6ffd640 1 -- 172.21.15.80:0/1413078543 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86d0005500 con 0x55d1fd728e30 2024-04-24T23:29:11.811 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.802+0000 7f86c6ffd640 1 -- 172.21.15.80:0/1413078543 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d0004d80 con 0x55d1fd728e30 2024-04-24T23:29:11.811 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.802+0000 7f86e13d5640 1 --2- 172.21.15.80:0/2195024662 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f86cc0a2b90 0x7f86cc0a5050 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x55d1fd729690 tx=0x7f86d4004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:11.811 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.803+0000 7f86e5c26740 1 -- 172.21.15.80:0/1413078543 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d1fd728e30 msgr2=0x55d1fd72d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:11.811 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.803+0000 7f86e5c26740 1 --2- 172.21.15.80:0/1413078543 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d1fd728e30 0x55d1fd72d930 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f86d000bcb0 tx=0x7f86d00107a0 comp rx=0 tx=0).stop 2024-04-24T23:29:11.811 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.803+0000 7f86e5c26740 1 -- 172.21.15.80:0/1413078543 shutdown_connections 2024-04-24T23:29:11.811 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.803+0000 7f86e5c26740 1 --2- 172.21.15.80:0/1413078543 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d1fd72dfa0 0x55d1fd730430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:11.811 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.803+0000 7f86e5c26740 1 --2- 172.21.15.80:0/1413078543 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d1fd728e30 0x55d1fd72d930 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:11.811 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.803+0000 7f86e5c26740 1 --2- 172.21.15.80:0/1413078543 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1fd728440 0x55d1fd728860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:11.811 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.803+0000 7f86e5c26740 1 -- 172.21.15.80:0/1413078543 >> 172.21.15.80:0/1413078543 conn(0x55d1fd723ab0 msgr2=0x55d1fd723eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:11.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.803+0000 7f86e5c26740 1 -- 172.21.15.80:0/1413078543 shutdown_connections 2024-04-24T23:29:11.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.803+0000 7f86e5c26740 1 -- 172.21.15.80:0/1413078543 wait complete. 2024-04-24T23:29:11.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.803+0000 7f86e5c26740 1 -- start start 2024-04-24T23:29:11.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.804+0000 7f86e5c26740 1 Processor -- start 2024-04-24T23:29:11.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.804+0000 7f86e5c26740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d1fd728440 0x55d1fd732870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:11.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.804+0000 7f86e5c26740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1fd728e30 0x55d1fd732db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:11.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.804+0000 7f86e5c26740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d1fd72dfa0 0x55d1fd739df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:11.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.804+0000 7f86e5c26740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x55d1fd730f50 con 0x55d1fd72dfa0 2024-04-24T23:29:11.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.804+0000 7f86e5c26740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d1fd647cf0 con 0x55d1fd728e30 2024-04-24T23:29:11.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.804+0000 7f86e5c26740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55d1fd7310d0 con 0x55d1fd728440 2024-04-24T23:29:11.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.804+0000 7f86e5c26740 10 client.0 _ll_register_callbacks cb 0x55d1fd6476e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:29:11.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.804+0000 7f86e13d5640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d1fd72dfa0 0x55d1fd739df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:11.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.804+0000 7f86e13d5640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d1fd72dfa0 0x55d1fd739df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:57570/0 (socket says 172.21.15.80:57570) 2024-04-24T23:29:11.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.804+0000 7f86e13d5640 1 -- 172.21.15.80:0/1730051560 learned_addr learned my addr 172.21.15.80:0/1730051560 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:29:11.813 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.804+0000 7f86dbfff640 1 --2- 172.21.15.80:0/1730051560 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1fd728e30 0x55d1fd732db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:11.813 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.804+0000 7f86e0bd4640 1 --2- 172.21.15.80:0/1730051560 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d1fd728440 0x55d1fd732870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:11.813 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.804+0000 7f86e13d5640 1 -- 172.21.15.80:0/1730051560 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d1fd728440 msgr2=0x55d1fd732870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:11.813 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.804+0000 7f86e13d5640 1 --2- 172.21.15.80:0/1730051560 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d1fd728440 0x55d1fd732870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:11.813 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.804+0000 7f86e13d5640 1 -- 172.21.15.80:0/1730051560 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1fd728e30 msgr2=0x55d1fd732db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:11.813 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.804+0000 7f86e13d5640 1 --2- 172.21.15.80:0/1730051560 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1fd728e30 0x55d1fd732db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:11.813 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.804+0000 7f86e13d5640 1 -- 172.21.15.80:0/1730051560 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1fd46a6b0 con 0x55d1fd72dfa0 2024-04-24T23:29:11.813 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.804+0000 7f86dbfff640 1 --2- 172.21.15.80:0/1730051560 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1fd728e30 0x55d1fd732db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:29:11.813 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.804+0000 7f86e0bd4640 1 --2- 172.21.15.80:0/1730051560 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d1fd728440 0x55d1fd732870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:29:11.813 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.804+0000 7f86e13d5640 1 --2- 172.21.15.80:0/1730051560 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d1fd72dfa0 0x55d1fd739df0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f86d4010a00 tx=0x7f86d4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51ee80e6bae8d26a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:11.813 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.804+0000 7f86bbfff640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-04-24T23:29:11.813 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.805+0000 7f86bbfff640 1 -- 172.21.15.80:0/1730051560 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d402b060 con 0x55d1fd72dfa0 2024-04-24T23:29:11.813 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.805+0000 7f86bbfff640 1 -- 172.21.15.80:0/1730051560 <== mon.0 v2:172.21.15.80:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f86d4018070 con 0x55d1fd72dfa0 2024-04-24T23:29:11.813 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.805+0000 7f86e5c26740 1 -- 172.21.15.80:0/1730051560 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86d0005500 con 0x55d1fd72dfa0 2024-04-24T23:29:11.813 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.805+0000 7f86bbfff640 1 -- 172.21.15.80:0/1730051560 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f86d4032050 con 0x55d1fd72dfa0 2024-04-24T23:29:11.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.805+0000 7f86e5c26740 10 client.15285 fetch_fsmap learned FSMap version 24 2024-04-24T23:29:11.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.805+0000 7f86e5c26740 1 -- 172.21.15.80:0/1730051560 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55d1fd73ce40 con 0x55d1fd72dfa0 2024-04-24T23:29:11.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.805+0000 7f86bbfff640 1 -- 172.21.15.80:0/1730051560 <== mon.0 v2:172.21.15.80:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86d0005500 con 0x55d1fd72dfa0 2024-04-24T23:29:11.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.805+0000 7f86bbfff640 1 -- 172.21.15.80:0/1730051560 <== mon.0 v2:172.21.15.80:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f86d400f040 con 0x55d1fd72dfa0 2024-04-24T23:29:11.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.805+0000 7f86e5c26740 10 client.15285 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T23:29:11.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.805+0000 7f86e5c26740 10 client.15285 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-24T23:29:11.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.805+0000 7f86e5c26740 20 client.15285 populate_metadata read hostname 'smithi080' 2024-04-24T23:29:11.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.805+0000 7f86e5c26740 1 --2- 172.21.15.80:0/1730051560 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x55d1fd73f530 0x55d1fd7409a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:11.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.805+0000 7f86e5c26740 4 client.15285 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:29:11.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.805+0000 7f86e5c26740 1 -- 172.21.15.80:0/1730051560 --> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d1fd699440 con 0x55d1fd73f530 2024-04-24T23:29:11.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.805+0000 7f86e0bd4640 1 --2- 172.21.15.80:0/1730051560 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x55d1fd73f530 0x55d1fd7409a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:11.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.806+0000 7f86bbfff640 10 client.15285 ms_handle_connect on v2:172.21.15.148:6838/2952757594 2024-04-24T23:29:11.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.806+0000 7f86e0bd4640 1 --2- 172.21.15.80:0/1730051560 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x55d1fd73f530 0x55d1fd7409a0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=686bc7c9d6659019 server_cookie=e912ff831841a614 in_seq=0 out_seq=0 2024-04-24T23:29:11.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.807+0000 7f86bbfff640 1 -- 172.21.15.80:0/1730051560 <== mds.0 v2:172.21.15.148:6838/2952757594 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d1fd699440 con 0x55d1fd73f530 2024-04-24T23:29:11.815 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.807+0000 7f86bbfff640 10 client.15285 handle_command_reply: tid=0 2024-04-24T23:29:11.815 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.807+0000 7f86e5c26740 1 client.15285 shutdown 2024-04-24T23:29:11.815 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.807+0000 7f86e5c26740 1 -- 172.21.15.80:0/1730051560 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d1fd72dfa0 msgr2=0x55d1fd739df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:11.815 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.807+0000 7f86e5c26740 1 --2- 172.21.15.80:0/1730051560 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d1fd72dfa0 0x55d1fd739df0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f86d4010a00 tx=0x7f86d4004ab0 comp rx=0 tx=0).stop 2024-04-24T23:29:11.815 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.807+0000 7f86e5c26740 1 -- 172.21.15.80:0/1730051560 shutdown_connections 2024-04-24T23:29:11.815 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.807+0000 7f86e5c26740 1 --2- 172.21.15.80:0/1730051560 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x55d1fd73f530 0x55d1fd7409a0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:11.815 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.807+0000 7f86e5c26740 1 --2- 172.21.15.80:0/1730051560 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d1fd72dfa0 0x55d1fd739df0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:11.815 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.807+0000 7f86e5c26740 1 --2- 172.21.15.80:0/1730051560 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1fd728e30 0x55d1fd732db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:11.815 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.807+0000 7f86bbfff640 0 client.15285 ms_handle_reset on v2:172.21.15.148:6838/2952757594 2024-04-24T23:29:11.815 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.807+0000 7f86e5c26740 1 --2- 172.21.15.80:0/1730051560 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d1fd728440 0x55d1fd732870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:11.815 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.807+0000 7f86e5c26740 1 -- 172.21.15.80:0/1730051560 >> 172.21.15.80:0/1730051560 conn(0x55d1fd723ab0 msgr2=0x55d1fd722ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:11.815 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.807+0000 7f86e5c26740 1 -- 172.21.15.80:0/1730051560 shutdown_connections 2024-04-24T23:29:11.815 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.807+0000 7f86e5c26740 1 -- 172.21.15.80:0/1730051560 wait complete. 2024-04-24T23:29:11.815 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.807+0000 7f86e5c26740 20 client.15285 trim_cache size 0 max 16384 2024-04-24T23:29:11.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.825+0000 7f86e5c26740 1 Processor -- start 2024-04-24T23:29:11.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.825+0000 7f86e5c26740 1 -- start start 2024-04-24T23:29:11.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.825+0000 7f86e5c26740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d1fd728440 0x55d1fd723040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:11.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.825+0000 7f86e5c26740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1fd728e30 0x55d1fd739db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:11.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.825+0000 7f86e5c26740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d1fd72dfa0 0x55d1fd71e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:11.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.825+0000 7f86e5c26740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x55d1fd73ece0 con 0x55d1fd728440 2024-04-24T23:29:11.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.825+0000 7f86e5c26740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d1fd73f3b0 con 0x55d1fd728e30 2024-04-24T23:29:11.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.825+0000 7f86e5c26740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55d1fd73fa80 con 0x55d1fd72dfa0 2024-04-24T23:29:11.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.825+0000 7f86e0bd4640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d1fd728440 0x55d1fd723040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:11.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.825+0000 7f86e0bd4640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d1fd728440 0x55d1fd723040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:57582/0 (socket says 172.21.15.80:57582) 2024-04-24T23:29:11.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.825+0000 7f86e0bd4640 1 -- 172.21.15.80:0/1925453629 learned_addr learned my addr 172.21.15.80:0/1925453629 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:29:11.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.825+0000 7f86dbfff640 1 --2- 172.21.15.80:0/1925453629 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1fd728e30 0x55d1fd739db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:11.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.825+0000 7f86e0bd4640 1 -- 172.21.15.80:0/1925453629 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d1fd72dfa0 msgr2=0x55d1fd71e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:11.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.825+0000 7f86e0bd4640 1 --2- 172.21.15.80:0/1925453629 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d1fd72dfa0 0x55d1fd71e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:11.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.825+0000 7f86e0bd4640 1 -- 172.21.15.80:0/1925453629 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1fd728e30 msgr2=0x55d1fd739db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:11.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.825+0000 7f86e0bd4640 1 --2- 172.21.15.80:0/1925453629 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1fd728e30 0x55d1fd739db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:11.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.825+0000 7f86e0bd4640 1 -- 172.21.15.80:0/1925453629 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1fd7296c0 con 0x55d1fd728440 2024-04-24T23:29:11.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.825+0000 7f86dbfff640 1 --2- 172.21.15.80:0/1925453629 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1fd728e30 0x55d1fd739db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:29:11.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.825+0000 7f86e0bd4640 1 --2- 172.21.15.80:0/1925453629 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d1fd728440 0x55d1fd723040 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f86d0011010 tx=0x7f86d0013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d55ab9c71966922 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:11.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.826+0000 7f86c67fc640 1 -- 172.21.15.80:0/1925453629 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d0028470 con 0x55d1fd728440 2024-04-24T23:29:11.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.826+0000 7f86c67fc640 1 -- 172.21.15.80:0/1925453629 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86d002d070 con 0x55d1fd728440 2024-04-24T23:29:11.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.826+0000 7f86c67fc640 1 -- 172.21.15.80:0/1925453629 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d0003830 con 0x55d1fd728440 2024-04-24T23:29:11.833 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.826+0000 7f86e5c26740 1 -- 172.21.15.80:0/1925453629 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d1fd728440 msgr2=0x55d1fd723040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:11.833 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.826+0000 7f86e5c26740 1 --2- 172.21.15.80:0/1925453629 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d1fd728440 0x55d1fd723040 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f86d0011010 tx=0x7f86d0013040 comp rx=0 tx=0).stop 2024-04-24T23:29:11.833 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.826+0000 7f86e5c26740 1 -- 172.21.15.80:0/1925453629 shutdown_connections 2024-04-24T23:29:11.833 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.826+0000 7f86e5c26740 1 --2- 172.21.15.80:0/1925453629 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d1fd72dfa0 0x55d1fd71e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:11.833 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.826+0000 7f86e5c26740 1 --2- 172.21.15.80:0/1925453629 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1fd728e30 0x55d1fd739db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:11.833 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.826+0000 7f86e5c26740 1 --2- 172.21.15.80:0/1925453629 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d1fd728440 0x55d1fd723040 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:11.833 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.826+0000 7f86e5c26740 1 -- 172.21.15.80:0/1925453629 >> 172.21.15.80:0/1925453629 conn(0x55d1fd723ab0 msgr2=0x55d1fd723e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:11.833 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.826+0000 7f86e5c26740 1 -- 172.21.15.80:0/1925453629 shutdown_connections 2024-04-24T23:29:11.833 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.826+0000 7f86e5c26740 1 -- 172.21.15.80:0/1925453629 wait complete. 2024-04-24T23:29:11.833 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.826+0000 7f86e5c26740 1 -- start start 2024-04-24T23:29:11.833 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.827+0000 7f86e5c26740 1 Processor -- start 2024-04-24T23:29:11.833 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.827+0000 7f86e5c26740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d1fd728440 0x55d1fd737d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:11.833 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.827+0000 7f86e5c26740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1fd728e30 0x55d1fd7a14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:11.833 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.827+0000 7f86e5c26740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d1fd72dfa0 0x55d1fd7a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:11.834 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.827+0000 7f86e5c26740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x55d1fd740820 con 0x55d1fd728440 2024-04-24T23:29:11.834 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.827+0000 7f86e5c26740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d1fd740150 con 0x55d1fd728e30 2024-04-24T23:29:11.834 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.827+0000 7f86e5c26740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55d1fd740ef0 con 0x55d1fd72dfa0 2024-04-24T23:29:11.834 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.827+0000 7f86e5c26740 10 client.0 _ll_register_callbacks cb 0x7f86d40106b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:29:11.834 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.827+0000 7f86e0bd4640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d1fd728440 0x55d1fd737d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:11.834 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.827+0000 7f86e0bd4640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d1fd728440 0x55d1fd737d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:57584/0 (socket says 172.21.15.80:57584) 2024-04-24T23:29:11.834 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.827+0000 7f86e0bd4640 1 -- 172.21.15.80:0/956328176 learned_addr learned my addr 172.21.15.80:0/956328176 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:29:11.834 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.827+0000 7f86dbfff640 1 --2- 172.21.15.80:0/956328176 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1fd728e30 0x55d1fd7a14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:11.834 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.827+0000 7f86e0bd4640 1 -- 172.21.15.80:0/956328176 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d1fd72dfa0 msgr2=0x55d1fd7a3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:11.834 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.827+0000 7f86e0bd4640 1 --2- 172.21.15.80:0/956328176 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d1fd72dfa0 0x55d1fd7a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:11.834 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.827+0000 7f86e0bd4640 1 -- 172.21.15.80:0/956328176 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1fd728e30 msgr2=0x55d1fd7a14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:11.834 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.827+0000 7f86e0bd4640 1 --2- 172.21.15.80:0/956328176 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1fd728e30 0x55d1fd7a14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:11.834 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.827+0000 7f86e0bd4640 1 -- 172.21.15.80:0/956328176 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1fd7a3fa0 con 0x55d1fd728440 2024-04-24T23:29:11.834 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.827+0000 7f86dbfff640 1 --2- 172.21.15.80:0/956328176 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1fd728e30 0x55d1fd7a14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:29:11.835 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.827+0000 7f86e0bd4640 1 --2- 172.21.15.80:0/956328176 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d1fd728440 0x55d1fd737d00 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f86d0005940 tx=0x7f86d000beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60be12a4ba7d3c87 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:11.835 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.827+0000 7f86bbfff640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-04-24T23:29:11.835 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.828+0000 7f86bbfff640 1 -- 172.21.15.80:0/956328176 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d0035070 con 0x55d1fd728440 2024-04-24T23:29:11.835 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.828+0000 7f86bbfff640 1 -- 172.21.15.80:0/956328176 <== mon.0 v2:172.21.15.80:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f86d00052b0 con 0x55d1fd728440 2024-04-24T23:29:11.835 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.828+0000 7f86e5c26740 1 -- 172.21.15.80:0/956328176 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86d002d070 con 0x55d1fd728440 2024-04-24T23:29:11.835 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.828+0000 7f86bbfff640 1 -- 172.21.15.80:0/956328176 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f86d00054b0 con 0x55d1fd728440 2024-04-24T23:29:11.835 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.828+0000 7f86e5c26740 10 client.15291 fetch_fsmap learned FSMap version 24 2024-04-24T23:29:11.835 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.828+0000 7f86e5c26740 1 -- 172.21.15.80:0/956328176 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55d1fd7a5210 con 0x55d1fd728440 2024-04-24T23:29:11.835 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.828+0000 7f86bbfff640 1 -- 172.21.15.80:0/956328176 <== mon.0 v2:172.21.15.80:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86d002d070 con 0x55d1fd728440 2024-04-24T23:29:11.835 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.828+0000 7f86bbfff640 1 -- 172.21.15.80:0/956328176 <== mon.0 v2:172.21.15.80:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f86d0011560 con 0x55d1fd728440 2024-04-24T23:29:11.835 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.828+0000 7f86e5c26740 10 client.15291 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T23:29:11.835 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.828+0000 7f86e5c26740 10 client.15291 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-24T23:29:11.835 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.828+0000 7f86e5c26740 20 client.15291 populate_metadata read hostname 'smithi080' 2024-04-24T23:29:11.835 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.828+0000 7f86e5c26740 1 --2- 172.21.15.80:0/956328176 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x55d1fd7a97f0 0x55d1fd7a8500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:11.835 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.828+0000 7f86e5c26740 4 client.15291 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T23:29:11.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.828+0000 7f86e5c26740 1 -- 172.21.15.80:0/956328176 --> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] -- command(tid 0: {"prefix": "status"}) -- 0x55d1fd693790 con 0x55d1fd7a97f0 2024-04-24T23:29:11.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.828+0000 7f86dbfff640 1 --2- 172.21.15.80:0/956328176 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x55d1fd7a97f0 0x55d1fd7a8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:11.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.829+0000 7f86bbfff640 10 client.15291 ms_handle_connect on v2:172.21.15.148:6838/2952757594 2024-04-24T23:29:11.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.829+0000 7f86dbfff640 1 --2- 172.21.15.80:0/956328176 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x55d1fd7a97f0 0x55d1fd7a8500 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa6439d52a94cb31 server_cookie=979d75f966e52daf in_seq=0 out_seq=0 2024-04-24T23:29:11.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.829+0000 7f86bbfff640 1 -- 172.21.15.80:0/956328176 <== mds.0 v2:172.21.15.148:6838/2952757594 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x55d1fd693790 con 0x55d1fd7a97f0 2024-04-24T23:29:11.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.829+0000 7f86bbfff640 10 client.15291 handle_command_reply: tid=0 2024-04-24T23:29:11.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.829+0000 7f86e5c26740 1 client.15291 shutdown 2024-04-24T23:29:11.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.829+0000 7f86e5c26740 1 -- 172.21.15.80:0/956328176 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d1fd728440 msgr2=0x55d1fd737d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:11.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.829+0000 7f86e5c26740 1 --2- 172.21.15.80:0/956328176 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d1fd728440 0x55d1fd737d00 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f86d0005940 tx=0x7f86d000beb0 comp rx=0 tx=0).stop 2024-04-24T23:29:11.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.829+0000 7f86e5c26740 1 -- 172.21.15.80:0/956328176 shutdown_connections 2024-04-24T23:29:11.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.829+0000 7f86e5c26740 1 --2- 172.21.15.80:0/956328176 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x55d1fd7a97f0 0x55d1fd7a8500 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:11.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.829+0000 7f86e5c26740 1 --2- 172.21.15.80:0/956328176 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d1fd72dfa0 0x55d1fd7a3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:11.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.829+0000 7f86bbfff640 0 client.15291 ms_handle_reset on v2:172.21.15.148:6838/2952757594 2024-04-24T23:29:11.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.829+0000 7f86e5c26740 1 --2- 172.21.15.80:0/956328176 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1fd728e30 0x55d1fd7a14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:11.837 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.829+0000 7f86e5c26740 1 --2- 172.21.15.80:0/956328176 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d1fd728440 0x55d1fd737d00 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:11.837 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.829+0000 7f86e5c26740 1 -- 172.21.15.80:0/956328176 >> 172.21.15.80:0/956328176 conn(0x55d1fd723ab0 msgr2=0x55d1fd71e690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:11.837 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.829+0000 7f86e5c26740 1 -- 172.21.15.80:0/956328176 shutdown_connections 2024-04-24T23:29:11.837 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.829+0000 7f86e5c26740 1 -- 172.21.15.80:0/956328176 wait complete. 2024-04-24T23:29:11.837 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.829+0000 7f86e5c26740 20 client.15291 trim_cache size 0 max 16384 2024-04-24T23:29:11.837 INFO:teuthology.orchestra.run.smithi080.stdout:{ 2024-04-24T23:29:11.837 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_fsid": "ec7885aa-0290-11ef-bc93-c7b262605968", 2024-04-24T23:29:11.837 INFO:teuthology.orchestra.run.smithi080.stdout: "whoami": 0, 2024-04-24T23:29:11.837 INFO:teuthology.orchestra.run.smithi080.stdout: "id": 24496, 2024-04-24T23:29:11.837 INFO:teuthology.orchestra.run.smithi080.stdout: "want_state": "up:active", 2024-04-24T23:29:11.837 INFO:teuthology.orchestra.run.smithi080.stdout: "state": "up:active", 2024-04-24T23:29:11.838 INFO:teuthology.orchestra.run.smithi080.stdout: "fs_name": "cephfs", 2024-04-24T23:29:11.838 INFO:teuthology.orchestra.run.smithi080.stdout: "rank_uptime": 111.642369378, 2024-04-24T23:29:11.838 INFO:teuthology.orchestra.run.smithi080.stdout: "mdsmap_epoch": 24, 2024-04-24T23:29:11.838 INFO:teuthology.orchestra.run.smithi080.stdout: "osdmap_epoch": 94, 2024-04-24T23:29:11.838 INFO:teuthology.orchestra.run.smithi080.stdout: "osdmap_epoch_barrier": 90, 2024-04-24T23:29:11.838 INFO:teuthology.orchestra.run.smithi080.stdout: "uptime": 127.235234377 2024-04-24T23:29:11.838 INFO:teuthology.orchestra.run.smithi080.stdout:} 2024-04-24T23:29:11.839 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.830+0000 7f86e2e38640 1 -- 172.21.15.80:0/2195024662 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f86cc0a2b90 msgr2=0x7f86cc0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:11.839 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.830+0000 7f86e2e38640 1 --2- 172.21.15.80:0/2195024662 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f86cc0a2b90 0x7f86cc0a5050 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x55d1fd729690 tx=0x7f86d4004a40 comp rx=0 tx=0).stop 2024-04-24T23:29:11.839 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.830+0000 7f86e2e38640 1 -- 172.21.15.80:0/2195024662 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f86dc101880 msgr2=0x7f86dc196f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:11.839 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.830+0000 7f86e2e38640 1 --2- 172.21.15.80:0/2195024662 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f86dc101880 0x7f86dc196f50 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f86c8005940 tx=0x7f86c8004360 comp rx=0 tx=0).stop 2024-04-24T23:29:11.839 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.830+0000 7f86e2e38640 1 -- 172.21.15.80:0/2195024662 shutdown_connections 2024-04-24T23:29:11.839 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.830+0000 7f86e2e38640 1 --2- 172.21.15.80:0/2195024662 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f86cc0a2b90 0x7f86cc0a5050 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:11.839 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.830+0000 7f86e2e38640 1 --2- 172.21.15.80:0/2195024662 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f86dc102240 0x7f86dc19dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:11.839 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.830+0000 7f86e2e38640 1 --2- 172.21.15.80:0/2195024662 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f86dc101880 0x7f86dc196f50 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:11.839 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.830+0000 7f86e2e38640 1 --2- 172.21.15.80:0/2195024662 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86dc100690 0x7f86dc196a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:11.839 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.830+0000 7f86e2e38640 1 -- 172.21.15.80:0/2195024662 >> 172.21.15.80:0/2195024662 conn(0x7f86dc0fbc00 msgr2=0x7f86dc104ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:11.839 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.830+0000 7f86e2e38640 1 -- 172.21.15.80:0/2195024662 shutdown_connections 2024-04-24T23:29:11.840 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:11.830+0000 7f86e2e38640 1 -- 172.21.15.80:0/2195024662 wait complete. 2024-04-24T23:29:11.842 INFO:tasks.cephadm:Setup complete, yielding 2024-04-24T23:29:11.842 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-24T23:29:11.853 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi080.front.sepia.ceph.com 2024-04-24T23:29:11.854 DEBUG:teuthology.orchestra.run.smithi080:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ec7885aa-0290-11ef-bc93-c7b262605968 -- bash -c 'ceph orch status' 2024-04-24T23:29:12.093 INFO:teuthology.orchestra.run.smithi080.stderr:Inferring config /var/lib/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:29:12.468 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:29:12 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/574571358' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:29:12.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:29:12 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/574571358' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:29:12.697 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:29:12 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/574571358' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:29:13.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:29:13 smithi080 ceph-mon[29152]: pgmap v357: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 895 B/s wr, 5 op/s; 14 B/s, 0 objects/s recovering 2024-04-24T23:29:13.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:29:13 smithi148 ceph-mon[32528]: pgmap v357: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 895 B/s wr, 5 op/s; 14 B/s, 0 objects/s recovering 2024-04-24T23:29:13.697 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:29:13 smithi134 ceph-mon[32619]: pgmap v357: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 895 B/s wr, 5 op/s; 14 B/s, 0 objects/s recovering 2024-04-24T23:29:14.009 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:14.008+0000 7fa9ac756700 1 -- 172.21.15.80:0/880365158 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa9a4103480 msgr2=0x7fa9a41038e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:14.009 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:14.008+0000 7fa9ac756700 1 --2- 172.21.15.80:0/880365158 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa9a4103480 0x7fa9a41038e0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fa9940099c0 tx=0x7fa994009cd0 comp rx=0 tx=0).stop 2024-04-24T23:29:14.009 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:14.008+0000 7fa9ac756700 1 -- 172.21.15.80:0/880365158 shutdown_connections 2024-04-24T23:29:14.009 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:14.008+0000 7fa9ac756700 1 --2- 172.21.15.80:0/880365158 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa9a4103e20 0x7fa9a410a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:14.009 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:14.008+0000 7fa9ac756700 1 --2- 172.21.15.80:0/880365158 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa9a4102280 0x7fa9a41026a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:14.009 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:14.008+0000 7fa9ac756700 1 --2- 172.21.15.80:0/880365158 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa9a4103480 0x7fa9a41038e0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:14.009 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:14.008+0000 7fa9ac756700 1 -- 172.21.15.80:0/880365158 >> 172.21.15.80:0/880365158 conn(0x7fa9a40fd820 msgr2=0x7fa9a40ffc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:14.009 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:14.008+0000 7fa9ac756700 1 -- 172.21.15.80:0/880365158 shutdown_connections 2024-04-24T23:29:14.009 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:14.008+0000 7fa9ac756700 1 -- 172.21.15.80:0/880365158 wait complete. 2024-04-24T23:29:14.010 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:14.008+0000 7fa9ac756700 1 Processor -- start 2024-04-24T23:29:14.010 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:14.008+0000 7fa9ac756700 1 -- start start 2024-04-24T23:29:14.011 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:14.009+0000 7fa9ac756700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa9a4102280 0x7fa9a41985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:14.012 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:14.009+0000 7fa9ac756700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa9a4103480 0x7fa9a4198b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:14.012 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:14.009+0000 7fa9ac756700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa9a4103e20 0x7fa9a419fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:14.012 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:14.009+0000 7fa9ac756700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fa9a410df60 con 0x7fa9a4103e20 2024-04-24T23:29:14.012 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:14.009+0000 7fa9ac756700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa9a410dde0 con 0x7fa9a4102280 2024-04-24T23:29:14.012 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:14.009+0000 7fa9ac756700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa9a410e0e0 con 0x7fa9a4103480 2024-04-24T23:29:14.012 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:14.009+0000 7fa9aacf3700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa9a4103e20 0x7fa9a419fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:14.012 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:14.009+0000 7fa9aacf3700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa9a4103e20 0x7fa9a419fb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:39768/0 (socket says 172.21.15.80:39768) 2024-04-24T23:29:14.012 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:14.009+0000 7fa9aacf3700 1 -- 172.21.15.80:0/2896226779 learned_addr learned my addr 172.21.15.80:0/2896226779 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:29:14.012 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:14.009+0000 7fa9aa4f2700 1 --2- 172.21.15.80:0/2896226779 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa9a4102280 0x7fa9a41985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:14.012 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:14.009+0000 7fa9a9cf1700 1 --2- 172.21.15.80:0/2896226779 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa9a4103480 0x7fa9a4198b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:14.012 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:14.009+0000 7fa9aacf3700 1 -- 172.21.15.80:0/2896226779 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa9a4103480 msgr2=0x7fa9a4198b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:14.012 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:14.009+0000 7fa9aacf3700 1 --2- 172.21.15.80:0/2896226779 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa9a4103480 0x7fa9a4198b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:14.012 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:14.009+0000 7fa9aacf3700 1 -- 172.21.15.80:0/2896226779 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa9a4102280 msgr2=0x7fa9a41985d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:14.013 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:14.009+0000 7fa9aacf3700 1 --2- 172.21.15.80:0/2896226779 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa9a4102280 0x7fa9a41985d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:14.013 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:14.009+0000 7fa9aacf3700 1 -- 172.21.15.80:0/2896226779 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9a41a0230 con 0x7fa9a4103e20 2024-04-24T23:29:14.013 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:14.009+0000 7fa9aa4f2700 1 --2- 172.21.15.80:0/2896226779 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa9a4102280 0x7fa9a41985d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:29:14.013 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:14.009+0000 7fa9aacf3700 1 --2- 172.21.15.80:0/2896226779 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa9a4103e20 0x7fa9a419fb90 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fa99c00b9a0 tx=0x7fa99c00bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:14.013 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:14.009+0000 7fa99b7fe700 1 -- 172.21.15.80:0/2896226779 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa99c019070 con 0x7fa9a4103e20 2024-04-24T23:29:14.013 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:14.009+0000 7fa99b7fe700 1 -- 172.21.15.80:0/2896226779 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa99c014440 con 0x7fa9a4103e20 2024-04-24T23:29:14.013 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:14.009+0000 7fa99b7fe700 1 -- 172.21.15.80:0/2896226779 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa99c0075d0 con 0x7fa9a4103e20 2024-04-24T23:29:14.013 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:14.010+0000 7fa9ac756700 1 -- 172.21.15.80:0/2896226779 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9a41a0520 con 0x7fa9a4103e20 2024-04-24T23:29:14.013 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:14.010+0000 7fa9ac756700 1 -- 172.21.15.80:0/2896226779 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9a41a0a90 con 0x7fa9a4103e20 2024-04-24T23:29:14.013 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:14.010+0000 7fa9ac756700 1 -- 172.21.15.80:0/2896226779 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9a4068f20 con 0x7fa9a4103e20 2024-04-24T23:29:14.014 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:14.012+0000 7fa99b7fe700 1 -- 172.21.15.80:0/2896226779 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa99c01c070 con 0x7fa9a4103e20 2024-04-24T23:29:14.014 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:14.012+0000 7fa99b7fe700 1 --2- 172.21.15.80:0/2896226779 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fa9900a2ca0 0x7fa9900a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:14.014 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:14.012+0000 7fa9aa4f2700 1 --2- 172.21.15.80:0/2896226779 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fa9900a2ca0 0x7fa9900a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:14.014 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:14.012+0000 7fa99b7fe700 1 -- 172.21.15.80:0/2896226779 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fa99c0cec10 con 0x7fa9a4103e20 2024-04-24T23:29:14.014 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:14.013+0000 7fa9aa4f2700 1 --2- 172.21.15.80:0/2896226779 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fa9900a2ca0 0x7fa9900a5160 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa9a41032e0 tx=0x7fa9a0008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:14.016 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:14.016+0000 7fa99b7fe700 1 -- 172.21.15.80:0/2896226779 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa99c09b830 con 0x7fa9a4103e20 2024-04-24T23:29:14.235 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:14.234+0000 7fa9ac756700 1 -- 172.21.15.80:0/2896226779 --> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7fa9a4062670 con 0x7fa9900a2ca0 2024-04-24T23:29:14.236 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:14.235+0000 7fa99b7fe700 1 -- 172.21.15.80:0/2896226779 <== mgr.14152 v2:172.21.15.80:6800/3830596922 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7fa9a4062670 con 0x7fa9900a2ca0 2024-04-24T23:29:14.236 INFO:teuthology.orchestra.run.smithi080.stdout:Backend: cephadm 2024-04-24T23:29:14.236 INFO:teuthology.orchestra.run.smithi080.stdout:Available: Yes 2024-04-24T23:29:14.236 INFO:teuthology.orchestra.run.smithi080.stdout:Paused: No 2024-04-24T23:29:14.239 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:14.238+0000 7fa9ac756700 1 -- 172.21.15.80:0/2896226779 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fa9900a2ca0 msgr2=0x7fa9900a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:14.239 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:14.238+0000 7fa9ac756700 1 --2- 172.21.15.80:0/2896226779 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fa9900a2ca0 0x7fa9900a5160 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa9a41032e0 tx=0x7fa9a0008040 comp rx=0 tx=0).stop 2024-04-24T23:29:14.239 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:14.238+0000 7fa9ac756700 1 -- 172.21.15.80:0/2896226779 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa9a4103e20 msgr2=0x7fa9a419fb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:14.239 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:14.238+0000 7fa9ac756700 1 --2- 172.21.15.80:0/2896226779 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa9a4103e20 0x7fa9a419fb90 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fa99c00b9a0 tx=0x7fa99c00bd60 comp rx=0 tx=0).stop 2024-04-24T23:29:14.239 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:14.238+0000 7fa9ac756700 1 -- 172.21.15.80:0/2896226779 shutdown_connections 2024-04-24T23:29:14.239 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:14.238+0000 7fa9ac756700 1 --2- 172.21.15.80:0/2896226779 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fa9900a2ca0 0x7fa9900a5160 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:14.240 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:14.238+0000 7fa9ac756700 1 --2- 172.21.15.80:0/2896226779 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa9a4103e20 0x7fa9a419fb90 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:14.240 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:14.238+0000 7fa9ac756700 1 --2- 172.21.15.80:0/2896226779 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa9a4102280 0x7fa9a41985d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:14.240 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:14.238+0000 7fa9ac756700 1 --2- 172.21.15.80:0/2896226779 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa9a4103480 0x7fa9a4198b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:14.240 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:14.238+0000 7fa9ac756700 1 -- 172.21.15.80:0/2896226779 >> 172.21.15.80:0/2896226779 conn(0x7fa9a40fd820 msgr2=0x7fa9a41067f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:14.240 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:14.238+0000 7fa9ac756700 1 -- 172.21.15.80:0/2896226779 shutdown_connections 2024-04-24T23:29:14.240 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:14.238+0000 7fa9ac756700 1 -- 172.21.15.80:0/2896226779 wait complete. 2024-04-24T23:29:15.048 DEBUG:teuthology.orchestra.run.smithi080:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ec7885aa-0290-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ps' 2024-04-24T23:29:15.288 INFO:teuthology.orchestra.run.smithi080.stderr:Inferring config /var/lib/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:29:16.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:29:15 smithi148 ceph-mon[32528]: from='client.15297 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:29:16.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:29:15 smithi148 ceph-mon[32528]: pgmap v358: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 895 B/s wr, 8 op/s; 14 B/s, 0 objects/s recovering 2024-04-24T23:29:16.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:29:15 smithi080 ceph-mon[29152]: from='client.15297 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:29:16.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:29:15 smithi080 ceph-mon[29152]: pgmap v358: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 895 B/s wr, 8 op/s; 14 B/s, 0 objects/s recovering 2024-04-24T23:29:16.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:29:15 smithi134 ceph-mon[32619]: from='client.15297 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:29:16.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:29:15 smithi134 ceph-mon[32619]: pgmap v358: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 895 B/s wr, 8 op/s; 14 B/s, 0 objects/s recovering 2024-04-24T23:29:16.994 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:16.992+0000 7fdb7e1eb700 1 -- 172.21.15.80:0/823077408 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdb78103e30 msgr2=0x7fdb7810a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:16.994 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:16.992+0000 7fdb7e1eb700 1 --2- 172.21.15.80:0/823077408 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdb78103e30 0x7fdb7810a6e0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fdb7000b0d0 tx=0x7fdb7000b3e0 comp rx=0 tx=0).stop 2024-04-24T23:29:16.994 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:16.992+0000 7fdb7e1eb700 1 -- 172.21.15.80:0/823077408 shutdown_connections 2024-04-24T23:29:16.994 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:16.992+0000 7fdb7e1eb700 1 --2- 172.21.15.80:0/823077408 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdb78103e30 0x7fdb7810a6e0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:16.994 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:16.992+0000 7fdb7e1eb700 1 --2- 172.21.15.80:0/823077408 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdb78102290 0x7fdb781026b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:16.994 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:16.992+0000 7fdb7e1eb700 1 --2- 172.21.15.80:0/823077408 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdb78103490 0x7fdb781038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:16.994 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:16.992+0000 7fdb7e1eb700 1 -- 172.21.15.80:0/823077408 >> 172.21.15.80:0/823077408 conn(0x7fdb780fd830 msgr2=0x7fdb780ffc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:16.994 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:16.993+0000 7fdb7e1eb700 1 -- 172.21.15.80:0/823077408 shutdown_connections 2024-04-24T23:29:16.995 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:16.993+0000 7fdb7e1eb700 1 -- 172.21.15.80:0/823077408 wait complete. 2024-04-24T23:29:16.995 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:16.993+0000 7fdb7e1eb700 1 Processor -- start 2024-04-24T23:29:16.996 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:16.993+0000 7fdb7e1eb700 1 -- start start 2024-04-24T23:29:16.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:16.994+0000 7fdb7e1eb700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdb78102290 0x7fdb781983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:16.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:16.994+0000 7fdb7e1eb700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdb78103490 0x7fdb78198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:16.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:16.994+0000 7fdb7e1eb700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdb78103e30 0x7fdb7819f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:16.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:16.994+0000 7fdb7e1eb700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fdb7810de50 con 0x7fdb78103e30 2024-04-24T23:29:16.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:16.994+0000 7fdb7e1eb700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fdb7810dcd0 con 0x7fdb78102290 2024-04-24T23:29:16.998 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:16.994+0000 7fdb7e1eb700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fdb7810dfd0 con 0x7fdb78103490 2024-04-24T23:29:16.998 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:16.994+0000 7fdb77fff700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdb78103e30 0x7fdb7819f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:16.998 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:16.994+0000 7fdb77fff700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdb78103e30 0x7fdb7819f990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:39778/0 (socket says 172.21.15.80:39778) 2024-04-24T23:29:16.998 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:16.994+0000 7fdb77fff700 1 -- 172.21.15.80:0/1614122321 learned_addr learned my addr 172.21.15.80:0/1614122321 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:29:16.999 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:16.994+0000 7fdb76ffd700 1 --2- 172.21.15.80:0/1614122321 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdb78103490 0x7fdb78198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:16.999 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:16.994+0000 7fdb777fe700 1 --2- 172.21.15.80:0/1614122321 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdb78102290 0x7fdb781983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:16.999 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:16.994+0000 7fdb77fff700 1 -- 172.21.15.80:0/1614122321 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdb78103490 msgr2=0x7fdb78198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:16.999 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:16.994+0000 7fdb77fff700 1 --2- 172.21.15.80:0/1614122321 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdb78103490 0x7fdb78198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:16.999 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:16.994+0000 7fdb77fff700 1 -- 172.21.15.80:0/1614122321 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdb78102290 msgr2=0x7fdb781983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:16.999 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:16.994+0000 7fdb77fff700 1 --2- 172.21.15.80:0/1614122321 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdb78102290 0x7fdb781983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:16.999 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:16.994+0000 7fdb77fff700 1 -- 172.21.15.80:0/1614122321 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb781a0030 con 0x7fdb78103e30 2024-04-24T23:29:16.999 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:16.994+0000 7fdb777fe700 1 --2- 172.21.15.80:0/1614122321 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdb78102290 0x7fdb781983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:29:16.999 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:16.994+0000 7fdb77fff700 1 --2- 172.21.15.80:0/1614122321 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdb78103e30 0x7fdb7819f990 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fdb70007d10 tx=0x7fdb70007d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:17.000 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:16.994+0000 7fdb74ff9700 1 -- 172.21.15.80:0/1614122321 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb70021020 con 0x7fdb78103e30 2024-04-24T23:29:17.000 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:16.994+0000 7fdb7e1eb700 1 -- 172.21.15.80:0/1614122321 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb781a02c0 con 0x7fdb78103e30 2024-04-24T23:29:17.000 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:16.994+0000 7fdb74ff9700 1 -- 172.21.15.80:0/1614122321 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb7001d950 con 0x7fdb78103e30 2024-04-24T23:29:17.000 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:16.994+0000 7fdb74ff9700 1 -- 172.21.15.80:0/1614122321 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb70005450 con 0x7fdb78103e30 2024-04-24T23:29:17.000 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:16.995+0000 7fdb7e1eb700 1 -- 172.21.15.80:0/1614122321 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb781a0850 con 0x7fdb78103e30 2024-04-24T23:29:17.000 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:16.996+0000 7fdb7e1eb700 1 -- 172.21.15.80:0/1614122321 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb78068f20 con 0x7fdb78103e30 2024-04-24T23:29:17.001 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:16.997+0000 7fdb74ff9700 1 -- 172.21.15.80:0/1614122321 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fdb70024020 con 0x7fdb78103e30 2024-04-24T23:29:17.001 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:16.997+0000 7fdb74ff9700 1 --2- 172.21.15.80:0/1614122321 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fdb600a2c20 0x7fdb600a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:17.001 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:16.997+0000 7fdb777fe700 1 --2- 172.21.15.80:0/1614122321 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fdb600a2c20 0x7fdb600a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:17.001 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:16.997+0000 7fdb74ff9700 1 -- 172.21.15.80:0/1614122321 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fdb7000f020 con 0x7fdb78103e30 2024-04-24T23:29:17.001 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:16.998+0000 7fdb777fe700 1 --2- 172.21.15.80:0/1614122321 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fdb600a2c20 0x7fdb600a50e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fdb68005fd0 tx=0x7fdb68005f00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:17.003 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:17.002+0000 7fdb74ff9700 1 -- 172.21.15.80:0/1614122321 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fdb7009d2d0 con 0x7fdb78103e30 2024-04-24T23:29:17.211 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:17.208+0000 7fdb7e1eb700 1 -- 172.21.15.80:0/1614122321 --> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7fdb78062670 con 0x7fdb600a2c20 2024-04-24T23:29:17.225 INFO:teuthology.orchestra.run.smithi080.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-24T23:29:17.225 INFO:teuthology.orchestra.run.smithi080.stdout:mds.a smithi080 running (2m) 21s ago 2m 15.2M - 19.0.0-2445-gd2c45d97 f68e37b807ef 0e7f74f1b192 2024-04-24T23:29:17.225 INFO:teuthology.orchestra.run.smithi080.stdout:mds.b smithi134 running (2m) 57s ago 2m 19.6M - 19.0.0-2445-gd2c45d97 f68e37b807ef 0e52ed6fdb94 2024-04-24T23:29:17.225 INFO:teuthology.orchestra.run.smithi080.stdout:mds.c smithi148 running (2m) 57s ago 2m 19.6M - 19.0.0-2445-gd2c45d97 f68e37b807ef 85fdccac975f 2024-04-24T23:29:17.225 INFO:teuthology.orchestra.run.smithi080.stdout:mds.d smithi080 running (2m) 21s ago 2m 14.0M - 19.0.0-2445-gd2c45d97 f68e37b807ef bc0ce64524d4 2024-04-24T23:29:17.225 INFO:teuthology.orchestra.run.smithi080.stdout:mds.e smithi134 running (2m) 57s ago 2m 17.4M - 19.0.0-2445-gd2c45d97 f68e37b807ef 366560a918ca 2024-04-24T23:29:17.225 INFO:teuthology.orchestra.run.smithi080.stdout:mds.f smithi148 running (2m) 57s ago 2m 18.4M - 19.0.0-2445-gd2c45d97 f68e37b807ef 9f0a8e178d1a 2024-04-24T23:29:17.225 INFO:teuthology.orchestra.run.smithi080.stdout:mds.g smithi080 running (2m) 21s ago 2m 13.5M - 19.0.0-2445-gd2c45d97 f68e37b807ef 5472b1bfb0cb 2024-04-24T23:29:17.225 INFO:teuthology.orchestra.run.smithi080.stdout:mds.h smithi134 running (2m) 57s ago 2m 17.3M - 19.0.0-2445-gd2c45d97 f68e37b807ef d8e5842d437b 2024-04-24T23:29:17.225 INFO:teuthology.orchestra.run.smithi080.stdout:mds.i smithi148 running (2m) 57s ago 2m 28.4M - 19.0.0-2445-gd2c45d97 f68e37b807ef b42a579c659d 2024-04-24T23:29:17.226 INFO:teuthology.orchestra.run.smithi080.stdout:mds.j smithi080 running (2m) 21s ago 2m 14.1M - 19.0.0-2445-gd2c45d97 f68e37b807ef 9bb75b8333b6 2024-04-24T23:29:17.226 INFO:teuthology.orchestra.run.smithi080.stdout:mds.k smithi134 running (2m) 57s ago 2m 13.1M - 19.0.0-2445-gd2c45d97 f68e37b807ef def966a79765 2024-04-24T23:29:17.226 INFO:teuthology.orchestra.run.smithi080.stdout:mds.l smithi148 running (2m) 57s ago 2m 16.4M - 19.0.0-2445-gd2c45d97 f68e37b807ef 8b2dcc3e6dc8 2024-04-24T23:29:17.226 INFO:teuthology.orchestra.run.smithi080.stdout:mgr.x smithi080 *:9283,8765 running (9m) 21s ago 9m 515M - 19.0.0-2445-gd2c45d97 f68e37b807ef 3dde3e4c4b21 2024-04-24T23:29:17.226 INFO:teuthology.orchestra.run.smithi080.stdout:mgr.y smithi134 *:8443,8765 running (7m) 57s ago 7m 452M - 19.0.0-2445-gd2c45d97 f68e37b807ef bb25bcb9fea2 2024-04-24T23:29:17.226 INFO:teuthology.orchestra.run.smithi080.stdout:mgr.z smithi148 *:8443,8765 running (8m) 57s ago 8m 453M - 19.0.0-2445-gd2c45d97 f68e37b807ef fd286875e6f7 2024-04-24T23:29:17.226 INFO:teuthology.orchestra.run.smithi080.stdout:mon.a smithi080 running (9m) 21s ago 10m 53.3M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef 789fd293dfe7 2024-04-24T23:29:17.226 INFO:teuthology.orchestra.run.smithi080.stdout:mon.b smithi134 running (8m) 57s ago 8m 39.0M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef a4ce1fe4d7fd 2024-04-24T23:29:17.226 INFO:teuthology.orchestra.run.smithi080.stdout:mon.c smithi148 running (8m) 57s ago 8m 39.4M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef 24ebdbf62b15 2024-04-24T23:29:17.226 INFO:teuthology.orchestra.run.smithi080.stdout:osd.0 smithi080 running (7m) 21s ago 7m 48.3M 4272M 19.0.0-2445-gd2c45d97 f68e37b807ef 1328aa490536 2024-04-24T23:29:17.226 INFO:teuthology.orchestra.run.smithi080.stdout:osd.1 smithi134 running (7m) 57s ago 7m 55.8M 4272M 19.0.0-2445-gd2c45d97 f68e37b807ef d344c11510cc 2024-04-24T23:29:17.226 INFO:teuthology.orchestra.run.smithi080.stdout:osd.2 smithi148 running (6m) 57s ago 6m 57.2M 4272M 19.0.0-2445-gd2c45d97 f68e37b807ef 61e90c42b296 2024-04-24T23:29:17.227 INFO:teuthology.orchestra.run.smithi080.stdout:osd.3 smithi080 running (6m) 21s ago 6m 56.0M 4272M 19.0.0-2445-gd2c45d97 f68e37b807ef 88c0764dff16 2024-04-24T23:29:17.227 INFO:teuthology.orchestra.run.smithi080.stdout:osd.4 smithi134 running (6m) 57s ago 6m 42.4M 4272M 19.0.0-2445-gd2c45d97 f68e37b807ef 3ddf7c1b2c4f 2024-04-24T23:29:17.227 INFO:teuthology.orchestra.run.smithi080.stdout:osd.5 smithi148 running (5m) 57s ago 5m 47.6M 4272M 19.0.0-2445-gd2c45d97 f68e37b807ef c2aa36f25152 2024-04-24T23:29:17.227 INFO:teuthology.orchestra.run.smithi080.stdout:osd.6 smithi080 running (5m) 21s ago 5m 56.6M 4272M 19.0.0-2445-gd2c45d97 f68e37b807ef c6137981ab8f 2024-04-24T23:29:17.227 INFO:teuthology.orchestra.run.smithi080.stdout:osd.7 smithi134 running (4m) 57s ago 4m 47.2M 4272M 19.0.0-2445-gd2c45d97 f68e37b807ef 569b518b7d18 2024-04-24T23:29:17.227 INFO:teuthology.orchestra.run.smithi080.stdout:osd.8 smithi148 running (4m) 57s ago 4m 42.7M 4272M 19.0.0-2445-gd2c45d97 f68e37b807ef 21714e30a426 2024-04-24T23:29:17.227 INFO:teuthology.orchestra.run.smithi080.stdout:osd.9 smithi080 running (3m) 21s ago 3m 54.8M 4272M 19.0.0-2445-gd2c45d97 f68e37b807ef 5d879543b86f 2024-04-24T23:29:17.227 INFO:teuthology.orchestra.run.smithi080.stdout:osd.10 smithi134 running (3m) 57s ago 3m 43.2M 4272M 19.0.0-2445-gd2c45d97 f68e37b807ef 7049d8168b25 2024-04-24T23:29:17.227 INFO:teuthology.orchestra.run.smithi080.stdout:osd.11 smithi148 running (2m) 57s ago 2m 54.3M 4272M 19.0.0-2445-gd2c45d97 f68e37b807ef 8912709582e1 2024-04-24T23:29:17.228 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:17.223+0000 7fdb74ff9700 1 -- 172.21.15.80:0/1614122321 <== mgr.14152 v2:172.21.15.80:6800/3830596922 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4092 (secure 0 0 0) 0x7fdb78062670 con 0x7fdb600a2c20 2024-04-24T23:29:17.228 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:17.225+0000 7fdb7e1eb700 1 -- 172.21.15.80:0/1614122321 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fdb600a2c20 msgr2=0x7fdb600a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:17.228 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:17.225+0000 7fdb7e1eb700 1 --2- 172.21.15.80:0/1614122321 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fdb600a2c20 0x7fdb600a50e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fdb68005fd0 tx=0x7fdb68005f00 comp rx=0 tx=0).stop 2024-04-24T23:29:17.228 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:17.225+0000 7fdb7e1eb700 1 -- 172.21.15.80:0/1614122321 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdb78103e30 msgr2=0x7fdb7819f990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:17.228 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:17.225+0000 7fdb7e1eb700 1 --2- 172.21.15.80:0/1614122321 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdb78103e30 0x7fdb7819f990 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fdb70007d10 tx=0x7fdb70007d40 comp rx=0 tx=0).stop 2024-04-24T23:29:17.228 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:17.225+0000 7fdb7e1eb700 1 -- 172.21.15.80:0/1614122321 shutdown_connections 2024-04-24T23:29:17.228 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:17.225+0000 7fdb7e1eb700 1 --2- 172.21.15.80:0/1614122321 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fdb600a2c20 0x7fdb600a50e0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:17.228 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:17.225+0000 7fdb7e1eb700 1 --2- 172.21.15.80:0/1614122321 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdb78103e30 0x7fdb7819f990 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:17.228 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:17.225+0000 7fdb7e1eb700 1 --2- 172.21.15.80:0/1614122321 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdb78102290 0x7fdb781983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:17.228 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:17.225+0000 7fdb7e1eb700 1 --2- 172.21.15.80:0/1614122321 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdb78103490 0x7fdb78198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:17.228 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:17.225+0000 7fdb7e1eb700 1 -- 172.21.15.80:0/1614122321 >> 172.21.15.80:0/1614122321 conn(0x7fdb780fd830 msgr2=0x7fdb78106790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:17.229 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:17.226+0000 7fdb7e1eb700 1 -- 172.21.15.80:0/1614122321 shutdown_connections 2024-04-24T23:29:17.229 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:17.226+0000 7fdb7e1eb700 1 -- 172.21.15.80:0/1614122321 wait complete. 2024-04-24T23:29:17.955 DEBUG:teuthology.orchestra.run.smithi080:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ec7885aa-0290-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ls' 2024-04-24T23:29:18.198 INFO:teuthology.orchestra.run.smithi080.stderr:Inferring config /var/lib/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:29:18.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:29:17 smithi148 ceph-mon[32528]: pgmap v359: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 738 B/s wr, 7 op/s; 12 B/s, 0 objects/s recovering 2024-04-24T23:29:18.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:29:17 smithi080 ceph-mon[29152]: pgmap v359: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 738 B/s wr, 7 op/s; 12 B/s, 0 objects/s recovering 2024-04-24T23:29:18.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:29:17 smithi134 ceph-mon[32619]: pgmap v359: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 738 B/s wr, 7 op/s; 12 B/s, 0 objects/s recovering 2024-04-24T23:29:19.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:29:18 smithi148 ceph-mon[32528]: from='client.15303 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:29:19.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:29:18 smithi080 ceph-mon[29152]: from='client.15303 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:29:19.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:29:18 smithi134 ceph-mon[32619]: from='client.15303 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:29:20.043 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:20.040+0000 7fd449b0f700 1 -- 172.21.15.80:0/2085393084 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd4441034a0 msgr2=0x7fd444103900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:20.043 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:20.040+0000 7fd449b0f700 1 --2- 172.21.15.80:0/2085393084 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd4441034a0 0x7fd444103900 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fd4340099c0 tx=0x7fd434009cd0 comp rx=0 tx=0).stop 2024-04-24T23:29:20.043 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:20.040+0000 7fd449b0f700 1 -- 172.21.15.80:0/2085393084 shutdown_connections 2024-04-24T23:29:20.043 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:20.040+0000 7fd449b0f700 1 --2- 172.21.15.80:0/2085393084 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd444103e40 0x7fd44410a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:20.043 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:20.040+0000 7fd449b0f700 1 --2- 172.21.15.80:0/2085393084 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4441022a0 0x7fd4441026c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:20.043 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:20.040+0000 7fd449b0f700 1 --2- 172.21.15.80:0/2085393084 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd4441034a0 0x7fd444103900 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:20.043 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:20.040+0000 7fd449b0f700 1 -- 172.21.15.80:0/2085393084 >> 172.21.15.80:0/2085393084 conn(0x7fd4440fd840 msgr2=0x7fd4440ffc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:20.043 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:20.040+0000 7fd449b0f700 1 -- 172.21.15.80:0/2085393084 shutdown_connections 2024-04-24T23:29:20.043 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:20.040+0000 7fd449b0f700 1 -- 172.21.15.80:0/2085393084 wait complete. 2024-04-24T23:29:20.043 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:20.040+0000 7fd449b0f700 1 Processor -- start 2024-04-24T23:29:20.043 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:20.041+0000 7fd449b0f700 1 -- start start 2024-04-24T23:29:20.044 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:20.041+0000 7fd449b0f700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd4441022a0 0x7fd444198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:20.044 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:20.041+0000 7fd449b0f700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4441034a0 0x7fd444198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:20.044 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:20.041+0000 7fd449b0f700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd444103e40 0x7fd44419f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:20.044 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:20.041+0000 7fd449b0f700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fd44410de80 con 0x7fd444103e40 2024-04-24T23:29:20.044 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:20.041+0000 7fd449b0f700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd44410dd00 con 0x7fd4441034a0 2024-04-24T23:29:20.044 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:20.041+0000 7fd449b0f700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd44410e000 con 0x7fd4441022a0 2024-04-24T23:29:20.044 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:20.041+0000 7fd443fff700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd444103e40 0x7fd44419f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:20.044 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:20.041+0000 7fd443fff700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd444103e40 0x7fd44419f9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:39798/0 (socket says 172.21.15.80:39798) 2024-04-24T23:29:20.044 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:20.041+0000 7fd443fff700 1 -- 172.21.15.80:0/4020729398 learned_addr learned my addr 172.21.15.80:0/4020729398 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:29:20.044 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:20.041+0000 7fd4437fe700 1 --2- 172.21.15.80:0/4020729398 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd4441022a0 0x7fd444198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:20.044 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:20.041+0000 7fd442ffd700 1 --2- 172.21.15.80:0/4020729398 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4441034a0 0x7fd444198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:20.044 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:20.041+0000 7fd443fff700 1 -- 172.21.15.80:0/4020729398 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd4441022a0 msgr2=0x7fd444198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:20.044 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:20.041+0000 7fd443fff700 1 --2- 172.21.15.80:0/4020729398 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd4441022a0 0x7fd444198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:20.044 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:20.041+0000 7fd443fff700 1 -- 172.21.15.80:0/4020729398 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4441034a0 msgr2=0x7fd444198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:20.044 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:20.041+0000 7fd443fff700 1 --2- 172.21.15.80:0/4020729398 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4441034a0 0x7fd444198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:20.045 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:20.041+0000 7fd443fff700 1 -- 172.21.15.80:0/4020729398 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4441a0090 con 0x7fd444103e40 2024-04-24T23:29:20.045 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:20.042+0000 7fd443fff700 1 --2- 172.21.15.80:0/4020729398 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd444103e40 0x7fd44419f9f0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fd43c007f10 tx=0x7fd43c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:20.045 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:20.042+0000 7fd440ff9700 1 -- 172.21.15.80:0/4020729398 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd43c004bb0 con 0x7fd444103e40 2024-04-24T23:29:20.045 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:20.042+0000 7fd449b0f700 1 -- 172.21.15.80:0/4020729398 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4441a0380 con 0x7fd444103e40 2024-04-24T23:29:20.045 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:20.042+0000 7fd440ff9700 1 -- 172.21.15.80:0/4020729398 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd43c004d50 con 0x7fd444103e40 2024-04-24T23:29:20.045 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:20.042+0000 7fd440ff9700 1 -- 172.21.15.80:0/4020729398 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd43c005a60 con 0x7fd444103e40 2024-04-24T23:29:20.045 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:20.042+0000 7fd449b0f700 1 -- 172.21.15.80:0/4020729398 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4441a08c0 con 0x7fd444103e40 2024-04-24T23:29:20.046 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:20.043+0000 7fd449b0f700 1 -- 172.21.15.80:0/4020729398 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd444068f20 con 0x7fd444103e40 2024-04-24T23:29:20.046 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:20.044+0000 7fd440ff9700 1 -- 172.21.15.80:0/4020729398 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd43c01a020 con 0x7fd444103e40 2024-04-24T23:29:20.046 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:20.044+0000 7fd440ff9700 1 --2- 172.21.15.80:0/4020729398 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fd42c0a2c20 0x7fd42c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:20.046 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:20.044+0000 7fd4437fe700 1 --2- 172.21.15.80:0/4020729398 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fd42c0a2c20 0x7fd42c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:20.046 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:20.044+0000 7fd440ff9700 1 -- 172.21.15.80:0/4020729398 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fd43c0cf220 con 0x7fd444103e40 2024-04-24T23:29:20.046 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:20.045+0000 7fd4437fe700 1 --2- 172.21.15.80:0/4020729398 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fd42c0a2c20 0x7fd42c0a50e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fd444103300 tx=0x7fd43800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:20.049 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:20.048+0000 7fd440ff9700 1 -- 172.21.15.80:0/4020729398 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fd43c010070 con 0x7fd444103e40 2024-04-24T23:29:20.255 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:20.254+0000 7fd449b0f700 1 -- 172.21.15.80:0/4020729398 --> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7fd444062670 con 0x7fd42c0a2c20 2024-04-24T23:29:20.255 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:29:19 smithi080 ceph-mon[29152]: pgmap v360: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 669 B/s wr, 8 op/s; 10 B/s, 0 objects/s recovering 2024-04-24T23:29:20.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:29:19 smithi148 ceph-mon[32528]: pgmap v360: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 669 B/s wr, 8 op/s; 10 B/s, 0 objects/s recovering 2024-04-24T23:29:20.263 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:20.263+0000 7fd440ff9700 1 -- 172.21.15.80:0/4020729398 <== mgr.14152 v2:172.21.15.80:6800/3830596922 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7fd444062670 con 0x7fd42c0a2c20 2024-04-24T23:29:20.263 INFO:teuthology.orchestra.run.smithi080.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-04-24T23:29:20.263 INFO:teuthology.orchestra.run.smithi080.stdout:mds.all 12/12 60s ago 2m smithi080=a;smithi080=d;smithi080=g;smithi080=j;smithi134=b;smithi134=e;smithi134=h;smithi134=k;smithi148=c;smithi148=f;smithi148=i;smithi148=l;count:12 2024-04-24T23:29:20.263 INFO:teuthology.orchestra.run.smithi080.stdout:mgr 3/3 60s ago 8m smithi080=x;smithi134=y;smithi148=z;count:3 2024-04-24T23:29:20.263 INFO:teuthology.orchestra.run.smithi080.stdout:mon 3/3 60s ago 8m smithi080:172.21.15.80=a;smithi134:172.21.15.134=b;smithi148:172.21.15.148=c;count:3 2024-04-24T23:29:20.263 INFO:teuthology.orchestra.run.smithi080.stdout:osd 12 60s ago - 2024-04-24T23:29:20.265 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:20.265+0000 7fd449b0f700 1 -- 172.21.15.80:0/4020729398 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fd42c0a2c20 msgr2=0x7fd42c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:20.265 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:20.265+0000 7fd449b0f700 1 --2- 172.21.15.80:0/4020729398 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fd42c0a2c20 0x7fd42c0a50e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fd444103300 tx=0x7fd43800b410 comp rx=0 tx=0).stop 2024-04-24T23:29:20.265 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:20.265+0000 7fd449b0f700 1 -- 172.21.15.80:0/4020729398 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd444103e40 msgr2=0x7fd44419f9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:20.265 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:20.265+0000 7fd449b0f700 1 --2- 172.21.15.80:0/4020729398 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd444103e40 0x7fd44419f9f0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fd43c007f10 tx=0x7fd43c00d3b0 comp rx=0 tx=0).stop 2024-04-24T23:29:20.265 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:20.265+0000 7fd449b0f700 1 -- 172.21.15.80:0/4020729398 shutdown_connections 2024-04-24T23:29:20.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:20.265+0000 7fd449b0f700 1 --2- 172.21.15.80:0/4020729398 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fd42c0a2c20 0x7fd42c0a50e0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:20.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:20.265+0000 7fd449b0f700 1 --2- 172.21.15.80:0/4020729398 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd444103e40 0x7fd44419f9f0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:20.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:20.265+0000 7fd449b0f700 1 --2- 172.21.15.80:0/4020729398 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd4441022a0 0x7fd444198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:20.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:20.265+0000 7fd449b0f700 1 --2- 172.21.15.80:0/4020729398 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4441034a0 0x7fd444198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:20.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:20.265+0000 7fd449b0f700 1 -- 172.21.15.80:0/4020729398 >> 172.21.15.80:0/4020729398 conn(0x7fd4440fd840 msgr2=0x7fd4441067a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:20.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:20.265+0000 7fd449b0f700 1 -- 172.21.15.80:0/4020729398 shutdown_connections 2024-04-24T23:29:20.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:20.265+0000 7fd449b0f700 1 -- 172.21.15.80:0/4020729398 wait complete. 2024-04-24T23:29:20.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:29:19 smithi134 ceph-mon[32619]: pgmap v360: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 669 B/s wr, 8 op/s; 10 B/s, 0 objects/s recovering 2024-04-24T23:29:20.961 DEBUG:teuthology.orchestra.run.smithi080:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ec7885aa-0290-11ef-bc93-c7b262605968 -- bash -c 'ceph orch host ls' 2024-04-24T23:29:21.202 INFO:teuthology.orchestra.run.smithi080.stderr:Inferring config /var/lib/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:29:22.249 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:29:21 smithi134 ceph-mon[32619]: from='client.15309 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:29:22.250 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:29:21 smithi134 ceph-mon[32619]: pgmap v361: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 170 B/s wr, 6 op/s; 9 B/s, 0 objects/s recovering 2024-04-24T23:29:22.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:29:21 smithi148 ceph-mon[32528]: from='client.15309 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:29:22.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:29:21 smithi148 ceph-mon[32528]: pgmap v361: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 170 B/s wr, 6 op/s; 9 B/s, 0 objects/s recovering 2024-04-24T23:29:22.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:29:21 smithi080 ceph-mon[29152]: from='client.15309 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:29:22.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:29:21 smithi080 ceph-mon[29152]: pgmap v361: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 170 B/s wr, 6 op/s; 9 B/s, 0 objects/s recovering 2024-04-24T23:29:22.941 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:22.938+0000 7f38f72a1700 1 -- 172.21.15.80:0/1932768206 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f38f01022c0 msgr2=0x7f38f01026e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:22.941 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:22.938+0000 7f38f72a1700 1 --2- 172.21.15.80:0/1932768206 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f38f01022c0 0x7f38f01026e0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f38ec0099c0 tx=0x7f38ec009cd0 comp rx=0 tx=0).stop 2024-04-24T23:29:22.941 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:22.938+0000 7f38f72a1700 1 -- 172.21.15.80:0/1932768206 shutdown_connections 2024-04-24T23:29:22.941 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:22.938+0000 7f38f72a1700 1 --2- 172.21.15.80:0/1932768206 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38f0103e60 0x7f38f010a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:22.941 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:22.938+0000 7f38f72a1700 1 --2- 172.21.15.80:0/1932768206 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f38f01022c0 0x7f38f01026e0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:22.941 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:22.938+0000 7f38f72a1700 1 --2- 172.21.15.80:0/1932768206 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38f01034c0 0x7f38f0103920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:22.941 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:22.938+0000 7f38f72a1700 1 -- 172.21.15.80:0/1932768206 >> 172.21.15.80:0/1932768206 conn(0x7f38f00fd880 msgr2=0x7f38f00ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:22.941 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:22.939+0000 7f38f72a1700 1 -- 172.21.15.80:0/1932768206 shutdown_connections 2024-04-24T23:29:22.941 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:22.939+0000 7f38f72a1700 1 -- 172.21.15.80:0/1932768206 wait complete. 2024-04-24T23:29:22.942 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:22.939+0000 7f38f72a1700 1 Processor -- start 2024-04-24T23:29:22.942 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:22.939+0000 7f38f72a1700 1 -- start start 2024-04-24T23:29:22.942 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:22.939+0000 7f38f72a1700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38f01022c0 0x7f38f0075fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:22.942 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:22.939+0000 7f38f72a1700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38f01034c0 0x7f38f0074620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:22.942 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:22.939+0000 7f38f72a1700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f38f0103e60 0x7f38f0074b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:22.942 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:22.939+0000 7f38f72a1700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f38f010df50 con 0x7f38f0103e60 2024-04-24T23:29:22.942 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:22.939+0000 7f38f72a1700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f38f010ddd0 con 0x7f38f01022c0 2024-04-24T23:29:22.942 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:22.939+0000 7f38f72a1700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f38f010e0d0 con 0x7f38f01034c0 2024-04-24T23:29:22.942 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:22.939+0000 7f38f583e700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f38f0103e60 0x7f38f0074b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:22.942 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:22.939+0000 7f38f583e700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f38f0103e60 0x7f38f0074b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:58638/0 (socket says 172.21.15.80:58638) 2024-04-24T23:29:22.942 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:22.939+0000 7f38f583e700 1 -- 172.21.15.80:0/1050794720 learned_addr learned my addr 172.21.15.80:0/1050794720 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:29:22.942 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:22.940+0000 7f38f503d700 1 --2- 172.21.15.80:0/1050794720 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38f01022c0 0x7f38f0075fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:22.942 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:22.940+0000 7f38f483c700 1 --2- 172.21.15.80:0/1050794720 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38f01034c0 0x7f38f0074620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:22.943 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:22.940+0000 7f38f583e700 1 -- 172.21.15.80:0/1050794720 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38f01034c0 msgr2=0x7f38f0074620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:22.943 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:22.940+0000 7f38f583e700 1 --2- 172.21.15.80:0/1050794720 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38f01034c0 0x7f38f0074620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:22.943 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:22.940+0000 7f38f583e700 1 -- 172.21.15.80:0/1050794720 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38f01022c0 msgr2=0x7f38f0075fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:22.943 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:22.940+0000 7f38f583e700 1 --2- 172.21.15.80:0/1050794720 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38f01022c0 0x7f38f0075fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:22.943 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:22.940+0000 7f38f583e700 1 -- 172.21.15.80:0/1050794720 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38f0075410 con 0x7f38f0103e60 2024-04-24T23:29:22.943 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:22.940+0000 7f38f583e700 1 --2- 172.21.15.80:0/1050794720 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f38f0103e60 0x7f38f0074b60 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f38e8007ee0 tx=0x7f38e800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:22.943 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:22.940+0000 7f38e67fc700 1 -- 172.21.15.80:0/1050794720 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38e800dbd0 con 0x7f38f0103e60 2024-04-24T23:29:22.943 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:22.940+0000 7f38e67fc700 1 -- 172.21.15.80:0/1050794720 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38e800dd70 con 0x7f38f0103e60 2024-04-24T23:29:22.943 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:22.940+0000 7f38f72a1700 1 -- 172.21.15.80:0/1050794720 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38f01a26f0 con 0x7f38f0103e60 2024-04-24T23:29:22.943 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:22.940+0000 7f38e67fc700 1 -- 172.21.15.80:0/1050794720 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38e8005a60 con 0x7f38f0103e60 2024-04-24T23:29:22.943 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:22.940+0000 7f38f72a1700 1 -- 172.21.15.80:0/1050794720 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38f01a2c80 con 0x7f38f0103e60 2024-04-24T23:29:22.943 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:22.941+0000 7f38f72a1700 1 -- 172.21.15.80:0/1050794720 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38f0068f20 con 0x7f38f0103e60 2024-04-24T23:29:22.944 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:22.942+0000 7f38e67fc700 1 -- 172.21.15.80:0/1050794720 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f38e801a020 con 0x7f38f0103e60 2024-04-24T23:29:22.944 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:22.943+0000 7f38e67fc700 1 --2- 172.21.15.80:0/1050794720 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f38dc0a2c20 0x7f38dc0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:22.944 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:22.943+0000 7f38f503d700 1 --2- 172.21.15.80:0/1050794720 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f38dc0a2c20 0x7f38dc0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:22.944 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:22.943+0000 7f38e67fc700 1 -- 172.21.15.80:0/1050794720 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f38e80cf210 con 0x7f38f0103e60 2024-04-24T23:29:22.944 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:22.943+0000 7f38f503d700 1 --2- 172.21.15.80:0/1050794720 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f38dc0a2c20 0x7f38dc0a50e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f38ec00b580 tx=0x7f38ec005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:22.946 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:22.946+0000 7f38e67fc700 1 -- 172.21.15.80:0/1050794720 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f38e8010070 con 0x7f38f0103e60 2024-04-24T23:29:23.154 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:23.154+0000 7f38f72a1700 1 -- 172.21.15.80:0/1050794720 --> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f38f0062670 con 0x7f38dc0a2c20 2024-04-24T23:29:23.155 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:23.155+0000 7f38e67fc700 1 -- 172.21.15.80:0/1050794720 <== mgr.14152 v2:172.21.15.80:6800/3830596922 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f38f0062670 con 0x7f38dc0a2c20 2024-04-24T23:29:23.155 INFO:teuthology.orchestra.run.smithi080.stdout:HOST ADDR LABELS STATUS 2024-04-24T23:29:23.155 INFO:teuthology.orchestra.run.smithi080.stdout:smithi080 172.21.15.80 2024-04-24T23:29:23.155 INFO:teuthology.orchestra.run.smithi080.stdout:smithi134 172.21.15.134 2024-04-24T23:29:23.155 INFO:teuthology.orchestra.run.smithi080.stdout:smithi148 172.21.15.148 2024-04-24T23:29:23.156 INFO:teuthology.orchestra.run.smithi080.stdout:3 hosts in cluster 2024-04-24T23:29:23.157 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:23.157+0000 7f38f72a1700 1 -- 172.21.15.80:0/1050794720 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f38dc0a2c20 msgr2=0x7f38dc0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:23.157 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:23.157+0000 7f38f72a1700 1 --2- 172.21.15.80:0/1050794720 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f38dc0a2c20 0x7f38dc0a50e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f38ec00b580 tx=0x7f38ec005fb0 comp rx=0 tx=0).stop 2024-04-24T23:29:23.157 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:23.157+0000 7f38f72a1700 1 -- 172.21.15.80:0/1050794720 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f38f0103e60 msgr2=0x7f38f0074b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:23.157 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:23.157+0000 7f38f72a1700 1 --2- 172.21.15.80:0/1050794720 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f38f0103e60 0x7f38f0074b60 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f38e8007ee0 tx=0x7f38e800d3b0 comp rx=0 tx=0).stop 2024-04-24T23:29:23.158 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:23.157+0000 7f38f72a1700 1 -- 172.21.15.80:0/1050794720 shutdown_connections 2024-04-24T23:29:23.158 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:23.157+0000 7f38f72a1700 1 --2- 172.21.15.80:0/1050794720 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f38dc0a2c20 0x7f38dc0a50e0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:23.158 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:23.157+0000 7f38f72a1700 1 --2- 172.21.15.80:0/1050794720 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f38f0103e60 0x7f38f0074b60 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:23.158 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:23.157+0000 7f38f72a1700 1 --2- 172.21.15.80:0/1050794720 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38f01022c0 0x7f38f0075fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:23.158 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:23.157+0000 7f38f72a1700 1 --2- 172.21.15.80:0/1050794720 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38f01034c0 0x7f38f0074620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:23.158 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:23.157+0000 7f38f72a1700 1 -- 172.21.15.80:0/1050794720 >> 172.21.15.80:0/1050794720 conn(0x7f38f00fd880 msgr2=0x7f38f01067c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:23.158 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:23.157+0000 7f38f72a1700 1 -- 172.21.15.80:0/1050794720 shutdown_connections 2024-04-24T23:29:23.158 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:23.157+0000 7f38f72a1700 1 -- 172.21.15.80:0/1050794720 wait complete. 2024-04-24T23:29:23.896 DEBUG:teuthology.orchestra.run.smithi080:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ec7885aa-0290-11ef-bc93-c7b262605968 -- bash -c 'ceph orch device ls' 2024-04-24T23:29:24.167 INFO:teuthology.orchestra.run.smithi080.stderr:Inferring config /var/lib/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:29:24.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:29:23 smithi148 ceph-mon[32528]: pgmap v362: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:29:24.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:29:23 smithi148 ceph-mon[32528]: from='client.15315 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:29:24.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:29:23 smithi080 ceph-mon[29152]: pgmap v362: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:29:24.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:29:23 smithi080 ceph-mon[29152]: from='client.15315 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:29:24.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:29:23 smithi134 ceph-mon[32619]: pgmap v362: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:29:24.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:29:23 smithi134 ceph-mon[32619]: from='client.15315 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:29:25.875 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:25.873+0000 7f1364b36700 1 -- 172.21.15.80:0/1600139473 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1360103950 msgr2=0x7f1360103d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:25.875 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:25.873+0000 7f1364b36700 1 --2- 172.21.15.80:0/1600139473 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1360103950 0x7f1360103d70 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f1354009990 tx=0x7f1354009ca0 comp rx=0 tx=0).stop 2024-04-24T23:29:25.875 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:25.873+0000 7f1364b36700 1 -- 172.21.15.80:0/1600139473 shutdown_connections 2024-04-24T23:29:25.876 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:25.873+0000 7f1364b36700 1 --2- 172.21.15.80:0/1600139473 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1360100d70 0x7f136010a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:25.876 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:25.873+0000 7f1364b36700 1 --2- 172.21.15.80:0/1600139473 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1360103950 0x7f1360103d70 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:25.876 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:25.873+0000 7f1364b36700 1 --2- 172.21.15.80:0/1600139473 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13601003d0 0x7f1360100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:25.876 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:25.873+0000 7f1364b36700 1 -- 172.21.15.80:0/1600139473 >> 172.21.15.80:0/1600139473 conn(0x7f13600fbf90 msgr2=0x7f13600fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:25.876 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:25.873+0000 7f1364b36700 1 -- 172.21.15.80:0/1600139473 shutdown_connections 2024-04-24T23:29:25.876 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:25.873+0000 7f1364b36700 1 -- 172.21.15.80:0/1600139473 wait complete. 2024-04-24T23:29:25.876 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:25.874+0000 7f1364b36700 1 Processor -- start 2024-04-24T23:29:25.876 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:25.874+0000 7f1364b36700 1 -- start start 2024-04-24T23:29:25.876 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:25.874+0000 7f1364b36700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13601003d0 0x7f1360198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:25.876 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:25.874+0000 7f1364b36700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1360100d70 0x7f1360198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:25.876 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:25.874+0000 7f1364b36700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1360103950 0x7f136019f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:25.876 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:25.874+0000 7f1364b36700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f136010df30 con 0x7f1360103950 2024-04-24T23:29:25.876 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:25.874+0000 7f1364b36700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f136010ddb0 con 0x7f13601003d0 2024-04-24T23:29:25.876 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:25.874+0000 7f1364b36700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f136010e0b0 con 0x7f1360100d70 2024-04-24T23:29:25.877 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:25.874+0000 7f135ed9d700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1360103950 0x7f136019f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:25.877 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:25.874+0000 7f135ed9d700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1360103950 0x7f136019f9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:58654/0 (socket says 172.21.15.80:58654) 2024-04-24T23:29:25.877 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:25.874+0000 7f135ed9d700 1 -- 172.21.15.80:0/650356520 learned_addr learned my addr 172.21.15.80:0/650356520 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:29:25.877 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:25.875+0000 7f135e59c700 1 --2- 172.21.15.80:0/650356520 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13601003d0 0x7f1360198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:25.877 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:25.875+0000 7f135dd9b700 1 --2- 172.21.15.80:0/650356520 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1360100d70 0x7f1360198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:25.877 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:25.875+0000 7f135ed9d700 1 -- 172.21.15.80:0/650356520 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1360100d70 msgr2=0x7f1360198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:25.877 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:25.875+0000 7f135ed9d700 1 --2- 172.21.15.80:0/650356520 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1360100d70 0x7f1360198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:25.877 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:25.875+0000 7f135ed9d700 1 -- 172.21.15.80:0/650356520 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13601003d0 msgr2=0x7f1360198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:25.877 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:25.875+0000 7f135ed9d700 1 --2- 172.21.15.80:0/650356520 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13601003d0 0x7f1360198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:25.877 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:25.875+0000 7f135ed9d700 1 -- 172.21.15.80:0/650356520 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13601a0080 con 0x7f1360103950 2024-04-24T23:29:25.877 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:25.875+0000 7f135ed9d700 1 --2- 172.21.15.80:0/650356520 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1360103950 0x7f136019f9e0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f1358007da0 tx=0x7f135800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:25.877 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:25.875+0000 7f134f7fe700 1 -- 172.21.15.80:0/650356520 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1358016020 con 0x7f1360103950 2024-04-24T23:29:25.877 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:25.875+0000 7f1364b36700 1 -- 172.21.15.80:0/650356520 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13601a0370 con 0x7f1360103950 2024-04-24T23:29:25.877 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:25.875+0000 7f134f7fe700 1 -- 172.21.15.80:0/650356520 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f135800ad00 con 0x7f1360103950 2024-04-24T23:29:25.877 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:25.875+0000 7f134f7fe700 1 -- 172.21.15.80:0/650356520 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13580115e0 con 0x7f1360103950 2024-04-24T23:29:25.878 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:25.875+0000 7f1364b36700 1 -- 172.21.15.80:0/650356520 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13601a0900 con 0x7f1360103950 2024-04-24T23:29:25.878 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:25.876+0000 7f1364b36700 1 -- 172.21.15.80:0/650356520 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1360068f20 con 0x7f1360103950 2024-04-24T23:29:25.878 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:25.877+0000 7f134f7fe700 1 -- 172.21.15.80:0/650356520 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1358019020 con 0x7f1360103950 2024-04-24T23:29:25.879 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:25.878+0000 7f134f7fe700 1 --2- 172.21.15.80:0/650356520 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f13480a2c20 0x7f13480a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:25.879 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:25.878+0000 7f134f7fe700 1 -- 172.21.15.80:0/650356520 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f13580cf8e0 con 0x7f1360103950 2024-04-24T23:29:25.879 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:25.878+0000 7f135e59c700 1 --2- 172.21.15.80:0/650356520 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f13480a2c20 0x7f13480a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:25.879 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:25.878+0000 7f135e59c700 1 --2- 172.21.15.80:0/650356520 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f13480a2c20 0x7f13480a50e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f135400b5c0 tx=0x7f13540038f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:25.881 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:25.882+0000 7f134f7fe700 1 -- 172.21.15.80:0/650356520 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f1358020020 con 0x7f1360103950 2024-04-24T23:29:26.087 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:26.087+0000 7f1364b36700 1 -- 172.21.15.80:0/650356520 --> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f1360062670 con 0x7f13480a2c20 2024-04-24T23:29:26.088 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:26.089+0000 7f134f7fe700 1 -- 172.21.15.80:0/650356520 <== mgr.14152 v2:172.21.15.80:6800/3830596922 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f1360062670 con 0x7f13480a2c20 2024-04-24T23:29:26.089 INFO:teuthology.orchestra.run.smithi080.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-04-24T23:29:26.089 INFO:teuthology.orchestra.run.smithi080.stdout:smithi080 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT6204014T400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-04-24T23:29:26.089 INFO:teuthology.orchestra.run.smithi080.stdout:smithi134 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT62420002400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-04-24T23:29:26.089 INFO:teuthology.orchestra.run.smithi080.stdout:smithi148 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6233005G400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-04-24T23:29:26.091 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:26.091+0000 7f1364b36700 1 -- 172.21.15.80:0/650356520 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f13480a2c20 msgr2=0x7f13480a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:26.091 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:26.091+0000 7f1364b36700 1 --2- 172.21.15.80:0/650356520 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f13480a2c20 0x7f13480a50e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f135400b5c0 tx=0x7f13540038f0 comp rx=0 tx=0).stop 2024-04-24T23:29:26.091 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:26.091+0000 7f1364b36700 1 -- 172.21.15.80:0/650356520 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1360103950 msgr2=0x7f136019f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:26.091 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:26.091+0000 7f1364b36700 1 --2- 172.21.15.80:0/650356520 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1360103950 0x7f136019f9e0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f1358007da0 tx=0x7f135800a340 comp rx=0 tx=0).stop 2024-04-24T23:29:26.091 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:26.091+0000 7f1364b36700 1 -- 172.21.15.80:0/650356520 shutdown_connections 2024-04-24T23:29:26.091 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:26.091+0000 7f1364b36700 1 --2- 172.21.15.80:0/650356520 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f13480a2c20 0x7f13480a50e0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:26.091 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:26.091+0000 7f1364b36700 1 --2- 172.21.15.80:0/650356520 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1360103950 0x7f136019f9e0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:26.091 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:26.091+0000 7f1364b36700 1 --2- 172.21.15.80:0/650356520 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13601003d0 0x7f1360198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:26.091 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:26.091+0000 7f1364b36700 1 --2- 172.21.15.80:0/650356520 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1360100d70 0x7f1360198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:26.091 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:26.091+0000 7f1364b36700 1 -- 172.21.15.80:0/650356520 >> 172.21.15.80:0/650356520 conn(0x7f13600fbf90 msgr2=0x7f1360106920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:26.091 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:26.091+0000 7f1364b36700 1 -- 172.21.15.80:0/650356520 shutdown_connections 2024-04-24T23:29:26.091 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:26.091+0000 7f1364b36700 1 -- 172.21.15.80:0/650356520 wait complete. 2024-04-24T23:29:26.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:29:25 smithi148 ceph-mon[32528]: pgmap v363: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T23:29:26.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:29:25 smithi080 ceph-mon[29152]: pgmap v363: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T23:29:26.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:29:25 smithi134 ceph-mon[32619]: pgmap v363: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T23:29:26.814 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-24T23:29:26.824 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi080.front.sepia.ceph.com 2024-04-24T23:29:26.824 DEBUG:teuthology.orchestra.run.smithi080:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ec7885aa-0290-11ef-bc93-c7b262605968 -- bash -c 'ceph fs dump' 2024-04-24T23:29:27.038 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:29:26 smithi080 ceph-mon[29152]: from='client.15321 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:29:27.075 INFO:teuthology.orchestra.run.smithi080.stderr:Inferring config /var/lib/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:29:27.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:29:26 smithi148 ceph-mon[32528]: from='client.15321 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:29:27.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:29:26 smithi134 ceph-mon[32619]: from='client.15321 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:29:28.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:29:27 smithi148 ceph-mon[32528]: pgmap v364: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:29:28.262 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:29:27 smithi080 ceph-mon[29152]: pgmap v364: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:29:28.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:29:27 smithi134 ceph-mon[32619]: pgmap v364: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:29:28.835 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:28.833+0000 7fe1eba1c700 1 -- 172.21.15.80:0/2075752552 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe1e4105640 msgr2=0x7fe1e4107a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:28.835 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:28.833+0000 7fe1eba1c700 1 --2- 172.21.15.80:0/2075752552 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe1e4105640 0x7fe1e4107a30 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fe1d4009990 tx=0x7fe1d4009ca0 comp rx=0 tx=0).stop 2024-04-24T23:29:28.835 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:28.833+0000 7fe1eba1c700 1 -- 172.21.15.80:0/2075752552 shutdown_connections 2024-04-24T23:29:28.835 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:28.833+0000 7fe1eba1c700 1 --2- 172.21.15.80:0/2075752552 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1e4107f70 0x7fe1e410a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:28.835 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:28.833+0000 7fe1eba1c700 1 --2- 172.21.15.80:0/2075752552 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe1e4102d10 0x7fe1e4105100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:28.835 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:28.833+0000 7fe1eba1c700 1 --2- 172.21.15.80:0/2075752552 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe1e4105640 0x7fe1e4107a30 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:28.835 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:28.833+0000 7fe1eba1c700 1 -- 172.21.15.80:0/2075752552 >> 172.21.15.80:0/2075752552 conn(0x7fe1e40fc720 msgr2=0x7fe1e40feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:28.835 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:28.833+0000 7fe1eba1c700 1 -- 172.21.15.80:0/2075752552 shutdown_connections 2024-04-24T23:29:28.835 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:28.833+0000 7fe1eba1c700 1 -- 172.21.15.80:0/2075752552 wait complete. 2024-04-24T23:29:28.835 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:28.834+0000 7fe1eba1c700 1 Processor -- start 2024-04-24T23:29:28.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:28.834+0000 7fe1eba1c700 1 -- start start 2024-04-24T23:29:28.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:28.834+0000 7fe1eba1c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe1e4102d10 0x7fe1e4102670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:28.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:28.834+0000 7fe1eba1c700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe1e4105640 0x7fe1e4100cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:28.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:28.834+0000 7fe1eba1c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1e4107f70 0x7fe1e4101200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:28.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:28.834+0000 7fe1eba1c700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fe1e410def0 con 0x7fe1e4105640 2024-04-24T23:29:28.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:28.834+0000 7fe1eba1c700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe1e410dd70 con 0x7fe1e4102d10 2024-04-24T23:29:28.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:28.834+0000 7fe1eba1c700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe1e410e070 con 0x7fe1e4107f70 2024-04-24T23:29:28.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:28.834+0000 7fe1e8fb7700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe1e4105640 0x7fe1e4100cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:28.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:28.834+0000 7fe1e8fb7700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe1e4105640 0x7fe1e4100cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:58680/0 (socket says 172.21.15.80:58680) 2024-04-24T23:29:28.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:28.834+0000 7fe1e8fb7700 1 -- 172.21.15.80:0/3305973598 learned_addr learned my addr 172.21.15.80:0/3305973598 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:29:28.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:28.834+0000 7fe1e9fb9700 1 --2- 172.21.15.80:0/3305973598 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1e4107f70 0x7fe1e4101200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:28.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:28.834+0000 7fe1e97b8700 1 --2- 172.21.15.80:0/3305973598 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe1e4102d10 0x7fe1e4102670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:28.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:28.834+0000 7fe1e8fb7700 1 -- 172.21.15.80:0/3305973598 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1e4107f70 msgr2=0x7fe1e4101200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:28.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:28.834+0000 7fe1e8fb7700 1 --2- 172.21.15.80:0/3305973598 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1e4107f70 0x7fe1e4101200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:28.837 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:28.834+0000 7fe1e8fb7700 1 -- 172.21.15.80:0/3305973598 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe1e4102d10 msgr2=0x7fe1e4102670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:28.837 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:28.834+0000 7fe1e8fb7700 1 --2- 172.21.15.80:0/3305973598 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe1e4102d10 0x7fe1e4102670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:28.837 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:28.834+0000 7fe1e8fb7700 1 -- 172.21.15.80:0/3305973598 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1e4101ab0 con 0x7fe1e4105640 2024-04-24T23:29:28.837 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:28.835+0000 7fe1e8fb7700 1 --2- 172.21.15.80:0/3305973598 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe1e4105640 0x7fe1e4100cc0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fe1d40098e0 tx=0x7fe1d40057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:28.837 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:28.835+0000 7fe1da7fc700 1 -- 172.21.15.80:0/3305973598 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1d4027070 con 0x7fe1e4105640 2024-04-24T23:29:28.837 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:28.835+0000 7fe1da7fc700 1 -- 172.21.15.80:0/3305973598 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe1d4004150 con 0x7fe1e4105640 2024-04-24T23:29:28.837 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:28.835+0000 7fe1eba1c700 1 -- 172.21.15.80:0/3305973598 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1e41a2630 con 0x7fe1e4105640 2024-04-24T23:29:28.837 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:28.835+0000 7fe1da7fc700 1 -- 172.21.15.80:0/3305973598 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1d400f460 con 0x7fe1e4105640 2024-04-24T23:29:28.837 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:28.835+0000 7fe1eba1c700 1 -- 172.21.15.80:0/3305973598 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1e41a2b00 con 0x7fe1e4105640 2024-04-24T23:29:28.838 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:28.836+0000 7fe1eba1c700 1 -- 172.21.15.80:0/3305973598 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1e4068f20 con 0x7fe1e4105640 2024-04-24T23:29:28.838 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:28.837+0000 7fe1da7fc700 1 -- 172.21.15.80:0/3305973598 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe1d402a070 con 0x7fe1e4105640 2024-04-24T23:29:28.838 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:28.837+0000 7fe1da7fc700 1 --2- 172.21.15.80:0/3305973598 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fe1d00a2c20 0x7fe1d00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:28.838 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:28.837+0000 7fe1e97b8700 1 --2- 172.21.15.80:0/3305973598 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fe1d00a2c20 0x7fe1d00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:28.838 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:28.837+0000 7fe1da7fc700 1 -- 172.21.15.80:0/3305973598 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fe1d40d0560 con 0x7fe1e4105640 2024-04-24T23:29:28.838 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:28.838+0000 7fe1e97b8700 1 --2- 172.21.15.80:0/3305973598 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fe1d00a2c20 0x7fe1d00a50e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc009f20 tx=0x7fe1dc0094f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:28.841 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:28.842+0000 7fe1da7fc700 1 -- 172.21.15.80:0/3305973598 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fe1d4013070 con 0x7fe1e4105640 2024-04-24T23:29:29.087 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:29.087+0000 7fe1eba1c700 1 -- 172.21.15.80:0/3305973598 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7fe1e4062e40 con 0x7fe1e4105640 2024-04-24T23:29:29.087 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:29.088+0000 7fe1da7fc700 1 -- 172.21.15.80:0/3305973598 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 24 v24) ==== 76+0+3051 (secure 0 0 0) 0x7fe1d409d180 con 0x7fe1e4105640 2024-04-24T23:29:29.088 INFO:teuthology.orchestra.run.smithi080.stdout:e24 2024-04-24T23:29:29.088 INFO:teuthology.orchestra.run.smithi080.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-24T23:29:29.088 INFO:teuthology.orchestra.run.smithi080.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-24T23:29:29.088 INFO:teuthology.orchestra.run.smithi080.stdout:legacy client fscid: 1 2024-04-24T23:29:29.088 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-24T23:29:29.088 INFO:teuthology.orchestra.run.smithi080.stdout:Filesystem 'cephfs' (1) 2024-04-24T23:29:29.088 INFO:teuthology.orchestra.run.smithi080.stdout:fs_name cephfs 2024-04-24T23:29:29.088 INFO:teuthology.orchestra.run.smithi080.stdout:epoch 24 2024-04-24T23:29:29.088 INFO:teuthology.orchestra.run.smithi080.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-24T23:29:29.088 INFO:teuthology.orchestra.run.smithi080.stdout:created 2024-04-24T23:27:19.300566+0000 2024-04-24T23:29:29.088 INFO:teuthology.orchestra.run.smithi080.stdout:modified 2024-04-24T23:27:45.981839+0000 2024-04-24T23:29:29.089 INFO:teuthology.orchestra.run.smithi080.stdout:tableserver 0 2024-04-24T23:29:29.089 INFO:teuthology.orchestra.run.smithi080.stdout:root 0 2024-04-24T23:29:29.089 INFO:teuthology.orchestra.run.smithi080.stdout:session_timeout 300 2024-04-24T23:29:29.089 INFO:teuthology.orchestra.run.smithi080.stdout:session_autoclose 300 2024-04-24T23:29:29.089 INFO:teuthology.orchestra.run.smithi080.stdout:max_file_size 1099511627776 2024-04-24T23:29:29.089 INFO:teuthology.orchestra.run.smithi080.stdout:max_xattr_size 65536 2024-04-24T23:29:29.089 INFO:teuthology.orchestra.run.smithi080.stdout:required_client_features {} 2024-04-24T23:29:29.089 INFO:teuthology.orchestra.run.smithi080.stdout:last_failure 0 2024-04-24T23:29:29.089 INFO:teuthology.orchestra.run.smithi080.stdout:last_failure_osd_epoch 0 2024-04-24T23:29:29.089 INFO:teuthology.orchestra.run.smithi080.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-24T23:29:29.089 INFO:teuthology.orchestra.run.smithi080.stdout:max_mds 3 2024-04-24T23:29:29.089 INFO:teuthology.orchestra.run.smithi080.stdout:in 0,1,2 2024-04-24T23:29:29.089 INFO:teuthology.orchestra.run.smithi080.stdout:up {0=24496,1=24488,2=24472} 2024-04-24T23:29:29.089 INFO:teuthology.orchestra.run.smithi080.stdout:failed 2024-04-24T23:29:29.090 INFO:teuthology.orchestra.run.smithi080.stdout:damaged 2024-04-24T23:29:29.090 INFO:teuthology.orchestra.run.smithi080.stdout:stopped 2024-04-24T23:29:29.090 INFO:teuthology.orchestra.run.smithi080.stdout:data_pools [3,4] 2024-04-24T23:29:29.090 INFO:teuthology.orchestra.run.smithi080.stdout:metadata_pool 2 2024-04-24T23:29:29.090 INFO:teuthology.orchestra.run.smithi080.stdout:inline_data disabled 2024-04-24T23:29:29.090 INFO:teuthology.orchestra.run.smithi080.stdout:balancer 2024-04-24T23:29:29.090 INFO:teuthology.orchestra.run.smithi080.stdout:bal_rank_mask -1 2024-04-24T23:29:29.090 INFO:teuthology.orchestra.run.smithi080.stdout:standby_count_wanted 0 2024-04-24T23:29:29.090 INFO:teuthology.orchestra.run.smithi080.stdout:qdb_cluster leader: 24496 members: 24472,24488,24496 2024-04-24T23:29:29.090 INFO:teuthology.orchestra.run.smithi080.stdout:[mds.i{0:24496} state up:active seq 5 addr [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T23:29:29.090 INFO:teuthology.orchestra.run.smithi080.stdout:[mds.f{0:24460} state up:standby-replay seq 1 addr [v2:172.21.15.148:6834/3002319310,v1:172.21.15.148:6835/3002319310] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T23:29:29.090 INFO:teuthology.orchestra.run.smithi080.stdout:[mds.b{1:24488} state up:active seq 9 addr [v2:172.21.15.134:6838/1743815021,v1:172.21.15.134:6839/1743815021] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T23:29:29.091 INFO:teuthology.orchestra.run.smithi080.stdout:[mds.h{1:24458} state up:standby-replay seq 1 addr [v2:172.21.15.134:6836/430512704,v1:172.21.15.134:6837/430512704] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T23:29:29.091 INFO:teuthology.orchestra.run.smithi080.stdout:[mds.c{2:24472} state up:active seq 13 addr [v2:172.21.15.148:6836/856528403,v1:172.21.15.148:6837/856528403] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T23:29:29.091 INFO:teuthology.orchestra.run.smithi080.stdout:[mds.e{2:24428} state up:standby-replay seq 1 addr [v2:172.21.15.134:6834/1399801120,v1:172.21.15.134:6835/1399801120] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T23:29:29.091 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-24T23:29:29.091 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-24T23:29:29.091 INFO:teuthology.orchestra.run.smithi080.stdout:Standby daemons: 2024-04-24T23:29:29.091 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-24T23:29:29.091 INFO:teuthology.orchestra.run.smithi080.stdout:[mds.j{-1:14556} state up:standby seq 1 addr [v2:172.21.15.80:6834/891591085,v1:172.21.15.80:6835/891591085] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T23:29:29.091 INFO:teuthology.orchestra.run.smithi080.stdout:[mds.d{-1:14574} state up:standby seq 1 addr [v2:172.21.15.80:6836/259529026,v1:172.21.15.80:6837/259529026] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T23:29:29.091 INFO:teuthology.orchestra.run.smithi080.stdout:[mds.g{-1:14646} state up:standby seq 1 addr [v2:172.21.15.80:6838/4194284854,v1:172.21.15.80:6839/4194284854] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T23:29:29.091 INFO:teuthology.orchestra.run.smithi080.stdout:[mds.a{-1:14658} state up:standby seq 1 addr [v2:172.21.15.80:6840/3145730958,v1:172.21.15.80:6841/3145730958] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T23:29:29.091 INFO:teuthology.orchestra.run.smithi080.stdout:[mds.k{-1:24395} state up:standby seq 1 addr [v2:172.21.15.134:6832/3736784385,v1:172.21.15.134:6833/3736784385] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T23:29:29.091 INFO:teuthology.orchestra.run.smithi080.stdout:[mds.l{-1:24421} state up:standby seq 1 addr [v2:172.21.15.148:6832/124777462,v1:172.21.15.148:6833/124777462] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T23:29:29.092 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:29.090+0000 7fe1eba1c700 1 -- 172.21.15.80:0/3305973598 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fe1d00a2c20 msgr2=0x7fe1d00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:29.092 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:29.090+0000 7fe1eba1c700 1 --2- 172.21.15.80:0/3305973598 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fe1d00a2c20 0x7fe1d00a50e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc009f20 tx=0x7fe1dc0094f0 comp rx=0 tx=0).stop 2024-04-24T23:29:29.092 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:29.090+0000 7fe1eba1c700 1 -- 172.21.15.80:0/3305973598 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe1e4105640 msgr2=0x7fe1e4100cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:29.092 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:29.090+0000 7fe1eba1c700 1 --2- 172.21.15.80:0/3305973598 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe1e4105640 0x7fe1e4100cc0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fe1d40098e0 tx=0x7fe1d40057c0 comp rx=0 tx=0).stop 2024-04-24T23:29:29.092 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:29.090+0000 7fe1eba1c700 1 -- 172.21.15.80:0/3305973598 shutdown_connections 2024-04-24T23:29:29.092 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:29.090+0000 7fe1eba1c700 1 --2- 172.21.15.80:0/3305973598 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1e4107f70 0x7fe1e4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:29.092 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:29.090+0000 7fe1eba1c700 1 --2- 172.21.15.80:0/3305973598 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe1e4102d10 0x7fe1e4102670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:29.092 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:29.090+0000 7fe1eba1c700 1 --2- 172.21.15.80:0/3305973598 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fe1d00a2c20 0x7fe1d00a50e0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:29.093 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:29.090+0000 7fe1eba1c700 1 --2- 172.21.15.80:0/3305973598 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe1e4105640 0x7fe1e4100cc0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:29.093 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:29.090+0000 7fe1eba1c700 1 -- 172.21.15.80:0/3305973598 >> 172.21.15.80:0/3305973598 conn(0x7fe1e40fc720 msgr2=0x7fe1e4103650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:29.093 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:29.090+0000 7fe1eba1c700 1 -- 172.21.15.80:0/3305973598 shutdown_connections 2024-04-24T23:29:29.093 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:29.090+0000 7fe1eba1c700 1 -- 172.21.15.80:0/3305973598 wait complete. 2024-04-24T23:29:29.093 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 24 2024-04-24T23:29:29.775 DEBUG:teuthology.orchestra.run.smithi080:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ec7885aa-0290-11ef-bc93-c7b262605968 -- bash -c 'ceph osd dump' 2024-04-24T23:29:30.016 INFO:teuthology.orchestra.run.smithi080.stderr:Inferring config /var/lib/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:29:30.038 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:29:29 smithi080 ceph-mon[29152]: pgmap v365: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T23:29:30.038 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:29:29 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/3305973598' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T23:29:30.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:29:29 smithi148 ceph-mon[32528]: pgmap v365: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T23:29:30.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:29:29 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/3305973598' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T23:29:30.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:29:29 smithi134 ceph-mon[32619]: pgmap v365: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T23:29:30.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:29:29 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/3305973598' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T23:29:31.758 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:31.755+0000 7fbb2c4df700 1 -- 172.21.15.80:0/3816888064 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbb24105650 msgr2=0x7fbb24107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:31.758 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:31.755+0000 7fbb2c4df700 1 --2- 172.21.15.80:0/3816888064 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbb24105650 0x7fbb24107a40 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fbb14009990 tx=0x7fbb14009ca0 comp rx=0 tx=0).stop 2024-04-24T23:29:31.758 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:31.755+0000 7fbb2c4df700 1 -- 172.21.15.80:0/3816888064 shutdown_connections 2024-04-24T23:29:31.758 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:31.755+0000 7fbb2c4df700 1 --2- 172.21.15.80:0/3816888064 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb24107f80 0x7fbb2410a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:31.758 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:31.755+0000 7fbb2c4df700 1 --2- 172.21.15.80:0/3816888064 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbb24102d20 0x7fbb24105110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:31.758 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:31.755+0000 7fbb2c4df700 1 --2- 172.21.15.80:0/3816888064 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbb24105650 0x7fbb24107a40 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:31.759 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:31.755+0000 7fbb2c4df700 1 -- 172.21.15.80:0/3816888064 >> 172.21.15.80:0/3816888064 conn(0x7fbb240fc730 msgr2=0x7fbb240feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:31.759 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:31.755+0000 7fbb2c4df700 1 -- 172.21.15.80:0/3816888064 shutdown_connections 2024-04-24T23:29:31.759 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:31.755+0000 7fbb2c4df700 1 -- 172.21.15.80:0/3816888064 wait complete. 2024-04-24T23:29:31.759 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:31.755+0000 7fbb2c4df700 1 Processor -- start 2024-04-24T23:29:31.759 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:31.756+0000 7fbb2c4df700 1 -- start start 2024-04-24T23:29:31.759 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:31.756+0000 7fbb2c4df700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbb24102d20 0x7fbb24198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:31.759 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:31.756+0000 7fbb2c4df700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbb24105650 0x7fbb24198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:31.760 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:31.756+0000 7fbb2c4df700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb24107f80 0x7fbb2419faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:31.760 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:31.756+0000 7fbb2c4df700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fbb2410dee0 con 0x7fbb24102d20 2024-04-24T23:29:31.760 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:31.756+0000 7fbb2c4df700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbb2410dd60 con 0x7fbb24105650 2024-04-24T23:29:31.760 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:31.756+0000 7fbb2c4df700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fbb2410e060 con 0x7fbb24107f80 2024-04-24T23:29:31.760 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:31.756+0000 7fbb2a27b700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbb24102d20 0x7fbb24198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:31.760 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:31.756+0000 7fbb2a27b700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbb24102d20 0x7fbb24198580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:58702/0 (socket says 172.21.15.80:58702) 2024-04-24T23:29:31.760 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:31.756+0000 7fbb2a27b700 1 -- 172.21.15.80:0/3239099706 learned_addr learned my addr 172.21.15.80:0/3239099706 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:29:31.760 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:31.756+0000 7fbb2aa7c700 1 --2- 172.21.15.80:0/3239099706 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb24107f80 0x7fbb2419faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:31.760 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:31.756+0000 7fbb29a7a700 1 --2- 172.21.15.80:0/3239099706 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbb24105650 0x7fbb24198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:31.761 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:31.756+0000 7fbb2a27b700 1 -- 172.21.15.80:0/3239099706 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb24107f80 msgr2=0x7fbb2419faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:31.761 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:31.756+0000 7fbb2a27b700 1 --2- 172.21.15.80:0/3239099706 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb24107f80 0x7fbb2419faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:31.761 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:31.756+0000 7fbb2a27b700 1 -- 172.21.15.80:0/3239099706 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbb24105650 msgr2=0x7fbb24198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:31.761 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:31.756+0000 7fbb2a27b700 1 --2- 172.21.15.80:0/3239099706 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbb24105650 0x7fbb24198ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:31.761 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:31.756+0000 7fbb2a27b700 1 -- 172.21.15.80:0/3239099706 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb241a0190 con 0x7fbb24102d20 2024-04-24T23:29:31.761 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:31.756+0000 7fbb2a27b700 1 --2- 172.21.15.80:0/3239099706 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbb24102d20 0x7fbb24198580 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fbb2000d980 tx=0x7fbb2000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:31.761 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:31.757+0000 7fbb1b7fe700 1 -- 172.21.15.80:0/3239099706 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb20016070 con 0x7fbb24102d20 2024-04-24T23:29:31.761 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:31.757+0000 7fbb1b7fe700 1 -- 172.21.15.80:0/3239099706 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb20009db0 con 0x7fbb24102d20 2024-04-24T23:29:31.761 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:31.757+0000 7fbb1b7fe700 1 -- 172.21.15.80:0/3239099706 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb2000f3c0 con 0x7fbb24102d20 2024-04-24T23:29:31.761 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:31.757+0000 7fbb2c4df700 1 -- 172.21.15.80:0/3239099706 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb241a0480 con 0x7fbb24102d20 2024-04-24T23:29:31.762 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:31.757+0000 7fbb2c4df700 1 -- 172.21.15.80:0/3239099706 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb241a0a40 con 0x7fbb24102d20 2024-04-24T23:29:31.763 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:31.758+0000 7fbb2c4df700 1 -- 172.21.15.80:0/3239099706 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb24068f20 con 0x7fbb24102d20 2024-04-24T23:29:31.763 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:31.759+0000 7fbb1b7fe700 1 -- 172.21.15.80:0/3239099706 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fbb20019070 con 0x7fbb24102d20 2024-04-24T23:29:31.763 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:31.759+0000 7fbb1b7fe700 1 --2- 172.21.15.80:0/3239099706 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fbb100a2c20 0x7fbb100a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:31.763 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:31.759+0000 7fbb29a7a700 1 --2- 172.21.15.80:0/3239099706 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fbb100a2c20 0x7fbb100a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:31.763 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:31.759+0000 7fbb1b7fe700 1 -- 172.21.15.80:0/3239099706 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fbb200ceef0 con 0x7fbb24102d20 2024-04-24T23:29:31.763 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:31.760+0000 7fbb29a7a700 1 --2- 172.21.15.80:0/3239099706 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fbb100a2c20 0x7fbb100a50e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fbb1400b5c0 tx=0x7fbb14003960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:31.764 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:31.764+0000 7fbb1b7fe700 1 -- 172.21.15.80:0/3239099706 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fbb2009ba80 con 0x7fbb24102d20 2024-04-24T23:29:31.962 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:31.961+0000 7fbb2c4df700 1 -- 172.21.15.80:0/3239099706 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7fbb24062e40 con 0x7fbb24102d20 2024-04-24T23:29:31.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:31.962+0000 7fbb1b7fe700 1 -- 172.21.15.80:0/3239099706 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v94) ==== 56+0+5702 (secure 0 0 0) 0x7fbb2009b380 con 0x7fbb24102d20 2024-04-24T23:29:31.963 INFO:teuthology.orchestra.run.smithi080.stdout:epoch 94 2024-04-24T23:29:31.963 INFO:teuthology.orchestra.run.smithi080.stdout:fsid ec7885aa-0290-11ef-bc93-c7b262605968 2024-04-24T23:29:31.964 INFO:teuthology.orchestra.run.smithi080.stdout:created 2024-04-24T23:19:05.586407+0000 2024-04-24T23:29:31.964 INFO:teuthology.orchestra.run.smithi080.stdout:modified 2024-04-24T23:29:08.209684+0000 2024-04-24T23:29:31.964 INFO:teuthology.orchestra.run.smithi080.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-04-24T23:29:31.964 INFO:teuthology.orchestra.run.smithi080.stdout:crush_version 27 2024-04-24T23:29:31.964 INFO:teuthology.orchestra.run.smithi080.stdout:full_ratio 0.95 2024-04-24T23:29:31.964 INFO:teuthology.orchestra.run.smithi080.stdout:backfillfull_ratio 0.9 2024-04-24T23:29:31.964 INFO:teuthology.orchestra.run.smithi080.stdout:nearfull_ratio 0.85 2024-04-24T23:29:31.964 INFO:teuthology.orchestra.run.smithi080.stdout:require_min_compat_client luminous 2024-04-24T23:29:31.964 INFO:teuthology.orchestra.run.smithi080.stdout:min_compat_client luminous 2024-04-24T23:29:31.964 INFO:teuthology.orchestra.run.smithi080.stdout:require_osd_release squid 2024-04-24T23:29:31.964 INFO:teuthology.orchestra.run.smithi080.stdout:stretch_mode_enabled false 2024-04-24T23:29:31.964 INFO:teuthology.orchestra.run.smithi080.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-04-24T23:29:31.964 INFO:teuthology.orchestra.run.smithi080.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 84 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-04-24T23:29:31.964 INFO:teuthology.orchestra.run.smithi080.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 85 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-04-24T23:29:31.965 INFO:teuthology.orchestra.run.smithi080.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 90 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-04-24T23:29:31.965 INFO:teuthology.orchestra.run.smithi080.stdout:max_osd 12 2024-04-24T23:29:31.965 INFO:teuthology.orchestra.run.smithi080.stdout:osd.0 up in weight 1 up_from 8 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.80:6802/2597718112,v1:172.21.15.80:6803/2597718112] [v2:172.21.15.80:6804/2597718112,v1:172.21.15.80:6805/2597718112] exists,up a525741d-b46c-4eef-aa64-3f057cac1497 2024-04-24T23:29:31.965 INFO:teuthology.orchestra.run.smithi080.stdout:osd.1 up in weight 1 up_from 13 up_thru 93 down_at 0 last_clean_interval [0,0) [v2:172.21.15.134:6800/2395610201,v1:172.21.15.134:6801/2395610201] [v2:172.21.15.134:6802/2395610201,v1:172.21.15.134:6803/2395610201] exists,up 2d0f0d67-6db4-40ac-91cc-58604ab6795a 2024-04-24T23:29:31.965 INFO:teuthology.orchestra.run.smithi080.stdout:osd.2 up in weight 1 up_from 18 up_thru 93 down_at 0 last_clean_interval [0,0) [v2:172.21.15.148:6800/1629152187,v1:172.21.15.148:6801/1629152187] [v2:172.21.15.148:6802/1629152187,v1:172.21.15.148:6803/1629152187] exists,up 781fb879-fedc-4c84-82fb-59e073bc4cf5 2024-04-24T23:29:31.965 INFO:teuthology.orchestra.run.smithi080.stdout:osd.3 up in weight 1 up_from 25 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.80:6810/3128298510,v1:172.21.15.80:6811/3128298510] [v2:172.21.15.80:6812/3128298510,v1:172.21.15.80:6813/3128298510] exists,up cd982757-f2f6-4296-b3e7-4cd40fbd8d9b 2024-04-24T23:29:31.965 INFO:teuthology.orchestra.run.smithi080.stdout:osd.4 up in weight 1 up_from 33 up_thru 93 down_at 0 last_clean_interval [0,0) [v2:172.21.15.134:6808/1917556060,v1:172.21.15.134:6809/1917556060] [v2:172.21.15.134:6810/1917556060,v1:172.21.15.134:6811/1917556060] exists,up dfc55a79-a7e8-417a-b898-ec44171a54af 2024-04-24T23:29:31.965 INFO:teuthology.orchestra.run.smithi080.stdout:osd.5 up in weight 1 up_from 38 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.148:6808/2715951281,v1:172.21.15.148:6809/2715951281] [v2:172.21.15.148:6810/2715951281,v1:172.21.15.148:6811/2715951281] exists,up b5491337-0694-440c-b679-312f3d14cf65 2024-04-24T23:29:31.965 INFO:teuthology.orchestra.run.smithi080.stdout:osd.6 up in weight 1 up_from 46 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.80:6818/953261724,v1:172.21.15.80:6819/953261724] [v2:172.21.15.80:6820/953261724,v1:172.21.15.80:6821/953261724] exists,up ba4b5df7-d86e-47cd-ab67-cf763c4e4d6c 2024-04-24T23:29:31.965 INFO:teuthology.orchestra.run.smithi080.stdout:osd.7 up in weight 1 up_from 51 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.134:6816/108354844,v1:172.21.15.134:6817/108354844] [v2:172.21.15.134:6818/108354844,v1:172.21.15.134:6819/108354844] exists,up 464a503a-75ae-49e7-ab6e-a7f9f2a6a307 2024-04-24T23:29:31.965 INFO:teuthology.orchestra.run.smithi080.stdout:osd.8 up in weight 1 up_from 59 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.148:6816/3559655329,v1:172.21.15.148:6817/3559655329] [v2:172.21.15.148:6818/3559655329,v1:172.21.15.148:6819/3559655329] exists,up ce544b2e-4540-47c0-9997-500c45351330 2024-04-24T23:29:31.965 INFO:teuthology.orchestra.run.smithi080.stdout:osd.9 up in weight 1 up_from 67 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.80:6826/4019913545,v1:172.21.15.80:6827/4019913545] [v2:172.21.15.80:6828/4019913545,v1:172.21.15.80:6829/4019913545] exists,up 06de3732-19b9-496c-a464-55322efa6759 2024-04-24T23:29:31.965 INFO:teuthology.orchestra.run.smithi080.stdout:osd.10 up in weight 1 up_from 72 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.134:6824/647343766,v1:172.21.15.134:6825/647343766] [v2:172.21.15.134:6826/647343766,v1:172.21.15.134:6827/647343766] exists,up 8d92f0d9-5c64-429a-b750-c33ae0e070c3 2024-04-24T23:29:31.965 INFO:teuthology.orchestra.run.smithi080.stdout:osd.11 up in weight 1 up_from 80 up_thru 93 down_at 0 last_clean_interval [0,0) [v2:172.21.15.148:6824/4068024116,v1:172.21.15.148:6825/4068024116] [v2:172.21.15.148:6826/4068024116,v1:172.21.15.148:6827/4068024116] exists,up 4c003a41-f65b-479f-bc80-b27d680cf936 2024-04-24T23:29:31.966 INFO:teuthology.orchestra.run.smithi080.stdout:pg_upmap_items 2.e [2,4] 2024-04-24T23:29:31.966 INFO:teuthology.orchestra.run.smithi080.stdout:pg_upmap_items 2.10 [2,1] 2024-04-24T23:29:31.966 INFO:teuthology.orchestra.run.smithi080.stdout:pg_upmap_items 2.1d [8,11] 2024-04-24T23:29:31.966 INFO:teuthology.orchestra.run.smithi080.stdout:pg_upmap_items 2.1f [7,11] 2024-04-24T23:29:31.966 INFO:teuthology.orchestra.run.smithi080.stdout:pg_upmap_items 3.e [5,9] 2024-04-24T23:29:31.966 INFO:teuthology.orchestra.run.smithi080.stdout:pg_upmap_items 3.14 [4,6] 2024-04-24T23:29:31.966 INFO:teuthology.orchestra.run.smithi080.stdout:pg_upmap_items 3.1e [3,9] 2024-04-24T23:29:31.966 INFO:teuthology.orchestra.run.smithi080.stdout:pg_upmap_items 3.21 [4,7] 2024-04-24T23:29:31.966 INFO:teuthology.orchestra.run.smithi080.stdout:pg_upmap_items 3.25 [5,9] 2024-04-24T23:29:31.966 INFO:teuthology.orchestra.run.smithi080.stdout:pg_upmap_items 3.31 [4,8] 2024-04-24T23:29:31.966 INFO:teuthology.orchestra.run.smithi080.stdout:pg_upmap_items 4.a [5,8] 2024-04-24T23:29:31.966 INFO:teuthology.orchestra.run.smithi080.stdout:pg_upmap_items 4.c [9,10] 2024-04-24T23:29:31.966 INFO:teuthology.orchestra.run.smithi080.stdout:pg_upmap_items 4.14 [9,8] 2024-04-24T23:29:31.966 INFO:teuthology.orchestra.run.smithi080.stdout:pg_upmap_items 4.1c [9,8] 2024-04-24T23:29:31.966 INFO:teuthology.orchestra.run.smithi080.stdout:blocklist 172.21.15.80:0/2828378749 expires 2024-04-25T23:20:06.793609+0000 2024-04-24T23:29:31.967 INFO:teuthology.orchestra.run.smithi080.stdout:blocklist 172.21.15.80:6801/233364790 expires 2024-04-25T23:20:06.793609+0000 2024-04-24T23:29:31.967 INFO:teuthology.orchestra.run.smithi080.stdout:blocklist 172.21.15.80:6800/1614691410 expires 2024-04-25T23:19:40.490692+0000 2024-04-24T23:29:31.967 INFO:teuthology.orchestra.run.smithi080.stdout:blocklist 172.21.15.80:0/1406187145 expires 2024-04-25T23:20:06.793609+0000 2024-04-24T23:29:31.967 INFO:teuthology.orchestra.run.smithi080.stdout:blocklist 172.21.15.80:0/92953396 expires 2024-04-25T23:19:40.490692+0000 2024-04-24T23:29:31.967 INFO:teuthology.orchestra.run.smithi080.stdout:blocklist 172.21.15.80:0/598129432 expires 2024-04-25T23:20:06.793609+0000 2024-04-24T23:29:31.967 INFO:teuthology.orchestra.run.smithi080.stdout:blocklist 172.21.15.80:0/232193118 expires 2024-04-25T23:19:40.490692+0000 2024-04-24T23:29:31.967 INFO:teuthology.orchestra.run.smithi080.stdout:blocklist 172.21.15.80:6800/233364790 expires 2024-04-25T23:20:06.793609+0000 2024-04-24T23:29:31.967 INFO:teuthology.orchestra.run.smithi080.stdout:blocklist 172.21.15.80:6801/1614691410 expires 2024-04-25T23:19:40.490692+0000 2024-04-24T23:29:31.967 INFO:teuthology.orchestra.run.smithi080.stdout:blocklist 172.21.15.80:0/2081562529 expires 2024-04-25T23:19:40.490692+0000 2024-04-24T23:29:31.968 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:31.964+0000 7fbb2c4df700 1 -- 172.21.15.80:0/3239099706 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fbb100a2c20 msgr2=0x7fbb100a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:31.968 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:31.964+0000 7fbb2c4df700 1 --2- 172.21.15.80:0/3239099706 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fbb100a2c20 0x7fbb100a50e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fbb1400b5c0 tx=0x7fbb14003960 comp rx=0 tx=0).stop 2024-04-24T23:29:31.968 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:31.964+0000 7fbb2c4df700 1 -- 172.21.15.80:0/3239099706 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbb24102d20 msgr2=0x7fbb24198580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:31.968 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:31.964+0000 7fbb2c4df700 1 --2- 172.21.15.80:0/3239099706 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbb24102d20 0x7fbb24198580 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fbb2000d980 tx=0x7fbb2000dc90 comp rx=0 tx=0).stop 2024-04-24T23:29:31.969 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:31.965+0000 7fbb2c4df700 1 -- 172.21.15.80:0/3239099706 shutdown_connections 2024-04-24T23:29:31.969 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:31.965+0000 7fbb2c4df700 1 --2- 172.21.15.80:0/3239099706 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb24107f80 0x7fbb2419faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:31.969 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:31.965+0000 7fbb2c4df700 1 --2- 172.21.15.80:0/3239099706 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fbb100a2c20 0x7fbb100a50e0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:31.969 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:31.965+0000 7fbb2c4df700 1 --2- 172.21.15.80:0/3239099706 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbb24102d20 0x7fbb24198580 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:31.969 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:31.965+0000 7fbb2c4df700 1 --2- 172.21.15.80:0/3239099706 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbb24105650 0x7fbb24198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:31.969 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:31.965+0000 7fbb2c4df700 1 -- 172.21.15.80:0/3239099706 >> 172.21.15.80:0/3239099706 conn(0x7fbb240fc730 msgr2=0x7fbb24103680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:31.969 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:31.965+0000 7fbb2c4df700 1 -- 172.21.15.80:0/3239099706 shutdown_connections 2024-04-24T23:29:31.969 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:31.965+0000 7fbb2c4df700 1 -- 172.21.15.80:0/3239099706 wait complete. 2024-04-24T23:29:32.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:29:31 smithi148 ceph-mon[32528]: pgmap v366: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T23:29:32.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:29:31 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/3239099706' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-24T23:29:32.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:29:31 smithi080 ceph-mon[29152]: pgmap v366: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T23:29:32.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:29:31 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/3239099706' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-24T23:29:32.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:29:31 smithi134 ceph-mon[32619]: pgmap v366: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T23:29:32.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:29:31 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/3239099706' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-24T23:29:32.792 INFO:teuthology.run_tasks:Running task fs.ready... 2024-04-24T23:29:32.806 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:29:32.912 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:32.908+0000 7f1fd87a9640 1 -- 172.21.15.80:0/2206392122 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1fd0101790 msgr2=0x7f1fd0101bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:32.912 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:32.908+0000 7f1fd87a9640 1 --2- 172.21.15.80:0/2206392122 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1fd0101790 0x7f1fd0101bb0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f1fc8009ca0 tx=0x7f1fc802a8c0 comp rx=0 tx=0).stop 2024-04-24T23:29:32.912 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:32.908+0000 7f1fd87a9640 1 -- 172.21.15.80:0/2206392122 shutdown_connections 2024-04-24T23:29:32.912 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:32.908+0000 7f1fd87a9640 1 --2- 172.21.15.80:0/2206392122 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1fd0103340 0x7f1fd0109bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:32.912 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:32.908+0000 7f1fd87a9640 1 --2- 172.21.15.80:0/2206392122 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1fd0102980 0x7f1fd0102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:32.912 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:32.908+0000 7f1fd87a9640 1 --2- 172.21.15.80:0/2206392122 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1fd0101790 0x7f1fd0101bb0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:32.912 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:32.908+0000 7f1fd87a9640 1 -- 172.21.15.80:0/2206392122 >> 172.21.15.80:0/2206392122 conn(0x7f1fd00fcd20 msgr2=0x7f1fd00ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:32.913 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:32.908+0000 7f1fd87a9640 1 -- 172.21.15.80:0/2206392122 shutdown_connections 2024-04-24T23:29:32.913 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:32.908+0000 7f1fd87a9640 1 -- 172.21.15.80:0/2206392122 wait complete. 2024-04-24T23:29:32.915 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:32.908+0000 7f1fd87a9640 1 Processor -- start 2024-04-24T23:29:32.915 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:32.909+0000 7f1fd87a9640 1 -- start start 2024-04-24T23:29:32.915 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:32.909+0000 7f1fd87a9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1fd0101790 0x7f1fd0197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:32.915 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:32.909+0000 7f1fd87a9640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1fd0102980 0x7f1fd0198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:32.915 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:32.909+0000 7f1fd87a9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1fd0103340 0x7f1fd019f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:32.915 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:32.909+0000 7f1fd87a9640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f1fd010d460 con 0x7f1fd0102980 2024-04-24T23:29:32.915 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:32.909+0000 7f1fd87a9640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1fd010d2e0 con 0x7f1fd0103340 2024-04-24T23:29:32.915 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:32.909+0000 7f1fd87a9640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1fd010d5e0 con 0x7f1fd0101790 2024-04-24T23:29:32.915 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:32.909+0000 7f1fd5d44640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1fd0102980 0x7f1fd0198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:32.915 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:32.909+0000 7f1fd5d44640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1fd0102980 0x7f1fd0198040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53572/0 (socket says 172.21.15.80:53572) 2024-04-24T23:29:32.915 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:32.909+0000 7f1fd5d44640 1 -- 172.21.15.80:0/3648061751 learned_addr learned my addr 172.21.15.80:0/3648061751 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:29:32.915 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:32.909+0000 7f1fd6d46640 1 --2- 172.21.15.80:0/3648061751 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1fd0103340 0x7f1fd019f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:32.915 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:32.909+0000 7f1fd5d44640 1 -- 172.21.15.80:0/3648061751 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1fd0101790 msgr2=0x7f1fd0197b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:32.916 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:32.909+0000 7f1fd5d44640 1 --2- 172.21.15.80:0/3648061751 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1fd0101790 0x7f1fd0197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:32.916 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:32.909+0000 7f1fd5d44640 1 -- 172.21.15.80:0/3648061751 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1fd0103340 msgr2=0x7f1fd019f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:32.916 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:32.909+0000 7f1fd5d44640 1 --2- 172.21.15.80:0/3648061751 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1fd0103340 0x7f1fd019f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:32.916 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:32.909+0000 7f1fd5d44640 1 -- 172.21.15.80:0/3648061751 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fd019f720 con 0x7f1fd0102980 2024-04-24T23:29:32.916 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:32.909+0000 7f1fd6d46640 1 --2- 172.21.15.80:0/3648061751 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1fd0103340 0x7f1fd019f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:29:32.916 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:32.909+0000 7f1fd5d44640 1 --2- 172.21.15.80:0/3648061751 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1fd0102980 0x7f1fd0198040 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f1fc400d950 tx=0x7f1fc400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:32.916 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:32.909+0000 7f1fcf7fe640 1 -- 172.21.15.80:0/3648061751 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fc4017070 con 0x7f1fd0102980 2024-04-24T23:29:32.916 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:32.909+0000 7f1fcf7fe640 1 -- 172.21.15.80:0/3648061751 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1fc4009c30 con 0x7f1fd0102980 2024-04-24T23:29:32.916 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:32.909+0000 7f1fcf7fe640 1 -- 172.21.15.80:0/3648061751 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fc400f6b0 con 0x7f1fd0102980 2024-04-24T23:29:32.916 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:32.909+0000 7f1fd87a9640 1 -- 172.21.15.80:0/3648061751 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fd019fa10 con 0x7f1fd0102980 2024-04-24T23:29:32.916 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:32.909+0000 7f1fd87a9640 1 -- 172.21.15.80:0/3648061751 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fd019ffe0 con 0x7f1fd0102980 2024-04-24T23:29:32.916 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:32.911+0000 7f1fd87a9640 1 -- 172.21.15.80:0/3648061751 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fd0108ab0 con 0x7f1fd0102980 2024-04-24T23:29:32.917 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:32.912+0000 7f1fcf7fe640 1 -- 172.21.15.80:0/3648061751 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1fc40109d0 con 0x7f1fd0102980 2024-04-24T23:29:32.917 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:32.912+0000 7f1fcf7fe640 1 --2- 172.21.15.80:0/3648061751 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f1fb80a2b90 0x7f1fb80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:32.917 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:32.912+0000 7f1fd6545640 1 --2- 172.21.15.80:0/3648061751 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f1fb80a2b90 0x7f1fb80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:32.917 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:32.912+0000 7f1fcf7fe640 1 -- 172.21.15.80:0/3648061751 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f1fc40cf110 con 0x7f1fd0102980 2024-04-24T23:29:32.918 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:32.913+0000 7f1fd6545640 1 --2- 172.21.15.80:0/3648061751 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f1fb80a2b90 0x7f1fb80a5050 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f1fc8005e30 tx=0x7f1fc8004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:32.918 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:32.917+0000 7f1fcf7fe640 1 -- 172.21.15.80:0/3648061751 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f1fc4009de0 con 0x7f1fd0102980 2024-04-24T23:29:33.094 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.094+0000 7f1fd87a9640 1 -- 172.21.15.80:0/3648061751 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1fd0062370 con 0x7f1fd0102980 2024-04-24T23:29:33.096 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.095+0000 7f1fcf7fe640 1 -- 172.21.15.80:0/3648061751 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24839 (secure 0 0 0) 0x7f1fc409bca0 con 0x7f1fd0102980 2024-04-24T23:29:33.097 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-24T23:29:33.097 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":94,"fsid":"ec7885aa-0290-11ef-bc93-c7b262605968","created":"2024-04-24T23:19:05.586407+0000","modified":"2024-04-24T23:29:08.209684+0000","last_up_change":"2024-04-24T23:26:25.938316+0000","last_in_change":"2024-04-24T23:26:04.786699+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:22:33.124428+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:27:17.575788+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:27:18.492762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T23:27:22.438503+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a525741d-b46c-4eef-aa64-3f057cac1497","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":2597718112},{"type":"v1","addr":"172.21.15.80:6803","nonce":2597718112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":2597718112},{"type":"v1","addr":"172.21.15.80:6805","nonce":2597718112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2597718112},{"type":"v1","addr":"172.21.15.80:6809","nonce":2597718112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":2597718112},{"type":"v1","addr":"172.21.15.80:6807","nonce":2597718112}]},"public_addr":"172.21.15.80:6803/2597718112","cluster_addr":"172.21.15.80:6805/2597718112","heartbeat_back_addr":"172.21.15.80:6809/2597718112","heartbeat_front_addr":"172.21.15.80:6807/2597718112","state":["exists","up"]},{"osd":1,"uuid":"2d0f0d67-6db4-40ac-91cc-58604ab6795a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":2395610201},{"type":"v1","addr":"172.21.15.134:6801","nonce":2395610201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":2395610201},{"type":"v1","addr":"172.21.15.134:6803","nonce":2395610201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":2395610201},{"type":"v1","addr":"172.21.15.134:6807","nonce":2395610201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":2395610201},{"type":"v1","addr":"172.21.15.134:6805","nonce":2395610201}]},"public_addr":"172.21.15.134:6801/2395610201","cluster_addr":"172.21.15.134:6803/2395610201","heartbeat_back_addr":"172.21.15.134:6807/2395610201","heartbeat_front_addr":"172.21.15.134:6805/2395610201","state":["exists","up"]},{"osd":2,"uuid":"781fb879-fedc-4c84-82fb-59e073bc4cf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1629152187},{"type":"v1","addr":"172.21.15.148:6801","nonce":1629152187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1629152187},{"type":"v1","addr":"172.21.15.148:6803","nonce":1629152187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1629152187},{"type":"v1","addr":"172.21.15.148:6807","nonce":1629152187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1629152187},{"type":"v1","addr":"172.21.15.148:6805","nonce":1629152187}]},"public_addr":"172.21.15.148:6801/1629152187","cluster_addr":"172.21.15.148:6803/1629152187","heartbeat_back_addr":"172.21.15.148:6807/1629152187","heartbeat_front_addr":"172.21.15.148:6805/1629152187","state":["exists","up"]},{"osd":3,"uuid":"cd982757-f2f6-4296-b3e7-4cd40fbd8d9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":3128298510},{"type":"v1","addr":"172.21.15.80:6811","nonce":3128298510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":3128298510},{"type":"v1","addr":"172.21.15.80:6813","nonce":3128298510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":3128298510},{"type":"v1","addr":"172.21.15.80:6817","nonce":3128298510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":3128298510},{"type":"v1","addr":"172.21.15.80:6815","nonce":3128298510}]},"public_addr":"172.21.15.80:6811/3128298510","cluster_addr":"172.21.15.80:6813/3128298510","heartbeat_back_addr":"172.21.15.80:6817/3128298510","heartbeat_front_addr":"172.21.15.80:6815/3128298510","state":["exists","up"]},{"osd":4,"uuid":"dfc55a79-a7e8-417a-b898-ec44171a54af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1917556060},{"type":"v1","addr":"172.21.15.134:6809","nonce":1917556060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1917556060},{"type":"v1","addr":"172.21.15.134:6811","nonce":1917556060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1917556060},{"type":"v1","addr":"172.21.15.134:6815","nonce":1917556060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1917556060},{"type":"v1","addr":"172.21.15.134:6813","nonce":1917556060}]},"public_addr":"172.21.15.134:6809/1917556060","cluster_addr":"172.21.15.134:6811/1917556060","heartbeat_back_addr":"172.21.15.134:6815/1917556060","heartbeat_front_addr":"172.21.15.134:6813/1917556060","state":["exists","up"]},{"osd":5,"uuid":"b5491337-0694-440c-b679-312f3d14cf65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":2715951281},{"type":"v1","addr":"172.21.15.148:6809","nonce":2715951281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":2715951281},{"type":"v1","addr":"172.21.15.148:6811","nonce":2715951281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":2715951281},{"type":"v1","addr":"172.21.15.148:6815","nonce":2715951281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":2715951281},{"type":"v1","addr":"172.21.15.148:6813","nonce":2715951281}]},"public_addr":"172.21.15.148:6809/2715951281","cluster_addr":"172.21.15.148:6811/2715951281","heartbeat_back_addr":"172.21.15.148:6815/2715951281","heartbeat_front_addr":"172.21.15.148:6813/2715951281","state":["exists","up"]},{"osd":6,"uuid":"ba4b5df7-d86e-47cd-ab67-cf763c4e4d6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":953261724},{"type":"v1","addr":"172.21.15.80:6819","nonce":953261724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":953261724},{"type":"v1","addr":"172.21.15.80:6821","nonce":953261724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":953261724},{"type":"v1","addr":"172.21.15.80:6825","nonce":953261724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":953261724},{"type":"v1","addr":"172.21.15.80:6823","nonce":953261724}]},"public_addr":"172.21.15.80:6819/953261724","cluster_addr":"172.21.15.80:6821/953261724","heartbeat_back_addr":"172.21.15.80:6825/953261724","heartbeat_front_addr":"172.21.15.80:6823/953261724","state":["exists","up"]},{"osd":7,"uuid":"464a503a-75ae-49e7-ab6e-a7f9f2a6a307","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":108354844},{"type":"v1","addr":"172.21.15.134:6817","nonce":108354844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":108354844},{"type":"v1","addr":"172.21.15.134:6819","nonce":108354844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":108354844},{"type":"v1","addr":"172.21.15.134:6823","nonce":108354844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":108354844},{"type":"v1","addr":"172.21.15.134:6821","nonce":108354844}]},"public_addr":"172.21.15.134:6817/108354844","cluster_addr":"172.21.15.134:6819/108354844","heartbeat_back_addr":"172.21.15.134:6823/108354844","heartbeat_front_addr":"172.21.15.134:6821/108354844","state":["exists","up"]},{"osd":8,"uuid":"ce544b2e-4540-47c0-9997-500c45351330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3559655329},{"type":"v1","addr":"172.21.15.148:6817","nonce":3559655329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":3559655329},{"type":"v1","addr":"172.21.15.148:6819","nonce":3559655329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":3559655329},{"type":"v1","addr":"172.21.15.148:6823","nonce":3559655329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3559655329},{"type":"v1","addr":"172.21.15.148:6821","nonce":3559655329}]},"public_addr":"172.21.15.148:6817/3559655329","cluster_addr":"172.21.15.148:6819/3559655329","heartbeat_back_addr":"172.21.15.148:6823/3559655329","heartbeat_front_addr":"172.21.15.148:6821/3559655329","state":["exists","up"]},{"osd":9,"uuid":"06de3732-19b9-496c-a464-55322efa6759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":4019913545},{"type":"v1","addr":"172.21.15.80:6827","nonce":4019913545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":4019913545},{"type":"v1","addr":"172.21.15.80:6829","nonce":4019913545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":4019913545},{"type":"v1","addr":"172.21.15.80:6833","nonce":4019913545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":4019913545},{"type":"v1","addr":"172.21.15.80:6831","nonce":4019913545}]},"public_addr":"172.21.15.80:6827/4019913545","cluster_addr":"172.21.15.80:6829/4019913545","heartbeat_back_addr":"172.21.15.80:6833/4019913545","heartbeat_front_addr":"172.21.15.80:6831/4019913545","state":["exists","up"]},{"osd":10,"uuid":"8d92f0d9-5c64-429a-b750-c33ae0e070c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":647343766},{"type":"v1","addr":"172.21.15.134:6825","nonce":647343766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":647343766},{"type":"v1","addr":"172.21.15.134:6827","nonce":647343766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":647343766},{"type":"v1","addr":"172.21.15.134:6831","nonce":647343766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":647343766},{"type":"v1","addr":"172.21.15.134:6829","nonce":647343766}]},"public_addr":"172.21.15.134:6825/647343766","cluster_addr":"172.21.15.134:6827/647343766","heartbeat_back_addr":"172.21.15.134:6831/647343766","heartbeat_front_addr":"172.21.15.134:6829/647343766","state":["exists","up"]},{"osd":11,"uuid":"4c003a41-f65b-479f-bc80-b27d680cf936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":4068024116},{"type":"v1","addr":"172.21.15.148:6825","nonce":4068024116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":4068024116},{"type":"v1","addr":"172.21.15.148:6827","nonce":4068024116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":4068024116},{"type":"v1","addr":"172.21.15.148:6831","nonce":4068024116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":4068024116},{"type":"v1","addr":"172.21.15.148:6829","nonce":4068024116}]},"public_addr":"172.21.15.148:6825/4068024116","cluster_addr":"172.21.15.148:6827/4068024116","heartbeat_back_addr":"172.21.15.148:6831/4068024116","heartbeat_front_addr":"172.21.15.148:6829/4068024116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:21:42.433631+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:22:06.145933+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:22:30.726143+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:22:55.729961+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:23:21.280913+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:23:46.609983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:24:12.585247+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:24:38.359287+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:25:04.491232+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:25:30.788089+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:25:56.891078+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:26:23.899113+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":7}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"4.a","mappings":[{"from":5,"to":8}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.80:0/2828378749":"2024-04-25T23:20:06.793609+0000","172.21.15.80:6801/233364790":"2024-04-25T23:20:06.793609+0000","172.21.15.80:6800/1614691410":"2024-04-25T23:19:40.490692+0000","172.21.15.80:0/1406187145":"2024-04-25T23:20:06.793609+0000","172.21.15.80:0/92953396":"2024-04-25T23:19:40.490692+0000","172.21.15.80:0/598129432":"2024-04-25T23:20:06.793609+0000","172.21.15.80:0/232193118":"2024-04-25T23:19:40.490692+0000","172.21.15.80:6800/233364790":"2024-04-25T23:20:06.793609+0000","172.21.15.80:6801/1614691410":"2024-04-25T23:19:40.490692+0000","172.21.15.80:0/2081562529":"2024-04-25T23:19:40.490692+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:29:33.099 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.097+0000 7f1fd87a9640 1 -- 172.21.15.80:0/3648061751 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f1fb80a2b90 msgr2=0x7f1fb80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:33.099 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.097+0000 7f1fd87a9640 1 --2- 172.21.15.80:0/3648061751 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f1fb80a2b90 0x7f1fb80a5050 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f1fc8005e30 tx=0x7f1fc8004a40 comp rx=0 tx=0).stop 2024-04-24T23:29:33.099 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.097+0000 7f1fd87a9640 1 -- 172.21.15.80:0/3648061751 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1fd0102980 msgr2=0x7f1fd0198040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:33.099 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.097+0000 7f1fd87a9640 1 --2- 172.21.15.80:0/3648061751 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1fd0102980 0x7f1fd0198040 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f1fc400d950 tx=0x7f1fc400de10 comp rx=0 tx=0).stop 2024-04-24T23:29:33.099 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.097+0000 7f1fd87a9640 1 -- 172.21.15.80:0/3648061751 shutdown_connections 2024-04-24T23:29:33.099 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.097+0000 7f1fd87a9640 1 --2- 172.21.15.80:0/3648061751 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f1fb80a2b90 0x7f1fb80a5050 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:33.099 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.097+0000 7f1fd87a9640 1 --2- 172.21.15.80:0/3648061751 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1fd0103340 0x7f1fd019f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:33.099 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.097+0000 7f1fd87a9640 1 --2- 172.21.15.80:0/3648061751 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1fd0102980 0x7f1fd0198040 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:33.099 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.097+0000 7f1fd87a9640 1 --2- 172.21.15.80:0/3648061751 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1fd0101790 0x7f1fd0197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:33.099 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.097+0000 7f1fd87a9640 1 -- 172.21.15.80:0/3648061751 >> 172.21.15.80:0/3648061751 conn(0x7f1fd00fcd20 msgr2=0x7f1fd0105c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:33.099 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.097+0000 7f1fd87a9640 1 -- 172.21.15.80:0/3648061751 shutdown_connections 2024-04-24T23:29:33.099 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.097+0000 7f1fd87a9640 1 -- 172.21.15.80:0/3648061751 wait complete. 2024-04-24T23:29:33.111 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:22:33.124428+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T23:27:17.575788+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T23:27:18.492762+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-24T23:27:22.438503+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-24T23:29:33.112 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:29:33.216 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.213+0000 7f2b0e8cd640 1 -- 172.21.15.80:0/3943713789 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2b08101790 msgr2=0x7f2b08101bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:33.217 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.213+0000 7f2b0e8cd640 1 --2- 172.21.15.80:0/3943713789 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2b08101790 0x7f2b08101bb0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f2afc009900 tx=0x7f2afc02a410 comp rx=0 tx=0).stop 2024-04-24T23:29:33.217 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.213+0000 7f2b0e8cd640 1 -- 172.21.15.80:0/3943713789 shutdown_connections 2024-04-24T23:29:33.217 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.213+0000 7f2b0e8cd640 1 --2- 172.21.15.80:0/3943713789 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2b08103340 0x7f2b08109bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:33.217 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.213+0000 7f2b0e8cd640 1 --2- 172.21.15.80:0/3943713789 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2b08102980 0x7f2b08102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:33.217 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.213+0000 7f2b0e8cd640 1 --2- 172.21.15.80:0/3943713789 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2b08101790 0x7f2b08101bb0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:33.217 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.213+0000 7f2b0e8cd640 1 -- 172.21.15.80:0/3943713789 >> 172.21.15.80:0/3943713789 conn(0x7f2b080fcd20 msgr2=0x7f2b080ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:33.217 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.213+0000 7f2b0e8cd640 1 -- 172.21.15.80:0/3943713789 shutdown_connections 2024-04-24T23:29:33.217 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.213+0000 7f2b0e8cd640 1 -- 172.21.15.80:0/3943713789 wait complete. 2024-04-24T23:29:33.217 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.213+0000 7f2b0e8cd640 1 Processor -- start 2024-04-24T23:29:33.217 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.214+0000 7f2b0e8cd640 1 -- start start 2024-04-24T23:29:33.217 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.214+0000 7f2b0e8cd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2b08101790 0x7f2b08197c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:33.218 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.214+0000 7f2b0e8cd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2b08102980 0x7f2b081981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:33.218 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.214+0000 7f2b0e8cd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2b08103340 0x7f2b0819f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:33.218 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.214+0000 7f2b0e8cd640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f2b0810d4e0 con 0x7f2b08101790 2024-04-24T23:29:33.218 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.214+0000 7f2b0e8cd640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2b0810d360 con 0x7f2b08102980 2024-04-24T23:29:33.218 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.214+0000 7f2b0e8cd640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f2b0810d660 con 0x7f2b08103340 2024-04-24T23:29:33.218 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.214+0000 7f2b07fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2b08101790 0x7f2b08197c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:33.218 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.214+0000 7f2b07fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2b08101790 0x7f2b08197c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53598/0 (socket says 172.21.15.80:53598) 2024-04-24T23:29:33.218 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.214+0000 7f2b07fff640 1 -- 172.21.15.80:0/4104929429 learned_addr learned my addr 172.21.15.80:0/4104929429 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:29:33.218 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.214+0000 7f2b0ce6a640 1 --2- 172.21.15.80:0/4104929429 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2b08103340 0x7f2b0819f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:33.218 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.214+0000 7f2b077fe640 1 --2- 172.21.15.80:0/4104929429 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2b08102980 0x7f2b081981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:33.218 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.214+0000 7f2b07fff640 1 -- 172.21.15.80:0/4104929429 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2b08103340 msgr2=0x7f2b0819f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:33.218 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.214+0000 7f2b07fff640 1 --2- 172.21.15.80:0/4104929429 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2b08103340 0x7f2b0819f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:33.219 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.214+0000 7f2b07fff640 1 -- 172.21.15.80:0/4104929429 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2b08102980 msgr2=0x7f2b081981c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:33.219 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.214+0000 7f2b07fff640 1 --2- 172.21.15.80:0/4104929429 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2b08102980 0x7f2b081981c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:33.219 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.214+0000 7f2b07fff640 1 -- 172.21.15.80:0/4104929429 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b0819f890 con 0x7f2b08101790 2024-04-24T23:29:33.219 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.214+0000 7f2b0ce6a640 1 --2- 172.21.15.80:0/4104929429 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2b08103340 0x7f2b0819f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:29:33.219 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.214+0000 7f2b077fe640 1 --2- 172.21.15.80:0/4104929429 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2b08102980 0x7f2b081981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:29:33.219 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.214+0000 7f2b07fff640 1 --2- 172.21.15.80:0/4104929429 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2b08101790 0x7f2b08197c80 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f2afc031610 tx=0x7f2afc0043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:33.219 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.215+0000 7f2b057fa640 1 -- 172.21.15.80:0/4104929429 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2afc03f030 con 0x7f2b08101790 2024-04-24T23:29:33.219 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.215+0000 7f2b0e8cd640 1 -- 172.21.15.80:0/4104929429 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b0819fb20 con 0x7f2b08101790 2024-04-24T23:29:33.219 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.215+0000 7f2b057fa640 1 -- 172.21.15.80:0/4104929429 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2afc003ce0 con 0x7f2b08101790 2024-04-24T23:29:33.219 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.215+0000 7f2b057fa640 1 -- 172.21.15.80:0/4104929429 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2afc035070 con 0x7f2b08101790 2024-04-24T23:29:33.219 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.215+0000 7f2b0e8cd640 1 -- 172.21.15.80:0/4104929429 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b081a0010 con 0x7f2b08101790 2024-04-24T23:29:33.219 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.216+0000 7f2b0e8cd640 1 -- 172.21.15.80:0/4104929429 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b08108b80 con 0x7f2b08101790 2024-04-24T23:29:33.220 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.217+0000 7f2b057fa640 1 -- 172.21.15.80:0/4104929429 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2afc042030 con 0x7f2b08101790 2024-04-24T23:29:33.220 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.217+0000 7f2b057fa640 1 --2- 172.21.15.80:0/4104929429 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f2af00a2b90 0x7f2af00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:33.220 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.217+0000 7f2b077fe640 1 --2- 172.21.15.80:0/4104929429 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f2af00a2b90 0x7f2af00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:33.220 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.218+0000 7f2b057fa640 1 -- 172.21.15.80:0/4104929429 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f2afc0efc00 con 0x7f2b08101790 2024-04-24T23:29:33.220 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.218+0000 7f2b077fe640 1 --2- 172.21.15.80:0/4104929429 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f2af00a2b90 0x7f2af00a5050 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f2af8006fd0 tx=0x7f2af8008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:33.223 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.222+0000 7f2b057fa640 1 -- 172.21.15.80:0/4104929429 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f2afc0f6080 con 0x7f2b08101790 2024-04-24T23:29:33.397 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.396+0000 7f2b0e8cd640 1 -- 172.21.15.80:0/4104929429 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f2b080623e0 con 0x7f2b08101790 2024-04-24T23:29:33.397 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.396+0000 7f2b057fa640 1 -- 172.21.15.80:0/4104929429 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7f2afc0bc790 con 0x7f2b08101790 2024-04-24T23:29:33.397 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 1 2024-04-24T23:29:33.399 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.398+0000 7f2b0e8cd640 1 -- 172.21.15.80:0/4104929429 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f2af00a2b90 msgr2=0x7f2af00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:33.399 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.398+0000 7f2b0e8cd640 1 --2- 172.21.15.80:0/4104929429 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f2af00a2b90 0x7f2af00a5050 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f2af8006fd0 tx=0x7f2af8008040 comp rx=0 tx=0).stop 2024-04-24T23:29:33.399 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.398+0000 7f2b0e8cd640 1 -- 172.21.15.80:0/4104929429 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2b08101790 msgr2=0x7f2b08197c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:33.400 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.398+0000 7f2b0e8cd640 1 --2- 172.21.15.80:0/4104929429 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2b08101790 0x7f2b08197c80 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f2afc031610 tx=0x7f2afc0043b0 comp rx=0 tx=0).stop 2024-04-24T23:29:33.400 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.398+0000 7f2b0e8cd640 1 -- 172.21.15.80:0/4104929429 shutdown_connections 2024-04-24T23:29:33.400 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.398+0000 7f2b0e8cd640 1 --2- 172.21.15.80:0/4104929429 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f2af00a2b90 0x7f2af00a5050 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:33.400 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.398+0000 7f2b0e8cd640 1 --2- 172.21.15.80:0/4104929429 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2b08103340 0x7f2b0819f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:33.400 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.398+0000 7f2b0e8cd640 1 --2- 172.21.15.80:0/4104929429 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2b08102980 0x7f2b081981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:33.400 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.398+0000 7f2b0e8cd640 1 --2- 172.21.15.80:0/4104929429 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2b08101790 0x7f2b08197c80 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:33.400 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.398+0000 7f2b0e8cd640 1 -- 172.21.15.80:0/4104929429 >> 172.21.15.80:0/4104929429 conn(0x7f2b080fcd20 msgr2=0x7f2b08105d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:33.400 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.398+0000 7f2b0e8cd640 1 -- 172.21.15.80:0/4104929429 shutdown_connections 2024-04-24T23:29:33.400 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.398+0000 7f2b0e8cd640 1 -- 172.21.15.80:0/4104929429 wait complete. 2024-04-24T23:29:33.411 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T23:29:33.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.510+0000 7f9c8cae1640 1 -- 172.21.15.80:0/4042571627 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9c88100680 msgr2=0x7f9c88100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:33.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.510+0000 7f9c8cae1640 1 --2- 172.21.15.80:0/4042571627 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9c88100680 0x7f9c88100aa0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f9c78009ca0 tx=0x7f9c7802a8c0 comp rx=0 tx=0).stop 2024-04-24T23:29:33.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.511+0000 7f9c8cae1640 1 -- 172.21.15.80:0/4042571627 shutdown_connections 2024-04-24T23:29:33.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.511+0000 7f9c8cae1640 1 --2- 172.21.15.80:0/4042571627 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9c88102230 0x7f9c88108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:33.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.511+0000 7f9c8cae1640 1 --2- 172.21.15.80:0/4042571627 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c88101870 0x7f9c88101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:33.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.511+0000 7f9c8cae1640 1 --2- 172.21.15.80:0/4042571627 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9c88100680 0x7f9c88100aa0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:33.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.511+0000 7f9c8cae1640 1 -- 172.21.15.80:0/4042571627 >> 172.21.15.80:0/4042571627 conn(0x7f9c880fbbf0 msgr2=0x7f9c880fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:33.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.511+0000 7f9c8cae1640 1 -- 172.21.15.80:0/4042571627 shutdown_connections 2024-04-24T23:29:33.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.511+0000 7f9c8cae1640 1 -- 172.21.15.80:0/4042571627 wait complete. 2024-04-24T23:29:33.516 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.511+0000 7f9c8cae1640 1 Processor -- start 2024-04-24T23:29:33.516 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.511+0000 7f9c8cae1640 1 -- start start 2024-04-24T23:29:33.516 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.511+0000 7f9c8cae1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c88100680 0x7f9c88069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:33.516 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.511+0000 7f9c8cae1640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9c88101870 0x7f9c8819c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:33.516 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.511+0000 7f9c8cae1640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9c88102230 0x7f9c8819eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:33.516 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.511+0000 7f9c8cae1640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f9c8810c350 con 0x7f9c88101870 2024-04-24T23:29:33.516 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.511+0000 7f9c8cae1640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9c8810c1d0 con 0x7f9c88102230 2024-04-24T23:29:33.516 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.511+0000 7f9c8cae1640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f9c8810c4d0 con 0x7f9c88100680 2024-04-24T23:29:33.516 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.512+0000 7f9c85d9b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9c88101870 0x7f9c8819c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:33.516 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.512+0000 7f9c85d9b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9c88101870 0x7f9c8819c8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53622/0 (socket says 172.21.15.80:53622) 2024-04-24T23:29:33.516 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.512+0000 7f9c85d9b640 1 -- 172.21.15.80:0/3063594653 learned_addr learned my addr 172.21.15.80:0/3063594653 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:29:33.516 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.512+0000 7f9c86d9d640 1 --2- 172.21.15.80:0/3063594653 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9c88102230 0x7f9c8819eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:33.517 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.512+0000 7f9c8659c640 1 --2- 172.21.15.80:0/3063594653 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c88100680 0x7f9c88069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:33.517 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.512+0000 7f9c85d9b640 1 -- 172.21.15.80:0/3063594653 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c88100680 msgr2=0x7f9c88069460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:33.517 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.512+0000 7f9c85d9b640 1 --2- 172.21.15.80:0/3063594653 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c88100680 0x7f9c88069460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:33.517 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.512+0000 7f9c85d9b640 1 -- 172.21.15.80:0/3063594653 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9c88102230 msgr2=0x7f9c8819eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:33.517 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.512+0000 7f9c85d9b640 1 --2- 172.21.15.80:0/3063594653 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9c88102230 0x7f9c8819eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:33.517 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.512+0000 7f9c85d9b640 1 -- 172.21.15.80:0/3063594653 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c8819f7a0 con 0x7f9c88101870 2024-04-24T23:29:33.518 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.512+0000 7f9c86d9d640 1 --2- 172.21.15.80:0/3063594653 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9c88102230 0x7f9c8819eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:29:33.518 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.512+0000 7f9c85d9b640 1 --2- 172.21.15.80:0/3063594653 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9c88101870 0x7f9c8819c8a0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c00b4a0 tx=0x7f9c7c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:33.518 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.512+0000 7f9c777fe640 1 -- 172.21.15.80:0/3063594653 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c7c014070 con 0x7f9c88101870 2024-04-24T23:29:33.518 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.512+0000 7f9c777fe640 1 -- 172.21.15.80:0/3063594653 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c7c010af0 con 0x7f9c88101870 2024-04-24T23:29:33.518 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.512+0000 7f9c8cae1640 1 -- 172.21.15.80:0/3063594653 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c8819fa90 con 0x7f9c88101870 2024-04-24T23:29:33.518 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.512+0000 7f9c777fe640 1 -- 172.21.15.80:0/3063594653 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c7c017d10 con 0x7f9c88101870 2024-04-24T23:29:33.518 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.512+0000 7f9c8cae1640 1 -- 172.21.15.80:0/3063594653 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c881a0060 con 0x7f9c88101870 2024-04-24T23:29:33.519 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.513+0000 7f9c8cae1640 1 -- 172.21.15.80:0/3063594653 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c88107a70 con 0x7f9c88101870 2024-04-24T23:29:33.521 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.514+0000 7f9c777fe640 1 -- 172.21.15.80:0/3063594653 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9c7c017070 con 0x7f9c88101870 2024-04-24T23:29:33.521 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.515+0000 7f9c777fe640 1 --2- 172.21.15.80:0/3063594653 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f9c700a2c10 0x7f9c700a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:33.521 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.515+0000 7f9c8659c640 1 --2- 172.21.15.80:0/3063594653 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f9c700a2c10 0x7f9c700a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:33.521 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.515+0000 7f9c777fe640 1 -- 172.21.15.80:0/3063594653 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f9c7c0ce890 con 0x7f9c88101870 2024-04-24T23:29:33.522 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.515+0000 7f9c8659c640 1 --2- 172.21.15.80:0/3063594653 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f9c700a2c10 0x7f9c700a50d0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f9c78006010 tx=0x7f9c78005f80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:33.522 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.519+0000 7f9c777fe640 1 -- 172.21.15.80:0/3063594653 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f9c7c09b3c0 con 0x7f9c88101870 2024-04-24T23:29:33.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.694+0000 7f9c8cae1640 1 -- 172.21.15.80:0/3063594653 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f9c88062340 con 0x7f9c88101870 2024-04-24T23:29:33.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.695+0000 7f9c777fe640 1 -- 172.21.15.80:0/3063594653 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) ==== 104+0+11 (secure 0 0 0) 0x7f9c7c09acc0 con 0x7f9c88101870 2024-04-24T23:29:33.695 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 32 2024-04-24T23:29:33.698 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.697+0000 7f9c8cae1640 1 -- 172.21.15.80:0/3063594653 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f9c700a2c10 msgr2=0x7f9c700a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:33.698 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.697+0000 7f9c8cae1640 1 --2- 172.21.15.80:0/3063594653 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f9c700a2c10 0x7f9c700a50d0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f9c78006010 tx=0x7f9c78005f80 comp rx=0 tx=0).stop 2024-04-24T23:29:33.698 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.697+0000 7f9c8cae1640 1 -- 172.21.15.80:0/3063594653 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9c88101870 msgr2=0x7f9c8819c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:33.698 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.697+0000 7f9c8cae1640 1 --2- 172.21.15.80:0/3063594653 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9c88101870 0x7f9c8819c8a0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c00b4a0 tx=0x7f9c7c00b960 comp rx=0 tx=0).stop 2024-04-24T23:29:33.698 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.697+0000 7f9c8cae1640 1 -- 172.21.15.80:0/3063594653 shutdown_connections 2024-04-24T23:29:33.698 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.697+0000 7f9c8cae1640 1 --2- 172.21.15.80:0/3063594653 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f9c700a2c10 0x7f9c700a50d0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:33.698 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.697+0000 7f9c8cae1640 1 --2- 172.21.15.80:0/3063594653 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9c88102230 0x7f9c8819eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:33.698 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.697+0000 7f9c8cae1640 1 --2- 172.21.15.80:0/3063594653 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9c88101870 0x7f9c8819c8a0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:33.698 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.697+0000 7f9c8cae1640 1 --2- 172.21.15.80:0/3063594653 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c88100680 0x7f9c88069460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:33.698 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.697+0000 7f9c8cae1640 1 -- 172.21.15.80:0/3063594653 >> 172.21.15.80:0/3063594653 conn(0x7f9c880fbbf0 msgr2=0x7f9c88104c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:33.698 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.697+0000 7f9c8cae1640 1 -- 172.21.15.80:0/3063594653 shutdown_connections 2024-04-24T23:29:33.699 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.697+0000 7f9c8cae1640 1 -- 172.21.15.80:0/3063594653 wait complete. 2024-04-24T23:29:33.709 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T23:29:33.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.809+0000 7fec7ae45640 1 -- 172.21.15.80:0/3378965580 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fec741017a0 msgr2=0x7fec74101bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:33.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.809+0000 7fec7ae45640 1 --2- 172.21.15.80:0/3378965580 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fec741017a0 0x7fec74101bc0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fec70009900 tx=0x7fec7002a410 comp rx=0 tx=0).stop 2024-04-24T23:29:33.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.809+0000 7fec7ae45640 1 -- 172.21.15.80:0/3378965580 shutdown_connections 2024-04-24T23:29:33.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.809+0000 7fec7ae45640 1 --2- 172.21.15.80:0/3378965580 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fec74103350 0x7fec74109be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:33.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.809+0000 7fec7ae45640 1 --2- 172.21.15.80:0/3378965580 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec74102990 0x7fec74102e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:33.813 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.809+0000 7fec7ae45640 1 --2- 172.21.15.80:0/3378965580 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fec741017a0 0x7fec74101bc0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:33.813 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.809+0000 7fec7ae45640 1 -- 172.21.15.80:0/3378965580 >> 172.21.15.80:0/3378965580 conn(0x7fec740fcd10 msgr2=0x7fec740ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:33.813 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.809+0000 7fec7ae45640 1 -- 172.21.15.80:0/3378965580 shutdown_connections 2024-04-24T23:29:33.813 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.809+0000 7fec7ae45640 1 -- 172.21.15.80:0/3378965580 wait complete. 2024-04-24T23:29:33.813 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.810+0000 7fec7ae45640 1 Processor -- start 2024-04-24T23:29:33.813 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.810+0000 7fec7ae45640 1 -- start start 2024-04-24T23:29:33.813 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.810+0000 7fec7ae45640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fec741017a0 0x7fec74199d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:33.813 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.810+0000 7fec7ae45640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fec74102990 0x7fec7419a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:33.813 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.810+0000 7fec7ae45640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec74103350 0x7fec741a12c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:33.813 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.810+0000 7fec7ae45640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fec74074f60 con 0x7fec741017a0 2024-04-24T23:29:33.813 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.810+0000 7fec7ae45640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fec74074de0 con 0x7fec74102990 2024-04-24T23:29:33.813 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.810+0000 7fec7ae45640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fec740750e0 con 0x7fec74103350 2024-04-24T23:29:33.813 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.810+0000 7fec78be1640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fec741017a0 0x7fec74199d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:33.813 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.810+0000 7fec78be1640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fec741017a0 0x7fec74199d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53650/0 (socket says 172.21.15.80:53650) 2024-04-24T23:29:33.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.810+0000 7fec78be1640 1 -- 172.21.15.80:0/1088706479 learned_addr learned my addr 172.21.15.80:0/1088706479 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:29:33.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.810+0000 7fec6bfff640 1 --2- 172.21.15.80:0/1088706479 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fec74102990 0x7fec7419a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:33.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.810+0000 7fec793e2640 1 --2- 172.21.15.80:0/1088706479 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec74103350 0x7fec741a12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:33.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.810+0000 7fec78be1640 1 -- 172.21.15.80:0/1088706479 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec74103350 msgr2=0x7fec741a12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:33.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.810+0000 7fec78be1640 1 --2- 172.21.15.80:0/1088706479 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec74103350 0x7fec741a12c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:33.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.810+0000 7fec78be1640 1 -- 172.21.15.80:0/1088706479 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fec74102990 msgr2=0x7fec7419a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:33.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.810+0000 7fec78be1640 1 --2- 172.21.15.80:0/1088706479 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fec74102990 0x7fec7419a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:33.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.810+0000 7fec78be1640 1 -- 172.21.15.80:0/1088706479 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec741a1960 con 0x7fec741017a0 2024-04-24T23:29:33.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.810+0000 7fec6bfff640 1 --2- 172.21.15.80:0/1088706479 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fec74102990 0x7fec7419a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:29:33.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.810+0000 7fec78be1640 1 --2- 172.21.15.80:0/1088706479 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fec741017a0 0x7fec74199d50 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fec70031610 tx=0x7fec700043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:33.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.811+0000 7fec69ffb640 1 -- 172.21.15.80:0/1088706479 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec7003f030 con 0x7fec741017a0 2024-04-24T23:29:33.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.811+0000 7fec7ae45640 1 -- 172.21.15.80:0/1088706479 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec741a1bf0 con 0x7fec741017a0 2024-04-24T23:29:33.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.811+0000 7fec69ffb640 1 -- 172.21.15.80:0/1088706479 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec70003ce0 con 0x7fec741017a0 2024-04-24T23:29:33.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.811+0000 7fec69ffb640 1 -- 172.21.15.80:0/1088706479 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec70035070 con 0x7fec741017a0 2024-04-24T23:29:33.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.811+0000 7fec7ae45640 1 -- 172.21.15.80:0/1088706479 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec741a20e0 con 0x7fec741017a0 2024-04-24T23:29:33.815 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.812+0000 7fec7ae45640 1 -- 172.21.15.80:0/1088706479 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec741089f0 con 0x7fec741017a0 2024-04-24T23:29:33.815 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.813+0000 7fec69ffb640 1 -- 172.21.15.80:0/1088706479 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fec70042030 con 0x7fec741017a0 2024-04-24T23:29:33.815 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.813+0000 7fec69ffb640 1 --2- 172.21.15.80:0/1088706479 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fec5c0a2b90 0x7fec5c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:33.816 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.814+0000 7fec69ffb640 1 -- 172.21.15.80:0/1088706479 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fec700efb20 con 0x7fec741017a0 2024-04-24T23:29:33.816 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.814+0000 7fec6bfff640 1 --2- 172.21.15.80:0/1088706479 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fec5c0a2b90 0x7fec5c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:33.816 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.814+0000 7fec6bfff640 1 --2- 172.21.15.80:0/1088706479 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fec5c0a2b90 0x7fec5c0a5050 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fec64009930 tx=0x7fec64008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:33.818 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.818+0000 7fec69ffb640 1 -- 172.21.15.80:0/1088706479 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fec700bc6b0 con 0x7fec741017a0 2024-04-24T23:29:33.994 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.993+0000 7fec7ae45640 1 -- 172.21.15.80:0/1088706479 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fec740623e0 con 0x7fec741017a0 2024-04-24T23:29:33.994 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.994+0000 7fec69ffb640 1 -- 172.21.15.80:0/1088706479 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) ==== 100+0+11 (secure 0 0 0) 0x7fec700bbfb0 con 0x7fec741017a0 2024-04-24T23:29:33.995 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 64 2024-04-24T23:29:33.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.996+0000 7fec7ae45640 1 -- 172.21.15.80:0/1088706479 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fec5c0a2b90 msgr2=0x7fec5c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:33.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.996+0000 7fec7ae45640 1 --2- 172.21.15.80:0/1088706479 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fec5c0a2b90 0x7fec5c0a5050 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fec64009930 tx=0x7fec64008040 comp rx=0 tx=0).stop 2024-04-24T23:29:33.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.996+0000 7fec7ae45640 1 -- 172.21.15.80:0/1088706479 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fec741017a0 msgr2=0x7fec74199d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:33.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.996+0000 7fec7ae45640 1 --2- 172.21.15.80:0/1088706479 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fec741017a0 0x7fec74199d50 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fec70031610 tx=0x7fec700043b0 comp rx=0 tx=0).stop 2024-04-24T23:29:33.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.996+0000 7fec7ae45640 1 -- 172.21.15.80:0/1088706479 shutdown_connections 2024-04-24T23:29:33.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.996+0000 7fec7ae45640 1 --2- 172.21.15.80:0/1088706479 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fec5c0a2b90 0x7fec5c0a5050 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:33.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.996+0000 7fec7ae45640 1 --2- 172.21.15.80:0/1088706479 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec74103350 0x7fec741a12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:33.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.996+0000 7fec7ae45640 1 --2- 172.21.15.80:0/1088706479 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fec74102990 0x7fec7419a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:33.998 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.996+0000 7fec7ae45640 1 --2- 172.21.15.80:0/1088706479 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fec741017a0 0x7fec74199d50 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:33.998 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.996+0000 7fec7ae45640 1 -- 172.21.15.80:0/1088706479 >> 172.21.15.80:0/1088706479 conn(0x7fec740fcd10 msgr2=0x7fec74105bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:33.998 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.996+0000 7fec7ae45640 1 -- 172.21.15.80:0/1088706479 shutdown_connections 2024-04-24T23:29:33.998 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:33.996+0000 7fec7ae45640 1 -- 172.21.15.80:0/1088706479 wait complete. 2024-04-24T23:29:34.008 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-24T23:29:34.115 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.107+0000 7f13227ed640 1 -- 172.21.15.80:0/269283188 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f131c104b30 msgr2=0x7f131c106f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:34.115 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.107+0000 7f13227ed640 1 --2- 172.21.15.80:0/269283188 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f131c104b30 0x7f131c106f20 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f1310009900 tx=0x7f131002a410 comp rx=0 tx=0).stop 2024-04-24T23:29:34.115 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.107+0000 7f13227ed640 1 -- 172.21.15.80:0/269283188 shutdown_connections 2024-04-24T23:29:34.115 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.107+0000 7f13227ed640 1 --2- 172.21.15.80:0/269283188 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f131c107460 0x7f131c1098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:34.115 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.107+0000 7f13227ed640 1 --2- 172.21.15.80:0/269283188 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f131c104b30 0x7f131c106f20 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:34.116 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.107+0000 7f13227ed640 1 --2- 172.21.15.80:0/269283188 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f131c102200 0x7f131c1045f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:34.116 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.107+0000 7f13227ed640 1 -- 172.21.15.80:0/269283188 >> 172.21.15.80:0/269283188 conn(0x7f131c0fbbf0 msgr2=0x7f131c0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:34.116 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.107+0000 7f13227ed640 1 -- 172.21.15.80:0/269283188 shutdown_connections 2024-04-24T23:29:34.116 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.107+0000 7f13227ed640 1 -- 172.21.15.80:0/269283188 wait complete. 2024-04-24T23:29:34.116 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.108+0000 7f13227ed640 1 Processor -- start 2024-04-24T23:29:34.116 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.108+0000 7f13227ed640 1 -- start start 2024-04-24T23:29:34.116 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.108+0000 7f13227ed640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f131c102200 0x7f131c197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:34.117 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.108+0000 7f13227ed640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f131c104b30 0x7f131c198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:34.117 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.108+0000 7f13227ed640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f131c107460 0x7f131c19f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:34.117 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.108+0000 7f13227ed640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f131c10d400 con 0x7f131c102200 2024-04-24T23:29:34.117 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.108+0000 7f13227ed640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f131c10d280 con 0x7f131c104b30 2024-04-24T23:29:34.117 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.108+0000 7f13227ed640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f131c10d580 con 0x7f131c107460 2024-04-24T23:29:34.117 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.108+0000 7f131bfff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f131c102200 0x7f131c197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:34.117 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.108+0000 7f131bfff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f131c102200 0x7f131c197b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53672/0 (socket says 172.21.15.80:53672) 2024-04-24T23:29:34.117 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.108+0000 7f131bfff640 1 -- 172.21.15.80:0/1066899453 learned_addr learned my addr 172.21.15.80:0/1066899453 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:29:34.117 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.108+0000 7f1320d8a640 1 --2- 172.21.15.80:0/1066899453 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f131c107460 0x7f131c19f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:34.118 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.109+0000 7f131bfff640 1 -- 172.21.15.80:0/1066899453 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f131c107460 msgr2=0x7f131c19f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:34.118 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.109+0000 7f131bfff640 1 --2- 172.21.15.80:0/1066899453 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f131c107460 0x7f131c19f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:34.118 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.109+0000 7f131b7fe640 1 --2- 172.21.15.80:0/1066899453 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f131c104b30 0x7f131c198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:34.118 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.109+0000 7f131bfff640 1 -- 172.21.15.80:0/1066899453 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f131c104b30 msgr2=0x7f131c198040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:34.118 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.109+0000 7f131bfff640 1 --2- 172.21.15.80:0/1066899453 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f131c104b30 0x7f131c198040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:34.118 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.109+0000 7f131bfff640 1 -- 172.21.15.80:0/1066899453 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f131c19f720 con 0x7f131c102200 2024-04-24T23:29:34.118 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.109+0000 7f1320d8a640 1 --2- 172.21.15.80:0/1066899453 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f131c107460 0x7f131c19f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:29:34.118 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.109+0000 7f131bfff640 1 --2- 172.21.15.80:0/1066899453 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f131c102200 0x7f131c197b00 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f130c00d950 tx=0x7f130c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:34.118 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.109+0000 7f13197fa640 1 -- 172.21.15.80:0/1066899453 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f130c017030 con 0x7f131c102200 2024-04-24T23:29:34.119 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.109+0000 7f13197fa640 1 -- 172.21.15.80:0/1066899453 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f130c004020 con 0x7f131c102200 2024-04-24T23:29:34.119 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.109+0000 7f13227ed640 1 -- 172.21.15.80:0/1066899453 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f131c19fa10 con 0x7f131c102200 2024-04-24T23:29:34.119 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.109+0000 7f13197fa640 1 -- 172.21.15.80:0/1066899453 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f130c0047a0 con 0x7f131c102200 2024-04-24T23:29:34.119 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.109+0000 7f13227ed640 1 -- 172.21.15.80:0/1066899453 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f131c19ff60 con 0x7f131c102200 2024-04-24T23:29:34.119 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.110+0000 7f13227ed640 1 -- 172.21.15.80:0/1066899453 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f131c0fd6e0 con 0x7f131c102200 2024-04-24T23:29:34.119 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.111+0000 7f13197fa640 1 -- 172.21.15.80:0/1066899453 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f130c0041d0 con 0x7f131c102200 2024-04-24T23:29:34.119 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.111+0000 7f13197fa640 1 --2- 172.21.15.80:0/1066899453 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f13040a2b90 0x7f13040a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:34.119 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.112+0000 7f131b7fe640 1 --2- 172.21.15.80:0/1066899453 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f13040a2b90 0x7f13040a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:34.119 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.112+0000 7f13197fa640 1 -- 172.21.15.80:0/1066899453 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f130c0cef30 con 0x7f131c102200 2024-04-24T23:29:34.119 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.112+0000 7f131b7fe640 1 --2- 172.21.15.80:0/1066899453 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f13040a2b90 0x7f13040a5050 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f1310006010 tx=0x7f1310000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:34.120 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.116+0000 7f13197fa640 1 -- 172.21.15.80:0/1066899453 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f130c09bac0 con 0x7f131c102200 2024-04-24T23:29:34.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:29:33 smithi148 ceph-mon[32528]: pgmap v367: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:29:34.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:29:33 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/3648061751' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:29:34.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:29:33 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/4104929429' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:29:34.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:29:33 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/3063594653' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T23:29:34.290 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:29:33 smithi080 ceph-mon[29152]: pgmap v367: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:29:34.290 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:29:33 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/3648061751' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:29:34.290 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:29:33 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/4104929429' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:29:34.290 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:29:33 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/3063594653' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T23:29:34.294 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.293+0000 7f13227ed640 1 -- 172.21.15.80:0/1066899453 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f131c062370 con 0x7f131c102200 2024-04-24T23:29:34.294 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.294+0000 7f13197fa640 1 -- 172.21.15.80:0/1066899453 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) ==== 103+0+11 (secure 0 0 0) 0x7f130c09b3c0 con 0x7f131c102200 2024-04-24T23:29:34.295 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 32 2024-04-24T23:29:34.297 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.296+0000 7f13227ed640 1 -- 172.21.15.80:0/1066899453 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f13040a2b90 msgr2=0x7f13040a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:34.297 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.296+0000 7f13227ed640 1 --2- 172.21.15.80:0/1066899453 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f13040a2b90 0x7f13040a5050 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f1310006010 tx=0x7f1310000bc0 comp rx=0 tx=0).stop 2024-04-24T23:29:34.297 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.296+0000 7f13227ed640 1 -- 172.21.15.80:0/1066899453 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f131c102200 msgr2=0x7f131c197b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:34.297 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.296+0000 7f13227ed640 1 --2- 172.21.15.80:0/1066899453 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f131c102200 0x7f131c197b00 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f130c00d950 tx=0x7f130c00de10 comp rx=0 tx=0).stop 2024-04-24T23:29:34.297 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.296+0000 7f13227ed640 1 -- 172.21.15.80:0/1066899453 shutdown_connections 2024-04-24T23:29:34.297 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.296+0000 7f13227ed640 1 --2- 172.21.15.80:0/1066899453 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f13040a2b90 0x7f13040a5050 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:34.297 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.296+0000 7f13227ed640 1 --2- 172.21.15.80:0/1066899453 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f131c107460 0x7f131c19f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:34.297 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.296+0000 7f13227ed640 1 --2- 172.21.15.80:0/1066899453 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f131c104b30 0x7f131c198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:34.297 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.296+0000 7f13227ed640 1 --2- 172.21.15.80:0/1066899453 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f131c102200 0x7f131c197b00 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:34.297 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.296+0000 7f13227ed640 1 -- 172.21.15.80:0/1066899453 >> 172.21.15.80:0/1066899453 conn(0x7f131c0fbbf0 msgr2=0x7f131c102ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:34.297 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.296+0000 7f13227ed640 1 -- 172.21.15.80:0/1066899453 shutdown_connections 2024-04-24T23:29:34.297 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.296+0000 7f13227ed640 1 -- 172.21.15.80:0/1066899453 wait complete. 2024-04-24T23:29:34.309 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:29:34.410 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.406+0000 7f4d8ba9a640 1 -- 172.21.15.80:0/1038255904 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4d84100680 msgr2=0x7f4d84100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:34.410 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.406+0000 7f4d8ba9a640 1 --2- 172.21.15.80:0/1038255904 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4d84100680 0x7f4d84100aa0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f4d80009ca0 tx=0x7f4d8002a8c0 comp rx=0 tx=0).stop 2024-04-24T23:29:34.410 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.406+0000 7f4d8ba9a640 1 -- 172.21.15.80:0/1038255904 shutdown_connections 2024-04-24T23:29:34.410 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.406+0000 7f4d8ba9a640 1 --2- 172.21.15.80:0/1038255904 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d84102230 0x7f4d84108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:34.410 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.406+0000 7f4d8ba9a640 1 --2- 172.21.15.80:0/1038255904 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d84101870 0x7f4d84101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:34.410 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.406+0000 7f4d8ba9a640 1 --2- 172.21.15.80:0/1038255904 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4d84100680 0x7f4d84100aa0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:34.410 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.406+0000 7f4d8ba9a640 1 -- 172.21.15.80:0/1038255904 >> 172.21.15.80:0/1038255904 conn(0x7f4d840fbbf0 msgr2=0x7f4d840fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:34.410 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.406+0000 7f4d8ba9a640 1 -- 172.21.15.80:0/1038255904 shutdown_connections 2024-04-24T23:29:34.410 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.407+0000 7f4d8ba9a640 1 -- 172.21.15.80:0/1038255904 wait complete. 2024-04-24T23:29:34.410 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.407+0000 7f4d8ba9a640 1 Processor -- start 2024-04-24T23:29:34.411 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.407+0000 7f4d8ba9a640 1 -- start start 2024-04-24T23:29:34.411 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.407+0000 7f4d8ba9a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d84100680 0x7f4d84069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:34.411 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.407+0000 7f4d8ba9a640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4d84101870 0x7f4d8419c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:34.411 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.407+0000 7f4d8ba9a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d84102230 0x7f4d8419ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:34.411 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.407+0000 7f4d8ba9a640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f4d8410c2d0 con 0x7f4d84101870 2024-04-24T23:29:34.411 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.407+0000 7f4d8ba9a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4d8410c150 con 0x7f4d84100680 2024-04-24T23:29:34.411 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.407+0000 7f4d8ba9a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4d8410c450 con 0x7f4d84102230 2024-04-24T23:29:34.411 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.407+0000 7f4d89035640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4d84101870 0x7f4d8419c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:34.411 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.407+0000 7f4d89035640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4d84101870 0x7f4d8419c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53678/0 (socket says 172.21.15.80:53678) 2024-04-24T23:29:34.411 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.407+0000 7f4d89035640 1 -- 172.21.15.80:0/2902549302 learned_addr learned my addr 172.21.15.80:0/2902549302 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:29:34.411 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.407+0000 7f4d8a037640 1 --2- 172.21.15.80:0/2902549302 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d84102230 0x7f4d8419ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:34.412 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.408+0000 7f4d89836640 1 --2- 172.21.15.80:0/2902549302 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d84100680 0x7f4d84069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:34.412 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.408+0000 7f4d89035640 1 -- 172.21.15.80:0/2902549302 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d84102230 msgr2=0x7f4d8419ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:34.412 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.408+0000 7f4d89035640 1 --2- 172.21.15.80:0/2902549302 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d84102230 0x7f4d8419ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:34.412 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.408+0000 7f4d89035640 1 -- 172.21.15.80:0/2902549302 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d84100680 msgr2=0x7f4d84069460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:34.412 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.408+0000 7f4d89035640 1 --2- 172.21.15.80:0/2902549302 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d84100680 0x7f4d84069460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:34.412 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.408+0000 7f4d89035640 1 -- 172.21.15.80:0/2902549302 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d8419f700 con 0x7f4d84101870 2024-04-24T23:29:34.412 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.408+0000 7f4d8a037640 1 --2- 172.21.15.80:0/2902549302 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d84102230 0x7f4d8419ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:29:34.412 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.408+0000 7f4d89035640 1 --2- 172.21.15.80:0/2902549302 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4d84101870 0x7f4d8419c800 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f4d7400d950 tx=0x7f4d7400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:34.412 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.408+0000 7f4d7affd640 1 -- 172.21.15.80:0/2902549302 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d74017030 con 0x7f4d84101870 2024-04-24T23:29:34.412 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.408+0000 7f4d7affd640 1 -- 172.21.15.80:0/2902549302 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d74004020 con 0x7f4d84101870 2024-04-24T23:29:34.412 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.408+0000 7f4d8ba9a640 1 -- 172.21.15.80:0/2902549302 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d8419f9f0 con 0x7f4d84101870 2024-04-24T23:29:34.412 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.408+0000 7f4d7affd640 1 -- 172.21.15.80:0/2902549302 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d740047a0 con 0x7f4d84101870 2024-04-24T23:29:34.412 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.408+0000 7f4d8ba9a640 1 -- 172.21.15.80:0/2902549302 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d8419ff40 con 0x7f4d84101870 2024-04-24T23:29:34.412 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.409+0000 7f4d8ba9a640 1 -- 172.21.15.80:0/2902549302 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d84107a10 con 0x7f4d84101870 2024-04-24T23:29:34.413 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.410+0000 7f4d7affd640 1 -- 172.21.15.80:0/2902549302 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4d740041d0 con 0x7f4d84101870 2024-04-24T23:29:34.413 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.411+0000 7f4d7affd640 1 --2- 172.21.15.80:0/2902549302 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f4d700a2b90 0x7f4d700a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:34.413 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.411+0000 7f4d89836640 1 --2- 172.21.15.80:0/2902549302 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f4d700a2b90 0x7f4d700a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:34.414 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.411+0000 7f4d7affd640 1 -- 172.21.15.80:0/2902549302 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f4d740cef10 con 0x7f4d84101870 2024-04-24T23:29:34.414 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.411+0000 7f4d89836640 1 --2- 172.21.15.80:0/2902549302 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f4d700a2b90 0x7f4d700a5050 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f4d80006010 tx=0x7f4d80009360 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:34.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.415+0000 7f4d7affd640 1 -- 172.21.15.80:0/2902549302 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f4d7409baa0 con 0x7f4d84101870 2024-04-24T23:29:34.449 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:29:33 smithi134 ceph-mon[32619]: pgmap v367: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:29:34.449 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:29:33 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/3648061751' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:29:34.449 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:29:33 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/4104929429' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:29:34.449 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:29:33 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/3063594653' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T23:29:34.627 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.626+0000 7f4d8ba9a640 1 -- 172.21.15.80:0/2902549302 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4d8402d170 con 0x7f4d84101870 2024-04-24T23:29:34.627 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.627+0000 7f4d7affd640 1 -- 172.21.15.80:0/2902549302 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11872 (secure 0 0 0) 0x7f4d7409b3a0 con 0x7f4d84101870 2024-04-24T23:29:34.628 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-24T23:29:34.628 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/891591085","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":891591085},{"type":"v1","addr":"172.21.15.80:6835","nonce":891591085}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6837/259529026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6836","nonce":259529026},{"type":"v1","addr":"172.21.15.80:6837","nonce":259529026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14646,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6839/4194284854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6838","nonce":4194284854},{"type":"v1","addr":"172.21.15.80:6839","nonce":4194284854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14658,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6841/3145730958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6840","nonce":3145730958},{"type":"v1","addr":"172.21.15.80:6841","nonce":3145730958}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24395,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/3736784385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3736784385},{"type":"v1","addr":"172.21.15.134:6833","nonce":3736784385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/124777462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":124777462},{"type":"v1","addr":"172.21.15.148:6833","nonce":124777462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T23:27:19.300566+0000","modified":"2024-04-24T23:27:45.981839+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24496,"mds_1":24488,"mds_2":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/1399801120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1399801120},{"type":"v1","addr":"172.21.15.134:6835","nonce":1399801120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/430512704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":430512704},{"type":"v1","addr":"172.21.15.134:6837","nonce":430512704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/3002319310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3002319310},{"type":"v1","addr":"172.21.15.148:6835","nonce":3002319310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.148:6837/856528403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":856528403},{"type":"v1","addr":"172.21.15.148:6837","nonce":856528403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6839/1743815021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":1743815021},{"type":"v1","addr":"172.21.15.134:6839","nonce":1743815021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.148:6839/2952757594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":2952757594},{"type":"v1","addr":"172.21.15.148:6839","nonce":2952757594}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24496,"qdb_cluster":[24472,24488,24496]},"id":1}]} 2024-04-24T23:29:34.628 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 24 2024-04-24T23:29:34.630 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.629+0000 7f4d8ba9a640 1 -- 172.21.15.80:0/2902549302 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f4d700a2b90 msgr2=0x7f4d700a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:34.630 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.629+0000 7f4d8ba9a640 1 --2- 172.21.15.80:0/2902549302 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f4d700a2b90 0x7f4d700a5050 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f4d80006010 tx=0x7f4d80009360 comp rx=0 tx=0).stop 2024-04-24T23:29:34.630 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.629+0000 7f4d8ba9a640 1 -- 172.21.15.80:0/2902549302 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4d84101870 msgr2=0x7f4d8419c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:34.630 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.629+0000 7f4d8ba9a640 1 --2- 172.21.15.80:0/2902549302 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4d84101870 0x7f4d8419c800 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f4d7400d950 tx=0x7f4d7400de10 comp rx=0 tx=0).stop 2024-04-24T23:29:34.630 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.629+0000 7f4d8ba9a640 1 -- 172.21.15.80:0/2902549302 shutdown_connections 2024-04-24T23:29:34.630 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.629+0000 7f4d8ba9a640 1 --2- 172.21.15.80:0/2902549302 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f4d700a2b90 0x7f4d700a5050 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:34.630 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.629+0000 7f4d8ba9a640 1 --2- 172.21.15.80:0/2902549302 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d84102230 0x7f4d8419ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:34.631 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.629+0000 7f4d8ba9a640 1 --2- 172.21.15.80:0/2902549302 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4d84101870 0x7f4d8419c800 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:34.631 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.629+0000 7f4d8ba9a640 1 --2- 172.21.15.80:0/2902549302 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d84100680 0x7f4d84069460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:34.631 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.629+0000 7f4d8ba9a640 1 -- 172.21.15.80:0/2902549302 >> 172.21.15.80:0/2902549302 conn(0x7f4d840fbbf0 msgr2=0x7f4d84104be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:34.631 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.629+0000 7f4d8ba9a640 1 -- 172.21.15.80:0/2902549302 shutdown_connections 2024-04-24T23:29:34.631 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.629+0000 7f4d8ba9a640 1 -- 172.21.15.80:0/2902549302 wait complete. 2024-04-24T23:29:34.642 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:29:34.743 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.739+0000 7f19113c1640 1 -- 172.21.15.80:0/2777743080 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f190c1013b0 msgr2=0x7f190c108c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:34.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.739+0000 7f19113c1640 1 --2- 172.21.15.80:0/2777743080 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f190c1013b0 0x7f190c108c50 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f1904009ca0 tx=0x7f190402a8c0 comp rx=0 tx=0).stop 2024-04-24T23:29:34.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.739+0000 7f19113c1640 1 -- 172.21.15.80:0/2777743080 shutdown_connections 2024-04-24T23:29:34.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.739+0000 7f19113c1640 1 --2- 172.21.15.80:0/2777743080 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f190c1013b0 0x7f190c108c50 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:34.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.739+0000 7f19113c1640 1 --2- 172.21.15.80:0/2777743080 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f190c1009f0 0x7f190c100e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:34.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.739+0000 7f19113c1640 1 --2- 172.21.15.80:0/2777743080 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f190c101cc0 0x7f190c1020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:34.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.739+0000 7f19113c1640 1 -- 172.21.15.80:0/2777743080 >> 172.21.15.80:0/2777743080 conn(0x7f190c0fb420 msgr2=0x7f190c0fd860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:34.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.739+0000 7f19113c1640 1 -- 172.21.15.80:0/2777743080 shutdown_connections 2024-04-24T23:29:34.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.739+0000 7f19113c1640 1 -- 172.21.15.80:0/2777743080 wait complete. 2024-04-24T23:29:34.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.739+0000 7f19113c1640 1 Processor -- start 2024-04-24T23:29:34.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.739+0000 7f19113c1640 1 -- start start 2024-04-24T23:29:34.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.739+0000 7f19113c1640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f190c1009f0 0x7f190c1047a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:34.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.739+0000 7f19113c1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f190c1013b0 0x7f190c102de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:34.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.739+0000 7f19113c1640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f190c101cc0 0x7f190c103320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:34.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.739+0000 7f19113c1640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f190c10c3b0 con 0x7f190c1009f0 2024-04-24T23:29:34.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.739+0000 7f19113c1640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f190c10c230 con 0x7f190c101cc0 2024-04-24T23:29:34.746 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.740+0000 7f19113c1640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f190c10c530 con 0x7f190c1013b0 2024-04-24T23:29:34.746 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.740+0000 7f190affd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f190c1009f0 0x7f190c1047a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:34.746 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.740+0000 7f190affd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f190c1009f0 0x7f190c1047a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53698/0 (socket says 172.21.15.80:53698) 2024-04-24T23:29:34.746 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.740+0000 7f190affd640 1 -- 172.21.15.80:0/2968618551 learned_addr learned my addr 172.21.15.80:0/2968618551 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:29:34.746 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.740+0000 7f190a7fc640 1 --2- 172.21.15.80:0/2968618551 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f190c1013b0 0x7f190c102de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:34.746 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.740+0000 7f190b7fe640 1 --2- 172.21.15.80:0/2968618551 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f190c101cc0 0x7f190c103320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:34.746 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.740+0000 7f190affd640 1 -- 172.21.15.80:0/2968618551 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f190c1013b0 msgr2=0x7f190c102de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:34.746 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.740+0000 7f190affd640 1 --2- 172.21.15.80:0/2968618551 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f190c1013b0 0x7f190c102de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:34.747 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.740+0000 7f190affd640 1 -- 172.21.15.80:0/2968618551 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f190c101cc0 msgr2=0x7f190c103320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:34.747 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.740+0000 7f190affd640 1 --2- 172.21.15.80:0/2968618551 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f190c101cc0 0x7f190c103320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:34.747 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.740+0000 7f190affd640 1 -- 172.21.15.80:0/2968618551 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f190c103bb0 con 0x7f190c1009f0 2024-04-24T23:29:34.747 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.740+0000 7f190b7fe640 1 --2- 172.21.15.80:0/2968618551 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f190c101cc0 0x7f190c103320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:29:34.747 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.740+0000 7f190affd640 1 --2- 172.21.15.80:0/2968618551 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f190c1009f0 0x7f190c1047a0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f190000e9c0 tx=0x7f190000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:34.747 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.740+0000 7f18f3fff640 1 -- 172.21.15.80:0/2968618551 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f190000cd80 con 0x7f190c1009f0 2024-04-24T23:29:34.747 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.740+0000 7f19113c1640 1 -- 172.21.15.80:0/2968618551 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f190c1a1c30 con 0x7f190c1009f0 2024-04-24T23:29:34.748 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.740+0000 7f18f3fff640 1 -- 172.21.15.80:0/2968618551 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19000035f0 con 0x7f190c1009f0 2024-04-24T23:29:34.748 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.740+0000 7f18f3fff640 1 -- 172.21.15.80:0/2968618551 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1900010690 con 0x7f190c1009f0 2024-04-24T23:29:34.748 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.740+0000 7f19113c1640 1 -- 172.21.15.80:0/2968618551 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f190c1a2130 con 0x7f190c1009f0 2024-04-24T23:29:34.748 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.741+0000 7f19113c1640 1 -- 172.21.15.80:0/2968618551 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f190c1a2360 con 0x7f190c1009f0 2024-04-24T23:29:34.748 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.742+0000 7f18f3fff640 1 -- 172.21.15.80:0/2968618551 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f19000037a0 con 0x7f190c1009f0 2024-04-24T23:29:34.748 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.743+0000 7f18f3fff640 1 --2- 172.21.15.80:0/2968618551 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f18f40a2b90 0x7f18f40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:34.748 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.743+0000 7f18f3fff640 1 -- 172.21.15.80:0/2968618551 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f19000cf8c0 con 0x7f190c1009f0 2024-04-24T23:29:34.748 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.743+0000 7f190a7fc640 1 --2- 172.21.15.80:0/2968618551 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f18f40a2b90 0x7f18f40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:34.749 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.743+0000 7f190a7fc640 1 --2- 172.21.15.80:0/2968618551 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f18f40a2b90 0x7f18f40a5050 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f190c104510 tx=0x7f18fc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:34.749 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.747+0000 7f18f3fff640 1 -- 172.21.15.80:0/2968618551 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f190009c450 con 0x7f190c1009f0 2024-04-24T23:29:34.922 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.922+0000 7f19113c1640 1 -- 172.21.15.80:0/2968618551 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f190c1a2530 con 0x7f190c1009f0 2024-04-24T23:29:34.924 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.923+0000 7f18f3fff640 1 -- 172.21.15.80:0/2968618551 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24839 (secure 0 0 0) 0x7f190009bd50 con 0x7f190c1009f0 2024-04-24T23:29:34.924 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-24T23:29:34.925 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":94,"fsid":"ec7885aa-0290-11ef-bc93-c7b262605968","created":"2024-04-24T23:19:05.586407+0000","modified":"2024-04-24T23:29:08.209684+0000","last_up_change":"2024-04-24T23:26:25.938316+0000","last_in_change":"2024-04-24T23:26:04.786699+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:22:33.124428+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:27:17.575788+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:27:18.492762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T23:27:22.438503+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a525741d-b46c-4eef-aa64-3f057cac1497","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":2597718112},{"type":"v1","addr":"172.21.15.80:6803","nonce":2597718112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":2597718112},{"type":"v1","addr":"172.21.15.80:6805","nonce":2597718112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2597718112},{"type":"v1","addr":"172.21.15.80:6809","nonce":2597718112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":2597718112},{"type":"v1","addr":"172.21.15.80:6807","nonce":2597718112}]},"public_addr":"172.21.15.80:6803/2597718112","cluster_addr":"172.21.15.80:6805/2597718112","heartbeat_back_addr":"172.21.15.80:6809/2597718112","heartbeat_front_addr":"172.21.15.80:6807/2597718112","state":["exists","up"]},{"osd":1,"uuid":"2d0f0d67-6db4-40ac-91cc-58604ab6795a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":2395610201},{"type":"v1","addr":"172.21.15.134:6801","nonce":2395610201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":2395610201},{"type":"v1","addr":"172.21.15.134:6803","nonce":2395610201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":2395610201},{"type":"v1","addr":"172.21.15.134:6807","nonce":2395610201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":2395610201},{"type":"v1","addr":"172.21.15.134:6805","nonce":2395610201}]},"public_addr":"172.21.15.134:6801/2395610201","cluster_addr":"172.21.15.134:6803/2395610201","heartbeat_back_addr":"172.21.15.134:6807/2395610201","heartbeat_front_addr":"172.21.15.134:6805/2395610201","state":["exists","up"]},{"osd":2,"uuid":"781fb879-fedc-4c84-82fb-59e073bc4cf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1629152187},{"type":"v1","addr":"172.21.15.148:6801","nonce":1629152187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1629152187},{"type":"v1","addr":"172.21.15.148:6803","nonce":1629152187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1629152187},{"type":"v1","addr":"172.21.15.148:6807","nonce":1629152187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1629152187},{"type":"v1","addr":"172.21.15.148:6805","nonce":1629152187}]},"public_addr":"172.21.15.148:6801/1629152187","cluster_addr":"172.21.15.148:6803/1629152187","heartbeat_back_addr":"172.21.15.148:6807/1629152187","heartbeat_front_addr":"172.21.15.148:6805/1629152187","state":["exists","up"]},{"osd":3,"uuid":"cd982757-f2f6-4296-b3e7-4cd40fbd8d9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":3128298510},{"type":"v1","addr":"172.21.15.80:6811","nonce":3128298510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":3128298510},{"type":"v1","addr":"172.21.15.80:6813","nonce":3128298510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":3128298510},{"type":"v1","addr":"172.21.15.80:6817","nonce":3128298510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":3128298510},{"type":"v1","addr":"172.21.15.80:6815","nonce":3128298510}]},"public_addr":"172.21.15.80:6811/3128298510","cluster_addr":"172.21.15.80:6813/3128298510","heartbeat_back_addr":"172.21.15.80:6817/3128298510","heartbeat_front_addr":"172.21.15.80:6815/3128298510","state":["exists","up"]},{"osd":4,"uuid":"dfc55a79-a7e8-417a-b898-ec44171a54af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1917556060},{"type":"v1","addr":"172.21.15.134:6809","nonce":1917556060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1917556060},{"type":"v1","addr":"172.21.15.134:6811","nonce":1917556060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1917556060},{"type":"v1","addr":"172.21.15.134:6815","nonce":1917556060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1917556060},{"type":"v1","addr":"172.21.15.134:6813","nonce":1917556060}]},"public_addr":"172.21.15.134:6809/1917556060","cluster_addr":"172.21.15.134:6811/1917556060","heartbeat_back_addr":"172.21.15.134:6815/1917556060","heartbeat_front_addr":"172.21.15.134:6813/1917556060","state":["exists","up"]},{"osd":5,"uuid":"b5491337-0694-440c-b679-312f3d14cf65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":2715951281},{"type":"v1","addr":"172.21.15.148:6809","nonce":2715951281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":2715951281},{"type":"v1","addr":"172.21.15.148:6811","nonce":2715951281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":2715951281},{"type":"v1","addr":"172.21.15.148:6815","nonce":2715951281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":2715951281},{"type":"v1","addr":"172.21.15.148:6813","nonce":2715951281}]},"public_addr":"172.21.15.148:6809/2715951281","cluster_addr":"172.21.15.148:6811/2715951281","heartbeat_back_addr":"172.21.15.148:6815/2715951281","heartbeat_front_addr":"172.21.15.148:6813/2715951281","state":["exists","up"]},{"osd":6,"uuid":"ba4b5df7-d86e-47cd-ab67-cf763c4e4d6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":953261724},{"type":"v1","addr":"172.21.15.80:6819","nonce":953261724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":953261724},{"type":"v1","addr":"172.21.15.80:6821","nonce":953261724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":953261724},{"type":"v1","addr":"172.21.15.80:6825","nonce":953261724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":953261724},{"type":"v1","addr":"172.21.15.80:6823","nonce":953261724}]},"public_addr":"172.21.15.80:6819/953261724","cluster_addr":"172.21.15.80:6821/953261724","heartbeat_back_addr":"172.21.15.80:6825/953261724","heartbeat_front_addr":"172.21.15.80:6823/953261724","state":["exists","up"]},{"osd":7,"uuid":"464a503a-75ae-49e7-ab6e-a7f9f2a6a307","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":108354844},{"type":"v1","addr":"172.21.15.134:6817","nonce":108354844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":108354844},{"type":"v1","addr":"172.21.15.134:6819","nonce":108354844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":108354844},{"type":"v1","addr":"172.21.15.134:6823","nonce":108354844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":108354844},{"type":"v1","addr":"172.21.15.134:6821","nonce":108354844}]},"public_addr":"172.21.15.134:6817/108354844","cluster_addr":"172.21.15.134:6819/108354844","heartbeat_back_addr":"172.21.15.134:6823/108354844","heartbeat_front_addr":"172.21.15.134:6821/108354844","state":["exists","up"]},{"osd":8,"uuid":"ce544b2e-4540-47c0-9997-500c45351330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3559655329},{"type":"v1","addr":"172.21.15.148:6817","nonce":3559655329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":3559655329},{"type":"v1","addr":"172.21.15.148:6819","nonce":3559655329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":3559655329},{"type":"v1","addr":"172.21.15.148:6823","nonce":3559655329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3559655329},{"type":"v1","addr":"172.21.15.148:6821","nonce":3559655329}]},"public_addr":"172.21.15.148:6817/3559655329","cluster_addr":"172.21.15.148:6819/3559655329","heartbeat_back_addr":"172.21.15.148:6823/3559655329","heartbeat_front_addr":"172.21.15.148:6821/3559655329","state":["exists","up"]},{"osd":9,"uuid":"06de3732-19b9-496c-a464-55322efa6759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":4019913545},{"type":"v1","addr":"172.21.15.80:6827","nonce":4019913545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":4019913545},{"type":"v1","addr":"172.21.15.80:6829","nonce":4019913545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":4019913545},{"type":"v1","addr":"172.21.15.80:6833","nonce":4019913545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":4019913545},{"type":"v1","addr":"172.21.15.80:6831","nonce":4019913545}]},"public_addr":"172.21.15.80:6827/4019913545","cluster_addr":"172.21.15.80:6829/4019913545","heartbeat_back_addr":"172.21.15.80:6833/4019913545","heartbeat_front_addr":"172.21.15.80:6831/4019913545","state":["exists","up"]},{"osd":10,"uuid":"8d92f0d9-5c64-429a-b750-c33ae0e070c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":647343766},{"type":"v1","addr":"172.21.15.134:6825","nonce":647343766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":647343766},{"type":"v1","addr":"172.21.15.134:6827","nonce":647343766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":647343766},{"type":"v1","addr":"172.21.15.134:6831","nonce":647343766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":647343766},{"type":"v1","addr":"172.21.15.134:6829","nonce":647343766}]},"public_addr":"172.21.15.134:6825/647343766","cluster_addr":"172.21.15.134:6827/647343766","heartbeat_back_addr":"172.21.15.134:6831/647343766","heartbeat_front_addr":"172.21.15.134:6829/647343766","state":["exists","up"]},{"osd":11,"uuid":"4c003a41-f65b-479f-bc80-b27d680cf936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":4068024116},{"type":"v1","addr":"172.21.15.148:6825","nonce":4068024116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":4068024116},{"type":"v1","addr":"172.21.15.148:6827","nonce":4068024116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":4068024116},{"type":"v1","addr":"172.21.15.148:6831","nonce":4068024116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":4068024116},{"type":"v1","addr":"172.21.15.148:6829","nonce":4068024116}]},"public_addr":"172.21.15.148:6825/4068024116","cluster_addr":"172.21.15.148:6827/4068024116","heartbeat_back_addr":"172.21.15.148:6831/4068024116","heartbeat_front_addr":"172.21.15.148:6829/4068024116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:21:42.433631+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:22:06.145933+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:22:30.726143+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:22:55.729961+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:23:21.280913+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:23:46.609983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:24:12.585247+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:24:38.359287+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:25:04.491232+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:25:30.788089+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:25:56.891078+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:26:23.899113+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":7}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"4.a","mappings":[{"from":5,"to":8}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.80:0/2828378749":"2024-04-25T23:20:06.793609+0000","172.21.15.80:6801/233364790":"2024-04-25T23:20:06.793609+0000","172.21.15.80:6800/1614691410":"2024-04-25T23:19:40.490692+0000","172.21.15.80:0/1406187145":"2024-04-25T23:20:06.793609+0000","172.21.15.80:0/92953396":"2024-04-25T23:19:40.490692+0000","172.21.15.80:0/598129432":"2024-04-25T23:20:06.793609+0000","172.21.15.80:0/232193118":"2024-04-25T23:19:40.490692+0000","172.21.15.80:6800/233364790":"2024-04-25T23:20:06.793609+0000","172.21.15.80:6801/1614691410":"2024-04-25T23:19:40.490692+0000","172.21.15.80:0/2081562529":"2024-04-25T23:19:40.490692+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:29:34.926 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.925+0000 7f19113c1640 1 -- 172.21.15.80:0/2968618551 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f18f40a2b90 msgr2=0x7f18f40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:34.927 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.925+0000 7f19113c1640 1 --2- 172.21.15.80:0/2968618551 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f18f40a2b90 0x7f18f40a5050 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f190c104510 tx=0x7f18fc005f90 comp rx=0 tx=0).stop 2024-04-24T23:29:34.927 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.925+0000 7f19113c1640 1 -- 172.21.15.80:0/2968618551 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f190c1009f0 msgr2=0x7f190c1047a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:34.927 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.925+0000 7f19113c1640 1 --2- 172.21.15.80:0/2968618551 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f190c1009f0 0x7f190c1047a0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f190000e9c0 tx=0x7f190000ee80 comp rx=0 tx=0).stop 2024-04-24T23:29:34.927 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.925+0000 7f19113c1640 1 -- 172.21.15.80:0/2968618551 shutdown_connections 2024-04-24T23:29:34.927 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.925+0000 7f19113c1640 1 --2- 172.21.15.80:0/2968618551 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f18f40a2b90 0x7f18f40a5050 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:34.927 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.925+0000 7f19113c1640 1 --2- 172.21.15.80:0/2968618551 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f190c101cc0 0x7f190c103320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:34.927 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.925+0000 7f19113c1640 1 --2- 172.21.15.80:0/2968618551 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f190c1013b0 0x7f190c102de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:34.927 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.925+0000 7f19113c1640 1 --2- 172.21.15.80:0/2968618551 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f190c1009f0 0x7f190c1047a0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:34.927 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.925+0000 7f19113c1640 1 -- 172.21.15.80:0/2968618551 >> 172.21.15.80:0/2968618551 conn(0x7f190c0fb420 msgr2=0x7f190c0fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:34.927 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.926+0000 7f19113c1640 1 -- 172.21.15.80:0/2968618551 shutdown_connections 2024-04-24T23:29:34.927 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:34.926+0000 7f19113c1640 1 -- 172.21.15.80:0/2968618551 wait complete. 2024-04-24T23:29:34.938 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:22:33.124428+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T23:27:17.575788+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T23:27:18.492762+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-24T23:27:22.438503+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-24T23:29:34.939 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:29:35.045 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.042+0000 7fa722415640 1 -- 172.21.15.80:0/2456217952 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa71c100690 msgr2=0x7fa71c100ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:35.045 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.042+0000 7fa722415640 1 --2- 172.21.15.80:0/2456217952 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa71c100690 0x7fa71c100ab0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fa704009900 tx=0x7fa70402a410 comp rx=0 tx=0).stop 2024-04-24T23:29:35.045 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.042+0000 7fa722415640 1 -- 172.21.15.80:0/2456217952 shutdown_connections 2024-04-24T23:29:35.046 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.042+0000 7fa722415640 1 --2- 172.21.15.80:0/2456217952 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa71c102240 0x7fa71c108ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:35.046 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.042+0000 7fa722415640 1 --2- 172.21.15.80:0/2456217952 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa71c101880 0x7fa71c101d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:35.046 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.042+0000 7fa722415640 1 --2- 172.21.15.80:0/2456217952 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa71c100690 0x7fa71c100ab0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:35.046 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.042+0000 7fa722415640 1 -- 172.21.15.80:0/2456217952 >> 172.21.15.80:0/2456217952 conn(0x7fa71c0fbc00 msgr2=0x7fa71c0fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:35.046 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.042+0000 7fa722415640 1 -- 172.21.15.80:0/2456217952 shutdown_connections 2024-04-24T23:29:35.046 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.042+0000 7fa722415640 1 -- 172.21.15.80:0/2456217952 wait complete. 2024-04-24T23:29:35.046 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.042+0000 7fa722415640 1 Processor -- start 2024-04-24T23:29:35.046 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.042+0000 7fa722415640 1 -- start start 2024-04-24T23:29:35.046 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.043+0000 7fa722415640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa71c100690 0x7fa71c1969b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:35.046 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.043+0000 7fa722415640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa71c101880 0x7fa71c196ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:35.046 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.043+0000 7fa722415640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa71c102240 0x7fa71c19df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:35.046 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.043+0000 7fa722415640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fa71c10c310 con 0x7fa71c101880 2024-04-24T23:29:35.047 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.043+0000 7fa722415640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa71c10c190 con 0x7fa71c102240 2024-04-24T23:29:35.047 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.043+0000 7fa722415640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa71c10c490 con 0x7fa71c100690 2024-04-24T23:29:35.047 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.043+0000 7fa71b7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa71c101880 0x7fa71c196ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:35.047 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.043+0000 7fa71b7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa71c101880 0x7fa71c196ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53722/0 (socket says 172.21.15.80:53722) 2024-04-24T23:29:35.047 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.043+0000 7fa71b7fe640 1 -- 172.21.15.80:0/1434744189 learned_addr learned my addr 172.21.15.80:0/1434744189 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:29:35.047 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.043+0000 7fa71bfff640 1 --2- 172.21.15.80:0/1434744189 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa71c100690 0x7fa71c1969b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:35.047 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.043+0000 7fa7209b2640 1 --2- 172.21.15.80:0/1434744189 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa71c102240 0x7fa71c19df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:35.047 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.043+0000 7fa71b7fe640 1 -- 172.21.15.80:0/1434744189 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa71c100690 msgr2=0x7fa71c1969b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:35.047 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.043+0000 7fa71b7fe640 1 --2- 172.21.15.80:0/1434744189 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa71c100690 0x7fa71c1969b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:35.047 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.043+0000 7fa71b7fe640 1 -- 172.21.15.80:0/1434744189 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa71c102240 msgr2=0x7fa71c19df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:35.047 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.043+0000 7fa71b7fe640 1 --2- 172.21.15.80:0/1434744189 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa71c102240 0x7fa71c19df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:35.047 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.043+0000 7fa71b7fe640 1 -- 172.21.15.80:0/1434744189 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa71c19e5d0 con 0x7fa71c101880 2024-04-24T23:29:35.047 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.043+0000 7fa7209b2640 1 --2- 172.21.15.80:0/1434744189 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa71c102240 0x7fa71c19df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:29:35.047 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.043+0000 7fa71b7fe640 1 --2- 172.21.15.80:0/1434744189 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa71c101880 0x7fa71c196ef0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fa70c00c890 tx=0x7fa70c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:35.048 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.043+0000 7fa7197fa640 1 -- 172.21.15.80:0/1434744189 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa70c004220 con 0x7fa71c101880 2024-04-24T23:29:35.048 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.043+0000 7fa722415640 1 -- 172.21.15.80:0/1434744189 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa71c19e8c0 con 0x7fa71c101880 2024-04-24T23:29:35.048 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.043+0000 7fa7197fa640 1 -- 172.21.15.80:0/1434744189 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa70c0043c0 con 0x7fa71c101880 2024-04-24T23:29:35.048 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.043+0000 7fa7197fa640 1 -- 172.21.15.80:0/1434744189 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa70c004a00 con 0x7fa71c101880 2024-04-24T23:29:35.048 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.043+0000 7fa722415640 1 -- 172.21.15.80:0/1434744189 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa71c19ee60 con 0x7fa71c101880 2024-04-24T23:29:35.048 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.045+0000 7fa722415640 1 -- 172.21.15.80:0/1434744189 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa71c107a30 con 0x7fa71c101880 2024-04-24T23:29:35.049 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.045+0000 7fa7197fa640 1 -- 172.21.15.80:0/1434744189 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa70c0079b0 con 0x7fa71c101880 2024-04-24T23:29:35.049 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.046+0000 7fa7197fa640 1 --2- 172.21.15.80:0/1434744189 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fa7080a2b90 0x7fa7080a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:35.049 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.046+0000 7fa71bfff640 1 --2- 172.21.15.80:0/1434744189 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fa7080a2b90 0x7fa7080a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:35.049 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.046+0000 7fa7197fa640 1 -- 172.21.15.80:0/1434744189 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fa70c0cf1c0 con 0x7fa71c101880 2024-04-24T23:29:35.049 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.046+0000 7fa71bfff640 1 --2- 172.21.15.80:0/1434744189 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fa7080a2b90 0x7fa7080a5050 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa704006010 tx=0x7fa704000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:35.051 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.050+0000 7fa7197fa640 1 -- 172.21.15.80:0/1434744189 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa70c014030 con 0x7fa71c101880 2024-04-24T23:29:35.226 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.225+0000 7fa722415640 1 -- 172.21.15.80:0/1434744189 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa71c062340 con 0x7fa71c101880 2024-04-24T23:29:35.226 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.225+0000 7fa7197fa640 1 -- 172.21.15.80:0/1434744189 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7fa70c09bd50 con 0x7fa71c101880 2024-04-24T23:29:35.226 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 1 2024-04-24T23:29:35.228 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.227+0000 7fa722415640 1 -- 172.21.15.80:0/1434744189 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fa7080a2b90 msgr2=0x7fa7080a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:35.228 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.227+0000 7fa722415640 1 --2- 172.21.15.80:0/1434744189 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fa7080a2b90 0x7fa7080a5050 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa704006010 tx=0x7fa704000bc0 comp rx=0 tx=0).stop 2024-04-24T23:29:35.228 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.227+0000 7fa722415640 1 -- 172.21.15.80:0/1434744189 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa71c101880 msgr2=0x7fa71c196ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:35.228 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.227+0000 7fa722415640 1 --2- 172.21.15.80:0/1434744189 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa71c101880 0x7fa71c196ef0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fa70c00c890 tx=0x7fa70c00cd50 comp rx=0 tx=0).stop 2024-04-24T23:29:35.228 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.227+0000 7fa722415640 1 -- 172.21.15.80:0/1434744189 shutdown_connections 2024-04-24T23:29:35.228 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.227+0000 7fa722415640 1 --2- 172.21.15.80:0/1434744189 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fa7080a2b90 0x7fa7080a5050 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:35.228 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.227+0000 7fa722415640 1 --2- 172.21.15.80:0/1434744189 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa71c102240 0x7fa71c19df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:35.229 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.227+0000 7fa722415640 1 --2- 172.21.15.80:0/1434744189 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa71c101880 0x7fa71c196ef0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:35.229 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.227+0000 7fa722415640 1 --2- 172.21.15.80:0/1434744189 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa71c100690 0x7fa71c1969b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:35.229 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.227+0000 7fa722415640 1 -- 172.21.15.80:0/1434744189 >> 172.21.15.80:0/1434744189 conn(0x7fa71c0fbc00 msgr2=0x7fa71c104c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:35.229 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.227+0000 7fa722415640 1 -- 172.21.15.80:0/1434744189 shutdown_connections 2024-04-24T23:29:35.229 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.227+0000 7fa722415640 1 -- 172.21.15.80:0/1434744189 wait complete. 2024-04-24T23:29:35.238 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:29:35 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/1088706479' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T23:29:35.239 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:29:35 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/1066899453' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T23:29:35.239 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:29:35 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/2902549302' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:29:35.239 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:29:35 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/2968618551' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:29:35.240 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T23:29:35.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:29:35 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/1088706479' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T23:29:35.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:29:35 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/1066899453' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T23:29:35.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:29:35 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/2902549302' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:29:35.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:29:35 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/2968618551' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:29:35.342 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.339+0000 7f690d8b3640 1 -- 172.21.15.80:0/3439444430 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6908100680 msgr2=0x7f6908100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:35.342 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.339+0000 7f690d8b3640 1 --2- 172.21.15.80:0/3439444430 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6908100680 0x7f6908100aa0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f68f8009900 tx=0x7f68f802a410 comp rx=0 tx=0).stop 2024-04-24T23:29:35.342 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.339+0000 7f690d8b3640 1 -- 172.21.15.80:0/3439444430 shutdown_connections 2024-04-24T23:29:35.342 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.339+0000 7f690d8b3640 1 --2- 172.21.15.80:0/3439444430 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6908102230 0x7f6908108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:35.342 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.339+0000 7f690d8b3640 1 --2- 172.21.15.80:0/3439444430 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6908101870 0x7f6908101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:35.342 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.339+0000 7f690d8b3640 1 --2- 172.21.15.80:0/3439444430 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6908100680 0x7f6908100aa0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:35.342 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.339+0000 7f690d8b3640 1 -- 172.21.15.80:0/3439444430 >> 172.21.15.80:0/3439444430 conn(0x7f69080fbbf0 msgr2=0x7f69080fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:35.342 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.339+0000 7f690d8b3640 1 -- 172.21.15.80:0/3439444430 shutdown_connections 2024-04-24T23:29:35.343 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.339+0000 7f690d8b3640 1 -- 172.21.15.80:0/3439444430 wait complete. 2024-04-24T23:29:35.343 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.340+0000 7f690d8b3640 1 Processor -- start 2024-04-24T23:29:35.343 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.340+0000 7f690d8b3640 1 -- start start 2024-04-24T23:29:35.343 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.340+0000 7f690d8b3640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6908100680 0x7f6908197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:35.343 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.340+0000 7f690d8b3640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6908101870 0x7f6908197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:35.343 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.340+0000 7f690d8b3640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6908102230 0x7f690819f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:35.343 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.340+0000 7f690d8b3640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f690810d420 con 0x7f6908100680 2024-04-24T23:29:35.343 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.340+0000 7f690d8b3640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f690810d2a0 con 0x7f6908101870 2024-04-24T23:29:35.343 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.340+0000 7f690d8b3640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f690810d5a0 con 0x7f6908102230 2024-04-24T23:29:35.343 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.340+0000 7f6906ffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6908100680 0x7f6908197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:35.343 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.340+0000 7f6906ffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6908100680 0x7f6908197aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53750/0 (socket says 172.21.15.80:53750) 2024-04-24T23:29:35.343 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.340+0000 7f6906ffd640 1 -- 172.21.15.80:0/2557334701 learned_addr learned my addr 172.21.15.80:0/2557334701 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:29:35.343 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.340+0000 7f6906ffd640 1 -- 172.21.15.80:0/2557334701 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6908102230 msgr2=0x7f690819f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:35.344 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.340+0000 7f6906ffd640 1 --2- 172.21.15.80:0/2557334701 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6908102230 0x7f690819f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:35.344 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.340+0000 7f6906ffd640 1 -- 172.21.15.80:0/2557334701 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6908101870 msgr2=0x7f6908197fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:35.344 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.340+0000 7f6906ffd640 1 --2- 172.21.15.80:0/2557334701 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6908101870 0x7f6908197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:35.344 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.340+0000 7f6906ffd640 1 -- 172.21.15.80:0/2557334701 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f690819f6c0 con 0x7f6908100680 2024-04-24T23:29:35.344 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.340+0000 7f6906ffd640 1 --2- 172.21.15.80:0/2557334701 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6908100680 0x7f6908197aa0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f68f8031610 tx=0x7f68f80043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:35.344 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.341+0000 7f690c8b1640 1 -- 172.21.15.80:0/2557334701 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f803f030 con 0x7f6908100680 2024-04-24T23:29:35.344 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.341+0000 7f690d8b3640 1 -- 172.21.15.80:0/2557334701 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f690819f950 con 0x7f6908100680 2024-04-24T23:29:35.344 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.341+0000 7f690c8b1640 1 -- 172.21.15.80:0/2557334701 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68f8003ce0 con 0x7f6908100680 2024-04-24T23:29:35.344 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.341+0000 7f690c8b1640 1 -- 172.21.15.80:0/2557334701 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f8035070 con 0x7f6908100680 2024-04-24T23:29:35.344 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.341+0000 7f690d8b3640 1 -- 172.21.15.80:0/2557334701 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f690819fe40 con 0x7f6908100680 2024-04-24T23:29:35.345 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.342+0000 7f690d8b3640 1 -- 172.21.15.80:0/2557334701 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69081079a0 con 0x7f6908100680 2024-04-24T23:29:35.345 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.343+0000 7f690c8b1640 1 -- 172.21.15.80:0/2557334701 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f68f8042030 con 0x7f6908100680 2024-04-24T23:29:35.345 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.343+0000 7f690c8b1640 1 --2- 172.21.15.80:0/2557334701 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f68f00a2b90 0x7f68f00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:35.345 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.343+0000 7f69067fc640 1 --2- 172.21.15.80:0/2557334701 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f68f00a2b90 0x7f68f00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:35.346 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.343+0000 7f690c8b1640 1 -- 172.21.15.80:0/2557334701 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f68f80efb70 con 0x7f6908100680 2024-04-24T23:29:35.346 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.344+0000 7f69067fc640 1 --2- 172.21.15.80:0/2557334701 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f68f00a2b90 0x7f68f00a5050 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f6908198fb0 tx=0x7f68fc005bf0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:35.348 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.348+0000 7f690c8b1640 1 -- 172.21.15.80:0/2557334701 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f68f80bc700 con 0x7f6908100680 2024-04-24T23:29:35.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:29:35 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/1088706479' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T23:29:35.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:29:35 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/1066899453' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T23:29:35.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:29:35 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/2902549302' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:29:35.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:29:35 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/2968618551' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:29:35.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.522+0000 7f690d8b3640 1 -- 172.21.15.80:0/2557334701 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f69080623e0 con 0x7f6908100680 2024-04-24T23:29:35.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.523+0000 7f690c8b1640 1 -- 172.21.15.80:0/2557334701 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) ==== 104+0+11 (secure 0 0 0) 0x7f68f80bc000 con 0x7f6908100680 2024-04-24T23:29:35.524 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 32 2024-04-24T23:29:35.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.525+0000 7f690d8b3640 1 -- 172.21.15.80:0/2557334701 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f68f00a2b90 msgr2=0x7f68f00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:35.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.525+0000 7f690d8b3640 1 --2- 172.21.15.80:0/2557334701 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f68f00a2b90 0x7f68f00a5050 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f6908198fb0 tx=0x7f68fc005bf0 comp rx=0 tx=0).stop 2024-04-24T23:29:35.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.525+0000 7f690d8b3640 1 -- 172.21.15.80:0/2557334701 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6908100680 msgr2=0x7f6908197aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:35.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.525+0000 7f690d8b3640 1 --2- 172.21.15.80:0/2557334701 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6908100680 0x7f6908197aa0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f68f8031610 tx=0x7f68f80043b0 comp rx=0 tx=0).stop 2024-04-24T23:29:35.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.525+0000 7f690d8b3640 1 -- 172.21.15.80:0/2557334701 shutdown_connections 2024-04-24T23:29:35.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.525+0000 7f690d8b3640 1 --2- 172.21.15.80:0/2557334701 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f68f00a2b90 0x7f68f00a5050 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:35.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.525+0000 7f690d8b3640 1 --2- 172.21.15.80:0/2557334701 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6908102230 0x7f690819f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:35.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.525+0000 7f690d8b3640 1 --2- 172.21.15.80:0/2557334701 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6908101870 0x7f6908197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:35.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.525+0000 7f690d8b3640 1 --2- 172.21.15.80:0/2557334701 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6908100680 0x7f6908197aa0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:35.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.525+0000 7f690d8b3640 1 -- 172.21.15.80:0/2557334701 >> 172.21.15.80:0/2557334701 conn(0x7f69080fbbf0 msgr2=0x7f6908104b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:35.527 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.525+0000 7f690d8b3640 1 -- 172.21.15.80:0/2557334701 shutdown_connections 2024-04-24T23:29:35.527 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.525+0000 7f690d8b3640 1 -- 172.21.15.80:0/2557334701 wait complete. 2024-04-24T23:29:35.537 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T23:29:35.641 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.637+0000 7fbc9f0ae640 1 -- 172.21.15.80:0/1013467219 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbc98100680 msgr2=0x7fbc98100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:35.641 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.637+0000 7fbc9f0ae640 1 --2- 172.21.15.80:0/1013467219 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbc98100680 0x7fbc98100aa0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fbc8c009900 tx=0x7fbc8c02a410 comp rx=0 tx=0).stop 2024-04-24T23:29:35.641 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.637+0000 7fbc9f0ae640 1 -- 172.21.15.80:0/1013467219 shutdown_connections 2024-04-24T23:29:35.641 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.637+0000 7fbc9f0ae640 1 --2- 172.21.15.80:0/1013467219 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc98102230 0x7fbc98108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:35.641 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.637+0000 7fbc9f0ae640 1 --2- 172.21.15.80:0/1013467219 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbc98101870 0x7fbc98101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:35.641 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.637+0000 7fbc9f0ae640 1 --2- 172.21.15.80:0/1013467219 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbc98100680 0x7fbc98100aa0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:35.641 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.637+0000 7fbc9f0ae640 1 -- 172.21.15.80:0/1013467219 >> 172.21.15.80:0/1013467219 conn(0x7fbc980fbbf0 msgr2=0x7fbc980fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:35.641 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.637+0000 7fbc9f0ae640 1 -- 172.21.15.80:0/1013467219 shutdown_connections 2024-04-24T23:29:35.641 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.637+0000 7fbc9f0ae640 1 -- 172.21.15.80:0/1013467219 wait complete. 2024-04-24T23:29:35.642 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.638+0000 7fbc9f0ae640 1 Processor -- start 2024-04-24T23:29:35.642 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.638+0000 7fbc9f0ae640 1 -- start start 2024-04-24T23:29:35.642 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.638+0000 7fbc9f0ae640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbc98100680 0x7fbc98196970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:35.642 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.638+0000 7fbc9f0ae640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc98101870 0x7fbc98196eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:35.642 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.638+0000 7fbc9f0ae640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbc98102230 0x7fbc9819df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:35.642 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.638+0000 7fbc9f0ae640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fbc9810c2d0 con 0x7fbc98100680 2024-04-24T23:29:35.642 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.638+0000 7fbc9f0ae640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbc9810c150 con 0x7fbc98102230 2024-04-24T23:29:35.642 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.638+0000 7fbc9f0ae640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fbc9810c450 con 0x7fbc98101870 2024-04-24T23:29:35.642 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.638+0000 7fbc9ce4a640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbc98100680 0x7fbc98196970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:35.642 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.638+0000 7fbc9ce4a640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbc98100680 0x7fbc98196970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53770/0 (socket says 172.21.15.80:53770) 2024-04-24T23:29:35.642 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.638+0000 7fbc9ce4a640 1 -- 172.21.15.80:0/1740723861 learned_addr learned my addr 172.21.15.80:0/1740723861 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:29:35.642 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.638+0000 7fbc97fff640 1 --2- 172.21.15.80:0/1740723861 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc98101870 0x7fbc98196eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:35.642 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.638+0000 7fbc9ce4a640 1 -- 172.21.15.80:0/1740723861 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc98101870 msgr2=0x7fbc98196eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:35.643 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.638+0000 7fbc9ce4a640 1 --2- 172.21.15.80:0/1740723861 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc98101870 0x7fbc98196eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:35.643 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.638+0000 7fbc9ce4a640 1 -- 172.21.15.80:0/1740723861 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbc98102230 msgr2=0x7fbc9819df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:35.643 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.638+0000 7fbc9ce4a640 1 --2- 172.21.15.80:0/1740723861 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbc98102230 0x7fbc9819df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:35.643 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.639+0000 7fbc9ce4a640 1 -- 172.21.15.80:0/1740723861 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc9819e590 con 0x7fbc98100680 2024-04-24T23:29:35.643 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.639+0000 7fbc97fff640 1 --2- 172.21.15.80:0/1740723861 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc98101870 0x7fbc98196eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:29:35.643 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.639+0000 7fbc9ce4a640 1 --2- 172.21.15.80:0/1740723861 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbc98100680 0x7fbc98196970 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fbc8c031610 tx=0x7fbc8c0043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:35.643 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.639+0000 7fbc95ffb640 1 -- 172.21.15.80:0/1740723861 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc8c03f030 con 0x7fbc98100680 2024-04-24T23:29:35.643 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.639+0000 7fbc95ffb640 1 -- 172.21.15.80:0/1740723861 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc8c003ce0 con 0x7fbc98100680 2024-04-24T23:29:35.643 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.639+0000 7fbc95ffb640 1 -- 172.21.15.80:0/1740723861 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc8c035070 con 0x7fbc98100680 2024-04-24T23:29:35.643 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.639+0000 7fbc9f0ae640 1 -- 172.21.15.80:0/1740723861 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc9819e820 con 0x7fbc98100680 2024-04-24T23:29:35.643 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.639+0000 7fbc9f0ae640 1 -- 172.21.15.80:0/1740723861 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc9819ed10 con 0x7fbc98100680 2024-04-24T23:29:35.643 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.640+0000 7fbc9f0ae640 1 -- 172.21.15.80:0/1740723861 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc98107a70 con 0x7fbc98100680 2024-04-24T23:29:35.644 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.641+0000 7fbc95ffb640 1 -- 172.21.15.80:0/1740723861 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fbc8c042030 con 0x7fbc98100680 2024-04-24T23:29:35.644 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.642+0000 7fbc95ffb640 1 --2- 172.21.15.80:0/1740723861 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fbc880a2b90 0x7fbc880a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:35.644 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.642+0000 7fbc97fff640 1 --2- 172.21.15.80:0/1740723861 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fbc880a2b90 0x7fbc880a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:35.644 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.642+0000 7fbc95ffb640 1 -- 172.21.15.80:0/1740723861 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fbc8c0efa60 con 0x7fbc98100680 2024-04-24T23:29:35.644 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.642+0000 7fbc97fff640 1 --2- 172.21.15.80:0/1740723861 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fbc880a2b90 0x7fbc880a5050 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fbc98197e80 tx=0x7fbc8400a5c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:35.647 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.646+0000 7fbc95ffb640 1 -- 172.21.15.80:0/1740723861 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fbc8c0bc5f0 con 0x7fbc98100680 2024-04-24T23:29:35.824 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.821+0000 7fbc9f0ae640 1 -- 172.21.15.80:0/1740723861 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fbc980623e0 con 0x7fbc98100680 2024-04-24T23:29:35.824 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.821+0000 7fbc95ffb640 1 -- 172.21.15.80:0/1740723861 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) ==== 100+0+11 (secure 0 0 0) 0x7fbc8c0bbef0 con 0x7fbc98100680 2024-04-24T23:29:35.824 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 64 2024-04-24T23:29:35.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.823+0000 7fbc9f0ae640 1 -- 172.21.15.80:0/1740723861 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fbc880a2b90 msgr2=0x7fbc880a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:35.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.823+0000 7fbc9f0ae640 1 --2- 172.21.15.80:0/1740723861 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fbc880a2b90 0x7fbc880a5050 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fbc98197e80 tx=0x7fbc8400a5c0 comp rx=0 tx=0).stop 2024-04-24T23:29:35.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.823+0000 7fbc9f0ae640 1 -- 172.21.15.80:0/1740723861 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbc98100680 msgr2=0x7fbc98196970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:35.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.823+0000 7fbc9f0ae640 1 --2- 172.21.15.80:0/1740723861 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbc98100680 0x7fbc98196970 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fbc8c031610 tx=0x7fbc8c0043b0 comp rx=0 tx=0).stop 2024-04-24T23:29:35.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.823+0000 7fbc9f0ae640 1 -- 172.21.15.80:0/1740723861 shutdown_connections 2024-04-24T23:29:35.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.823+0000 7fbc9f0ae640 1 --2- 172.21.15.80:0/1740723861 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fbc880a2b90 0x7fbc880a5050 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:35.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.823+0000 7fbc9f0ae640 1 --2- 172.21.15.80:0/1740723861 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbc98102230 0x7fbc9819df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:35.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.823+0000 7fbc9f0ae640 1 --2- 172.21.15.80:0/1740723861 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc98101870 0x7fbc98196eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:35.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.823+0000 7fbc9f0ae640 1 --2- 172.21.15.80:0/1740723861 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbc98100680 0x7fbc98196970 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:35.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.823+0000 7fbc9f0ae640 1 -- 172.21.15.80:0/1740723861 >> 172.21.15.80:0/1740723861 conn(0x7fbc980fbbf0 msgr2=0x7fbc98104c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:35.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.823+0000 7fbc9f0ae640 1 -- 172.21.15.80:0/1740723861 shutdown_connections 2024-04-24T23:29:35.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.823+0000 7fbc9f0ae640 1 -- 172.21.15.80:0/1740723861 wait complete. 2024-04-24T23:29:35.835 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-24T23:29:35.938 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.935+0000 7f3cf0c38640 1 -- 172.21.15.80:0/2292284856 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3cec0ff7f0 msgr2=0x7f3cec0ffc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:35.938 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.935+0000 7f3cf0c38640 1 --2- 172.21.15.80:0/2292284856 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3cec0ff7f0 0x7f3cec0ffc70 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f3cdc009900 tx=0x7f3cdc02a410 comp rx=0 tx=0).stop 2024-04-24T23:29:35.938 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.935+0000 7f3cf0c38640 1 -- 172.21.15.80:0/2292284856 shutdown_connections 2024-04-24T23:29:35.938 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.935+0000 7f3cf0c38640 1 --2- 172.21.15.80:0/2292284856 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3cec1001b0 0x7f3cec109d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:35.938 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.935+0000 7f3cf0c38640 1 --2- 172.21.15.80:0/2292284856 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3cec0ff7f0 0x7f3cec0ffc70 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:35.938 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.935+0000 7f3cf0c38640 1 --2- 172.21.15.80:0/2292284856 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3cec102d80 0x7f3cec1031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:35.938 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.935+0000 7f3cf0c38640 1 -- 172.21.15.80:0/2292284856 >> 172.21.15.80:0/2292284856 conn(0x7f3cec0fb370 msgr2=0x7f3cec0fd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:35.938 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.935+0000 7f3cf0c38640 1 -- 172.21.15.80:0/2292284856 shutdown_connections 2024-04-24T23:29:35.939 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.935+0000 7f3cf0c38640 1 -- 172.21.15.80:0/2292284856 wait complete. 2024-04-24T23:29:35.939 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.936+0000 7f3cf0c38640 1 Processor -- start 2024-04-24T23:29:35.939 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.936+0000 7f3cf0c38640 1 -- start start 2024-04-24T23:29:35.939 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.936+0000 7f3cf0c38640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3cec0ff7f0 0x7f3cec197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:35.939 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.936+0000 7f3cf0c38640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3cec1001b0 0x7f3cec197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:35.939 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.936+0000 7f3cf0c38640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3cec102d80 0x7f3cec19f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:35.939 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.936+0000 7f3cf0c38640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f3cec10d430 con 0x7f3cec102d80 2024-04-24T23:29:35.939 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.936+0000 7f3cf0c38640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3cec10d2b0 con 0x7f3cec0ff7f0 2024-04-24T23:29:35.939 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.936+0000 7f3cf0c38640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3cec10d5b0 con 0x7f3cec1001b0 2024-04-24T23:29:35.939 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.936+0000 7f3cead9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3cec102d80 0x7f3cec19f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:35.939 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.936+0000 7f3cead9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3cec102d80 0x7f3cec19f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53792/0 (socket says 172.21.15.80:53792) 2024-04-24T23:29:35.939 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.936+0000 7f3cead9d640 1 -- 172.21.15.80:0/339848005 learned_addr learned my addr 172.21.15.80:0/339848005 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:29:35.939 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.936+0000 7f3cea59c640 1 --2- 172.21.15.80:0/339848005 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3cec0ff7f0 0x7f3cec197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:35.940 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.936+0000 7f3ce9d9b640 1 --2- 172.21.15.80:0/339848005 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3cec1001b0 0x7f3cec197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:35.940 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.936+0000 7f3cead9d640 1 -- 172.21.15.80:0/339848005 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3cec1001b0 msgr2=0x7f3cec197ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:35.940 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.936+0000 7f3cead9d640 1 --2- 172.21.15.80:0/339848005 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3cec1001b0 0x7f3cec197ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:35.940 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.936+0000 7f3cead9d640 1 -- 172.21.15.80:0/339848005 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3cec0ff7f0 msgr2=0x7f3cec197ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:35.940 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.936+0000 7f3cead9d640 1 --2- 172.21.15.80:0/339848005 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3cec0ff7f0 0x7f3cec197ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:35.940 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.936+0000 7f3cead9d640 1 -- 172.21.15.80:0/339848005 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cec19f6d0 con 0x7f3cec102d80 2024-04-24T23:29:35.940 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.936+0000 7f3ce9d9b640 1 --2- 172.21.15.80:0/339848005 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3cec1001b0 0x7f3cec197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:29:35.940 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.936+0000 7f3cead9d640 1 --2- 172.21.15.80:0/339848005 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3cec102d80 0x7f3cec19f060 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f3ce4009fd0 tx=0x7f3ce400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:35.940 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.936+0000 7f3ce37fe640 1 -- 172.21.15.80:0/339848005 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ce4017070 con 0x7f3cec102d80 2024-04-24T23:29:35.940 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.937+0000 7f3ce37fe640 1 -- 172.21.15.80:0/339848005 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ce4003e50 con 0x7f3cec102d80 2024-04-24T23:29:35.940 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.937+0000 7f3cf0c38640 1 -- 172.21.15.80:0/339848005 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cec19f9c0 con 0x7f3cec102d80 2024-04-24T23:29:35.940 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.937+0000 7f3ce37fe640 1 -- 172.21.15.80:0/339848005 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ce40106c0 con 0x7f3cec102d80 2024-04-24T23:29:35.940 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.937+0000 7f3cf0c38640 1 -- 172.21.15.80:0/339848005 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cec19fe90 con 0x7f3cec102d80 2024-04-24T23:29:35.942 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.938+0000 7f3cf0c38640 1 -- 172.21.15.80:0/339848005 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3cec108cc0 con 0x7f3cec102d80 2024-04-24T23:29:35.942 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.939+0000 7f3ce37fe640 1 -- 172.21.15.80:0/339848005 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3ce4003970 con 0x7f3cec102d80 2024-04-24T23:29:35.942 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.939+0000 7f3ce37fe640 1 --2- 172.21.15.80:0/339848005 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f3cd80a2b90 0x7f3cd80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:35.942 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.939+0000 7f3cea59c640 1 --2- 172.21.15.80:0/339848005 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f3cd80a2b90 0x7f3cd80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:35.942 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.939+0000 7f3ce37fe640 1 -- 172.21.15.80:0/339848005 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f3ce40cf1f0 con 0x7f3cec102d80 2024-04-24T23:29:35.943 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.940+0000 7f3cea59c640 1 --2- 172.21.15.80:0/339848005 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f3cd80a2b90 0x7f3cd80a5050 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f3cd4005fd0 tx=0x7f3cd400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:35.944 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:35.944+0000 7f3ce37fe640 1 -- 172.21.15.80:0/339848005 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f3ce409bd80 con 0x7f3cec102d80 2024-04-24T23:29:36.126 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.125+0000 7f3cf0c38640 1 -- 172.21.15.80:0/339848005 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f3cec062340 con 0x7f3cec102d80 2024-04-24T23:29:36.126 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.125+0000 7f3ce37fe640 1 -- 172.21.15.80:0/339848005 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) ==== 103+0+11 (secure 0 0 0) 0x7f3ce409b680 con 0x7f3cec102d80 2024-04-24T23:29:36.127 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 32 2024-04-24T23:29:36.129 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.128+0000 7f3cf0c38640 1 -- 172.21.15.80:0/339848005 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f3cd80a2b90 msgr2=0x7f3cd80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:36.129 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.128+0000 7f3cf0c38640 1 --2- 172.21.15.80:0/339848005 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f3cd80a2b90 0x7f3cd80a5050 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f3cd4005fd0 tx=0x7f3cd400b390 comp rx=0 tx=0).stop 2024-04-24T23:29:36.129 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.128+0000 7f3cf0c38640 1 -- 172.21.15.80:0/339848005 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3cec102d80 msgr2=0x7f3cec19f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:36.129 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.128+0000 7f3cf0c38640 1 --2- 172.21.15.80:0/339848005 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3cec102d80 0x7f3cec19f060 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f3ce4009fd0 tx=0x7f3ce400b260 comp rx=0 tx=0).stop 2024-04-24T23:29:36.129 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.128+0000 7f3cf0c38640 1 -- 172.21.15.80:0/339848005 shutdown_connections 2024-04-24T23:29:36.129 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.128+0000 7f3cf0c38640 1 --2- 172.21.15.80:0/339848005 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f3cd80a2b90 0x7f3cd80a5050 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:36.130 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.128+0000 7f3cf0c38640 1 --2- 172.21.15.80:0/339848005 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3cec102d80 0x7f3cec19f060 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:36.130 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.128+0000 7f3cf0c38640 1 --2- 172.21.15.80:0/339848005 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3cec1001b0 0x7f3cec197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:36.130 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.128+0000 7f3cf0c38640 1 --2- 172.21.15.80:0/339848005 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3cec0ff7f0 0x7f3cec197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:36.130 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.128+0000 7f3cf0c38640 1 -- 172.21.15.80:0/339848005 >> 172.21.15.80:0/339848005 conn(0x7f3cec0fb370 msgr2=0x7f3cec105e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:36.130 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.128+0000 7f3cf0c38640 1 -- 172.21.15.80:0/339848005 shutdown_connections 2024-04-24T23:29:36.130 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.128+0000 7f3cf0c38640 1 -- 172.21.15.80:0/339848005 wait complete. 2024-04-24T23:29:36.139 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:29:36 smithi080 ceph-mon[29152]: pgmap v368: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T23:29:36.139 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:29:36 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/1434744189' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:29:36.139 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:29:36 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/2557334701' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T23:29:36.139 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:29:36 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/1740723861' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T23:29:36.140 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:29:36.241 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.237+0000 7f352c56a640 1 -- 172.21.15.80:0/319568576 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3524100680 msgr2=0x7f3524100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:36.241 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.237+0000 7f352c56a640 1 --2- 172.21.15.80:0/319568576 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3524100680 0x7f3524100aa0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f3514009900 tx=0x7f351402a410 comp rx=0 tx=0).stop 2024-04-24T23:29:36.241 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.238+0000 7f352c56a640 1 -- 172.21.15.80:0/319568576 shutdown_connections 2024-04-24T23:29:36.241 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.238+0000 7f352c56a640 1 --2- 172.21.15.80:0/319568576 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3524102230 0x7f3524108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:36.241 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.238+0000 7f352c56a640 1 --2- 172.21.15.80:0/319568576 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3524101870 0x7f3524101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:36.241 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.238+0000 7f352c56a640 1 --2- 172.21.15.80:0/319568576 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3524100680 0x7f3524100aa0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:36.241 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.238+0000 7f352c56a640 1 -- 172.21.15.80:0/319568576 >> 172.21.15.80:0/319568576 conn(0x7f35240fbbf0 msgr2=0x7f35240fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:36.241 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.238+0000 7f352c56a640 1 -- 172.21.15.80:0/319568576 shutdown_connections 2024-04-24T23:29:36.241 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.238+0000 7f352c56a640 1 -- 172.21.15.80:0/319568576 wait complete. 2024-04-24T23:29:36.242 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.238+0000 7f352c56a640 1 Processor -- start 2024-04-24T23:29:36.242 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.238+0000 7f352c56a640 1 -- start start 2024-04-24T23:29:36.242 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.238+0000 7f352c56a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3524100680 0x7f3524069440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:36.242 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.238+0000 7f352c56a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3524101870 0x7f352419c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:36.242 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.238+0000 7f352c56a640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3524102230 0x7f352419edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:36.242 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.238+0000 7f352c56a640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f352410c2d0 con 0x7f3524102230 2024-04-24T23:29:36.242 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.238+0000 7f352c56a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f352410c150 con 0x7f3524100680 2024-04-24T23:29:36.242 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.238+0000 7f352c56a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f352410c450 con 0x7f3524101870 2024-04-24T23:29:36.242 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.238+0000 7f352ab07640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3524102230 0x7f352419edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:36.242 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.238+0000 7f352ab07640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3524102230 0x7f352419edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53810/0 (socket says 172.21.15.80:53810) 2024-04-24T23:29:36.242 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.238+0000 7f352ab07640 1 -- 172.21.15.80:0/592350597 learned_addr learned my addr 172.21.15.80:0/592350597 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:29:36.242 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.239+0000 7f3529b05640 1 --2- 172.21.15.80:0/592350597 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3524101870 0x7f352419c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:36.243 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.239+0000 7f352a306640 1 --2- 172.21.15.80:0/592350597 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3524100680 0x7f3524069440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:36.243 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.239+0000 7f352ab07640 1 -- 172.21.15.80:0/592350597 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3524101870 msgr2=0x7f352419c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:36.243 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.239+0000 7f352ab07640 1 --2- 172.21.15.80:0/592350597 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3524101870 0x7f352419c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:36.243 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.239+0000 7f352ab07640 1 -- 172.21.15.80:0/592350597 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3524100680 msgr2=0x7f3524069440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:36.243 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.239+0000 7f352ab07640 1 --2- 172.21.15.80:0/592350597 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3524100680 0x7f3524069440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:36.244 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.239+0000 7f352ab07640 1 -- 172.21.15.80:0/592350597 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f352419f6c0 con 0x7f3524102230 2024-04-24T23:29:36.244 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.239+0000 7f352a306640 1 --2- 172.21.15.80:0/592350597 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3524100680 0x7f3524069440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:29:36.244 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.239+0000 7f352ab07640 1 --2- 172.21.15.80:0/592350597 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3524102230 0x7f352419edb0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f351c00cad0 tx=0x7f351c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:36.244 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.239+0000 7f351b7fe640 1 -- 172.21.15.80:0/592350597 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f351c00ae20 con 0x7f3524102230 2024-04-24T23:29:36.244 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.239+0000 7f351b7fe640 1 -- 172.21.15.80:0/592350597 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f351c005020 con 0x7f3524102230 2024-04-24T23:29:36.244 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.239+0000 7f352c56a640 1 -- 172.21.15.80:0/592350597 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f352419f950 con 0x7f3524102230 2024-04-24T23:29:36.244 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.239+0000 7f351b7fe640 1 -- 172.21.15.80:0/592350597 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f351c00f6f0 con 0x7f3524102230 2024-04-24T23:29:36.244 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.239+0000 7f352c56a640 1 -- 172.21.15.80:0/592350597 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f352419fe50 con 0x7f3524102230 2024-04-24T23:29:36.244 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.240+0000 7f352c56a640 1 -- 172.21.15.80:0/592350597 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35241079a0 con 0x7f3524102230 2024-04-24T23:29:36.245 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.241+0000 7f351b7fe640 1 -- 172.21.15.80:0/592350597 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f351c0051d0 con 0x7f3524102230 2024-04-24T23:29:36.245 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.242+0000 7f351b7fe640 1 --2- 172.21.15.80:0/592350597 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f35100a2b90 0x7f35100a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:36.245 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.242+0000 7f352a306640 1 --2- 172.21.15.80:0/592350597 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f35100a2b90 0x7f35100a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:36.246 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.242+0000 7f351b7fe640 1 -- 172.21.15.80:0/592350597 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f351c0cf8d0 con 0x7f3524102230 2024-04-24T23:29:36.246 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.242+0000 7f352a306640 1 --2- 172.21.15.80:0/592350597 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f35100a2b90 0x7f35100a5050 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f3514000c00 tx=0x7f3514031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:36.246 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.245+0000 7f351b7fe640 1 -- 172.21.15.80:0/592350597 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f351c014030 con 0x7f3524102230 2024-04-24T23:29:36.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:29:36 smithi148 ceph-mon[32528]: pgmap v368: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T23:29:36.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:29:36 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/1434744189' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:29:36.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:29:36 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/2557334701' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T23:29:36.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:29:36 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/1740723861' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T23:29:36.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:29:36 smithi134 ceph-mon[32619]: pgmap v368: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T23:29:36.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:29:36 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/1434744189' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:29:36.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:29:36 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/2557334701' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T23:29:36.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:29:36 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/1740723861' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T23:29:36.458 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.458+0000 7f352c56a640 1 -- 172.21.15.80:0/592350597 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f352402d170 con 0x7f3524102230 2024-04-24T23:29:36.459 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.459+0000 7f351b7fe640 1 -- 172.21.15.80:0/592350597 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11872 (secure 0 0 0) 0x7f351c09c460 con 0x7f3524102230 2024-04-24T23:29:36.459 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-24T23:29:36.460 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/891591085","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":891591085},{"type":"v1","addr":"172.21.15.80:6835","nonce":891591085}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6837/259529026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6836","nonce":259529026},{"type":"v1","addr":"172.21.15.80:6837","nonce":259529026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14646,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6839/4194284854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6838","nonce":4194284854},{"type":"v1","addr":"172.21.15.80:6839","nonce":4194284854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14658,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6841/3145730958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6840","nonce":3145730958},{"type":"v1","addr":"172.21.15.80:6841","nonce":3145730958}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24395,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/3736784385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3736784385},{"type":"v1","addr":"172.21.15.134:6833","nonce":3736784385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/124777462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":124777462},{"type":"v1","addr":"172.21.15.148:6833","nonce":124777462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T23:27:19.300566+0000","modified":"2024-04-24T23:27:45.981839+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24496,"mds_1":24488,"mds_2":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/1399801120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1399801120},{"type":"v1","addr":"172.21.15.134:6835","nonce":1399801120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/430512704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":430512704},{"type":"v1","addr":"172.21.15.134:6837","nonce":430512704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/3002319310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3002319310},{"type":"v1","addr":"172.21.15.148:6835","nonce":3002319310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.148:6837/856528403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":856528403},{"type":"v1","addr":"172.21.15.148:6837","nonce":856528403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6839/1743815021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":1743815021},{"type":"v1","addr":"172.21.15.134:6839","nonce":1743815021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.148:6839/2952757594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":2952757594},{"type":"v1","addr":"172.21.15.148:6839","nonce":2952757594}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24496,"qdb_cluster":[24472,24488,24496]},"id":1}]} 2024-04-24T23:29:36.460 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 24 2024-04-24T23:29:36.461 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.460+0000 7f352c56a640 1 -- 172.21.15.80:0/592350597 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f35100a2b90 msgr2=0x7f35100a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:36.461 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.460+0000 7f352c56a640 1 --2- 172.21.15.80:0/592350597 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f35100a2b90 0x7f35100a5050 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f3514000c00 tx=0x7f3514031400 comp rx=0 tx=0).stop 2024-04-24T23:29:36.461 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.460+0000 7f352c56a640 1 -- 172.21.15.80:0/592350597 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3524102230 msgr2=0x7f352419edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:36.462 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.460+0000 7f352c56a640 1 --2- 172.21.15.80:0/592350597 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3524102230 0x7f352419edb0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f351c00cad0 tx=0x7f351c00cf90 comp rx=0 tx=0).stop 2024-04-24T23:29:36.462 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.461+0000 7f352c56a640 1 -- 172.21.15.80:0/592350597 shutdown_connections 2024-04-24T23:29:36.462 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.461+0000 7f352c56a640 1 --2- 172.21.15.80:0/592350597 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f35100a2b90 0x7f35100a5050 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:36.462 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.461+0000 7f352c56a640 1 --2- 172.21.15.80:0/592350597 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3524102230 0x7f352419edb0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:36.462 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.461+0000 7f352c56a640 1 --2- 172.21.15.80:0/592350597 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3524101870 0x7f352419c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:36.462 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.461+0000 7f352c56a640 1 --2- 172.21.15.80:0/592350597 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3524100680 0x7f3524069440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:36.462 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.461+0000 7f352c56a640 1 -- 172.21.15.80:0/592350597 >> 172.21.15.80:0/592350597 conn(0x7f35240fbbf0 msgr2=0x7f3524104b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:36.462 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.461+0000 7f352c56a640 1 -- 172.21.15.80:0/592350597 shutdown_connections 2024-04-24T23:29:36.462 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.461+0000 7f352c56a640 1 -- 172.21.15.80:0/592350597 wait complete. 2024-04-24T23:29:36.473 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:29:36.574 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.570+0000 7f09fc930640 1 -- 172.21.15.80:0/934427702 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f09f8101930 msgr2=0x7f09f8108250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:36.575 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.570+0000 7f09fc930640 1 --2- 172.21.15.80:0/934427702 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f09f8101930 0x7f09f8108250 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f09e8009900 tx=0x7f09e802a410 comp rx=0 tx=0).stop 2024-04-24T23:29:36.575 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.570+0000 7f09fc930640 1 -- 172.21.15.80:0/934427702 shutdown_connections 2024-04-24T23:29:36.575 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.570+0000 7f09fc930640 1 --2- 172.21.15.80:0/934427702 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f09f8101930 0x7f09f8108250 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:36.575 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.570+0000 7f09fc930640 1 --2- 172.21.15.80:0/934427702 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09f8100f70 0x7f09f81013f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:36.575 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.570+0000 7f09fc930640 1 --2- 172.21.15.80:0/934427702 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09f80ffe20 0x7f09f8100240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:36.575 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.570+0000 7f09fc930640 1 -- 172.21.15.80:0/934427702 >> 172.21.15.80:0/934427702 conn(0x7f09f80fb3b0 msgr2=0x7f09f80fd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:36.575 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.570+0000 7f09fc930640 1 -- 172.21.15.80:0/934427702 shutdown_connections 2024-04-24T23:29:36.575 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.570+0000 7f09fc930640 1 -- 172.21.15.80:0/934427702 wait complete. 2024-04-24T23:29:36.575 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.570+0000 7f09fc930640 1 Processor -- start 2024-04-24T23:29:36.575 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.571+0000 7f09fc930640 1 -- start start 2024-04-24T23:29:36.575 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.571+0000 7f09fc930640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09f80ffe20 0x7f09f8197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:36.575 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.571+0000 7f09fc930640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f09f8100f70 0x7f09f8197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:36.575 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.571+0000 7f09fc930640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09f8101930 0x7f09f819f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:36.575 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.571+0000 7f09fc930640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f09f810d4c0 con 0x7f09f8100f70 2024-04-24T23:29:36.576 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.571+0000 7f09fc930640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f09f810d340 con 0x7f09f8101930 2024-04-24T23:29:36.576 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.571+0000 7f09fc930640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f09f810d640 con 0x7f09f80ffe20 2024-04-24T23:29:36.576 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.571+0000 7f09f5d9b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f09f8100f70 0x7f09f8197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:36.576 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.571+0000 7f09f5d9b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f09f8100f70 0x7f09f8197fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53822/0 (socket says 172.21.15.80:53822) 2024-04-24T23:29:36.576 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.571+0000 7f09f5d9b640 1 -- 172.21.15.80:0/79943504 learned_addr learned my addr 172.21.15.80:0/79943504 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:29:36.576 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.571+0000 7f09f6d9d640 1 --2- 172.21.15.80:0/79943504 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09f8101930 0x7f09f819f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:36.576 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.571+0000 7f09f659c640 1 --2- 172.21.15.80:0/79943504 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09f80ffe20 0x7f09f8197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:36.576 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.571+0000 7f09f5d9b640 1 -- 172.21.15.80:0/79943504 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09f80ffe20 msgr2=0x7f09f8197a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:36.576 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.571+0000 7f09f5d9b640 1 --2- 172.21.15.80:0/79943504 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09f80ffe20 0x7f09f8197a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:36.576 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.571+0000 7f09f5d9b640 1 -- 172.21.15.80:0/79943504 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09f8101930 msgr2=0x7f09f819f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:36.576 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.571+0000 7f09f5d9b640 1 --2- 172.21.15.80:0/79943504 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09f8101930 0x7f09f819f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:36.576 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.571+0000 7f09f5d9b640 1 -- 172.21.15.80:0/79943504 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09f819f6a0 con 0x7f09f8100f70 2024-04-24T23:29:36.576 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.571+0000 7f09f5d9b640 1 --2- 172.21.15.80:0/79943504 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f09f8100f70 0x7f09f8197fc0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f09ec00ccb0 tx=0x7f09ec00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:36.576 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.571+0000 7f09e77fe640 1 -- 172.21.15.80:0/79943504 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09ec004b20 con 0x7f09f8100f70 2024-04-24T23:29:36.576 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.571+0000 7f09fc930640 1 -- 172.21.15.80:0/79943504 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09f819f990 con 0x7f09f8100f70 2024-04-24T23:29:36.577 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.571+0000 7f09e77fe640 1 -- 172.21.15.80:0/79943504 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09ec004cc0 con 0x7f09f8100f70 2024-04-24T23:29:36.577 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.571+0000 7f09e77fe640 1 -- 172.21.15.80:0/79943504 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09ec005020 con 0x7f09f8100f70 2024-04-24T23:29:36.577 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.572+0000 7f09fc930640 1 -- 172.21.15.80:0/79943504 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09f819ff10 con 0x7f09f8100f70 2024-04-24T23:29:36.577 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.573+0000 7f09fc930640 1 -- 172.21.15.80:0/79943504 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09f81070d0 con 0x7f09f8100f70 2024-04-24T23:29:36.577 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.574+0000 7f09e77fe640 1 -- 172.21.15.80:0/79943504 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f09ec004000 con 0x7f09f8100f70 2024-04-24T23:29:36.577 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.574+0000 7f09e77fe640 1 --2- 172.21.15.80:0/79943504 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f09e00a2b90 0x7f09e00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:36.578 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.574+0000 7f09f659c640 1 --2- 172.21.15.80:0/79943504 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f09e00a2b90 0x7f09e00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:36.578 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.574+0000 7f09e77fe640 1 -- 172.21.15.80:0/79943504 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f09ec0cf590 con 0x7f09f8100f70 2024-04-24T23:29:36.578 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.575+0000 7f09f659c640 1 --2- 172.21.15.80:0/79943504 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f09e00a2b90 0x7f09e00a5050 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f09f0009890 tx=0x7f09f0008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:36.579 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.578+0000 7f09e77fe640 1 -- 172.21.15.80:0/79943504 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f09ec014030 con 0x7f09f8100f70 2024-04-24T23:29:36.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.756+0000 7f09fc930640 1 -- 172.21.15.80:0/79943504 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f09f8062340 con 0x7f09f8100f70 2024-04-24T23:29:36.758 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.758+0000 7f09e77fe640 1 -- 172.21.15.80:0/79943504 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24839 (secure 0 0 0) 0x7f09ec09c120 con 0x7f09f8100f70 2024-04-24T23:29:36.759 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-24T23:29:36.759 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":94,"fsid":"ec7885aa-0290-11ef-bc93-c7b262605968","created":"2024-04-24T23:19:05.586407+0000","modified":"2024-04-24T23:29:08.209684+0000","last_up_change":"2024-04-24T23:26:25.938316+0000","last_in_change":"2024-04-24T23:26:04.786699+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:22:33.124428+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:27:17.575788+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:27:18.492762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T23:27:22.438503+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a525741d-b46c-4eef-aa64-3f057cac1497","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":2597718112},{"type":"v1","addr":"172.21.15.80:6803","nonce":2597718112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":2597718112},{"type":"v1","addr":"172.21.15.80:6805","nonce":2597718112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2597718112},{"type":"v1","addr":"172.21.15.80:6809","nonce":2597718112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":2597718112},{"type":"v1","addr":"172.21.15.80:6807","nonce":2597718112}]},"public_addr":"172.21.15.80:6803/2597718112","cluster_addr":"172.21.15.80:6805/2597718112","heartbeat_back_addr":"172.21.15.80:6809/2597718112","heartbeat_front_addr":"172.21.15.80:6807/2597718112","state":["exists","up"]},{"osd":1,"uuid":"2d0f0d67-6db4-40ac-91cc-58604ab6795a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":2395610201},{"type":"v1","addr":"172.21.15.134:6801","nonce":2395610201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":2395610201},{"type":"v1","addr":"172.21.15.134:6803","nonce":2395610201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":2395610201},{"type":"v1","addr":"172.21.15.134:6807","nonce":2395610201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":2395610201},{"type":"v1","addr":"172.21.15.134:6805","nonce":2395610201}]},"public_addr":"172.21.15.134:6801/2395610201","cluster_addr":"172.21.15.134:6803/2395610201","heartbeat_back_addr":"172.21.15.134:6807/2395610201","heartbeat_front_addr":"172.21.15.134:6805/2395610201","state":["exists","up"]},{"osd":2,"uuid":"781fb879-fedc-4c84-82fb-59e073bc4cf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1629152187},{"type":"v1","addr":"172.21.15.148:6801","nonce":1629152187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1629152187},{"type":"v1","addr":"172.21.15.148:6803","nonce":1629152187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1629152187},{"type":"v1","addr":"172.21.15.148:6807","nonce":1629152187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1629152187},{"type":"v1","addr":"172.21.15.148:6805","nonce":1629152187}]},"public_addr":"172.21.15.148:6801/1629152187","cluster_addr":"172.21.15.148:6803/1629152187","heartbeat_back_addr":"172.21.15.148:6807/1629152187","heartbeat_front_addr":"172.21.15.148:6805/1629152187","state":["exists","up"]},{"osd":3,"uuid":"cd982757-f2f6-4296-b3e7-4cd40fbd8d9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":3128298510},{"type":"v1","addr":"172.21.15.80:6811","nonce":3128298510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":3128298510},{"type":"v1","addr":"172.21.15.80:6813","nonce":3128298510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":3128298510},{"type":"v1","addr":"172.21.15.80:6817","nonce":3128298510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":3128298510},{"type":"v1","addr":"172.21.15.80:6815","nonce":3128298510}]},"public_addr":"172.21.15.80:6811/3128298510","cluster_addr":"172.21.15.80:6813/3128298510","heartbeat_back_addr":"172.21.15.80:6817/3128298510","heartbeat_front_addr":"172.21.15.80:6815/3128298510","state":["exists","up"]},{"osd":4,"uuid":"dfc55a79-a7e8-417a-b898-ec44171a54af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1917556060},{"type":"v1","addr":"172.21.15.134:6809","nonce":1917556060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1917556060},{"type":"v1","addr":"172.21.15.134:6811","nonce":1917556060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1917556060},{"type":"v1","addr":"172.21.15.134:6815","nonce":1917556060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1917556060},{"type":"v1","addr":"172.21.15.134:6813","nonce":1917556060}]},"public_addr":"172.21.15.134:6809/1917556060","cluster_addr":"172.21.15.134:6811/1917556060","heartbeat_back_addr":"172.21.15.134:6815/1917556060","heartbeat_front_addr":"172.21.15.134:6813/1917556060","state":["exists","up"]},{"osd":5,"uuid":"b5491337-0694-440c-b679-312f3d14cf65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":2715951281},{"type":"v1","addr":"172.21.15.148:6809","nonce":2715951281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":2715951281},{"type":"v1","addr":"172.21.15.148:6811","nonce":2715951281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":2715951281},{"type":"v1","addr":"172.21.15.148:6815","nonce":2715951281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":2715951281},{"type":"v1","addr":"172.21.15.148:6813","nonce":2715951281}]},"public_addr":"172.21.15.148:6809/2715951281","cluster_addr":"172.21.15.148:6811/2715951281","heartbeat_back_addr":"172.21.15.148:6815/2715951281","heartbeat_front_addr":"172.21.15.148:6813/2715951281","state":["exists","up"]},{"osd":6,"uuid":"ba4b5df7-d86e-47cd-ab67-cf763c4e4d6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":953261724},{"type":"v1","addr":"172.21.15.80:6819","nonce":953261724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":953261724},{"type":"v1","addr":"172.21.15.80:6821","nonce":953261724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":953261724},{"type":"v1","addr":"172.21.15.80:6825","nonce":953261724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":953261724},{"type":"v1","addr":"172.21.15.80:6823","nonce":953261724}]},"public_addr":"172.21.15.80:6819/953261724","cluster_addr":"172.21.15.80:6821/953261724","heartbeat_back_addr":"172.21.15.80:6825/953261724","heartbeat_front_addr":"172.21.15.80:6823/953261724","state":["exists","up"]},{"osd":7,"uuid":"464a503a-75ae-49e7-ab6e-a7f9f2a6a307","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":108354844},{"type":"v1","addr":"172.21.15.134:6817","nonce":108354844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":108354844},{"type":"v1","addr":"172.21.15.134:6819","nonce":108354844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":108354844},{"type":"v1","addr":"172.21.15.134:6823","nonce":108354844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":108354844},{"type":"v1","addr":"172.21.15.134:6821","nonce":108354844}]},"public_addr":"172.21.15.134:6817/108354844","cluster_addr":"172.21.15.134:6819/108354844","heartbeat_back_addr":"172.21.15.134:6823/108354844","heartbeat_front_addr":"172.21.15.134:6821/108354844","state":["exists","up"]},{"osd":8,"uuid":"ce544b2e-4540-47c0-9997-500c45351330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3559655329},{"type":"v1","addr":"172.21.15.148:6817","nonce":3559655329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":3559655329},{"type":"v1","addr":"172.21.15.148:6819","nonce":3559655329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":3559655329},{"type":"v1","addr":"172.21.15.148:6823","nonce":3559655329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3559655329},{"type":"v1","addr":"172.21.15.148:6821","nonce":3559655329}]},"public_addr":"172.21.15.148:6817/3559655329","cluster_addr":"172.21.15.148:6819/3559655329","heartbeat_back_addr":"172.21.15.148:6823/3559655329","heartbeat_front_addr":"172.21.15.148:6821/3559655329","state":["exists","up"]},{"osd":9,"uuid":"06de3732-19b9-496c-a464-55322efa6759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":4019913545},{"type":"v1","addr":"172.21.15.80:6827","nonce":4019913545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":4019913545},{"type":"v1","addr":"172.21.15.80:6829","nonce":4019913545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":4019913545},{"type":"v1","addr":"172.21.15.80:6833","nonce":4019913545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":4019913545},{"type":"v1","addr":"172.21.15.80:6831","nonce":4019913545}]},"public_addr":"172.21.15.80:6827/4019913545","cluster_addr":"172.21.15.80:6829/4019913545","heartbeat_back_addr":"172.21.15.80:6833/4019913545","heartbeat_front_addr":"172.21.15.80:6831/4019913545","state":["exists","up"]},{"osd":10,"uuid":"8d92f0d9-5c64-429a-b750-c33ae0e070c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":647343766},{"type":"v1","addr":"172.21.15.134:6825","nonce":647343766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":647343766},{"type":"v1","addr":"172.21.15.134:6827","nonce":647343766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":647343766},{"type":"v1","addr":"172.21.15.134:6831","nonce":647343766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":647343766},{"type":"v1","addr":"172.21.15.134:6829","nonce":647343766}]},"public_addr":"172.21.15.134:6825/647343766","cluster_addr":"172.21.15.134:6827/647343766","heartbeat_back_addr":"172.21.15.134:6831/647343766","heartbeat_front_addr":"172.21.15.134:6829/647343766","state":["exists","up"]},{"osd":11,"uuid":"4c003a41-f65b-479f-bc80-b27d680cf936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":4068024116},{"type":"v1","addr":"172.21.15.148:6825","nonce":4068024116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":4068024116},{"type":"v1","addr":"172.21.15.148:6827","nonce":4068024116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":4068024116},{"type":"v1","addr":"172.21.15.148:6831","nonce":4068024116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":4068024116},{"type":"v1","addr":"172.21.15.148:6829","nonce":4068024116}]},"public_addr":"172.21.15.148:6825/4068024116","cluster_addr":"172.21.15.148:6827/4068024116","heartbeat_back_addr":"172.21.15.148:6831/4068024116","heartbeat_front_addr":"172.21.15.148:6829/4068024116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:21:42.433631+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:22:06.145933+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:22:30.726143+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:22:55.729961+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:23:21.280913+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:23:46.609983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:24:12.585247+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:24:38.359287+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:25:04.491232+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:25:30.788089+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:25:56.891078+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:26:23.899113+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":7}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"4.a","mappings":[{"from":5,"to":8}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.80:0/2828378749":"2024-04-25T23:20:06.793609+0000","172.21.15.80:6801/233364790":"2024-04-25T23:20:06.793609+0000","172.21.15.80:6800/1614691410":"2024-04-25T23:19:40.490692+0000","172.21.15.80:0/1406187145":"2024-04-25T23:20:06.793609+0000","172.21.15.80:0/92953396":"2024-04-25T23:19:40.490692+0000","172.21.15.80:0/598129432":"2024-04-25T23:20:06.793609+0000","172.21.15.80:0/232193118":"2024-04-25T23:19:40.490692+0000","172.21.15.80:6800/233364790":"2024-04-25T23:20:06.793609+0000","172.21.15.80:6801/1614691410":"2024-04-25T23:19:40.490692+0000","172.21.15.80:0/2081562529":"2024-04-25T23:19:40.490692+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:29:36.761 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.759+0000 7f09fc930640 1 -- 172.21.15.80:0/79943504 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f09e00a2b90 msgr2=0x7f09e00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:36.761 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.759+0000 7f09fc930640 1 --2- 172.21.15.80:0/79943504 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f09e00a2b90 0x7f09e00a5050 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f09f0009890 tx=0x7f09f0008040 comp rx=0 tx=0).stop 2024-04-24T23:29:36.761 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.759+0000 7f09fc930640 1 -- 172.21.15.80:0/79943504 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f09f8100f70 msgr2=0x7f09f8197fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:36.761 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.760+0000 7f09fc930640 1 --2- 172.21.15.80:0/79943504 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f09f8100f70 0x7f09f8197fc0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f09ec00ccb0 tx=0x7f09ec00a620 comp rx=0 tx=0).stop 2024-04-24T23:29:36.761 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.760+0000 7f09fc930640 1 -- 172.21.15.80:0/79943504 shutdown_connections 2024-04-24T23:29:36.761 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.760+0000 7f09fc930640 1 --2- 172.21.15.80:0/79943504 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f09e00a2b90 0x7f09e00a5050 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:36.761 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.760+0000 7f09fc930640 1 --2- 172.21.15.80:0/79943504 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09f8101930 0x7f09f819f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:36.761 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.760+0000 7f09fc930640 1 --2- 172.21.15.80:0/79943504 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f09f8100f70 0x7f09f8197fc0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:36.761 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.760+0000 7f09fc930640 1 --2- 172.21.15.80:0/79943504 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09f80ffe20 0x7f09f8197a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:36.761 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.760+0000 7f09fc930640 1 -- 172.21.15.80:0/79943504 >> 172.21.15.80:0/79943504 conn(0x7f09f80fb3b0 msgr2=0x7f09f81042a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:36.761 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.760+0000 7f09fc930640 1 -- 172.21.15.80:0/79943504 shutdown_connections 2024-04-24T23:29:36.761 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.760+0000 7f09fc930640 1 -- 172.21.15.80:0/79943504 wait complete. 2024-04-24T23:29:36.773 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-24T23:27:19.300566+0000', 'modified': '2024-04-24T23:27:45.981839+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24496, 'mds_1': 24488, 'mds_2': 24472}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24428': {'gid': 24428, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.134:6835/1399801120', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6834', 'nonce': 1399801120}, {'type': 'v1', 'addr': '172.21.15.134:6835', 'nonce': 1399801120}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24458': {'gid': 24458, 'name': 'h', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.134:6837/430512704', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6836', 'nonce': 430512704}, {'type': 'v1', 'addr': '172.21.15.134:6837', 'nonce': 430512704}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24460': {'gid': 24460, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.148:6835/3002319310', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6834', 'nonce': 3002319310}, {'type': 'v1', 'addr': '172.21.15.148:6835', 'nonce': 3002319310}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24472': {'gid': 24472, 'name': 'c', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.148:6837/856528403', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6836', 'nonce': 856528403}, {'type': 'v1', 'addr': '172.21.15.148:6837', 'nonce': 856528403}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24488': {'gid': 24488, 'name': 'b', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.134:6839/1743815021', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6838', 'nonce': 1743815021}, {'type': 'v1', 'addr': '172.21.15.134:6839', 'nonce': 1743815021}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24496': {'gid': 24496, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.148:6839/2952757594', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6838', 'nonce': 2952757594}, {'type': 'v1', 'addr': '172.21.15.148:6839', 'nonce': 2952757594}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24496, 'qdb_cluster': [24472, 24488, 24496]} 2024-04-24T23:29:36.773 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-04-24T23:29:36.773 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-24T23:29:36.874 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.871+0000 7ff180c12640 1 -- 172.21.15.80:0/215638637 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff17c101870 msgr2=0x7ff17c101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:36.875 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.871+0000 7ff180c12640 1 --2- 172.21.15.80:0/215638637 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff17c101870 0x7ff17c101cf0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7ff164009900 tx=0x7ff16402a410 comp rx=0 tx=0).stop 2024-04-24T23:29:36.875 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.871+0000 7ff180c12640 1 -- 172.21.15.80:0/215638637 shutdown_connections 2024-04-24T23:29:36.875 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.871+0000 7ff180c12640 1 --2- 172.21.15.80:0/215638637 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff17c102230 0x7ff17c108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:36.875 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.871+0000 7ff180c12640 1 --2- 172.21.15.80:0/215638637 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff17c101870 0x7ff17c101cf0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:36.875 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.871+0000 7ff180c12640 1 --2- 172.21.15.80:0/215638637 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff17c100680 0x7ff17c100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:36.875 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.871+0000 7ff180c12640 1 -- 172.21.15.80:0/215638637 >> 172.21.15.80:0/215638637 conn(0x7ff17c0fbbf0 msgr2=0x7ff17c0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:36.875 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.872+0000 7ff180c12640 1 -- 172.21.15.80:0/215638637 shutdown_connections 2024-04-24T23:29:36.875 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.872+0000 7ff180c12640 1 -- 172.21.15.80:0/215638637 wait complete. 2024-04-24T23:29:36.875 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.872+0000 7ff180c12640 1 Processor -- start 2024-04-24T23:29:36.875 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.872+0000 7ff180c12640 1 -- start start 2024-04-24T23:29:36.875 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.872+0000 7ff180c12640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff17c100680 0x7ff17c069440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:36.875 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.872+0000 7ff180c12640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff17c101870 0x7ff17c19c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:36.875 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.872+0000 7ff180c12640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff17c102230 0x7ff17c19ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:36.876 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.872+0000 7ff180c12640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7ff17c10c300 con 0x7ff17c100680 2024-04-24T23:29:36.876 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.872+0000 7ff180c12640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff17c10c180 con 0x7ff17c101870 2024-04-24T23:29:36.876 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.872+0000 7ff180c12640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff17c10c480 con 0x7ff17c102230 2024-04-24T23:29:36.876 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.873+0000 7ff17a59c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff17c100680 0x7ff17c069440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:36.876 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.873+0000 7ff17a59c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff17c100680 0x7ff17c069440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53842/0 (socket says 172.21.15.80:53842) 2024-04-24T23:29:36.876 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.873+0000 7ff17a59c640 1 -- 172.21.15.80:0/2120929828 learned_addr learned my addr 172.21.15.80:0/2120929828 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:29:36.876 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.873+0000 7ff17ad9d640 1 --2- 172.21.15.80:0/2120929828 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff17c102230 0x7ff17c19ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:36.876 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.873+0000 7ff173fff640 1 --2- 172.21.15.80:0/2120929828 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff17c101870 0x7ff17c19c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:36.876 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.873+0000 7ff17a59c640 1 -- 172.21.15.80:0/2120929828 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff17c102230 msgr2=0x7ff17c19ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:36.876 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.873+0000 7ff17a59c640 1 --2- 172.21.15.80:0/2120929828 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff17c102230 0x7ff17c19ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:36.876 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.873+0000 7ff17a59c640 1 -- 172.21.15.80:0/2120929828 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff17c101870 msgr2=0x7ff17c19c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:36.877 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.873+0000 7ff17a59c640 1 --2- 172.21.15.80:0/2120929828 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff17c101870 0x7ff17c19c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:36.877 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.873+0000 7ff17a59c640 1 -- 172.21.15.80:0/2120929828 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff17c19f720 con 0x7ff17c100680 2024-04-24T23:29:36.877 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.873+0000 7ff173fff640 1 --2- 172.21.15.80:0/2120929828 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff17c101870 0x7ff17c19c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:29:36.877 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.873+0000 7ff17a59c640 1 --2- 172.21.15.80:0/2120929828 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff17c100680 0x7ff17c069440 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7ff16c00c970 tx=0x7ff16c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:36.877 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.873+0000 7ff1737fe640 1 -- 172.21.15.80:0/2120929828 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff16c004220 con 0x7ff17c100680 2024-04-24T23:29:36.877 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.873+0000 7ff1737fe640 1 -- 172.21.15.80:0/2120929828 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff16c0043c0 con 0x7ff17c100680 2024-04-24T23:29:36.877 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.873+0000 7ff180c12640 1 -- 172.21.15.80:0/2120929828 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff17c19f9b0 con 0x7ff17c100680 2024-04-24T23:29:36.877 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.873+0000 7ff1737fe640 1 -- 172.21.15.80:0/2120929828 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff16c00f450 con 0x7ff17c100680 2024-04-24T23:29:36.877 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.873+0000 7ff180c12640 1 -- 172.21.15.80:0/2120929828 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff17c19ff00 con 0x7ff17c100680 2024-04-24T23:29:36.885 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.875+0000 7ff1737fe640 1 -- 172.21.15.80:0/2120929828 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff16c0079b0 con 0x7ff17c100680 2024-04-24T23:29:36.885 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.875+0000 7ff1839f6740 1 Processor -- start 2024-04-24T23:29:36.886 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.876+0000 7ff1839f6740 1 -- start start 2024-04-24T23:29:36.886 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.876+0000 7ff1839f6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c436d64440 0x55c436d64860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:36.886 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.876+0000 7ff1839f6740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c436d64e30 0x55c436d69930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:36.886 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.876+0000 7ff1839f6740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c436d69fa0 0x55c436d6c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:36.886 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.876+0000 7ff1839f6740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x55c436cc7f00 con 0x55c436d64e30 2024-04-24T23:29:36.886 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.876+0000 7ff1839f6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c436c70750 con 0x55c436d64440 2024-04-24T23:29:36.886 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.876+0000 7ff1839f6740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55c436cc3ec0 con 0x55c436d69fa0 2024-04-24T23:29:36.886 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.876+0000 7ff1737fe640 1 --2- 172.21.15.80:0/2120929828 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7ff1680a2b90 0x7ff1680a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:36.886 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.876+0000 7ff17a59c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c436d64e30 0x55c436d69930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:36.886 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.876+0000 7ff17ad9d640 1 --2- 172.21.15.80:0/2120929828 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7ff1680a2b90 0x7ff1680a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:36.886 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.876+0000 7ff17a59c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c436d64e30 0x55c436d69930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53856/0 (socket says 172.21.15.80:53856) 2024-04-24T23:29:36.886 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.876+0000 7ff17a59c640 1 -- 172.21.15.80:0/2193634876 learned_addr learned my addr 172.21.15.80:0/2193634876 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:29:36.886 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.876+0000 7ff17ad9d640 1 --2- 172.21.15.80:0/2193634876 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c436d64440 0x55c436d64860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:36.887 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.876+0000 7ff173fff640 1 --2- 172.21.15.80:0/2193634876 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c436d69fa0 0x55c436d6c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:36.887 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.876+0000 7ff1737fe640 1 -- 172.21.15.80:0/2120929828 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7ff16c0cf250 con 0x7ff17c100680 2024-04-24T23:29:36.887 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.876+0000 7ff17a59c640 1 -- 172.21.15.80:0/2193634876 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c436d69fa0 msgr2=0x55c436d6c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:36.887 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.876+0000 7ff17a59c640 1 --2- 172.21.15.80:0/2193634876 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c436d69fa0 0x55c436d6c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:36.887 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.876+0000 7ff17a59c640 1 -- 172.21.15.80:0/2193634876 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c436d64440 msgr2=0x55c436d64860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:36.887 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.876+0000 7ff17a59c640 1 --2- 172.21.15.80:0/2193634876 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c436d64440 0x55c436d64860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:36.887 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.876+0000 7ff17a59c640 1 -- 172.21.15.80:0/2193634876 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4369dc2d0 con 0x55c436d64e30 2024-04-24T23:29:36.887 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.876+0000 7ff17ad9d640 1 --2- 172.21.15.80:0/2193634876 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c436d64440 0x55c436d64860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:29:36.887 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.877+0000 7ff17ad9d640 1 --2- 172.21.15.80:0/2120929828 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7ff1680a2b90 0x7ff1680a5050 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff174005fd0 tx=0x7ff174012a30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:36.887 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.877+0000 7ff17a59c640 1 --2- 172.21.15.80:0/2193634876 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c436d64e30 0x55c436d69930 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7ff16c0cc980 tx=0x7ff16c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b66dfe6384fb3b94 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:36.887 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.877+0000 7ff170ff9640 1 -- 172.21.15.80:0/2193634876 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff16c01d070 con 0x55c436d64e30 2024-04-24T23:29:36.887 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.877+0000 7ff170ff9640 1 -- 172.21.15.80:0/2193634876 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff16c09fa90 con 0x55c436d64e30 2024-04-24T23:29:36.887 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.877+0000 7ff170ff9640 1 -- 172.21.15.80:0/2193634876 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff16c007ca0 con 0x55c436d64e30 2024-04-24T23:29:36.887 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.877+0000 7ff1839f6740 1 -- 172.21.15.80:0/2193634876 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c436d64e30 msgr2=0x55c436d69930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:36.887 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.877+0000 7ff1839f6740 1 --2- 172.21.15.80:0/2193634876 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c436d64e30 0x55c436d69930 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7ff16c0cc980 tx=0x7ff16c000bc0 comp rx=0 tx=0).stop 2024-04-24T23:29:36.888 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.877+0000 7ff1839f6740 1 -- 172.21.15.80:0/2193634876 shutdown_connections 2024-04-24T23:29:36.888 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.877+0000 7ff1839f6740 1 --2- 172.21.15.80:0/2193634876 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c436d69fa0 0x55c436d6c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:36.888 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.877+0000 7ff1839f6740 1 --2- 172.21.15.80:0/2193634876 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c436d64e30 0x55c436d69930 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:36.888 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.877+0000 7ff1839f6740 1 --2- 172.21.15.80:0/2193634876 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c436d64440 0x55c436d64860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:36.888 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.878+0000 7ff1839f6740 1 -- 172.21.15.80:0/2193634876 >> 172.21.15.80:0/2193634876 conn(0x55c436d5fab0 msgr2=0x55c436d5feb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:36.888 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.878+0000 7ff1839f6740 1 -- 172.21.15.80:0/2193634876 shutdown_connections 2024-04-24T23:29:36.888 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.878+0000 7ff1839f6740 1 -- 172.21.15.80:0/2193634876 wait complete. 2024-04-24T23:29:36.888 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.878+0000 7ff1839f6740 1 -- start start 2024-04-24T23:29:36.888 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.878+0000 7ff1839f6740 1 Processor -- start 2024-04-24T23:29:36.888 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.878+0000 7ff1839f6740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c436d64440 0x55c436d6e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:36.888 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.878+0000 7ff1839f6740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c436d64e30 0x55c436d6ec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:36.889 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.878+0000 7ff1839f6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c436d69fa0 0x55c436d75cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:36.889 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.878+0000 7ff1839f6740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x55c436d6ce90 con 0x55c436d64440 2024-04-24T23:29:36.890 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.878+0000 7ff1839f6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c436c83cf0 con 0x55c436d69fa0 2024-04-24T23:29:36.890 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.878+0000 7ff1839f6740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55c436d6d010 con 0x55c436d64e30 2024-04-24T23:29:36.890 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.878+0000 7ff1839f6740 10 client.0 _ll_register_callbacks cb 0x55c436c836e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:29:36.890 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.878+0000 7ff17a59c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c436d64440 0x55c436d6e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:36.890 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.878+0000 7ff17a59c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c436d64440 0x55c436d6e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53870/0 (socket says 172.21.15.80:53870) 2024-04-24T23:29:36.890 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.878+0000 7ff17a59c640 1 -- 172.21.15.80:0/4180255636 learned_addr learned my addr 172.21.15.80:0/4180255636 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:29:36.890 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.878+0000 7ff173fff640 1 --2- 172.21.15.80:0/4180255636 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c436d64e30 0x55c436d6ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:36.890 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.878+0000 7ff17a59c640 1 -- 172.21.15.80:0/4180255636 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c436d64e30 msgr2=0x55c436d6ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:36.890 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.879+0000 7ff17a59c640 1 --2- 172.21.15.80:0/4180255636 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c436d64e30 0x55c436d6ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:36.890 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.879+0000 7ff17a59c640 1 -- 172.21.15.80:0/4180255636 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c436d69fa0 msgr2=0x55c436d75cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:36.890 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.879+0000 7ff17a59c640 1 --2- 172.21.15.80:0/4180255636 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c436d69fa0 0x55c436d75cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:36.890 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.879+0000 7ff17a59c640 1 -- 172.21.15.80:0/4180255636 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c436aa66b0 con 0x55c436d64440 2024-04-24T23:29:36.891 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.879+0000 7ff173fff640 1 --2- 172.21.15.80:0/4180255636 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c436d64e30 0x55c436d6ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:29:36.891 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.879+0000 7ff17a59c640 1 --2- 172.21.15.80:0/4180255636 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c436d64440 0x55c436d6e730 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7ff16c097b40 tx=0x7ff16c004570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60f1b8783746665f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:36.891 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.879+0000 7ff161ffb640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-04-24T23:29:36.891 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.879+0000 7ff161ffb640 1 -- 172.21.15.80:0/4180255636 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff16c01d040 con 0x55c436d64440 2024-04-24T23:29:36.891 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.879+0000 7ff161ffb640 1 -- 172.21.15.80:0/4180255636 <== mon.0 v2:172.21.15.80:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7ff16c09ad30 con 0x55c436d64440 2024-04-24T23:29:36.891 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.879+0000 7ff1839f6740 1 -- 172.21.15.80:0/4180255636 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff16c09fa90 con 0x55c436d64440 2024-04-24T23:29:36.891 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.879+0000 7ff161ffb640 1 -- 172.21.15.80:0/4180255636 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7ff16c0860e0 con 0x55c436d64440 2024-04-24T23:29:36.891 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.879+0000 7ff1839f6740 10 client.15423 fetch_fsmap learned FSMap version 24 2024-04-24T23:29:36.891 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.879+0000 7ff1839f6740 1 -- 172.21.15.80:0/4180255636 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55c436d78c90 con 0x55c436d64440 2024-04-24T23:29:36.891 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.879+0000 7ff161ffb640 1 -- 172.21.15.80:0/4180255636 <== mon.0 v2:172.21.15.80:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff16c09fa90 con 0x55c436d64440 2024-04-24T23:29:36.891 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.879+0000 7ff161ffb640 1 -- 172.21.15.80:0/4180255636 <== mon.0 v2:172.21.15.80:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7ff16c09a7a0 con 0x55c436d64440 2024-04-24T23:29:36.891 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.879+0000 7ff1839f6740 10 client.15423 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T23:29:36.891 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.879+0000 7ff1839f6740 10 client.15423 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-24T23:29:36.891 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.879+0000 7ff1839f6740 20 client.15423 populate_metadata read hostname 'smithi080' 2024-04-24T23:29:36.892 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.879+0000 7ff1839f6740 1 --2- 172.21.15.80:0/4180255636 >> [v2:172.21.15.148:6836/856528403,v1:172.21.15.148:6837/856528403] conn(0x55c436d7ae80 0x55c436d7c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:36.892 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.879+0000 7ff1839f6740 4 client.15423 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:29:36.892 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.879+0000 7ff1839f6740 1 -- 172.21.15.80:0/4180255636 --> [v2:172.21.15.148:6836/856528403,v1:172.21.15.148:6837/856528403] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c436cd5440 con 0x55c436d7ae80 2024-04-24T23:29:36.892 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.880+0000 7ff173fff640 1 --2- 172.21.15.80:0/4180255636 >> [v2:172.21.15.148:6836/856528403,v1:172.21.15.148:6837/856528403] conn(0x55c436d7ae80 0x55c436d7c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:36.892 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.880+0000 7ff173fff640 1 --2- 172.21.15.80:0/4180255636 >> [v2:172.21.15.148:6836/856528403,v1:172.21.15.148:6837/856528403] conn(0x55c436d7ae80 0x55c436d7c2f0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7e393ac745a0dde3 server_cookie=d9ae35369199b507 in_seq=0 out_seq=0 2024-04-24T23:29:36.892 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.880+0000 7ff161ffb640 10 client.15423 ms_handle_connect on v2:172.21.15.148:6836/856528403 2024-04-24T23:29:36.892 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.881+0000 7ff161ffb640 1 -- 172.21.15.80:0/4180255636 <== mds.2 v2:172.21.15.148:6836/856528403 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c436cd5440 con 0x55c436d7ae80 2024-04-24T23:29:36.892 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.881+0000 7ff161ffb640 10 client.15423 handle_command_reply: tid=0 2024-04-24T23:29:36.892 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.881+0000 7ff1839f6740 1 client.15423 shutdown 2024-04-24T23:29:36.892 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.881+0000 7ff1839f6740 1 -- 172.21.15.80:0/4180255636 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c436d64440 msgr2=0x55c436d6e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:36.892 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.881+0000 7ff1839f6740 1 --2- 172.21.15.80:0/4180255636 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c436d64440 0x55c436d6e730 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7ff16c097b40 tx=0x7ff16c004570 comp rx=0 tx=0).stop 2024-04-24T23:29:36.892 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.881+0000 7ff1839f6740 1 -- 172.21.15.80:0/4180255636 shutdown_connections 2024-04-24T23:29:36.892 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.881+0000 7ff1839f6740 1 --2- 172.21.15.80:0/4180255636 >> [v2:172.21.15.148:6836/856528403,v1:172.21.15.148:6837/856528403] conn(0x55c436d7ae80 0x55c436d7c2f0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:36.892 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.881+0000 7ff1839f6740 1 --2- 172.21.15.80:0/4180255636 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c436d69fa0 0x55c436d75cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:36.893 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.881+0000 7ff161ffb640 0 client.15423 ms_handle_reset on v2:172.21.15.148:6836/856528403 2024-04-24T23:29:36.893 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.881+0000 7ff1839f6740 1 --2- 172.21.15.80:0/4180255636 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c436d64e30 0x55c436d6ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:36.893 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.881+0000 7ff1839f6740 1 --2- 172.21.15.80:0/4180255636 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c436d64440 0x55c436d6e730 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:36.893 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.881+0000 7ff1839f6740 1 -- 172.21.15.80:0/4180255636 >> 172.21.15.80:0/4180255636 conn(0x55c436d5fab0 msgr2=0x55c436d5eae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:36.893 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.881+0000 7ff1839f6740 1 -- 172.21.15.80:0/4180255636 shutdown_connections 2024-04-24T23:29:36.893 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.881+0000 7ff1839f6740 1 -- 172.21.15.80:0/4180255636 wait complete. 2024-04-24T23:29:36.893 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.881+0000 7ff1839f6740 20 client.15423 trim_cache size 0 max 16384 2024-04-24T23:29:36.907 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.899+0000 7ff1839f6740 1 Processor -- start 2024-04-24T23:29:36.907 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.899+0000 7ff1839f6740 1 -- start start 2024-04-24T23:29:36.907 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.899+0000 7ff1839f6740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c436d64440 0x55c436d656c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:36.907 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.899+0000 7ff1839f6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c436d64e30 0x55c436d75c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:36.907 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.899+0000 7ff1839f6740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c436d69fa0 0x55c436d6e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:36.907 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.899+0000 7ff1839f6740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x55c436d7a630 con 0x55c436d64440 2024-04-24T23:29:36.907 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.899+0000 7ff1839f6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c436d7ad00 con 0x55c436d64e30 2024-04-24T23:29:36.907 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.899+0000 7ff1839f6740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55c436d7b3d0 con 0x55c436d69fa0 2024-04-24T23:29:36.907 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.899+0000 7ff17a59c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c436d64440 0x55c436d656c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:36.908 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.899+0000 7ff17a59c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c436d64440 0x55c436d656c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53872/0 (socket says 172.21.15.80:53872) 2024-04-24T23:29:36.909 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.899+0000 7ff17a59c640 1 -- 172.21.15.80:0/303504546 learned_addr learned my addr 172.21.15.80:0/303504546 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:29:36.909 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.900+0000 7ff17a59c640 1 -- 172.21.15.80:0/303504546 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c436d69fa0 msgr2=0x55c436d6e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:36.909 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.900+0000 7ff17ad9d640 1 --2- 172.21.15.80:0/303504546 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c436d69fa0 0x55c436d6e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:36.909 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.900+0000 7ff173fff640 1 --2- 172.21.15.80:0/303504546 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c436d64e30 0x55c436d75c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:36.909 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.900+0000 7ff17a59c640 1 --2- 172.21.15.80:0/303504546 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c436d69fa0 0x55c436d6e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:36.909 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.900+0000 7ff17a59c640 1 -- 172.21.15.80:0/303504546 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c436d64e30 msgr2=0x55c436d75c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:36.909 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.900+0000 7ff17a59c640 1 --2- 172.21.15.80:0/303504546 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c436d64e30 0x55c436d75c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:36.909 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.900+0000 7ff17a59c640 1 -- 172.21.15.80:0/303504546 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c436d7bf50 con 0x55c436d64440 2024-04-24T23:29:36.909 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.900+0000 7ff17ad9d640 1 --2- 172.21.15.80:0/303504546 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c436d69fa0 0x55c436d6e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:29:36.909 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.900+0000 7ff173fff640 1 --2- 172.21.15.80:0/303504546 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c436d64e30 0x55c436d75c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:29:36.910 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.900+0000 7ff17a59c640 1 --2- 172.21.15.80:0/303504546 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c436d64440 0x55c436d656c0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7ff16c007590 tx=0x7ff16c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62098213b45140f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:36.910 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.900+0000 7ff163fff640 1 -- 172.21.15.80:0/303504546 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff16c01d070 con 0x55c436d64440 2024-04-24T23:29:36.910 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.900+0000 7ff163fff640 1 -- 172.21.15.80:0/303504546 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff16c09fa90 con 0x55c436d64440 2024-04-24T23:29:36.910 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.900+0000 7ff163fff640 1 -- 172.21.15.80:0/303504546 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff16c0bb740 con 0x55c436d64440 2024-04-24T23:29:36.910 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.900+0000 7ff1839f6740 1 -- 172.21.15.80:0/303504546 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c436d64440 msgr2=0x55c436d656c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:36.910 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.900+0000 7ff1839f6740 1 --2- 172.21.15.80:0/303504546 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c436d64440 0x55c436d656c0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7ff16c007590 tx=0x7ff16c000bc0 comp rx=0 tx=0).stop 2024-04-24T23:29:36.910 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.901+0000 7ff1839f6740 1 -- 172.21.15.80:0/303504546 shutdown_connections 2024-04-24T23:29:36.910 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.901+0000 7ff1839f6740 1 --2- 172.21.15.80:0/303504546 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c436d69fa0 0x55c436d6e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:36.910 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.901+0000 7ff1839f6740 1 --2- 172.21.15.80:0/303504546 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c436d64e30 0x55c436d75c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:36.910 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.901+0000 7ff1839f6740 1 --2- 172.21.15.80:0/303504546 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c436d64440 0x55c436d656c0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:36.910 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.901+0000 7ff1839f6740 1 -- 172.21.15.80:0/303504546 >> 172.21.15.80:0/303504546 conn(0x55c436d5fab0 msgr2=0x55c436d5a690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:36.910 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.901+0000 7ff1839f6740 1 -- 172.21.15.80:0/303504546 shutdown_connections 2024-04-24T23:29:36.910 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.901+0000 7ff1839f6740 1 -- 172.21.15.80:0/303504546 wait complete. 2024-04-24T23:29:36.910 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.901+0000 7ff1839f6740 1 -- start start 2024-04-24T23:29:36.911 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.901+0000 7ff1839f6740 1 Processor -- start 2024-04-24T23:29:36.911 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.901+0000 7ff1839f6740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c436d64440 0x55c436d71e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:36.911 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.901+0000 7ff1839f6740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c436d64e30 0x55c436d723b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:36.911 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.901+0000 7ff1839f6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c436d69fa0 0x55c436d74e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:36.911 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.901+0000 7ff1839f6740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x55c436d7c170 con 0x55c436d64e30 2024-04-24T23:29:36.911 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.901+0000 7ff1839f6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c436d7baa0 con 0x55c436d69fa0 2024-04-24T23:29:36.911 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.901+0000 7ff1839f6740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55c436d7c840 con 0x55c436d64440 2024-04-24T23:29:36.911 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.901+0000 7ff1839f6740 10 client.0 _ll_register_callbacks cb 0x7ff16c0972c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:29:36.912 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.901+0000 7ff173fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c436d64e30 0x55c436d723b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:36.912 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.901+0000 7ff173fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c436d64e30 0x55c436d723b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53880/0 (socket says 172.21.15.80:53880) 2024-04-24T23:29:36.912 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.901+0000 7ff173fff640 1 -- 172.21.15.80:0/1100331892 learned_addr learned my addr 172.21.15.80:0/1100331892 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:29:36.912 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.901+0000 7ff17a59c640 1 --2- 172.21.15.80:0/1100331892 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c436d64440 0x55c436d71e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:36.912 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.901+0000 7ff173fff640 1 -- 172.21.15.80:0/1100331892 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c436d64440 msgr2=0x55c436d71e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:36.912 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.901+0000 7ff173fff640 1 --2- 172.21.15.80:0/1100331892 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c436d64440 0x55c436d71e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:36.912 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.901+0000 7ff173fff640 1 -- 172.21.15.80:0/1100331892 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c436d69fa0 msgr2=0x55c436d74e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:36.912 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.901+0000 7ff173fff640 1 --2- 172.21.15.80:0/1100331892 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c436d69fa0 0x55c436d74e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:36.912 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.901+0000 7ff173fff640 1 -- 172.21.15.80:0/1100331892 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c436d75430 con 0x55c436d64e30 2024-04-24T23:29:36.912 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.902+0000 7ff173fff640 1 --2- 172.21.15.80:0/1100331892 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c436d64e30 0x55c436d723b0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7ff1640314e0 tx=0x7ff164009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed25391298fabc3c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:36.912 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.902+0000 7ff161ffb640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-04-24T23:29:36.912 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.902+0000 7ff161ffb640 1 -- 172.21.15.80:0/1100331892 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff164035040 con 0x55c436d64e30 2024-04-24T23:29:36.912 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.902+0000 7ff161ffb640 1 -- 172.21.15.80:0/1100331892 <== mon.0 v2:172.21.15.80:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7ff164042050 con 0x55c436d64e30 2024-04-24T23:29:36.912 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.902+0000 7ff1839f6740 1 -- 172.21.15.80:0/1100331892 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff16c09fa90 con 0x55c436d64e30 2024-04-24T23:29:36.913 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.902+0000 7ff161ffb640 1 -- 172.21.15.80:0/1100331892 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7ff16404f050 con 0x55c436d64e30 2024-04-24T23:29:36.913 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.902+0000 7ff1839f6740 10 client.15429 fetch_fsmap learned FSMap version 24 2024-04-24T23:29:36.913 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.902+0000 7ff1839f6740 1 -- 172.21.15.80:0/1100331892 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55c436de1350 con 0x55c436d64e30 2024-04-24T23:29:36.913 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.902+0000 7ff161ffb640 1 -- 172.21.15.80:0/1100331892 <== mon.0 v2:172.21.15.80:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff16c09fa90 con 0x55c436d64e30 2024-04-24T23:29:36.913 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.902+0000 7ff161ffb640 1 -- 172.21.15.80:0/1100331892 <== mon.0 v2:172.21.15.80:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7ff164031a80 con 0x55c436d64e30 2024-04-24T23:29:36.913 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.902+0000 7ff1839f6740 10 client.15429 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T23:29:36.913 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.902+0000 7ff1839f6740 10 client.15429 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-24T23:29:36.913 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.902+0000 7ff1839f6740 20 client.15429 populate_metadata read hostname 'smithi080' 2024-04-24T23:29:36.913 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.902+0000 7ff1839f6740 1 --2- 172.21.15.80:0/1100331892 >> [v2:172.21.15.148:6836/856528403,v1:172.21.15.148:6837/856528403] conn(0x55c436de6000 0x55c436de2440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:36.913 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.902+0000 7ff1839f6740 4 client.15429 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T23:29:36.913 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.902+0000 7ff1839f6740 1 -- 172.21.15.80:0/1100331892 --> [v2:172.21.15.148:6836/856528403,v1:172.21.15.148:6837/856528403] -- command(tid 0: {"prefix": "status"}) -- 0x55c436ccf790 con 0x55c436de6000 2024-04-24T23:29:36.913 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.903+0000 7ff17a59c640 1 --2- 172.21.15.80:0/1100331892 >> [v2:172.21.15.148:6836/856528403,v1:172.21.15.148:6837/856528403] conn(0x55c436de6000 0x55c436de2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:36.913 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.903+0000 7ff161ffb640 10 client.15429 ms_handle_connect on v2:172.21.15.148:6836/856528403 2024-04-24T23:29:36.913 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.903+0000 7ff17a59c640 1 --2- 172.21.15.80:0/1100331892 >> [v2:172.21.15.148:6836/856528403,v1:172.21.15.148:6837/856528403] conn(0x55c436de6000 0x55c436de2440 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8543dc2dbf4b41e4 server_cookie=84e129bfa17563b3 in_seq=0 out_seq=0 2024-04-24T23:29:36.914 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.903+0000 7ff161ffb640 1 -- 172.21.15.80:0/1100331892 <== mds.2 v2:172.21.15.148:6836/856528403 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x55c436ccf790 con 0x55c436de6000 2024-04-24T23:29:36.914 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.903+0000 7ff161ffb640 10 client.15429 handle_command_reply: tid=0 2024-04-24T23:29:36.914 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.903+0000 7ff1839f6740 1 client.15429 shutdown 2024-04-24T23:29:36.914 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.903+0000 7ff1839f6740 1 -- 172.21.15.80:0/1100331892 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c436d64e30 msgr2=0x55c436d723b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:36.914 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.903+0000 7ff1839f6740 1 --2- 172.21.15.80:0/1100331892 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c436d64e30 0x55c436d723b0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7ff1640314e0 tx=0x7ff164009680 comp rx=0 tx=0).stop 2024-04-24T23:29:36.914 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.903+0000 7ff1839f6740 1 -- 172.21.15.80:0/1100331892 shutdown_connections 2024-04-24T23:29:36.914 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.903+0000 7ff1839f6740 1 --2- 172.21.15.80:0/1100331892 >> [v2:172.21.15.148:6836/856528403,v1:172.21.15.148:6837/856528403] conn(0x55c436de6000 0x55c436de2440 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:36.914 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.903+0000 7ff1839f6740 1 --2- 172.21.15.80:0/1100331892 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c436d69fa0 0x55c436d74e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:36.914 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.903+0000 7ff1839f6740 1 --2- 172.21.15.80:0/1100331892 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c436d64e30 0x55c436d723b0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:36.914 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.903+0000 7ff1839f6740 1 --2- 172.21.15.80:0/1100331892 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c436d64440 0x55c436d71e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:36.914 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.903+0000 7ff161ffb640 0 client.15429 ms_handle_reset on v2:172.21.15.148:6836/856528403 2024-04-24T23:29:36.914 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.903+0000 7ff1839f6740 1 -- 172.21.15.80:0/1100331892 >> 172.21.15.80:0/1100331892 conn(0x55c436d5fab0 msgr2=0x55c436d656c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:36.914 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.903+0000 7ff1839f6740 1 -- 172.21.15.80:0/1100331892 shutdown_connections 2024-04-24T23:29:36.915 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.903+0000 7ff1839f6740 1 -- 172.21.15.80:0/1100331892 wait complete. 2024-04-24T23:29:36.915 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.903+0000 7ff1839f6740 20 client.15429 trim_cache size 0 max 16384 2024-04-24T23:29:36.915 INFO:teuthology.orchestra.run.smithi080.stdout:{ 2024-04-24T23:29:36.915 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_fsid": "ec7885aa-0290-11ef-bc93-c7b262605968", 2024-04-24T23:29:36.915 INFO:teuthology.orchestra.run.smithi080.stdout: "whoami": 2, 2024-04-24T23:29:36.915 INFO:teuthology.orchestra.run.smithi080.stdout: "id": 24472, 2024-04-24T23:29:36.915 INFO:teuthology.orchestra.run.smithi080.stdout: "want_state": "up:active", 2024-04-24T23:29:36.915 INFO:teuthology.orchestra.run.smithi080.stdout: "state": "up:active", 2024-04-24T23:29:36.915 INFO:teuthology.orchestra.run.smithi080.stdout: "fs_name": "cephfs", 2024-04-24T23:29:36.916 INFO:teuthology.orchestra.run.smithi080.stdout: "rank_uptime": 111.92642411, 2024-04-24T23:29:36.916 INFO:teuthology.orchestra.run.smithi080.stdout: "mdsmap_epoch": 24, 2024-04-24T23:29:36.916 INFO:teuthology.orchestra.run.smithi080.stdout: "osdmap_epoch": 94, 2024-04-24T23:29:36.916 INFO:teuthology.orchestra.run.smithi080.stdout: "osdmap_epoch_barrier": 90, 2024-04-24T23:29:36.916 INFO:teuthology.orchestra.run.smithi080.stdout: "uptime": 158.13972223299999 2024-04-24T23:29:36.916 INFO:teuthology.orchestra.run.smithi080.stdout:} 2024-04-24T23:29:36.917 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.904+0000 7ff180c12640 1 -- 172.21.15.80:0/2120929828 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7ff1680a2b90 msgr2=0x7ff1680a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:36.917 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.904+0000 7ff180c12640 1 --2- 172.21.15.80:0/2120929828 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7ff1680a2b90 0x7ff1680a5050 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff174005fd0 tx=0x7ff174012a30 comp rx=0 tx=0).stop 2024-04-24T23:29:36.917 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.904+0000 7ff180c12640 1 -- 172.21.15.80:0/2120929828 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff17c100680 msgr2=0x7ff17c069440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:36.917 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.904+0000 7ff180c12640 1 --2- 172.21.15.80:0/2120929828 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff17c100680 0x7ff17c069440 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7ff16c00c970 tx=0x7ff16c00ce30 comp rx=0 tx=0).stop 2024-04-24T23:29:36.917 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.904+0000 7ff180c12640 1 -- 172.21.15.80:0/2120929828 shutdown_connections 2024-04-24T23:29:36.918 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.904+0000 7ff180c12640 1 --2- 172.21.15.80:0/2120929828 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7ff1680a2b90 0x7ff1680a5050 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:36.918 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.904+0000 7ff180c12640 1 --2- 172.21.15.80:0/2120929828 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff17c102230 0x7ff17c19ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:36.918 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.904+0000 7ff180c12640 1 --2- 172.21.15.80:0/2120929828 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff17c101870 0x7ff17c19c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:36.918 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.904+0000 7ff180c12640 1 --2- 172.21.15.80:0/2120929828 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff17c100680 0x7ff17c069440 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:36.918 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.904+0000 7ff180c12640 1 -- 172.21.15.80:0/2120929828 >> 172.21.15.80:0/2120929828 conn(0x7ff17c0fbbf0 msgr2=0x7ff17c104b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:36.918 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.904+0000 7ff180c12640 1 -- 172.21.15.80:0/2120929828 shutdown_connections 2024-04-24T23:29:36.918 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:36.904+0000 7ff180c12640 1 -- 172.21.15.80:0/2120929828 wait complete. 2024-04-24T23:29:36.918 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-24T23:29:37.021 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.018+0000 7fd0f0fa2640 1 -- 172.21.15.80:0/3622337072 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd0ec1033e0 msgr2=0x7fd0ec109c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:37.021 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.018+0000 7fd0f0fa2640 1 --2- 172.21.15.80:0/3622337072 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd0ec1033e0 0x7fd0ec109c70 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fd0dc009930 tx=0x7fd0dc02a420 comp rx=0 tx=0).stop 2024-04-24T23:29:37.021 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.018+0000 7fd0f0fa2640 1 -- 172.21.15.80:0/3622337072 shutdown_connections 2024-04-24T23:29:37.021 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.018+0000 7fd0f0fa2640 1 --2- 172.21.15.80:0/3622337072 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd0ec1033e0 0x7fd0ec109c70 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:37.021 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.018+0000 7fd0f0fa2640 1 --2- 172.21.15.80:0/3622337072 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd0ec102a20 0x7fd0ec102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:37.021 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.018+0000 7fd0f0fa2640 1 --2- 172.21.15.80:0/3622337072 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0ec0ffa50 0x7fd0ec0ffe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:37.021 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.018+0000 7fd0f0fa2640 1 -- 172.21.15.80:0/3622337072 >> 172.21.15.80:0/3622337072 conn(0x7fd0ec0fb380 msgr2=0x7fd0ec0fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:37.021 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.018+0000 7fd0f0fa2640 1 -- 172.21.15.80:0/3622337072 shutdown_connections 2024-04-24T23:29:37.021 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.018+0000 7fd0f0fa2640 1 -- 172.21.15.80:0/3622337072 wait complete. 2024-04-24T23:29:37.021 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.018+0000 7fd0f0fa2640 1 Processor -- start 2024-04-24T23:29:37.021 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.019+0000 7fd0f0fa2640 1 -- start start 2024-04-24T23:29:37.022 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.019+0000 7fd0f0fa2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0ec0ffa50 0x7fd0ec197a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:37.022 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.019+0000 7fd0f0fa2640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd0ec102a20 0x7fd0ec197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:37.022 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.019+0000 7fd0f0fa2640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd0ec1033e0 0x7fd0ec19eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:37.022 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.019+0000 7fd0f0fa2640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fd0ec10d400 con 0x7fd0ec102a20 2024-04-24T23:29:37.022 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.019+0000 7fd0f0fa2640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd0ec10d280 con 0x7fd0ec0ffa50 2024-04-24T23:29:37.022 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.019+0000 7fd0f0fa2640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd0ec10d580 con 0x7fd0ec1033e0 2024-04-24T23:29:37.022 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.019+0000 7fd0e9d9b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd0ec102a20 0x7fd0ec197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:37.022 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.019+0000 7fd0e9d9b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd0ec102a20 0x7fd0ec197f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53892/0 (socket says 172.21.15.80:53892) 2024-04-24T23:29:37.022 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.019+0000 7fd0e9d9b640 1 -- 172.21.15.80:0/3637959638 learned_addr learned my addr 172.21.15.80:0/3637959638 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:29:37.022 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.019+0000 7fd0ea59c640 1 --2- 172.21.15.80:0/3637959638 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0ec0ffa50 0x7fd0ec197a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:37.022 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.019+0000 7fd0ead9d640 1 --2- 172.21.15.80:0/3637959638 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd0ec1033e0 0x7fd0ec19eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:37.022 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.019+0000 7fd0e9d9b640 1 -- 172.21.15.80:0/3637959638 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd0ec1033e0 msgr2=0x7fd0ec19eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:37.022 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.019+0000 7fd0e9d9b640 1 --2- 172.21.15.80:0/3637959638 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd0ec1033e0 0x7fd0ec19eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:37.022 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.019+0000 7fd0e9d9b640 1 -- 172.21.15.80:0/3637959638 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0ec0ffa50 msgr2=0x7fd0ec197a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:37.023 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.019+0000 7fd0e9d9b640 1 --2- 172.21.15.80:0/3637959638 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0ec0ffa50 0x7fd0ec197a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:37.023 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.019+0000 7fd0e9d9b640 1 -- 172.21.15.80:0/3637959638 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0ec19f660 con 0x7fd0ec102a20 2024-04-24T23:29:37.023 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.019+0000 7fd0ea59c640 1 --2- 172.21.15.80:0/3637959638 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0ec0ffa50 0x7fd0ec197a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:29:37.023 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.019+0000 7fd0e9d9b640 1 --2- 172.21.15.80:0/3637959638 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd0ec102a20 0x7fd0ec197f80 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fd0d400d950 tx=0x7fd0d400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:37.023 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.019+0000 7fd0e37fe640 1 -- 172.21.15.80:0/3637959638 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0d4017030 con 0x7fd0ec102a20 2024-04-24T23:29:37.023 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.019+0000 7fd0f0fa2640 1 -- 172.21.15.80:0/3637959638 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0ec19f950 con 0x7fd0ec102a20 2024-04-24T23:29:37.023 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.019+0000 7fd0e37fe640 1 -- 172.21.15.80:0/3637959638 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0d4004020 con 0x7fd0ec102a20 2024-04-24T23:29:37.023 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.019+0000 7fd0e37fe640 1 -- 172.21.15.80:0/3637959638 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0d40047a0 con 0x7fd0ec102a20 2024-04-24T23:29:37.023 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.020+0000 7fd0f0fa2640 1 -- 172.21.15.80:0/3637959638 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0ec19fea0 con 0x7fd0ec102a20 2024-04-24T23:29:37.030 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.021+0000 7fd0e37fe640 1 -- 172.21.15.80:0/3637959638 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd0d40041d0 con 0x7fd0ec102a20 2024-04-24T23:29:37.030 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.022+0000 7fd0f3d90740 1 Processor -- start 2024-04-24T23:29:37.030 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.022+0000 7fd0f3d90740 1 -- start start 2024-04-24T23:29:37.030 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.022+0000 7fd0f3d90740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5609c4f42440 0x5609c4f42860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:37.030 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.022+0000 7fd0f3d90740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5609c4f42e30 0x5609c4f47930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:37.030 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.022+0000 7fd0f3d90740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5609c4f47fa0 0x5609c4f4a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:37.030 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.022+0000 7fd0f3d90740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x5609c4ea5f00 con 0x5609c4f47fa0 2024-04-24T23:29:37.030 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.022+0000 7fd0f3d90740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5609c4e4e750 con 0x5609c4f42440 2024-04-24T23:29:37.030 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.022+0000 7fd0f3d90740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5609c4ea1ec0 con 0x5609c4f42e30 2024-04-24T23:29:37.030 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.022+0000 7fd0e37fe640 1 --2- 172.21.15.80:0/3637959638 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fd0d80a2b90 0x7fd0d80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:37.030 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.022+0000 7fd0e9d9b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5609c4f47fa0 0x5609c4f4a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:37.031 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.022+0000 7fd0ead9d640 1 --2- 172.21.15.80:0/3637959638 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fd0d80a2b90 0x7fd0d80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:37.031 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.022+0000 7fd0ea59c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5609c4f42e30 0x5609c4f47930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:37.031 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.022+0000 7fd0e9d9b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5609c4f47fa0 0x5609c4f4a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53900/0 (socket says 172.21.15.80:53900) 2024-04-24T23:29:37.031 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.022+0000 7fd0e9d9b640 1 -- 172.21.15.80:0/256551940 learned_addr learned my addr 172.21.15.80:0/256551940 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:29:37.031 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.023+0000 7fd0e37fe640 1 -- 172.21.15.80:0/3637959638 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fd0d40ced80 con 0x7fd0ec102a20 2024-04-24T23:29:37.031 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.023+0000 7fd0ead9d640 1 --2- 172.21.15.80:0/256551940 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5609c4f42440 0x5609c4f42860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:37.031 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.023+0000 7fd0e9d9b640 1 -- 172.21.15.80:0/256551940 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5609c4f42e30 msgr2=0x5609c4f47930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:37.031 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.023+0000 7fd0e9d9b640 1 --2- 172.21.15.80:0/256551940 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5609c4f42e30 0x5609c4f47930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:37.031 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.023+0000 7fd0e9d9b640 1 -- 172.21.15.80:0/256551940 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5609c4f42440 msgr2=0x5609c4f42860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:37.031 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.023+0000 7fd0e9d9b640 1 --2- 172.21.15.80:0/256551940 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5609c4f42440 0x5609c4f42860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:37.031 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.023+0000 7fd0e9d9b640 1 -- 172.21.15.80:0/256551940 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609c4bba2d0 con 0x5609c4f47fa0 2024-04-24T23:29:37.031 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.023+0000 7fd0ea59c640 1 --2- 172.21.15.80:0/256551940 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5609c4f42e30 0x5609c4f47930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:29:37.031 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.023+0000 7fd0ead9d640 1 --2- 172.21.15.80:0/3637959638 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fd0d80a2b90 0x7fd0d80a5050 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fd0dc005ab0 tx=0x7fd0dc005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:37.031 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.023+0000 7fd0e9d9b640 1 --2- 172.21.15.80:0/256551940 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5609c4f47fa0 0x5609c4f4a430 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fd0d4097be0 tx=0x7fd0d406e2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d4ab25510da6173 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:37.031 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.023+0000 7fd0e0ff9640 1 -- 172.21.15.80:0/256551940 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0d409ab70 con 0x5609c4f47fa0 2024-04-24T23:29:37.032 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.023+0000 7fd0e0ff9640 1 -- 172.21.15.80:0/256551940 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0d409b910 con 0x5609c4f47fa0 2024-04-24T23:29:37.032 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.023+0000 7fd0e0ff9640 1 -- 172.21.15.80:0/256551940 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0d401fac0 con 0x5609c4f47fa0 2024-04-24T23:29:37.032 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.024+0000 7fd0f3d90740 1 -- 172.21.15.80:0/256551940 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5609c4f47fa0 msgr2=0x5609c4f4a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:37.032 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.024+0000 7fd0f3d90740 1 --2- 172.21.15.80:0/256551940 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5609c4f47fa0 0x5609c4f4a430 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fd0d4097be0 tx=0x7fd0d406e2d0 comp rx=0 tx=0).stop 2024-04-24T23:29:37.032 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.024+0000 7fd0f3d90740 1 -- 172.21.15.80:0/256551940 shutdown_connections 2024-04-24T23:29:37.032 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.024+0000 7fd0f3d90740 1 --2- 172.21.15.80:0/256551940 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5609c4f47fa0 0x5609c4f4a430 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:37.032 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.024+0000 7fd0f3d90740 1 --2- 172.21.15.80:0/256551940 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5609c4f42e30 0x5609c4f47930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:37.032 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.024+0000 7fd0f3d90740 1 --2- 172.21.15.80:0/256551940 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5609c4f42440 0x5609c4f42860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:37.032 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.024+0000 7fd0f3d90740 1 -- 172.21.15.80:0/256551940 >> 172.21.15.80:0/256551940 conn(0x5609c4f3dab0 msgr2=0x5609c4f3deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:37.032 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.024+0000 7fd0f3d90740 1 -- 172.21.15.80:0/256551940 shutdown_connections 2024-04-24T23:29:37.032 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.024+0000 7fd0f3d90740 1 -- 172.21.15.80:0/256551940 wait complete. 2024-04-24T23:29:37.032 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.024+0000 7fd0f3d90740 1 -- start start 2024-04-24T23:29:37.032 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.024+0000 7fd0f3d90740 1 Processor -- start 2024-04-24T23:29:37.032 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.024+0000 7fd0f3d90740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5609c4f42440 0x5609c4f4c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:37.032 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.024+0000 7fd0f3d90740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5609c4f42e30 0x5609c4f4cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:37.033 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.024+0000 7fd0f3d90740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5609c4f47fa0 0x5609c4f53db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:37.033 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.024+0000 7fd0f3d90740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x5609c4f4aef0 con 0x5609c4f42e30 2024-04-24T23:29:37.033 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.024+0000 7fd0f3d90740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5609c4e61cf0 con 0x5609c4f47fa0 2024-04-24T23:29:37.033 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.024+0000 7fd0f3d90740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5609c4f4b070 con 0x5609c4f42440 2024-04-24T23:29:37.033 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.024+0000 7fd0e9d9b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5609c4f42e30 0x5609c4f4cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:37.033 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.024+0000 7fd0f3d90740 10 client.0 _ll_register_callbacks cb 0x5609c4e616e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:29:37.033 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.025+0000 7fd0e9d9b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5609c4f42e30 0x5609c4f4cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53910/0 (socket says 172.21.15.80:53910) 2024-04-24T23:29:37.033 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.025+0000 7fd0e9d9b640 1 -- 172.21.15.80:0/2086929525 learned_addr learned my addr 172.21.15.80:0/2086929525 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:29:37.033 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.025+0000 7fd0ead9d640 1 --2- 172.21.15.80:0/2086929525 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5609c4f47fa0 0x5609c4f53db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:37.033 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.025+0000 7fd0ea59c640 1 --2- 172.21.15.80:0/2086929525 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5609c4f42440 0x5609c4f4c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:37.033 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.025+0000 7fd0e9d9b640 1 -- 172.21.15.80:0/2086929525 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5609c4f42440 msgr2=0x5609c4f4c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:37.033 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.025+0000 7fd0e9d9b640 1 --2- 172.21.15.80:0/2086929525 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5609c4f42440 0x5609c4f4c8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:37.033 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.025+0000 7fd0e9d9b640 1 -- 172.21.15.80:0/2086929525 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5609c4f47fa0 msgr2=0x5609c4f53db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:37.033 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.025+0000 7fd0e9d9b640 1 --2- 172.21.15.80:0/2086929525 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5609c4f47fa0 0x5609c4f53db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:37.033 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.025+0000 7fd0e9d9b640 1 -- 172.21.15.80:0/2086929525 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609c4c846b0 con 0x5609c4f42e30 2024-04-24T23:29:37.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.025+0000 7fd0ead9d640 1 --2- 172.21.15.80:0/2086929525 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5609c4f47fa0 0x5609c4f53db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:29:37.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.025+0000 7fd0e9d9b640 1 --2- 172.21.15.80:0/2086929525 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5609c4f42e30 0x5609c4f4cde0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fd0d40a3950 tx=0x7fd0d4097ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6ad7eca360b60ac server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:37.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.025+0000 7fd0d1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-04-24T23:29:37.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.025+0000 7fd0d1ffb640 1 -- 172.21.15.80:0/2086929525 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0d40b4060 con 0x5609c4f42e30 2024-04-24T23:29:37.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.025+0000 7fd0d1ffb640 1 -- 172.21.15.80:0/2086929525 <== mon.0 v2:172.21.15.80:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fd0d409b680 con 0x5609c4f42e30 2024-04-24T23:29:37.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.025+0000 7fd0f3d90740 1 -- 172.21.15.80:0/2086929525 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0d409b910 con 0x5609c4f42e30 2024-04-24T23:29:37.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.025+0000 7fd0d1ffb640 1 -- 172.21.15.80:0/2086929525 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fd0d40b4200 con 0x5609c4f42e30 2024-04-24T23:29:37.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.025+0000 7fd0f3d90740 10 client.15441 fetch_fsmap learned FSMap version 24 2024-04-24T23:29:37.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.025+0000 7fd0f3d90740 1 -- 172.21.15.80:0/2086929525 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5609c4f56d90 con 0x5609c4f42e30 2024-04-24T23:29:37.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.025+0000 7fd0d1ffb640 1 -- 172.21.15.80:0/2086929525 <== mon.0 v2:172.21.15.80:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0d409b910 con 0x5609c4f42e30 2024-04-24T23:29:37.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.025+0000 7fd0d1ffb640 1 -- 172.21.15.80:0/2086929525 <== mon.0 v2:172.21.15.80:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fd0d40a39f0 con 0x5609c4f42e30 2024-04-24T23:29:37.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.026+0000 7fd0f3d90740 10 client.15441 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T23:29:37.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.026+0000 7fd0f3d90740 10 client.15441 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-24T23:29:37.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.026+0000 7fd0f3d90740 20 client.15441 populate_metadata read hostname 'smithi080' 2024-04-24T23:29:37.035 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.026+0000 7fd0f3d90740 1 --2- 172.21.15.80:0/2086929525 >> [v2:172.21.15.134:6838/1743815021,v1:172.21.15.134:6839/1743815021] conn(0x5609c4f58fd0 0x5609c4f5a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:37.035 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.026+0000 7fd0f3d90740 4 client.15441 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:29:37.035 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.026+0000 7fd0f3d90740 1 -- 172.21.15.80:0/2086929525 --> [v2:172.21.15.134:6838/1743815021,v1:172.21.15.134:6839/1743815021] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5609c4eb3440 con 0x5609c4f58fd0 2024-04-24T23:29:37.035 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.026+0000 7fd0ea59c640 1 --2- 172.21.15.80:0/2086929525 >> [v2:172.21.15.134:6838/1743815021,v1:172.21.15.134:6839/1743815021] conn(0x5609c4f58fd0 0x5609c4f5a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:37.035 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.026+0000 7fd0ea59c640 1 --2- 172.21.15.80:0/2086929525 >> [v2:172.21.15.134:6838/1743815021,v1:172.21.15.134:6839/1743815021] conn(0x5609c4f58fd0 0x5609c4f5a440 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f34821ecbc39e131 server_cookie=ea682c4b90cb32e7 in_seq=0 out_seq=0 2024-04-24T23:29:37.035 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.026+0000 7fd0d1ffb640 10 client.15441 ms_handle_connect on v2:172.21.15.134:6838/1743815021 2024-04-24T23:29:37.035 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.027+0000 7fd0d1ffb640 1 -- 172.21.15.80:0/2086929525 <== mds.1 v2:172.21.15.134:6838/1743815021 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5609c4eb3440 con 0x5609c4f58fd0 2024-04-24T23:29:37.035 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.027+0000 7fd0d1ffb640 10 client.15441 handle_command_reply: tid=0 2024-04-24T23:29:37.035 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.027+0000 7fd0f3d90740 1 client.15441 shutdown 2024-04-24T23:29:37.035 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.027+0000 7fd0f3d90740 1 -- 172.21.15.80:0/2086929525 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5609c4f42e30 msgr2=0x5609c4f4cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:37.035 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.027+0000 7fd0f3d90740 1 --2- 172.21.15.80:0/2086929525 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5609c4f42e30 0x5609c4f4cde0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fd0d40a3950 tx=0x7fd0d4097ab0 comp rx=0 tx=0).stop 2024-04-24T23:29:37.035 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.027+0000 7fd0f3d90740 1 -- 172.21.15.80:0/2086929525 shutdown_connections 2024-04-24T23:29:37.035 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.027+0000 7fd0f3d90740 1 --2- 172.21.15.80:0/2086929525 >> [v2:172.21.15.134:6838/1743815021,v1:172.21.15.134:6839/1743815021] conn(0x5609c4f58fd0 0x5609c4f5a440 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:37.035 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.027+0000 7fd0f3d90740 1 --2- 172.21.15.80:0/2086929525 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5609c4f47fa0 0x5609c4f53db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:37.035 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.027+0000 7fd0d1ffb640 0 client.15441 ms_handle_reset on v2:172.21.15.134:6838/1743815021 2024-04-24T23:29:37.035 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.027+0000 7fd0f3d90740 1 --2- 172.21.15.80:0/2086929525 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5609c4f42e30 0x5609c4f4cde0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:37.036 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.027+0000 7fd0f3d90740 1 --2- 172.21.15.80:0/2086929525 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5609c4f42440 0x5609c4f4c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:37.036 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.027+0000 7fd0f3d90740 1 -- 172.21.15.80:0/2086929525 >> 172.21.15.80:0/2086929525 conn(0x5609c4f3dab0 msgr2=0x5609c4f3cae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:37.036 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.027+0000 7fd0f3d90740 1 -- 172.21.15.80:0/2086929525 shutdown_connections 2024-04-24T23:29:37.036 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.027+0000 7fd0f3d90740 1 -- 172.21.15.80:0/2086929525 wait complete. 2024-04-24T23:29:37.036 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.027+0000 7fd0f3d90740 20 client.15441 trim_cache size 0 max 16384 2024-04-24T23:29:37.038 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:29:37 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/339848005' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T23:29:37.038 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:29:37 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/592350597' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:29:37.038 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:29:37 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/79943504' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:29:37.057 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.045+0000 7fd0f3d90740 1 Processor -- start 2024-04-24T23:29:37.058 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.045+0000 7fd0f3d90740 1 -- start start 2024-04-24T23:29:37.058 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.046+0000 7fd0f3d90740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5609c4f42440 0x5609c4f4cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:37.058 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.046+0000 7fd0f3d90740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5609c4f42e30 0x5609c4f53d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:37.058 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.046+0000 7fd0f3d90740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5609c4f47fa0 0x5609c4f38690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:37.058 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.046+0000 7fd0f3d90740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x5609c4f58780 con 0x5609c4f42440 2024-04-24T23:29:37.058 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.046+0000 7fd0f3d90740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5609c4f58e50 con 0x5609c4f47fa0 2024-04-24T23:29:37.058 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.046+0000 7fd0f3d90740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5609c4f59520 con 0x5609c4f42e30 2024-04-24T23:29:37.058 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.046+0000 7fd0ea59c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5609c4f42440 0x5609c4f4cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:37.059 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.046+0000 7fd0ea59c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5609c4f42440 0x5609c4f4cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53916/0 (socket says 172.21.15.80:53916) 2024-04-24T23:29:37.059 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.046+0000 7fd0ea59c640 1 -- 172.21.15.80:0/3170148864 learned_addr learned my addr 172.21.15.80:0/3170148864 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:29:37.059 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.046+0000 7fd0ead9d640 1 --2- 172.21.15.80:0/3170148864 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5609c4f47fa0 0x5609c4f38690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:37.059 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.046+0000 7fd0ea59c640 1 -- 172.21.15.80:0/3170148864 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5609c4f42e30 msgr2=0x5609c4f53d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:37.059 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.046+0000 7fd0ea59c640 1 --2- 172.21.15.80:0/3170148864 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5609c4f42e30 0x5609c4f53d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:37.060 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.046+0000 7fd0ea59c640 1 -- 172.21.15.80:0/3170148864 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5609c4f47fa0 msgr2=0x5609c4f38690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:37.060 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.046+0000 7fd0ea59c640 1 --2- 172.21.15.80:0/3170148864 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5609c4f47fa0 0x5609c4f38690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:37.060 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.046+0000 7fd0ea59c640 1 -- 172.21.15.80:0/3170148864 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609c4f3d8e0 con 0x5609c4f42440 2024-04-24T23:29:37.060 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.046+0000 7fd0ead9d640 1 --2- 172.21.15.80:0/3170148864 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5609c4f47fa0 0x5609c4f38690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:29:37.060 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.046+0000 7fd0ea59c640 1 --2- 172.21.15.80:0/3170148864 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5609c4f42440 0x5609c4f4cd40 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fd0e4000f80 tx=0x7fd0e4004620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=436d515c6e3deb0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:37.060 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.046+0000 7fd0d3fff640 1 -- 172.21.15.80:0/3170148864 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0e4004b20 con 0x5609c4f42440 2024-04-24T23:29:37.060 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.046+0000 7fd0d3fff640 1 -- 172.21.15.80:0/3170148864 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0e4004cc0 con 0x5609c4f42440 2024-04-24T23:29:37.060 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.046+0000 7fd0d3fff640 1 -- 172.21.15.80:0/3170148864 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0e4005020 con 0x5609c4f42440 2024-04-24T23:29:37.060 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.047+0000 7fd0f3d90740 1 -- 172.21.15.80:0/3170148864 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5609c4f42440 msgr2=0x5609c4f4cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:37.060 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.047+0000 7fd0f3d90740 1 --2- 172.21.15.80:0/3170148864 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5609c4f42440 0x5609c4f4cd40 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fd0e4000f80 tx=0x7fd0e4004620 comp rx=0 tx=0).stop 2024-04-24T23:29:37.060 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.047+0000 7fd0f3d90740 1 -- 172.21.15.80:0/3170148864 shutdown_connections 2024-04-24T23:29:37.060 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.047+0000 7fd0f3d90740 1 --2- 172.21.15.80:0/3170148864 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5609c4f47fa0 0x5609c4f38690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:37.060 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.047+0000 7fd0f3d90740 1 --2- 172.21.15.80:0/3170148864 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5609c4f42e30 0x5609c4f53d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:37.061 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.047+0000 7fd0f3d90740 1 --2- 172.21.15.80:0/3170148864 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5609c4f42440 0x5609c4f4cd40 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:37.061 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.047+0000 7fd0f3d90740 1 -- 172.21.15.80:0/3170148864 >> 172.21.15.80:0/3170148864 conn(0x5609c4f3dab0 msgr2=0x5609c4f3de90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:37.061 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.047+0000 7fd0f3d90740 1 -- 172.21.15.80:0/3170148864 shutdown_connections 2024-04-24T23:29:37.061 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.047+0000 7fd0f3d90740 1 -- 172.21.15.80:0/3170148864 wait complete. 2024-04-24T23:29:37.061 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.047+0000 7fd0f3d90740 1 -- start start 2024-04-24T23:29:37.061 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.047+0000 7fd0f3d90740 1 Processor -- start 2024-04-24T23:29:37.061 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.047+0000 7fd0f3d90740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5609c4f42440 0x5609c4f4ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:37.061 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.047+0000 7fd0f3d90740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5609c4f42e30 0x5609c4f504e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:37.061 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.047+0000 7fd0f3d90740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5609c4f47fa0 0x5609c4f53070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:37.061 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.047+0000 7fd0f3d90740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x5609c4f5a2c0 con 0x5609c4f47fa0 2024-04-24T23:29:37.061 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.047+0000 7fd0f3d90740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5609c4f59bf0 con 0x5609c4f42e30 2024-04-24T23:29:37.061 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.047+0000 7fd0f3d90740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5609c4f5a990 con 0x5609c4f42440 2024-04-24T23:29:37.061 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.048+0000 7fd0f3d90740 10 client.0 _ll_register_callbacks cb 0x7fd0d40ba440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:29:37.061 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.048+0000 7fd0ead9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5609c4f47fa0 0x5609c4f53070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:37.062 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.048+0000 7fd0ead9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5609c4f47fa0 0x5609c4f53070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53928/0 (socket says 172.21.15.80:53928) 2024-04-24T23:29:37.062 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.048+0000 7fd0ead9d640 1 -- 172.21.15.80:0/1024951061 learned_addr learned my addr 172.21.15.80:0/1024951061 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:29:37.062 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.048+0000 7fd0e9d9b640 1 --2- 172.21.15.80:0/1024951061 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5609c4f42e30 0x5609c4f504e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:37.062 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.048+0000 7fd0ea59c640 1 --2- 172.21.15.80:0/1024951061 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5609c4f42440 0x5609c4f4ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:37.062 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.048+0000 7fd0ead9d640 1 -- 172.21.15.80:0/1024951061 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5609c4f42440 msgr2=0x5609c4f4ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:37.062 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.048+0000 7fd0ead9d640 1 --2- 172.21.15.80:0/1024951061 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5609c4f42440 0x5609c4f4ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:37.062 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.048+0000 7fd0ead9d640 1 -- 172.21.15.80:0/1024951061 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5609c4f42e30 msgr2=0x5609c4f504e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:37.062 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.048+0000 7fd0ead9d640 1 --2- 172.21.15.80:0/1024951061 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5609c4f42e30 0x5609c4f504e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:37.062 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.048+0000 7fd0ead9d640 1 -- 172.21.15.80:0/1024951061 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609c4f53660 con 0x5609c4f47fa0 2024-04-24T23:29:37.062 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.048+0000 7fd0ead9d640 1 --2- 172.21.15.80:0/1024951061 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5609c4f47fa0 0x5609c4f53070 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fd0dc0314e0 tx=0x7fd0dc0315b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21ecd7b500d89f6b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:37.062 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.048+0000 7fd0d1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-04-24T23:29:37.062 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.048+0000 7fd0d1ffb640 1 -- 172.21.15.80:0/1024951061 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0dc04f070 con 0x5609c4f47fa0 2024-04-24T23:29:37.062 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.048+0000 7fd0d1ffb640 1 -- 172.21.15.80:0/1024951061 <== mon.0 v2:172.21.15.80:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fd0dc0035f0 con 0x5609c4f47fa0 2024-04-24T23:29:37.062 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.048+0000 7fd0f3d90740 1 -- 172.21.15.80:0/1024951061 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0e4004cc0 con 0x5609c4f47fa0 2024-04-24T23:29:37.063 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.048+0000 7fd0d1ffb640 1 -- 172.21.15.80:0/1024951061 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fd0dc05b050 con 0x5609c4f47fa0 2024-04-24T23:29:37.063 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.048+0000 7fd0f3d90740 10 client.15447 fetch_fsmap learned FSMap version 24 2024-04-24T23:29:37.063 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.048+0000 7fd0f3d90740 1 -- 172.21.15.80:0/1024951061 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5609c4fbf5c0 con 0x5609c4f47fa0 2024-04-24T23:29:37.063 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.049+0000 7fd0d1ffb640 1 -- 172.21.15.80:0/1024951061 <== mon.0 v2:172.21.15.80:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0e4004cc0 con 0x5609c4f47fa0 2024-04-24T23:29:37.063 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.049+0000 7fd0d1ffb640 1 -- 172.21.15.80:0/1024951061 <== mon.0 v2:172.21.15.80:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fd0dc03e880 con 0x5609c4f47fa0 2024-04-24T23:29:37.063 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.049+0000 7fd0f3d90740 10 client.15447 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T23:29:37.063 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.049+0000 7fd0f3d90740 10 client.15447 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-24T23:29:37.063 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.049+0000 7fd0f3d90740 20 client.15447 populate_metadata read hostname 'smithi080' 2024-04-24T23:29:37.063 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.049+0000 7fd0f3d90740 1 --2- 172.21.15.80:0/1024951061 >> [v2:172.21.15.134:6838/1743815021,v1:172.21.15.134:6839/1743815021] conn(0x5609c4fc4220 0x5609c4fc2f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:37.063 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.049+0000 7fd0f3d90740 4 client.15447 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T23:29:37.063 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.049+0000 7fd0f3d90740 1 -- 172.21.15.80:0/1024951061 --> [v2:172.21.15.134:6838/1743815021,v1:172.21.15.134:6839/1743815021] -- command(tid 0: {"prefix": "status"}) -- 0x5609c4ead790 con 0x5609c4fc4220 2024-04-24T23:29:37.063 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.049+0000 7fd0ea59c640 1 --2- 172.21.15.80:0/1024951061 >> [v2:172.21.15.134:6838/1743815021,v1:172.21.15.134:6839/1743815021] conn(0x5609c4fc4220 0x5609c4fc2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:37.063 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.049+0000 7fd0ea59c640 1 --2- 172.21.15.80:0/1024951061 >> [v2:172.21.15.134:6838/1743815021,v1:172.21.15.134:6839/1743815021] conn(0x5609c4fc4220 0x5609c4fc2f30 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bf8e938654b74a0f server_cookie=2f197b5ea77fb86d in_seq=0 out_seq=0 2024-04-24T23:29:37.063 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.049+0000 7fd0d1ffb640 10 client.15447 ms_handle_connect on v2:172.21.15.134:6838/1743815021 2024-04-24T23:29:37.064 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.050+0000 7fd0d1ffb640 1 -- 172.21.15.80:0/1024951061 <== mds.1 v2:172.21.15.134:6838/1743815021 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5609c4ead790 con 0x5609c4fc4220 2024-04-24T23:29:37.064 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.050+0000 7fd0d1ffb640 10 client.15447 handle_command_reply: tid=0 2024-04-24T23:29:37.064 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.050+0000 7fd0f3d90740 1 client.15447 shutdown 2024-04-24T23:29:37.064 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.050+0000 7fd0f3d90740 1 -- 172.21.15.80:0/1024951061 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5609c4f47fa0 msgr2=0x5609c4f53070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:37.064 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.050+0000 7fd0f3d90740 1 --2- 172.21.15.80:0/1024951061 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5609c4f47fa0 0x5609c4f53070 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fd0dc0314e0 tx=0x7fd0dc0315b0 comp rx=0 tx=0).stop 2024-04-24T23:29:37.064 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.050+0000 7fd0f3d90740 1 -- 172.21.15.80:0/1024951061 shutdown_connections 2024-04-24T23:29:37.064 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.050+0000 7fd0f3d90740 1 --2- 172.21.15.80:0/1024951061 >> [v2:172.21.15.134:6838/1743815021,v1:172.21.15.134:6839/1743815021] conn(0x5609c4fc4220 0x5609c4fc2f30 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:37.064 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.050+0000 7fd0f3d90740 1 --2- 172.21.15.80:0/1024951061 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5609c4f47fa0 0x5609c4f53070 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:37.064 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.050+0000 7fd0d1ffb640 0 client.15447 ms_handle_reset on v2:172.21.15.134:6838/1743815021 2024-04-24T23:29:37.064 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.050+0000 7fd0f3d90740 1 --2- 172.21.15.80:0/1024951061 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5609c4f42e30 0x5609c4f504e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:37.064 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.050+0000 7fd0f3d90740 1 --2- 172.21.15.80:0/1024951061 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5609c4f42440 0x5609c4f4ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:37.064 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.050+0000 7fd0f3d90740 1 -- 172.21.15.80:0/1024951061 >> 172.21.15.80:0/1024951061 conn(0x5609c4f3dab0 msgr2=0x5609c4f38690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:37.064 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.050+0000 7fd0f3d90740 1 -- 172.21.15.80:0/1024951061 shutdown_connections 2024-04-24T23:29:37.064 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.050+0000 7fd0f3d90740 1 -- 172.21.15.80:0/1024951061 wait complete. 2024-04-24T23:29:37.064 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.050+0000 7fd0f3d90740 20 client.15447 trim_cache size 0 max 16384 2024-04-24T23:29:37.065 INFO:teuthology.orchestra.run.smithi080.stdout:{ 2024-04-24T23:29:37.065 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_fsid": "ec7885aa-0290-11ef-bc93-c7b262605968", 2024-04-24T23:29:37.065 INFO:teuthology.orchestra.run.smithi080.stdout: "whoami": 1, 2024-04-24T23:29:37.065 INFO:teuthology.orchestra.run.smithi080.stdout: "id": 24488, 2024-04-24T23:29:37.065 INFO:teuthology.orchestra.run.smithi080.stdout: "want_state": "up:active", 2024-04-24T23:29:37.065 INFO:teuthology.orchestra.run.smithi080.stdout: "state": "up:active", 2024-04-24T23:29:37.065 INFO:teuthology.orchestra.run.smithi080.stdout: "fs_name": "cephfs", 2024-04-24T23:29:37.065 INFO:teuthology.orchestra.run.smithi080.stdout: "rank_uptime": 113.08679204400001, 2024-04-24T23:29:37.065 INFO:teuthology.orchestra.run.smithi080.stdout: "mdsmap_epoch": 24, 2024-04-24T23:29:37.065 INFO:teuthology.orchestra.run.smithi080.stdout: "osdmap_epoch": 94, 2024-04-24T23:29:37.065 INFO:teuthology.orchestra.run.smithi080.stdout: "osdmap_epoch_barrier": 90, 2024-04-24T23:29:37.066 INFO:teuthology.orchestra.run.smithi080.stdout: "uptime": 143.84558847700001 2024-04-24T23:29:37.066 INFO:teuthology.orchestra.run.smithi080.stdout:} 2024-04-24T23:29:37.067 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.050+0000 7fd0f0fa2640 1 -- 172.21.15.80:0/3637959638 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fd0d80a2b90 msgr2=0x7fd0d80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:37.067 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.050+0000 7fd0f0fa2640 1 --2- 172.21.15.80:0/3637959638 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fd0d80a2b90 0x7fd0d80a5050 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fd0dc005ab0 tx=0x7fd0dc005fd0 comp rx=0 tx=0).stop 2024-04-24T23:29:37.067 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.050+0000 7fd0f0fa2640 1 -- 172.21.15.80:0/3637959638 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd0ec102a20 msgr2=0x7fd0ec197f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:37.067 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.050+0000 7fd0f0fa2640 1 --2- 172.21.15.80:0/3637959638 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd0ec102a20 0x7fd0ec197f80 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fd0d400d950 tx=0x7fd0d400de10 comp rx=0 tx=0).stop 2024-04-24T23:29:37.068 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.050+0000 7fd0f0fa2640 1 -- 172.21.15.80:0/3637959638 shutdown_connections 2024-04-24T23:29:37.068 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.050+0000 7fd0f0fa2640 1 --2- 172.21.15.80:0/3637959638 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fd0d80a2b90 0x7fd0d80a5050 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:37.068 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.050+0000 7fd0f0fa2640 1 --2- 172.21.15.80:0/3637959638 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd0ec1033e0 0x7fd0ec19eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:37.068 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.050+0000 7fd0f0fa2640 1 --2- 172.21.15.80:0/3637959638 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd0ec102a20 0x7fd0ec197f80 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:37.068 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.050+0000 7fd0f0fa2640 1 --2- 172.21.15.80:0/3637959638 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0ec0ffa50 0x7fd0ec197a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:37.068 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.050+0000 7fd0f0fa2640 1 -- 172.21.15.80:0/3637959638 >> 172.21.15.80:0/3637959638 conn(0x7fd0ec0fb380 msgr2=0x7fd0ec105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:37.068 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.050+0000 7fd0f0fa2640 1 -- 172.21.15.80:0/3637959638 shutdown_connections 2024-04-24T23:29:37.068 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.050+0000 7fd0f0fa2640 1 -- 172.21.15.80:0/3637959638 wait complete. 2024-04-24T23:29:37.069 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-24T23:29:37.170 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.167+0000 7f71a95a5640 1 -- 172.21.15.80:0/2344127139 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f71a4101790 msgr2=0x7f71a4101bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:37.170 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.167+0000 7f71a95a5640 1 --2- 172.21.15.80:0/2344127139 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f71a4101790 0x7f71a4101bb0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f7194009ca0 tx=0x7f719402a8c0 comp rx=0 tx=0).stop 2024-04-24T23:29:37.170 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.167+0000 7f71a95a5640 1 -- 172.21.15.80:0/2344127139 shutdown_connections 2024-04-24T23:29:37.170 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.167+0000 7f71a95a5640 1 --2- 172.21.15.80:0/2344127139 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f71a4103340 0x7f71a4109bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:37.170 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.167+0000 7f71a95a5640 1 --2- 172.21.15.80:0/2344127139 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f71a4102980 0x7f71a4102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:37.171 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.167+0000 7f71a95a5640 1 --2- 172.21.15.80:0/2344127139 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f71a4101790 0x7f71a4101bb0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:37.171 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.167+0000 7f71a95a5640 1 -- 172.21.15.80:0/2344127139 >> 172.21.15.80:0/2344127139 conn(0x7f71a40fcd20 msgr2=0x7f71a40ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:37.171 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.167+0000 7f71a95a5640 1 -- 172.21.15.80:0/2344127139 shutdown_connections 2024-04-24T23:29:37.171 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.167+0000 7f71a95a5640 1 -- 172.21.15.80:0/2344127139 wait complete. 2024-04-24T23:29:37.171 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.168+0000 7f71a95a5640 1 Processor -- start 2024-04-24T23:29:37.171 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.168+0000 7f71a95a5640 1 -- start start 2024-04-24T23:29:37.171 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.168+0000 7f71a95a5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f71a4101790 0x7f71a4197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:37.171 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.168+0000 7f71a95a5640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f71a4102980 0x7f71a4198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:37.171 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.168+0000 7f71a95a5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f71a4103340 0x7f71a419f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:37.171 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.168+0000 7f71a95a5640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f71a410d3e0 con 0x7f71a4102980 2024-04-24T23:29:37.171 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.168+0000 7f71a95a5640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f71a410d260 con 0x7f71a4101790 2024-04-24T23:29:37.171 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.168+0000 7f71a95a5640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f71a410d560 con 0x7f71a4103340 2024-04-24T23:29:37.171 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.168+0000 7f71a27fc640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f71a4102980 0x7f71a4198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:37.171 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.168+0000 7f71a27fc640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f71a4102980 0x7f71a4198000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53940/0 (socket says 172.21.15.80:53940) 2024-04-24T23:29:37.171 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.168+0000 7f71a27fc640 1 -- 172.21.15.80:0/1233006120 learned_addr learned my addr 172.21.15.80:0/1233006120 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:29:37.172 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.168+0000 7f71a2ffd640 1 --2- 172.21.15.80:0/1233006120 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f71a4101790 0x7f71a4197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:37.172 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.168+0000 7f71a27fc640 1 -- 172.21.15.80:0/1233006120 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f71a4103340 msgr2=0x7f71a419f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:37.172 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.168+0000 7f71a27fc640 1 --2- 172.21.15.80:0/1233006120 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f71a4103340 0x7f71a419f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:37.172 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.168+0000 7f71a27fc640 1 -- 172.21.15.80:0/1233006120 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f71a4101790 msgr2=0x7f71a4197ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:37.172 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.168+0000 7f71a27fc640 1 --2- 172.21.15.80:0/1233006120 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f71a4101790 0x7f71a4197ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:37.172 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.168+0000 7f71a27fc640 1 -- 172.21.15.80:0/1233006120 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71a419f6e0 con 0x7f71a4102980 2024-04-24T23:29:37.172 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.168+0000 7f71a2ffd640 1 --2- 172.21.15.80:0/1233006120 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f71a4101790 0x7f71a4197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:29:37.172 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.169+0000 7f71a27fc640 1 --2- 172.21.15.80:0/1233006120 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f71a4102980 0x7f71a4198000 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f719800d950 tx=0x7f719800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:37.172 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.169+0000 7f718bfff640 1 -- 172.21.15.80:0/1233006120 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7198017030 con 0x7f71a4102980 2024-04-24T23:29:37.172 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.169+0000 7f718bfff640 1 -- 172.21.15.80:0/1233006120 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7198004020 con 0x7f71a4102980 2024-04-24T23:29:37.172 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.169+0000 7f71a95a5640 1 -- 172.21.15.80:0/1233006120 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71a419f9d0 con 0x7f71a4102980 2024-04-24T23:29:37.172 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.169+0000 7f718bfff640 1 -- 172.21.15.80:0/1233006120 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71980047a0 con 0x7f71a4102980 2024-04-24T23:29:37.172 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.169+0000 7f71a95a5640 1 -- 172.21.15.80:0/1233006120 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71a419ff20 con 0x7f71a4102980 2024-04-24T23:29:37.181 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.171+0000 7f718bfff640 1 -- 172.21.15.80:0/1233006120 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f71980041d0 con 0x7f71a4102980 2024-04-24T23:29:37.181 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.171+0000 7f71ac38c740 1 Processor -- start 2024-04-24T23:29:37.181 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.171+0000 7f71ac38c740 1 -- start start 2024-04-24T23:29:37.181 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.171+0000 7f71ac38c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d78bb6a440 0x55d78bb6a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:37.181 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.171+0000 7f71ac38c740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d78bb6ae30 0x55d78bb6f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:37.181 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.171+0000 7f71ac38c740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d78bb6ffa0 0x55d78bb72430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:37.181 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.171+0000 7f71ac38c740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x55d78bacdf00 con 0x55d78bb6ffa0 2024-04-24T23:29:37.181 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.171+0000 7f71ac38c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d78ba76750 con 0x55d78bb6a440 2024-04-24T23:29:37.182 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.171+0000 7f71ac38c740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55d78bac9ec0 con 0x55d78bb6ae30 2024-04-24T23:29:37.182 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.171+0000 7f718bfff640 1 --2- 172.21.15.80:0/1233006120 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f718c0a2b90 0x7f718c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:37.182 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.172+0000 7f71a27fc640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d78bb6ffa0 0x55d78bb72430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:37.182 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.172+0000 7f71a27fc640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d78bb6ffa0 0x55d78bb72430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53948/0 (socket says 172.21.15.80:53948) 2024-04-24T23:29:37.182 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.172+0000 7f71a37fe640 1 --2- 172.21.15.80:0/1233006120 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f718c0a2b90 0x7f718c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:37.182 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.172+0000 7f71a27fc640 1 -- 172.21.15.80:0/4217076623 learned_addr learned my addr 172.21.15.80:0/4217076623 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:29:37.182 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.172+0000 7f71a37fe640 1 --2- 172.21.15.80:0/4217076623 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d78bb6a440 0x55d78bb6a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:37.183 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.172+0000 7f718bfff640 1 -- 172.21.15.80:0/1233006120 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f71980cef30 con 0x7f71a4102980 2024-04-24T23:29:37.183 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.172+0000 7f71a2ffd640 1 --2- 172.21.15.80:0/4217076623 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d78bb6ae30 0x55d78bb6f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:37.183 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.172+0000 7f71a27fc640 1 -- 172.21.15.80:0/4217076623 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d78bb6ae30 msgr2=0x55d78bb6f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:37.183 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.172+0000 7f71a27fc640 1 --2- 172.21.15.80:0/4217076623 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d78bb6ae30 0x55d78bb6f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:37.183 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.172+0000 7f71a27fc640 1 -- 172.21.15.80:0/4217076623 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d78bb6a440 msgr2=0x55d78bb6a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:37.183 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.172+0000 7f71a27fc640 1 --2- 172.21.15.80:0/4217076623 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d78bb6a440 0x55d78bb6a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:37.183 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.172+0000 7f71a27fc640 1 -- 172.21.15.80:0/4217076623 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d78b7e22d0 con 0x55d78bb6ffa0 2024-04-24T23:29:37.183 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.172+0000 7f71a37fe640 1 --2- 172.21.15.80:0/1233006120 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f718c0a2b90 0x7f718c0a5050 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f719c009460 tx=0x7f719c009980 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:37.183 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.172+0000 7f71a27fc640 1 --2- 172.21.15.80:0/4217076623 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d78bb6ffa0 0x55d78bb72430 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f7198005fd0 tx=0x7f7198096f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6ec3c84634466b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:37.183 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.172+0000 7f71897fa640 1 -- 172.21.15.80:0/4217076623 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f719809ad20 con 0x55d78bb6ffa0 2024-04-24T23:29:37.183 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.172+0000 7f71897fa640 1 -- 172.21.15.80:0/4217076623 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f719809bac0 con 0x55d78bb6ffa0 2024-04-24T23:29:37.184 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.172+0000 7f71897fa640 1 -- 172.21.15.80:0/4217076623 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7198099ef0 con 0x55d78bb6ffa0 2024-04-24T23:29:37.184 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.173+0000 7f71ac38c740 1 -- 172.21.15.80:0/4217076623 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d78bb6ffa0 msgr2=0x55d78bb72430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:37.184 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.173+0000 7f71ac38c740 1 --2- 172.21.15.80:0/4217076623 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d78bb6ffa0 0x55d78bb72430 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f7198005fd0 tx=0x7f7198096f40 comp rx=0 tx=0).stop 2024-04-24T23:29:37.184 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.173+0000 7f71ac38c740 1 -- 172.21.15.80:0/4217076623 shutdown_connections 2024-04-24T23:29:37.184 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.173+0000 7f71ac38c740 1 --2- 172.21.15.80:0/4217076623 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d78bb6ffa0 0x55d78bb72430 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:37.184 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.173+0000 7f71ac38c740 1 --2- 172.21.15.80:0/4217076623 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d78bb6ae30 0x55d78bb6f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:37.184 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.173+0000 7f71ac38c740 1 --2- 172.21.15.80:0/4217076623 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d78bb6a440 0x55d78bb6a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:37.184 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.173+0000 7f71ac38c740 1 -- 172.21.15.80:0/4217076623 >> 172.21.15.80:0/4217076623 conn(0x55d78bb65ab0 msgr2=0x55d78bb65eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:37.184 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.173+0000 7f71ac38c740 1 -- 172.21.15.80:0/4217076623 shutdown_connections 2024-04-24T23:29:37.184 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.173+0000 7f71ac38c740 1 -- 172.21.15.80:0/4217076623 wait complete. 2024-04-24T23:29:37.184 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.173+0000 7f71ac38c740 1 -- start start 2024-04-24T23:29:37.185 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.173+0000 7f71ac38c740 1 Processor -- start 2024-04-24T23:29:37.185 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.173+0000 7f71ac38c740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d78bb6a440 0x55d78bb78b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:37.185 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.173+0000 7f71ac38c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d78bb6ae30 0x55d78bb79060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:37.185 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.174+0000 7f71ac38c740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d78bb6ffa0 0x55d78bb80030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:37.185 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.174+0000 7f71ac38c740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x55d78bb72e90 con 0x55d78bb6ffa0 2024-04-24T23:29:37.185 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.174+0000 7f71ac38c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d78ba89cf0 con 0x55d78bb6ae30 2024-04-24T23:29:37.185 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.174+0000 7f71ac38c740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55d78bb73010 con 0x55d78bb6a440 2024-04-24T23:29:37.186 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.174+0000 7f71ac38c740 10 client.0 _ll_register_callbacks cb 0x55d78ba896e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:29:37.186 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.174+0000 7f71a37fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d78bb6ffa0 0x55d78bb80030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:37.186 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.174+0000 7f71a37fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d78bb6ffa0 0x55d78bb80030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53962/0 (socket says 172.21.15.80:53962) 2024-04-24T23:29:37.186 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.174+0000 7f71a37fe640 1 -- 172.21.15.80:0/2954494700 learned_addr learned my addr 172.21.15.80:0/2954494700 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:29:37.186 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.174+0000 7f71a2ffd640 1 --2- 172.21.15.80:0/2954494700 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d78bb6a440 0x55d78bb78b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:37.186 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.174+0000 7f71a27fc640 1 --2- 172.21.15.80:0/2954494700 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d78bb6ae30 0x55d78bb79060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:37.186 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.174+0000 7f71a37fe640 1 -- 172.21.15.80:0/2954494700 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d78bb6a440 msgr2=0x55d78bb78b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:37.186 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.174+0000 7f71a37fe640 1 --2- 172.21.15.80:0/2954494700 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d78bb6a440 0x55d78bb78b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:37.186 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.174+0000 7f71a37fe640 1 -- 172.21.15.80:0/2954494700 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d78bb6ae30 msgr2=0x55d78bb79060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:37.186 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.174+0000 7f71a37fe640 1 --2- 172.21.15.80:0/2954494700 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d78bb6ae30 0x55d78bb79060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:37.187 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.174+0000 7f71a37fe640 1 -- 172.21.15.80:0/2954494700 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d78b8ac6b0 con 0x55d78bb6ffa0 2024-04-24T23:29:37.187 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.174+0000 7f71a37fe640 1 --2- 172.21.15.80:0/2954494700 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d78bb6ffa0 0x55d78bb80030 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f719c0108b0 tx=0x7f719c01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=288177a2f5a5aaf7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:37.187 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.174+0000 7f7186ff5640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-04-24T23:29:37.187 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.174+0000 7f7186ff5640 1 -- 172.21.15.80:0/2954494700 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f719c011bc0 con 0x55d78bb6ffa0 2024-04-24T23:29:37.187 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.174+0000 7f7186ff5640 1 -- 172.21.15.80:0/2954494700 <== mon.0 v2:172.21.15.80:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f719c011d60 con 0x55d78bb6ffa0 2024-04-24T23:29:37.187 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.174+0000 7f71ac38c740 1 -- 172.21.15.80:0/2954494700 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f719809bac0 con 0x55d78bb6ffa0 2024-04-24T23:29:37.187 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.174+0000 7f7186ff5640 1 -- 172.21.15.80:0/2954494700 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f719c032050 con 0x55d78bb6ffa0 2024-04-24T23:29:37.187 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.174+0000 7f71ac38c740 10 client.15459 fetch_fsmap learned FSMap version 24 2024-04-24T23:29:37.187 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.174+0000 7f71ac38c740 1 -- 172.21.15.80:0/2954494700 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55d78bb83080 con 0x55d78bb6ffa0 2024-04-24T23:29:37.187 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.175+0000 7f7186ff5640 1 -- 172.21.15.80:0/2954494700 <== mon.0 v2:172.21.15.80:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f719809bac0 con 0x55d78bb6ffa0 2024-04-24T23:29:37.188 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.175+0000 7f7186ff5640 1 -- 172.21.15.80:0/2954494700 <== mon.0 v2:172.21.15.80:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f719c00e450 con 0x55d78bb6ffa0 2024-04-24T23:29:37.188 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.175+0000 7f71ac38c740 10 client.15459 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T23:29:37.188 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.175+0000 7f71ac38c740 10 client.15459 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-24T23:29:37.188 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.175+0000 7f71ac38c740 20 client.15459 populate_metadata read hostname 'smithi080' 2024-04-24T23:29:37.188 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.175+0000 7f71ac38c740 1 --2- 172.21.15.80:0/2954494700 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x55d78bb85770 0x55d78bb86be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:37.188 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.175+0000 7f71ac38c740 4 client.15459 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:29:37.188 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.175+0000 7f71ac38c740 1 -- 172.21.15.80:0/2954494700 --> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d78badb440 con 0x55d78bb85770 2024-04-24T23:29:37.188 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.175+0000 7f71a2ffd640 1 --2- 172.21.15.80:0/2954494700 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x55d78bb85770 0x55d78bb86be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:37.188 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.175+0000 7f71a2ffd640 1 --2- 172.21.15.80:0/2954494700 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x55d78bb85770 0x55d78bb86be0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=645547e424722311 server_cookie=fa29a7c720e968e9 in_seq=0 out_seq=0 2024-04-24T23:29:37.188 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.175+0000 7f7186ff5640 10 client.15459 ms_handle_connect on v2:172.21.15.148:6838/2952757594 2024-04-24T23:29:37.189 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.176+0000 7f7186ff5640 1 -- 172.21.15.80:0/2954494700 <== mds.0 v2:172.21.15.148:6838/2952757594 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d78badb440 con 0x55d78bb85770 2024-04-24T23:29:37.189 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.176+0000 7f7186ff5640 10 client.15459 handle_command_reply: tid=0 2024-04-24T23:29:37.189 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.176+0000 7f71ac38c740 1 client.15459 shutdown 2024-04-24T23:29:37.189 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.176+0000 7f71ac38c740 1 -- 172.21.15.80:0/2954494700 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d78bb6ffa0 msgr2=0x55d78bb80030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:37.189 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.176+0000 7f71ac38c740 1 --2- 172.21.15.80:0/2954494700 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d78bb6ffa0 0x55d78bb80030 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f719c0108b0 tx=0x7f719c01a000 comp rx=0 tx=0).stop 2024-04-24T23:29:37.189 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.177+0000 7f71ac38c740 1 -- 172.21.15.80:0/2954494700 shutdown_connections 2024-04-24T23:29:37.189 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.177+0000 7f71ac38c740 1 --2- 172.21.15.80:0/2954494700 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x55d78bb85770 0x55d78bb86be0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:37.189 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.177+0000 7f71ac38c740 1 --2- 172.21.15.80:0/2954494700 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d78bb6ffa0 0x55d78bb80030 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:37.189 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.177+0000 7f71ac38c740 1 --2- 172.21.15.80:0/2954494700 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d78bb6ae30 0x55d78bb79060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:37.189 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.177+0000 7f71ac38c740 1 --2- 172.21.15.80:0/2954494700 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d78bb6a440 0x55d78bb78b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:37.190 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.177+0000 7f7186ff5640 0 client.15459 ms_handle_reset on v2:172.21.15.148:6838/2952757594 2024-04-24T23:29:37.190 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.177+0000 7f71ac38c740 1 -- 172.21.15.80:0/2954494700 >> 172.21.15.80:0/2954494700 conn(0x55d78bb65ab0 msgr2=0x55d78bb64ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:37.190 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.177+0000 7f71ac38c740 1 -- 172.21.15.80:0/2954494700 shutdown_connections 2024-04-24T23:29:37.190 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.177+0000 7f71ac38c740 1 -- 172.21.15.80:0/2954494700 wait complete. 2024-04-24T23:29:37.190 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.177+0000 7f71ac38c740 20 client.15459 trim_cache size 0 max 16384 2024-04-24T23:29:37.201 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.195+0000 7f71ac38c740 1 Processor -- start 2024-04-24T23:29:37.202 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.195+0000 7f71ac38c740 1 -- start start 2024-04-24T23:29:37.202 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.195+0000 7f71ac38c740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d78bb6a440 0x55d78bb60690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:37.202 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.195+0000 7f71ac38c740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d78bb6ae30 0x55d78bb7fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:37.202 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.195+0000 7f71ac38c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d78bb6ffa0 0x55d78bb78b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:37.202 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.195+0000 7f71ac38c740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x55d78bb84f20 con 0x55d78bb6ae30 2024-04-24T23:29:37.202 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.195+0000 7f71ac38c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d78bb855f0 con 0x55d78bb6ffa0 2024-04-24T23:29:37.202 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.195+0000 7f71ac38c740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55d78bb85cc0 con 0x55d78bb6a440 2024-04-24T23:29:37.202 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.195+0000 7f71a27fc640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d78bb6ae30 0x55d78bb7fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:37.202 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.195+0000 7f71a27fc640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d78bb6ae30 0x55d78bb7fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53978/0 (socket says 172.21.15.80:53978) 2024-04-24T23:29:37.202 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.195+0000 7f71a27fc640 1 -- 172.21.15.80:0/918652240 learned_addr learned my addr 172.21.15.80:0/918652240 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:29:37.203 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.195+0000 7f71a2ffd640 1 --2- 172.21.15.80:0/918652240 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d78bb6a440 0x55d78bb60690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:37.203 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.195+0000 7f71a27fc640 1 -- 172.21.15.80:0/918652240 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d78bb6a440 msgr2=0x55d78bb60690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:37.203 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.195+0000 7f71a27fc640 1 --2- 172.21.15.80:0/918652240 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d78bb6a440 0x55d78bb60690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:37.203 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.195+0000 7f71a27fc640 1 -- 172.21.15.80:0/918652240 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d78bb6ffa0 msgr2=0x55d78bb78b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:37.203 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.195+0000 7f71a27fc640 1 --2- 172.21.15.80:0/918652240 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d78bb6ffa0 0x55d78bb78b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:37.203 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.195+0000 7f71a27fc640 1 -- 172.21.15.80:0/918652240 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d78bb853d0 con 0x55d78bb6ae30 2024-04-24T23:29:37.203 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.195+0000 7f71a27fc640 1 --2- 172.21.15.80:0/918652240 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d78bb6ae30 0x55d78bb7fff0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f719809a280 tx=0x7f719800b530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26e04959025e69e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:37.203 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.195+0000 7f7188ff9640 1 -- 172.21.15.80:0/918652240 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71980a4030 con 0x55d78bb6ae30 2024-04-24T23:29:37.203 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.196+0000 7f7188ff9640 1 -- 172.21.15.80:0/918652240 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f719809f770 con 0x55d78bb6ae30 2024-04-24T23:29:37.203 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.196+0000 7f7188ff9640 1 -- 172.21.15.80:0/918652240 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f719809a440 con 0x55d78bb6ae30 2024-04-24T23:29:37.203 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.196+0000 7f71ac38c740 1 -- 172.21.15.80:0/918652240 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d78bb6ae30 msgr2=0x55d78bb7fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:37.203 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.196+0000 7f71ac38c740 1 --2- 172.21.15.80:0/918652240 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d78bb6ae30 0x55d78bb7fff0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f719809a280 tx=0x7f719800b530 comp rx=0 tx=0).stop 2024-04-24T23:29:37.203 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.196+0000 7f71ac38c740 1 -- 172.21.15.80:0/918652240 shutdown_connections 2024-04-24T23:29:37.203 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.196+0000 7f71ac38c740 1 --2- 172.21.15.80:0/918652240 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d78bb6ffa0 0x55d78bb78b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:37.204 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.196+0000 7f71ac38c740 1 --2- 172.21.15.80:0/918652240 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d78bb6ae30 0x55d78bb7fff0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:37.204 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.196+0000 7f71ac38c740 1 --2- 172.21.15.80:0/918652240 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d78bb6a440 0x55d78bb60690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:37.204 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.196+0000 7f71ac38c740 1 -- 172.21.15.80:0/918652240 >> 172.21.15.80:0/918652240 conn(0x55d78bb65ab0 msgr2=0x55d78bb65e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:37.204 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.196+0000 7f71ac38c740 1 -- 172.21.15.80:0/918652240 shutdown_connections 2024-04-24T23:29:37.204 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.196+0000 7f71ac38c740 1 -- 172.21.15.80:0/918652240 wait complete. 2024-04-24T23:29:37.204 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.196+0000 7f71ac38c740 1 -- start start 2024-04-24T23:29:37.204 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.196+0000 7f71ac38c740 1 Processor -- start 2024-04-24T23:29:37.204 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.197+0000 7f71ac38c740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d78bb6a440 0x55d78bb7c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:37.204 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.197+0000 7f71ac38c740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d78bb6ae30 0x55d78bb7c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:37.204 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.197+0000 7f71ac38c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d78bb6ffa0 0x55d78bb7ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:37.204 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.197+0000 7f71ac38c740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x55d78bb86a60 con 0x55d78bb6a440 2024-04-24T23:29:37.204 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.197+0000 7f71ac38c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d78bb86390 con 0x55d78bb6ffa0 2024-04-24T23:29:37.205 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.197+0000 7f71ac38c740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55d78bb87130 con 0x55d78bb6ae30 2024-04-24T23:29:37.205 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.197+0000 7f71ac38c740 10 client.0 _ll_register_callbacks cb 0x7f719c009c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:29:37.205 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.197+0000 7f71a2ffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d78bb6a440 0x55d78bb7c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:37.205 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.197+0000 7f71a2ffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d78bb6a440 0x55d78bb7c180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53986/0 (socket says 172.21.15.80:53986) 2024-04-24T23:29:37.205 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.197+0000 7f71a2ffd640 1 -- 172.21.15.80:0/4273365968 learned_addr learned my addr 172.21.15.80:0/4273365968 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:29:37.205 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.197+0000 7f71a37fe640 1 --2- 172.21.15.80:0/4273365968 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d78bb6ffa0 0x55d78bb7ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:37.205 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.197+0000 7f71a27fc640 1 --2- 172.21.15.80:0/4273365968 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d78bb6ae30 0x55d78bb7c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:37.205 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.197+0000 7f71a2ffd640 1 -- 172.21.15.80:0/4273365968 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d78bb6ae30 msgr2=0x55d78bb7c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:37.205 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.197+0000 7f71a2ffd640 1 --2- 172.21.15.80:0/4273365968 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d78bb6ae30 0x55d78bb7c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:37.205 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.197+0000 7f71a2ffd640 1 -- 172.21.15.80:0/4273365968 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d78bb6ffa0 msgr2=0x55d78bb7ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:37.205 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.197+0000 7f71a2ffd640 1 --2- 172.21.15.80:0/4273365968 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d78bb6ffa0 0x55d78bb7ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:37.205 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.197+0000 7f71a2ffd640 1 -- 172.21.15.80:0/4273365968 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d78bb7f550 con 0x55d78bb6a440 2024-04-24T23:29:37.205 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.197+0000 7f71a37fe640 1 --2- 172.21.15.80:0/4273365968 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d78bb6ffa0 0x55d78bb7ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:29:37.206 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.197+0000 7f71a2ffd640 1 --2- 172.21.15.80:0/4273365968 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d78bb6a440 0x55d78bb7c180 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f7194009620 tx=0x7f719402f4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd397c461d38cf2b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:37.206 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.197+0000 7f7186ff5640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-04-24T23:29:37.206 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.197+0000 7f7186ff5640 1 -- 172.21.15.80:0/4273365968 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f719404a030 con 0x55d78bb6a440 2024-04-24T23:29:37.206 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.197+0000 7f71ac38c740 1 -- 172.21.15.80:0/4273365968 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f719809f770 con 0x55d78bb6a440 2024-04-24T23:29:37.206 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.197+0000 7f7186ff5640 1 -- 172.21.15.80:0/4273365968 <== mon.0 v2:172.21.15.80:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f7194031660 con 0x55d78bb6a440 2024-04-24T23:29:37.206 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.197+0000 7f7186ff5640 1 -- 172.21.15.80:0/4273365968 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f7194057050 con 0x55d78bb6a440 2024-04-24T23:29:37.206 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.197+0000 7f71ac38c740 10 client.15465 fetch_fsmap learned FSMap version 24 2024-04-24T23:29:37.206 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.197+0000 7f71ac38c740 1 -- 172.21.15.80:0/4273365968 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55d78bbebab0 con 0x55d78bb6a440 2024-04-24T23:29:37.206 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.198+0000 7f7186ff5640 1 -- 172.21.15.80:0/4273365968 <== mon.0 v2:172.21.15.80:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f719809f770 con 0x55d78bb6a440 2024-04-24T23:29:37.206 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.198+0000 7f7186ff5640 1 -- 172.21.15.80:0/4273365968 <== mon.0 v2:172.21.15.80:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f7194004040 con 0x55d78bb6a440 2024-04-24T23:29:37.206 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.198+0000 7f71ac38c740 10 client.15465 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T23:29:37.206 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.198+0000 7f71ac38c740 10 client.15465 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-24T23:29:37.206 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.198+0000 7f71ac38c740 20 client.15465 populate_metadata read hostname 'smithi080' 2024-04-24T23:29:37.206 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.198+0000 7f71ac38c740 1 --2- 172.21.15.80:0/4273365968 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x55d78bbf06d0 0x55d78bbecb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:37.207 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.198+0000 7f71ac38c740 4 client.15465 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T23:29:37.207 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.198+0000 7f71ac38c740 1 -- 172.21.15.80:0/4273365968 --> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] -- command(tid 0: {"prefix": "status"}) -- 0x55d78bad5790 con 0x55d78bbf06d0 2024-04-24T23:29:37.207 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.198+0000 7f71a27fc640 1 --2- 172.21.15.80:0/4273365968 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x55d78bbf06d0 0x55d78bbecb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:37.207 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.198+0000 7f71a27fc640 1 --2- 172.21.15.80:0/4273365968 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x55d78bbf06d0 0x55d78bbecb10 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b21ca570a4f327e8 server_cookie=da9e2ff72448f4c8 in_seq=0 out_seq=0 2024-04-24T23:29:37.207 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.198+0000 7f7186ff5640 10 client.15465 ms_handle_connect on v2:172.21.15.148:6838/2952757594 2024-04-24T23:29:37.207 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.199+0000 7f7186ff5640 1 -- 172.21.15.80:0/4273365968 <== mds.0 v2:172.21.15.148:6838/2952757594 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55d78bad5790 con 0x55d78bbf06d0 2024-04-24T23:29:37.207 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.199+0000 7f7186ff5640 10 client.15465 handle_command_reply: tid=0 2024-04-24T23:29:37.207 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.199+0000 7f71ac38c740 1 client.15465 shutdown 2024-04-24T23:29:37.207 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.199+0000 7f71ac38c740 1 -- 172.21.15.80:0/4273365968 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d78bb6a440 msgr2=0x55d78bb7c180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:37.207 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.199+0000 7f71ac38c740 1 --2- 172.21.15.80:0/4273365968 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d78bb6a440 0x55d78bb7c180 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f7194009620 tx=0x7f719402f4c0 comp rx=0 tx=0).stop 2024-04-24T23:29:37.207 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.199+0000 7f71ac38c740 1 -- 172.21.15.80:0/4273365968 shutdown_connections 2024-04-24T23:29:37.207 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.199+0000 7f71ac38c740 1 --2- 172.21.15.80:0/4273365968 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x55d78bbf06d0 0x55d78bbecb10 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:37.207 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.199+0000 7f71ac38c740 1 --2- 172.21.15.80:0/4273365968 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d78bb6ffa0 0x55d78bb7ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:37.207 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.199+0000 7f71ac38c740 1 --2- 172.21.15.80:0/4273365968 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d78bb6ae30 0x55d78bb7c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:37.208 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.199+0000 7f7186ff5640 0 client.15465 ms_handle_reset on v2:172.21.15.148:6838/2952757594 2024-04-24T23:29:37.208 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.199+0000 7f71ac38c740 1 --2- 172.21.15.80:0/4273365968 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d78bb6a440 0x55d78bb7c180 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:37.208 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.199+0000 7f71ac38c740 1 -- 172.21.15.80:0/4273365968 >> 172.21.15.80:0/4273365968 conn(0x55d78bb65ab0 msgr2=0x55d78bb60690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:37.208 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.199+0000 7f71ac38c740 1 -- 172.21.15.80:0/4273365968 shutdown_connections 2024-04-24T23:29:37.208 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.199+0000 7f71ac38c740 1 -- 172.21.15.80:0/4273365968 wait complete. 2024-04-24T23:29:37.208 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.199+0000 7f71ac38c740 20 client.15465 trim_cache size 0 max 16384 2024-04-24T23:29:37.208 INFO:teuthology.orchestra.run.smithi080.stdout:{ 2024-04-24T23:29:37.208 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_fsid": "ec7885aa-0290-11ef-bc93-c7b262605968", 2024-04-24T23:29:37.209 INFO:teuthology.orchestra.run.smithi080.stdout: "whoami": 0, 2024-04-24T23:29:37.209 INFO:teuthology.orchestra.run.smithi080.stdout: "id": 24496, 2024-04-24T23:29:37.209 INFO:teuthology.orchestra.run.smithi080.stdout: "want_state": "up:active", 2024-04-24T23:29:37.209 INFO:teuthology.orchestra.run.smithi080.stdout: "state": "up:active", 2024-04-24T23:29:37.209 INFO:teuthology.orchestra.run.smithi080.stdout: "fs_name": "cephfs", 2024-04-24T23:29:37.209 INFO:teuthology.orchestra.run.smithi080.stdout: "rank_uptime": 137.01211149299999, 2024-04-24T23:29:37.209 INFO:teuthology.orchestra.run.smithi080.stdout: "mdsmap_epoch": 24, 2024-04-24T23:29:37.209 INFO:teuthology.orchestra.run.smithi080.stdout: "osdmap_epoch": 94, 2024-04-24T23:29:37.209 INFO:teuthology.orchestra.run.smithi080.stdout: "osdmap_epoch_barrier": 90, 2024-04-24T23:29:37.209 INFO:teuthology.orchestra.run.smithi080.stdout: "uptime": 152.60497690700001 2024-04-24T23:29:37.209 INFO:teuthology.orchestra.run.smithi080.stdout:} 2024-04-24T23:29:37.210 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.199+0000 7f71a95a5640 1 -- 172.21.15.80:0/1233006120 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f718c0a2b90 msgr2=0x7f718c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:37.210 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.199+0000 7f71a95a5640 1 --2- 172.21.15.80:0/1233006120 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f718c0a2b90 0x7f718c0a5050 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f719c009460 tx=0x7f719c009980 comp rx=0 tx=0).stop 2024-04-24T23:29:37.210 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.199+0000 7f71a95a5640 1 -- 172.21.15.80:0/1233006120 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f71a4102980 msgr2=0x7f71a4198000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:37.210 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.199+0000 7f71a95a5640 1 --2- 172.21.15.80:0/1233006120 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f71a4102980 0x7f71a4198000 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f719800d950 tx=0x7f719800de10 comp rx=0 tx=0).stop 2024-04-24T23:29:37.211 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.199+0000 7f71a95a5640 1 -- 172.21.15.80:0/1233006120 shutdown_connections 2024-04-24T23:29:37.211 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.199+0000 7f71a95a5640 1 --2- 172.21.15.80:0/1233006120 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f718c0a2b90 0x7f718c0a5050 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:37.211 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.199+0000 7f71a95a5640 1 --2- 172.21.15.80:0/1233006120 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f71a4103340 0x7f71a419f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:37.211 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.199+0000 7f71a95a5640 1 --2- 172.21.15.80:0/1233006120 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f71a4102980 0x7f71a4198000 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:37.211 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.199+0000 7f71a95a5640 1 --2- 172.21.15.80:0/1233006120 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f71a4101790 0x7f71a4197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:37.211 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.199+0000 7f71a95a5640 1 -- 172.21.15.80:0/1233006120 >> 172.21.15.80:0/1233006120 conn(0x7f71a40fcd20 msgr2=0x7f71a4105ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:37.211 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.200+0000 7f71a95a5640 1 -- 172.21.15.80:0/1233006120 shutdown_connections 2024-04-24T23:29:37.211 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:37.200+0000 7f71a95a5640 1 -- 172.21.15.80:0/1233006120 wait complete. 2024-04-24T23:29:37.212 INFO:teuthology.run_tasks:Running task sequential... 2024-04-24T23:29:37.225 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-24T23:29:37.225 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-24T23:29:37.225 INFO:teuthology.task.print:Enabling mgr modules 2024-04-24T23:29:37.226 INFO:teuthology.run_tasks:Running task kclient... 2024-04-24T23:29:37.246 INFO:tasks.kclient:Mounting kernel clients... 2024-04-24T23:29:37.247 INFO:tasks.kclient:config is {'client.0': {}} 2024-04-24T23:29:37.247 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:29:37.247 DEBUG:teuthology.orchestra.run.smithi080:> ip netns list 2024-04-24T23:29:37.263 INFO:teuthology.orchestra.run.smithi080.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-04-24T23:29:37.264 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:29:37.264 DEBUG:teuthology.orchestra.run.smithi080:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-24T23:29:37.328 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:29:37.328 DEBUG:teuthology.orchestra.run.smithi080:> sudo ip link delete ceph-brx 2024-04-24T23:29:37.418 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=secure', 'wsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2024-04-24T23:29:37.418 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-24T23:29:37.418 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-24T23:29:37.419 DEBUG:teuthology.orchestra.run.smithi080:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid ec7885aa-0290-11ef-bc93-c7b262605968 -- ceph fs ls 2024-04-24T23:29:37.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:29:37 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/339848005' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T23:29:37.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:29:37 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/592350597' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:29:37.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:29:37 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/79943504' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:29:37.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:29:37 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/339848005' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T23:29:37.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:29:37 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/592350597' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:29:37.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:29:37 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/79943504' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:29:37.657 INFO:teuthology.orchestra.run.smithi080.stderr:Inferring config /var/lib/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:29:38.038 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:29:38 smithi080 ceph-mon[29152]: pgmap v369: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:29:38.446 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:29:38 smithi134 ceph-mon[32619]: pgmap v369: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:29:38.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:29:38 smithi148 ceph-mon[32528]: pgmap v369: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:29:39.387 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:39.386+0000 7f42ca471700 1 -- 172.21.15.80:0/1675144982 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f42c41071a0 msgr2=0x7f42c4107580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:39.388 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:39.386+0000 7f42ca471700 1 --2- 172.21.15.80:0/1675144982 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f42c41071a0 0x7f42c4107580 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f42b4009990 tx=0x7f42b4009ca0 comp rx=0 tx=0).stop 2024-04-24T23:29:39.388 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:39.386+0000 7f42ca471700 1 -- 172.21.15.80:0/1675144982 shutdown_connections 2024-04-24T23:29:39.388 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:39.386+0000 7f42ca471700 1 --2- 172.21.15.80:0/1675144982 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f42c4101af0 0x7f42c410dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:39.388 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:39.386+0000 7f42ca471700 1 --2- 172.21.15.80:0/1675144982 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f42c41071a0 0x7f42c4107580 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:39.388 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:39.386+0000 7f42ca471700 1 --2- 172.21.15.80:0/1675144982 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42c4101130 0x7f42c41015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:39.388 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:39.386+0000 7f42ca471700 1 -- 172.21.15.80:0/1675144982 >> 172.21.15.80:0/1675144982 conn(0x7f42c40780e0 msgr2=0x7f42c40784f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:39.388 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:39.386+0000 7f42ca471700 1 -- 172.21.15.80:0/1675144982 shutdown_connections 2024-04-24T23:29:39.388 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:39.386+0000 7f42ca471700 1 -- 172.21.15.80:0/1675144982 wait complete. 2024-04-24T23:29:39.390 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:39.387+0000 7f42ca471700 1 Processor -- start 2024-04-24T23:29:39.390 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:39.388+0000 7f42ca471700 1 -- start start 2024-04-24T23:29:39.390 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:39.388+0000 7f42ca471700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42c4101130 0x7f42c4199840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:39.390 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:39.388+0000 7f42ca471700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f42c4101af0 0x7f42c4199d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:39.390 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:39.388+0000 7f42ca471700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f42c41071a0 0x7f42c41a48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:39.391 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:39.388+0000 7f42ca471700 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f42c4111440 con 0x7f42c41071a0 2024-04-24T23:29:39.391 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:39.388+0000 7f42ca471700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f42c41112c0 con 0x7f42c4101130 2024-04-24T23:29:39.391 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:39.388+0000 7f42ca471700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f42c41115c0 con 0x7f42c4101af0 2024-04-24T23:29:39.391 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:39.388+0000 7f42c8a0e700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f42c41071a0 0x7f42c41a48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:39.391 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:39.388+0000 7f42c8a0e700 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f42c41071a0 0x7f42c41a48c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:54008/0 (socket says 172.21.15.80:54008) 2024-04-24T23:29:39.391 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:39.388+0000 7f42c8a0e700 1 -- 172.21.15.80:0/971736271 learned_addr learned my addr 172.21.15.80:0/971736271 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:29:39.391 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:39.388+0000 7f42c3fff700 1 --2- 172.21.15.80:0/971736271 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42c4101130 0x7f42c4199840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:39.391 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:39.388+0000 7f42c37fe700 1 --2- 172.21.15.80:0/971736271 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f42c4101af0 0x7f42c4199d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:39.391 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:39.388+0000 7f42c8a0e700 1 -- 172.21.15.80:0/971736271 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f42c4101af0 msgr2=0x7f42c4199d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:39.391 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:39.388+0000 7f42c8a0e700 1 --2- 172.21.15.80:0/971736271 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f42c4101af0 0x7f42c4199d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:39.391 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:39.388+0000 7f42c8a0e700 1 -- 172.21.15.80:0/971736271 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42c4101130 msgr2=0x7f42c4199840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:39.391 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:39.388+0000 7f42c8a0e700 1 --2- 172.21.15.80:0/971736271 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42c4101130 0x7f42c4199840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:39.392 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:39.389+0000 7f42c8a0e700 1 -- 172.21.15.80:0/971736271 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42c41a4e00 con 0x7f42c41071a0 2024-04-24T23:29:39.392 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:39.389+0000 7f42c8a0e700 1 --2- 172.21.15.80:0/971736271 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f42c41071a0 0x7f42c41a48c0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f42bc00ada0 tx=0x7f42bc00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:39.392 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:39.389+0000 7f42c17fa700 1 -- 172.21.15.80:0/971736271 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42bc017020 con 0x7f42c41071a0 2024-04-24T23:29:39.392 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:39.389+0000 7f42c17fa700 1 -- 172.21.15.80:0/971736271 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f42bc00cdb0 con 0x7f42c41071a0 2024-04-24T23:29:39.392 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:39.389+0000 7f42ca471700 1 -- 172.21.15.80:0/971736271 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42c41a5030 con 0x7f42c41071a0 2024-04-24T23:29:39.392 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:39.389+0000 7f42c17fa700 1 -- 172.21.15.80:0/971736271 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42bc0053e0 con 0x7f42c41071a0 2024-04-24T23:29:39.392 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:39.389+0000 7f42ca471700 1 -- 172.21.15.80:0/971736271 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42c41a54f0 con 0x7f42c41071a0 2024-04-24T23:29:39.393 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:39.390+0000 7f42ca471700 1 -- 172.21.15.80:0/971736271 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42c4068f20 con 0x7f42c41071a0 2024-04-24T23:29:39.393 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:39.391+0000 7f42c17fa700 1 -- 172.21.15.80:0/971736271 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f42bc00c8d0 con 0x7f42c41071a0 2024-04-24T23:29:39.393 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:39.391+0000 7f42c17fa700 1 --2- 172.21.15.80:0/971736271 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f42ac0a2c20 0x7f42ac0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:39.394 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:39.392+0000 7f42c3fff700 1 --2- 172.21.15.80:0/971736271 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f42ac0a2c20 0x7f42ac0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:39.394 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:39.392+0000 7f42c17fa700 1 -- 172.21.15.80:0/971736271 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f42bc0ceac0 con 0x7f42c41071a0 2024-04-24T23:29:39.394 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:39.392+0000 7f42c3fff700 1 --2- 172.21.15.80:0/971736271 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f42ac0a2c20 0x7f42ac0a50e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f42b400b5c0 tx=0x7f42b4005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:39.397 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:39.396+0000 7f42c17fa700 1 -- 172.21.15.80:0/971736271 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f42bc021020 con 0x7f42c41071a0 2024-04-24T23:29:39.642 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:39.641+0000 7f42ca471700 1 -- 172.21.15.80:0/971736271 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f42c402db30 con 0x7f42c41071a0 2024-04-24T23:29:39.642 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:39.641+0000 7f42c17fa700 1 -- 172.21.15.80:0/971736271 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v24) ==== 53+0+88 (secure 0 0 0) 0x7f42bc09b6e0 con 0x7f42c41071a0 2024-04-24T23:29:39.642 INFO:teuthology.orchestra.run.smithi080.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-24T23:29:39.645 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:39.643+0000 7f42ca471700 1 -- 172.21.15.80:0/971736271 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f42ac0a2c20 msgr2=0x7f42ac0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:39.645 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:39.643+0000 7f42ca471700 1 --2- 172.21.15.80:0/971736271 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f42ac0a2c20 0x7f42ac0a50e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f42b400b5c0 tx=0x7f42b4005960 comp rx=0 tx=0).stop 2024-04-24T23:29:39.646 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:39.643+0000 7f42ca471700 1 -- 172.21.15.80:0/971736271 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f42c41071a0 msgr2=0x7f42c41a48c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:39.646 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:39.643+0000 7f42ca471700 1 --2- 172.21.15.80:0/971736271 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f42c41071a0 0x7f42c41a48c0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f42bc00ada0 tx=0x7f42bc00c330 comp rx=0 tx=0).stop 2024-04-24T23:29:39.646 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:39.643+0000 7f42ca471700 1 -- 172.21.15.80:0/971736271 shutdown_connections 2024-04-24T23:29:39.646 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:39.643+0000 7f42ca471700 1 --2- 172.21.15.80:0/971736271 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f42ac0a2c20 0x7f42ac0a50e0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:39.646 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:39.643+0000 7f42ca471700 1 --2- 172.21.15.80:0/971736271 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f42c41071a0 0x7f42c41a48c0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:39.646 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:39.643+0000 7f42ca471700 1 --2- 172.21.15.80:0/971736271 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42c4101130 0x7f42c4199840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:39.646 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:39.643+0000 7f42ca471700 1 --2- 172.21.15.80:0/971736271 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f42c4101af0 0x7f42c4199d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:39.646 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:39.643+0000 7f42ca471700 1 -- 172.21.15.80:0/971736271 >> 172.21.15.80:0/971736271 conn(0x7f42c40780e0 msgr2=0x7f42c40ffe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:39.646 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:39.643+0000 7f42ca471700 1 -- 172.21.15.80:0/971736271 shutdown_connections 2024-04-24T23:29:39.646 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:39.643+0000 7f42ca471700 1 -- 172.21.15.80:0/971736271 wait complete. 2024-04-24T23:29:40.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:29:40 smithi080 ceph-mon[29152]: pgmap v370: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T23:29:40.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:29:40 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/971736271' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-24T23:29:40.308 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:29:40 smithi148 ceph-mon[32528]: pgmap v370: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T23:29:40.309 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:29:40 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/971736271' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-24T23:29:40.354 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-24T23:29:40.354 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-24T23:29:40.354 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi080.front.sepia.ceph.com 2024-04-24T23:29:40.354 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-24T23:29:40.354 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-24T23:29:40.354 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-24T23:29:40.354 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-24T23:29:40.354 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-24T23:29:40.354 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-24T23:29:40.355 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:29:40.355 DEBUG:teuthology.orchestra.run.smithi080:> ip addr 2024-04-24T23:29:40.371 INFO:teuthology.orchestra.run.smithi080.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-24T23:29:40.371 INFO:teuthology.orchestra.run.smithi080.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-24T23:29:40.371 INFO:teuthology.orchestra.run.smithi080.stdout: inet 127.0.0.1/8 scope host lo 2024-04-24T23:29:40.371 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:29:40.372 INFO:teuthology.orchestra.run.smithi080.stdout: inet6 ::1/128 scope host 2024-04-24T23:29:40.372 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:29:40.372 INFO:teuthology.orchestra.run.smithi080.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:29:40.372 INFO:teuthology.orchestra.run.smithi080.stdout: link/ether 0c:c4:7a:d8:d9:b2 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:29:40.372 INFO:teuthology.orchestra.run.smithi080.stdout: altname enp7s0f0 2024-04-24T23:29:40.372 INFO:teuthology.orchestra.run.smithi080.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:29:40.372 INFO:teuthology.orchestra.run.smithi080.stdout: link/ether 0c:c4:7a:d8:d9:b3 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:29:40.372 INFO:teuthology.orchestra.run.smithi080.stdout: altname enp7s0f1 2024-04-24T23:29:40.372 INFO:teuthology.orchestra.run.smithi080.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:29:40.372 INFO:teuthology.orchestra.run.smithi080.stdout: link/ether 0c:c4:7a:8f:cb:a8 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:29:40.372 INFO:teuthology.orchestra.run.smithi080.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-24T23:29:40.372 INFO:teuthology.orchestra.run.smithi080.stdout: link/ether 0c:c4:7a:8f:cb:a9 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:29:40.372 INFO:teuthology.orchestra.run.smithi080.stdout: inet 172.21.15.80/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-24T23:29:40.372 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft 42112sec preferred_lft 42112sec 2024-04-24T23:29:40.373 INFO:teuthology.orchestra.run.smithi080.stdout: inet6 fe80::477b:93ee:6d5c:3d53/64 scope link noprefixroute 2024-04-24T23:29:40.373 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:29:40.373 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-24T23:29:40.373 DEBUG:teuthology.orchestra.run.smithi080:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T23:29:40.373 DEBUG:teuthology.orchestra.run.smithi080:> set -e 2024-04-24T23:29:40.373 DEBUG:teuthology.orchestra.run.smithi080:> sudo ip link add name ceph-brx type bridge 2024-04-24T23:29:40.373 DEBUG:teuthology.orchestra.run.smithi080:> sudo ip addr flush dev ceph-brx 2024-04-24T23:29:40.373 DEBUG:teuthology.orchestra.run.smithi080:> sudo ip link set ceph-brx up 2024-04-24T23:29:40.373 DEBUG:teuthology.orchestra.run.smithi080:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-24T23:29:40.374 DEBUG:teuthology.orchestra.run.smithi080:> ') 2024-04-24T23:29:40.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:29:40 smithi134 ceph-mon[32619]: pgmap v370: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T23:29:40.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:29:40 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/971736271' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-24T23:29:40.459 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:29:40.501 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:29:40.509 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:29:40.509 DEBUG:teuthology.orchestra.run.smithi080:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-24T23:29:40.531 INFO:teuthology.orchestra.run.smithi080.stdout:1 2024-04-24T23:29:40.532 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:29:40.532 DEBUG:teuthology.orchestra.run.smithi080:> route 2024-04-24T23:29:40.588 INFO:teuthology.orchestra.run.smithi080.stdout:Kernel IP routing table 2024-04-24T23:29:40.588 INFO:teuthology.orchestra.run.smithi080.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-24T23:29:40.588 INFO:teuthology.orchestra.run.smithi080.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-24T23:29:40.588 INFO:teuthology.orchestra.run.smithi080.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-24T23:29:40.588 INFO:teuthology.orchestra.run.smithi080.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-24T23:29:40.589 DEBUG:teuthology.orchestra.run.smithi080:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T23:29:40.589 DEBUG:teuthology.orchestra.run.smithi080:> set -e 2024-04-24T23:29:40.589 DEBUG:teuthology.orchestra.run.smithi080:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-24T23:29:40.589 DEBUG:teuthology.orchestra.run.smithi080:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-24T23:29:40.589 DEBUG:teuthology.orchestra.run.smithi080:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-24T23:29:40.589 DEBUG:teuthology.orchestra.run.smithi080:> ') 2024-04-24T23:29:40.677 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:29:40.707 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:29:40.713 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:29:40.714 DEBUG:teuthology.orchestra.run.smithi080:> ip netns list 2024-04-24T23:29:40.769 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:29:40.769 DEBUG:teuthology.orchestra.run.smithi080:> ip netns list-id 2024-04-24T23:29:40.828 DEBUG:teuthology.orchestra.run.smithi080:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T23:29:40.828 DEBUG:teuthology.orchestra.run.smithi080:> set -e 2024-04-24T23:29:40.828 DEBUG:teuthology.orchestra.run.smithi080:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T23:29:40.828 DEBUG:teuthology.orchestra.run.smithi080:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-24T23:29:40.828 DEBUG:teuthology.orchestra.run.smithi080:> ') 2024-04-24T23:29:40.917 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:29:40.938 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:29:40.944 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-24T23:29:40.944 DEBUG:teuthology.orchestra.run.smithi080:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T23:29:40.944 DEBUG:teuthology.orchestra.run.smithi080:> set -e 2024-04-24T23:29:40.944 DEBUG:teuthology.orchestra.run.smithi080:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-24T23:29:40.945 DEBUG:teuthology.orchestra.run.smithi080:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-24T23:29:40.945 DEBUG:teuthology.orchestra.run.smithi080:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-24T23:29:40.945 DEBUG:teuthology.orchestra.run.smithi080:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-24T23:29:40.945 DEBUG:teuthology.orchestra.run.smithi080:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-24T23:29:40.945 DEBUG:teuthology.orchestra.run.smithi080:> ') 2024-04-24T23:29:40.993 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:29:41.048 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:29:41.055 DEBUG:teuthology.orchestra.run.smithi080:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T23:29:41.056 DEBUG:teuthology.orchestra.run.smithi080:> set -e 2024-04-24T23:29:41.056 DEBUG:teuthology.orchestra.run.smithi080:> sudo ip link set brx.0 up 2024-04-24T23:29:41.056 DEBUG:teuthology.orchestra.run.smithi080:> sudo ip link set dev brx.0 master ceph-brx 2024-04-24T23:29:41.056 DEBUG:teuthology.orchestra.run.smithi080:> ') 2024-04-24T23:29:41.104 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:41 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:29:41.125 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:29:41.135 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T23:29:41.135 DEBUG:teuthology.orchestra.run.smithi080:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-24T23:29:41.152 INFO:teuthology.orchestra.run.smithi080.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-24T23:29:41.152 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T23:29:41.153 DEBUG:teuthology.orchestra.run.smithi080:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-24T23:29:41.207 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-04-24T23:29:41.207 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-04-24T23:29:41.207 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/521b5c7e-1c80-4acd-899a-7184a5da80b8 2024-04-24T23:29:41.207 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:29:41.207 DEBUG:teuthology.orchestra.run.smithi080:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/521b5c7e-1c80-4acd-899a-7184a5da80b8 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync 2024-04-24T23:29:41.288 INFO:teuthology.orchestra.run.smithi080.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync 2024-04-24T23:29:41.289 INFO:teuthology.orchestra.run.smithi080.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync". 2024-04-24T23:29:41.289 INFO:teuthology.orchestra.run.smithi080.stdout:invalid new device string format 2024-04-24T23:29:41.417 INFO:teuthology.orchestra.run.smithi080.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync 2024-04-24T23:29:41.417 INFO:teuthology.orchestra.run.smithi080.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync". 2024-04-24T23:29:41.417 INFO:teuthology.orchestra.run.smithi080.stdout:invalid new device string format 2024-04-24T23:29:41.417 INFO:teuthology.orchestra.run.smithi080.stdout:mount.ceph: resolved to: "172.21.15.80:3300,172.21.15.134:3300,172.21.15.148:3300" 2024-04-24T23:29:41.417 INFO:teuthology.orchestra.run.smithi080.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.80:3300,172.21.15.134:3300,172.21.15.148:3300:/volumes/qa/sv_1/521b5c7e-1c80-4acd-899a-7184a5da80b8 2024-04-24T23:29:41.418 INFO:teuthology.orchestra.run.smithi080.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync,key=0,fsid=ec7885aa-0290-11ef-bc93-c7b262605968" will pass to kernel 2024-04-24T23:29:41.418 INFO:teuthology.orchestra.run.smithi080.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-24T23:29:41.418 INFO:teuthology.orchestra.run.smithi080.stdout: You just mounted a file system that supports labels which does not 2024-04-24T23:29:41.418 INFO:teuthology.orchestra.run.smithi080.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-24T23:29:41.418 INFO:teuthology.orchestra.run.smithi080.stdout: applications will generate AVC messages and not be allowed access to 2024-04-24T23:29:41.418 INFO:teuthology.orchestra.run.smithi080.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-24T23:29:41.419 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-24T23:29:41.419 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:29:41.419 DEBUG:teuthology.orchestra.run.smithi080:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-24T23:29:41.489 DEBUG:teuthology.orchestra.run.smithi080:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-24T23:29:41.578 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:41 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-24T23:29:41.580 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:29:41.586 INFO:teuthology.orchestra.run.smithi080.stdout:client15474 2024-04-24T23:29:41.586 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-24T23:29:41.586 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:29:41.603 DEBUG:teuthology.orchestra.run.smithi080:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-24T23:29:41.691 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:41 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-24T23:29:41.694 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:29:41.700 INFO:teuthology.orchestra.run.smithi080.stdout:client15474 2024-04-24T23:29:41.700 DEBUG:teuthology.orchestra.run.smithi080:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-24T23:29:41.748 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:41 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-24T23:29:41.750 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:29:41.756 INFO:teuthology.orchestra.run.smithi080.stdout:client15474 2024-04-24T23:29:41.756 DEBUG:teuthology.orchestra.run.smithi080:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/ec7885aa-0290-11ef-bc93-c7b262605968.client15474/status') 2024-04-24T23:29:41.804 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:41 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:29:41.813 INFO:teuthology.orchestra.run.smithi080.stdout:instance: client.15474 (3)192.168.0.1:0/875212370 2024-04-24T23:29:41.813 INFO:teuthology.orchestra.run.smithi080.stdout:blocklisted: false 2024-04-24T23:29:41.813 INFO:teuthology.orchestra.run.smithi080.stderr:0+1 records in 2024-04-24T23:29:41.813 INFO:teuthology.orchestra.run.smithi080.stderr:0+1 records out 2024-04-24T23:29:41.813 INFO:teuthology.orchestra.run.smithi080.stderr:69 bytes copied, 2.2163e-05 s, 3.1 MB/s 2024-04-24T23:29:41.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:29:41.820 INFO:teuthology.run_tasks:Running task exec... 2024-04-24T23:29:41.832 INFO:teuthology.task.exec:Executing custom commands... 2024-04-24T23:29:41.833 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi080.front.sepia.ceph.com 2024-04-24T23:29:41.833 DEBUG:teuthology.orchestra.run.smithi080:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-04-24T23:29:41.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:41.932+0000 7f89d8da9640 1 -- 172.21.15.80:0/2366385102 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f89d4107590 msgr2=0x7f89d4109980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:41.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:41.932+0000 7f89d8da9640 1 --2- 172.21.15.80:0/2366385102 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f89d4107590 0x7f89d4109980 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f89cc009ca0 tx=0x7f89cc02a8c0 comp rx=0 tx=0).stop 2024-04-24T23:29:41.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:41.932+0000 7f89d8da9640 1 -- 172.21.15.80:0/2366385102 shutdown_connections 2024-04-24T23:29:41.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:41.932+0000 7f89d8da9640 1 --2- 172.21.15.80:0/2366385102 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f89d4107590 0x7f89d4109980 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:41.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:41.932+0000 7f89d8da9640 1 --2- 172.21.15.80:0/2366385102 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f89d4104c60 0x7f89d4107050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:41.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:41.932+0000 7f89d8da9640 1 --2- 172.21.15.80:0/2366385102 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f89d4102330 0x7f89d4104720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:41.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:41.932+0000 7f89d8da9640 1 -- 172.21.15.80:0/2366385102 >> 172.21.15.80:0/2366385102 conn(0x7f89d40fbcc0 msgr2=0x7f89d40fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:41.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:41.932+0000 7f89d8da9640 1 -- 172.21.15.80:0/2366385102 shutdown_connections 2024-04-24T23:29:41.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:41.932+0000 7f89d8da9640 1 -- 172.21.15.80:0/2366385102 wait complete. 2024-04-24T23:29:41.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:41.933+0000 7f89d8da9640 1 Processor -- start 2024-04-24T23:29:41.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:41.933+0000 7f89d8da9640 1 -- start start 2024-04-24T23:29:41.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:41.933+0000 7f89d8da9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f89d4102330 0x7f89d4197990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:41.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:41.933+0000 7f89d8da9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f89d4104c60 0x7f89d4197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:41.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:41.933+0000 7f89d8da9640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f89d4107590 0x7f89d419ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:41.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:41.933+0000 7f89d8da9640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f89d410d480 con 0x7f89d4107590 2024-04-24T23:29:41.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:41.933+0000 7f89d8da9640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f89d410d300 con 0x7f89d4104c60 2024-04-24T23:29:41.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:41.933+0000 7f89d8da9640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f89d410d600 con 0x7f89d4102330 2024-04-24T23:29:41.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:41.933+0000 7f89d2d9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f89d4107590 0x7f89d419ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:41.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:41.933+0000 7f89d2d9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f89d4107590 0x7f89d419ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:54032/0 (socket says 172.21.15.80:54032) 2024-04-24T23:29:41.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:41.933+0000 7f89d2d9d640 1 -- 172.21.15.80:0/1820105275 learned_addr learned my addr 172.21.15.80:0/1820105275 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:29:41.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:41.933+0000 7f89d259c640 1 --2- 172.21.15.80:0/1820105275 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f89d4102330 0x7f89d4197990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:41.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:41.933+0000 7f89cbfff640 1 --2- 172.21.15.80:0/1820105275 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f89d4104c60 0x7f89d4197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:41.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:41.933+0000 7f89d2d9d640 1 -- 172.21.15.80:0/1820105275 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f89d4102330 msgr2=0x7f89d4197990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:41.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:41.933+0000 7f89d2d9d640 1 --2- 172.21.15.80:0/1820105275 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f89d4102330 0x7f89d4197990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:41.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:41.933+0000 7f89d2d9d640 1 -- 172.21.15.80:0/1820105275 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f89d4104c60 msgr2=0x7f89d4197ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:41.937 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:41.933+0000 7f89d2d9d640 1 --2- 172.21.15.80:0/1820105275 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f89d4104c60 0x7f89d4197ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:41.937 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:41.933+0000 7f89d2d9d640 1 -- 172.21.15.80:0/1820105275 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89d419f5b0 con 0x7f89d4107590 2024-04-24T23:29:41.937 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:41.933+0000 7f89d2d9d640 1 --2- 172.21.15.80:0/1820105275 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f89d4107590 0x7f89d419ef40 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f89cc005e30 tx=0x7f89cc031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:41.937 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:41.934+0000 7f89cb7fe640 1 -- 172.21.15.80:0/1820105275 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89cc003c30 con 0x7f89d4107590 2024-04-24T23:29:41.937 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:41.934+0000 7f89cb7fe640 1 -- 172.21.15.80:0/1820105275 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89cc003dd0 con 0x7f89d4107590 2024-04-24T23:29:41.937 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:41.934+0000 7f89d8da9640 1 -- 172.21.15.80:0/1820105275 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89d419f840 con 0x7f89d4107590 2024-04-24T23:29:41.937 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:41.934+0000 7f89cb7fe640 1 -- 172.21.15.80:0/1820105275 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89cc02fde0 con 0x7f89d4107590 2024-04-24T23:29:41.937 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:41.934+0000 7f89d8da9640 1 -- 172.21.15.80:0/1820105275 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89d419fd90 con 0x7f89d4107590 2024-04-24T23:29:41.938 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:41.935+0000 7f89d8da9640 1 -- 172.21.15.80:0/1820105275 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89d40fd7b0 con 0x7f89d4107590 2024-04-24T23:29:41.938 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:41.936+0000 7f89cb7fe640 1 -- 172.21.15.80:0/1820105275 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f89cc004050 con 0x7f89d4107590 2024-04-24T23:29:41.938 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:41.936+0000 7f89cb7fe640 1 --2- 172.21.15.80:0/1820105275 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f89c00a2c30 0x7f89c00a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:41.938 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:41.936+0000 7f89d259c640 1 --2- 172.21.15.80:0/1820105275 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f89c00a2c30 0x7f89c00a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:41.938 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:41.936+0000 7f89cb7fe640 1 -- 172.21.15.80:0/1820105275 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f89cc0f0830 con 0x7f89d4107590 2024-04-24T23:29:41.938 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:41.937+0000 7f89d259c640 1 --2- 172.21.15.80:0/1820105275 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f89c00a2c30 0x7f89c00a50f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f89c4005fd0 tx=0x7f89c400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:41.941 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:41.941+0000 7f89cb7fe640 1 -- 172.21.15.80:0/1820105275 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f89cc0bd3c0 con 0x7f89d4107590 2024-04-24T23:29:42.157 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:42.157+0000 7f89d8da9640 1 -- 172.21.15.80:0/1820105275 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7f89d4062340 con 0x7f89d4107590 2024-04-24T23:29:42.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:29:42 smithi080 ceph-mon[29152]: pgmap v371: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T23:29:42.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:29:42 smithi134 ceph-mon[32619]: pgmap v371: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T23:29:42.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:29:42 smithi148 ceph-mon[32528]: pgmap v371: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T23:29:43.019 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.018+0000 7f89cb7fe640 1 -- 172.21.15.80:0/1820105275 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v25) ==== 117+0+0 (secure 0 0 0) 0x7f89cc0bccc0 con 0x7f89d4107590 2024-04-24T23:29:43.022 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.021+0000 7f89d8da9640 1 -- 172.21.15.80:0/1820105275 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f89c00a2c30 msgr2=0x7f89c00a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:43.022 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.021+0000 7f89d8da9640 1 --2- 172.21.15.80:0/1820105275 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f89c00a2c30 0x7f89c00a50f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f89c4005fd0 tx=0x7f89c400b390 comp rx=0 tx=0).stop 2024-04-24T23:29:43.022 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.021+0000 7f89d8da9640 1 -- 172.21.15.80:0/1820105275 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f89d4107590 msgr2=0x7f89d419ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:43.022 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.021+0000 7f89d8da9640 1 --2- 172.21.15.80:0/1820105275 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f89d4107590 0x7f89d419ef40 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f89cc005e30 tx=0x7f89cc031ee0 comp rx=0 tx=0).stop 2024-04-24T23:29:43.022 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.021+0000 7f89d8da9640 1 -- 172.21.15.80:0/1820105275 shutdown_connections 2024-04-24T23:29:43.022 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.021+0000 7f89d8da9640 1 --2- 172.21.15.80:0/1820105275 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f89c00a2c30 0x7f89c00a50f0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:43.022 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.021+0000 7f89d8da9640 1 --2- 172.21.15.80:0/1820105275 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f89d4107590 0x7f89d419ef40 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:43.022 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.021+0000 7f89d8da9640 1 --2- 172.21.15.80:0/1820105275 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f89d4104c60 0x7f89d4197ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:43.022 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.021+0000 7f89d8da9640 1 --2- 172.21.15.80:0/1820105275 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f89d4102330 0x7f89d4197990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:43.022 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.021+0000 7f89d8da9640 1 -- 172.21.15.80:0/1820105275 >> 172.21.15.80:0/1820105275 conn(0x7f89d40fbcc0 msgr2=0x7f89d4102c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:43.023 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.021+0000 7f89d8da9640 1 -- 172.21.15.80:0/1820105275 shutdown_connections 2024-04-24T23:29:43.023 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.021+0000 7f89d8da9640 1 -- 172.21.15.80:0/1820105275 wait complete. 2024-04-24T23:29:43.034 INFO:teuthology.run_tasks:Running task check-counter... 2024-04-24T23:29:43.053 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-04-24T23:29:43.073 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:29:43.176 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.173+0000 7f939886d640 1 -- 172.21.15.80:0/554284783 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9394075720 msgr2=0x7f9394073b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:43.176 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.173+0000 7f939886d640 1 --2- 172.21.15.80:0/554284783 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9394075720 0x7f9394073b70 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f937c009900 tx=0x7f937c02a480 comp rx=0 tx=0).stop 2024-04-24T23:29:43.176 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.173+0000 7f939886d640 1 -- 172.21.15.80:0/554284783 shutdown_connections 2024-04-24T23:29:43.176 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.173+0000 7f939886d640 1 --2- 172.21.15.80:0/554284783 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9394109640 0x7f939410bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:43.176 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.173+0000 7f939886d640 1 --2- 172.21.15.80:0/554284783 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9394074140 0x7f93940745a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:43.176 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.173+0000 7f939886d640 1 --2- 172.21.15.80:0/554284783 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9394075720 0x7f9394073b70 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:43.176 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.173+0000 7f939886d640 1 -- 172.21.15.80:0/554284783 >> 172.21.15.80:0/554284783 conn(0x7f93940fcd40 msgr2=0x7f93940ff160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:43.176 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.173+0000 7f939886d640 1 -- 172.21.15.80:0/554284783 shutdown_connections 2024-04-24T23:29:43.176 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.173+0000 7f939886d640 1 -- 172.21.15.80:0/554284783 wait complete. 2024-04-24T23:29:43.177 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.174+0000 7f939886d640 1 Processor -- start 2024-04-24T23:29:43.177 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.174+0000 7f939886d640 1 -- start start 2024-04-24T23:29:43.177 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.174+0000 7f939886d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9394074140 0x7f9394199d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:43.177 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.174+0000 7f939886d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9394075720 0x7f939419a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:43.177 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.174+0000 7f939886d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9394109640 0x7f93941a1270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:43.177 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.174+0000 7f939886d640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f939410f680 con 0x7f9394074140 2024-04-24T23:29:43.177 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.174+0000 7f939886d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f939410f500 con 0x7f9394075720 2024-04-24T23:29:43.177 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.174+0000 7f939886d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f939410f800 con 0x7f9394109640 2024-04-24T23:29:43.177 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.174+0000 7f939259c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9394074140 0x7f9394199d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:43.177 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.174+0000 7f939259c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9394074140 0x7f9394199d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:51812/0 (socket says 172.21.15.80:51812) 2024-04-24T23:29:43.177 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.174+0000 7f939259c640 1 -- 172.21.15.80:0/3677435279 learned_addr learned my addr 172.21.15.80:0/3677435279 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:29:43.177 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.174+0000 7f9392d9d640 1 --2- 172.21.15.80:0/3677435279 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9394109640 0x7f93941a1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:43.178 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.174+0000 7f9391d9b640 1 --2- 172.21.15.80:0/3677435279 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9394075720 0x7f939419a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:43.178 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.174+0000 7f939259c640 1 -- 172.21.15.80:0/3677435279 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9394109640 msgr2=0x7f93941a1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:43.178 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.174+0000 7f939259c640 1 --2- 172.21.15.80:0/3677435279 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9394109640 0x7f93941a1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:43.178 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.174+0000 7f939259c640 1 -- 172.21.15.80:0/3677435279 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9394075720 msgr2=0x7f939419a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:43.178 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.174+0000 7f939259c640 1 --2- 172.21.15.80:0/3677435279 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9394075720 0x7f939419a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:43.178 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.174+0000 7f939259c640 1 -- 172.21.15.80:0/3677435279 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93941a1910 con 0x7f9394074140 2024-04-24T23:29:43.179 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.174+0000 7f939259c640 1 --2- 172.21.15.80:0/3677435279 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9394074140 0x7f9394199d00 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f937c009230 tx=0x7f937c004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:43.179 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.174+0000 7f938b7fe640 1 -- 172.21.15.80:0/3677435279 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f937c03f030 con 0x7f9394074140 2024-04-24T23:29:43.179 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.175+0000 7f938b7fe640 1 -- 172.21.15.80:0/3677435279 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f937c003ce0 con 0x7f9394074140 2024-04-24T23:29:43.179 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.175+0000 7f939886d640 1 -- 172.21.15.80:0/3677435279 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93941a1ba0 con 0x7f9394074140 2024-04-24T23:29:43.179 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.175+0000 7f938b7fe640 1 -- 172.21.15.80:0/3677435279 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f937c035070 con 0x7f9394074140 2024-04-24T23:29:43.179 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.175+0000 7f939886d640 1 -- 172.21.15.80:0/3677435279 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93941a2090 con 0x7f9394074140 2024-04-24T23:29:43.180 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.175+0000 7f939886d640 1 -- 172.21.15.80:0/3677435279 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9394108400 con 0x7f9394074140 2024-04-24T23:29:43.181 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.177+0000 7f938b7fe640 1 -- 172.21.15.80:0/3677435279 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f937c042030 con 0x7f9394074140 2024-04-24T23:29:43.181 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.177+0000 7f938b7fe640 1 --2- 172.21.15.80:0/3677435279 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f93800a2b90 0x7f93800a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:43.181 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.177+0000 7f9391d9b640 1 --2- 172.21.15.80:0/3677435279 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f93800a2b90 0x7f93800a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:43.181 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.178+0000 7f938b7fe640 1 -- 172.21.15.80:0/3677435279 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f937c0efde0 con 0x7f9394074140 2024-04-24T23:29:43.181 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.178+0000 7f9391d9b640 1 --2- 172.21.15.80:0/3677435279 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f93800a2b90 0x7f93800a5050 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f938400bbe0 tx=0x7f938400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:43.182 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.182+0000 7f938b7fe640 1 -- 172.21.15.80:0/3677435279 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f937c0bc320 con 0x7f9394074140 2024-04-24T23:29:43.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:29:43 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/1820105275' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-04-24T23:29:43.361 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.360+0000 7f939886d640 1 -- 172.21.15.80:0/3677435279 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f93940623e0 con 0x7f9394074140 2024-04-24T23:29:43.363 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.362+0000 7f938b7fe640 1 -- 172.21.15.80:0/3677435279 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24839 (secure 0 0 0) 0x7f937c0c0620 con 0x7f9394074140 2024-04-24T23:29:43.363 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-24T23:29:43.363 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":94,"fsid":"ec7885aa-0290-11ef-bc93-c7b262605968","created":"2024-04-24T23:19:05.586407+0000","modified":"2024-04-24T23:29:08.209684+0000","last_up_change":"2024-04-24T23:26:25.938316+0000","last_in_change":"2024-04-24T23:26:04.786699+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:22:33.124428+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:27:17.575788+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:27:18.492762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T23:27:22.438503+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a525741d-b46c-4eef-aa64-3f057cac1497","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":2597718112},{"type":"v1","addr":"172.21.15.80:6803","nonce":2597718112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":2597718112},{"type":"v1","addr":"172.21.15.80:6805","nonce":2597718112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2597718112},{"type":"v1","addr":"172.21.15.80:6809","nonce":2597718112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":2597718112},{"type":"v1","addr":"172.21.15.80:6807","nonce":2597718112}]},"public_addr":"172.21.15.80:6803/2597718112","cluster_addr":"172.21.15.80:6805/2597718112","heartbeat_back_addr":"172.21.15.80:6809/2597718112","heartbeat_front_addr":"172.21.15.80:6807/2597718112","state":["exists","up"]},{"osd":1,"uuid":"2d0f0d67-6db4-40ac-91cc-58604ab6795a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":2395610201},{"type":"v1","addr":"172.21.15.134:6801","nonce":2395610201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":2395610201},{"type":"v1","addr":"172.21.15.134:6803","nonce":2395610201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":2395610201},{"type":"v1","addr":"172.21.15.134:6807","nonce":2395610201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":2395610201},{"type":"v1","addr":"172.21.15.134:6805","nonce":2395610201}]},"public_addr":"172.21.15.134:6801/2395610201","cluster_addr":"172.21.15.134:6803/2395610201","heartbeat_back_addr":"172.21.15.134:6807/2395610201","heartbeat_front_addr":"172.21.15.134:6805/2395610201","state":["exists","up"]},{"osd":2,"uuid":"781fb879-fedc-4c84-82fb-59e073bc4cf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1629152187},{"type":"v1","addr":"172.21.15.148:6801","nonce":1629152187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1629152187},{"type":"v1","addr":"172.21.15.148:6803","nonce":1629152187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1629152187},{"type":"v1","addr":"172.21.15.148:6807","nonce":1629152187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1629152187},{"type":"v1","addr":"172.21.15.148:6805","nonce":1629152187}]},"public_addr":"172.21.15.148:6801/1629152187","cluster_addr":"172.21.15.148:6803/1629152187","heartbeat_back_addr":"172.21.15.148:6807/1629152187","heartbeat_front_addr":"172.21.15.148:6805/1629152187","state":["exists","up"]},{"osd":3,"uuid":"cd982757-f2f6-4296-b3e7-4cd40fbd8d9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":3128298510},{"type":"v1","addr":"172.21.15.80:6811","nonce":3128298510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":3128298510},{"type":"v1","addr":"172.21.15.80:6813","nonce":3128298510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":3128298510},{"type":"v1","addr":"172.21.15.80:6817","nonce":3128298510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":3128298510},{"type":"v1","addr":"172.21.15.80:6815","nonce":3128298510}]},"public_addr":"172.21.15.80:6811/3128298510","cluster_addr":"172.21.15.80:6813/3128298510","heartbeat_back_addr":"172.21.15.80:6817/3128298510","heartbeat_front_addr":"172.21.15.80:6815/3128298510","state":["exists","up"]},{"osd":4,"uuid":"dfc55a79-a7e8-417a-b898-ec44171a54af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1917556060},{"type":"v1","addr":"172.21.15.134:6809","nonce":1917556060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1917556060},{"type":"v1","addr":"172.21.15.134:6811","nonce":1917556060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1917556060},{"type":"v1","addr":"172.21.15.134:6815","nonce":1917556060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1917556060},{"type":"v1","addr":"172.21.15.134:6813","nonce":1917556060}]},"public_addr":"172.21.15.134:6809/1917556060","cluster_addr":"172.21.15.134:6811/1917556060","heartbeat_back_addr":"172.21.15.134:6815/1917556060","heartbeat_front_addr":"172.21.15.134:6813/1917556060","state":["exists","up"]},{"osd":5,"uuid":"b5491337-0694-440c-b679-312f3d14cf65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":2715951281},{"type":"v1","addr":"172.21.15.148:6809","nonce":2715951281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":2715951281},{"type":"v1","addr":"172.21.15.148:6811","nonce":2715951281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":2715951281},{"type":"v1","addr":"172.21.15.148:6815","nonce":2715951281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":2715951281},{"type":"v1","addr":"172.21.15.148:6813","nonce":2715951281}]},"public_addr":"172.21.15.148:6809/2715951281","cluster_addr":"172.21.15.148:6811/2715951281","heartbeat_back_addr":"172.21.15.148:6815/2715951281","heartbeat_front_addr":"172.21.15.148:6813/2715951281","state":["exists","up"]},{"osd":6,"uuid":"ba4b5df7-d86e-47cd-ab67-cf763c4e4d6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":953261724},{"type":"v1","addr":"172.21.15.80:6819","nonce":953261724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":953261724},{"type":"v1","addr":"172.21.15.80:6821","nonce":953261724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":953261724},{"type":"v1","addr":"172.21.15.80:6825","nonce":953261724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":953261724},{"type":"v1","addr":"172.21.15.80:6823","nonce":953261724}]},"public_addr":"172.21.15.80:6819/953261724","cluster_addr":"172.21.15.80:6821/953261724","heartbeat_back_addr":"172.21.15.80:6825/953261724","heartbeat_front_addr":"172.21.15.80:6823/953261724","state":["exists","up"]},{"osd":7,"uuid":"464a503a-75ae-49e7-ab6e-a7f9f2a6a307","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":108354844},{"type":"v1","addr":"172.21.15.134:6817","nonce":108354844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":108354844},{"type":"v1","addr":"172.21.15.134:6819","nonce":108354844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":108354844},{"type":"v1","addr":"172.21.15.134:6823","nonce":108354844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":108354844},{"type":"v1","addr":"172.21.15.134:6821","nonce":108354844}]},"public_addr":"172.21.15.134:6817/108354844","cluster_addr":"172.21.15.134:6819/108354844","heartbeat_back_addr":"172.21.15.134:6823/108354844","heartbeat_front_addr":"172.21.15.134:6821/108354844","state":["exists","up"]},{"osd":8,"uuid":"ce544b2e-4540-47c0-9997-500c45351330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3559655329},{"type":"v1","addr":"172.21.15.148:6817","nonce":3559655329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":3559655329},{"type":"v1","addr":"172.21.15.148:6819","nonce":3559655329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":3559655329},{"type":"v1","addr":"172.21.15.148:6823","nonce":3559655329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3559655329},{"type":"v1","addr":"172.21.15.148:6821","nonce":3559655329}]},"public_addr":"172.21.15.148:6817/3559655329","cluster_addr":"172.21.15.148:6819/3559655329","heartbeat_back_addr":"172.21.15.148:6823/3559655329","heartbeat_front_addr":"172.21.15.148:6821/3559655329","state":["exists","up"]},{"osd":9,"uuid":"06de3732-19b9-496c-a464-55322efa6759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":4019913545},{"type":"v1","addr":"172.21.15.80:6827","nonce":4019913545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":4019913545},{"type":"v1","addr":"172.21.15.80:6829","nonce":4019913545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":4019913545},{"type":"v1","addr":"172.21.15.80:6833","nonce":4019913545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":4019913545},{"type":"v1","addr":"172.21.15.80:6831","nonce":4019913545}]},"public_addr":"172.21.15.80:6827/4019913545","cluster_addr":"172.21.15.80:6829/4019913545","heartbeat_back_addr":"172.21.15.80:6833/4019913545","heartbeat_front_addr":"172.21.15.80:6831/4019913545","state":["exists","up"]},{"osd":10,"uuid":"8d92f0d9-5c64-429a-b750-c33ae0e070c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":647343766},{"type":"v1","addr":"172.21.15.134:6825","nonce":647343766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":647343766},{"type":"v1","addr":"172.21.15.134:6827","nonce":647343766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":647343766},{"type":"v1","addr":"172.21.15.134:6831","nonce":647343766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":647343766},{"type":"v1","addr":"172.21.15.134:6829","nonce":647343766}]},"public_addr":"172.21.15.134:6825/647343766","cluster_addr":"172.21.15.134:6827/647343766","heartbeat_back_addr":"172.21.15.134:6831/647343766","heartbeat_front_addr":"172.21.15.134:6829/647343766","state":["exists","up"]},{"osd":11,"uuid":"4c003a41-f65b-479f-bc80-b27d680cf936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":4068024116},{"type":"v1","addr":"172.21.15.148:6825","nonce":4068024116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":4068024116},{"type":"v1","addr":"172.21.15.148:6827","nonce":4068024116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":4068024116},{"type":"v1","addr":"172.21.15.148:6831","nonce":4068024116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":4068024116},{"type":"v1","addr":"172.21.15.148:6829","nonce":4068024116}]},"public_addr":"172.21.15.148:6825/4068024116","cluster_addr":"172.21.15.148:6827/4068024116","heartbeat_back_addr":"172.21.15.148:6831/4068024116","heartbeat_front_addr":"172.21.15.148:6829/4068024116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:21:42.433631+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:22:06.145933+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:22:30.726143+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:22:55.729961+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:23:21.280913+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:23:46.609983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:24:12.585247+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:24:38.359287+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:25:04.491232+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:25:30.788089+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:25:56.891078+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:26:23.899113+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":7}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"4.a","mappings":[{"from":5,"to":8}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.80:0/2828378749":"2024-04-25T23:20:06.793609+0000","172.21.15.80:6801/233364790":"2024-04-25T23:20:06.793609+0000","172.21.15.80:6800/1614691410":"2024-04-25T23:19:40.490692+0000","172.21.15.80:0/1406187145":"2024-04-25T23:20:06.793609+0000","172.21.15.80:0/92953396":"2024-04-25T23:19:40.490692+0000","172.21.15.80:0/598129432":"2024-04-25T23:20:06.793609+0000","172.21.15.80:0/232193118":"2024-04-25T23:19:40.490692+0000","172.21.15.80:6800/233364790":"2024-04-25T23:20:06.793609+0000","172.21.15.80:6801/1614691410":"2024-04-25T23:19:40.490692+0000","172.21.15.80:0/2081562529":"2024-04-25T23:19:40.490692+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:29:43.365 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.364+0000 7f939886d640 1 -- 172.21.15.80:0/3677435279 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f93800a2b90 msgr2=0x7f93800a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:43.365 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.364+0000 7f939886d640 1 --2- 172.21.15.80:0/3677435279 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f93800a2b90 0x7f93800a5050 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f938400bbe0 tx=0x7f938400b390 comp rx=0 tx=0).stop 2024-04-24T23:29:43.365 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.364+0000 7f939886d640 1 -- 172.21.15.80:0/3677435279 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9394074140 msgr2=0x7f9394199d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:43.365 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.364+0000 7f939886d640 1 --2- 172.21.15.80:0/3677435279 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9394074140 0x7f9394199d00 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f937c009230 tx=0x7f937c004360 comp rx=0 tx=0).stop 2024-04-24T23:29:43.365 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.364+0000 7f939886d640 1 -- 172.21.15.80:0/3677435279 shutdown_connections 2024-04-24T23:29:43.365 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.364+0000 7f939886d640 1 --2- 172.21.15.80:0/3677435279 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f93800a2b90 0x7f93800a5050 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:43.365 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.364+0000 7f939886d640 1 --2- 172.21.15.80:0/3677435279 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9394109640 0x7f93941a1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:43.366 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.364+0000 7f939886d640 1 --2- 172.21.15.80:0/3677435279 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9394075720 0x7f939419a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:43.366 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.364+0000 7f939886d640 1 --2- 172.21.15.80:0/3677435279 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9394074140 0x7f9394199d00 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:43.366 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.364+0000 7f939886d640 1 -- 172.21.15.80:0/3677435279 >> 172.21.15.80:0/3677435279 conn(0x7f93940fcd40 msgr2=0x7f93941055d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:43.366 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.364+0000 7f939886d640 1 -- 172.21.15.80:0/3677435279 shutdown_connections 2024-04-24T23:29:43.366 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.364+0000 7f939886d640 1 -- 172.21.15.80:0/3677435279 wait complete. 2024-04-24T23:29:43.377 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:22:33.124428+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T23:27:17.575788+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T23:27:18.492762+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-24T23:27:22.438503+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-24T23:29:43.377 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:29:43.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:29:43 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/1820105275' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-04-24T23:29:43.482 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.479+0000 7f858dba2640 1 -- 172.21.15.80:0/2986536040 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8588107520 msgr2=0x7f8588109910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:43.482 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.479+0000 7f858dba2640 1 --2- 172.21.15.80:0/2986536040 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8588107520 0x7f8588109910 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f8580009ca0 tx=0x7f858002a8c0 comp rx=0 tx=0).stop 2024-04-24T23:29:43.482 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.479+0000 7f858dba2640 1 -- 172.21.15.80:0/2986536040 shutdown_connections 2024-04-24T23:29:43.482 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.479+0000 7f858dba2640 1 --2- 172.21.15.80:0/2986536040 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8588107520 0x7f8588109910 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:43.482 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.479+0000 7f858dba2640 1 --2- 172.21.15.80:0/2986536040 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8588104bf0 0x7f8588106fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:43.482 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.479+0000 7f858dba2640 1 --2- 172.21.15.80:0/2986536040 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85881022c0 0x7f85881046b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:43.482 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.479+0000 7f858dba2640 1 -- 172.21.15.80:0/2986536040 >> 172.21.15.80:0/2986536040 conn(0x7f85880fbc30 msgr2=0x7f85880fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:43.482 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.479+0000 7f858dba2640 1 -- 172.21.15.80:0/2986536040 shutdown_connections 2024-04-24T23:29:43.482 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.479+0000 7f858dba2640 1 -- 172.21.15.80:0/2986536040 wait complete. 2024-04-24T23:29:43.482 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.479+0000 7f858dba2640 1 Processor -- start 2024-04-24T23:29:43.483 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.479+0000 7f858dba2640 1 -- start start 2024-04-24T23:29:43.483 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.480+0000 7f858dba2640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f85881022c0 0x7f8588101c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:43.483 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.480+0000 7f858dba2640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8588104bf0 0x7f8588100270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:43.483 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.480+0000 7f858dba2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8588107520 0x7f85881007b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:43.483 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.480+0000 7f858dba2640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f858810d440 con 0x7f85881022c0 2024-04-24T23:29:43.483 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.480+0000 7f858dba2640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f858810d2c0 con 0x7f8588107520 2024-04-24T23:29:43.483 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.480+0000 7f858dba2640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f858810d5c0 con 0x7f8588104bf0 2024-04-24T23:29:43.483 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.480+0000 7f85877fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f85881022c0 0x7f8588101c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:43.483 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.480+0000 7f85877fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f85881022c0 0x7f8588101c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:51822/0 (socket says 172.21.15.80:51822) 2024-04-24T23:29:43.483 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.480+0000 7f85877fe640 1 -- 172.21.15.80:0/3091560622 learned_addr learned my addr 172.21.15.80:0/3091560622 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:29:43.483 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.480+0000 7f8587fff640 1 --2- 172.21.15.80:0/3091560622 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8588107520 0x7f85881007b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:43.484 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.480+0000 7f8586ffd640 1 --2- 172.21.15.80:0/3091560622 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8588104bf0 0x7f8588100270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:43.484 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.480+0000 7f85877fe640 1 -- 172.21.15.80:0/3091560622 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8588104bf0 msgr2=0x7f8588100270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:43.484 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.480+0000 7f85877fe640 1 --2- 172.21.15.80:0/3091560622 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8588104bf0 0x7f8588100270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:43.484 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.480+0000 7f85877fe640 1 -- 172.21.15.80:0/3091560622 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8588107520 msgr2=0x7f85881007b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:43.484 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.480+0000 7f85877fe640 1 --2- 172.21.15.80:0/3091560622 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8588107520 0x7f85881007b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:43.484 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.480+0000 7f85877fe640 1 -- 172.21.15.80:0/3091560622 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8588101040 con 0x7f85881022c0 2024-04-24T23:29:43.484 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.480+0000 7f85877fe640 1 --2- 172.21.15.80:0/3091560622 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f85881022c0 0x7f8588101c30 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f857800d950 tx=0x7f857800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:43.484 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.480+0000 7f8584ff9640 1 -- 172.21.15.80:0/3091560622 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8578017070 con 0x7f85881022c0 2024-04-24T23:29:43.484 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.480+0000 7f858dba2640 1 -- 172.21.15.80:0/3091560622 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85881a1bb0 con 0x7f85881022c0 2024-04-24T23:29:43.484 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.480+0000 7f8584ff9640 1 -- 172.21.15.80:0/3091560622 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8578009c30 con 0x7f85881022c0 2024-04-24T23:29:43.484 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.480+0000 7f8584ff9640 1 -- 172.21.15.80:0/3091560622 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8578020060 con 0x7f85881022c0 2024-04-24T23:29:43.485 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.480+0000 7f858dba2640 1 -- 172.21.15.80:0/3091560622 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85881a2100 con 0x7f85881022c0 2024-04-24T23:29:43.485 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.481+0000 7f858dba2640 1 -- 172.21.15.80:0/3091560622 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85880fd720 con 0x7f85881022c0 2024-04-24T23:29:43.486 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.482+0000 7f8584ff9640 1 -- 172.21.15.80:0/3091560622 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f85780109d0 con 0x7f85881022c0 2024-04-24T23:29:43.486 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.483+0000 7f8584ff9640 1 --2- 172.21.15.80:0/3091560622 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f85700a2b90 0x7f85700a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:43.486 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.483+0000 7f8586ffd640 1 --2- 172.21.15.80:0/3091560622 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f85700a2b90 0x7f85700a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:43.486 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.483+0000 7f8584ff9640 1 -- 172.21.15.80:0/3091560622 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f85780cf3f0 con 0x7f85881022c0 2024-04-24T23:29:43.486 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.483+0000 7f8586ffd640 1 --2- 172.21.15.80:0/3091560622 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f85700a2b90 0x7f85700a5050 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f85881019a0 tx=0x7f857c005c60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:43.488 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.487+0000 7f8584ff9640 1 -- 172.21.15.80:0/3091560622 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f8578009de0 con 0x7f85881022c0 2024-04-24T23:29:43.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:29:43 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/1820105275' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-04-24T23:29:43.662 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.661+0000 7f858dba2640 1 -- 172.21.15.80:0/3091560622 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f858802d170 con 0x7f85881022c0 2024-04-24T23:29:43.662 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.661+0000 7f8584ff9640 1 -- 172.21.15.80:0/3091560622 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7f858802d170 con 0x7f85881022c0 2024-04-24T23:29:43.662 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 1 2024-04-24T23:29:43.664 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.663+0000 7f858dba2640 1 -- 172.21.15.80:0/3091560622 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f85700a2b90 msgr2=0x7f85700a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:43.665 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.663+0000 7f858dba2640 1 --2- 172.21.15.80:0/3091560622 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f85700a2b90 0x7f85700a5050 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f85881019a0 tx=0x7f857c005c60 comp rx=0 tx=0).stop 2024-04-24T23:29:43.665 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.663+0000 7f858dba2640 1 -- 172.21.15.80:0/3091560622 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f85881022c0 msgr2=0x7f8588101c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:43.665 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.663+0000 7f858dba2640 1 --2- 172.21.15.80:0/3091560622 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f85881022c0 0x7f8588101c30 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f857800d950 tx=0x7f857800de10 comp rx=0 tx=0).stop 2024-04-24T23:29:43.665 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.663+0000 7f858dba2640 1 -- 172.21.15.80:0/3091560622 shutdown_connections 2024-04-24T23:29:43.665 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.663+0000 7f858dba2640 1 --2- 172.21.15.80:0/3091560622 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f85700a2b90 0x7f85700a5050 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:43.665 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.663+0000 7f858dba2640 1 --2- 172.21.15.80:0/3091560622 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8588107520 0x7f85881007b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:43.665 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.663+0000 7f858dba2640 1 --2- 172.21.15.80:0/3091560622 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8588104bf0 0x7f8588100270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:43.665 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.663+0000 7f858dba2640 1 --2- 172.21.15.80:0/3091560622 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f85881022c0 0x7f8588101c30 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:43.665 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.663+0000 7f858dba2640 1 -- 172.21.15.80:0/3091560622 >> 172.21.15.80:0/3091560622 conn(0x7f85880fbc30 msgr2=0x7f8588102c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:43.665 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.664+0000 7f858dba2640 1 -- 172.21.15.80:0/3091560622 shutdown_connections 2024-04-24T23:29:43.665 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.664+0000 7f858dba2640 1 -- 172.21.15.80:0/3091560622 wait complete. 2024-04-24T23:29:43.676 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T23:29:43.779 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.776+0000 7fa2901ed640 1 -- 172.21.15.80:0/2794234669 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa288102290 msgr2=0x7fa288104680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:43.779 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.776+0000 7fa2901ed640 1 --2- 172.21.15.80:0/2794234669 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa288102290 0x7fa288104680 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fa280009ca0 tx=0x7fa28002a8c0 comp rx=0 tx=0).stop 2024-04-24T23:29:43.779 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.776+0000 7fa2901ed640 1 -- 172.21.15.80:0/2794234669 shutdown_connections 2024-04-24T23:29:43.779 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.776+0000 7fa2901ed640 1 --2- 172.21.15.80:0/2794234669 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2881074f0 0x7fa2881098e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:43.779 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.776+0000 7fa2901ed640 1 --2- 172.21.15.80:0/2794234669 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa288104bc0 0x7fa288106fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:43.779 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.776+0000 7fa2901ed640 1 --2- 172.21.15.80:0/2794234669 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa288102290 0x7fa288104680 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:43.779 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.776+0000 7fa2901ed640 1 -- 172.21.15.80:0/2794234669 >> 172.21.15.80:0/2794234669 conn(0x7fa2880fbc00 msgr2=0x7fa2880fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:43.779 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.776+0000 7fa2901ed640 1 -- 172.21.15.80:0/2794234669 shutdown_connections 2024-04-24T23:29:43.779 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.776+0000 7fa2901ed640 1 -- 172.21.15.80:0/2794234669 wait complete. 2024-04-24T23:29:43.779 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.776+0000 7fa2901ed640 1 Processor -- start 2024-04-24T23:29:43.779 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.776+0000 7fa2901ed640 1 -- start start 2024-04-24T23:29:43.780 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.776+0000 7fa2901ed640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa288102290 0x7fa288101c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:43.780 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.776+0000 7fa2901ed640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa288104bc0 0x7fa288100240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:43.780 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.776+0000 7fa2901ed640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa2881074f0 0x7fa288100780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:43.780 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.776+0000 7fa2901ed640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fa28810d400 con 0x7fa288104bc0 2024-04-24T23:29:43.780 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.776+0000 7fa2901ed640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa28810d280 con 0x7fa2881074f0 2024-04-24T23:29:43.780 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.776+0000 7fa2901ed640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa28810d580 con 0x7fa288102290 2024-04-24T23:29:43.780 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.777+0000 7fa28d788640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa288104bc0 0x7fa288100240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:43.780 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.777+0000 7fa28d788640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa288104bc0 0x7fa288100240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:51832/0 (socket says 172.21.15.80:51832) 2024-04-24T23:29:43.780 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.777+0000 7fa28d788640 1 -- 172.21.15.80:0/188621338 learned_addr learned my addr 172.21.15.80:0/188621338 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:29:43.780 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.777+0000 7fa28df89640 1 --2- 172.21.15.80:0/188621338 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa288102290 0x7fa288101c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:43.780 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.777+0000 7fa28d788640 1 -- 172.21.15.80:0/188621338 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa288102290 msgr2=0x7fa288101c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:43.780 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.777+0000 7fa28d788640 1 --2- 172.21.15.80:0/188621338 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa288102290 0x7fa288101c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:43.780 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.777+0000 7fa28e78a640 1 --2- 172.21.15.80:0/188621338 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa2881074f0 0x7fa288100780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:43.780 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.777+0000 7fa28d788640 1 -- 172.21.15.80:0/188621338 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa2881074f0 msgr2=0x7fa288100780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:43.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.777+0000 7fa28d788640 1 --2- 172.21.15.80:0/188621338 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa2881074f0 0x7fa288100780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:43.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.777+0000 7fa28d788640 1 -- 172.21.15.80:0/188621338 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa288101010 con 0x7fa288104bc0 2024-04-24T23:29:43.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.777+0000 7fa28d788640 1 --2- 172.21.15.80:0/188621338 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa288104bc0 0x7fa288100240 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fa27800c970 tx=0x7fa27800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:43.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.777+0000 7fa27effd640 1 -- 172.21.15.80:0/188621338 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa278007bb0 con 0x7fa288104bc0 2024-04-24T23:29:43.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.777+0000 7fa27effd640 1 -- 172.21.15.80:0/188621338 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa278007d50 con 0x7fa288104bc0 2024-04-24T23:29:43.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.777+0000 7fa2901ed640 1 -- 172.21.15.80:0/188621338 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2881a1c20 con 0x7fa288104bc0 2024-04-24T23:29:43.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.777+0000 7fa27effd640 1 -- 172.21.15.80:0/188621338 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa27800f450 con 0x7fa288104bc0 2024-04-24T23:29:43.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.777+0000 7fa2901ed640 1 -- 172.21.15.80:0/188621338 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2881a2170 con 0x7fa288104bc0 2024-04-24T23:29:43.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.778+0000 7fa2901ed640 1 -- 172.21.15.80:0/188621338 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2880fd6f0 con 0x7fa288104bc0 2024-04-24T23:29:43.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.780+0000 7fa27effd640 1 -- 172.21.15.80:0/188621338 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa2780046c0 con 0x7fa288104bc0 2024-04-24T23:29:43.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.780+0000 7fa27effd640 1 --2- 172.21.15.80:0/188621338 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fa2740a2b90 0x7fa2740a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:43.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.781+0000 7fa28df89640 1 --2- 172.21.15.80:0/188621338 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fa2740a2b90 0x7fa2740a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:43.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.781+0000 7fa27effd640 1 -- 172.21.15.80:0/188621338 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fa2780cf490 con 0x7fa288104bc0 2024-04-24T23:29:43.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.781+0000 7fa28df89640 1 --2- 172.21.15.80:0/188621338 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fa2740a2b90 0x7fa2740a5050 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa280006010 tx=0x7fa280000f40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:43.785 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.784+0000 7fa27effd640 1 -- 172.21.15.80:0/188621338 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa278014030 con 0x7fa288104bc0 2024-04-24T23:29:43.960 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.959+0000 7fa2901ed640 1 -- 172.21.15.80:0/188621338 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fa28802d370 con 0x7fa288104bc0 2024-04-24T23:29:43.960 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.960+0000 7fa27effd640 1 -- 172.21.15.80:0/188621338 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) ==== 104+0+11 (secure 0 0 0) 0x7fa27809c020 con 0x7fa288104bc0 2024-04-24T23:29:43.961 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 32 2024-04-24T23:29:43.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.962+0000 7fa2901ed640 1 -- 172.21.15.80:0/188621338 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fa2740a2b90 msgr2=0x7fa2740a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:43.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.962+0000 7fa2901ed640 1 --2- 172.21.15.80:0/188621338 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fa2740a2b90 0x7fa2740a5050 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa280006010 tx=0x7fa280000f40 comp rx=0 tx=0).stop 2024-04-24T23:29:43.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.962+0000 7fa2901ed640 1 -- 172.21.15.80:0/188621338 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa288104bc0 msgr2=0x7fa288100240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:43.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.962+0000 7fa2901ed640 1 --2- 172.21.15.80:0/188621338 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa288104bc0 0x7fa288100240 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fa27800c970 tx=0x7fa27800ce30 comp rx=0 tx=0).stop 2024-04-24T23:29:43.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.962+0000 7fa2901ed640 1 -- 172.21.15.80:0/188621338 shutdown_connections 2024-04-24T23:29:43.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.962+0000 7fa2901ed640 1 --2- 172.21.15.80:0/188621338 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fa2740a2b90 0x7fa2740a5050 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:43.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.962+0000 7fa2901ed640 1 --2- 172.21.15.80:0/188621338 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa2881074f0 0x7fa288100780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:43.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.962+0000 7fa2901ed640 1 --2- 172.21.15.80:0/188621338 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa288104bc0 0x7fa288100240 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:43.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.962+0000 7fa2901ed640 1 --2- 172.21.15.80:0/188621338 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa288102290 0x7fa288101c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:43.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.962+0000 7fa2901ed640 1 -- 172.21.15.80:0/188621338 >> 172.21.15.80:0/188621338 conn(0x7fa2880fbc00 msgr2=0x7fa288102d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:43.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.962+0000 7fa2901ed640 1 -- 172.21.15.80:0/188621338 shutdown_connections 2024-04-24T23:29:43.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:43.962+0000 7fa2901ed640 1 -- 172.21.15.80:0/188621338 wait complete. 2024-04-24T23:29:43.974 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T23:29:44.081 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.078+0000 7ffa6af8b640 1 -- 172.21.15.80:0/3626292654 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffa64101890 msgr2=0x7ffa64101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:44.081 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.078+0000 7ffa6af8b640 1 --2- 172.21.15.80:0/3626292654 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffa64101890 0x7ffa64101d10 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7ffa50009900 tx=0x7ffa5002a410 comp rx=0 tx=0).stop 2024-04-24T23:29:44.081 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.078+0000 7ffa6af8b640 1 -- 172.21.15.80:0/3626292654 shutdown_connections 2024-04-24T23:29:44.081 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.078+0000 7ffa6af8b640 1 --2- 172.21.15.80:0/3626292654 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa64102250 0x7ffa64108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:44.081 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.078+0000 7ffa6af8b640 1 --2- 172.21.15.80:0/3626292654 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffa64101890 0x7ffa64101d10 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:44.081 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.078+0000 7ffa6af8b640 1 --2- 172.21.15.80:0/3626292654 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa641006a0 0x7ffa64100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:44.081 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.078+0000 7ffa6af8b640 1 -- 172.21.15.80:0/3626292654 >> 172.21.15.80:0/3626292654 conn(0x7ffa640fbc10 msgr2=0x7ffa640fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:44.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.078+0000 7ffa6af8b640 1 -- 172.21.15.80:0/3626292654 shutdown_connections 2024-04-24T23:29:44.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.078+0000 7ffa6af8b640 1 -- 172.21.15.80:0/3626292654 wait complete. 2024-04-24T23:29:44.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.078+0000 7ffa6af8b640 1 Processor -- start 2024-04-24T23:29:44.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.078+0000 7ffa6af8b640 1 -- start start 2024-04-24T23:29:44.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.078+0000 7ffa6af8b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffa641006a0 0x7ffa6406a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:44.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.078+0000 7ffa6af8b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa64101890 0x7ffa6419ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:44.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.078+0000 7ffa6af8b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa64102250 0x7ffa641a11e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:44.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.078+0000 7ffa6af8b640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7ffa6410c3a0 con 0x7ffa641006a0 2024-04-24T23:29:44.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.078+0000 7ffa6af8b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ffa6410c220 con 0x7ffa64101890 2024-04-24T23:29:44.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.078+0000 7ffa6af8b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ffa6410c520 con 0x7ffa64102250 2024-04-24T23:29:44.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.079+0000 7ffa68d27640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffa641006a0 0x7ffa6406a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:44.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.079+0000 7ffa68d27640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffa641006a0 0x7ffa6406a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:51858/0 (socket says 172.21.15.80:51858) 2024-04-24T23:29:44.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.079+0000 7ffa68d27640 1 -- 172.21.15.80:0/4005212306 learned_addr learned my addr 172.21.15.80:0/4005212306 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:29:44.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.079+0000 7ffa63fff640 1 --2- 172.21.15.80:0/4005212306 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa64101890 0x7ffa6419ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:44.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.079+0000 7ffa68d27640 1 -- 172.21.15.80:0/4005212306 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa64102250 msgr2=0x7ffa641a11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:44.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.079+0000 7ffa69528640 1 --2- 172.21.15.80:0/4005212306 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa64102250 0x7ffa641a11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:44.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.079+0000 7ffa68d27640 1 --2- 172.21.15.80:0/4005212306 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa64102250 0x7ffa641a11e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:44.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.079+0000 7ffa68d27640 1 -- 172.21.15.80:0/4005212306 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa64101890 msgr2=0x7ffa6419ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:44.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.079+0000 7ffa68d27640 1 --2- 172.21.15.80:0/4005212306 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa64101890 0x7ffa6419ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:44.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.079+0000 7ffa68d27640 1 -- 172.21.15.80:0/4005212306 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa641a19d0 con 0x7ffa641006a0 2024-04-24T23:29:44.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.079+0000 7ffa63fff640 1 --2- 172.21.15.80:0/4005212306 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa64101890 0x7ffa6419ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:29:44.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.079+0000 7ffa69528640 1 --2- 172.21.15.80:0/4005212306 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa64102250 0x7ffa641a11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:29:44.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.079+0000 7ffa68d27640 1 --2- 172.21.15.80:0/4005212306 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffa641006a0 0x7ffa6406a8b0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7ffa5800c970 tx=0x7ffa5800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:44.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.079+0000 7ffa61ffb640 1 -- 172.21.15.80:0/4005212306 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa58004220 con 0x7ffa641006a0 2024-04-24T23:29:44.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.079+0000 7ffa61ffb640 1 -- 172.21.15.80:0/4005212306 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa580043c0 con 0x7ffa641006a0 2024-04-24T23:29:44.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.079+0000 7ffa6af8b640 1 -- 172.21.15.80:0/4005212306 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa641a1c60 con 0x7ffa641006a0 2024-04-24T23:29:44.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.079+0000 7ffa61ffb640 1 -- 172.21.15.80:0/4005212306 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa58004a00 con 0x7ffa641006a0 2024-04-24T23:29:44.084 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.079+0000 7ffa6af8b640 1 -- 172.21.15.80:0/4005212306 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa641a2160 con 0x7ffa641006a0 2024-04-24T23:29:44.084 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.080+0000 7ffa6af8b640 1 -- 172.21.15.80:0/4005212306 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa641079c0 con 0x7ffa641006a0 2024-04-24T23:29:44.084 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.081+0000 7ffa61ffb640 1 -- 172.21.15.80:0/4005212306 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ffa580079b0 con 0x7ffa641006a0 2024-04-24T23:29:44.084 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.082+0000 7ffa61ffb640 1 --2- 172.21.15.80:0/4005212306 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7ffa540a2bc0 0x7ffa540a5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:44.085 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.082+0000 7ffa61ffb640 1 -- 172.21.15.80:0/4005212306 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7ffa580cf1f0 con 0x7ffa641006a0 2024-04-24T23:29:44.085 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.082+0000 7ffa63fff640 1 --2- 172.21.15.80:0/4005212306 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7ffa540a2bc0 0x7ffa540a5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:44.085 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.082+0000 7ffa63fff640 1 --2- 172.21.15.80:0/4005212306 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7ffa540a2bc0 0x7ffa540a5080 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ffa50000c00 tx=0x7ffa50031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:44.087 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.086+0000 7ffa61ffb640 1 -- 172.21.15.80:0/4005212306 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ffa58014030 con 0x7ffa641006a0 2024-04-24T23:29:44.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.265+0000 7ffa6af8b640 1 -- 172.21.15.80:0/4005212306 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ffa640623e0 con 0x7ffa641006a0 2024-04-24T23:29:44.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.265+0000 7ffa61ffb640 1 -- 172.21.15.80:0/4005212306 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) ==== 100+0+11 (secure 0 0 0) 0x7ffa5809bd80 con 0x7ffa641006a0 2024-04-24T23:29:44.266 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 64 2024-04-24T23:29:44.269 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.267+0000 7ffa6af8b640 1 -- 172.21.15.80:0/4005212306 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7ffa540a2bc0 msgr2=0x7ffa540a5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:44.269 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.267+0000 7ffa6af8b640 1 --2- 172.21.15.80:0/4005212306 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7ffa540a2bc0 0x7ffa540a5080 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ffa50000c00 tx=0x7ffa50031400 comp rx=0 tx=0).stop 2024-04-24T23:29:44.269 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.267+0000 7ffa6af8b640 1 -- 172.21.15.80:0/4005212306 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffa641006a0 msgr2=0x7ffa6406a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:44.269 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.267+0000 7ffa6af8b640 1 --2- 172.21.15.80:0/4005212306 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffa641006a0 0x7ffa6406a8b0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7ffa5800c970 tx=0x7ffa5800ce30 comp rx=0 tx=0).stop 2024-04-24T23:29:44.269 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.267+0000 7ffa6af8b640 1 -- 172.21.15.80:0/4005212306 shutdown_connections 2024-04-24T23:29:44.270 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.267+0000 7ffa6af8b640 1 --2- 172.21.15.80:0/4005212306 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7ffa540a2bc0 0x7ffa540a5080 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:44.270 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.267+0000 7ffa6af8b640 1 --2- 172.21.15.80:0/4005212306 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa64102250 0x7ffa641a11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:44.270 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.267+0000 7ffa6af8b640 1 --2- 172.21.15.80:0/4005212306 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa64101890 0x7ffa6419ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:44.270 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.267+0000 7ffa6af8b640 1 --2- 172.21.15.80:0/4005212306 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffa641006a0 0x7ffa6406a8b0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:44.270 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.267+0000 7ffa6af8b640 1 -- 172.21.15.80:0/4005212306 >> 172.21.15.80:0/4005212306 conn(0x7ffa640fbc10 msgr2=0x7ffa64104b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:44.270 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.268+0000 7ffa6af8b640 1 -- 172.21.15.80:0/4005212306 shutdown_connections 2024-04-24T23:29:44.270 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.268+0000 7ffa6af8b640 1 -- 172.21.15.80:0/4005212306 wait complete. 2024-04-24T23:29:44.278 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:29:44 smithi080 ceph-mon[29152]: pgmap v372: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:29:44.279 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:29:44 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/1820105275' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-04-24T23:29:44.279 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:29:44 smithi080 ceph-mon[29152]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T23:29:44.279 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:29:44 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/3677435279' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:29:44.279 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:29:44 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/3091560622' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:29:44.279 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:29:44 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/188621338' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T23:29:44.280 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-24T23:29:44.383 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.380+0000 7fa5d1ba9640 1 -- 172.21.15.80:0/1311099870 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa5cc102250 msgr2=0x7fa5cc108ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:44.383 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.380+0000 7fa5d1ba9640 1 --2- 172.21.15.80:0/1311099870 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa5cc102250 0x7fa5cc108ae0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fa5c4009ca0 tx=0x7fa5c402a8c0 comp rx=0 tx=0).stop 2024-04-24T23:29:44.383 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.380+0000 7fa5d1ba9640 1 -- 172.21.15.80:0/1311099870 shutdown_connections 2024-04-24T23:29:44.383 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.380+0000 7fa5d1ba9640 1 --2- 172.21.15.80:0/1311099870 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa5cc102250 0x7fa5cc108ae0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:44.383 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.380+0000 7fa5d1ba9640 1 --2- 172.21.15.80:0/1311099870 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5cc101890 0x7fa5cc101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:44.383 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.380+0000 7fa5d1ba9640 1 --2- 172.21.15.80:0/1311099870 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa5cc1006a0 0x7fa5cc100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:44.383 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.380+0000 7fa5d1ba9640 1 -- 172.21.15.80:0/1311099870 >> 172.21.15.80:0/1311099870 conn(0x7fa5cc0fbc30 msgr2=0x7fa5cc0fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:44.383 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.380+0000 7fa5d1ba9640 1 -- 172.21.15.80:0/1311099870 shutdown_connections 2024-04-24T23:29:44.384 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.380+0000 7fa5d1ba9640 1 -- 172.21.15.80:0/1311099870 wait complete. 2024-04-24T23:29:44.384 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.381+0000 7fa5d1ba9640 1 Processor -- start 2024-04-24T23:29:44.384 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.381+0000 7fa5d1ba9640 1 -- start start 2024-04-24T23:29:44.384 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.381+0000 7fa5d1ba9640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa5cc1006a0 0x7fa5cc06a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:44.384 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.381+0000 7fa5d1ba9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5cc101890 0x7fa5cc19c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:44.384 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.381+0000 7fa5d1ba9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa5cc102250 0x7fa5cc19ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:44.384 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.381+0000 7fa5d1ba9640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fa5cc10c350 con 0x7fa5cc1006a0 2024-04-24T23:29:44.384 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.381+0000 7fa5d1ba9640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa5cc10c1d0 con 0x7fa5cc102250 2024-04-24T23:29:44.384 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.381+0000 7fa5d1ba9640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa5cc10c4d0 con 0x7fa5cc101890 2024-04-24T23:29:44.384 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.381+0000 7fa5cb7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa5cc1006a0 0x7fa5cc06a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:44.384 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.381+0000 7fa5cb7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa5cc1006a0 0x7fa5cc06a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:51876/0 (socket says 172.21.15.80:51876) 2024-04-24T23:29:44.384 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.381+0000 7fa5cb7fe640 1 -- 172.21.15.80:0/359884436 learned_addr learned my addr 172.21.15.80:0/359884436 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:29:44.384 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.381+0000 7fa5cb7fe640 1 -- 172.21.15.80:0/359884436 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5cc101890 msgr2=0x7fa5cc19c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:44.384 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.381+0000 7fa5cb7fe640 1 --2- 172.21.15.80:0/359884436 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5cc101890 0x7fa5cc19c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:44.384 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.381+0000 7fa5cb7fe640 1 -- 172.21.15.80:0/359884436 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa5cc102250 msgr2=0x7fa5cc19ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:44.385 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.381+0000 7fa5cb7fe640 1 --2- 172.21.15.80:0/359884436 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa5cc102250 0x7fa5cc19ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:44.385 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.381+0000 7fa5cb7fe640 1 -- 172.21.15.80:0/359884436 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5cc19f650 con 0x7fa5cc1006a0 2024-04-24T23:29:44.385 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.381+0000 7fa5cb7fe640 1 --2- 172.21.15.80:0/359884436 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa5cc1006a0 0x7fa5cc06a8b0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc00c970 tx=0x7fa5bc00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:44.385 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.382+0000 7fa5c8ff9640 1 -- 172.21.15.80:0/359884436 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5bc007c10 con 0x7fa5cc1006a0 2024-04-24T23:29:44.385 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.382+0000 7fa5c8ff9640 1 -- 172.21.15.80:0/359884436 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5bc007db0 con 0x7fa5cc1006a0 2024-04-24T23:29:44.385 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.382+0000 7fa5c8ff9640 1 -- 172.21.15.80:0/359884436 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5bc00f940 con 0x7fa5cc1006a0 2024-04-24T23:29:44.385 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.382+0000 7fa5d1ba9640 1 -- 172.21.15.80:0/359884436 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5cc19f8e0 con 0x7fa5cc1006a0 2024-04-24T23:29:44.385 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.382+0000 7fa5d1ba9640 1 -- 172.21.15.80:0/359884436 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5cc19fde0 con 0x7fa5cc1006a0 2024-04-24T23:29:44.386 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.383+0000 7fa5d1ba9640 1 -- 172.21.15.80:0/359884436 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5cc107960 con 0x7fa5cc1006a0 2024-04-24T23:29:44.386 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.384+0000 7fa5c8ff9640 1 -- 172.21.15.80:0/359884436 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa5bc00fae0 con 0x7fa5cc1006a0 2024-04-24T23:29:44.386 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.384+0000 7fa5c8ff9640 1 --2- 172.21.15.80:0/359884436 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fa5b40a2b90 0x7fa5b40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:44.386 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.384+0000 7fa5caffd640 1 --2- 172.21.15.80:0/359884436 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fa5b40a2b90 0x7fa5b40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:44.386 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.384+0000 7fa5c8ff9640 1 -- 172.21.15.80:0/359884436 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fa5bc0cf890 con 0x7fa5cc1006a0 2024-04-24T23:29:44.386 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.385+0000 7fa5caffd640 1 --2- 172.21.15.80:0/359884436 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fa5b40a2b90 0x7fa5b40a5050 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fa5c0005fd0 tx=0x7fa5c0005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:44.389 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.389+0000 7fa5c8ff9640 1 -- 172.21.15.80:0/359884436 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa5bc09c4b0 con 0x7fa5cc1006a0 2024-04-24T23:29:44.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:29:44 smithi134 ceph-mon[32619]: pgmap v372: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:29:44.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:29:44 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/1820105275' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-04-24T23:29:44.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:29:44 smithi134 ceph-mon[32619]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T23:29:44.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:29:44 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/3677435279' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:29:44.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:29:44 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/3091560622' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:29:44.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:29:44 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/188621338' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T23:29:44.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:29:44 smithi148 ceph-mon[32528]: pgmap v372: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:29:44.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:29:44 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/1820105275' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-04-24T23:29:44.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:29:44 smithi148 ceph-mon[32528]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T23:29:44.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:29:44 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/3677435279' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:29:44.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:29:44 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/3091560622' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:29:44.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:29:44 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/188621338' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T23:29:44.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.563+0000 7fa5d1ba9640 1 -- 172.21.15.80:0/359884436 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fa5cc062480 con 0x7fa5cc1006a0 2024-04-24T23:29:44.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.563+0000 7fa5c8ff9640 1 -- 172.21.15.80:0/359884436 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) ==== 103+0+11 (secure 0 0 0) 0x7fa5bc09bdb0 con 0x7fa5cc1006a0 2024-04-24T23:29:44.564 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 32 2024-04-24T23:29:44.566 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.565+0000 7fa5d1ba9640 1 -- 172.21.15.80:0/359884436 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fa5b40a2b90 msgr2=0x7fa5b40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:44.566 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.565+0000 7fa5d1ba9640 1 --2- 172.21.15.80:0/359884436 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fa5b40a2b90 0x7fa5b40a5050 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fa5c0005fd0 tx=0x7fa5c0005850 comp rx=0 tx=0).stop 2024-04-24T23:29:44.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.565+0000 7fa5d1ba9640 1 -- 172.21.15.80:0/359884436 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa5cc1006a0 msgr2=0x7fa5cc06a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:44.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.565+0000 7fa5d1ba9640 1 --2- 172.21.15.80:0/359884436 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa5cc1006a0 0x7fa5cc06a8b0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc00c970 tx=0x7fa5bc00ce30 comp rx=0 tx=0).stop 2024-04-24T23:29:44.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.565+0000 7fa5d1ba9640 1 -- 172.21.15.80:0/359884436 shutdown_connections 2024-04-24T23:29:44.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.565+0000 7fa5d1ba9640 1 --2- 172.21.15.80:0/359884436 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fa5b40a2b90 0x7fa5b40a5050 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:44.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.565+0000 7fa5d1ba9640 1 --2- 172.21.15.80:0/359884436 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa5cc102250 0x7fa5cc19ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:44.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.565+0000 7fa5d1ba9640 1 --2- 172.21.15.80:0/359884436 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5cc101890 0x7fa5cc19c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:44.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.565+0000 7fa5d1ba9640 1 --2- 172.21.15.80:0/359884436 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa5cc1006a0 0x7fa5cc06a8b0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:44.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.565+0000 7fa5d1ba9640 1 -- 172.21.15.80:0/359884436 >> 172.21.15.80:0/359884436 conn(0x7fa5cc0fbc30 msgr2=0x7fa5cc104b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:44.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.566+0000 7fa5d1ba9640 1 -- 172.21.15.80:0/359884436 shutdown_connections 2024-04-24T23:29:44.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.566+0000 7fa5d1ba9640 1 -- 172.21.15.80:0/359884436 wait complete. 2024-04-24T23:29:44.579 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:29:44.679 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.677+0000 7fca26a1c640 1 -- 172.21.15.80:0/3413175369 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fca20104b50 msgr2=0x7fca20106f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:44.682 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.677+0000 7fca26a1c640 1 --2- 172.21.15.80:0/3413175369 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fca20104b50 0x7fca20106f40 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fca14009900 tx=0x7fca1402a410 comp rx=0 tx=0).stop 2024-04-24T23:29:44.682 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.677+0000 7fca26a1c640 1 -- 172.21.15.80:0/3413175369 shutdown_connections 2024-04-24T23:29:44.682 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.677+0000 7fca26a1c640 1 --2- 172.21.15.80:0/3413175369 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fca20107480 0x7fca201098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:44.682 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.677+0000 7fca26a1c640 1 --2- 172.21.15.80:0/3413175369 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fca20104b50 0x7fca20106f40 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:44.683 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.677+0000 7fca26a1c640 1 --2- 172.21.15.80:0/3413175369 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fca20102220 0x7fca20104610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:44.683 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.677+0000 7fca26a1c640 1 -- 172.21.15.80:0/3413175369 >> 172.21.15.80:0/3413175369 conn(0x7fca200fbc30 msgr2=0x7fca200fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:44.683 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.677+0000 7fca26a1c640 1 -- 172.21.15.80:0/3413175369 shutdown_connections 2024-04-24T23:29:44.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.677+0000 7fca26a1c640 1 -- 172.21.15.80:0/3413175369 wait complete. 2024-04-24T23:29:44.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.678+0000 7fca26a1c640 1 Processor -- start 2024-04-24T23:29:44.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.678+0000 7fca26a1c640 1 -- start start 2024-04-24T23:29:44.690 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.678+0000 7fca26a1c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fca20102220 0x7fca20101b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:44.690 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.678+0000 7fca26a1c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fca20104b50 0x7fca201001d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:44.690 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.678+0000 7fca26a1c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fca20107480 0x7fca20100710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:44.690 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.678+0000 7fca26a1c640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fca2010d420 con 0x7fca20107480 2024-04-24T23:29:44.691 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.678+0000 7fca26a1c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fca2010d2a0 con 0x7fca20102220 2024-04-24T23:29:44.691 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.678+0000 7fca26a1c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fca2010d5a0 con 0x7fca20104b50 2024-04-24T23:29:44.691 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.678+0000 7fca24fb9640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fca20107480 0x7fca20100710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:44.691 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.678+0000 7fca24fb9640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fca20107480 0x7fca20100710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:51902/0 (socket says 172.21.15.80:51902) 2024-04-24T23:29:44.691 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.678+0000 7fca24fb9640 1 -- 172.21.15.80:0/2759412716 learned_addr learned my addr 172.21.15.80:0/2759412716 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:29:44.691 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.678+0000 7fca1f7fe640 1 --2- 172.21.15.80:0/2759412716 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fca20104b50 0x7fca201001d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:44.691 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.678+0000 7fca1ffff640 1 --2- 172.21.15.80:0/2759412716 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fca20102220 0x7fca20101b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:44.691 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.678+0000 7fca24fb9640 1 -- 172.21.15.80:0/2759412716 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fca20104b50 msgr2=0x7fca201001d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:44.691 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.678+0000 7fca24fb9640 1 --2- 172.21.15.80:0/2759412716 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fca20104b50 0x7fca201001d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:44.691 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.678+0000 7fca24fb9640 1 -- 172.21.15.80:0/2759412716 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fca20102220 msgr2=0x7fca20101b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:44.692 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.678+0000 7fca24fb9640 1 --2- 172.21.15.80:0/2759412716 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fca20102220 0x7fca20101b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:44.692 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.678+0000 7fca24fb9640 1 -- 172.21.15.80:0/2759412716 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca20100fa0 con 0x7fca20107480 2024-04-24T23:29:44.692 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.678+0000 7fca1ffff640 1 --2- 172.21.15.80:0/2759412716 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fca20102220 0x7fca20101b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:29:44.692 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.679+0000 7fca24fb9640 1 --2- 172.21.15.80:0/2759412716 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fca20107480 0x7fca20100710 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fca1800eaa0 tx=0x7fca1800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:44.692 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.679+0000 7fca1d7fa640 1 -- 172.21.15.80:0/2759412716 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca1800cdb0 con 0x7fca20107480 2024-04-24T23:29:44.692 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.679+0000 7fca1d7fa640 1 -- 172.21.15.80:0/2759412716 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca18005020 con 0x7fca20107480 2024-04-24T23:29:44.692 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.679+0000 7fca26a1c640 1 -- 172.21.15.80:0/2759412716 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca201a1c10 con 0x7fca20107480 2024-04-24T23:29:44.692 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.679+0000 7fca1d7fa640 1 -- 172.21.15.80:0/2759412716 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca180116b0 con 0x7fca20107480 2024-04-24T23:29:44.692 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.679+0000 7fca26a1c640 1 -- 172.21.15.80:0/2759412716 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca201a2130 con 0x7fca20107480 2024-04-24T23:29:44.693 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.680+0000 7fca26a1c640 1 -- 172.21.15.80:0/2759412716 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca200689b0 con 0x7fca20107480 2024-04-24T23:29:44.693 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.681+0000 7fca1d7fa640 1 -- 172.21.15.80:0/2759412716 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fca18005560 con 0x7fca20107480 2024-04-24T23:29:44.693 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.681+0000 7fca1d7fa640 1 --2- 172.21.15.80:0/2759412716 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fca080a2b90 0x7fca080a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:44.693 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.681+0000 7fca1ffff640 1 --2- 172.21.15.80:0/2759412716 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fca080a2b90 0x7fca080a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:44.693 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.682+0000 7fca1d7fa640 1 -- 172.21.15.80:0/2759412716 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fca180cf6c0 con 0x7fca20107480 2024-04-24T23:29:44.693 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.682+0000 7fca1ffff640 1 --2- 172.21.15.80:0/2759412716 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fca080a2b90 0x7fca080a5050 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fca10005d00 tx=0x7fca10005ed0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:44.693 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.686+0000 7fca1d7fa640 1 -- 172.21.15.80:0/2759412716 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fca18016030 con 0x7fca20107480 2024-04-24T23:29:44.865 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.864+0000 7fca26a1c640 1 -- 172.21.15.80:0/2759412716 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fca20062340 con 0x7fca20107480 2024-04-24T23:29:44.866 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.865+0000 7fca1d7fa640 1 -- 172.21.15.80:0/2759412716 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24839 (secure 0 0 0) 0x7fca1809c250 con 0x7fca20107480 2024-04-24T23:29:44.867 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-24T23:29:44.867 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":94,"fsid":"ec7885aa-0290-11ef-bc93-c7b262605968","created":"2024-04-24T23:19:05.586407+0000","modified":"2024-04-24T23:29:08.209684+0000","last_up_change":"2024-04-24T23:26:25.938316+0000","last_in_change":"2024-04-24T23:26:04.786699+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:22:33.124428+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:27:17.575788+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:27:18.492762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T23:27:22.438503+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a525741d-b46c-4eef-aa64-3f057cac1497","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":2597718112},{"type":"v1","addr":"172.21.15.80:6803","nonce":2597718112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":2597718112},{"type":"v1","addr":"172.21.15.80:6805","nonce":2597718112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2597718112},{"type":"v1","addr":"172.21.15.80:6809","nonce":2597718112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":2597718112},{"type":"v1","addr":"172.21.15.80:6807","nonce":2597718112}]},"public_addr":"172.21.15.80:6803/2597718112","cluster_addr":"172.21.15.80:6805/2597718112","heartbeat_back_addr":"172.21.15.80:6809/2597718112","heartbeat_front_addr":"172.21.15.80:6807/2597718112","state":["exists","up"]},{"osd":1,"uuid":"2d0f0d67-6db4-40ac-91cc-58604ab6795a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":2395610201},{"type":"v1","addr":"172.21.15.134:6801","nonce":2395610201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":2395610201},{"type":"v1","addr":"172.21.15.134:6803","nonce":2395610201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":2395610201},{"type":"v1","addr":"172.21.15.134:6807","nonce":2395610201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":2395610201},{"type":"v1","addr":"172.21.15.134:6805","nonce":2395610201}]},"public_addr":"172.21.15.134:6801/2395610201","cluster_addr":"172.21.15.134:6803/2395610201","heartbeat_back_addr":"172.21.15.134:6807/2395610201","heartbeat_front_addr":"172.21.15.134:6805/2395610201","state":["exists","up"]},{"osd":2,"uuid":"781fb879-fedc-4c84-82fb-59e073bc4cf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1629152187},{"type":"v1","addr":"172.21.15.148:6801","nonce":1629152187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1629152187},{"type":"v1","addr":"172.21.15.148:6803","nonce":1629152187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1629152187},{"type":"v1","addr":"172.21.15.148:6807","nonce":1629152187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1629152187},{"type":"v1","addr":"172.21.15.148:6805","nonce":1629152187}]},"public_addr":"172.21.15.148:6801/1629152187","cluster_addr":"172.21.15.148:6803/1629152187","heartbeat_back_addr":"172.21.15.148:6807/1629152187","heartbeat_front_addr":"172.21.15.148:6805/1629152187","state":["exists","up"]},{"osd":3,"uuid":"cd982757-f2f6-4296-b3e7-4cd40fbd8d9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":3128298510},{"type":"v1","addr":"172.21.15.80:6811","nonce":3128298510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":3128298510},{"type":"v1","addr":"172.21.15.80:6813","nonce":3128298510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":3128298510},{"type":"v1","addr":"172.21.15.80:6817","nonce":3128298510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":3128298510},{"type":"v1","addr":"172.21.15.80:6815","nonce":3128298510}]},"public_addr":"172.21.15.80:6811/3128298510","cluster_addr":"172.21.15.80:6813/3128298510","heartbeat_back_addr":"172.21.15.80:6817/3128298510","heartbeat_front_addr":"172.21.15.80:6815/3128298510","state":["exists","up"]},{"osd":4,"uuid":"dfc55a79-a7e8-417a-b898-ec44171a54af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1917556060},{"type":"v1","addr":"172.21.15.134:6809","nonce":1917556060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1917556060},{"type":"v1","addr":"172.21.15.134:6811","nonce":1917556060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1917556060},{"type":"v1","addr":"172.21.15.134:6815","nonce":1917556060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1917556060},{"type":"v1","addr":"172.21.15.134:6813","nonce":1917556060}]},"public_addr":"172.21.15.134:6809/1917556060","cluster_addr":"172.21.15.134:6811/1917556060","heartbeat_back_addr":"172.21.15.134:6815/1917556060","heartbeat_front_addr":"172.21.15.134:6813/1917556060","state":["exists","up"]},{"osd":5,"uuid":"b5491337-0694-440c-b679-312f3d14cf65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":2715951281},{"type":"v1","addr":"172.21.15.148:6809","nonce":2715951281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":2715951281},{"type":"v1","addr":"172.21.15.148:6811","nonce":2715951281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":2715951281},{"type":"v1","addr":"172.21.15.148:6815","nonce":2715951281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":2715951281},{"type":"v1","addr":"172.21.15.148:6813","nonce":2715951281}]},"public_addr":"172.21.15.148:6809/2715951281","cluster_addr":"172.21.15.148:6811/2715951281","heartbeat_back_addr":"172.21.15.148:6815/2715951281","heartbeat_front_addr":"172.21.15.148:6813/2715951281","state":["exists","up"]},{"osd":6,"uuid":"ba4b5df7-d86e-47cd-ab67-cf763c4e4d6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":953261724},{"type":"v1","addr":"172.21.15.80:6819","nonce":953261724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":953261724},{"type":"v1","addr":"172.21.15.80:6821","nonce":953261724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":953261724},{"type":"v1","addr":"172.21.15.80:6825","nonce":953261724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":953261724},{"type":"v1","addr":"172.21.15.80:6823","nonce":953261724}]},"public_addr":"172.21.15.80:6819/953261724","cluster_addr":"172.21.15.80:6821/953261724","heartbeat_back_addr":"172.21.15.80:6825/953261724","heartbeat_front_addr":"172.21.15.80:6823/953261724","state":["exists","up"]},{"osd":7,"uuid":"464a503a-75ae-49e7-ab6e-a7f9f2a6a307","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":108354844},{"type":"v1","addr":"172.21.15.134:6817","nonce":108354844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":108354844},{"type":"v1","addr":"172.21.15.134:6819","nonce":108354844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":108354844},{"type":"v1","addr":"172.21.15.134:6823","nonce":108354844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":108354844},{"type":"v1","addr":"172.21.15.134:6821","nonce":108354844}]},"public_addr":"172.21.15.134:6817/108354844","cluster_addr":"172.21.15.134:6819/108354844","heartbeat_back_addr":"172.21.15.134:6823/108354844","heartbeat_front_addr":"172.21.15.134:6821/108354844","state":["exists","up"]},{"osd":8,"uuid":"ce544b2e-4540-47c0-9997-500c45351330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3559655329},{"type":"v1","addr":"172.21.15.148:6817","nonce":3559655329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":3559655329},{"type":"v1","addr":"172.21.15.148:6819","nonce":3559655329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":3559655329},{"type":"v1","addr":"172.21.15.148:6823","nonce":3559655329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3559655329},{"type":"v1","addr":"172.21.15.148:6821","nonce":3559655329}]},"public_addr":"172.21.15.148:6817/3559655329","cluster_addr":"172.21.15.148:6819/3559655329","heartbeat_back_addr":"172.21.15.148:6823/3559655329","heartbeat_front_addr":"172.21.15.148:6821/3559655329","state":["exists","up"]},{"osd":9,"uuid":"06de3732-19b9-496c-a464-55322efa6759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":4019913545},{"type":"v1","addr":"172.21.15.80:6827","nonce":4019913545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":4019913545},{"type":"v1","addr":"172.21.15.80:6829","nonce":4019913545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":4019913545},{"type":"v1","addr":"172.21.15.80:6833","nonce":4019913545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":4019913545},{"type":"v1","addr":"172.21.15.80:6831","nonce":4019913545}]},"public_addr":"172.21.15.80:6827/4019913545","cluster_addr":"172.21.15.80:6829/4019913545","heartbeat_back_addr":"172.21.15.80:6833/4019913545","heartbeat_front_addr":"172.21.15.80:6831/4019913545","state":["exists","up"]},{"osd":10,"uuid":"8d92f0d9-5c64-429a-b750-c33ae0e070c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":647343766},{"type":"v1","addr":"172.21.15.134:6825","nonce":647343766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":647343766},{"type":"v1","addr":"172.21.15.134:6827","nonce":647343766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":647343766},{"type":"v1","addr":"172.21.15.134:6831","nonce":647343766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":647343766},{"type":"v1","addr":"172.21.15.134:6829","nonce":647343766}]},"public_addr":"172.21.15.134:6825/647343766","cluster_addr":"172.21.15.134:6827/647343766","heartbeat_back_addr":"172.21.15.134:6831/647343766","heartbeat_front_addr":"172.21.15.134:6829/647343766","state":["exists","up"]},{"osd":11,"uuid":"4c003a41-f65b-479f-bc80-b27d680cf936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":4068024116},{"type":"v1","addr":"172.21.15.148:6825","nonce":4068024116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":4068024116},{"type":"v1","addr":"172.21.15.148:6827","nonce":4068024116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":4068024116},{"type":"v1","addr":"172.21.15.148:6831","nonce":4068024116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":4068024116},{"type":"v1","addr":"172.21.15.148:6829","nonce":4068024116}]},"public_addr":"172.21.15.148:6825/4068024116","cluster_addr":"172.21.15.148:6827/4068024116","heartbeat_back_addr":"172.21.15.148:6831/4068024116","heartbeat_front_addr":"172.21.15.148:6829/4068024116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:21:42.433631+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:22:06.145933+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:22:30.726143+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:22:55.729961+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:23:21.280913+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:23:46.609983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:24:12.585247+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:24:38.359287+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:25:04.491232+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:25:30.788089+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:25:56.891078+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:26:23.899113+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":7}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"4.a","mappings":[{"from":5,"to":8}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.80:0/2828378749":"2024-04-25T23:20:06.793609+0000","172.21.15.80:6801/233364790":"2024-04-25T23:20:06.793609+0000","172.21.15.80:6800/1614691410":"2024-04-25T23:19:40.490692+0000","172.21.15.80:0/1406187145":"2024-04-25T23:20:06.793609+0000","172.21.15.80:0/92953396":"2024-04-25T23:19:40.490692+0000","172.21.15.80:0/598129432":"2024-04-25T23:20:06.793609+0000","172.21.15.80:0/232193118":"2024-04-25T23:19:40.490692+0000","172.21.15.80:6800/233364790":"2024-04-25T23:20:06.793609+0000","172.21.15.80:6801/1614691410":"2024-04-25T23:19:40.490692+0000","172.21.15.80:0/2081562529":"2024-04-25T23:19:40.490692+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:29:44.869 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.867+0000 7fca26a1c640 1 -- 172.21.15.80:0/2759412716 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fca080a2b90 msgr2=0x7fca080a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:44.869 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.867+0000 7fca26a1c640 1 --2- 172.21.15.80:0/2759412716 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fca080a2b90 0x7fca080a5050 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fca10005d00 tx=0x7fca10005ed0 comp rx=0 tx=0).stop 2024-04-24T23:29:44.869 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.867+0000 7fca26a1c640 1 -- 172.21.15.80:0/2759412716 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fca20107480 msgr2=0x7fca20100710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:44.869 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.867+0000 7fca26a1c640 1 --2- 172.21.15.80:0/2759412716 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fca20107480 0x7fca20100710 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fca1800eaa0 tx=0x7fca1800ef60 comp rx=0 tx=0).stop 2024-04-24T23:29:44.869 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.867+0000 7fca26a1c640 1 -- 172.21.15.80:0/2759412716 shutdown_connections 2024-04-24T23:29:44.869 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.867+0000 7fca26a1c640 1 --2- 172.21.15.80:0/2759412716 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fca080a2b90 0x7fca080a5050 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:44.869 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.867+0000 7fca26a1c640 1 --2- 172.21.15.80:0/2759412716 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fca20107480 0x7fca20100710 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:44.870 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.867+0000 7fca26a1c640 1 --2- 172.21.15.80:0/2759412716 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fca20104b50 0x7fca201001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:44.870 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.867+0000 7fca26a1c640 1 --2- 172.21.15.80:0/2759412716 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fca20102220 0x7fca20101b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:44.870 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.867+0000 7fca26a1c640 1 -- 172.21.15.80:0/2759412716 >> 172.21.15.80:0/2759412716 conn(0x7fca200fbc30 msgr2=0x7fca20102b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:44.870 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.867+0000 7fca26a1c640 1 -- 172.21.15.80:0/2759412716 shutdown_connections 2024-04-24T23:29:44.870 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.868+0000 7fca26a1c640 1 -- 172.21.15.80:0/2759412716 wait complete. 2024-04-24T23:29:44.881 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:22:33.124428+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T23:27:17.575788+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T23:27:18.492762+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-24T23:27:22.438503+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-24T23:29:44.882 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:29:44.990 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.986+0000 7f67e36da640 1 -- 172.21.15.80:0/2360633240 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f67dc101870 msgr2=0x7f67dc101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:44.990 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.986+0000 7f67e36da640 1 --2- 172.21.15.80:0/2360633240 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f67dc101870 0x7f67dc101cf0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f67cc009900 tx=0x7f67cc02a410 comp rx=0 tx=0).stop 2024-04-24T23:29:44.990 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.986+0000 7f67e36da640 1 -- 172.21.15.80:0/2360633240 shutdown_connections 2024-04-24T23:29:44.990 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.986+0000 7f67e36da640 1 --2- 172.21.15.80:0/2360633240 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f67dc102230 0x7f67dc108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:44.990 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.986+0000 7f67e36da640 1 --2- 172.21.15.80:0/2360633240 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f67dc101870 0x7f67dc101cf0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:44.990 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.986+0000 7f67e36da640 1 --2- 172.21.15.80:0/2360633240 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67dc100680 0x7f67dc100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:44.990 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.986+0000 7f67e36da640 1 -- 172.21.15.80:0/2360633240 >> 172.21.15.80:0/2360633240 conn(0x7f67dc0fbbf0 msgr2=0x7f67dc0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:44.990 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.987+0000 7f67e36da640 1 -- 172.21.15.80:0/2360633240 shutdown_connections 2024-04-24T23:29:44.990 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.987+0000 7f67e36da640 1 -- 172.21.15.80:0/2360633240 wait complete. 2024-04-24T23:29:44.990 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.987+0000 7f67e36da640 1 Processor -- start 2024-04-24T23:29:44.991 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.987+0000 7f67e36da640 1 -- start start 2024-04-24T23:29:44.991 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.987+0000 7f67e36da640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f67dc100680 0x7f67dc1969f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:44.991 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.987+0000 7f67e36da640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67dc101870 0x7f67dc196f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:44.991 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.987+0000 7f67e36da640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f67dc102230 0x7f67dc19dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:44.991 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.987+0000 7f67e36da640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f67dc10c300 con 0x7f67dc100680 2024-04-24T23:29:44.991 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.987+0000 7f67e36da640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f67dc10c180 con 0x7f67dc102230 2024-04-24T23:29:44.991 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.987+0000 7f67e36da640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f67dc10c480 con 0x7f67dc101870 2024-04-24T23:29:44.991 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.987+0000 7f67e1476640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f67dc100680 0x7f67dc1969f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:44.991 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.987+0000 7f67e1476640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f67dc100680 0x7f67dc1969f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:51924/0 (socket says 172.21.15.80:51924) 2024-04-24T23:29:44.991 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.987+0000 7f67e1476640 1 -- 172.21.15.80:0/726820089 learned_addr learned my addr 172.21.15.80:0/726820089 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:29:44.992 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.988+0000 7f67e0c75640 1 --2- 172.21.15.80:0/726820089 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67dc101870 0x7f67dc196f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:44.992 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.988+0000 7f67e1c77640 1 --2- 172.21.15.80:0/726820089 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f67dc102230 0x7f67dc19dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:44.992 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.988+0000 7f67e1476640 1 -- 172.21.15.80:0/726820089 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67dc101870 msgr2=0x7f67dc196f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:44.992 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.988+0000 7f67e1476640 1 --2- 172.21.15.80:0/726820089 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67dc101870 0x7f67dc196f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:44.992 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.988+0000 7f67e1476640 1 -- 172.21.15.80:0/726820089 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f67dc102230 msgr2=0x7f67dc19dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:44.992 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.988+0000 7f67e1476640 1 --2- 172.21.15.80:0/726820089 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f67dc102230 0x7f67dc19dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:44.992 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.988+0000 7f67e1476640 1 -- 172.21.15.80:0/726820089 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67dc19e610 con 0x7f67dc100680 2024-04-24T23:29:44.992 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.988+0000 7f67e1476640 1 --2- 172.21.15.80:0/726820089 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f67dc100680 0x7f67dc1969f0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f67d800c970 tx=0x7f67d800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:44.992 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.988+0000 7f67d27fc640 1 -- 172.21.15.80:0/726820089 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d8004220 con 0x7f67dc100680 2024-04-24T23:29:44.992 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.988+0000 7f67d27fc640 1 -- 172.21.15.80:0/726820089 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67d80043c0 con 0x7f67dc100680 2024-04-24T23:29:44.992 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.988+0000 7f67e36da640 1 -- 172.21.15.80:0/726820089 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67dc19e900 con 0x7f67dc100680 2024-04-24T23:29:44.992 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.988+0000 7f67d27fc640 1 -- 172.21.15.80:0/726820089 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d8004a00 con 0x7f67dc100680 2024-04-24T23:29:44.992 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.988+0000 7f67e36da640 1 -- 172.21.15.80:0/726820089 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67dc19edd0 con 0x7f67dc100680 2024-04-24T23:29:44.992 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.989+0000 7f67e36da640 1 -- 172.21.15.80:0/726820089 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67dc107a90 con 0x7f67dc100680 2024-04-24T23:29:44.994 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.990+0000 7f67d27fc640 1 -- 172.21.15.80:0/726820089 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f67d80079b0 con 0x7f67dc100680 2024-04-24T23:29:44.994 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.990+0000 7f67d27fc640 1 --2- 172.21.15.80:0/726820089 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f67c80a2b90 0x7f67c80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:44.994 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.991+0000 7f67e0c75640 1 --2- 172.21.15.80:0/726820089 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f67c80a2b90 0x7f67c80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:44.994 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.991+0000 7f67d27fc640 1 -- 172.21.15.80:0/726820089 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f67d80cf280 con 0x7f67dc100680 2024-04-24T23:29:44.994 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.991+0000 7f67e0c75640 1 --2- 172.21.15.80:0/726820089 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f67c80a2b90 0x7f67c80a5050 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f67cc000c00 tx=0x7f67cc0315d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:44.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:44.995+0000 7f67d27fc640 1 -- 172.21.15.80:0/726820089 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f67d8014030 con 0x7f67dc100680 2024-04-24T23:29:45.173 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.171+0000 7f67e36da640 1 -- 172.21.15.80:0/726820089 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f67dc062340 con 0x7f67dc100680 2024-04-24T23:29:45.173 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.171+0000 7f67d27fc640 1 -- 172.21.15.80:0/726820089 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7f67d809be10 con 0x7f67dc100680 2024-04-24T23:29:45.173 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 1 2024-04-24T23:29:45.175 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.173+0000 7f67e36da640 1 -- 172.21.15.80:0/726820089 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f67c80a2b90 msgr2=0x7f67c80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:45.175 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.173+0000 7f67e36da640 1 --2- 172.21.15.80:0/726820089 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f67c80a2b90 0x7f67c80a5050 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f67cc000c00 tx=0x7f67cc0315d0 comp rx=0 tx=0).stop 2024-04-24T23:29:45.175 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.173+0000 7f67e36da640 1 -- 172.21.15.80:0/726820089 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f67dc100680 msgr2=0x7f67dc1969f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:45.175 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.173+0000 7f67e36da640 1 --2- 172.21.15.80:0/726820089 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f67dc100680 0x7f67dc1969f0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f67d800c970 tx=0x7f67d800ce30 comp rx=0 tx=0).stop 2024-04-24T23:29:45.175 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.174+0000 7f67e36da640 1 -- 172.21.15.80:0/726820089 shutdown_connections 2024-04-24T23:29:45.175 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.174+0000 7f67e36da640 1 --2- 172.21.15.80:0/726820089 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f67c80a2b90 0x7f67c80a5050 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:45.175 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.174+0000 7f67e36da640 1 --2- 172.21.15.80:0/726820089 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f67dc102230 0x7f67dc19dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:45.175 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.174+0000 7f67e36da640 1 --2- 172.21.15.80:0/726820089 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67dc101870 0x7f67dc196f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:45.175 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.174+0000 7f67e36da640 1 --2- 172.21.15.80:0/726820089 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f67dc100680 0x7f67dc1969f0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:45.175 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.174+0000 7f67e36da640 1 -- 172.21.15.80:0/726820089 >> 172.21.15.80:0/726820089 conn(0x7f67dc0fbbf0 msgr2=0x7f67dc104c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:45.175 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.174+0000 7f67e36da640 1 -- 172.21.15.80:0/726820089 shutdown_connections 2024-04-24T23:29:45.175 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.174+0000 7f67e36da640 1 -- 172.21.15.80:0/726820089 wait complete. 2024-04-24T23:29:45.184 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:29:45 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/4005212306' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T23:29:45.185 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:29:45 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/359884436' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T23:29:45.185 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:29:45 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/2759412716' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:29:45.186 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T23:29:45.300 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.292+0000 7f7defed4640 1 -- 172.21.15.80:0/1438002923 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7de8101870 msgr2=0x7f7de8101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:45.301 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.292+0000 7f7defed4640 1 --2- 172.21.15.80:0/1438002923 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7de8101870 0x7f7de8101cf0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f7dd8009900 tx=0x7f7dd802a410 comp rx=0 tx=0).stop 2024-04-24T23:29:45.301 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.292+0000 7f7defed4640 1 -- 172.21.15.80:0/1438002923 shutdown_connections 2024-04-24T23:29:45.301 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.292+0000 7f7defed4640 1 --2- 172.21.15.80:0/1438002923 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7de8102230 0x7f7de8108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:45.301 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.292+0000 7f7defed4640 1 --2- 172.21.15.80:0/1438002923 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7de8101870 0x7f7de8101cf0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:45.301 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.292+0000 7f7defed4640 1 --2- 172.21.15.80:0/1438002923 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7de8100680 0x7f7de8100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:45.301 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.292+0000 7f7defed4640 1 -- 172.21.15.80:0/1438002923 >> 172.21.15.80:0/1438002923 conn(0x7f7de80fbbf0 msgr2=0x7f7de80fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:45.301 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.292+0000 7f7defed4640 1 -- 172.21.15.80:0/1438002923 shutdown_connections 2024-04-24T23:29:45.301 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.292+0000 7f7defed4640 1 -- 172.21.15.80:0/1438002923 wait complete. 2024-04-24T23:29:45.302 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.293+0000 7f7defed4640 1 Processor -- start 2024-04-24T23:29:45.302 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.293+0000 7f7defed4640 1 -- start start 2024-04-24T23:29:45.302 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.293+0000 7f7defed4640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7de8100680 0x7f7de8069440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:45.302 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.293+0000 7f7defed4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7de8101870 0x7f7de819c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:45.302 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.293+0000 7f7defed4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7de8102230 0x7f7de819eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:45.302 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.293+0000 7f7defed4640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f7de810c3a0 con 0x7f7de8100680 2024-04-24T23:29:45.302 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.293+0000 7f7defed4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7de810c220 con 0x7f7de8101870 2024-04-24T23:29:45.302 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.293+0000 7f7defed4640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f7de810c520 con 0x7f7de8102230 2024-04-24T23:29:45.302 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.293+0000 7f7dedc70640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7de8100680 0x7f7de8069440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:45.302 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.293+0000 7f7dedc70640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7de8100680 0x7f7de8069440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:51940/0 (socket says 172.21.15.80:51940) 2024-04-24T23:29:45.303 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.293+0000 7f7dedc70640 1 -- 172.21.15.80:0/1754267977 learned_addr learned my addr 172.21.15.80:0/1754267977 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:29:45.303 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.293+0000 7f7dee471640 1 --2- 172.21.15.80:0/1754267977 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7de8102230 0x7f7de819eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:45.303 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.293+0000 7f7ded46f640 1 --2- 172.21.15.80:0/1754267977 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7de8101870 0x7f7de819c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:45.303 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.293+0000 7f7dedc70640 1 -- 172.21.15.80:0/1754267977 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7de8102230 msgr2=0x7f7de819eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:45.303 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.293+0000 7f7dedc70640 1 --2- 172.21.15.80:0/1754267977 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7de8102230 0x7f7de819eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:45.303 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.293+0000 7f7dedc70640 1 -- 172.21.15.80:0/1754267977 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7de8101870 msgr2=0x7f7de819c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:45.303 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.293+0000 7f7dedc70640 1 --2- 172.21.15.80:0/1754267977 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7de8101870 0x7f7de819c8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:45.303 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.293+0000 7f7dedc70640 1 -- 172.21.15.80:0/1754267977 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7de819f7f0 con 0x7f7de8100680 2024-04-24T23:29:45.303 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.293+0000 7f7dedc70640 1 --2- 172.21.15.80:0/1754267977 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7de8100680 0x7f7de8069440 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f7de400b450 tx=0x7f7de400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:45.303 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.293+0000 7f7ddeffd640 1 -- 172.21.15.80:0/1754267977 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de4015020 con 0x7f7de8100680 2024-04-24T23:29:45.304 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.293+0000 7f7ddeffd640 1 -- 172.21.15.80:0/1754267977 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7de4011d70 con 0x7f7de8100680 2024-04-24T23:29:45.304 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.293+0000 7f7defed4640 1 -- 172.21.15.80:0/1754267977 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7de819fa80 con 0x7f7de8100680 2024-04-24T23:29:45.304 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.294+0000 7f7ddeffd640 1 -- 172.21.15.80:0/1754267977 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de400f340 con 0x7f7de8100680 2024-04-24T23:29:45.304 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.294+0000 7f7defed4640 1 -- 172.21.15.80:0/1754267977 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7de819ffd0 con 0x7f7de8100680 2024-04-24T23:29:45.304 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.294+0000 7f7defed4640 1 -- 172.21.15.80:0/1754267977 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7de81079f0 con 0x7f7de8100680 2024-04-24T23:29:45.304 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.296+0000 7f7ddeffd640 1 -- 172.21.15.80:0/1754267977 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7de4018020 con 0x7f7de8100680 2024-04-24T23:29:45.304 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.296+0000 7f7ddeffd640 1 --2- 172.21.15.80:0/1754267977 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f7dd40a2b90 0x7f7dd40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:45.304 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.296+0000 7f7ded46f640 1 --2- 172.21.15.80:0/1754267977 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f7dd40a2b90 0x7f7dd40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:45.304 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.296+0000 7f7ddeffd640 1 -- 172.21.15.80:0/1754267977 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f7de40cf3a0 con 0x7f7de8100680 2024-04-24T23:29:45.305 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.297+0000 7f7ded46f640 1 --2- 172.21.15.80:0/1754267977 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f7dd40a2b90 0x7f7dd40a5050 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f7dd8031610 tx=0x7f7dd8000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:45.305 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.300+0000 7f7ddeffd640 1 -- 172.21.15.80:0/1754267977 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f7de401f020 con 0x7f7de8100680 2024-04-24T23:29:45.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:29:45 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/4005212306' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T23:29:45.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:29:45 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/359884436' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T23:29:45.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:29:45 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/2759412716' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:29:45.476 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.475+0000 7f7defed4640 1 -- 172.21.15.80:0/1754267977 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f7de802d3c0 con 0x7f7de8100680 2024-04-24T23:29:45.477 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.476+0000 7f7ddeffd640 1 -- 172.21.15.80:0/1754267977 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) ==== 104+0+11 (secure 0 0 0) 0x7f7de400d040 con 0x7f7de8100680 2024-04-24T23:29:45.477 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 32 2024-04-24T23:29:45.479 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.478+0000 7f7defed4640 1 -- 172.21.15.80:0/1754267977 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f7dd40a2b90 msgr2=0x7f7dd40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:45.479 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.478+0000 7f7defed4640 1 --2- 172.21.15.80:0/1754267977 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f7dd40a2b90 0x7f7dd40a5050 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f7dd8031610 tx=0x7f7dd8000bc0 comp rx=0 tx=0).stop 2024-04-24T23:29:45.479 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.478+0000 7f7defed4640 1 -- 172.21.15.80:0/1754267977 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7de8100680 msgr2=0x7f7de8069440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:45.479 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.478+0000 7f7defed4640 1 --2- 172.21.15.80:0/1754267977 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7de8100680 0x7f7de8069440 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f7de400b450 tx=0x7f7de400b910 comp rx=0 tx=0).stop 2024-04-24T23:29:45.479 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.478+0000 7f7defed4640 1 -- 172.21.15.80:0/1754267977 shutdown_connections 2024-04-24T23:29:45.479 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.478+0000 7f7defed4640 1 --2- 172.21.15.80:0/1754267977 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f7dd40a2b90 0x7f7dd40a5050 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:45.479 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.478+0000 7f7defed4640 1 --2- 172.21.15.80:0/1754267977 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7de8102230 0x7f7de819eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:45.479 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.478+0000 7f7defed4640 1 --2- 172.21.15.80:0/1754267977 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7de8101870 0x7f7de819c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:45.479 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.478+0000 7f7defed4640 1 --2- 172.21.15.80:0/1754267977 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7de8100680 0x7f7de8069440 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:45.480 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.478+0000 7f7defed4640 1 -- 172.21.15.80:0/1754267977 >> 172.21.15.80:0/1754267977 conn(0x7f7de80fbbf0 msgr2=0x7f7de8104bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:45.480 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.478+0000 7f7defed4640 1 -- 172.21.15.80:0/1754267977 shutdown_connections 2024-04-24T23:29:45.480 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.478+0000 7f7defed4640 1 -- 172.21.15.80:0/1754267977 wait complete. 2024-04-24T23:29:45.490 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T23:29:45.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:29:45 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/4005212306' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T23:29:45.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:29:45 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/359884436' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T23:29:45.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:29:45 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/2759412716' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:29:45.594 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.591+0000 7fcfbc163640 1 -- 172.21.15.80:0/3980257342 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcfb4100680 msgr2=0x7fcfb4100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:45.595 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.592+0000 7fcfbc163640 1 --2- 172.21.15.80:0/3980257342 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcfb4100680 0x7fcfb4100aa0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fcfb0009900 tx=0x7fcfb002a410 comp rx=0 tx=0).stop 2024-04-24T23:29:45.595 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.592+0000 7fcfbc163640 1 -- 172.21.15.80:0/3980257342 shutdown_connections 2024-04-24T23:29:45.595 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.592+0000 7fcfbc163640 1 --2- 172.21.15.80:0/3980257342 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcfb4102230 0x7fcfb4108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:45.595 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.592+0000 7fcfbc163640 1 --2- 172.21.15.80:0/3980257342 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcfb4101870 0x7fcfb4101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:45.595 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.592+0000 7fcfbc163640 1 --2- 172.21.15.80:0/3980257342 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcfb4100680 0x7fcfb4100aa0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:45.595 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.592+0000 7fcfbc163640 1 -- 172.21.15.80:0/3980257342 >> 172.21.15.80:0/3980257342 conn(0x7fcfb40fbbf0 msgr2=0x7fcfb40fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:45.595 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.592+0000 7fcfbc163640 1 -- 172.21.15.80:0/3980257342 shutdown_connections 2024-04-24T23:29:45.595 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.592+0000 7fcfbc163640 1 -- 172.21.15.80:0/3980257342 wait complete. 2024-04-24T23:29:45.595 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.592+0000 7fcfbc163640 1 Processor -- start 2024-04-24T23:29:45.595 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.592+0000 7fcfbc163640 1 -- start start 2024-04-24T23:29:45.595 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.592+0000 7fcfbc163640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcfb4100680 0x7fcfb4197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:45.595 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.592+0000 7fcfbc163640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcfb4101870 0x7fcfb4198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:45.595 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.592+0000 7fcfbc163640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcfb4102230 0x7fcfb419f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:45.596 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.592+0000 7fcfbc163640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fcfb410c350 con 0x7fcfb4100680 2024-04-24T23:29:45.596 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.592+0000 7fcfbc163640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcfb410c1d0 con 0x7fcfb4101870 2024-04-24T23:29:45.596 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.592+0000 7fcfbc163640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fcfb410c4d0 con 0x7fcfb4102230 2024-04-24T23:29:45.596 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.592+0000 7fcfb9eff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcfb4100680 0x7fcfb4197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:45.596 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.593+0000 7fcfb9eff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcfb4100680 0x7fcfb4197b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:51958/0 (socket says 172.21.15.80:51958) 2024-04-24T23:29:45.596 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.593+0000 7fcfb9eff640 1 -- 172.21.15.80:0/2263964437 learned_addr learned my addr 172.21.15.80:0/2263964437 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:29:45.596 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.593+0000 7fcfba700640 1 --2- 172.21.15.80:0/2263964437 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcfb4102230 0x7fcfb419f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:45.596 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.593+0000 7fcfb96fe640 1 --2- 172.21.15.80:0/2263964437 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcfb4101870 0x7fcfb4198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:45.596 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.593+0000 7fcfb9eff640 1 -- 172.21.15.80:0/2263964437 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcfb4102230 msgr2=0x7fcfb419f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:45.596 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.593+0000 7fcfb9eff640 1 --2- 172.21.15.80:0/2263964437 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcfb4102230 0x7fcfb419f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:45.596 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.593+0000 7fcfb9eff640 1 -- 172.21.15.80:0/2263964437 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcfb4101870 msgr2=0x7fcfb4198090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:45.596 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.593+0000 7fcfb9eff640 1 --2- 172.21.15.80:0/2263964437 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcfb4101870 0x7fcfb4198090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:45.596 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.593+0000 7fcfb9eff640 1 -- 172.21.15.80:0/2263964437 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfb419f770 con 0x7fcfb4100680 2024-04-24T23:29:45.596 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.593+0000 7fcfba700640 1 --2- 172.21.15.80:0/2263964437 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcfb4102230 0x7fcfb419f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:29:45.596 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.593+0000 7fcfb96fe640 1 --2- 172.21.15.80:0/2263964437 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcfb4101870 0x7fcfb4198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:29:45.597 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.593+0000 7fcfb9eff640 1 --2- 172.21.15.80:0/2263964437 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcfb4100680 0x7fcfb4197b50 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fcfb0031610 tx=0x7fcfb00043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:45.597 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.593+0000 7fcfaaffd640 1 -- 172.21.15.80:0/2263964437 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb003f030 con 0x7fcfb4100680 2024-04-24T23:29:45.597 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.593+0000 7fcfaaffd640 1 -- 172.21.15.80:0/2263964437 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfb0003ce0 con 0x7fcfb4100680 2024-04-24T23:29:45.597 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.593+0000 7fcfbc163640 1 -- 172.21.15.80:0/2263964437 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfb419fa00 con 0x7fcfb4100680 2024-04-24T23:29:45.597 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.593+0000 7fcfaaffd640 1 -- 172.21.15.80:0/2263964437 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb0035070 con 0x7fcfb4100680 2024-04-24T23:29:45.597 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.593+0000 7fcfbc163640 1 -- 172.21.15.80:0/2263964437 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfb419ff70 con 0x7fcfb4100680 2024-04-24T23:29:45.599 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.594+0000 7fcfbc163640 1 -- 172.21.15.80:0/2263964437 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfb41a0140 con 0x7fcfb4100680 2024-04-24T23:29:45.599 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.595+0000 7fcfaaffd640 1 -- 172.21.15.80:0/2263964437 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fcfb0042030 con 0x7fcfb4100680 2024-04-24T23:29:45.599 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.596+0000 7fcfaaffd640 1 --2- 172.21.15.80:0/2263964437 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fcfa00a2b90 0x7fcfa00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:45.599 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.596+0000 7fcfb96fe640 1 --2- 172.21.15.80:0/2263964437 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fcfa00a2b90 0x7fcfa00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:45.599 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.596+0000 7fcfaaffd640 1 -- 172.21.15.80:0/2263964437 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fcfb00efb70 con 0x7fcfb4100680 2024-04-24T23:29:45.600 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.597+0000 7fcfb96fe640 1 --2- 172.21.15.80:0/2263964437 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fcfa00a2b90 0x7fcfa00a5050 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fcfb4199060 tx=0x7fcfa4005d30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:45.603 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.601+0000 7fcfaaffd640 1 -- 172.21.15.80:0/2263964437 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fcfb00bc700 con 0x7fcfb4100680 2024-04-24T23:29:45.777 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.776+0000 7fcfbc163640 1 -- 172.21.15.80:0/2263964437 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fcfb41a0310 con 0x7fcfb4100680 2024-04-24T23:29:45.777 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.776+0000 7fcfaaffd640 1 -- 172.21.15.80:0/2263964437 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) ==== 100+0+11 (secure 0 0 0) 0x7fcfb00bc000 con 0x7fcfb4100680 2024-04-24T23:29:45.777 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 64 2024-04-24T23:29:45.779 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.778+0000 7fcfbc163640 1 -- 172.21.15.80:0/2263964437 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fcfa00a2b90 msgr2=0x7fcfa00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:45.779 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.778+0000 7fcfbc163640 1 --2- 172.21.15.80:0/2263964437 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fcfa00a2b90 0x7fcfa00a5050 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fcfb4199060 tx=0x7fcfa4005d30 comp rx=0 tx=0).stop 2024-04-24T23:29:45.779 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.778+0000 7fcfbc163640 1 -- 172.21.15.80:0/2263964437 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcfb4100680 msgr2=0x7fcfb4197b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:45.779 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.778+0000 7fcfbc163640 1 --2- 172.21.15.80:0/2263964437 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcfb4100680 0x7fcfb4197b50 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fcfb0031610 tx=0x7fcfb00043b0 comp rx=0 tx=0).stop 2024-04-24T23:29:45.779 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.778+0000 7fcfbc163640 1 -- 172.21.15.80:0/2263964437 shutdown_connections 2024-04-24T23:29:45.779 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.778+0000 7fcfbc163640 1 --2- 172.21.15.80:0/2263964437 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fcfa00a2b90 0x7fcfa00a5050 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:45.779 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.778+0000 7fcfbc163640 1 --2- 172.21.15.80:0/2263964437 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcfb4102230 0x7fcfb419f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:45.779 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.778+0000 7fcfbc163640 1 --2- 172.21.15.80:0/2263964437 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcfb4101870 0x7fcfb4198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:45.779 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.778+0000 7fcfbc163640 1 --2- 172.21.15.80:0/2263964437 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcfb4100680 0x7fcfb4197b50 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:45.779 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.778+0000 7fcfbc163640 1 -- 172.21.15.80:0/2263964437 >> 172.21.15.80:0/2263964437 conn(0x7fcfb40fbbf0 msgr2=0x7fcfb4068900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:45.780 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.778+0000 7fcfbc163640 1 -- 172.21.15.80:0/2263964437 shutdown_connections 2024-04-24T23:29:45.780 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.778+0000 7fcfbc163640 1 -- 172.21.15.80:0/2263964437 wait complete. 2024-04-24T23:29:45.790 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-24T23:29:45.891 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.890+0000 7f27de6a5640 1 -- 172.21.15.80:0/4288767651 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f27d8102230 msgr2=0x7f27d8108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:45.892 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.890+0000 7f27de6a5640 1 --2- 172.21.15.80:0/4288767651 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f27d8102230 0x7f27d8108ac0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f27d0009ca0 tx=0x7f27d002a8c0 comp rx=0 tx=0).stop 2024-04-24T23:29:45.892 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.890+0000 7f27de6a5640 1 -- 172.21.15.80:0/4288767651 shutdown_connections 2024-04-24T23:29:45.892 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.890+0000 7f27de6a5640 1 --2- 172.21.15.80:0/4288767651 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f27d8102230 0x7f27d8108ac0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:45.892 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.890+0000 7f27de6a5640 1 --2- 172.21.15.80:0/4288767651 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27d8101870 0x7f27d8101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:45.892 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.890+0000 7f27de6a5640 1 --2- 172.21.15.80:0/4288767651 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f27d8100680 0x7f27d8100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:45.892 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.890+0000 7f27de6a5640 1 -- 172.21.15.80:0/4288767651 >> 172.21.15.80:0/4288767651 conn(0x7f27d80fbbf0 msgr2=0x7f27d80fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:45.892 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.890+0000 7f27de6a5640 1 -- 172.21.15.80:0/4288767651 shutdown_connections 2024-04-24T23:29:45.892 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.890+0000 7f27de6a5640 1 -- 172.21.15.80:0/4288767651 wait complete. 2024-04-24T23:29:45.894 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.891+0000 7f27de6a5640 1 Processor -- start 2024-04-24T23:29:45.895 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.891+0000 7f27de6a5640 1 -- start start 2024-04-24T23:29:45.895 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.891+0000 7f27de6a5640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f27d8100680 0x7f27d8069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:45.895 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.891+0000 7f27de6a5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27d8101870 0x7f27d819c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:45.895 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.891+0000 7f27de6a5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f27d8102230 0x7f27d819ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:45.895 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.891+0000 7f27de6a5640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f27d810c300 con 0x7f27d8100680 2024-04-24T23:29:45.895 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.891+0000 7f27de6a5640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f27d810c180 con 0x7f27d8102230 2024-04-24T23:29:45.895 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.891+0000 7f27de6a5640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f27d810c480 con 0x7f27d8101870 2024-04-24T23:29:45.895 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.891+0000 7f27d7fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f27d8100680 0x7f27d8069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:45.896 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.891+0000 7f27d7fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f27d8100680 0x7f27d8069460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:51970/0 (socket says 172.21.15.80:51970) 2024-04-24T23:29:45.896 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.891+0000 7f27d7fff640 1 -- 172.21.15.80:0/3621144815 learned_addr learned my addr 172.21.15.80:0/3621144815 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:29:45.896 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.891+0000 7f27d77fe640 1 --2- 172.21.15.80:0/3621144815 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27d8101870 0x7f27d819c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:45.896 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.891+0000 7f27dcc42640 1 --2- 172.21.15.80:0/3621144815 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f27d8102230 0x7f27d819ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:45.896 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.892+0000 7f27d7fff640 1 -- 172.21.15.80:0/3621144815 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27d8101870 msgr2=0x7f27d819c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:45.896 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.892+0000 7f27d7fff640 1 --2- 172.21.15.80:0/3621144815 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27d8101870 0x7f27d819c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:45.896 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.892+0000 7f27d7fff640 1 -- 172.21.15.80:0/3621144815 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f27d8102230 msgr2=0x7f27d819ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:45.896 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.892+0000 7f27d7fff640 1 --2- 172.21.15.80:0/3621144815 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f27d8102230 0x7f27d819ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:45.896 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.892+0000 7f27d7fff640 1 -- 172.21.15.80:0/3621144815 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27d819f640 con 0x7f27d8100680 2024-04-24T23:29:45.896 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.892+0000 7f27dcc42640 1 --2- 172.21.15.80:0/3621144815 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f27d8102230 0x7f27d819ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:29:45.896 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.892+0000 7f27d7fff640 1 --2- 172.21.15.80:0/3621144815 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f27d8100680 0x7f27d8069460 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f27c800c970 tx=0x7f27c800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:45.896 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.892+0000 7f27d57fa640 1 -- 172.21.15.80:0/3621144815 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27c8007c10 con 0x7f27d8100680 2024-04-24T23:29:45.896 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.892+0000 7f27d57fa640 1 -- 172.21.15.80:0/3621144815 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27c8007db0 con 0x7f27d8100680 2024-04-24T23:29:45.896 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.892+0000 7f27de6a5640 1 -- 172.21.15.80:0/3621144815 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27d819f930 con 0x7f27d8100680 2024-04-24T23:29:45.897 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.892+0000 7f27d57fa640 1 -- 172.21.15.80:0/3621144815 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27c800f940 con 0x7f27d8100680 2024-04-24T23:29:45.897 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.892+0000 7f27de6a5640 1 -- 172.21.15.80:0/3621144815 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27d819fe80 con 0x7f27d8100680 2024-04-24T23:29:45.897 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.893+0000 7f27de6a5640 1 -- 172.21.15.80:0/3621144815 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27d8107940 con 0x7f27d8100680 2024-04-24T23:29:45.897 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.894+0000 7f27d57fa640 1 -- 172.21.15.80:0/3621144815 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f27c800fae0 con 0x7f27d8100680 2024-04-24T23:29:45.897 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.894+0000 7f27d57fa640 1 --2- 172.21.15.80:0/3621144815 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f27c00a2b90 0x7f27c00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:45.898 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.895+0000 7f27d77fe640 1 --2- 172.21.15.80:0/3621144815 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f27c00a2b90 0x7f27c00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:45.898 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.895+0000 7f27d57fa640 1 -- 172.21.15.80:0/3621144815 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f27c80cfb30 con 0x7f27d8100680 2024-04-24T23:29:45.898 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.895+0000 7f27d77fe640 1 --2- 172.21.15.80:0/3621144815 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f27c00a2b90 0x7f27c00a5050 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f27d80691d0 tx=0x7f27cc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:45.899 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:45.898+0000 7f27d57fa640 1 -- 172.21.15.80:0/3621144815 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f27c809c7e0 con 0x7f27d8100680 2024-04-24T23:29:46.076 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.075+0000 7f27de6a5640 1 -- 172.21.15.80:0/3621144815 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f27d802d300 con 0x7f27d8100680 2024-04-24T23:29:46.076 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.075+0000 7f27d57fa640 1 -- 172.21.15.80:0/3621144815 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) ==== 103+0+11 (secure 0 0 0) 0x7f27c809c0e0 con 0x7f27d8100680 2024-04-24T23:29:46.076 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 32 2024-04-24T23:29:46.078 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.077+0000 7f27de6a5640 1 -- 172.21.15.80:0/3621144815 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f27c00a2b90 msgr2=0x7f27c00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:46.078 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.077+0000 7f27de6a5640 1 --2- 172.21.15.80:0/3621144815 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f27c00a2b90 0x7f27c00a5050 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f27d80691d0 tx=0x7f27cc005f90 comp rx=0 tx=0).stop 2024-04-24T23:29:46.078 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.077+0000 7f27de6a5640 1 -- 172.21.15.80:0/3621144815 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f27d8100680 msgr2=0x7f27d8069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:46.078 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.077+0000 7f27de6a5640 1 --2- 172.21.15.80:0/3621144815 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f27d8100680 0x7f27d8069460 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f27c800c970 tx=0x7f27c800ce30 comp rx=0 tx=0).stop 2024-04-24T23:29:46.078 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.077+0000 7f27de6a5640 1 -- 172.21.15.80:0/3621144815 shutdown_connections 2024-04-24T23:29:46.078 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.077+0000 7f27de6a5640 1 --2- 172.21.15.80:0/3621144815 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f27c00a2b90 0x7f27c00a5050 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:46.078 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.077+0000 7f27de6a5640 1 --2- 172.21.15.80:0/3621144815 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f27d8102230 0x7f27d819ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:46.079 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.077+0000 7f27de6a5640 1 --2- 172.21.15.80:0/3621144815 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27d8101870 0x7f27d819c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:46.079 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.077+0000 7f27de6a5640 1 --2- 172.21.15.80:0/3621144815 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f27d8100680 0x7f27d8069460 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:46.079 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.077+0000 7f27de6a5640 1 -- 172.21.15.80:0/3621144815 >> 172.21.15.80:0/3621144815 conn(0x7f27d80fbbf0 msgr2=0x7f27d8104b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:46.079 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.077+0000 7f27de6a5640 1 -- 172.21.15.80:0/3621144815 shutdown_connections 2024-04-24T23:29:46.079 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.077+0000 7f27de6a5640 1 -- 172.21.15.80:0/3621144815 wait complete. 2024-04-24T23:29:46.088 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:29:46 smithi080 ceph-mon[29152]: pgmap v373: 129 pgs: 129 active+clean; 640 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 8 op/s 2024-04-24T23:29:46.088 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:29:46 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/726820089' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:29:46.088 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:29:46 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/1754267977' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T23:29:46.088 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:29:46 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/2263964437' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T23:29:46.089 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-04-24T23:29:46.089 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:29:46.190 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.186+0000 7fe837314640 1 -- 172.21.15.80:0/1391196119 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe830102230 msgr2=0x7fe830108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:46.190 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.186+0000 7fe837314640 1 --2- 172.21.15.80:0/1391196119 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe830102230 0x7fe830108ac0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fe828009ca0 tx=0x7fe82802a8c0 comp rx=0 tx=0).stop 2024-04-24T23:29:46.190 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.186+0000 7fe837314640 1 -- 172.21.15.80:0/1391196119 shutdown_connections 2024-04-24T23:29:46.190 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.186+0000 7fe837314640 1 --2- 172.21.15.80:0/1391196119 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe830102230 0x7fe830108ac0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:46.190 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.186+0000 7fe837314640 1 --2- 172.21.15.80:0/1391196119 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe830101870 0x7fe830101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:46.190 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.186+0000 7fe837314640 1 --2- 172.21.15.80:0/1391196119 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe830100680 0x7fe830100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:46.190 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.186+0000 7fe837314640 1 -- 172.21.15.80:0/1391196119 >> 172.21.15.80:0/1391196119 conn(0x7fe8300fbbf0 msgr2=0x7fe8300fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:46.190 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.187+0000 7fe837314640 1 -- 172.21.15.80:0/1391196119 shutdown_connections 2024-04-24T23:29:46.190 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.187+0000 7fe837314640 1 -- 172.21.15.80:0/1391196119 wait complete. 2024-04-24T23:29:46.190 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.187+0000 7fe837314640 1 Processor -- start 2024-04-24T23:29:46.191 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.187+0000 7fe837314640 1 -- start start 2024-04-24T23:29:46.191 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.187+0000 7fe837314640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe830100680 0x7fe830196900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:46.191 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.187+0000 7fe837314640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe830101870 0x7fe830196e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:46.191 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.187+0000 7fe837314640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe830102230 0x7fe83019deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:46.191 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.187+0000 7fe837314640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fe83010c370 con 0x7fe830101870 2024-04-24T23:29:46.191 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.187+0000 7fe837314640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe83010c1f0 con 0x7fe830102230 2024-04-24T23:29:46.191 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.187+0000 7fe837314640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe83010c4f0 con 0x7fe830100680 2024-04-24T23:29:46.191 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.187+0000 7fe8348af640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe830101870 0x7fe830196e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:46.191 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.187+0000 7fe8348af640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe830101870 0x7fe830196e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:51994/0 (socket says 172.21.15.80:51994) 2024-04-24T23:29:46.191 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.187+0000 7fe8348af640 1 -- 172.21.15.80:0/905843497 learned_addr learned my addr 172.21.15.80:0/905843497 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:29:46.191 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.187+0000 7fe8350b0640 1 --2- 172.21.15.80:0/905843497 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe830100680 0x7fe830196900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:46.191 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.188+0000 7fe8358b1640 1 --2- 172.21.15.80:0/905843497 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe830102230 0x7fe83019deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:46.191 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.188+0000 7fe8348af640 1 -- 172.21.15.80:0/905843497 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe830100680 msgr2=0x7fe830196900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:46.191 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.188+0000 7fe8348af640 1 --2- 172.21.15.80:0/905843497 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe830100680 0x7fe830196900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:46.192 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.188+0000 7fe8348af640 1 -- 172.21.15.80:0/905843497 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe830102230 msgr2=0x7fe83019deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:46.192 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.188+0000 7fe8348af640 1 --2- 172.21.15.80:0/905843497 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe830102230 0x7fe83019deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:46.192 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.188+0000 7fe8348af640 1 -- 172.21.15.80:0/905843497 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe83019e520 con 0x7fe830101870 2024-04-24T23:29:46.192 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.188+0000 7fe8358b1640 1 --2- 172.21.15.80:0/905843497 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe830102230 0x7fe83019deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:29:46.192 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.188+0000 7fe8348af640 1 --2- 172.21.15.80:0/905843497 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe830101870 0x7fe830196e40 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fe82000c920 tx=0x7fe82000cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:46.192 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.188+0000 7fe8267fc640 1 -- 172.21.15.80:0/905843497 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe820007bb0 con 0x7fe830101870 2024-04-24T23:29:46.192 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.188+0000 7fe8267fc640 1 -- 172.21.15.80:0/905843497 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe820007d50 con 0x7fe830101870 2024-04-24T23:29:46.192 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.188+0000 7fe837314640 1 -- 172.21.15.80:0/905843497 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe83019e810 con 0x7fe830101870 2024-04-24T23:29:46.192 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.188+0000 7fe8267fc640 1 -- 172.21.15.80:0/905843497 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8200187d0 con 0x7fe830101870 2024-04-24T23:29:46.192 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.188+0000 7fe837314640 1 -- 172.21.15.80:0/905843497 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe83019ee30 con 0x7fe830101870 2024-04-24T23:29:46.192 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.189+0000 7fe837314640 1 -- 172.21.15.80:0/905843497 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8301078d0 con 0x7fe830101870 2024-04-24T23:29:46.193 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.190+0000 7fe8267fc640 1 -- 172.21.15.80:0/905843497 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe82000f450 con 0x7fe830101870 2024-04-24T23:29:46.193 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.191+0000 7fe8267fc640 1 --2- 172.21.15.80:0/905843497 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fe81c0a2c10 0x7fe81c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:46.193 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.191+0000 7fe8350b0640 1 --2- 172.21.15.80:0/905843497 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fe81c0a2c10 0x7fe81c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:46.193 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.191+0000 7fe8267fc640 1 -- 172.21.15.80:0/905843497 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fe8200cf4f0 con 0x7fe830101870 2024-04-24T23:29:46.193 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.191+0000 7fe8350b0640 1 --2- 172.21.15.80:0/905843497 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fe81c0a2c10 0x7fe81c0a50d0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fe8301016d0 tx=0x7fe82c009110 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:46.195 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.195+0000 7fe8267fc640 1 -- 172.21.15.80:0/905843497 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fe820014030 con 0x7fe830101870 2024-04-24T23:29:46.406 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.406+0000 7fe837314640 1 -- 172.21.15.80:0/905843497 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe83002d170 con 0x7fe830101870 2024-04-24T23:29:46.407 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.407+0000 7fe8267fc640 1 -- 172.21.15.80:0/905843497 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11872 (secure 0 0 0) 0x7fe82000de20 con 0x7fe830101870 2024-04-24T23:29:46.407 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-24T23:29:46.408 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/891591085","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":891591085},{"type":"v1","addr":"172.21.15.80:6835","nonce":891591085}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6837/259529026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6836","nonce":259529026},{"type":"v1","addr":"172.21.15.80:6837","nonce":259529026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14646,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6839/4194284854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6838","nonce":4194284854},{"type":"v1","addr":"172.21.15.80:6839","nonce":4194284854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14658,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6841/3145730958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6840","nonce":3145730958},{"type":"v1","addr":"172.21.15.80:6841","nonce":3145730958}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24395,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/3736784385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3736784385},{"type":"v1","addr":"172.21.15.134:6833","nonce":3736784385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/124777462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":124777462},{"type":"v1","addr":"172.21.15.148:6833","nonce":124777462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T23:27:19.300566+0000","modified":"2024-04-24T23:29:42.158567+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24496,"mds_1":24488,"mds_2":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/1399801120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1399801120},{"type":"v1","addr":"172.21.15.134:6835","nonce":1399801120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/430512704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":430512704},{"type":"v1","addr":"172.21.15.134:6837","nonce":430512704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/3002319310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3002319310},{"type":"v1","addr":"172.21.15.148:6835","nonce":3002319310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.148:6837/856528403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":856528403},{"type":"v1","addr":"172.21.15.148:6837","nonce":856528403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6839/1743815021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":1743815021},{"type":"v1","addr":"172.21.15.134:6839","nonce":1743815021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.148:6839/2952757594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":2952757594},{"type":"v1","addr":"172.21.15.148:6839","nonce":2952757594}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24496,"qdb_cluster":[24496,24472,24488]},"id":1}]} 2024-04-24T23:29:46.408 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 25 2024-04-24T23:29:46.410 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.408+0000 7fe837314640 1 -- 172.21.15.80:0/905843497 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fe81c0a2c10 msgr2=0x7fe81c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:46.410 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.408+0000 7fe837314640 1 --2- 172.21.15.80:0/905843497 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fe81c0a2c10 0x7fe81c0a50d0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fe8301016d0 tx=0x7fe82c009110 comp rx=0 tx=0).stop 2024-04-24T23:29:46.410 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.409+0000 7fe837314640 1 -- 172.21.15.80:0/905843497 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe830101870 msgr2=0x7fe830196e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:46.410 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.409+0000 7fe837314640 1 --2- 172.21.15.80:0/905843497 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe830101870 0x7fe830196e40 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fe82000c920 tx=0x7fe82000cd20 comp rx=0 tx=0).stop 2024-04-24T23:29:46.410 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.409+0000 7fe837314640 1 -- 172.21.15.80:0/905843497 shutdown_connections 2024-04-24T23:29:46.410 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.409+0000 7fe837314640 1 --2- 172.21.15.80:0/905843497 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fe81c0a2c10 0x7fe81c0a50d0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:46.410 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.409+0000 7fe837314640 1 --2- 172.21.15.80:0/905843497 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe830102230 0x7fe83019deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:46.410 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.409+0000 7fe837314640 1 --2- 172.21.15.80:0/905843497 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe830101870 0x7fe830196e40 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:46.410 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.409+0000 7fe837314640 1 --2- 172.21.15.80:0/905843497 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe830100680 0x7fe830196900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:46.410 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.409+0000 7fe837314640 1 -- 172.21.15.80:0/905843497 >> 172.21.15.80:0/905843497 conn(0x7fe8300fbbf0 msgr2=0x7fe830104aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:46.410 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.409+0000 7fe837314640 1 -- 172.21.15.80:0/905843497 shutdown_connections 2024-04-24T23:29:46.410 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.409+0000 7fe837314640 1 -- 172.21.15.80:0/905843497 wait complete. 2024-04-24T23:29:46.421 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-04-24T23:29:46.421 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-04-24T23:29:46.421 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T23:29:46.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:29:46 smithi134 ceph-mon[32619]: pgmap v373: 129 pgs: 129 active+clean; 640 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 8 op/s 2024-04-24T23:29:46.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:29:46 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/726820089' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:29:46.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:29:46 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/1754267977' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T23:29:46.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:29:46 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/2263964437' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T23:29:46.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:29:46 smithi148 ceph-mon[32528]: pgmap v373: 129 pgs: 129 active+clean; 640 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 8 op/s 2024-04-24T23:29:46.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:29:46 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/726820089' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:29:46.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:29:46 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/1754267977' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T23:29:46.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:29:46 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/2263964437' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T23:29:46.527 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.518+0000 7ff6b5a95640 1 -- 172.21.15.80:0/1358432394 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff6b0101890 msgr2=0x7ff6b0101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:46.527 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.518+0000 7ff6b5a95640 1 --2- 172.21.15.80:0/1358432394 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff6b0101890 0x7ff6b0101d10 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7ff6a4009900 tx=0x7ff6a402a410 comp rx=0 tx=0).stop 2024-04-24T23:29:46.527 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.518+0000 7ff6b5a95640 1 -- 172.21.15.80:0/1358432394 shutdown_connections 2024-04-24T23:29:46.527 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.518+0000 7ff6b5a95640 1 --2- 172.21.15.80:0/1358432394 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff6b0102250 0x7ff6b0108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:46.528 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.518+0000 7ff6b5a95640 1 --2- 172.21.15.80:0/1358432394 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff6b0101890 0x7ff6b0101d10 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:46.528 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.518+0000 7ff6b5a95640 1 --2- 172.21.15.80:0/1358432394 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff6b01006a0 0x7ff6b0100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:46.528 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.518+0000 7ff6b5a95640 1 -- 172.21.15.80:0/1358432394 >> 172.21.15.80:0/1358432394 conn(0x7ff6b00fbc30 msgr2=0x7ff6b00fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:46.528 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.518+0000 7ff6b5a95640 1 -- 172.21.15.80:0/1358432394 shutdown_connections 2024-04-24T23:29:46.528 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.518+0000 7ff6b5a95640 1 -- 172.21.15.80:0/1358432394 wait complete. 2024-04-24T23:29:46.528 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.519+0000 7ff6b5a95640 1 Processor -- start 2024-04-24T23:29:46.528 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.519+0000 7ff6b5a95640 1 -- start start 2024-04-24T23:29:46.528 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.519+0000 7ff6b5a95640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff6b01006a0 0x7ff6b006a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:46.528 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.519+0000 7ff6b5a95640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff6b0101890 0x7ff6b019c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:46.528 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.519+0000 7ff6b5a95640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff6b0102250 0x7ff6b019eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:46.529 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.519+0000 7ff6b5a95640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7ff6b010c3a0 con 0x7ff6b0101890 2024-04-24T23:29:46.529 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.519+0000 7ff6b5a95640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff6b010c220 con 0x7ff6b01006a0 2024-04-24T23:29:46.529 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.519+0000 7ff6b5a95640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff6b010c520 con 0x7ff6b0102250 2024-04-24T23:29:46.529 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.519+0000 7ff6aeffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff6b0101890 0x7ff6b019c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:46.529 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.519+0000 7ff6aeffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff6b0101890 0x7ff6b019c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:52014/0 (socket says 172.21.15.80:52014) 2024-04-24T23:29:46.529 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.519+0000 7ff6aeffd640 1 -- 172.21.15.80:0/3133829482 learned_addr learned my addr 172.21.15.80:0/3133829482 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:29:46.530 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.519+0000 7ff6affff640 1 --2- 172.21.15.80:0/3133829482 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff6b0102250 0x7ff6b019eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:46.530 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.519+0000 7ff6af7fe640 1 --2- 172.21.15.80:0/3133829482 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff6b01006a0 0x7ff6b006a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:46.531 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.519+0000 7ff6aeffd640 1 -- 172.21.15.80:0/3133829482 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff6b0102250 msgr2=0x7ff6b019eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:46.531 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.519+0000 7ff6aeffd640 1 --2- 172.21.15.80:0/3133829482 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff6b0102250 0x7ff6b019eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:46.531 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.519+0000 7ff6aeffd640 1 -- 172.21.15.80:0/3133829482 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff6b01006a0 msgr2=0x7ff6b006a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:46.531 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.519+0000 7ff6aeffd640 1 --2- 172.21.15.80:0/3133829482 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff6b01006a0 0x7ff6b006a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:46.531 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.519+0000 7ff6aeffd640 1 -- 172.21.15.80:0/3133829482 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6b019f780 con 0x7ff6b0101890 2024-04-24T23:29:46.531 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.519+0000 7ff6af7fe640 1 --2- 172.21.15.80:0/3133829482 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff6b01006a0 0x7ff6b006a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:29:46.531 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.520+0000 7ff6aeffd640 1 --2- 172.21.15.80:0/3133829482 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff6b0101890 0x7ff6b019c880 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7ff6a4009230 tx=0x7ff6a4004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:46.531 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.520+0000 7ff6acff9640 1 -- 172.21.15.80:0/3133829482 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6a403f030 con 0x7ff6b0101890 2024-04-24T23:29:46.531 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.520+0000 7ff6acff9640 1 -- 172.21.15.80:0/3133829482 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6a4003ce0 con 0x7ff6b0101890 2024-04-24T23:29:46.531 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.520+0000 7ff6b5a95640 1 -- 172.21.15.80:0/3133829482 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6b019fa10 con 0x7ff6b0101890 2024-04-24T23:29:46.531 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.520+0000 7ff6acff9640 1 -- 172.21.15.80:0/3133829482 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6a4035070 con 0x7ff6b0101890 2024-04-24T23:29:46.531 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.520+0000 7ff6b5a95640 1 -- 172.21.15.80:0/3133829482 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6b019ff80 con 0x7ff6b0101890 2024-04-24T23:29:46.531 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.521+0000 7ff6b5a95640 1 -- 172.21.15.80:0/3133829482 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6b01079c0 con 0x7ff6b0101890 2024-04-24T23:29:46.531 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.522+0000 7ff6acff9640 1 -- 172.21.15.80:0/3133829482 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff6a4042030 con 0x7ff6b0101890 2024-04-24T23:29:46.532 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.522+0000 7ff6acff9640 1 --2- 172.21.15.80:0/3133829482 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7ff6980a2b90 0x7ff6980a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:46.532 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.522+0000 7ff6af7fe640 1 --2- 172.21.15.80:0/3133829482 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7ff6980a2b90 0x7ff6980a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:46.532 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.522+0000 7ff6acff9640 1 -- 172.21.15.80:0/3133829482 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7ff6a40efeb0 con 0x7ff6b0101890 2024-04-24T23:29:46.532 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.523+0000 7ff6af7fe640 1 --2- 172.21.15.80:0/3133829482 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7ff6980a2b90 0x7ff6980a5050 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7ff6b01016f0 tx=0x7ff6a0005e70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:46.532 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.526+0000 7ff6acff9640 1 -- 172.21.15.80:0/3133829482 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff6a40bca40 con 0x7ff6b0101890 2024-04-24T23:29:46.698 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.697+0000 7ff6b5a95640 1 -- 172.21.15.80:0/3133829482 --> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff6b0061b80 con 0x7ff6980a2b90 2024-04-24T23:29:46.707 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.706+0000 7ff6acff9640 1 -- 172.21.15.80:0/3133829482 <== mgr.14152 v2:172.21.15.80:6800/3830596922 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340774 (secure 0 0 0) 0x7ff6b0061b80 con 0x7ff6980a2b90 2024-04-24T23:29:46.711 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-24T23:29:46.713 INFO:teuthology.orchestra.run.smithi080.stderr:dumped all 2024-04-24T23:29:46.715 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.710+0000 7ff6b5a95640 1 -- 172.21.15.80:0/3133829482 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7ff6980a2b90 msgr2=0x7ff6980a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:46.715 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.710+0000 7ff6b5a95640 1 --2- 172.21.15.80:0/3133829482 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7ff6980a2b90 0x7ff6980a5050 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7ff6b01016f0 tx=0x7ff6a0005e70 comp rx=0 tx=0).stop 2024-04-24T23:29:46.715 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.710+0000 7ff6b5a95640 1 -- 172.21.15.80:0/3133829482 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff6b0101890 msgr2=0x7ff6b019c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:46.715 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.710+0000 7ff6b5a95640 1 --2- 172.21.15.80:0/3133829482 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff6b0101890 0x7ff6b019c880 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7ff6a4009230 tx=0x7ff6a4004360 comp rx=0 tx=0).stop 2024-04-24T23:29:46.715 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.710+0000 7ff6b5a95640 1 -- 172.21.15.80:0/3133829482 shutdown_connections 2024-04-24T23:29:46.715 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.710+0000 7ff6b5a95640 1 --2- 172.21.15.80:0/3133829482 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7ff6980a2b90 0x7ff6980a5050 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:46.715 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.710+0000 7ff6b5a95640 1 --2- 172.21.15.80:0/3133829482 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff6b0102250 0x7ff6b019eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:46.715 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.710+0000 7ff6b5a95640 1 --2- 172.21.15.80:0/3133829482 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff6b0101890 0x7ff6b019c880 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:46.715 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.710+0000 7ff6b5a95640 1 --2- 172.21.15.80:0/3133829482 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff6b01006a0 0x7ff6b006a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:46.716 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.710+0000 7ff6b5a95640 1 -- 172.21.15.80:0/3133829482 >> 172.21.15.80:0/3133829482 conn(0x7ff6b00fbc30 msgr2=0x7ff6b0104b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:46.716 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.710+0000 7ff6b5a95640 1 -- 172.21.15.80:0/3133829482 shutdown_connections 2024-04-24T23:29:46.716 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.710+0000 7ff6b5a95640 1 -- 172.21.15.80:0/3133829482 wait complete. 2024-04-24T23:29:46.722 INFO:teuthology.orchestra.run.smithi080.stdout:{"pg_ready":true,"pg_map":{"version":373,"stamp":"2024-04-24T23:29:44.919286+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655012,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":867,"num_read_kb":550,"num_write":357,"num_write_kb":3063,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":1181547,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":790,"ondisk_log_size":790,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":365940,"kb_used_data":12584,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124575884,"statfs":{"total":1151940427776,"available":1151565705216,"internally_reserved":0,"allocated":12886016,"data_stored":8291294,"data_compressed":462777,"data_compressed_allocated":626688,"data_compressed_original":2752512,"omap_allocated":19141,"internal_metadata":361674043},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":616,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":92,"num_read_kb":47,"num_write":6,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003151"},"pg_stats":[{"pgid":"3.3f","version":"92'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.217305+0000","last_change":"2024-04-24T23:27:19.978807+0000","last_active":"2024-04-24T23:29:08.217305+0000","last_peered":"2024-04-24T23:29:08.217305+0000","last_clean":"2024-04-24T23:29:08.217305+0000","last_became_active":"2024-04-24T23:27:19.978436+0000","last_became_peered":"2024-04-24T23:27:19.978436+0000","last_unstale":"2024-04-24T23:29:08.217305+0000","last_undegraded":"2024-04-24T23:29:08.217305+0000","last_fullsized":"2024-04-24T23:29:08.217305+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:00:13.126518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"92'4","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.387809+0000","last_change":"2024-04-24T23:27:19.980327+0000","last_active":"2024-04-24T23:29:08.387809+0000","last_peered":"2024-04-24T23:29:08.387809+0000","last_clean":"2024-04-24T23:29:08.387809+0000","last_became_active":"2024-04-24T23:27:19.979912+0000","last_became_peered":"2024-04-24T23:27:19.979912+0000","last_unstale":"2024-04-24T23:29:08.387809+0000","last_undegraded":"2024-04-24T23:29:08.387809+0000","last_fullsized":"2024-04-24T23:29:08.387809+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:25:41.657205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"92'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.387723+0000","last_change":"2024-04-24T23:27:19.976451+0000","last_active":"2024-04-24T23:29:08.387723+0000","last_peered":"2024-04-24T23:29:08.387723+0000","last_clean":"2024-04-24T23:29:08.387723+0000","last_became_active":"2024-04-24T23:27:19.975976+0000","last_became_peered":"2024-04-24T23:27:19.975976+0000","last_unstale":"2024-04-24T23:29:08.387723+0000","last_undegraded":"2024-04-24T23:29:08.387723+0000","last_fullsized":"2024-04-24T23:29:08.387723+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:14:51.605217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"92'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.218709+0000","last_change":"2024-04-24T23:27:19.991245+0000","last_active":"2024-04-24T23:29:08.218709+0000","last_peered":"2024-04-24T23:29:08.218709+0000","last_clean":"2024-04-24T23:29:08.218709+0000","last_became_active":"2024-04-24T23:27:19.990448+0000","last_became_peered":"2024-04-24T23:27:19.990448+0000","last_unstale":"2024-04-24T23:29:08.218709+0000","last_undegraded":"2024-04-24T23:29:08.218709+0000","last_fullsized":"2024-04-24T23:29:08.218709+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:03:21.013005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"92'4","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.221241+0000","last_change":"2024-04-24T23:27:19.977664+0000","last_active":"2024-04-24T23:29:08.221241+0000","last_peered":"2024-04-24T23:29:08.221241+0000","last_clean":"2024-04-24T23:29:08.221241+0000","last_became_active":"2024-04-24T23:27:19.977300+0000","last_became_peered":"2024-04-24T23:27:19.977300+0000","last_unstale":"2024-04-24T23:29:08.221241+0000","last_undegraded":"2024-04-24T23:29:08.221241+0000","last_fullsized":"2024-04-24T23:29:08.221241+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:19:27.229712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"92'6","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.249580+0000","last_change":"2024-04-24T23:27:19.989994+0000","last_active":"2024-04-24T23:29:08.249580+0000","last_peered":"2024-04-24T23:29:08.249580+0000","last_clean":"2024-04-24T23:29:08.249580+0000","last_became_active":"2024-04-24T23:27:19.989639+0000","last_became_peered":"2024-04-24T23:27:19.989639+0000","last_unstale":"2024-04-24T23:29:08.249580+0000","last_undegraded":"2024-04-24T23:29:08.249580+0000","last_fullsized":"2024-04-24T23:29:08.249580+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:20:43.931086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"92'5","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.221180+0000","last_change":"2024-04-24T23:27:19.992886+0000","last_active":"2024-04-24T23:29:08.221180+0000","last_peered":"2024-04-24T23:29:08.221180+0000","last_clean":"2024-04-24T23:29:08.221180+0000","last_became_active":"2024-04-24T23:27:19.992438+0000","last_became_peered":"2024-04-24T23:27:19.992438+0000","last_unstale":"2024-04-24T23:29:08.221180+0000","last_undegraded":"2024-04-24T23:29:08.221180+0000","last_fullsized":"2024-04-24T23:29:08.221180+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:02:34.110984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"92'9","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.212229+0000","last_change":"2024-04-24T23:27:19.981592+0000","last_active":"2024-04-24T23:29:08.212229+0000","last_peered":"2024-04-24T23:29:08.212229+0000","last_clean":"2024-04-24T23:29:08.212229+0000","last_became_active":"2024-04-24T23:27:19.981248+0000","last_became_peered":"2024-04-24T23:27:19.981248+0000","last_unstale":"2024-04-24T23:29:08.212229+0000","last_undegraded":"2024-04-24T23:29:08.212229+0000","last_fullsized":"2024-04-24T23:29:08.212229+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:53:31.191318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"92'11","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.221670+0000","last_change":"2024-04-24T23:27:19.975995+0000","last_active":"2024-04-24T23:29:08.221670+0000","last_peered":"2024-04-24T23:29:08.221670+0000","last_clean":"2024-04-24T23:29:08.221670+0000","last_became_active":"2024-04-24T23:27:19.975614+0000","last_became_peered":"2024-04-24T23:27:19.975614+0000","last_unstale":"2024-04-24T23:29:08.221670+0000","last_undegraded":"2024-04-24T23:29:08.221670+0000","last_fullsized":"2024-04-24T23:29:08.221670+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:33:22.439998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"92'16","reported_seq":47,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.250021+0000","last_change":"2024-04-24T23:27:19.991265+0000","last_active":"2024-04-24T23:29:08.250021+0000","last_peered":"2024-04-24T23:29:08.250021+0000","last_clean":"2024-04-24T23:29:08.250021+0000","last_became_active":"2024-04-24T23:27:19.990928+0000","last_became_peered":"2024-04-24T23:27:19.990928+0000","last_unstale":"2024-04-24T23:29:08.250021+0000","last_undegraded":"2024-04-24T23:29:08.250021+0000","last_fullsized":"2024-04-24T23:29:08.250021+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:20:41.244669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"92'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.222538+0000","last_change":"2024-04-24T23:27:19.985182+0000","last_active":"2024-04-24T23:29:08.222538+0000","last_peered":"2024-04-24T23:29:08.222538+0000","last_clean":"2024-04-24T23:29:08.222538+0000","last_became_active":"2024-04-24T23:27:19.984600+0000","last_became_peered":"2024-04-24T23:27:19.984600+0000","last_unstale":"2024-04-24T23:29:08.222538+0000","last_undegraded":"2024-04-24T23:29:08.222538+0000","last_fullsized":"2024-04-24T23:29:08.222538+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:16:09.919736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"92'7","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.218365+0000","last_change":"2024-04-24T23:27:19.994271+0000","last_active":"2024-04-24T23:29:08.218365+0000","last_peered":"2024-04-24T23:29:08.218365+0000","last_clean":"2024-04-24T23:29:08.218365+0000","last_became_active":"2024-04-24T23:27:19.993926+0000","last_became_peered":"2024-04-24T23:27:19.993926+0000","last_unstale":"2024-04-24T23:29:08.218365+0000","last_undegraded":"2024-04-24T23:29:08.218365+0000","last_fullsized":"2024-04-24T23:29:08.218365+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:49:07.500370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"92'10","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.249303+0000","last_change":"2024-04-24T23:27:19.984230+0000","last_active":"2024-04-24T23:29:08.249303+0000","last_peered":"2024-04-24T23:29:08.249303+0000","last_clean":"2024-04-24T23:29:08.249303+0000","last_became_active":"2024-04-24T23:27:19.983902+0000","last_became_peered":"2024-04-24T23:27:19.983902+0000","last_unstale":"2024-04-24T23:29:08.249303+0000","last_undegraded":"2024-04-24T23:29:08.249303+0000","last_fullsized":"2024-04-24T23:29:08.249303+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:12:40.985092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"92'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.387496+0000","last_change":"2024-04-24T23:27:19.979822+0000","last_active":"2024-04-24T23:29:08.387496+0000","last_peered":"2024-04-24T23:29:08.387496+0000","last_clean":"2024-04-24T23:29:08.387496+0000","last_became_active":"2024-04-24T23:27:19.979452+0000","last_became_peered":"2024-04-24T23:27:19.979452+0000","last_unstale":"2024-04-24T23:29:08.387496+0000","last_undegraded":"2024-04-24T23:29:08.387496+0000","last_fullsized":"2024-04-24T23:29:08.387496+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:08:53.945221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"92'7","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.387472+0000","last_change":"2024-04-24T23:28:08.053470+0000","last_active":"2024-04-24T23:29:08.387472+0000","last_peered":"2024-04-24T23:29:08.387472+0000","last_clean":"2024-04-24T23:29:08.387472+0000","last_became_active":"2024-04-24T23:28:08.053064+0000","last_became_peered":"2024-04-24T23:28:08.053064+0000","last_unstale":"2024-04-24T23:29:08.387472+0000","last_undegraded":"2024-04-24T23:29:08.387472+0000","last_fullsized":"2024-04-24T23:29:08.387472+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:13:34.489239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"92'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.220541+0000","last_change":"2024-04-24T23:27:19.984346+0000","last_active":"2024-04-24T23:29:08.220541+0000","last_peered":"2024-04-24T23:29:08.220541+0000","last_clean":"2024-04-24T23:29:08.220541+0000","last_became_active":"2024-04-24T23:27:19.983999+0000","last_became_peered":"2024-04-24T23:27:19.983999+0000","last_unstale":"2024-04-24T23:29:08.220541+0000","last_undegraded":"2024-04-24T23:29:08.220541+0000","last_fullsized":"2024-04-24T23:29:08.220541+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:15:35.329454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"92'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.217118+0000","last_change":"2024-04-24T23:27:19.979939+0000","last_active":"2024-04-24T23:29:08.217118+0000","last_peered":"2024-04-24T23:29:08.217118+0000","last_clean":"2024-04-24T23:29:08.217118+0000","last_became_active":"2024-04-24T23:27:19.979600+0000","last_became_peered":"2024-04-24T23:27:19.979600+0000","last_unstale":"2024-04-24T23:29:08.217118+0000","last_undegraded":"2024-04-24T23:29:08.217118+0000","last_fullsized":"2024-04-24T23:29:08.217118+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:37:52.250143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"92'10","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.221118+0000","last_change":"2024-04-24T23:27:19.984797+0000","last_active":"2024-04-24T23:29:08.221118+0000","last_peered":"2024-04-24T23:29:08.221118+0000","last_clean":"2024-04-24T23:29:08.221118+0000","last_became_active":"2024-04-24T23:27:19.983881+0000","last_became_peered":"2024-04-24T23:27:19.983881+0000","last_unstale":"2024-04-24T23:29:08.221118+0000","last_undegraded":"2024-04-24T23:29:08.221118+0000","last_fullsized":"2024-04-24T23:29:08.221118+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:17:24.630821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"92'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.214520+0000","last_change":"2024-04-24T23:27:19.977558+0000","last_active":"2024-04-24T23:29:08.214520+0000","last_peered":"2024-04-24T23:29:08.214520+0000","last_clean":"2024-04-24T23:29:08.214520+0000","last_became_active":"2024-04-24T23:27:19.977220+0000","last_became_peered":"2024-04-24T23:27:19.977220+0000","last_unstale":"2024-04-24T23:29:08.214520+0000","last_undegraded":"2024-04-24T23:29:08.214520+0000","last_fullsized":"2024-04-24T23:29:08.214520+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:38:49.615455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.214190+0000","last_change":"2024-04-24T23:27:24.234304+0000","last_active":"2024-04-24T23:29:08.214190+0000","last_peered":"2024-04-24T23:29:08.214190+0000","last_clean":"2024-04-24T23:29:08.214190+0000","last_became_active":"2024-04-24T23:27:24.233865+0000","last_became_peered":"2024-04-24T23:27:24.233865+0000","last_unstale":"2024-04-24T23:29:08.214190+0000","last_undegraded":"2024-04-24T23:29:08.214190+0000","last_fullsized":"2024-04-24T23:29:08.214190+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:46:20.853060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'9","reported_seq":60,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.221270+0000","last_change":"2024-04-24T23:28:08.061595+0000","last_active":"2024-04-24T23:29:08.221270+0000","last_peered":"2024-04-24T23:29:08.221270+0000","last_clean":"2024-04-24T23:29:08.221270+0000","last_became_active":"2024-04-24T23:28:08.053973+0000","last_became_peered":"2024-04-24T23:28:08.053973+0000","last_unstale":"2024-04-24T23:29:08.221270+0000","last_undegraded":"2024-04-24T23:29:08.221270+0000","last_fullsized":"2024-04-24T23:29:08.221270+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:57:55.667666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,8],"acting":[9,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"92'5","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.221020+0000","last_change":"2024-04-24T23:28:08.110963+0000","last_active":"2024-04-24T23:29:08.221020+0000","last_peered":"2024-04-24T23:29:08.221020+0000","last_clean":"2024-04-24T23:29:08.221020+0000","last_became_active":"2024-04-24T23:28:08.110633+0000","last_became_peered":"2024-04-24T23:28:08.110633+0000","last_unstale":"2024-04-24T23:29:08.221020+0000","last_undegraded":"2024-04-24T23:29:08.221020+0000","last_fullsized":"2024-04-24T23:29:08.221020+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:43:21.694477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.217862+0000","last_change":"2024-04-24T23:27:24.234134+0000","last_active":"2024-04-24T23:29:08.217862+0000","last_peered":"2024-04-24T23:29:08.217862+0000","last_clean":"2024-04-24T23:29:08.217862+0000","last_became_active":"2024-04-24T23:27:24.233650+0000","last_became_peered":"2024-04-24T23:27:24.233650+0000","last_unstale":"2024-04-24T23:29:08.217862+0000","last_undegraded":"2024-04-24T23:29:08.217862+0000","last_fullsized":"2024-04-24T23:29:08.217862+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:04:54.191186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.212945+0000","last_change":"2024-04-24T23:27:18.969572+0000","last_active":"2024-04-24T23:29:08.212945+0000","last_peered":"2024-04-24T23:29:08.212945+0000","last_clean":"2024-04-24T23:29:08.212945+0000","last_became_active":"2024-04-24T23:27:18.968897+0000","last_became_peered":"2024-04-24T23:27:18.968897+0000","last_unstale":"2024-04-24T23:29:08.212945+0000","last_undegraded":"2024-04-24T23:29:08.212945+0000","last_fullsized":"2024-04-24T23:29:08.212945+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:42:51.380386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"92'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.387791+0000","last_change":"2024-04-24T23:27:19.977290+0000","last_active":"2024-04-24T23:29:08.387791+0000","last_peered":"2024-04-24T23:29:08.387791+0000","last_clean":"2024-04-24T23:29:08.387791+0000","last_became_active":"2024-04-24T23:27:19.976802+0000","last_became_peered":"2024-04-24T23:27:19.976802+0000","last_unstale":"2024-04-24T23:29:08.387791+0000","last_undegraded":"2024-04-24T23:29:08.387791+0000","last_fullsized":"2024-04-24T23:29:08.387791+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:55:15.827279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.218034+0000","last_change":"2024-04-24T23:27:24.235706+0000","last_active":"2024-04-24T23:29:08.218034+0000","last_peered":"2024-04-24T23:29:08.218034+0000","last_clean":"2024-04-24T23:29:08.218034+0000","last_became_active":"2024-04-24T23:27:24.235367+0000","last_became_peered":"2024-04-24T23:27:24.235367+0000","last_unstale":"2024-04-24T23:29:08.218034+0000","last_undegraded":"2024-04-24T23:29:08.218034+0000","last_fullsized":"2024-04-24T23:29:08.218034+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:17.444845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.217153+0000","last_change":"2024-04-24T23:28:08.061398+0000","last_active":"2024-04-24T23:29:08.217153+0000","last_peered":"2024-04-24T23:29:08.217153+0000","last_clean":"2024-04-24T23:29:08.217153+0000","last_became_active":"2024-04-24T23:28:08.056810+0000","last_became_peered":"2024-04-24T23:28:08.056810+0000","last_unstale":"2024-04-24T23:29:08.217153+0000","last_undegraded":"2024-04-24T23:29:08.217153+0000","last_fullsized":"2024-04-24T23:29:08.217153+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:10:56.126730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"92'9","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.220961+0000","last_change":"2024-04-24T23:27:19.975222+0000","last_active":"2024-04-24T23:29:08.220961+0000","last_peered":"2024-04-24T23:29:08.220961+0000","last_clean":"2024-04-24T23:29:08.220961+0000","last_became_active":"2024-04-24T23:27:19.974881+0000","last_became_peered":"2024-04-24T23:27:19.974881+0000","last_unstale":"2024-04-24T23:29:08.220961+0000","last_undegraded":"2024-04-24T23:29:08.220961+0000","last_fullsized":"2024-04-24T23:29:08.220961+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:16:17.884748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.217945+0000","last_change":"2024-04-24T23:27:24.232445+0000","last_active":"2024-04-24T23:29:08.217945+0000","last_peered":"2024-04-24T23:29:08.217945+0000","last_clean":"2024-04-24T23:29:08.217945+0000","last_became_active":"2024-04-24T23:27:24.231754+0000","last_became_peered":"2024-04-24T23:27:24.231754+0000","last_unstale":"2024-04-24T23:29:08.217945+0000","last_undegraded":"2024-04-24T23:29:08.217945+0000","last_fullsized":"2024-04-24T23:29:08.217945+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:45:43.630508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.221798+0000","last_change":"2024-04-24T23:27:18.966558+0000","last_active":"2024-04-24T23:29:08.221798+0000","last_peered":"2024-04-24T23:29:08.221798+0000","last_clean":"2024-04-24T23:29:08.221798+0000","last_became_active":"2024-04-24T23:27:18.966212+0000","last_became_peered":"2024-04-24T23:27:18.966212+0000","last_unstale":"2024-04-24T23:29:08.221798+0000","last_undegraded":"2024-04-24T23:29:08.221798+0000","last_fullsized":"2024-04-24T23:29:08.221798+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:24:17.262587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"92'15","reported_seq":46,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.249544+0000","last_change":"2024-04-24T23:27:19.976811+0000","last_active":"2024-04-24T23:29:08.249544+0000","last_peered":"2024-04-24T23:29:08.249544+0000","last_clean":"2024-04-24T23:29:08.249544+0000","last_became_active":"2024-04-24T23:27:19.976456+0000","last_became_peered":"2024-04-24T23:27:19.976456+0000","last_unstale":"2024-04-24T23:29:08.249544+0000","last_undegraded":"2024-04-24T23:29:08.249544+0000","last_fullsized":"2024-04-24T23:29:08.249544+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:43:13.755833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.214292+0000","last_change":"2024-04-24T23:27:24.232136+0000","last_active":"2024-04-24T23:29:08.214292+0000","last_peered":"2024-04-24T23:29:08.214292+0000","last_clean":"2024-04-24T23:29:08.214292+0000","last_became_active":"2024-04-24T23:27:24.231459+0000","last_became_peered":"2024-04-24T23:27:24.231459+0000","last_unstale":"2024-04-24T23:29:08.214292+0000","last_undegraded":"2024-04-24T23:29:08.214292+0000","last_fullsized":"2024-04-24T23:29:08.214292+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:51:51.114787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"92'7","reported_seq":162,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:42.159846+0000","last_change":"2024-04-24T23:27:18.975333+0000","last_active":"2024-04-24T23:29:42.159846+0000","last_peered":"2024-04-24T23:29:42.159846+0000","last_clean":"2024-04-24T23:29:42.159846+0000","last_became_active":"2024-04-24T23:27:18.974869+0000","last_became_peered":"2024-04-24T23:27:18.974869+0000","last_unstale":"2024-04-24T23:29:42.159846+0000","last_undegraded":"2024-04-24T23:29:42.159846+0000","last_fullsized":"2024-04-24T23:29:42.159846+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:30:26.758596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":122,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"92'7","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.221237+0000","last_change":"2024-04-24T23:27:19.974989+0000","last_active":"2024-04-24T23:29:08.221237+0000","last_peered":"2024-04-24T23:29:08.221237+0000","last_clean":"2024-04-24T23:29:08.221237+0000","last_became_active":"2024-04-24T23:27:19.974312+0000","last_became_peered":"2024-04-24T23:27:19.974312+0000","last_unstale":"2024-04-24T23:29:08.221237+0000","last_undegraded":"2024-04-24T23:29:08.221237+0000","last_fullsized":"2024-04-24T23:29:08.221237+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:16:39.685430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.250906+0000","last_change":"2024-04-24T23:29:08.250906+0000","last_active":"2024-04-24T23:29:08.250906+0000","last_peered":"2024-04-24T23:29:08.250906+0000","last_clean":"2024-04-24T23:29:08.250906+0000","last_became_active":"2024-04-24T23:29:08.250264+0000","last_became_peered":"2024-04-24T23:29:08.250264+0000","last_unstale":"2024-04-24T23:29:08.250906+0000","last_undegraded":"2024-04-24T23:29:08.250906+0000","last_fullsized":"2024-04-24T23:29:08.250906+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:41:54.427428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.249568+0000","last_change":"2024-04-24T23:27:18.971313+0000","last_active":"2024-04-24T23:29:08.249568+0000","last_peered":"2024-04-24T23:29:08.249568+0000","last_clean":"2024-04-24T23:29:08.249568+0000","last_became_active":"2024-04-24T23:27:18.970888+0000","last_became_peered":"2024-04-24T23:27:18.970888+0000","last_unstale":"2024-04-24T23:29:08.249568+0000","last_undegraded":"2024-04-24T23:29:08.249568+0000","last_fullsized":"2024-04-24T23:29:08.249568+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:34:20.118693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"92'8","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.249768+0000","last_change":"2024-04-24T23:27:19.978939+0000","last_active":"2024-04-24T23:29:08.249768+0000","last_peered":"2024-04-24T23:29:08.249768+0000","last_clean":"2024-04-24T23:29:08.249768+0000","last_became_active":"2024-04-24T23:27:19.978556+0000","last_became_peered":"2024-04-24T23:27:19.978556+0000","last_unstale":"2024-04-24T23:29:08.249768+0000","last_undegraded":"2024-04-24T23:29:08.249768+0000","last_fullsized":"2024-04-24T23:29:08.249768+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:28:43.118561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.217032+0000","last_change":"2024-04-24T23:27:24.232384+0000","last_active":"2024-04-24T23:29:08.217032+0000","last_peered":"2024-04-24T23:29:08.217032+0000","last_clean":"2024-04-24T23:29:08.217032+0000","last_became_active":"2024-04-24T23:27:24.231497+0000","last_became_peered":"2024-04-24T23:27:24.231497+0000","last_unstale":"2024-04-24T23:29:08.217032+0000","last_undegraded":"2024-04-24T23:29:08.217032+0000","last_fullsized":"2024-04-24T23:29:08.217032+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:26:54.716821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.248992+0000","last_change":"2024-04-24T23:27:18.968949+0000","last_active":"2024-04-24T23:29:08.248992+0000","last_peered":"2024-04-24T23:29:08.248992+0000","last_clean":"2024-04-24T23:29:08.248992+0000","last_became_active":"2024-04-24T23:27:18.968612+0000","last_became_peered":"2024-04-24T23:27:18.968612+0000","last_unstale":"2024-04-24T23:29:08.248992+0000","last_undegraded":"2024-04-24T23:29:08.248992+0000","last_fullsized":"2024-04-24T23:29:08.248992+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:03:45.908080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"92'8","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.212368+0000","last_change":"2024-04-24T23:27:19.979670+0000","last_active":"2024-04-24T23:29:08.212368+0000","last_peered":"2024-04-24T23:29:08.212368+0000","last_clean":"2024-04-24T23:29:08.212368+0000","last_became_active":"2024-04-24T23:27:19.979277+0000","last_became_peered":"2024-04-24T23:27:19.979277+0000","last_unstale":"2024-04-24T23:29:08.212368+0000","last_undegraded":"2024-04-24T23:29:08.212368+0000","last_fullsized":"2024-04-24T23:29:08.212368+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:05:54.958747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.387084+0000","last_change":"2024-04-24T23:27:24.237085+0000","last_active":"2024-04-24T23:29:08.387084+0000","last_peered":"2024-04-24T23:29:08.387084+0000","last_clean":"2024-04-24T23:29:08.387084+0000","last_became_active":"2024-04-24T23:27:24.236671+0000","last_became_peered":"2024-04-24T23:27:24.236671+0000","last_unstale":"2024-04-24T23:29:08.387084+0000","last_undegraded":"2024-04-24T23:29:08.387084+0000","last_fullsized":"2024-04-24T23:29:08.387084+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:48:38.482374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.250147+0000","last_change":"2024-04-24T23:27:18.974146+0000","last_active":"2024-04-24T23:29:08.250147+0000","last_peered":"2024-04-24T23:29:08.250147+0000","last_clean":"2024-04-24T23:29:08.250147+0000","last_became_active":"2024-04-24T23:27:18.973814+0000","last_became_peered":"2024-04-24T23:27:18.973814+0000","last_unstale":"2024-04-24T23:29:08.250147+0000","last_undegraded":"2024-04-24T23:29:08.250147+0000","last_fullsized":"2024-04-24T23:29:08.250147+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:22:52.409448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"92'11","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.220653+0000","last_change":"2024-04-24T23:27:19.984325+0000","last_active":"2024-04-24T23:29:08.220653+0000","last_peered":"2024-04-24T23:29:08.220653+0000","last_clean":"2024-04-24T23:29:08.220653+0000","last_became_active":"2024-04-24T23:27:19.983918+0000","last_became_peered":"2024-04-24T23:27:19.983918+0000","last_unstale":"2024-04-24T23:29:08.220653+0000","last_undegraded":"2024-04-24T23:29:08.220653+0000","last_fullsized":"2024-04-24T23:29:08.220653+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:24:07.640983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.212601+0000","last_change":"2024-04-24T23:27:24.223561+0000","last_active":"2024-04-24T23:29:08.212601+0000","last_peered":"2024-04-24T23:29:08.212601+0000","last_clean":"2024-04-24T23:29:08.212601+0000","last_became_active":"2024-04-24T23:27:24.223127+0000","last_became_peered":"2024-04-24T23:27:24.223127+0000","last_unstale":"2024-04-24T23:29:08.212601+0000","last_undegraded":"2024-04-24T23:29:08.212601+0000","last_fullsized":"2024-04-24T23:29:08.212601+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:31:28.079633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"92'4","reported_seq":156,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:43.160939+0000","last_change":"2024-04-24T23:27:18.971374+0000","last_active":"2024-04-24T23:29:43.160939+0000","last_peered":"2024-04-24T23:29:43.160939+0000","last_clean":"2024-04-24T23:29:43.160939+0000","last_became_active":"2024-04-24T23:27:18.970818+0000","last_became_peered":"2024-04-24T23:27:18.970818+0000","last_unstale":"2024-04-24T23:29:43.160939+0000","last_undegraded":"2024-04-24T23:29:43.160939+0000","last_fullsized":"2024-04-24T23:29:43.160939+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:12:17.547968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":118,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"92'10","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.222062+0000","last_change":"2024-04-24T23:27:19.980275+0000","last_active":"2024-04-24T23:29:08.222062+0000","last_peered":"2024-04-24T23:29:08.222062+0000","last_clean":"2024-04-24T23:29:08.222062+0000","last_became_active":"2024-04-24T23:27:19.979925+0000","last_became_peered":"2024-04-24T23:27:19.979925+0000","last_unstale":"2024-04-24T23:29:08.222062+0000","last_undegraded":"2024-04-24T23:29:08.222062+0000","last_fullsized":"2024-04-24T23:29:08.222062+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:21:10.681169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.220570+0000","last_change":"2024-04-24T23:27:24.231389+0000","last_active":"2024-04-24T23:29:08.220570+0000","last_peered":"2024-04-24T23:29:08.220570+0000","last_clean":"2024-04-24T23:29:08.220570+0000","last_became_active":"2024-04-24T23:27:24.230973+0000","last_became_peered":"2024-04-24T23:27:24.230973+0000","last_unstale":"2024-04-24T23:29:08.220570+0000","last_undegraded":"2024-04-24T23:29:08.220570+0000","last_fullsized":"2024-04-24T23:29:08.220570+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:35.943122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.222447+0000","last_change":"2024-04-24T23:27:18.977009+0000","last_active":"2024-04-24T23:29:08.222447+0000","last_peered":"2024-04-24T23:29:08.222447+0000","last_clean":"2024-04-24T23:29:08.222447+0000","last_became_active":"2024-04-24T23:27:18.976667+0000","last_became_peered":"2024-04-24T23:27:18.976667+0000","last_unstale":"2024-04-24T23:29:08.222447+0000","last_undegraded":"2024-04-24T23:29:08.222447+0000","last_fullsized":"2024-04-24T23:29:08.222447+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:19:47.817961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"92'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.387836+0000","last_change":"2024-04-24T23:27:19.988589+0000","last_active":"2024-04-24T23:29:08.387836+0000","last_peered":"2024-04-24T23:29:08.387836+0000","last_clean":"2024-04-24T23:29:08.387836+0000","last_became_active":"2024-04-24T23:27:19.988261+0000","last_became_peered":"2024-04-24T23:27:19.988261+0000","last_unstale":"2024-04-24T23:29:08.387836+0000","last_undegraded":"2024-04-24T23:29:08.387836+0000","last_fullsized":"2024-04-24T23:29:08.387836+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:44:15.809776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.218303+0000","last_change":"2024-04-24T23:27:24.231256+0000","last_active":"2024-04-24T23:29:08.218303+0000","last_peered":"2024-04-24T23:29:08.218303+0000","last_clean":"2024-04-24T23:29:08.218303+0000","last_became_active":"2024-04-24T23:27:24.230545+0000","last_became_peered":"2024-04-24T23:27:24.230545+0000","last_unstale":"2024-04-24T23:29:08.218303+0000","last_undegraded":"2024-04-24T23:29:08.218303+0000","last_fullsized":"2024-04-24T23:29:08.218303+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:43:40.460473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.249368+0000","last_change":"2024-04-24T23:27:18.968805+0000","last_active":"2024-04-24T23:29:08.249368+0000","last_peered":"2024-04-24T23:29:08.249368+0000","last_clean":"2024-04-24T23:29:08.249368+0000","last_became_active":"2024-04-24T23:27:18.968423+0000","last_became_peered":"2024-04-24T23:27:18.968423+0000","last_unstale":"2024-04-24T23:29:08.249368+0000","last_undegraded":"2024-04-24T23:29:08.249368+0000","last_fullsized":"2024-04-24T23:29:08.249368+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:14:04.077773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"92'11","reported_seq":49,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.221213+0000","last_change":"2024-04-24T23:28:08.056901+0000","last_active":"2024-04-24T23:29:08.221213+0000","last_peered":"2024-04-24T23:29:08.221213+0000","last_clean":"2024-04-24T23:29:08.221213+0000","last_became_active":"2024-04-24T23:28:08.056422+0000","last_became_peered":"2024-04-24T23:28:08.056422+0000","last_unstale":"2024-04-24T23:29:08.221213+0000","last_undegraded":"2024-04-24T23:29:08.221213+0000","last_fullsized":"2024-04-24T23:29:08.221213+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:10:01.716479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.220583+0000","last_change":"2024-04-24T23:27:24.225481+0000","last_active":"2024-04-24T23:29:08.220583+0000","last_peered":"2024-04-24T23:29:08.220583+0000","last_clean":"2024-04-24T23:29:08.220583+0000","last_became_active":"2024-04-24T23:27:24.224363+0000","last_became_peered":"2024-04-24T23:27:24.224363+0000","last_unstale":"2024-04-24T23:29:08.220583+0000","last_undegraded":"2024-04-24T23:29:08.220583+0000","last_fullsized":"2024-04-24T23:29:08.220583+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:36:17.715596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"94'35","reported_seq":314,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:43.173813+0000","last_change":"2024-04-24T23:27:18.968264+0000","last_active":"2024-04-24T23:29:43.173813+0000","last_peered":"2024-04-24T23:29:43.173813+0000","last_clean":"2024-04-24T23:29:43.173813+0000","last_became_active":"2024-04-24T23:27:18.967856+0000","last_became_peered":"2024-04-24T23:27:18.967856+0000","last_unstale":"2024-04-24T23:29:43.173813+0000","last_undegraded":"2024-04-24T23:29:43.173813+0000","last_fullsized":"2024-04-24T23:29:43.173813+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:48:44.347254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59428,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":245,"num_read_kb":181,"num_write":36,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"92'6","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.222016+0000","last_change":"2024-04-24T23:27:19.984719+0000","last_active":"2024-04-24T23:29:08.222016+0000","last_peered":"2024-04-24T23:29:08.222016+0000","last_clean":"2024-04-24T23:29:08.222016+0000","last_became_active":"2024-04-24T23:27:19.984386+0000","last_became_peered":"2024-04-24T23:27:19.984386+0000","last_unstale":"2024-04-24T23:29:08.222016+0000","last_undegraded":"2024-04-24T23:29:08.222016+0000","last_fullsized":"2024-04-24T23:29:08.222016+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:02:13.739759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.248640+0000","last_change":"2024-04-24T23:27:24.230795+0000","last_active":"2024-04-24T23:29:08.248640+0000","last_peered":"2024-04-24T23:29:08.248640+0000","last_clean":"2024-04-24T23:29:08.248640+0000","last_became_active":"2024-04-24T23:27:24.230360+0000","last_became_peered":"2024-04-24T23:27:24.230360+0000","last_unstale":"2024-04-24T23:29:08.248640+0000","last_undegraded":"2024-04-24T23:29:08.248640+0000","last_fullsized":"2024-04-24T23:29:08.248640+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:51:24.875668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.214478+0000","last_change":"2024-04-24T23:27:18.964964+0000","last_active":"2024-04-24T23:29:08.214478+0000","last_peered":"2024-04-24T23:29:08.214478+0000","last_clean":"2024-04-24T23:29:08.214478+0000","last_became_active":"2024-04-24T23:27:18.964632+0000","last_became_peered":"2024-04-24T23:27:18.964632+0000","last_unstale":"2024-04-24T23:29:08.214478+0000","last_undegraded":"2024-04-24T23:29:08.214478+0000","last_fullsized":"2024-04-24T23:29:08.214478+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:52:23.144629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"92'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.249122+0000","last_change":"2024-04-24T23:27:19.987258+0000","last_active":"2024-04-24T23:29:08.249122+0000","last_peered":"2024-04-24T23:29:08.249122+0000","last_clean":"2024-04-24T23:29:08.249122+0000","last_became_active":"2024-04-24T23:27:19.986864+0000","last_became_peered":"2024-04-24T23:27:19.986864+0000","last_unstale":"2024-04-24T23:29:08.249122+0000","last_undegraded":"2024-04-24T23:29:08.249122+0000","last_fullsized":"2024-04-24T23:29:08.249122+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:57:40.503506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.251155+0000","last_change":"2024-04-24T23:29:08.251155+0000","last_active":"2024-04-24T23:29:08.251155+0000","last_peered":"2024-04-24T23:29:08.251155+0000","last_clean":"2024-04-24T23:29:08.251155+0000","last_became_active":"2024-04-24T23:29:08.250662+0000","last_became_peered":"2024-04-24T23:29:08.250662+0000","last_unstale":"2024-04-24T23:29:08.251155+0000","last_undegraded":"2024-04-24T23:29:08.251155+0000","last_fullsized":"2024-04-24T23:29:08.251155+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:32:01.403466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.249469+0000","last_change":"2024-04-24T23:27:18.974229+0000","last_active":"2024-04-24T23:29:08.249469+0000","last_peered":"2024-04-24T23:29:08.249469+0000","last_clean":"2024-04-24T23:29:08.249469+0000","last_became_active":"2024-04-24T23:27:18.973893+0000","last_became_peered":"2024-04-24T23:27:18.973893+0000","last_unstale":"2024-04-24T23:29:08.249469+0000","last_undegraded":"2024-04-24T23:29:08.249469+0000","last_fullsized":"2024-04-24T23:29:08.249469+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:41:45.699709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"92'15","reported_seq":47,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.221880+0000","last_change":"2024-04-24T23:27:19.986910+0000","last_active":"2024-04-24T23:29:08.221880+0000","last_peered":"2024-04-24T23:29:08.221880+0000","last_clean":"2024-04-24T23:29:08.221880+0000","last_became_active":"2024-04-24T23:27:19.986556+0000","last_became_peered":"2024-04-24T23:27:19.986556+0000","last_unstale":"2024-04-24T23:29:08.221880+0000","last_undegraded":"2024-04-24T23:29:08.221880+0000","last_fullsized":"2024-04-24T23:29:08.221880+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:06:28.506556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.221054+0000","last_change":"2024-04-24T23:27:24.233115+0000","last_active":"2024-04-24T23:29:08.221054+0000","last_peered":"2024-04-24T23:29:08.221054+0000","last_clean":"2024-04-24T23:29:08.221054+0000","last_became_active":"2024-04-24T23:27:24.232714+0000","last_became_peered":"2024-04-24T23:27:24.232714+0000","last_unstale":"2024-04-24T23:29:08.221054+0000","last_undegraded":"2024-04-24T23:29:08.221054+0000","last_fullsized":"2024-04-24T23:29:08.221054+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:04.365515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"94'6","reported_seq":163,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:42.159532+0000","last_change":"2024-04-24T23:27:18.966426+0000","last_active":"2024-04-24T23:29:42.159532+0000","last_peered":"2024-04-24T23:29:42.159532+0000","last_clean":"2024-04-24T23:29:42.159532+0000","last_became_active":"2024-04-24T23:27:18.966070+0000","last_became_peered":"2024-04-24T23:27:18.966070+0000","last_unstale":"2024-04-24T23:29:42.159532+0000","last_undegraded":"2024-04-24T23:29:42.159532+0000","last_fullsized":"2024-04-24T23:29:42.159532+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:56:21.679639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1428,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":124,"num_read_kb":6,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"92'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.221340+0000","last_change":"2024-04-24T23:27:19.991829+0000","last_active":"2024-04-24T23:29:08.221340+0000","last_peered":"2024-04-24T23:29:08.221340+0000","last_clean":"2024-04-24T23:29:08.221340+0000","last_became_active":"2024-04-24T23:27:19.991492+0000","last_became_peered":"2024-04-24T23:27:19.991492+0000","last_unstale":"2024-04-24T23:29:08.221340+0000","last_undegraded":"2024-04-24T23:29:08.221340+0000","last_fullsized":"2024-04-24T23:29:08.221340+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:23:45.195217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.248500+0000","last_change":"2024-04-24T23:27:24.229794+0000","last_active":"2024-04-24T23:29:08.248500+0000","last_peered":"2024-04-24T23:29:08.248500+0000","last_clean":"2024-04-24T23:29:08.248500+0000","last_became_active":"2024-04-24T23:27:24.229410+0000","last_became_peered":"2024-04-24T23:27:24.229410+0000","last_unstale":"2024-04-24T23:29:08.248500+0000","last_undegraded":"2024-04-24T23:29:08.248500+0000","last_fullsized":"2024-04-24T23:29:08.248500+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:00:53.188307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":74,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.221241+0000","last_change":"2024-04-24T23:28:08.068516+0000","last_active":"2024-04-24T23:29:08.221241+0000","last_peered":"2024-04-24T23:29:08.221241+0000","last_clean":"2024-04-24T23:29:08.221241+0000","last_became_active":"2024-04-24T23:28:08.056740+0000","last_became_peered":"2024-04-24T23:28:08.056740+0000","last_unstale":"2024-04-24T23:29:08.221241+0000","last_undegraded":"2024-04-24T23:29:08.221241+0000","last_fullsized":"2024-04-24T23:29:08.221241+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:47:35.568377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,1],"acting":[8,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"92'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.249623+0000","last_change":"2024-04-24T23:27:19.982608+0000","last_active":"2024-04-24T23:29:08.249623+0000","last_peered":"2024-04-24T23:29:08.249623+0000","last_clean":"2024-04-24T23:29:08.249623+0000","last_became_active":"2024-04-24T23:27:19.982270+0000","last_became_peered":"2024-04-24T23:27:19.982270+0000","last_unstale":"2024-04-24T23:29:08.249623+0000","last_undegraded":"2024-04-24T23:29:08.249623+0000","last_fullsized":"2024-04-24T23:29:08.249623+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:01:10.525164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.248615+0000","last_change":"2024-04-24T23:27:24.236143+0000","last_active":"2024-04-24T23:29:08.248615+0000","last_peered":"2024-04-24T23:29:08.248615+0000","last_clean":"2024-04-24T23:29:08.248615+0000","last_became_active":"2024-04-24T23:27:24.235724+0000","last_became_peered":"2024-04-24T23:27:24.235724+0000","last_unstale":"2024-04-24T23:29:08.248615+0000","last_undegraded":"2024-04-24T23:29:08.248615+0000","last_fullsized":"2024-04-24T23:29:08.248615+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:55:41.317044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.222036+0000","last_change":"2024-04-24T23:27:18.971312+0000","last_active":"2024-04-24T23:29:08.222036+0000","last_peered":"2024-04-24T23:29:08.222036+0000","last_clean":"2024-04-24T23:29:08.222036+0000","last_became_active":"2024-04-24T23:27:18.970929+0000","last_became_peered":"2024-04-24T23:27:18.970929+0000","last_unstale":"2024-04-24T23:29:08.222036+0000","last_undegraded":"2024-04-24T23:29:08.222036+0000","last_fullsized":"2024-04-24T23:29:08.222036+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:54:20.318916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"92'5","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.217096+0000","last_change":"2024-04-24T23:28:08.056700+0000","last_active":"2024-04-24T23:29:08.217096+0000","last_peered":"2024-04-24T23:29:08.217096+0000","last_clean":"2024-04-24T23:29:08.217096+0000","last_became_active":"2024-04-24T23:28:08.056271+0000","last_became_peered":"2024-04-24T23:28:08.056271+0000","last_unstale":"2024-04-24T23:29:08.217096+0000","last_undegraded":"2024-04-24T23:29:08.217096+0000","last_fullsized":"2024-04-24T23:29:08.217096+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:03:34.808028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.221630+0000","last_change":"2024-04-24T23:27:24.718948+0000","last_active":"2024-04-24T23:29:08.221630+0000","last_peered":"2024-04-24T23:29:08.221630+0000","last_clean":"2024-04-24T23:29:08.221630+0000","last_became_active":"2024-04-24T23:27:24.718424+0000","last_became_peered":"2024-04-24T23:27:24.718424+0000","last_unstale":"2024-04-24T23:29:08.221630+0000","last_undegraded":"2024-04-24T23:29:08.221630+0000","last_fullsized":"2024-04-24T23:29:08.221630+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:22:01.984976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.220935+0000","last_change":"2024-04-24T23:27:18.957073+0000","last_active":"2024-04-24T23:29:08.220935+0000","last_peered":"2024-04-24T23:29:08.220935+0000","last_clean":"2024-04-24T23:29:08.220935+0000","last_became_active":"2024-04-24T23:27:18.956713+0000","last_became_peered":"2024-04-24T23:27:18.956713+0000","last_unstale":"2024-04-24T23:29:08.220935+0000","last_undegraded":"2024-04-24T23:29:08.220935+0000","last_fullsized":"2024-04-24T23:29:08.220935+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:16:21.015902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"92'4","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.249932+0000","last_change":"2024-04-24T23:27:19.985058+0000","last_active":"2024-04-24T23:29:08.249932+0000","last_peered":"2024-04-24T23:29:08.249932+0000","last_clean":"2024-04-24T23:29:08.249932+0000","last_became_active":"2024-04-24T23:27:19.984675+0000","last_became_peered":"2024-04-24T23:27:19.984675+0000","last_unstale":"2024-04-24T23:29:08.249932+0000","last_undegraded":"2024-04-24T23:29:08.249932+0000","last_fullsized":"2024-04-24T23:29:08.249932+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:22:35.252839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.248925+0000","last_change":"2024-04-24T23:27:24.236117+0000","last_active":"2024-04-24T23:29:08.248925+0000","last_peered":"2024-04-24T23:29:08.248925+0000","last_clean":"2024-04-24T23:29:08.248925+0000","last_became_active":"2024-04-24T23:27:24.235720+0000","last_became_peered":"2024-04-24T23:27:24.235720+0000","last_unstale":"2024-04-24T23:29:08.248925+0000","last_undegraded":"2024-04-24T23:29:08.248925+0000","last_fullsized":"2024-04-24T23:29:08.248925+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:46:40.604988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.221627+0000","last_change":"2024-04-24T23:27:18.973690+0000","last_active":"2024-04-24T23:29:08.221627+0000","last_peered":"2024-04-24T23:29:08.221627+0000","last_clean":"2024-04-24T23:29:08.221627+0000","last_became_active":"2024-04-24T23:27:18.973182+0000","last_became_peered":"2024-04-24T23:27:18.973182+0000","last_unstale":"2024-04-24T23:29:08.221627+0000","last_undegraded":"2024-04-24T23:29:08.221627+0000","last_fullsized":"2024-04-24T23:29:08.221627+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:00:44.809880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"92'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.249631+0000","last_change":"2024-04-24T23:27:19.991252+0000","last_active":"2024-04-24T23:29:08.249631+0000","last_peered":"2024-04-24T23:29:08.249631+0000","last_clean":"2024-04-24T23:29:08.249631+0000","last_became_active":"2024-04-24T23:27:19.990892+0000","last_became_peered":"2024-04-24T23:27:19.990892+0000","last_unstale":"2024-04-24T23:29:08.249631+0000","last_undegraded":"2024-04-24T23:29:08.249631+0000","last_fullsized":"2024-04-24T23:29:08.249631+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:29:37.835521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.249473+0000","last_change":"2024-04-24T23:27:24.718929+0000","last_active":"2024-04-24T23:29:08.249473+0000","last_peered":"2024-04-24T23:29:08.249473+0000","last_clean":"2024-04-24T23:29:08.249473+0000","last_became_active":"2024-04-24T23:27:24.718486+0000","last_became_peered":"2024-04-24T23:27:24.718486+0000","last_unstale":"2024-04-24T23:29:08.249473+0000","last_undegraded":"2024-04-24T23:29:08.249473+0000","last_fullsized":"2024-04-24T23:29:08.249473+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:31:13.359829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"92'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.249896+0000","last_change":"2024-04-24T23:27:19.983181+0000","last_active":"2024-04-24T23:29:08.249896+0000","last_peered":"2024-04-24T23:29:08.249896+0000","last_clean":"2024-04-24T23:29:08.249896+0000","last_became_active":"2024-04-24T23:27:19.981764+0000","last_became_peered":"2024-04-24T23:27:19.981764+0000","last_unstale":"2024-04-24T23:29:08.249896+0000","last_undegraded":"2024-04-24T23:29:08.249896+0000","last_fullsized":"2024-04-24T23:29:08.249896+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:01:20.653572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"94'6","reported_seq":159,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:43.160323+0000","last_change":"2024-04-24T23:27:18.976351+0000","last_active":"2024-04-24T23:29:43.160323+0000","last_peered":"2024-04-24T23:29:43.160323+0000","last_clean":"2024-04-24T23:29:43.160323+0000","last_became_active":"2024-04-24T23:27:18.975552+0000","last_became_peered":"2024-04-24T23:27:18.975552+0000","last_unstale":"2024-04-24T23:29:43.160323+0000","last_undegraded":"2024-04-24T23:29:43.160323+0000","last_fullsized":"2024-04-24T23:29:43.160323+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:26:29.551439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":119,"num_read_kb":119,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":27,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.222548+0000","last_change":"2024-04-24T23:27:24.229528+0000","last_active":"2024-04-24T23:29:08.222548+0000","last_peered":"2024-04-24T23:29:08.222548+0000","last_clean":"2024-04-24T23:29:08.222548+0000","last_became_active":"2024-04-24T23:27:24.229099+0000","last_became_peered":"2024-04-24T23:27:24.229099+0000","last_unstale":"2024-04-24T23:29:08.222548+0000","last_undegraded":"2024-04-24T23:29:08.222548+0000","last_fullsized":"2024-04-24T23:29:08.222548+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:51:51.421912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"92'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.212843+0000","last_change":"2024-04-24T23:27:19.980094+0000","last_active":"2024-04-24T23:29:08.212843+0000","last_peered":"2024-04-24T23:29:08.212843+0000","last_clean":"2024-04-24T23:29:08.212843+0000","last_became_active":"2024-04-24T23:27:19.979710+0000","last_became_peered":"2024-04-24T23:27:19.979710+0000","last_unstale":"2024-04-24T23:29:08.212843+0000","last_undegraded":"2024-04-24T23:29:08.212843+0000","last_fullsized":"2024-04-24T23:29:08.212843+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:50:37.920235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.218213+0000","last_change":"2024-04-24T23:27:18.963224+0000","last_active":"2024-04-24T23:29:08.218213+0000","last_peered":"2024-04-24T23:29:08.218213+0000","last_clean":"2024-04-24T23:29:08.218213+0000","last_became_active":"2024-04-24T23:27:18.962794+0000","last_became_peered":"2024-04-24T23:27:18.962794+0000","last_unstale":"2024-04-24T23:29:08.218213+0000","last_undegraded":"2024-04-24T23:29:08.218213+0000","last_fullsized":"2024-04-24T23:29:08.218213+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:05:36.869062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":358,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.220962+0000","last_change":"2024-04-24T23:26:02.307599+0000","last_active":"2024-04-24T23:29:08.220962+0000","last_peered":"2024-04-24T23:29:08.220962+0000","last_clean":"2024-04-24T23:29:08.220962+0000","last_became_active":"2024-04-24T23:26:02.307186+0000","last_became_peered":"2024-04-24T23:26:02.307186+0000","last_unstale":"2024-04-24T23:29:08.220962+0000","last_undegraded":"2024-04-24T23:29:08.220962+0000","last_fullsized":"2024-04-24T23:29:08.220962+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:22:33.824274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:22:33.824274+0000","last_clean_scrub_stamp":"2024-04-24T23:22:33.824274+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:19:26.789955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.249319+0000","last_change":"2024-04-24T23:27:24.719315+0000","last_active":"2024-04-24T23:29:08.249319+0000","last_peered":"2024-04-24T23:29:08.249319+0000","last_clean":"2024-04-24T23:29:08.249319+0000","last_became_active":"2024-04-24T23:27:24.718868+0000","last_became_peered":"2024-04-24T23:27:24.718868+0000","last_unstale":"2024-04-24T23:29:08.249319+0000","last_undegraded":"2024-04-24T23:29:08.249319+0000","last_fullsized":"2024-04-24T23:29:08.249319+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:43:32.122873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"92'5","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.248863+0000","last_change":"2024-04-24T23:27:19.983711+0000","last_active":"2024-04-24T23:29:08.248863+0000","last_peered":"2024-04-24T23:29:08.248863+0000","last_clean":"2024-04-24T23:29:08.248863+0000","last_became_active":"2024-04-24T23:27:19.983302+0000","last_became_peered":"2024-04-24T23:27:19.983302+0000","last_unstale":"2024-04-24T23:29:08.248863+0000","last_undegraded":"2024-04-24T23:29:08.248863+0000","last_fullsized":"2024-04-24T23:29:08.248863+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:44:49.251679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.221222+0000","last_change":"2024-04-24T23:27:18.976766+0000","last_active":"2024-04-24T23:29:08.221222+0000","last_peered":"2024-04-24T23:29:08.221222+0000","last_clean":"2024-04-24T23:29:08.221222+0000","last_became_active":"2024-04-24T23:27:18.976420+0000","last_became_peered":"2024-04-24T23:27:18.976420+0000","last_unstale":"2024-04-24T23:29:08.221222+0000","last_undegraded":"2024-04-24T23:29:08.221222+0000","last_fullsized":"2024-04-24T23:29:08.221222+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:59:34.262774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.216819+0000","last_change":"2024-04-24T23:27:24.226979+0000","last_active":"2024-04-24T23:29:08.216819+0000","last_peered":"2024-04-24T23:29:08.216819+0000","last_clean":"2024-04-24T23:29:08.216819+0000","last_became_active":"2024-04-24T23:27:24.226596+0000","last_became_peered":"2024-04-24T23:27:24.226596+0000","last_unstale":"2024-04-24T23:29:08.216819+0000","last_undegraded":"2024-04-24T23:29:08.216819+0000","last_fullsized":"2024-04-24T23:29:08.216819+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:50:48.585067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.222082+0000","last_change":"2024-04-24T23:27:18.970811+0000","last_active":"2024-04-24T23:29:08.222082+0000","last_peered":"2024-04-24T23:29:08.222082+0000","last_clean":"2024-04-24T23:29:08.222082+0000","last_became_active":"2024-04-24T23:27:18.970292+0000","last_became_peered":"2024-04-24T23:27:18.970292+0000","last_unstale":"2024-04-24T23:29:08.222082+0000","last_undegraded":"2024-04-24T23:29:08.222082+0000","last_fullsized":"2024-04-24T23:29:08.222082+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:06:37.556527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"92'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.249285+0000","last_change":"2024-04-24T23:27:19.975455+0000","last_active":"2024-04-24T23:29:08.249285+0000","last_peered":"2024-04-24T23:29:08.249285+0000","last_clean":"2024-04-24T23:29:08.249285+0000","last_became_active":"2024-04-24T23:27:19.973504+0000","last_became_peered":"2024-04-24T23:27:19.973504+0000","last_unstale":"2024-04-24T23:29:08.249285+0000","last_undegraded":"2024-04-24T23:29:08.249285+0000","last_fullsized":"2024-04-24T23:29:08.249285+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:50:34.450370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.212709+0000","last_change":"2024-04-24T23:27:24.719253+0000","last_active":"2024-04-24T23:29:08.212709+0000","last_peered":"2024-04-24T23:29:08.212709+0000","last_clean":"2024-04-24T23:29:08.212709+0000","last_became_active":"2024-04-24T23:27:24.718357+0000","last_became_peered":"2024-04-24T23:27:24.718357+0000","last_unstale":"2024-04-24T23:29:08.212709+0000","last_undegraded":"2024-04-24T23:29:08.212709+0000","last_fullsized":"2024-04-24T23:29:08.212709+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:31:03.798037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"94'10","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:28.012182+0000","last_change":"2024-04-24T23:27:18.974310+0000","last_active":"2024-04-24T23:29:28.012182+0000","last_peered":"2024-04-24T23:29:28.012182+0000","last_clean":"2024-04-24T23:29:28.012182+0000","last_became_active":"2024-04-24T23:27:18.973956+0000","last_became_peered":"2024-04-24T23:27:18.973956+0000","last_unstale":"2024-04-24T23:29:28.012182+0000","last_undegraded":"2024-04-24T23:29:28.012182+0000","last_fullsized":"2024-04-24T23:29:28.012182+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:44:27.233155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"92'5","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.213085+0000","last_change":"2024-04-24T23:27:19.985763+0000","last_active":"2024-04-24T23:29:08.213085+0000","last_peered":"2024-04-24T23:29:08.213085+0000","last_clean":"2024-04-24T23:29:08.213085+0000","last_became_active":"2024-04-24T23:27:19.985413+0000","last_became_peered":"2024-04-24T23:27:19.985413+0000","last_unstale":"2024-04-24T23:29:08.213085+0000","last_undegraded":"2024-04-24T23:29:08.213085+0000","last_fullsized":"2024-04-24T23:29:08.213085+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:04:54.910407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.214033+0000","last_change":"2024-04-24T23:27:24.225615+0000","last_active":"2024-04-24T23:29:08.214033+0000","last_peered":"2024-04-24T23:29:08.214033+0000","last_clean":"2024-04-24T23:29:08.214033+0000","last_became_active":"2024-04-24T23:27:24.225029+0000","last_became_peered":"2024-04-24T23:27:24.225029+0000","last_unstale":"2024-04-24T23:29:08.214033+0000","last_undegraded":"2024-04-24T23:29:08.214033+0000","last_fullsized":"2024-04-24T23:29:08.214033+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:56:02.956669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.218561+0000","last_change":"2024-04-24T23:27:18.963235+0000","last_active":"2024-04-24T23:29:08.218561+0000","last_peered":"2024-04-24T23:29:08.218561+0000","last_clean":"2024-04-24T23:29:08.218561+0000","last_became_active":"2024-04-24T23:27:18.962835+0000","last_became_peered":"2024-04-24T23:27:18.962835+0000","last_unstale":"2024-04-24T23:29:08.218561+0000","last_undegraded":"2024-04-24T23:29:08.218561+0000","last_fullsized":"2024-04-24T23:29:08.218561+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:34:29.376856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"92'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.387790+0000","last_change":"2024-04-24T23:27:19.987386+0000","last_active":"2024-04-24T23:29:08.387790+0000","last_peered":"2024-04-24T23:29:08.387790+0000","last_clean":"2024-04-24T23:29:08.387790+0000","last_became_active":"2024-04-24T23:27:19.987003+0000","last_became_peered":"2024-04-24T23:27:19.987003+0000","last_unstale":"2024-04-24T23:29:08.387790+0000","last_undegraded":"2024-04-24T23:29:08.387790+0000","last_fullsized":"2024-04-24T23:29:08.387790+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:20:25.635732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.218484+0000","last_change":"2024-04-24T23:27:24.719291+0000","last_active":"2024-04-24T23:29:08.218484+0000","last_peered":"2024-04-24T23:29:08.218484+0000","last_clean":"2024-04-24T23:29:08.218484+0000","last_became_active":"2024-04-24T23:27:24.718796+0000","last_became_peered":"2024-04-24T23:27:24.718796+0000","last_unstale":"2024-04-24T23:29:08.218484+0000","last_undegraded":"2024-04-24T23:29:08.218484+0000","last_fullsized":"2024-04-24T23:29:08.218484+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:20:25.135902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.220641+0000","last_change":"2024-04-24T23:27:18.967856+0000","last_active":"2024-04-24T23:29:08.220641+0000","last_peered":"2024-04-24T23:29:08.220641+0000","last_clean":"2024-04-24T23:29:08.220641+0000","last_became_active":"2024-04-24T23:27:18.967497+0000","last_became_peered":"2024-04-24T23:27:18.967497+0000","last_unstale":"2024-04-24T23:29:08.220641+0000","last_undegraded":"2024-04-24T23:29:08.220641+0000","last_fullsized":"2024-04-24T23:29:08.220641+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:21:29.841791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"92'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.221174+0000","last_change":"2024-04-24T23:27:19.981302+0000","last_active":"2024-04-24T23:29:08.221174+0000","last_peered":"2024-04-24T23:29:08.221174+0000","last_clean":"2024-04-24T23:29:08.221174+0000","last_became_active":"2024-04-24T23:27:19.980938+0000","last_became_peered":"2024-04-24T23:27:19.980938+0000","last_unstale":"2024-04-24T23:29:08.221174+0000","last_undegraded":"2024-04-24T23:29:08.221174+0000","last_fullsized":"2024-04-24T23:29:08.221174+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:24:53.939882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.221231+0000","last_change":"2024-04-24T23:27:24.224626+0000","last_active":"2024-04-24T23:29:08.221231+0000","last_peered":"2024-04-24T23:29:08.221231+0000","last_clean":"2024-04-24T23:29:08.221231+0000","last_became_active":"2024-04-24T23:27:24.224175+0000","last_became_peered":"2024-04-24T23:27:24.224175+0000","last_unstale":"2024-04-24T23:29:08.221231+0000","last_undegraded":"2024-04-24T23:29:08.221231+0000","last_fullsized":"2024-04-24T23:29:08.221231+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:54:44.155637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"90'3","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.212460+0000","last_change":"2024-04-24T23:27:18.975662+0000","last_active":"2024-04-24T23:29:08.212460+0000","last_peered":"2024-04-24T23:29:08.212460+0000","last_clean":"2024-04-24T23:29:08.212460+0000","last_became_active":"2024-04-24T23:27:18.975326+0000","last_became_peered":"2024-04-24T23:27:18.975326+0000","last_unstale":"2024-04-24T23:29:08.212460+0000","last_undegraded":"2024-04-24T23:29:08.212460+0000","last_fullsized":"2024-04-24T23:29:08.212460+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:39.440866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"92'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.216748+0000","last_change":"2024-04-24T23:27:19.978818+0000","last_active":"2024-04-24T23:29:08.216748+0000","last_peered":"2024-04-24T23:29:08.216748+0000","last_clean":"2024-04-24T23:29:08.216748+0000","last_became_active":"2024-04-24T23:27:19.978107+0000","last_became_peered":"2024-04-24T23:27:19.978107+0000","last_unstale":"2024-04-24T23:29:08.216748+0000","last_undegraded":"2024-04-24T23:29:08.216748+0000","last_fullsized":"2024-04-24T23:29:08.216748+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:22:25.797996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.255331+0000","last_change":"2024-04-24T23:29:08.254854+0000","last_active":"2024-04-24T23:29:08.255331+0000","last_peered":"2024-04-24T23:29:08.255331+0000","last_clean":"2024-04-24T23:29:08.255331+0000","last_became_active":"2024-04-24T23:29:08.250700+0000","last_became_peered":"2024-04-24T23:29:08.250700+0000","last_unstale":"2024-04-24T23:29:08.255331+0000","last_undegraded":"2024-04-24T23:29:08.255331+0000","last_fullsized":"2024-04-24T23:29:08.255331+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:25:31.400400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'1","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.214173+0000","last_change":"2024-04-24T23:27:18.967165+0000","last_active":"2024-04-24T23:29:08.214173+0000","last_peered":"2024-04-24T23:29:08.214173+0000","last_clean":"2024-04-24T23:29:08.214173+0000","last_became_active":"2024-04-24T23:27:18.966628+0000","last_became_peered":"2024-04-24T23:27:18.966628+0000","last_unstale":"2024-04-24T23:29:08.214173+0000","last_undegraded":"2024-04-24T23:29:08.214173+0000","last_fullsized":"2024-04-24T23:29:08.214173+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:12:20.520777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"92'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.212534+0000","last_change":"2024-04-24T23:27:19.988069+0000","last_active":"2024-04-24T23:29:08.212534+0000","last_peered":"2024-04-24T23:29:08.212534+0000","last_clean":"2024-04-24T23:29:08.212534+0000","last_became_active":"2024-04-24T23:27:19.987660+0000","last_became_peered":"2024-04-24T23:27:19.987660+0000","last_unstale":"2024-04-24T23:29:08.212534+0000","last_undegraded":"2024-04-24T23:29:08.212534+0000","last_fullsized":"2024-04-24T23:29:08.212534+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:21:03.505313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.218455+0000","last_change":"2024-04-24T23:27:24.719041+0000","last_active":"2024-04-24T23:29:08.218455+0000","last_peered":"2024-04-24T23:29:08.218455+0000","last_clean":"2024-04-24T23:29:08.218455+0000","last_became_active":"2024-04-24T23:27:24.718523+0000","last_became_peered":"2024-04-24T23:27:24.718523+0000","last_unstale":"2024-04-24T23:29:08.218455+0000","last_undegraded":"2024-04-24T23:29:08.218455+0000","last_fullsized":"2024-04-24T23:29:08.218455+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:27:57.396800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.221356+0000","last_change":"2024-04-24T23:27:18.975034+0000","last_active":"2024-04-24T23:29:08.221356+0000","last_peered":"2024-04-24T23:29:08.221356+0000","last_clean":"2024-04-24T23:29:08.221356+0000","last_became_active":"2024-04-24T23:27:18.973764+0000","last_became_peered":"2024-04-24T23:27:18.973764+0000","last_unstale":"2024-04-24T23:29:08.221356+0000","last_undegraded":"2024-04-24T23:29:08.221356+0000","last_fullsized":"2024-04-24T23:29:08.221356+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:43:23.451904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"92'5","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.221861+0000","last_change":"2024-04-24T23:27:19.984948+0000","last_active":"2024-04-24T23:29:08.221861+0000","last_peered":"2024-04-24T23:29:08.221861+0000","last_clean":"2024-04-24T23:29:08.221861+0000","last_became_active":"2024-04-24T23:27:19.984484+0000","last_became_peered":"2024-04-24T23:27:19.984484+0000","last_unstale":"2024-04-24T23:29:08.221861+0000","last_undegraded":"2024-04-24T23:29:08.221861+0000","last_fullsized":"2024-04-24T23:29:08.221861+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:17:43.335983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.251471+0000","last_change":"2024-04-24T23:29:08.250873+0000","last_active":"2024-04-24T23:29:08.251471+0000","last_peered":"2024-04-24T23:29:08.251471+0000","last_clean":"2024-04-24T23:29:08.251471+0000","last_became_active":"2024-04-24T23:29:08.250147+0000","last_became_peered":"2024-04-24T23:29:08.250147+0000","last_unstale":"2024-04-24T23:29:08.251471+0000","last_undegraded":"2024-04-24T23:29:08.251471+0000","last_fullsized":"2024-04-24T23:29:08.251471+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:14:42.358098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6,7],"acting":[1,8,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.214665+0000","last_change":"2024-04-24T23:27:18.964232+0000","last_active":"2024-04-24T23:29:08.214665+0000","last_peered":"2024-04-24T23:29:08.214665+0000","last_clean":"2024-04-24T23:29:08.214665+0000","last_became_active":"2024-04-24T23:27:18.963855+0000","last_became_peered":"2024-04-24T23:27:18.963855+0000","last_unstale":"2024-04-24T23:29:08.214665+0000","last_undegraded":"2024-04-24T23:29:08.214665+0000","last_fullsized":"2024-04-24T23:29:08.214665+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:02:21.000326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"92'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.222653+0000","last_change":"2024-04-24T23:27:19.984354+0000","last_active":"2024-04-24T23:29:08.222653+0000","last_peered":"2024-04-24T23:29:08.222653+0000","last_clean":"2024-04-24T23:29:08.222653+0000","last_became_active":"2024-04-24T23:27:19.983972+0000","last_became_peered":"2024-04-24T23:27:19.983972+0000","last_unstale":"2024-04-24T23:29:08.222653+0000","last_undegraded":"2024-04-24T23:29:08.222653+0000","last_fullsized":"2024-04-24T23:29:08.222653+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:28:28.390283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.387070+0000","last_change":"2024-04-24T23:27:24.238404+0000","last_active":"2024-04-24T23:29:08.387070+0000","last_peered":"2024-04-24T23:29:08.387070+0000","last_clean":"2024-04-24T23:29:08.387070+0000","last_became_active":"2024-04-24T23:27:24.237976+0000","last_became_peered":"2024-04-24T23:27:24.237976+0000","last_unstale":"2024-04-24T23:29:08.387070+0000","last_undegraded":"2024-04-24T23:29:08.387070+0000","last_fullsized":"2024-04-24T23:29:08.387070+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:19:31.476789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.249641+0000","last_change":"2024-04-24T23:27:18.971259+0000","last_active":"2024-04-24T23:29:08.249641+0000","last_peered":"2024-04-24T23:29:08.249641+0000","last_clean":"2024-04-24T23:29:08.249641+0000","last_became_active":"2024-04-24T23:27:18.970856+0000","last_became_peered":"2024-04-24T23:27:18.970856+0000","last_unstale":"2024-04-24T23:29:08.249641+0000","last_undegraded":"2024-04-24T23:29:08.249641+0000","last_fullsized":"2024-04-24T23:29:08.249641+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:54:58.642528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"92'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.222288+0000","last_change":"2024-04-24T23:27:19.986320+0000","last_active":"2024-04-24T23:29:08.222288+0000","last_peered":"2024-04-24T23:29:08.222288+0000","last_clean":"2024-04-24T23:29:08.222288+0000","last_became_active":"2024-04-24T23:27:19.985951+0000","last_became_peered":"2024-04-24T23:27:19.985951+0000","last_unstale":"2024-04-24T23:29:08.222288+0000","last_undegraded":"2024-04-24T23:29:08.222288+0000","last_fullsized":"2024-04-24T23:29:08.222288+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:05:35.427735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.248604+0000","last_change":"2024-04-24T23:27:24.228149+0000","last_active":"2024-04-24T23:29:08.248604+0000","last_peered":"2024-04-24T23:29:08.248604+0000","last_clean":"2024-04-24T23:29:08.248604+0000","last_became_active":"2024-04-24T23:27:24.227683+0000","last_became_peered":"2024-04-24T23:27:24.227683+0000","last_unstale":"2024-04-24T23:29:08.248604+0000","last_undegraded":"2024-04-24T23:29:08.248604+0000","last_fullsized":"2024-04-24T23:29:08.248604+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:55:50.408834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":46,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.249519+0000","last_change":"2024-04-24T23:28:08.059374+0000","last_active":"2024-04-24T23:29:08.249519+0000","last_peered":"2024-04-24T23:29:08.249519+0000","last_clean":"2024-04-24T23:29:08.249519+0000","last_became_active":"2024-04-24T23:28:08.056104+0000","last_became_peered":"2024-04-24T23:28:08.056104+0000","last_unstale":"2024-04-24T23:29:08.249519+0000","last_undegraded":"2024-04-24T23:29:08.249519+0000","last_fullsized":"2024-04-24T23:29:08.249519+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:07:17.185948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"92'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.222499+0000","last_change":"2024-04-24T23:27:19.990935+0000","last_active":"2024-04-24T23:29:08.222499+0000","last_peered":"2024-04-24T23:29:08.222499+0000","last_clean":"2024-04-24T23:29:08.222499+0000","last_became_active":"2024-04-24T23:27:19.990524+0000","last_became_peered":"2024-04-24T23:27:19.990524+0000","last_unstale":"2024-04-24T23:29:08.222499+0000","last_undegraded":"2024-04-24T23:29:08.222499+0000","last_fullsized":"2024-04-24T23:29:08.222499+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:54:02.463248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"92'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.216879+0000","last_change":"2024-04-24T23:27:19.979142+0000","last_active":"2024-04-24T23:29:08.216879+0000","last_peered":"2024-04-24T23:29:08.216879+0000","last_clean":"2024-04-24T23:29:08.216879+0000","last_became_active":"2024-04-24T23:27:19.978799+0000","last_became_peered":"2024-04-24T23:27:19.978799+0000","last_unstale":"2024-04-24T23:29:08.216879+0000","last_undegraded":"2024-04-24T23:29:08.216879+0000","last_fullsized":"2024-04-24T23:29:08.216879+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:07:19.046313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"92'12","reported_seq":50,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.214206+0000","last_change":"2024-04-24T23:28:08.109820+0000","last_active":"2024-04-24T23:29:08.214206+0000","last_peered":"2024-04-24T23:29:08.214206+0000","last_clean":"2024-04-24T23:29:08.214206+0000","last_became_active":"2024-04-24T23:28:08.109500+0000","last_became_peered":"2024-04-24T23:28:08.109500+0000","last_unstale":"2024-04-24T23:29:08.214206+0000","last_undegraded":"2024-04-24T23:29:08.214206+0000","last_fullsized":"2024-04-24T23:29:08.214206+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:34:32.814256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"92'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.214320+0000","last_change":"2024-04-24T23:27:19.980008+0000","last_active":"2024-04-24T23:29:08.214320+0000","last_peered":"2024-04-24T23:29:08.214320+0000","last_clean":"2024-04-24T23:29:08.214320+0000","last_became_active":"2024-04-24T23:27:19.979424+0000","last_became_peered":"2024-04-24T23:27:19.979424+0000","last_unstale":"2024-04-24T23:29:08.214320+0000","last_undegraded":"2024-04-24T23:29:08.214320+0000","last_fullsized":"2024-04-24T23:29:08.214320+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:44.449245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"92'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.387685+0000","last_change":"2024-04-24T23:27:19.987574+0000","last_active":"2024-04-24T23:29:08.387685+0000","last_peered":"2024-04-24T23:29:08.387685+0000","last_clean":"2024-04-24T23:29:08.387685+0000","last_became_active":"2024-04-24T23:27:19.987242+0000","last_became_peered":"2024-04-24T23:27:19.987242+0000","last_unstale":"2024-04-24T23:29:08.387685+0000","last_undegraded":"2024-04-24T23:29:08.387685+0000","last_fullsized":"2024-04-24T23:29:08.387685+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:39:10.777152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"92'4","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.249324+0000","last_change":"2024-04-24T23:27:19.980983+0000","last_active":"2024-04-24T23:29:08.249324+0000","last_peered":"2024-04-24T23:29:08.249324+0000","last_clean":"2024-04-24T23:29:08.249324+0000","last_became_active":"2024-04-24T23:27:19.980659+0000","last_became_peered":"2024-04-24T23:27:19.980659+0000","last_unstale":"2024-04-24T23:29:08.249324+0000","last_undegraded":"2024-04-24T23:29:08.249324+0000","last_fullsized":"2024-04-24T23:29:08.249324+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:02:52.647687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"92'6","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.217260+0000","last_change":"2024-04-24T23:28:08.055819+0000","last_active":"2024-04-24T23:29:08.217260+0000","last_peered":"2024-04-24T23:29:08.217260+0000","last_clean":"2024-04-24T23:29:08.217260+0000","last_became_active":"2024-04-24T23:28:08.055386+0000","last_became_peered":"2024-04-24T23:28:08.055386+0000","last_unstale":"2024-04-24T23:29:08.217260+0000","last_undegraded":"2024-04-24T23:29:08.217260+0000","last_fullsized":"2024-04-24T23:29:08.217260+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:59:30.488812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"92'15","reported_seq":47,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.214979+0000","last_change":"2024-04-24T23:27:19.985864+0000","last_active":"2024-04-24T23:29:08.214979+0000","last_peered":"2024-04-24T23:29:08.214979+0000","last_clean":"2024-04-24T23:29:08.214979+0000","last_became_active":"2024-04-24T23:27:19.984877+0000","last_became_peered":"2024-04-24T23:27:19.984877+0000","last_unstale":"2024-04-24T23:29:08.214979+0000","last_undegraded":"2024-04-24T23:29:08.214979+0000","last_fullsized":"2024-04-24T23:29:08.214979+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:14:43.458970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"92'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.213153+0000","last_change":"2024-04-24T23:27:19.988831+0000","last_active":"2024-04-24T23:29:08.213153+0000","last_peered":"2024-04-24T23:29:08.213153+0000","last_clean":"2024-04-24T23:29:08.213153+0000","last_became_active":"2024-04-24T23:27:19.988478+0000","last_became_peered":"2024-04-24T23:27:19.988478+0000","last_unstale":"2024-04-24T23:29:08.213153+0000","last_undegraded":"2024-04-24T23:29:08.213153+0000","last_fullsized":"2024-04-24T23:29:08.213153+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:18:37.488710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"92'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.249789+0000","last_change":"2024-04-24T23:27:19.983553+0000","last_active":"2024-04-24T23:29:08.249789+0000","last_peered":"2024-04-24T23:29:08.249789+0000","last_clean":"2024-04-24T23:29:08.249789+0000","last_became_active":"2024-04-24T23:27:19.981042+0000","last_became_peered":"2024-04-24T23:27:19.981042+0000","last_unstale":"2024-04-24T23:29:08.249789+0000","last_undegraded":"2024-04-24T23:29:08.249789+0000","last_fullsized":"2024-04-24T23:29:08.249789+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:09:42.608238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"92'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.212461+0000","last_change":"2024-04-24T23:27:19.985425+0000","last_active":"2024-04-24T23:29:08.212461+0000","last_peered":"2024-04-24T23:29:08.212461+0000","last_clean":"2024-04-24T23:29:08.212461+0000","last_became_active":"2024-04-24T23:27:19.985094+0000","last_became_peered":"2024-04-24T23:27:19.985094+0000","last_unstale":"2024-04-24T23:29:08.212461+0000","last_undegraded":"2024-04-24T23:29:08.212461+0000","last_fullsized":"2024-04-24T23:29:08.212461+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:07:15.612247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"92'10","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.220939+0000","last_change":"2024-04-24T23:27:19.988224+0000","last_active":"2024-04-24T23:29:08.220939+0000","last_peered":"2024-04-24T23:29:08.220939+0000","last_clean":"2024-04-24T23:29:08.220939+0000","last_became_active":"2024-04-24T23:27:19.987820+0000","last_became_peered":"2024-04-24T23:27:19.987820+0000","last_unstale":"2024-04-24T23:29:08.220939+0000","last_undegraded":"2024-04-24T23:29:08.220939+0000","last_fullsized":"2024-04-24T23:29:08.220939+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:27.653290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"92'11","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.249349+0000","last_change":"2024-04-24T23:27:19.989672+0000","last_active":"2024-04-24T23:29:08.249349+0000","last_peered":"2024-04-24T23:29:08.249349+0000","last_clean":"2024-04-24T23:29:08.249349+0000","last_became_active":"2024-04-24T23:27:19.989337+0000","last_became_peered":"2024-04-24T23:27:19.989337+0000","last_unstale":"2024-04-24T23:29:08.249349+0000","last_undegraded":"2024-04-24T23:29:08.249349+0000","last_fullsized":"2024-04-24T23:29:08.249349+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:07:13.399366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"92'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.216974+0000","last_change":"2024-04-24T23:27:19.987703+0000","last_active":"2024-04-24T23:29:08.216974+0000","last_peered":"2024-04-24T23:29:08.216974+0000","last_clean":"2024-04-24T23:29:08.216974+0000","last_became_active":"2024-04-24T23:27:19.987355+0000","last_became_peered":"2024-04-24T23:27:19.987355+0000","last_unstale":"2024-04-24T23:29:08.216974+0000","last_undegraded":"2024-04-24T23:29:08.216974+0000","last_fullsized":"2024-04-24T23:29:08.216974+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:03:15.812726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":280134,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":64410,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":741,"num_read_kb":441,"num_write":174,"num_write_kb":123,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":694,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":193230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":151,"ondisk_log_size":151,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383723,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1036288,"data_stored":529776,"data_compressed":29657,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645360,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28168,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716984,"statfs":{"total":95995035648,"available":95966191616,"internally_reserved":0,"allocated":1179648,"data_stored":1180826,"data_compressed":65288,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808886,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32208,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712944,"statfs":{"total":95995035648,"available":95962054656,"internally_reserved":0,"allocated":1048576,"data_stored":529430,"data_compressed":29657,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070523,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1048576,"data_stored":529350,"data_compressed":29657,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332160,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28088,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717064,"statfs":{"total":95995035648,"available":95966273536,"internally_reserved":0,"allocated":1093632,"data_stored":1125658,"data_compressed":65288,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495686,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1028096,"data_stored":524606,"data_compressed":29657,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574943,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36412,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708740,"statfs":{"total":95995035648,"available":95957749760,"internally_reserved":0,"allocated":1101824,"data_stored":584056,"data_compressed":29657,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738467,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1052672,"data_stored":525994,"data_compressed":29657,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411418,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32260,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712892,"statfs":{"total":95995035648,"available":95962001408,"internally_reserved":0,"allocated":1110016,"data_stored":585714,"data_compressed":29657,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182485,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1040384,"data_stored":524800,"data_compressed":29657,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920848,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1040384,"data_stored":530096,"data_compressed":29657,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757323,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1105920,"data_stored":1120988,"data_compressed":65288,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33993,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33993,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33993,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35631,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35631,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35631,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35631,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35631,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":59540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":61198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":61846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T23:29:46.728 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T23:29:46.862 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.859+0000 7fa4e7e1c640 1 -- 172.21.15.80:0/2184785063 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa4e0103360 msgr2=0x7fa4e0109bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:46.863 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.859+0000 7fa4e7e1c640 1 --2- 172.21.15.80:0/2184785063 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa4e0103360 0x7fa4e0109bf0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fa4d8009ca0 tx=0x7fa4d802a8c0 comp rx=0 tx=0).stop 2024-04-24T23:29:46.863 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.859+0000 7fa4e7e1c640 1 -- 172.21.15.80:0/2184785063 shutdown_connections 2024-04-24T23:29:46.863 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.859+0000 7fa4e7e1c640 1 --2- 172.21.15.80:0/2184785063 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa4e0103360 0x7fa4e0109bf0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:46.863 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.859+0000 7fa4e7e1c640 1 --2- 172.21.15.80:0/2184785063 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4e01029a0 0x7fa4e0102e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:46.863 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.859+0000 7fa4e7e1c640 1 --2- 172.21.15.80:0/2184785063 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa4e01017b0 0x7fa4e0101bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:46.863 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.859+0000 7fa4e7e1c640 1 -- 172.21.15.80:0/2184785063 >> 172.21.15.80:0/2184785063 conn(0x7fa4e00fcd40 msgr2=0x7fa4e00ff160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:46.863 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.859+0000 7fa4e7e1c640 1 -- 172.21.15.80:0/2184785063 shutdown_connections 2024-04-24T23:29:46.863 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.859+0000 7fa4e7e1c640 1 -- 172.21.15.80:0/2184785063 wait complete. 2024-04-24T23:29:46.863 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.860+0000 7fa4e7e1c640 1 Processor -- start 2024-04-24T23:29:46.863 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.860+0000 7fa4e7e1c640 1 -- start start 2024-04-24T23:29:46.863 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.860+0000 7fa4e7e1c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa4e01017b0 0x7fa4e0197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:46.864 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.860+0000 7fa4e7e1c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4e01029a0 0x7fa4e0197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:46.864 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.860+0000 7fa4e7e1c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa4e0103360 0x7fa4e019f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:46.864 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.860+0000 7fa4e7e1c640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fa4e010d430 con 0x7fa4e0103360 2024-04-24T23:29:46.864 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.860+0000 7fa4e7e1c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa4e010d2b0 con 0x7fa4e01017b0 2024-04-24T23:29:46.864 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.860+0000 7fa4e7e1c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa4e010d5b0 con 0x7fa4e01029a0 2024-04-24T23:29:46.864 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.860+0000 7fa4e63b9640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa4e0103360 0x7fa4e019f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:46.864 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.860+0000 7fa4e63b9640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa4e0103360 0x7fa4e019f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:52032/0 (socket says 172.21.15.80:52032) 2024-04-24T23:29:46.864 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.860+0000 7fa4e63b9640 1 -- 172.21.15.80:0/404145689 learned_addr learned my addr 172.21.15.80:0/404145689 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:29:46.864 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.860+0000 7fa4e5bb8640 1 --2- 172.21.15.80:0/404145689 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa4e01017b0 0x7fa4e0197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:46.864 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.860+0000 7fa4e53b7640 1 --2- 172.21.15.80:0/404145689 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4e01029a0 0x7fa4e0197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:46.864 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.860+0000 7fa4e63b9640 1 -- 172.21.15.80:0/404145689 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4e01029a0 msgr2=0x7fa4e0197fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:46.864 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.860+0000 7fa4e63b9640 1 --2- 172.21.15.80:0/404145689 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4e01029a0 0x7fa4e0197fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:46.865 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.860+0000 7fa4e63b9640 1 -- 172.21.15.80:0/404145689 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa4e01017b0 msgr2=0x7fa4e0197aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:46.865 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.860+0000 7fa4e63b9640 1 --2- 172.21.15.80:0/404145689 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa4e01017b0 0x7fa4e0197aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:46.865 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.860+0000 7fa4e63b9640 1 -- 172.21.15.80:0/404145689 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4e019f6c0 con 0x7fa4e0103360 2024-04-24T23:29:46.865 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.860+0000 7fa4e5bb8640 1 --2- 172.21.15.80:0/404145689 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa4e01017b0 0x7fa4e0197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:29:46.865 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.861+0000 7fa4e63b9640 1 --2- 172.21.15.80:0/404145689 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa4e0103360 0x7fa4e019f050 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fa4d8005e30 tx=0x7fa4d8031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:46.865 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.861+0000 7fa4d6ffd640 1 -- 172.21.15.80:0/404145689 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d8003c30 con 0x7fa4e0103360 2024-04-24T23:29:46.865 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.861+0000 7fa4d6ffd640 1 -- 172.21.15.80:0/404145689 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4d8003dd0 con 0x7fa4e0103360 2024-04-24T23:29:46.865 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.861+0000 7fa4e7e1c640 1 -- 172.21.15.80:0/404145689 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4e019f950 con 0x7fa4e0103360 2024-04-24T23:29:46.865 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.861+0000 7fa4d6ffd640 1 -- 172.21.15.80:0/404145689 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d802fde0 con 0x7fa4e0103360 2024-04-24T23:29:46.865 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.861+0000 7fa4e7e1c640 1 -- 172.21.15.80:0/404145689 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4e019fea0 con 0x7fa4e0103360 2024-04-24T23:29:46.865 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.862+0000 7fa4e7e1c640 1 -- 172.21.15.80:0/404145689 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4e0108ba0 con 0x7fa4e0103360 2024-04-24T23:29:46.866 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.863+0000 7fa4d6ffd640 1 -- 172.21.15.80:0/404145689 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa4d8004050 con 0x7fa4e0103360 2024-04-24T23:29:46.866 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.863+0000 7fa4d6ffd640 1 --2- 172.21.15.80:0/404145689 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fa4cc0a2b90 0x7fa4cc0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:46.866 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.864+0000 7fa4e5bb8640 1 --2- 172.21.15.80:0/404145689 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fa4cc0a2b90 0x7fa4cc0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:46.866 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.864+0000 7fa4d6ffd640 1 -- 172.21.15.80:0/404145689 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fa4d80f09d0 con 0x7fa4e0103360 2024-04-24T23:29:46.866 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.864+0000 7fa4e5bb8640 1 --2- 172.21.15.80:0/404145689 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fa4cc0a2b90 0x7fa4cc0a5050 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fa4e0102800 tx=0x7fa4dc00a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:46.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:46.868+0000 7fa4d6ffd640 1 -- 172.21.15.80:0/404145689 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa4d80bd680 con 0x7fa4e0103360 2024-04-24T23:29:47.045 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.045+0000 7fa4e7e1c640 1 -- 172.21.15.80:0/404145689 --> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa4e0061b80 con 0x7fa4cc0a2b90 2024-04-24T23:29:47.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.054+0000 7fa4d6ffd640 1 -- 172.21.15.80:0/404145689 <== mgr.14152 v2:172.21.15.80:6800/3830596922 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340774 (secure 0 0 0) 0x7fa4e0061b80 con 0x7fa4cc0a2b90 2024-04-24T23:29:47.058 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-24T23:29:47.060 INFO:teuthology.orchestra.run.smithi080.stderr:dumped all 2024-04-24T23:29:47.061 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.057+0000 7fa4e7e1c640 1 -- 172.21.15.80:0/404145689 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fa4cc0a2b90 msgr2=0x7fa4cc0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:47.061 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.057+0000 7fa4e7e1c640 1 --2- 172.21.15.80:0/404145689 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fa4cc0a2b90 0x7fa4cc0a5050 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fa4e0102800 tx=0x7fa4dc00a560 comp rx=0 tx=0).stop 2024-04-24T23:29:47.062 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.057+0000 7fa4e7e1c640 1 -- 172.21.15.80:0/404145689 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa4e0103360 msgr2=0x7fa4e019f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:47.062 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.057+0000 7fa4e7e1c640 1 --2- 172.21.15.80:0/404145689 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa4e0103360 0x7fa4e019f050 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fa4d8005e30 tx=0x7fa4d8031ee0 comp rx=0 tx=0).stop 2024-04-24T23:29:47.062 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.057+0000 7fa4e7e1c640 1 -- 172.21.15.80:0/404145689 shutdown_connections 2024-04-24T23:29:47.062 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.057+0000 7fa4e7e1c640 1 --2- 172.21.15.80:0/404145689 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fa4cc0a2b90 0x7fa4cc0a5050 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:47.062 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.057+0000 7fa4e7e1c640 1 --2- 172.21.15.80:0/404145689 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa4e0103360 0x7fa4e019f050 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:47.062 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.057+0000 7fa4e7e1c640 1 --2- 172.21.15.80:0/404145689 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4e01029a0 0x7fa4e0197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:47.062 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.057+0000 7fa4e7e1c640 1 --2- 172.21.15.80:0/404145689 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa4e01017b0 0x7fa4e0197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:47.062 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.057+0000 7fa4e7e1c640 1 -- 172.21.15.80:0/404145689 >> 172.21.15.80:0/404145689 conn(0x7fa4e00fcd40 msgr2=0x7fa4e0105d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:47.062 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.057+0000 7fa4e7e1c640 1 -- 172.21.15.80:0/404145689 shutdown_connections 2024-04-24T23:29:47.062 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.057+0000 7fa4e7e1c640 1 -- 172.21.15.80:0/404145689 wait complete. 2024-04-24T23:29:47.069 INFO:teuthology.orchestra.run.smithi080.stdout:{"pg_ready":true,"pg_map":{"version":374,"stamp":"2024-04-24T23:29:46.919868+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655012,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":867,"num_read_kb":550,"num_write":359,"num_write_kb":3064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":1181547,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":791,"ondisk_log_size":791,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":365932,"kb_used_data":12584,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124575892,"statfs":{"total":1151940427776,"available":1151565713408,"internally_reserved":0,"allocated":12886016,"data_stored":8291294,"data_compressed":462777,"data_compressed_allocated":626688,"data_compressed_original":2752512,"omap_allocated":19141,"internal_metadata":361674043},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":616,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":62,"num_read_kb":32,"num_write":8,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003273"},"pg_stats":[{"pgid":"3.3f","version":"92'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.217305+0000","last_change":"2024-04-24T23:27:19.978807+0000","last_active":"2024-04-24T23:29:08.217305+0000","last_peered":"2024-04-24T23:29:08.217305+0000","last_clean":"2024-04-24T23:29:08.217305+0000","last_became_active":"2024-04-24T23:27:19.978436+0000","last_became_peered":"2024-04-24T23:27:19.978436+0000","last_unstale":"2024-04-24T23:29:08.217305+0000","last_undegraded":"2024-04-24T23:29:08.217305+0000","last_fullsized":"2024-04-24T23:29:08.217305+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:00:13.126518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"92'4","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.387809+0000","last_change":"2024-04-24T23:27:19.980327+0000","last_active":"2024-04-24T23:29:08.387809+0000","last_peered":"2024-04-24T23:29:08.387809+0000","last_clean":"2024-04-24T23:29:08.387809+0000","last_became_active":"2024-04-24T23:27:19.979912+0000","last_became_peered":"2024-04-24T23:27:19.979912+0000","last_unstale":"2024-04-24T23:29:08.387809+0000","last_undegraded":"2024-04-24T23:29:08.387809+0000","last_fullsized":"2024-04-24T23:29:08.387809+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:25:41.657205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"92'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.387723+0000","last_change":"2024-04-24T23:27:19.976451+0000","last_active":"2024-04-24T23:29:08.387723+0000","last_peered":"2024-04-24T23:29:08.387723+0000","last_clean":"2024-04-24T23:29:08.387723+0000","last_became_active":"2024-04-24T23:27:19.975976+0000","last_became_peered":"2024-04-24T23:27:19.975976+0000","last_unstale":"2024-04-24T23:29:08.387723+0000","last_undegraded":"2024-04-24T23:29:08.387723+0000","last_fullsized":"2024-04-24T23:29:08.387723+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:14:51.605217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"92'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.218709+0000","last_change":"2024-04-24T23:27:19.991245+0000","last_active":"2024-04-24T23:29:08.218709+0000","last_peered":"2024-04-24T23:29:08.218709+0000","last_clean":"2024-04-24T23:29:08.218709+0000","last_became_active":"2024-04-24T23:27:19.990448+0000","last_became_peered":"2024-04-24T23:27:19.990448+0000","last_unstale":"2024-04-24T23:29:08.218709+0000","last_undegraded":"2024-04-24T23:29:08.218709+0000","last_fullsized":"2024-04-24T23:29:08.218709+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:03:21.013005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"92'4","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.221241+0000","last_change":"2024-04-24T23:27:19.977664+0000","last_active":"2024-04-24T23:29:08.221241+0000","last_peered":"2024-04-24T23:29:08.221241+0000","last_clean":"2024-04-24T23:29:08.221241+0000","last_became_active":"2024-04-24T23:27:19.977300+0000","last_became_peered":"2024-04-24T23:27:19.977300+0000","last_unstale":"2024-04-24T23:29:08.221241+0000","last_undegraded":"2024-04-24T23:29:08.221241+0000","last_fullsized":"2024-04-24T23:29:08.221241+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:19:27.229712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"92'6","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.249580+0000","last_change":"2024-04-24T23:27:19.989994+0000","last_active":"2024-04-24T23:29:08.249580+0000","last_peered":"2024-04-24T23:29:08.249580+0000","last_clean":"2024-04-24T23:29:08.249580+0000","last_became_active":"2024-04-24T23:27:19.989639+0000","last_became_peered":"2024-04-24T23:27:19.989639+0000","last_unstale":"2024-04-24T23:29:08.249580+0000","last_undegraded":"2024-04-24T23:29:08.249580+0000","last_fullsized":"2024-04-24T23:29:08.249580+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:20:43.931086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"92'5","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.221180+0000","last_change":"2024-04-24T23:27:19.992886+0000","last_active":"2024-04-24T23:29:08.221180+0000","last_peered":"2024-04-24T23:29:08.221180+0000","last_clean":"2024-04-24T23:29:08.221180+0000","last_became_active":"2024-04-24T23:27:19.992438+0000","last_became_peered":"2024-04-24T23:27:19.992438+0000","last_unstale":"2024-04-24T23:29:08.221180+0000","last_undegraded":"2024-04-24T23:29:08.221180+0000","last_fullsized":"2024-04-24T23:29:08.221180+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:02:34.110984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"92'9","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.212229+0000","last_change":"2024-04-24T23:27:19.981592+0000","last_active":"2024-04-24T23:29:08.212229+0000","last_peered":"2024-04-24T23:29:08.212229+0000","last_clean":"2024-04-24T23:29:08.212229+0000","last_became_active":"2024-04-24T23:27:19.981248+0000","last_became_peered":"2024-04-24T23:27:19.981248+0000","last_unstale":"2024-04-24T23:29:08.212229+0000","last_undegraded":"2024-04-24T23:29:08.212229+0000","last_fullsized":"2024-04-24T23:29:08.212229+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:53:31.191318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"92'11","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.221670+0000","last_change":"2024-04-24T23:27:19.975995+0000","last_active":"2024-04-24T23:29:08.221670+0000","last_peered":"2024-04-24T23:29:08.221670+0000","last_clean":"2024-04-24T23:29:08.221670+0000","last_became_active":"2024-04-24T23:27:19.975614+0000","last_became_peered":"2024-04-24T23:27:19.975614+0000","last_unstale":"2024-04-24T23:29:08.221670+0000","last_undegraded":"2024-04-24T23:29:08.221670+0000","last_fullsized":"2024-04-24T23:29:08.221670+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:33:22.439998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"92'16","reported_seq":47,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.250021+0000","last_change":"2024-04-24T23:27:19.991265+0000","last_active":"2024-04-24T23:29:08.250021+0000","last_peered":"2024-04-24T23:29:08.250021+0000","last_clean":"2024-04-24T23:29:08.250021+0000","last_became_active":"2024-04-24T23:27:19.990928+0000","last_became_peered":"2024-04-24T23:27:19.990928+0000","last_unstale":"2024-04-24T23:29:08.250021+0000","last_undegraded":"2024-04-24T23:29:08.250021+0000","last_fullsized":"2024-04-24T23:29:08.250021+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:20:41.244669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"92'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.222538+0000","last_change":"2024-04-24T23:27:19.985182+0000","last_active":"2024-04-24T23:29:08.222538+0000","last_peered":"2024-04-24T23:29:08.222538+0000","last_clean":"2024-04-24T23:29:08.222538+0000","last_became_active":"2024-04-24T23:27:19.984600+0000","last_became_peered":"2024-04-24T23:27:19.984600+0000","last_unstale":"2024-04-24T23:29:08.222538+0000","last_undegraded":"2024-04-24T23:29:08.222538+0000","last_fullsized":"2024-04-24T23:29:08.222538+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:16:09.919736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"92'7","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.218365+0000","last_change":"2024-04-24T23:27:19.994271+0000","last_active":"2024-04-24T23:29:08.218365+0000","last_peered":"2024-04-24T23:29:08.218365+0000","last_clean":"2024-04-24T23:29:08.218365+0000","last_became_active":"2024-04-24T23:27:19.993926+0000","last_became_peered":"2024-04-24T23:27:19.993926+0000","last_unstale":"2024-04-24T23:29:08.218365+0000","last_undegraded":"2024-04-24T23:29:08.218365+0000","last_fullsized":"2024-04-24T23:29:08.218365+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:49:07.500370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"92'10","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.249303+0000","last_change":"2024-04-24T23:27:19.984230+0000","last_active":"2024-04-24T23:29:08.249303+0000","last_peered":"2024-04-24T23:29:08.249303+0000","last_clean":"2024-04-24T23:29:08.249303+0000","last_became_active":"2024-04-24T23:27:19.983902+0000","last_became_peered":"2024-04-24T23:27:19.983902+0000","last_unstale":"2024-04-24T23:29:08.249303+0000","last_undegraded":"2024-04-24T23:29:08.249303+0000","last_fullsized":"2024-04-24T23:29:08.249303+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:12:40.985092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"92'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.387496+0000","last_change":"2024-04-24T23:27:19.979822+0000","last_active":"2024-04-24T23:29:08.387496+0000","last_peered":"2024-04-24T23:29:08.387496+0000","last_clean":"2024-04-24T23:29:08.387496+0000","last_became_active":"2024-04-24T23:27:19.979452+0000","last_became_peered":"2024-04-24T23:27:19.979452+0000","last_unstale":"2024-04-24T23:29:08.387496+0000","last_undegraded":"2024-04-24T23:29:08.387496+0000","last_fullsized":"2024-04-24T23:29:08.387496+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:08:53.945221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"92'7","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.387472+0000","last_change":"2024-04-24T23:28:08.053470+0000","last_active":"2024-04-24T23:29:08.387472+0000","last_peered":"2024-04-24T23:29:08.387472+0000","last_clean":"2024-04-24T23:29:08.387472+0000","last_became_active":"2024-04-24T23:28:08.053064+0000","last_became_peered":"2024-04-24T23:28:08.053064+0000","last_unstale":"2024-04-24T23:29:08.387472+0000","last_undegraded":"2024-04-24T23:29:08.387472+0000","last_fullsized":"2024-04-24T23:29:08.387472+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:13:34.489239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"92'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.220541+0000","last_change":"2024-04-24T23:27:19.984346+0000","last_active":"2024-04-24T23:29:08.220541+0000","last_peered":"2024-04-24T23:29:08.220541+0000","last_clean":"2024-04-24T23:29:08.220541+0000","last_became_active":"2024-04-24T23:27:19.983999+0000","last_became_peered":"2024-04-24T23:27:19.983999+0000","last_unstale":"2024-04-24T23:29:08.220541+0000","last_undegraded":"2024-04-24T23:29:08.220541+0000","last_fullsized":"2024-04-24T23:29:08.220541+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:15:35.329454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"92'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.217118+0000","last_change":"2024-04-24T23:27:19.979939+0000","last_active":"2024-04-24T23:29:08.217118+0000","last_peered":"2024-04-24T23:29:08.217118+0000","last_clean":"2024-04-24T23:29:08.217118+0000","last_became_active":"2024-04-24T23:27:19.979600+0000","last_became_peered":"2024-04-24T23:27:19.979600+0000","last_unstale":"2024-04-24T23:29:08.217118+0000","last_undegraded":"2024-04-24T23:29:08.217118+0000","last_fullsized":"2024-04-24T23:29:08.217118+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:37:52.250143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"92'10","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.221118+0000","last_change":"2024-04-24T23:27:19.984797+0000","last_active":"2024-04-24T23:29:08.221118+0000","last_peered":"2024-04-24T23:29:08.221118+0000","last_clean":"2024-04-24T23:29:08.221118+0000","last_became_active":"2024-04-24T23:27:19.983881+0000","last_became_peered":"2024-04-24T23:27:19.983881+0000","last_unstale":"2024-04-24T23:29:08.221118+0000","last_undegraded":"2024-04-24T23:29:08.221118+0000","last_fullsized":"2024-04-24T23:29:08.221118+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:17:24.630821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"92'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.214520+0000","last_change":"2024-04-24T23:27:19.977558+0000","last_active":"2024-04-24T23:29:08.214520+0000","last_peered":"2024-04-24T23:29:08.214520+0000","last_clean":"2024-04-24T23:29:08.214520+0000","last_became_active":"2024-04-24T23:27:19.977220+0000","last_became_peered":"2024-04-24T23:27:19.977220+0000","last_unstale":"2024-04-24T23:29:08.214520+0000","last_undegraded":"2024-04-24T23:29:08.214520+0000","last_fullsized":"2024-04-24T23:29:08.214520+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:38:49.615455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.214190+0000","last_change":"2024-04-24T23:27:24.234304+0000","last_active":"2024-04-24T23:29:08.214190+0000","last_peered":"2024-04-24T23:29:08.214190+0000","last_clean":"2024-04-24T23:29:08.214190+0000","last_became_active":"2024-04-24T23:27:24.233865+0000","last_became_peered":"2024-04-24T23:27:24.233865+0000","last_unstale":"2024-04-24T23:29:08.214190+0000","last_undegraded":"2024-04-24T23:29:08.214190+0000","last_fullsized":"2024-04-24T23:29:08.214190+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:46:20.853060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'9","reported_seq":60,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.221270+0000","last_change":"2024-04-24T23:28:08.061595+0000","last_active":"2024-04-24T23:29:08.221270+0000","last_peered":"2024-04-24T23:29:08.221270+0000","last_clean":"2024-04-24T23:29:08.221270+0000","last_became_active":"2024-04-24T23:28:08.053973+0000","last_became_peered":"2024-04-24T23:28:08.053973+0000","last_unstale":"2024-04-24T23:29:08.221270+0000","last_undegraded":"2024-04-24T23:29:08.221270+0000","last_fullsized":"2024-04-24T23:29:08.221270+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:57:55.667666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,8],"acting":[9,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"92'5","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.221020+0000","last_change":"2024-04-24T23:28:08.110963+0000","last_active":"2024-04-24T23:29:08.221020+0000","last_peered":"2024-04-24T23:29:08.221020+0000","last_clean":"2024-04-24T23:29:08.221020+0000","last_became_active":"2024-04-24T23:28:08.110633+0000","last_became_peered":"2024-04-24T23:28:08.110633+0000","last_unstale":"2024-04-24T23:29:08.221020+0000","last_undegraded":"2024-04-24T23:29:08.221020+0000","last_fullsized":"2024-04-24T23:29:08.221020+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:43:21.694477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.217862+0000","last_change":"2024-04-24T23:27:24.234134+0000","last_active":"2024-04-24T23:29:08.217862+0000","last_peered":"2024-04-24T23:29:08.217862+0000","last_clean":"2024-04-24T23:29:08.217862+0000","last_became_active":"2024-04-24T23:27:24.233650+0000","last_became_peered":"2024-04-24T23:27:24.233650+0000","last_unstale":"2024-04-24T23:29:08.217862+0000","last_undegraded":"2024-04-24T23:29:08.217862+0000","last_fullsized":"2024-04-24T23:29:08.217862+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:04:54.191186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.212945+0000","last_change":"2024-04-24T23:27:18.969572+0000","last_active":"2024-04-24T23:29:08.212945+0000","last_peered":"2024-04-24T23:29:08.212945+0000","last_clean":"2024-04-24T23:29:08.212945+0000","last_became_active":"2024-04-24T23:27:18.968897+0000","last_became_peered":"2024-04-24T23:27:18.968897+0000","last_unstale":"2024-04-24T23:29:08.212945+0000","last_undegraded":"2024-04-24T23:29:08.212945+0000","last_fullsized":"2024-04-24T23:29:08.212945+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:42:51.380386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"92'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.387791+0000","last_change":"2024-04-24T23:27:19.977290+0000","last_active":"2024-04-24T23:29:08.387791+0000","last_peered":"2024-04-24T23:29:08.387791+0000","last_clean":"2024-04-24T23:29:08.387791+0000","last_became_active":"2024-04-24T23:27:19.976802+0000","last_became_peered":"2024-04-24T23:27:19.976802+0000","last_unstale":"2024-04-24T23:29:08.387791+0000","last_undegraded":"2024-04-24T23:29:08.387791+0000","last_fullsized":"2024-04-24T23:29:08.387791+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:55:15.827279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.218034+0000","last_change":"2024-04-24T23:27:24.235706+0000","last_active":"2024-04-24T23:29:08.218034+0000","last_peered":"2024-04-24T23:29:08.218034+0000","last_clean":"2024-04-24T23:29:08.218034+0000","last_became_active":"2024-04-24T23:27:24.235367+0000","last_became_peered":"2024-04-24T23:27:24.235367+0000","last_unstale":"2024-04-24T23:29:08.218034+0000","last_undegraded":"2024-04-24T23:29:08.218034+0000","last_fullsized":"2024-04-24T23:29:08.218034+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:17.444845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.217153+0000","last_change":"2024-04-24T23:28:08.061398+0000","last_active":"2024-04-24T23:29:08.217153+0000","last_peered":"2024-04-24T23:29:08.217153+0000","last_clean":"2024-04-24T23:29:08.217153+0000","last_became_active":"2024-04-24T23:28:08.056810+0000","last_became_peered":"2024-04-24T23:28:08.056810+0000","last_unstale":"2024-04-24T23:29:08.217153+0000","last_undegraded":"2024-04-24T23:29:08.217153+0000","last_fullsized":"2024-04-24T23:29:08.217153+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:10:56.126730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"92'9","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.220961+0000","last_change":"2024-04-24T23:27:19.975222+0000","last_active":"2024-04-24T23:29:08.220961+0000","last_peered":"2024-04-24T23:29:08.220961+0000","last_clean":"2024-04-24T23:29:08.220961+0000","last_became_active":"2024-04-24T23:27:19.974881+0000","last_became_peered":"2024-04-24T23:27:19.974881+0000","last_unstale":"2024-04-24T23:29:08.220961+0000","last_undegraded":"2024-04-24T23:29:08.220961+0000","last_fullsized":"2024-04-24T23:29:08.220961+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:16:17.884748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.217945+0000","last_change":"2024-04-24T23:27:24.232445+0000","last_active":"2024-04-24T23:29:08.217945+0000","last_peered":"2024-04-24T23:29:08.217945+0000","last_clean":"2024-04-24T23:29:08.217945+0000","last_became_active":"2024-04-24T23:27:24.231754+0000","last_became_peered":"2024-04-24T23:27:24.231754+0000","last_unstale":"2024-04-24T23:29:08.217945+0000","last_undegraded":"2024-04-24T23:29:08.217945+0000","last_fullsized":"2024-04-24T23:29:08.217945+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:45:43.630508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.221798+0000","last_change":"2024-04-24T23:27:18.966558+0000","last_active":"2024-04-24T23:29:08.221798+0000","last_peered":"2024-04-24T23:29:08.221798+0000","last_clean":"2024-04-24T23:29:08.221798+0000","last_became_active":"2024-04-24T23:27:18.966212+0000","last_became_peered":"2024-04-24T23:27:18.966212+0000","last_unstale":"2024-04-24T23:29:08.221798+0000","last_undegraded":"2024-04-24T23:29:08.221798+0000","last_fullsized":"2024-04-24T23:29:08.221798+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:24:17.262587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"92'15","reported_seq":46,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.249544+0000","last_change":"2024-04-24T23:27:19.976811+0000","last_active":"2024-04-24T23:29:08.249544+0000","last_peered":"2024-04-24T23:29:08.249544+0000","last_clean":"2024-04-24T23:29:08.249544+0000","last_became_active":"2024-04-24T23:27:19.976456+0000","last_became_peered":"2024-04-24T23:27:19.976456+0000","last_unstale":"2024-04-24T23:29:08.249544+0000","last_undegraded":"2024-04-24T23:29:08.249544+0000","last_fullsized":"2024-04-24T23:29:08.249544+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:43:13.755833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.214292+0000","last_change":"2024-04-24T23:27:24.232136+0000","last_active":"2024-04-24T23:29:08.214292+0000","last_peered":"2024-04-24T23:29:08.214292+0000","last_clean":"2024-04-24T23:29:08.214292+0000","last_became_active":"2024-04-24T23:27:24.231459+0000","last_became_peered":"2024-04-24T23:27:24.231459+0000","last_unstale":"2024-04-24T23:29:08.214292+0000","last_undegraded":"2024-04-24T23:29:08.214292+0000","last_fullsized":"2024-04-24T23:29:08.214292+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:51:51.114787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"92'7","reported_seq":162,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:42.159846+0000","last_change":"2024-04-24T23:27:18.975333+0000","last_active":"2024-04-24T23:29:42.159846+0000","last_peered":"2024-04-24T23:29:42.159846+0000","last_clean":"2024-04-24T23:29:42.159846+0000","last_became_active":"2024-04-24T23:27:18.974869+0000","last_became_peered":"2024-04-24T23:27:18.974869+0000","last_unstale":"2024-04-24T23:29:42.159846+0000","last_undegraded":"2024-04-24T23:29:42.159846+0000","last_fullsized":"2024-04-24T23:29:42.159846+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:30:26.758596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":122,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"92'7","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.221237+0000","last_change":"2024-04-24T23:27:19.974989+0000","last_active":"2024-04-24T23:29:08.221237+0000","last_peered":"2024-04-24T23:29:08.221237+0000","last_clean":"2024-04-24T23:29:08.221237+0000","last_became_active":"2024-04-24T23:27:19.974312+0000","last_became_peered":"2024-04-24T23:27:19.974312+0000","last_unstale":"2024-04-24T23:29:08.221237+0000","last_undegraded":"2024-04-24T23:29:08.221237+0000","last_fullsized":"2024-04-24T23:29:08.221237+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:16:39.685430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.250906+0000","last_change":"2024-04-24T23:29:08.250906+0000","last_active":"2024-04-24T23:29:08.250906+0000","last_peered":"2024-04-24T23:29:08.250906+0000","last_clean":"2024-04-24T23:29:08.250906+0000","last_became_active":"2024-04-24T23:29:08.250264+0000","last_became_peered":"2024-04-24T23:29:08.250264+0000","last_unstale":"2024-04-24T23:29:08.250906+0000","last_undegraded":"2024-04-24T23:29:08.250906+0000","last_fullsized":"2024-04-24T23:29:08.250906+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:41:54.427428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.249568+0000","last_change":"2024-04-24T23:27:18.971313+0000","last_active":"2024-04-24T23:29:08.249568+0000","last_peered":"2024-04-24T23:29:08.249568+0000","last_clean":"2024-04-24T23:29:08.249568+0000","last_became_active":"2024-04-24T23:27:18.970888+0000","last_became_peered":"2024-04-24T23:27:18.970888+0000","last_unstale":"2024-04-24T23:29:08.249568+0000","last_undegraded":"2024-04-24T23:29:08.249568+0000","last_fullsized":"2024-04-24T23:29:08.249568+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:34:20.118693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"92'8","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.249768+0000","last_change":"2024-04-24T23:27:19.978939+0000","last_active":"2024-04-24T23:29:08.249768+0000","last_peered":"2024-04-24T23:29:08.249768+0000","last_clean":"2024-04-24T23:29:08.249768+0000","last_became_active":"2024-04-24T23:27:19.978556+0000","last_became_peered":"2024-04-24T23:27:19.978556+0000","last_unstale":"2024-04-24T23:29:08.249768+0000","last_undegraded":"2024-04-24T23:29:08.249768+0000","last_fullsized":"2024-04-24T23:29:08.249768+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:28:43.118561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.217032+0000","last_change":"2024-04-24T23:27:24.232384+0000","last_active":"2024-04-24T23:29:08.217032+0000","last_peered":"2024-04-24T23:29:08.217032+0000","last_clean":"2024-04-24T23:29:08.217032+0000","last_became_active":"2024-04-24T23:27:24.231497+0000","last_became_peered":"2024-04-24T23:27:24.231497+0000","last_unstale":"2024-04-24T23:29:08.217032+0000","last_undegraded":"2024-04-24T23:29:08.217032+0000","last_fullsized":"2024-04-24T23:29:08.217032+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:26:54.716821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.248992+0000","last_change":"2024-04-24T23:27:18.968949+0000","last_active":"2024-04-24T23:29:08.248992+0000","last_peered":"2024-04-24T23:29:08.248992+0000","last_clean":"2024-04-24T23:29:08.248992+0000","last_became_active":"2024-04-24T23:27:18.968612+0000","last_became_peered":"2024-04-24T23:27:18.968612+0000","last_unstale":"2024-04-24T23:29:08.248992+0000","last_undegraded":"2024-04-24T23:29:08.248992+0000","last_fullsized":"2024-04-24T23:29:08.248992+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:03:45.908080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"92'8","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.212368+0000","last_change":"2024-04-24T23:27:19.979670+0000","last_active":"2024-04-24T23:29:08.212368+0000","last_peered":"2024-04-24T23:29:08.212368+0000","last_clean":"2024-04-24T23:29:08.212368+0000","last_became_active":"2024-04-24T23:27:19.979277+0000","last_became_peered":"2024-04-24T23:27:19.979277+0000","last_unstale":"2024-04-24T23:29:08.212368+0000","last_undegraded":"2024-04-24T23:29:08.212368+0000","last_fullsized":"2024-04-24T23:29:08.212368+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:05:54.958747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.387084+0000","last_change":"2024-04-24T23:27:24.237085+0000","last_active":"2024-04-24T23:29:08.387084+0000","last_peered":"2024-04-24T23:29:08.387084+0000","last_clean":"2024-04-24T23:29:08.387084+0000","last_became_active":"2024-04-24T23:27:24.236671+0000","last_became_peered":"2024-04-24T23:27:24.236671+0000","last_unstale":"2024-04-24T23:29:08.387084+0000","last_undegraded":"2024-04-24T23:29:08.387084+0000","last_fullsized":"2024-04-24T23:29:08.387084+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:48:38.482374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.250147+0000","last_change":"2024-04-24T23:27:18.974146+0000","last_active":"2024-04-24T23:29:08.250147+0000","last_peered":"2024-04-24T23:29:08.250147+0000","last_clean":"2024-04-24T23:29:08.250147+0000","last_became_active":"2024-04-24T23:27:18.973814+0000","last_became_peered":"2024-04-24T23:27:18.973814+0000","last_unstale":"2024-04-24T23:29:08.250147+0000","last_undegraded":"2024-04-24T23:29:08.250147+0000","last_fullsized":"2024-04-24T23:29:08.250147+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:22:52.409448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"92'11","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.220653+0000","last_change":"2024-04-24T23:27:19.984325+0000","last_active":"2024-04-24T23:29:08.220653+0000","last_peered":"2024-04-24T23:29:08.220653+0000","last_clean":"2024-04-24T23:29:08.220653+0000","last_became_active":"2024-04-24T23:27:19.983918+0000","last_became_peered":"2024-04-24T23:27:19.983918+0000","last_unstale":"2024-04-24T23:29:08.220653+0000","last_undegraded":"2024-04-24T23:29:08.220653+0000","last_fullsized":"2024-04-24T23:29:08.220653+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:24:07.640983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.212601+0000","last_change":"2024-04-24T23:27:24.223561+0000","last_active":"2024-04-24T23:29:08.212601+0000","last_peered":"2024-04-24T23:29:08.212601+0000","last_clean":"2024-04-24T23:29:08.212601+0000","last_became_active":"2024-04-24T23:27:24.223127+0000","last_became_peered":"2024-04-24T23:27:24.223127+0000","last_unstale":"2024-04-24T23:29:08.212601+0000","last_undegraded":"2024-04-24T23:29:08.212601+0000","last_fullsized":"2024-04-24T23:29:08.212601+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:31:28.079633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"92'4","reported_seq":156,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:43.160939+0000","last_change":"2024-04-24T23:27:18.971374+0000","last_active":"2024-04-24T23:29:43.160939+0000","last_peered":"2024-04-24T23:29:43.160939+0000","last_clean":"2024-04-24T23:29:43.160939+0000","last_became_active":"2024-04-24T23:27:18.970818+0000","last_became_peered":"2024-04-24T23:27:18.970818+0000","last_unstale":"2024-04-24T23:29:43.160939+0000","last_undegraded":"2024-04-24T23:29:43.160939+0000","last_fullsized":"2024-04-24T23:29:43.160939+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:12:17.547968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":118,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"92'10","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.222062+0000","last_change":"2024-04-24T23:27:19.980275+0000","last_active":"2024-04-24T23:29:08.222062+0000","last_peered":"2024-04-24T23:29:08.222062+0000","last_clean":"2024-04-24T23:29:08.222062+0000","last_became_active":"2024-04-24T23:27:19.979925+0000","last_became_peered":"2024-04-24T23:27:19.979925+0000","last_unstale":"2024-04-24T23:29:08.222062+0000","last_undegraded":"2024-04-24T23:29:08.222062+0000","last_fullsized":"2024-04-24T23:29:08.222062+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:21:10.681169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.220570+0000","last_change":"2024-04-24T23:27:24.231389+0000","last_active":"2024-04-24T23:29:08.220570+0000","last_peered":"2024-04-24T23:29:08.220570+0000","last_clean":"2024-04-24T23:29:08.220570+0000","last_became_active":"2024-04-24T23:27:24.230973+0000","last_became_peered":"2024-04-24T23:27:24.230973+0000","last_unstale":"2024-04-24T23:29:08.220570+0000","last_undegraded":"2024-04-24T23:29:08.220570+0000","last_fullsized":"2024-04-24T23:29:08.220570+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:35.943122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.222447+0000","last_change":"2024-04-24T23:27:18.977009+0000","last_active":"2024-04-24T23:29:08.222447+0000","last_peered":"2024-04-24T23:29:08.222447+0000","last_clean":"2024-04-24T23:29:08.222447+0000","last_became_active":"2024-04-24T23:27:18.976667+0000","last_became_peered":"2024-04-24T23:27:18.976667+0000","last_unstale":"2024-04-24T23:29:08.222447+0000","last_undegraded":"2024-04-24T23:29:08.222447+0000","last_fullsized":"2024-04-24T23:29:08.222447+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:19:47.817961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"92'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.387836+0000","last_change":"2024-04-24T23:27:19.988589+0000","last_active":"2024-04-24T23:29:08.387836+0000","last_peered":"2024-04-24T23:29:08.387836+0000","last_clean":"2024-04-24T23:29:08.387836+0000","last_became_active":"2024-04-24T23:27:19.988261+0000","last_became_peered":"2024-04-24T23:27:19.988261+0000","last_unstale":"2024-04-24T23:29:08.387836+0000","last_undegraded":"2024-04-24T23:29:08.387836+0000","last_fullsized":"2024-04-24T23:29:08.387836+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:44:15.809776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.218303+0000","last_change":"2024-04-24T23:27:24.231256+0000","last_active":"2024-04-24T23:29:08.218303+0000","last_peered":"2024-04-24T23:29:08.218303+0000","last_clean":"2024-04-24T23:29:08.218303+0000","last_became_active":"2024-04-24T23:27:24.230545+0000","last_became_peered":"2024-04-24T23:27:24.230545+0000","last_unstale":"2024-04-24T23:29:08.218303+0000","last_undegraded":"2024-04-24T23:29:08.218303+0000","last_fullsized":"2024-04-24T23:29:08.218303+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:43:40.460473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.249368+0000","last_change":"2024-04-24T23:27:18.968805+0000","last_active":"2024-04-24T23:29:08.249368+0000","last_peered":"2024-04-24T23:29:08.249368+0000","last_clean":"2024-04-24T23:29:08.249368+0000","last_became_active":"2024-04-24T23:27:18.968423+0000","last_became_peered":"2024-04-24T23:27:18.968423+0000","last_unstale":"2024-04-24T23:29:08.249368+0000","last_undegraded":"2024-04-24T23:29:08.249368+0000","last_fullsized":"2024-04-24T23:29:08.249368+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:14:04.077773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"92'11","reported_seq":49,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.221213+0000","last_change":"2024-04-24T23:28:08.056901+0000","last_active":"2024-04-24T23:29:08.221213+0000","last_peered":"2024-04-24T23:29:08.221213+0000","last_clean":"2024-04-24T23:29:08.221213+0000","last_became_active":"2024-04-24T23:28:08.056422+0000","last_became_peered":"2024-04-24T23:28:08.056422+0000","last_unstale":"2024-04-24T23:29:08.221213+0000","last_undegraded":"2024-04-24T23:29:08.221213+0000","last_fullsized":"2024-04-24T23:29:08.221213+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:10:01.716479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.220583+0000","last_change":"2024-04-24T23:27:24.225481+0000","last_active":"2024-04-24T23:29:08.220583+0000","last_peered":"2024-04-24T23:29:08.220583+0000","last_clean":"2024-04-24T23:29:08.220583+0000","last_became_active":"2024-04-24T23:27:24.224363+0000","last_became_peered":"2024-04-24T23:27:24.224363+0000","last_unstale":"2024-04-24T23:29:08.220583+0000","last_undegraded":"2024-04-24T23:29:08.220583+0000","last_fullsized":"2024-04-24T23:29:08.220583+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:36:17.715596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"94'35","reported_seq":314,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:43.173813+0000","last_change":"2024-04-24T23:27:18.968264+0000","last_active":"2024-04-24T23:29:43.173813+0000","last_peered":"2024-04-24T23:29:43.173813+0000","last_clean":"2024-04-24T23:29:43.173813+0000","last_became_active":"2024-04-24T23:27:18.967856+0000","last_became_peered":"2024-04-24T23:27:18.967856+0000","last_unstale":"2024-04-24T23:29:43.173813+0000","last_undegraded":"2024-04-24T23:29:43.173813+0000","last_fullsized":"2024-04-24T23:29:43.173813+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:48:44.347254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59428,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":245,"num_read_kb":181,"num_write":36,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"92'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.222070+0000","last_change":"2024-04-24T23:27:19.984719+0000","last_active":"2024-04-24T23:29:08.222070+0000","last_peered":"2024-04-24T23:29:08.222070+0000","last_clean":"2024-04-24T23:29:08.222070+0000","last_became_active":"2024-04-24T23:27:19.984386+0000","last_became_peered":"2024-04-24T23:27:19.984386+0000","last_unstale":"2024-04-24T23:29:08.222070+0000","last_undegraded":"2024-04-24T23:29:08.222070+0000","last_fullsized":"2024-04-24T23:29:08.222070+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:02:13.739759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.248640+0000","last_change":"2024-04-24T23:27:24.230795+0000","last_active":"2024-04-24T23:29:08.248640+0000","last_peered":"2024-04-24T23:29:08.248640+0000","last_clean":"2024-04-24T23:29:08.248640+0000","last_became_active":"2024-04-24T23:27:24.230360+0000","last_became_peered":"2024-04-24T23:27:24.230360+0000","last_unstale":"2024-04-24T23:29:08.248640+0000","last_undegraded":"2024-04-24T23:29:08.248640+0000","last_fullsized":"2024-04-24T23:29:08.248640+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:51:24.875668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.214478+0000","last_change":"2024-04-24T23:27:18.964964+0000","last_active":"2024-04-24T23:29:08.214478+0000","last_peered":"2024-04-24T23:29:08.214478+0000","last_clean":"2024-04-24T23:29:08.214478+0000","last_became_active":"2024-04-24T23:27:18.964632+0000","last_became_peered":"2024-04-24T23:27:18.964632+0000","last_unstale":"2024-04-24T23:29:08.214478+0000","last_undegraded":"2024-04-24T23:29:08.214478+0000","last_fullsized":"2024-04-24T23:29:08.214478+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:52:23.144629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"92'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.249122+0000","last_change":"2024-04-24T23:27:19.987258+0000","last_active":"2024-04-24T23:29:08.249122+0000","last_peered":"2024-04-24T23:29:08.249122+0000","last_clean":"2024-04-24T23:29:08.249122+0000","last_became_active":"2024-04-24T23:27:19.986864+0000","last_became_peered":"2024-04-24T23:27:19.986864+0000","last_unstale":"2024-04-24T23:29:08.249122+0000","last_undegraded":"2024-04-24T23:29:08.249122+0000","last_fullsized":"2024-04-24T23:29:08.249122+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:57:40.503506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.251754+0000","last_change":"2024-04-24T23:29:08.251155+0000","last_active":"2024-04-24T23:29:08.251754+0000","last_peered":"2024-04-24T23:29:08.251754+0000","last_clean":"2024-04-24T23:29:08.251754+0000","last_became_active":"2024-04-24T23:29:08.250662+0000","last_became_peered":"2024-04-24T23:29:08.250662+0000","last_unstale":"2024-04-24T23:29:08.251754+0000","last_undegraded":"2024-04-24T23:29:08.251754+0000","last_fullsized":"2024-04-24T23:29:08.251754+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:32:01.403466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.249469+0000","last_change":"2024-04-24T23:27:18.974229+0000","last_active":"2024-04-24T23:29:08.249469+0000","last_peered":"2024-04-24T23:29:08.249469+0000","last_clean":"2024-04-24T23:29:08.249469+0000","last_became_active":"2024-04-24T23:27:18.973893+0000","last_became_peered":"2024-04-24T23:27:18.973893+0000","last_unstale":"2024-04-24T23:29:08.249469+0000","last_undegraded":"2024-04-24T23:29:08.249469+0000","last_fullsized":"2024-04-24T23:29:08.249469+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:41:45.699709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"92'15","reported_seq":47,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.221880+0000","last_change":"2024-04-24T23:27:19.986910+0000","last_active":"2024-04-24T23:29:08.221880+0000","last_peered":"2024-04-24T23:29:08.221880+0000","last_clean":"2024-04-24T23:29:08.221880+0000","last_became_active":"2024-04-24T23:27:19.986556+0000","last_became_peered":"2024-04-24T23:27:19.986556+0000","last_unstale":"2024-04-24T23:29:08.221880+0000","last_undegraded":"2024-04-24T23:29:08.221880+0000","last_fullsized":"2024-04-24T23:29:08.221880+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:06:28.506556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.221054+0000","last_change":"2024-04-24T23:27:24.233115+0000","last_active":"2024-04-24T23:29:08.221054+0000","last_peered":"2024-04-24T23:29:08.221054+0000","last_clean":"2024-04-24T23:29:08.221054+0000","last_became_active":"2024-04-24T23:27:24.232714+0000","last_became_peered":"2024-04-24T23:27:24.232714+0000","last_unstale":"2024-04-24T23:29:08.221054+0000","last_undegraded":"2024-04-24T23:29:08.221054+0000","last_fullsized":"2024-04-24T23:29:08.221054+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:04.365515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"94'6","reported_seq":163,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:42.159532+0000","last_change":"2024-04-24T23:27:18.966426+0000","last_active":"2024-04-24T23:29:42.159532+0000","last_peered":"2024-04-24T23:29:42.159532+0000","last_clean":"2024-04-24T23:29:42.159532+0000","last_became_active":"2024-04-24T23:27:18.966070+0000","last_became_peered":"2024-04-24T23:27:18.966070+0000","last_unstale":"2024-04-24T23:29:42.159532+0000","last_undegraded":"2024-04-24T23:29:42.159532+0000","last_fullsized":"2024-04-24T23:29:42.159532+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:56:21.679639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1428,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":124,"num_read_kb":6,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"92'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.221340+0000","last_change":"2024-04-24T23:27:19.991829+0000","last_active":"2024-04-24T23:29:08.221340+0000","last_peered":"2024-04-24T23:29:08.221340+0000","last_clean":"2024-04-24T23:29:08.221340+0000","last_became_active":"2024-04-24T23:27:19.991492+0000","last_became_peered":"2024-04-24T23:27:19.991492+0000","last_unstale":"2024-04-24T23:29:08.221340+0000","last_undegraded":"2024-04-24T23:29:08.221340+0000","last_fullsized":"2024-04-24T23:29:08.221340+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:23:45.195217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.248500+0000","last_change":"2024-04-24T23:27:24.229794+0000","last_active":"2024-04-24T23:29:08.248500+0000","last_peered":"2024-04-24T23:29:08.248500+0000","last_clean":"2024-04-24T23:29:08.248500+0000","last_became_active":"2024-04-24T23:27:24.229410+0000","last_became_peered":"2024-04-24T23:27:24.229410+0000","last_unstale":"2024-04-24T23:29:08.248500+0000","last_undegraded":"2024-04-24T23:29:08.248500+0000","last_fullsized":"2024-04-24T23:29:08.248500+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:00:53.188307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":74,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.221241+0000","last_change":"2024-04-24T23:28:08.068516+0000","last_active":"2024-04-24T23:29:08.221241+0000","last_peered":"2024-04-24T23:29:08.221241+0000","last_clean":"2024-04-24T23:29:08.221241+0000","last_became_active":"2024-04-24T23:28:08.056740+0000","last_became_peered":"2024-04-24T23:28:08.056740+0000","last_unstale":"2024-04-24T23:29:08.221241+0000","last_undegraded":"2024-04-24T23:29:08.221241+0000","last_fullsized":"2024-04-24T23:29:08.221241+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:47:35.568377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,1],"acting":[8,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"92'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.249623+0000","last_change":"2024-04-24T23:27:19.982608+0000","last_active":"2024-04-24T23:29:08.249623+0000","last_peered":"2024-04-24T23:29:08.249623+0000","last_clean":"2024-04-24T23:29:08.249623+0000","last_became_active":"2024-04-24T23:27:19.982270+0000","last_became_peered":"2024-04-24T23:27:19.982270+0000","last_unstale":"2024-04-24T23:29:08.249623+0000","last_undegraded":"2024-04-24T23:29:08.249623+0000","last_fullsized":"2024-04-24T23:29:08.249623+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:01:10.525164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.248615+0000","last_change":"2024-04-24T23:27:24.236143+0000","last_active":"2024-04-24T23:29:08.248615+0000","last_peered":"2024-04-24T23:29:08.248615+0000","last_clean":"2024-04-24T23:29:08.248615+0000","last_became_active":"2024-04-24T23:27:24.235724+0000","last_became_peered":"2024-04-24T23:27:24.235724+0000","last_unstale":"2024-04-24T23:29:08.248615+0000","last_undegraded":"2024-04-24T23:29:08.248615+0000","last_fullsized":"2024-04-24T23:29:08.248615+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:55:41.317044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"94'3","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:36.689756+0000","last_change":"2024-04-24T23:27:18.971312+0000","last_active":"2024-04-24T23:29:36.689756+0000","last_peered":"2024-04-24T23:29:36.689756+0000","last_clean":"2024-04-24T23:29:36.689756+0000","last_became_active":"2024-04-24T23:27:18.970929+0000","last_became_peered":"2024-04-24T23:27:18.970929+0000","last_unstale":"2024-04-24T23:29:36.689756+0000","last_undegraded":"2024-04-24T23:29:36.689756+0000","last_fullsized":"2024-04-24T23:29:36.689756+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:54:20.318916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"92'5","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.217096+0000","last_change":"2024-04-24T23:28:08.056700+0000","last_active":"2024-04-24T23:29:08.217096+0000","last_peered":"2024-04-24T23:29:08.217096+0000","last_clean":"2024-04-24T23:29:08.217096+0000","last_became_active":"2024-04-24T23:28:08.056271+0000","last_became_peered":"2024-04-24T23:28:08.056271+0000","last_unstale":"2024-04-24T23:29:08.217096+0000","last_undegraded":"2024-04-24T23:29:08.217096+0000","last_fullsized":"2024-04-24T23:29:08.217096+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:03:34.808028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.221630+0000","last_change":"2024-04-24T23:27:24.718948+0000","last_active":"2024-04-24T23:29:08.221630+0000","last_peered":"2024-04-24T23:29:08.221630+0000","last_clean":"2024-04-24T23:29:08.221630+0000","last_became_active":"2024-04-24T23:27:24.718424+0000","last_became_peered":"2024-04-24T23:27:24.718424+0000","last_unstale":"2024-04-24T23:29:08.221630+0000","last_undegraded":"2024-04-24T23:29:08.221630+0000","last_fullsized":"2024-04-24T23:29:08.221630+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:22:01.984976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.220935+0000","last_change":"2024-04-24T23:27:18.957073+0000","last_active":"2024-04-24T23:29:08.220935+0000","last_peered":"2024-04-24T23:29:08.220935+0000","last_clean":"2024-04-24T23:29:08.220935+0000","last_became_active":"2024-04-24T23:27:18.956713+0000","last_became_peered":"2024-04-24T23:27:18.956713+0000","last_unstale":"2024-04-24T23:29:08.220935+0000","last_undegraded":"2024-04-24T23:29:08.220935+0000","last_fullsized":"2024-04-24T23:29:08.220935+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:16:21.015902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"92'4","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.249932+0000","last_change":"2024-04-24T23:27:19.985058+0000","last_active":"2024-04-24T23:29:08.249932+0000","last_peered":"2024-04-24T23:29:08.249932+0000","last_clean":"2024-04-24T23:29:08.249932+0000","last_became_active":"2024-04-24T23:27:19.984675+0000","last_became_peered":"2024-04-24T23:27:19.984675+0000","last_unstale":"2024-04-24T23:29:08.249932+0000","last_undegraded":"2024-04-24T23:29:08.249932+0000","last_fullsized":"2024-04-24T23:29:08.249932+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:22:35.252839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.248925+0000","last_change":"2024-04-24T23:27:24.236117+0000","last_active":"2024-04-24T23:29:08.248925+0000","last_peered":"2024-04-24T23:29:08.248925+0000","last_clean":"2024-04-24T23:29:08.248925+0000","last_became_active":"2024-04-24T23:27:24.235720+0000","last_became_peered":"2024-04-24T23:27:24.235720+0000","last_unstale":"2024-04-24T23:29:08.248925+0000","last_undegraded":"2024-04-24T23:29:08.248925+0000","last_fullsized":"2024-04-24T23:29:08.248925+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:46:40.604988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.221627+0000","last_change":"2024-04-24T23:27:18.973690+0000","last_active":"2024-04-24T23:29:08.221627+0000","last_peered":"2024-04-24T23:29:08.221627+0000","last_clean":"2024-04-24T23:29:08.221627+0000","last_became_active":"2024-04-24T23:27:18.973182+0000","last_became_peered":"2024-04-24T23:27:18.973182+0000","last_unstale":"2024-04-24T23:29:08.221627+0000","last_undegraded":"2024-04-24T23:29:08.221627+0000","last_fullsized":"2024-04-24T23:29:08.221627+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:00:44.809880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"92'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.249631+0000","last_change":"2024-04-24T23:27:19.991252+0000","last_active":"2024-04-24T23:29:08.249631+0000","last_peered":"2024-04-24T23:29:08.249631+0000","last_clean":"2024-04-24T23:29:08.249631+0000","last_became_active":"2024-04-24T23:27:19.990892+0000","last_became_peered":"2024-04-24T23:27:19.990892+0000","last_unstale":"2024-04-24T23:29:08.249631+0000","last_undegraded":"2024-04-24T23:29:08.249631+0000","last_fullsized":"2024-04-24T23:29:08.249631+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:29:37.835521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.249473+0000","last_change":"2024-04-24T23:27:24.718929+0000","last_active":"2024-04-24T23:29:08.249473+0000","last_peered":"2024-04-24T23:29:08.249473+0000","last_clean":"2024-04-24T23:29:08.249473+0000","last_became_active":"2024-04-24T23:27:24.718486+0000","last_became_peered":"2024-04-24T23:27:24.718486+0000","last_unstale":"2024-04-24T23:29:08.249473+0000","last_undegraded":"2024-04-24T23:29:08.249473+0000","last_fullsized":"2024-04-24T23:29:08.249473+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:31:13.359829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"92'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.249896+0000","last_change":"2024-04-24T23:27:19.983181+0000","last_active":"2024-04-24T23:29:08.249896+0000","last_peered":"2024-04-24T23:29:08.249896+0000","last_clean":"2024-04-24T23:29:08.249896+0000","last_became_active":"2024-04-24T23:27:19.981764+0000","last_became_peered":"2024-04-24T23:27:19.981764+0000","last_unstale":"2024-04-24T23:29:08.249896+0000","last_undegraded":"2024-04-24T23:29:08.249896+0000","last_fullsized":"2024-04-24T23:29:08.249896+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:01:20.653572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"94'6","reported_seq":159,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:43.160323+0000","last_change":"2024-04-24T23:27:18.976351+0000","last_active":"2024-04-24T23:29:43.160323+0000","last_peered":"2024-04-24T23:29:43.160323+0000","last_clean":"2024-04-24T23:29:43.160323+0000","last_became_active":"2024-04-24T23:27:18.975552+0000","last_became_peered":"2024-04-24T23:27:18.975552+0000","last_unstale":"2024-04-24T23:29:43.160323+0000","last_undegraded":"2024-04-24T23:29:43.160323+0000","last_fullsized":"2024-04-24T23:29:43.160323+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:26:29.551439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":119,"num_read_kb":119,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":27,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.222548+0000","last_change":"2024-04-24T23:27:24.229528+0000","last_active":"2024-04-24T23:29:08.222548+0000","last_peered":"2024-04-24T23:29:08.222548+0000","last_clean":"2024-04-24T23:29:08.222548+0000","last_became_active":"2024-04-24T23:27:24.229099+0000","last_became_peered":"2024-04-24T23:27:24.229099+0000","last_unstale":"2024-04-24T23:29:08.222548+0000","last_undegraded":"2024-04-24T23:29:08.222548+0000","last_fullsized":"2024-04-24T23:29:08.222548+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:51:51.421912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"92'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.212843+0000","last_change":"2024-04-24T23:27:19.980094+0000","last_active":"2024-04-24T23:29:08.212843+0000","last_peered":"2024-04-24T23:29:08.212843+0000","last_clean":"2024-04-24T23:29:08.212843+0000","last_became_active":"2024-04-24T23:27:19.979710+0000","last_became_peered":"2024-04-24T23:27:19.979710+0000","last_unstale":"2024-04-24T23:29:08.212843+0000","last_undegraded":"2024-04-24T23:29:08.212843+0000","last_fullsized":"2024-04-24T23:29:08.212843+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:50:37.920235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.218213+0000","last_change":"2024-04-24T23:27:18.963224+0000","last_active":"2024-04-24T23:29:08.218213+0000","last_peered":"2024-04-24T23:29:08.218213+0000","last_clean":"2024-04-24T23:29:08.218213+0000","last_became_active":"2024-04-24T23:27:18.962794+0000","last_became_peered":"2024-04-24T23:27:18.962794+0000","last_unstale":"2024-04-24T23:29:08.218213+0000","last_undegraded":"2024-04-24T23:29:08.218213+0000","last_fullsized":"2024-04-24T23:29:08.218213+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:05:36.869062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":358,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.220962+0000","last_change":"2024-04-24T23:26:02.307599+0000","last_active":"2024-04-24T23:29:08.220962+0000","last_peered":"2024-04-24T23:29:08.220962+0000","last_clean":"2024-04-24T23:29:08.220962+0000","last_became_active":"2024-04-24T23:26:02.307186+0000","last_became_peered":"2024-04-24T23:26:02.307186+0000","last_unstale":"2024-04-24T23:29:08.220962+0000","last_undegraded":"2024-04-24T23:29:08.220962+0000","last_fullsized":"2024-04-24T23:29:08.220962+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:22:33.824274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:22:33.824274+0000","last_clean_scrub_stamp":"2024-04-24T23:22:33.824274+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:19:26.789955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.249319+0000","last_change":"2024-04-24T23:27:24.719315+0000","last_active":"2024-04-24T23:29:08.249319+0000","last_peered":"2024-04-24T23:29:08.249319+0000","last_clean":"2024-04-24T23:29:08.249319+0000","last_became_active":"2024-04-24T23:27:24.718868+0000","last_became_peered":"2024-04-24T23:27:24.718868+0000","last_unstale":"2024-04-24T23:29:08.249319+0000","last_undegraded":"2024-04-24T23:29:08.249319+0000","last_fullsized":"2024-04-24T23:29:08.249319+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:43:32.122873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"92'5","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.248863+0000","last_change":"2024-04-24T23:27:19.983711+0000","last_active":"2024-04-24T23:29:08.248863+0000","last_peered":"2024-04-24T23:29:08.248863+0000","last_clean":"2024-04-24T23:29:08.248863+0000","last_became_active":"2024-04-24T23:27:19.983302+0000","last_became_peered":"2024-04-24T23:27:19.983302+0000","last_unstale":"2024-04-24T23:29:08.248863+0000","last_undegraded":"2024-04-24T23:29:08.248863+0000","last_fullsized":"2024-04-24T23:29:08.248863+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:44:49.251679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.221222+0000","last_change":"2024-04-24T23:27:18.976766+0000","last_active":"2024-04-24T23:29:08.221222+0000","last_peered":"2024-04-24T23:29:08.221222+0000","last_clean":"2024-04-24T23:29:08.221222+0000","last_became_active":"2024-04-24T23:27:18.976420+0000","last_became_peered":"2024-04-24T23:27:18.976420+0000","last_unstale":"2024-04-24T23:29:08.221222+0000","last_undegraded":"2024-04-24T23:29:08.221222+0000","last_fullsized":"2024-04-24T23:29:08.221222+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:59:34.262774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.216819+0000","last_change":"2024-04-24T23:27:24.226979+0000","last_active":"2024-04-24T23:29:08.216819+0000","last_peered":"2024-04-24T23:29:08.216819+0000","last_clean":"2024-04-24T23:29:08.216819+0000","last_became_active":"2024-04-24T23:27:24.226596+0000","last_became_peered":"2024-04-24T23:27:24.226596+0000","last_unstale":"2024-04-24T23:29:08.216819+0000","last_undegraded":"2024-04-24T23:29:08.216819+0000","last_fullsized":"2024-04-24T23:29:08.216819+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:50:48.585067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.222082+0000","last_change":"2024-04-24T23:27:18.970811+0000","last_active":"2024-04-24T23:29:08.222082+0000","last_peered":"2024-04-24T23:29:08.222082+0000","last_clean":"2024-04-24T23:29:08.222082+0000","last_became_active":"2024-04-24T23:27:18.970292+0000","last_became_peered":"2024-04-24T23:27:18.970292+0000","last_unstale":"2024-04-24T23:29:08.222082+0000","last_undegraded":"2024-04-24T23:29:08.222082+0000","last_fullsized":"2024-04-24T23:29:08.222082+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:06:37.556527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"92'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.249285+0000","last_change":"2024-04-24T23:27:19.975455+0000","last_active":"2024-04-24T23:29:08.249285+0000","last_peered":"2024-04-24T23:29:08.249285+0000","last_clean":"2024-04-24T23:29:08.249285+0000","last_became_active":"2024-04-24T23:27:19.973504+0000","last_became_peered":"2024-04-24T23:27:19.973504+0000","last_unstale":"2024-04-24T23:29:08.249285+0000","last_undegraded":"2024-04-24T23:29:08.249285+0000","last_fullsized":"2024-04-24T23:29:08.249285+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:50:34.450370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.212709+0000","last_change":"2024-04-24T23:27:24.719253+0000","last_active":"2024-04-24T23:29:08.212709+0000","last_peered":"2024-04-24T23:29:08.212709+0000","last_clean":"2024-04-24T23:29:08.212709+0000","last_became_active":"2024-04-24T23:27:24.718357+0000","last_became_peered":"2024-04-24T23:27:24.718357+0000","last_unstale":"2024-04-24T23:29:08.212709+0000","last_undegraded":"2024-04-24T23:29:08.212709+0000","last_fullsized":"2024-04-24T23:29:08.212709+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:31:03.798037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"94'10","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:28.012182+0000","last_change":"2024-04-24T23:27:18.974310+0000","last_active":"2024-04-24T23:29:28.012182+0000","last_peered":"2024-04-24T23:29:28.012182+0000","last_clean":"2024-04-24T23:29:28.012182+0000","last_became_active":"2024-04-24T23:27:18.973956+0000","last_became_peered":"2024-04-24T23:27:18.973956+0000","last_unstale":"2024-04-24T23:29:28.012182+0000","last_undegraded":"2024-04-24T23:29:28.012182+0000","last_fullsized":"2024-04-24T23:29:28.012182+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:44:27.233155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"92'5","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.213085+0000","last_change":"2024-04-24T23:27:19.985763+0000","last_active":"2024-04-24T23:29:08.213085+0000","last_peered":"2024-04-24T23:29:08.213085+0000","last_clean":"2024-04-24T23:29:08.213085+0000","last_became_active":"2024-04-24T23:27:19.985413+0000","last_became_peered":"2024-04-24T23:27:19.985413+0000","last_unstale":"2024-04-24T23:29:08.213085+0000","last_undegraded":"2024-04-24T23:29:08.213085+0000","last_fullsized":"2024-04-24T23:29:08.213085+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:04:54.910407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.214033+0000","last_change":"2024-04-24T23:27:24.225615+0000","last_active":"2024-04-24T23:29:08.214033+0000","last_peered":"2024-04-24T23:29:08.214033+0000","last_clean":"2024-04-24T23:29:08.214033+0000","last_became_active":"2024-04-24T23:27:24.225029+0000","last_became_peered":"2024-04-24T23:27:24.225029+0000","last_unstale":"2024-04-24T23:29:08.214033+0000","last_undegraded":"2024-04-24T23:29:08.214033+0000","last_fullsized":"2024-04-24T23:29:08.214033+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:56:02.956669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.218561+0000","last_change":"2024-04-24T23:27:18.963235+0000","last_active":"2024-04-24T23:29:08.218561+0000","last_peered":"2024-04-24T23:29:08.218561+0000","last_clean":"2024-04-24T23:29:08.218561+0000","last_became_active":"2024-04-24T23:27:18.962835+0000","last_became_peered":"2024-04-24T23:27:18.962835+0000","last_unstale":"2024-04-24T23:29:08.218561+0000","last_undegraded":"2024-04-24T23:29:08.218561+0000","last_fullsized":"2024-04-24T23:29:08.218561+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:34:29.376856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"92'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.387790+0000","last_change":"2024-04-24T23:27:19.987386+0000","last_active":"2024-04-24T23:29:08.387790+0000","last_peered":"2024-04-24T23:29:08.387790+0000","last_clean":"2024-04-24T23:29:08.387790+0000","last_became_active":"2024-04-24T23:27:19.987003+0000","last_became_peered":"2024-04-24T23:27:19.987003+0000","last_unstale":"2024-04-24T23:29:08.387790+0000","last_undegraded":"2024-04-24T23:29:08.387790+0000","last_fullsized":"2024-04-24T23:29:08.387790+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:20:25.635732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.218484+0000","last_change":"2024-04-24T23:27:24.719291+0000","last_active":"2024-04-24T23:29:08.218484+0000","last_peered":"2024-04-24T23:29:08.218484+0000","last_clean":"2024-04-24T23:29:08.218484+0000","last_became_active":"2024-04-24T23:27:24.718796+0000","last_became_peered":"2024-04-24T23:27:24.718796+0000","last_unstale":"2024-04-24T23:29:08.218484+0000","last_undegraded":"2024-04-24T23:29:08.218484+0000","last_fullsized":"2024-04-24T23:29:08.218484+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:20:25.135902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.220641+0000","last_change":"2024-04-24T23:27:18.967856+0000","last_active":"2024-04-24T23:29:08.220641+0000","last_peered":"2024-04-24T23:29:08.220641+0000","last_clean":"2024-04-24T23:29:08.220641+0000","last_became_active":"2024-04-24T23:27:18.967497+0000","last_became_peered":"2024-04-24T23:27:18.967497+0000","last_unstale":"2024-04-24T23:29:08.220641+0000","last_undegraded":"2024-04-24T23:29:08.220641+0000","last_fullsized":"2024-04-24T23:29:08.220641+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:21:29.841791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"92'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.221174+0000","last_change":"2024-04-24T23:27:19.981302+0000","last_active":"2024-04-24T23:29:08.221174+0000","last_peered":"2024-04-24T23:29:08.221174+0000","last_clean":"2024-04-24T23:29:08.221174+0000","last_became_active":"2024-04-24T23:27:19.980938+0000","last_became_peered":"2024-04-24T23:27:19.980938+0000","last_unstale":"2024-04-24T23:29:08.221174+0000","last_undegraded":"2024-04-24T23:29:08.221174+0000","last_fullsized":"2024-04-24T23:29:08.221174+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:24:53.939882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.221231+0000","last_change":"2024-04-24T23:27:24.224626+0000","last_active":"2024-04-24T23:29:08.221231+0000","last_peered":"2024-04-24T23:29:08.221231+0000","last_clean":"2024-04-24T23:29:08.221231+0000","last_became_active":"2024-04-24T23:27:24.224175+0000","last_became_peered":"2024-04-24T23:27:24.224175+0000","last_unstale":"2024-04-24T23:29:08.221231+0000","last_undegraded":"2024-04-24T23:29:08.221231+0000","last_fullsized":"2024-04-24T23:29:08.221231+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:54:44.155637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"90'3","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.212460+0000","last_change":"2024-04-24T23:27:18.975662+0000","last_active":"2024-04-24T23:29:08.212460+0000","last_peered":"2024-04-24T23:29:08.212460+0000","last_clean":"2024-04-24T23:29:08.212460+0000","last_became_active":"2024-04-24T23:27:18.975326+0000","last_became_peered":"2024-04-24T23:27:18.975326+0000","last_unstale":"2024-04-24T23:29:08.212460+0000","last_undegraded":"2024-04-24T23:29:08.212460+0000","last_fullsized":"2024-04-24T23:29:08.212460+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:39.440866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"92'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.216748+0000","last_change":"2024-04-24T23:27:19.978818+0000","last_active":"2024-04-24T23:29:08.216748+0000","last_peered":"2024-04-24T23:29:08.216748+0000","last_clean":"2024-04-24T23:29:08.216748+0000","last_became_active":"2024-04-24T23:27:19.978107+0000","last_became_peered":"2024-04-24T23:27:19.978107+0000","last_unstale":"2024-04-24T23:29:08.216748+0000","last_undegraded":"2024-04-24T23:29:08.216748+0000","last_fullsized":"2024-04-24T23:29:08.216748+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:22:25.797996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.255331+0000","last_change":"2024-04-24T23:29:08.254854+0000","last_active":"2024-04-24T23:29:08.255331+0000","last_peered":"2024-04-24T23:29:08.255331+0000","last_clean":"2024-04-24T23:29:08.255331+0000","last_became_active":"2024-04-24T23:29:08.250700+0000","last_became_peered":"2024-04-24T23:29:08.250700+0000","last_unstale":"2024-04-24T23:29:08.255331+0000","last_undegraded":"2024-04-24T23:29:08.255331+0000","last_fullsized":"2024-04-24T23:29:08.255331+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:25:31.400400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'1","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.214173+0000","last_change":"2024-04-24T23:27:18.967165+0000","last_active":"2024-04-24T23:29:08.214173+0000","last_peered":"2024-04-24T23:29:08.214173+0000","last_clean":"2024-04-24T23:29:08.214173+0000","last_became_active":"2024-04-24T23:27:18.966628+0000","last_became_peered":"2024-04-24T23:27:18.966628+0000","last_unstale":"2024-04-24T23:29:08.214173+0000","last_undegraded":"2024-04-24T23:29:08.214173+0000","last_fullsized":"2024-04-24T23:29:08.214173+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:12:20.520777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"92'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.212534+0000","last_change":"2024-04-24T23:27:19.988069+0000","last_active":"2024-04-24T23:29:08.212534+0000","last_peered":"2024-04-24T23:29:08.212534+0000","last_clean":"2024-04-24T23:29:08.212534+0000","last_became_active":"2024-04-24T23:27:19.987660+0000","last_became_peered":"2024-04-24T23:27:19.987660+0000","last_unstale":"2024-04-24T23:29:08.212534+0000","last_undegraded":"2024-04-24T23:29:08.212534+0000","last_fullsized":"2024-04-24T23:29:08.212534+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:21:03.505313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.218455+0000","last_change":"2024-04-24T23:27:24.719041+0000","last_active":"2024-04-24T23:29:08.218455+0000","last_peered":"2024-04-24T23:29:08.218455+0000","last_clean":"2024-04-24T23:29:08.218455+0000","last_became_active":"2024-04-24T23:27:24.718523+0000","last_became_peered":"2024-04-24T23:27:24.718523+0000","last_unstale":"2024-04-24T23:29:08.218455+0000","last_undegraded":"2024-04-24T23:29:08.218455+0000","last_fullsized":"2024-04-24T23:29:08.218455+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:27:57.396800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.221356+0000","last_change":"2024-04-24T23:27:18.975034+0000","last_active":"2024-04-24T23:29:08.221356+0000","last_peered":"2024-04-24T23:29:08.221356+0000","last_clean":"2024-04-24T23:29:08.221356+0000","last_became_active":"2024-04-24T23:27:18.973764+0000","last_became_peered":"2024-04-24T23:27:18.973764+0000","last_unstale":"2024-04-24T23:29:08.221356+0000","last_undegraded":"2024-04-24T23:29:08.221356+0000","last_fullsized":"2024-04-24T23:29:08.221356+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:43:23.451904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"92'5","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.221861+0000","last_change":"2024-04-24T23:27:19.984948+0000","last_active":"2024-04-24T23:29:08.221861+0000","last_peered":"2024-04-24T23:29:08.221861+0000","last_clean":"2024-04-24T23:29:08.221861+0000","last_became_active":"2024-04-24T23:27:19.984484+0000","last_became_peered":"2024-04-24T23:27:19.984484+0000","last_unstale":"2024-04-24T23:29:08.221861+0000","last_undegraded":"2024-04-24T23:29:08.221861+0000","last_fullsized":"2024-04-24T23:29:08.221861+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:17:43.335983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.251471+0000","last_change":"2024-04-24T23:29:08.250873+0000","last_active":"2024-04-24T23:29:08.251471+0000","last_peered":"2024-04-24T23:29:08.251471+0000","last_clean":"2024-04-24T23:29:08.251471+0000","last_became_active":"2024-04-24T23:29:08.250147+0000","last_became_peered":"2024-04-24T23:29:08.250147+0000","last_unstale":"2024-04-24T23:29:08.251471+0000","last_undegraded":"2024-04-24T23:29:08.251471+0000","last_fullsized":"2024-04-24T23:29:08.251471+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:14:42.358098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6,7],"acting":[1,8,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.214665+0000","last_change":"2024-04-24T23:27:18.964232+0000","last_active":"2024-04-24T23:29:08.214665+0000","last_peered":"2024-04-24T23:29:08.214665+0000","last_clean":"2024-04-24T23:29:08.214665+0000","last_became_active":"2024-04-24T23:27:18.963855+0000","last_became_peered":"2024-04-24T23:27:18.963855+0000","last_unstale":"2024-04-24T23:29:08.214665+0000","last_undegraded":"2024-04-24T23:29:08.214665+0000","last_fullsized":"2024-04-24T23:29:08.214665+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:02:21.000326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"92'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.222653+0000","last_change":"2024-04-24T23:27:19.984354+0000","last_active":"2024-04-24T23:29:08.222653+0000","last_peered":"2024-04-24T23:29:08.222653+0000","last_clean":"2024-04-24T23:29:08.222653+0000","last_became_active":"2024-04-24T23:27:19.983972+0000","last_became_peered":"2024-04-24T23:27:19.983972+0000","last_unstale":"2024-04-24T23:29:08.222653+0000","last_undegraded":"2024-04-24T23:29:08.222653+0000","last_fullsized":"2024-04-24T23:29:08.222653+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:28:28.390283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.387070+0000","last_change":"2024-04-24T23:27:24.238404+0000","last_active":"2024-04-24T23:29:08.387070+0000","last_peered":"2024-04-24T23:29:08.387070+0000","last_clean":"2024-04-24T23:29:08.387070+0000","last_became_active":"2024-04-24T23:27:24.237976+0000","last_became_peered":"2024-04-24T23:27:24.237976+0000","last_unstale":"2024-04-24T23:29:08.387070+0000","last_undegraded":"2024-04-24T23:29:08.387070+0000","last_fullsized":"2024-04-24T23:29:08.387070+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:19:31.476789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.249641+0000","last_change":"2024-04-24T23:27:18.971259+0000","last_active":"2024-04-24T23:29:08.249641+0000","last_peered":"2024-04-24T23:29:08.249641+0000","last_clean":"2024-04-24T23:29:08.249641+0000","last_became_active":"2024-04-24T23:27:18.970856+0000","last_became_peered":"2024-04-24T23:27:18.970856+0000","last_unstale":"2024-04-24T23:29:08.249641+0000","last_undegraded":"2024-04-24T23:29:08.249641+0000","last_fullsized":"2024-04-24T23:29:08.249641+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:54:58.642528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"92'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.222288+0000","last_change":"2024-04-24T23:27:19.986320+0000","last_active":"2024-04-24T23:29:08.222288+0000","last_peered":"2024-04-24T23:29:08.222288+0000","last_clean":"2024-04-24T23:29:08.222288+0000","last_became_active":"2024-04-24T23:27:19.985951+0000","last_became_peered":"2024-04-24T23:27:19.985951+0000","last_unstale":"2024-04-24T23:29:08.222288+0000","last_undegraded":"2024-04-24T23:29:08.222288+0000","last_fullsized":"2024-04-24T23:29:08.222288+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:05:35.427735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.248604+0000","last_change":"2024-04-24T23:27:24.228149+0000","last_active":"2024-04-24T23:29:08.248604+0000","last_peered":"2024-04-24T23:29:08.248604+0000","last_clean":"2024-04-24T23:29:08.248604+0000","last_became_active":"2024-04-24T23:27:24.227683+0000","last_became_peered":"2024-04-24T23:27:24.227683+0000","last_unstale":"2024-04-24T23:29:08.248604+0000","last_undegraded":"2024-04-24T23:29:08.248604+0000","last_fullsized":"2024-04-24T23:29:08.248604+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:55:50.408834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":46,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.249519+0000","last_change":"2024-04-24T23:28:08.059374+0000","last_active":"2024-04-24T23:29:08.249519+0000","last_peered":"2024-04-24T23:29:08.249519+0000","last_clean":"2024-04-24T23:29:08.249519+0000","last_became_active":"2024-04-24T23:28:08.056104+0000","last_became_peered":"2024-04-24T23:28:08.056104+0000","last_unstale":"2024-04-24T23:29:08.249519+0000","last_undegraded":"2024-04-24T23:29:08.249519+0000","last_fullsized":"2024-04-24T23:29:08.249519+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:07:17.185948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"92'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.222499+0000","last_change":"2024-04-24T23:27:19.990935+0000","last_active":"2024-04-24T23:29:08.222499+0000","last_peered":"2024-04-24T23:29:08.222499+0000","last_clean":"2024-04-24T23:29:08.222499+0000","last_became_active":"2024-04-24T23:27:19.990524+0000","last_became_peered":"2024-04-24T23:27:19.990524+0000","last_unstale":"2024-04-24T23:29:08.222499+0000","last_undegraded":"2024-04-24T23:29:08.222499+0000","last_fullsized":"2024-04-24T23:29:08.222499+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:54:02.463248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"92'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.216879+0000","last_change":"2024-04-24T23:27:19.979142+0000","last_active":"2024-04-24T23:29:08.216879+0000","last_peered":"2024-04-24T23:29:08.216879+0000","last_clean":"2024-04-24T23:29:08.216879+0000","last_became_active":"2024-04-24T23:27:19.978799+0000","last_became_peered":"2024-04-24T23:27:19.978799+0000","last_unstale":"2024-04-24T23:29:08.216879+0000","last_undegraded":"2024-04-24T23:29:08.216879+0000","last_fullsized":"2024-04-24T23:29:08.216879+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:07:19.046313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"92'12","reported_seq":50,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.214206+0000","last_change":"2024-04-24T23:28:08.109820+0000","last_active":"2024-04-24T23:29:08.214206+0000","last_peered":"2024-04-24T23:29:08.214206+0000","last_clean":"2024-04-24T23:29:08.214206+0000","last_became_active":"2024-04-24T23:28:08.109500+0000","last_became_peered":"2024-04-24T23:28:08.109500+0000","last_unstale":"2024-04-24T23:29:08.214206+0000","last_undegraded":"2024-04-24T23:29:08.214206+0000","last_fullsized":"2024-04-24T23:29:08.214206+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:34:32.814256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"92'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.214320+0000","last_change":"2024-04-24T23:27:19.980008+0000","last_active":"2024-04-24T23:29:08.214320+0000","last_peered":"2024-04-24T23:29:08.214320+0000","last_clean":"2024-04-24T23:29:08.214320+0000","last_became_active":"2024-04-24T23:27:19.979424+0000","last_became_peered":"2024-04-24T23:27:19.979424+0000","last_unstale":"2024-04-24T23:29:08.214320+0000","last_undegraded":"2024-04-24T23:29:08.214320+0000","last_fullsized":"2024-04-24T23:29:08.214320+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:44.449245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"92'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.387685+0000","last_change":"2024-04-24T23:27:19.987574+0000","last_active":"2024-04-24T23:29:08.387685+0000","last_peered":"2024-04-24T23:29:08.387685+0000","last_clean":"2024-04-24T23:29:08.387685+0000","last_became_active":"2024-04-24T23:27:19.987242+0000","last_became_peered":"2024-04-24T23:27:19.987242+0000","last_unstale":"2024-04-24T23:29:08.387685+0000","last_undegraded":"2024-04-24T23:29:08.387685+0000","last_fullsized":"2024-04-24T23:29:08.387685+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:39:10.777152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"92'4","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.249324+0000","last_change":"2024-04-24T23:27:19.980983+0000","last_active":"2024-04-24T23:29:08.249324+0000","last_peered":"2024-04-24T23:29:08.249324+0000","last_clean":"2024-04-24T23:29:08.249324+0000","last_became_active":"2024-04-24T23:27:19.980659+0000","last_became_peered":"2024-04-24T23:27:19.980659+0000","last_unstale":"2024-04-24T23:29:08.249324+0000","last_undegraded":"2024-04-24T23:29:08.249324+0000","last_fullsized":"2024-04-24T23:29:08.249324+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:02:52.647687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"92'6","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.217260+0000","last_change":"2024-04-24T23:28:08.055819+0000","last_active":"2024-04-24T23:29:08.217260+0000","last_peered":"2024-04-24T23:29:08.217260+0000","last_clean":"2024-04-24T23:29:08.217260+0000","last_became_active":"2024-04-24T23:28:08.055386+0000","last_became_peered":"2024-04-24T23:28:08.055386+0000","last_unstale":"2024-04-24T23:29:08.217260+0000","last_undegraded":"2024-04-24T23:29:08.217260+0000","last_fullsized":"2024-04-24T23:29:08.217260+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:59:30.488812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"92'15","reported_seq":47,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.214979+0000","last_change":"2024-04-24T23:27:19.985864+0000","last_active":"2024-04-24T23:29:08.214979+0000","last_peered":"2024-04-24T23:29:08.214979+0000","last_clean":"2024-04-24T23:29:08.214979+0000","last_became_active":"2024-04-24T23:27:19.984877+0000","last_became_peered":"2024-04-24T23:27:19.984877+0000","last_unstale":"2024-04-24T23:29:08.214979+0000","last_undegraded":"2024-04-24T23:29:08.214979+0000","last_fullsized":"2024-04-24T23:29:08.214979+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:14:43.458970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"92'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.213153+0000","last_change":"2024-04-24T23:27:19.988831+0000","last_active":"2024-04-24T23:29:08.213153+0000","last_peered":"2024-04-24T23:29:08.213153+0000","last_clean":"2024-04-24T23:29:08.213153+0000","last_became_active":"2024-04-24T23:27:19.988478+0000","last_became_peered":"2024-04-24T23:27:19.988478+0000","last_unstale":"2024-04-24T23:29:08.213153+0000","last_undegraded":"2024-04-24T23:29:08.213153+0000","last_fullsized":"2024-04-24T23:29:08.213153+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:18:37.488710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"92'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.249789+0000","last_change":"2024-04-24T23:27:19.983553+0000","last_active":"2024-04-24T23:29:08.249789+0000","last_peered":"2024-04-24T23:29:08.249789+0000","last_clean":"2024-04-24T23:29:08.249789+0000","last_became_active":"2024-04-24T23:27:19.981042+0000","last_became_peered":"2024-04-24T23:27:19.981042+0000","last_unstale":"2024-04-24T23:29:08.249789+0000","last_undegraded":"2024-04-24T23:29:08.249789+0000","last_fullsized":"2024-04-24T23:29:08.249789+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:09:42.608238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"92'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.212461+0000","last_change":"2024-04-24T23:27:19.985425+0000","last_active":"2024-04-24T23:29:08.212461+0000","last_peered":"2024-04-24T23:29:08.212461+0000","last_clean":"2024-04-24T23:29:08.212461+0000","last_became_active":"2024-04-24T23:27:19.985094+0000","last_became_peered":"2024-04-24T23:27:19.985094+0000","last_unstale":"2024-04-24T23:29:08.212461+0000","last_undegraded":"2024-04-24T23:29:08.212461+0000","last_fullsized":"2024-04-24T23:29:08.212461+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:07:15.612247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"92'10","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.220939+0000","last_change":"2024-04-24T23:27:19.988224+0000","last_active":"2024-04-24T23:29:08.220939+0000","last_peered":"2024-04-24T23:29:08.220939+0000","last_clean":"2024-04-24T23:29:08.220939+0000","last_became_active":"2024-04-24T23:27:19.987820+0000","last_became_peered":"2024-04-24T23:27:19.987820+0000","last_unstale":"2024-04-24T23:29:08.220939+0000","last_undegraded":"2024-04-24T23:29:08.220939+0000","last_fullsized":"2024-04-24T23:29:08.220939+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:27.653290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"92'11","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.249349+0000","last_change":"2024-04-24T23:27:19.989672+0000","last_active":"2024-04-24T23:29:08.249349+0000","last_peered":"2024-04-24T23:29:08.249349+0000","last_clean":"2024-04-24T23:29:08.249349+0000","last_became_active":"2024-04-24T23:27:19.989337+0000","last_became_peered":"2024-04-24T23:27:19.989337+0000","last_unstale":"2024-04-24T23:29:08.249349+0000","last_undegraded":"2024-04-24T23:29:08.249349+0000","last_fullsized":"2024-04-24T23:29:08.249349+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:07:13.399366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"92'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.216974+0000","last_change":"2024-04-24T23:27:19.987703+0000","last_active":"2024-04-24T23:29:08.216974+0000","last_peered":"2024-04-24T23:29:08.216974+0000","last_clean":"2024-04-24T23:29:08.216974+0000","last_became_active":"2024-04-24T23:27:19.987355+0000","last_became_peered":"2024-04-24T23:27:19.987355+0000","last_unstale":"2024-04-24T23:29:08.216974+0000","last_undegraded":"2024-04-24T23:29:08.216974+0000","last_fullsized":"2024-04-24T23:29:08.216974+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:03:15.812726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":280134,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":64410,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":741,"num_read_kb":441,"num_write":176,"num_write_kb":124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":694,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":193230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":152,"ondisk_log_size":152,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383724,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1036288,"data_stored":529776,"data_compressed":29657,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645360,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28168,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716984,"statfs":{"total":95995035648,"available":95966191616,"internally_reserved":0,"allocated":1179648,"data_stored":1180826,"data_compressed":65288,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808886,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32208,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712944,"statfs":{"total":95995035648,"available":95962054656,"internally_reserved":0,"allocated":1048576,"data_stored":529430,"data_compressed":29657,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070524,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1048576,"data_stored":529350,"data_compressed":29657,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332160,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28088,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717064,"statfs":{"total":95995035648,"available":95966273536,"internally_reserved":0,"allocated":1093632,"data_stored":1125658,"data_compressed":65288,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495686,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1028096,"data_stored":524606,"data_compressed":29657,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574943,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36412,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708740,"statfs":{"total":95995035648,"available":95957749760,"internally_reserved":0,"allocated":1101824,"data_stored":584056,"data_compressed":29657,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738467,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1052672,"data_stored":525994,"data_compressed":29657,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411418,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32260,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712892,"statfs":{"total":95995035648,"available":95962001408,"internally_reserved":0,"allocated":1110016,"data_stored":585714,"data_compressed":29657,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182485,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1040384,"data_stored":524800,"data_compressed":29657,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920848,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1040384,"data_stored":530096,"data_compressed":29657,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757323,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1105920,"data_stored":1120988,"data_compressed":65288,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33993,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33993,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33993,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35631,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35631,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35631,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35631,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35631,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":59540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":61198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":61846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T23:29:47.074 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-04-24T23:29:47.074 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T23:29:47.211 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.207+0000 7fd8fa0ee640 1 -- 172.21.15.80:0/3727678165 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd8f4100690 msgr2=0x7fd8f4100ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:47.211 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.207+0000 7fd8fa0ee640 1 --2- 172.21.15.80:0/3727678165 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd8f4100690 0x7fd8f4100ab0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fd8e4009900 tx=0x7fd8e402a410 comp rx=0 tx=0).stop 2024-04-24T23:29:47.211 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.207+0000 7fd8fa0ee640 1 -- 172.21.15.80:0/3727678165 shutdown_connections 2024-04-24T23:29:47.211 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.207+0000 7fd8fa0ee640 1 --2- 172.21.15.80:0/3727678165 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd8f4102240 0x7fd8f4108ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:47.211 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.207+0000 7fd8fa0ee640 1 --2- 172.21.15.80:0/3727678165 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8f4101880 0x7fd8f4101d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:47.211 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.207+0000 7fd8fa0ee640 1 --2- 172.21.15.80:0/3727678165 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd8f4100690 0x7fd8f4100ab0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:47.211 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.207+0000 7fd8fa0ee640 1 -- 172.21.15.80:0/3727678165 >> 172.21.15.80:0/3727678165 conn(0x7fd8f40fbc20 msgr2=0x7fd8f40fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:47.211 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.207+0000 7fd8fa0ee640 1 -- 172.21.15.80:0/3727678165 shutdown_connections 2024-04-24T23:29:47.211 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.207+0000 7fd8fa0ee640 1 -- 172.21.15.80:0/3727678165 wait complete. 2024-04-24T23:29:47.212 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.208+0000 7fd8fa0ee640 1 Processor -- start 2024-04-24T23:29:47.212 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.208+0000 7fd8fa0ee640 1 -- start start 2024-04-24T23:29:47.212 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.208+0000 7fd8fa0ee640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd8f4100690 0x7fd8f4197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:47.212 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.208+0000 7fd8fa0ee640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8f4101880 0x7fd8f4198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:47.212 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.208+0000 7fd8fa0ee640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd8f4102240 0x7fd8f419f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:47.212 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.208+0000 7fd8fa0ee640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fd8f410d430 con 0x7fd8f4102240 2024-04-24T23:29:47.212 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.208+0000 7fd8fa0ee640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd8f410d2b0 con 0x7fd8f4100690 2024-04-24T23:29:47.212 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.208+0000 7fd8fa0ee640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd8f410d5b0 con 0x7fd8f4101880 2024-04-24T23:29:47.212 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.208+0000 7fd8f3fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd8f4102240 0x7fd8f419f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:47.212 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.208+0000 7fd8f3fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd8f4102240 0x7fd8f419f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:52048/0 (socket says 172.21.15.80:52048) 2024-04-24T23:29:47.212 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.208+0000 7fd8f3fff640 1 -- 172.21.15.80:0/3944823030 learned_addr learned my addr 172.21.15.80:0/3944823030 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:29:47.213 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.208+0000 7fd8f2ffd640 1 --2- 172.21.15.80:0/3944823030 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8f4101880 0x7fd8f4198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:47.213 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.208+0000 7fd8f37fe640 1 --2- 172.21.15.80:0/3944823030 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd8f4100690 0x7fd8f4197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:47.213 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.209+0000 7fd8f3fff640 1 -- 172.21.15.80:0/3944823030 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8f4101880 msgr2=0x7fd8f4198030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:47.213 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.209+0000 7fd8f3fff640 1 --2- 172.21.15.80:0/3944823030 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8f4101880 0x7fd8f4198030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:47.213 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.209+0000 7fd8f3fff640 1 -- 172.21.15.80:0/3944823030 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd8f4100690 msgr2=0x7fd8f4197af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:47.213 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.209+0000 7fd8f3fff640 1 --2- 172.21.15.80:0/3944823030 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd8f4100690 0x7fd8f4197af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:47.213 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.209+0000 7fd8f3fff640 1 -- 172.21.15.80:0/3944823030 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8f419f710 con 0x7fd8f4102240 2024-04-24T23:29:47.213 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.209+0000 7fd8f3fff640 1 --2- 172.21.15.80:0/3944823030 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd8f4102240 0x7fd8f419f0a0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fd8ec009fd0 tx=0x7fd8ec00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:47.213 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.209+0000 7fd8f0ff9640 1 -- 172.21.15.80:0/3944823030 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8ec017070 con 0x7fd8f4102240 2024-04-24T23:29:47.214 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.209+0000 7fd8fa0ee640 1 -- 172.21.15.80:0/3944823030 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8f419fa00 con 0x7fd8f4102240 2024-04-24T23:29:47.214 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.209+0000 7fd8f0ff9640 1 -- 172.21.15.80:0/3944823030 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd8ec003e50 con 0x7fd8f4102240 2024-04-24T23:29:47.214 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.209+0000 7fd8f0ff9640 1 -- 172.21.15.80:0/3944823030 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8ec0106c0 con 0x7fd8f4102240 2024-04-24T23:29:47.214 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.209+0000 7fd8fa0ee640 1 -- 172.21.15.80:0/3944823030 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8f419ff00 con 0x7fd8f4102240 2024-04-24T23:29:47.215 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.210+0000 7fd8fa0ee640 1 -- 172.21.15.80:0/3944823030 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8f4107a30 con 0x7fd8f4102240 2024-04-24T23:29:47.215 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.211+0000 7fd8f0ff9640 1 -- 172.21.15.80:0/3944823030 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd8ec003970 con 0x7fd8f4102240 2024-04-24T23:29:47.216 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.212+0000 7fd8f0ff9640 1 --2- 172.21.15.80:0/3944823030 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fd8dc0a2b90 0x7fd8dc0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:47.216 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.212+0000 7fd8f37fe640 1 --2- 172.21.15.80:0/3944823030 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fd8dc0a2b90 0x7fd8dc0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:47.216 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.212+0000 7fd8f0ff9640 1 -- 172.21.15.80:0/3944823030 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fd8ec0cf120 con 0x7fd8f4102240 2024-04-24T23:29:47.216 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.212+0000 7fd8f37fe640 1 --2- 172.21.15.80:0/3944823030 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fd8dc0a2b90 0x7fd8dc0a5050 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd8e4005ab0 tx=0x7fd8e4000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:47.217 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.216+0000 7fd8f0ff9640 1 -- 172.21.15.80:0/3944823030 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fd8ec09b660 con 0x7fd8f4102240 2024-04-24T23:29:47.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:29:47 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/3621144815' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T23:29:47.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:29:47 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/905843497' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:29:47.389 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.388+0000 7fd8fa0ee640 1 -- 172.21.15.80:0/3944823030 --> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd8f4061b80 con 0x7fd8dc0a2b90 2024-04-24T23:29:47.398 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.397+0000 7fd8f0ff9640 1 -- 172.21.15.80:0/3944823030 <== mgr.14152 v2:172.21.15.80:6800/3830596922 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340774 (secure 0 0 0) 0x7fd8f4061b80 con 0x7fd8dc0a2b90 2024-04-24T23:29:47.402 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-24T23:29:47.403 INFO:teuthology.orchestra.run.smithi080.stderr:dumped all 2024-04-24T23:29:47.405 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.401+0000 7fd8fa0ee640 1 -- 172.21.15.80:0/3944823030 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fd8dc0a2b90 msgr2=0x7fd8dc0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:47.405 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.401+0000 7fd8fa0ee640 1 --2- 172.21.15.80:0/3944823030 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fd8dc0a2b90 0x7fd8dc0a5050 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd8e4005ab0 tx=0x7fd8e4000bc0 comp rx=0 tx=0).stop 2024-04-24T23:29:47.405 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.401+0000 7fd8fa0ee640 1 -- 172.21.15.80:0/3944823030 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd8f4102240 msgr2=0x7fd8f419f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:47.405 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.401+0000 7fd8fa0ee640 1 --2- 172.21.15.80:0/3944823030 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd8f4102240 0x7fd8f419f0a0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fd8ec009fd0 tx=0x7fd8ec00b260 comp rx=0 tx=0).stop 2024-04-24T23:29:47.405 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.401+0000 7fd8fa0ee640 1 -- 172.21.15.80:0/3944823030 shutdown_connections 2024-04-24T23:29:47.405 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.401+0000 7fd8fa0ee640 1 --2- 172.21.15.80:0/3944823030 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fd8dc0a2b90 0x7fd8dc0a5050 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:47.405 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.401+0000 7fd8fa0ee640 1 --2- 172.21.15.80:0/3944823030 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd8f4102240 0x7fd8f419f0a0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:47.405 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.401+0000 7fd8fa0ee640 1 --2- 172.21.15.80:0/3944823030 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8f4101880 0x7fd8f4198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:47.406 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.401+0000 7fd8fa0ee640 1 --2- 172.21.15.80:0/3944823030 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd8f4100690 0x7fd8f4197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:47.406 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.401+0000 7fd8fa0ee640 1 -- 172.21.15.80:0/3944823030 >> 172.21.15.80:0/3944823030 conn(0x7fd8f40fbc20 msgr2=0x7fd8f4104c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:47.406 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.401+0000 7fd8fa0ee640 1 -- 172.21.15.80:0/3944823030 shutdown_connections 2024-04-24T23:29:47.406 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.401+0000 7fd8fa0ee640 1 -- 172.21.15.80:0/3944823030 wait complete. 2024-04-24T23:29:47.414 INFO:teuthology.orchestra.run.smithi080.stdout:{"pg_ready":true,"pg_map":{"version":374,"stamp":"2024-04-24T23:29:46.919868+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655012,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":867,"num_read_kb":550,"num_write":359,"num_write_kb":3064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":1181547,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":791,"ondisk_log_size":791,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":365932,"kb_used_data":12584,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124575892,"statfs":{"total":1151940427776,"available":1151565713408,"internally_reserved":0,"allocated":12886016,"data_stored":8291294,"data_compressed":462777,"data_compressed_allocated":626688,"data_compressed_original":2752512,"omap_allocated":19141,"internal_metadata":361674043},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":616,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":62,"num_read_kb":32,"num_write":8,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003273"},"pg_stats":[{"pgid":"3.3f","version":"92'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.217305+0000","last_change":"2024-04-24T23:27:19.978807+0000","last_active":"2024-04-24T23:29:08.217305+0000","last_peered":"2024-04-24T23:29:08.217305+0000","last_clean":"2024-04-24T23:29:08.217305+0000","last_became_active":"2024-04-24T23:27:19.978436+0000","last_became_peered":"2024-04-24T23:27:19.978436+0000","last_unstale":"2024-04-24T23:29:08.217305+0000","last_undegraded":"2024-04-24T23:29:08.217305+0000","last_fullsized":"2024-04-24T23:29:08.217305+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:00:13.126518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"92'4","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.387809+0000","last_change":"2024-04-24T23:27:19.980327+0000","last_active":"2024-04-24T23:29:08.387809+0000","last_peered":"2024-04-24T23:29:08.387809+0000","last_clean":"2024-04-24T23:29:08.387809+0000","last_became_active":"2024-04-24T23:27:19.979912+0000","last_became_peered":"2024-04-24T23:27:19.979912+0000","last_unstale":"2024-04-24T23:29:08.387809+0000","last_undegraded":"2024-04-24T23:29:08.387809+0000","last_fullsized":"2024-04-24T23:29:08.387809+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:25:41.657205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"92'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.387723+0000","last_change":"2024-04-24T23:27:19.976451+0000","last_active":"2024-04-24T23:29:08.387723+0000","last_peered":"2024-04-24T23:29:08.387723+0000","last_clean":"2024-04-24T23:29:08.387723+0000","last_became_active":"2024-04-24T23:27:19.975976+0000","last_became_peered":"2024-04-24T23:27:19.975976+0000","last_unstale":"2024-04-24T23:29:08.387723+0000","last_undegraded":"2024-04-24T23:29:08.387723+0000","last_fullsized":"2024-04-24T23:29:08.387723+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:14:51.605217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"92'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.218709+0000","last_change":"2024-04-24T23:27:19.991245+0000","last_active":"2024-04-24T23:29:08.218709+0000","last_peered":"2024-04-24T23:29:08.218709+0000","last_clean":"2024-04-24T23:29:08.218709+0000","last_became_active":"2024-04-24T23:27:19.990448+0000","last_became_peered":"2024-04-24T23:27:19.990448+0000","last_unstale":"2024-04-24T23:29:08.218709+0000","last_undegraded":"2024-04-24T23:29:08.218709+0000","last_fullsized":"2024-04-24T23:29:08.218709+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:03:21.013005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"92'4","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.221241+0000","last_change":"2024-04-24T23:27:19.977664+0000","last_active":"2024-04-24T23:29:08.221241+0000","last_peered":"2024-04-24T23:29:08.221241+0000","last_clean":"2024-04-24T23:29:08.221241+0000","last_became_active":"2024-04-24T23:27:19.977300+0000","last_became_peered":"2024-04-24T23:27:19.977300+0000","last_unstale":"2024-04-24T23:29:08.221241+0000","last_undegraded":"2024-04-24T23:29:08.221241+0000","last_fullsized":"2024-04-24T23:29:08.221241+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:19:27.229712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"92'6","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.249580+0000","last_change":"2024-04-24T23:27:19.989994+0000","last_active":"2024-04-24T23:29:08.249580+0000","last_peered":"2024-04-24T23:29:08.249580+0000","last_clean":"2024-04-24T23:29:08.249580+0000","last_became_active":"2024-04-24T23:27:19.989639+0000","last_became_peered":"2024-04-24T23:27:19.989639+0000","last_unstale":"2024-04-24T23:29:08.249580+0000","last_undegraded":"2024-04-24T23:29:08.249580+0000","last_fullsized":"2024-04-24T23:29:08.249580+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:20:43.931086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"92'5","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.221180+0000","last_change":"2024-04-24T23:27:19.992886+0000","last_active":"2024-04-24T23:29:08.221180+0000","last_peered":"2024-04-24T23:29:08.221180+0000","last_clean":"2024-04-24T23:29:08.221180+0000","last_became_active":"2024-04-24T23:27:19.992438+0000","last_became_peered":"2024-04-24T23:27:19.992438+0000","last_unstale":"2024-04-24T23:29:08.221180+0000","last_undegraded":"2024-04-24T23:29:08.221180+0000","last_fullsized":"2024-04-24T23:29:08.221180+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:02:34.110984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"92'9","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.212229+0000","last_change":"2024-04-24T23:27:19.981592+0000","last_active":"2024-04-24T23:29:08.212229+0000","last_peered":"2024-04-24T23:29:08.212229+0000","last_clean":"2024-04-24T23:29:08.212229+0000","last_became_active":"2024-04-24T23:27:19.981248+0000","last_became_peered":"2024-04-24T23:27:19.981248+0000","last_unstale":"2024-04-24T23:29:08.212229+0000","last_undegraded":"2024-04-24T23:29:08.212229+0000","last_fullsized":"2024-04-24T23:29:08.212229+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:53:31.191318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"92'11","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.221670+0000","last_change":"2024-04-24T23:27:19.975995+0000","last_active":"2024-04-24T23:29:08.221670+0000","last_peered":"2024-04-24T23:29:08.221670+0000","last_clean":"2024-04-24T23:29:08.221670+0000","last_became_active":"2024-04-24T23:27:19.975614+0000","last_became_peered":"2024-04-24T23:27:19.975614+0000","last_unstale":"2024-04-24T23:29:08.221670+0000","last_undegraded":"2024-04-24T23:29:08.221670+0000","last_fullsized":"2024-04-24T23:29:08.221670+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:33:22.439998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"92'16","reported_seq":47,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.250021+0000","last_change":"2024-04-24T23:27:19.991265+0000","last_active":"2024-04-24T23:29:08.250021+0000","last_peered":"2024-04-24T23:29:08.250021+0000","last_clean":"2024-04-24T23:29:08.250021+0000","last_became_active":"2024-04-24T23:27:19.990928+0000","last_became_peered":"2024-04-24T23:27:19.990928+0000","last_unstale":"2024-04-24T23:29:08.250021+0000","last_undegraded":"2024-04-24T23:29:08.250021+0000","last_fullsized":"2024-04-24T23:29:08.250021+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:20:41.244669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"92'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.222538+0000","last_change":"2024-04-24T23:27:19.985182+0000","last_active":"2024-04-24T23:29:08.222538+0000","last_peered":"2024-04-24T23:29:08.222538+0000","last_clean":"2024-04-24T23:29:08.222538+0000","last_became_active":"2024-04-24T23:27:19.984600+0000","last_became_peered":"2024-04-24T23:27:19.984600+0000","last_unstale":"2024-04-24T23:29:08.222538+0000","last_undegraded":"2024-04-24T23:29:08.222538+0000","last_fullsized":"2024-04-24T23:29:08.222538+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:16:09.919736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"92'7","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.218365+0000","last_change":"2024-04-24T23:27:19.994271+0000","last_active":"2024-04-24T23:29:08.218365+0000","last_peered":"2024-04-24T23:29:08.218365+0000","last_clean":"2024-04-24T23:29:08.218365+0000","last_became_active":"2024-04-24T23:27:19.993926+0000","last_became_peered":"2024-04-24T23:27:19.993926+0000","last_unstale":"2024-04-24T23:29:08.218365+0000","last_undegraded":"2024-04-24T23:29:08.218365+0000","last_fullsized":"2024-04-24T23:29:08.218365+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:49:07.500370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"92'10","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.249303+0000","last_change":"2024-04-24T23:27:19.984230+0000","last_active":"2024-04-24T23:29:08.249303+0000","last_peered":"2024-04-24T23:29:08.249303+0000","last_clean":"2024-04-24T23:29:08.249303+0000","last_became_active":"2024-04-24T23:27:19.983902+0000","last_became_peered":"2024-04-24T23:27:19.983902+0000","last_unstale":"2024-04-24T23:29:08.249303+0000","last_undegraded":"2024-04-24T23:29:08.249303+0000","last_fullsized":"2024-04-24T23:29:08.249303+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:12:40.985092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"92'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.387496+0000","last_change":"2024-04-24T23:27:19.979822+0000","last_active":"2024-04-24T23:29:08.387496+0000","last_peered":"2024-04-24T23:29:08.387496+0000","last_clean":"2024-04-24T23:29:08.387496+0000","last_became_active":"2024-04-24T23:27:19.979452+0000","last_became_peered":"2024-04-24T23:27:19.979452+0000","last_unstale":"2024-04-24T23:29:08.387496+0000","last_undegraded":"2024-04-24T23:29:08.387496+0000","last_fullsized":"2024-04-24T23:29:08.387496+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:08:53.945221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"92'7","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.387472+0000","last_change":"2024-04-24T23:28:08.053470+0000","last_active":"2024-04-24T23:29:08.387472+0000","last_peered":"2024-04-24T23:29:08.387472+0000","last_clean":"2024-04-24T23:29:08.387472+0000","last_became_active":"2024-04-24T23:28:08.053064+0000","last_became_peered":"2024-04-24T23:28:08.053064+0000","last_unstale":"2024-04-24T23:29:08.387472+0000","last_undegraded":"2024-04-24T23:29:08.387472+0000","last_fullsized":"2024-04-24T23:29:08.387472+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:13:34.489239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"92'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.220541+0000","last_change":"2024-04-24T23:27:19.984346+0000","last_active":"2024-04-24T23:29:08.220541+0000","last_peered":"2024-04-24T23:29:08.220541+0000","last_clean":"2024-04-24T23:29:08.220541+0000","last_became_active":"2024-04-24T23:27:19.983999+0000","last_became_peered":"2024-04-24T23:27:19.983999+0000","last_unstale":"2024-04-24T23:29:08.220541+0000","last_undegraded":"2024-04-24T23:29:08.220541+0000","last_fullsized":"2024-04-24T23:29:08.220541+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:15:35.329454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"92'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.217118+0000","last_change":"2024-04-24T23:27:19.979939+0000","last_active":"2024-04-24T23:29:08.217118+0000","last_peered":"2024-04-24T23:29:08.217118+0000","last_clean":"2024-04-24T23:29:08.217118+0000","last_became_active":"2024-04-24T23:27:19.979600+0000","last_became_peered":"2024-04-24T23:27:19.979600+0000","last_unstale":"2024-04-24T23:29:08.217118+0000","last_undegraded":"2024-04-24T23:29:08.217118+0000","last_fullsized":"2024-04-24T23:29:08.217118+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:37:52.250143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"92'10","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.221118+0000","last_change":"2024-04-24T23:27:19.984797+0000","last_active":"2024-04-24T23:29:08.221118+0000","last_peered":"2024-04-24T23:29:08.221118+0000","last_clean":"2024-04-24T23:29:08.221118+0000","last_became_active":"2024-04-24T23:27:19.983881+0000","last_became_peered":"2024-04-24T23:27:19.983881+0000","last_unstale":"2024-04-24T23:29:08.221118+0000","last_undegraded":"2024-04-24T23:29:08.221118+0000","last_fullsized":"2024-04-24T23:29:08.221118+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:17:24.630821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"92'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.214520+0000","last_change":"2024-04-24T23:27:19.977558+0000","last_active":"2024-04-24T23:29:08.214520+0000","last_peered":"2024-04-24T23:29:08.214520+0000","last_clean":"2024-04-24T23:29:08.214520+0000","last_became_active":"2024-04-24T23:27:19.977220+0000","last_became_peered":"2024-04-24T23:27:19.977220+0000","last_unstale":"2024-04-24T23:29:08.214520+0000","last_undegraded":"2024-04-24T23:29:08.214520+0000","last_fullsized":"2024-04-24T23:29:08.214520+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:38:49.615455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.214190+0000","last_change":"2024-04-24T23:27:24.234304+0000","last_active":"2024-04-24T23:29:08.214190+0000","last_peered":"2024-04-24T23:29:08.214190+0000","last_clean":"2024-04-24T23:29:08.214190+0000","last_became_active":"2024-04-24T23:27:24.233865+0000","last_became_peered":"2024-04-24T23:27:24.233865+0000","last_unstale":"2024-04-24T23:29:08.214190+0000","last_undegraded":"2024-04-24T23:29:08.214190+0000","last_fullsized":"2024-04-24T23:29:08.214190+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:46:20.853060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'9","reported_seq":60,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.221270+0000","last_change":"2024-04-24T23:28:08.061595+0000","last_active":"2024-04-24T23:29:08.221270+0000","last_peered":"2024-04-24T23:29:08.221270+0000","last_clean":"2024-04-24T23:29:08.221270+0000","last_became_active":"2024-04-24T23:28:08.053973+0000","last_became_peered":"2024-04-24T23:28:08.053973+0000","last_unstale":"2024-04-24T23:29:08.221270+0000","last_undegraded":"2024-04-24T23:29:08.221270+0000","last_fullsized":"2024-04-24T23:29:08.221270+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:57:55.667666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,8],"acting":[9,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"92'5","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.221020+0000","last_change":"2024-04-24T23:28:08.110963+0000","last_active":"2024-04-24T23:29:08.221020+0000","last_peered":"2024-04-24T23:29:08.221020+0000","last_clean":"2024-04-24T23:29:08.221020+0000","last_became_active":"2024-04-24T23:28:08.110633+0000","last_became_peered":"2024-04-24T23:28:08.110633+0000","last_unstale":"2024-04-24T23:29:08.221020+0000","last_undegraded":"2024-04-24T23:29:08.221020+0000","last_fullsized":"2024-04-24T23:29:08.221020+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:43:21.694477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.217862+0000","last_change":"2024-04-24T23:27:24.234134+0000","last_active":"2024-04-24T23:29:08.217862+0000","last_peered":"2024-04-24T23:29:08.217862+0000","last_clean":"2024-04-24T23:29:08.217862+0000","last_became_active":"2024-04-24T23:27:24.233650+0000","last_became_peered":"2024-04-24T23:27:24.233650+0000","last_unstale":"2024-04-24T23:29:08.217862+0000","last_undegraded":"2024-04-24T23:29:08.217862+0000","last_fullsized":"2024-04-24T23:29:08.217862+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:04:54.191186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.212945+0000","last_change":"2024-04-24T23:27:18.969572+0000","last_active":"2024-04-24T23:29:08.212945+0000","last_peered":"2024-04-24T23:29:08.212945+0000","last_clean":"2024-04-24T23:29:08.212945+0000","last_became_active":"2024-04-24T23:27:18.968897+0000","last_became_peered":"2024-04-24T23:27:18.968897+0000","last_unstale":"2024-04-24T23:29:08.212945+0000","last_undegraded":"2024-04-24T23:29:08.212945+0000","last_fullsized":"2024-04-24T23:29:08.212945+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:42:51.380386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"92'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.387791+0000","last_change":"2024-04-24T23:27:19.977290+0000","last_active":"2024-04-24T23:29:08.387791+0000","last_peered":"2024-04-24T23:29:08.387791+0000","last_clean":"2024-04-24T23:29:08.387791+0000","last_became_active":"2024-04-24T23:27:19.976802+0000","last_became_peered":"2024-04-24T23:27:19.976802+0000","last_unstale":"2024-04-24T23:29:08.387791+0000","last_undegraded":"2024-04-24T23:29:08.387791+0000","last_fullsized":"2024-04-24T23:29:08.387791+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:55:15.827279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.218034+0000","last_change":"2024-04-24T23:27:24.235706+0000","last_active":"2024-04-24T23:29:08.218034+0000","last_peered":"2024-04-24T23:29:08.218034+0000","last_clean":"2024-04-24T23:29:08.218034+0000","last_became_active":"2024-04-24T23:27:24.235367+0000","last_became_peered":"2024-04-24T23:27:24.235367+0000","last_unstale":"2024-04-24T23:29:08.218034+0000","last_undegraded":"2024-04-24T23:29:08.218034+0000","last_fullsized":"2024-04-24T23:29:08.218034+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:17.444845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.217153+0000","last_change":"2024-04-24T23:28:08.061398+0000","last_active":"2024-04-24T23:29:08.217153+0000","last_peered":"2024-04-24T23:29:08.217153+0000","last_clean":"2024-04-24T23:29:08.217153+0000","last_became_active":"2024-04-24T23:28:08.056810+0000","last_became_peered":"2024-04-24T23:28:08.056810+0000","last_unstale":"2024-04-24T23:29:08.217153+0000","last_undegraded":"2024-04-24T23:29:08.217153+0000","last_fullsized":"2024-04-24T23:29:08.217153+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:10:56.126730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"92'9","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.220961+0000","last_change":"2024-04-24T23:27:19.975222+0000","last_active":"2024-04-24T23:29:08.220961+0000","last_peered":"2024-04-24T23:29:08.220961+0000","last_clean":"2024-04-24T23:29:08.220961+0000","last_became_active":"2024-04-24T23:27:19.974881+0000","last_became_peered":"2024-04-24T23:27:19.974881+0000","last_unstale":"2024-04-24T23:29:08.220961+0000","last_undegraded":"2024-04-24T23:29:08.220961+0000","last_fullsized":"2024-04-24T23:29:08.220961+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:16:17.884748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.217945+0000","last_change":"2024-04-24T23:27:24.232445+0000","last_active":"2024-04-24T23:29:08.217945+0000","last_peered":"2024-04-24T23:29:08.217945+0000","last_clean":"2024-04-24T23:29:08.217945+0000","last_became_active":"2024-04-24T23:27:24.231754+0000","last_became_peered":"2024-04-24T23:27:24.231754+0000","last_unstale":"2024-04-24T23:29:08.217945+0000","last_undegraded":"2024-04-24T23:29:08.217945+0000","last_fullsized":"2024-04-24T23:29:08.217945+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:45:43.630508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.221798+0000","last_change":"2024-04-24T23:27:18.966558+0000","last_active":"2024-04-24T23:29:08.221798+0000","last_peered":"2024-04-24T23:29:08.221798+0000","last_clean":"2024-04-24T23:29:08.221798+0000","last_became_active":"2024-04-24T23:27:18.966212+0000","last_became_peered":"2024-04-24T23:27:18.966212+0000","last_unstale":"2024-04-24T23:29:08.221798+0000","last_undegraded":"2024-04-24T23:29:08.221798+0000","last_fullsized":"2024-04-24T23:29:08.221798+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:24:17.262587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"92'15","reported_seq":46,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.249544+0000","last_change":"2024-04-24T23:27:19.976811+0000","last_active":"2024-04-24T23:29:08.249544+0000","last_peered":"2024-04-24T23:29:08.249544+0000","last_clean":"2024-04-24T23:29:08.249544+0000","last_became_active":"2024-04-24T23:27:19.976456+0000","last_became_peered":"2024-04-24T23:27:19.976456+0000","last_unstale":"2024-04-24T23:29:08.249544+0000","last_undegraded":"2024-04-24T23:29:08.249544+0000","last_fullsized":"2024-04-24T23:29:08.249544+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:43:13.755833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.214292+0000","last_change":"2024-04-24T23:27:24.232136+0000","last_active":"2024-04-24T23:29:08.214292+0000","last_peered":"2024-04-24T23:29:08.214292+0000","last_clean":"2024-04-24T23:29:08.214292+0000","last_became_active":"2024-04-24T23:27:24.231459+0000","last_became_peered":"2024-04-24T23:27:24.231459+0000","last_unstale":"2024-04-24T23:29:08.214292+0000","last_undegraded":"2024-04-24T23:29:08.214292+0000","last_fullsized":"2024-04-24T23:29:08.214292+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:51:51.114787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"92'7","reported_seq":162,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:42.159846+0000","last_change":"2024-04-24T23:27:18.975333+0000","last_active":"2024-04-24T23:29:42.159846+0000","last_peered":"2024-04-24T23:29:42.159846+0000","last_clean":"2024-04-24T23:29:42.159846+0000","last_became_active":"2024-04-24T23:27:18.974869+0000","last_became_peered":"2024-04-24T23:27:18.974869+0000","last_unstale":"2024-04-24T23:29:42.159846+0000","last_undegraded":"2024-04-24T23:29:42.159846+0000","last_fullsized":"2024-04-24T23:29:42.159846+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:30:26.758596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":122,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"92'7","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.221237+0000","last_change":"2024-04-24T23:27:19.974989+0000","last_active":"2024-04-24T23:29:08.221237+0000","last_peered":"2024-04-24T23:29:08.221237+0000","last_clean":"2024-04-24T23:29:08.221237+0000","last_became_active":"2024-04-24T23:27:19.974312+0000","last_became_peered":"2024-04-24T23:27:19.974312+0000","last_unstale":"2024-04-24T23:29:08.221237+0000","last_undegraded":"2024-04-24T23:29:08.221237+0000","last_fullsized":"2024-04-24T23:29:08.221237+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:16:39.685430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.250906+0000","last_change":"2024-04-24T23:29:08.250906+0000","last_active":"2024-04-24T23:29:08.250906+0000","last_peered":"2024-04-24T23:29:08.250906+0000","last_clean":"2024-04-24T23:29:08.250906+0000","last_became_active":"2024-04-24T23:29:08.250264+0000","last_became_peered":"2024-04-24T23:29:08.250264+0000","last_unstale":"2024-04-24T23:29:08.250906+0000","last_undegraded":"2024-04-24T23:29:08.250906+0000","last_fullsized":"2024-04-24T23:29:08.250906+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:41:54.427428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.249568+0000","last_change":"2024-04-24T23:27:18.971313+0000","last_active":"2024-04-24T23:29:08.249568+0000","last_peered":"2024-04-24T23:29:08.249568+0000","last_clean":"2024-04-24T23:29:08.249568+0000","last_became_active":"2024-04-24T23:27:18.970888+0000","last_became_peered":"2024-04-24T23:27:18.970888+0000","last_unstale":"2024-04-24T23:29:08.249568+0000","last_undegraded":"2024-04-24T23:29:08.249568+0000","last_fullsized":"2024-04-24T23:29:08.249568+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:34:20.118693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"92'8","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.249768+0000","last_change":"2024-04-24T23:27:19.978939+0000","last_active":"2024-04-24T23:29:08.249768+0000","last_peered":"2024-04-24T23:29:08.249768+0000","last_clean":"2024-04-24T23:29:08.249768+0000","last_became_active":"2024-04-24T23:27:19.978556+0000","last_became_peered":"2024-04-24T23:27:19.978556+0000","last_unstale":"2024-04-24T23:29:08.249768+0000","last_undegraded":"2024-04-24T23:29:08.249768+0000","last_fullsized":"2024-04-24T23:29:08.249768+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:28:43.118561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.217032+0000","last_change":"2024-04-24T23:27:24.232384+0000","last_active":"2024-04-24T23:29:08.217032+0000","last_peered":"2024-04-24T23:29:08.217032+0000","last_clean":"2024-04-24T23:29:08.217032+0000","last_became_active":"2024-04-24T23:27:24.231497+0000","last_became_peered":"2024-04-24T23:27:24.231497+0000","last_unstale":"2024-04-24T23:29:08.217032+0000","last_undegraded":"2024-04-24T23:29:08.217032+0000","last_fullsized":"2024-04-24T23:29:08.217032+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:26:54.716821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.248992+0000","last_change":"2024-04-24T23:27:18.968949+0000","last_active":"2024-04-24T23:29:08.248992+0000","last_peered":"2024-04-24T23:29:08.248992+0000","last_clean":"2024-04-24T23:29:08.248992+0000","last_became_active":"2024-04-24T23:27:18.968612+0000","last_became_peered":"2024-04-24T23:27:18.968612+0000","last_unstale":"2024-04-24T23:29:08.248992+0000","last_undegraded":"2024-04-24T23:29:08.248992+0000","last_fullsized":"2024-04-24T23:29:08.248992+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:03:45.908080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"92'8","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.212368+0000","last_change":"2024-04-24T23:27:19.979670+0000","last_active":"2024-04-24T23:29:08.212368+0000","last_peered":"2024-04-24T23:29:08.212368+0000","last_clean":"2024-04-24T23:29:08.212368+0000","last_became_active":"2024-04-24T23:27:19.979277+0000","last_became_peered":"2024-04-24T23:27:19.979277+0000","last_unstale":"2024-04-24T23:29:08.212368+0000","last_undegraded":"2024-04-24T23:29:08.212368+0000","last_fullsized":"2024-04-24T23:29:08.212368+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:05:54.958747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.387084+0000","last_change":"2024-04-24T23:27:24.237085+0000","last_active":"2024-04-24T23:29:08.387084+0000","last_peered":"2024-04-24T23:29:08.387084+0000","last_clean":"2024-04-24T23:29:08.387084+0000","last_became_active":"2024-04-24T23:27:24.236671+0000","last_became_peered":"2024-04-24T23:27:24.236671+0000","last_unstale":"2024-04-24T23:29:08.387084+0000","last_undegraded":"2024-04-24T23:29:08.387084+0000","last_fullsized":"2024-04-24T23:29:08.387084+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:48:38.482374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.250147+0000","last_change":"2024-04-24T23:27:18.974146+0000","last_active":"2024-04-24T23:29:08.250147+0000","last_peered":"2024-04-24T23:29:08.250147+0000","last_clean":"2024-04-24T23:29:08.250147+0000","last_became_active":"2024-04-24T23:27:18.973814+0000","last_became_peered":"2024-04-24T23:27:18.973814+0000","last_unstale":"2024-04-24T23:29:08.250147+0000","last_undegraded":"2024-04-24T23:29:08.250147+0000","last_fullsized":"2024-04-24T23:29:08.250147+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:22:52.409448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"92'11","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.220653+0000","last_change":"2024-04-24T23:27:19.984325+0000","last_active":"2024-04-24T23:29:08.220653+0000","last_peered":"2024-04-24T23:29:08.220653+0000","last_clean":"2024-04-24T23:29:08.220653+0000","last_became_active":"2024-04-24T23:27:19.983918+0000","last_became_peered":"2024-04-24T23:27:19.983918+0000","last_unstale":"2024-04-24T23:29:08.220653+0000","last_undegraded":"2024-04-24T23:29:08.220653+0000","last_fullsized":"2024-04-24T23:29:08.220653+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:24:07.640983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.212601+0000","last_change":"2024-04-24T23:27:24.223561+0000","last_active":"2024-04-24T23:29:08.212601+0000","last_peered":"2024-04-24T23:29:08.212601+0000","last_clean":"2024-04-24T23:29:08.212601+0000","last_became_active":"2024-04-24T23:27:24.223127+0000","last_became_peered":"2024-04-24T23:27:24.223127+0000","last_unstale":"2024-04-24T23:29:08.212601+0000","last_undegraded":"2024-04-24T23:29:08.212601+0000","last_fullsized":"2024-04-24T23:29:08.212601+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:31:28.079633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"92'4","reported_seq":156,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:43.160939+0000","last_change":"2024-04-24T23:27:18.971374+0000","last_active":"2024-04-24T23:29:43.160939+0000","last_peered":"2024-04-24T23:29:43.160939+0000","last_clean":"2024-04-24T23:29:43.160939+0000","last_became_active":"2024-04-24T23:27:18.970818+0000","last_became_peered":"2024-04-24T23:27:18.970818+0000","last_unstale":"2024-04-24T23:29:43.160939+0000","last_undegraded":"2024-04-24T23:29:43.160939+0000","last_fullsized":"2024-04-24T23:29:43.160939+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:12:17.547968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":118,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"92'10","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.222062+0000","last_change":"2024-04-24T23:27:19.980275+0000","last_active":"2024-04-24T23:29:08.222062+0000","last_peered":"2024-04-24T23:29:08.222062+0000","last_clean":"2024-04-24T23:29:08.222062+0000","last_became_active":"2024-04-24T23:27:19.979925+0000","last_became_peered":"2024-04-24T23:27:19.979925+0000","last_unstale":"2024-04-24T23:29:08.222062+0000","last_undegraded":"2024-04-24T23:29:08.222062+0000","last_fullsized":"2024-04-24T23:29:08.222062+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:21:10.681169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.220570+0000","last_change":"2024-04-24T23:27:24.231389+0000","last_active":"2024-04-24T23:29:08.220570+0000","last_peered":"2024-04-24T23:29:08.220570+0000","last_clean":"2024-04-24T23:29:08.220570+0000","last_became_active":"2024-04-24T23:27:24.230973+0000","last_became_peered":"2024-04-24T23:27:24.230973+0000","last_unstale":"2024-04-24T23:29:08.220570+0000","last_undegraded":"2024-04-24T23:29:08.220570+0000","last_fullsized":"2024-04-24T23:29:08.220570+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:35.943122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.222447+0000","last_change":"2024-04-24T23:27:18.977009+0000","last_active":"2024-04-24T23:29:08.222447+0000","last_peered":"2024-04-24T23:29:08.222447+0000","last_clean":"2024-04-24T23:29:08.222447+0000","last_became_active":"2024-04-24T23:27:18.976667+0000","last_became_peered":"2024-04-24T23:27:18.976667+0000","last_unstale":"2024-04-24T23:29:08.222447+0000","last_undegraded":"2024-04-24T23:29:08.222447+0000","last_fullsized":"2024-04-24T23:29:08.222447+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:19:47.817961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"92'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.387836+0000","last_change":"2024-04-24T23:27:19.988589+0000","last_active":"2024-04-24T23:29:08.387836+0000","last_peered":"2024-04-24T23:29:08.387836+0000","last_clean":"2024-04-24T23:29:08.387836+0000","last_became_active":"2024-04-24T23:27:19.988261+0000","last_became_peered":"2024-04-24T23:27:19.988261+0000","last_unstale":"2024-04-24T23:29:08.387836+0000","last_undegraded":"2024-04-24T23:29:08.387836+0000","last_fullsized":"2024-04-24T23:29:08.387836+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:44:15.809776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.218303+0000","last_change":"2024-04-24T23:27:24.231256+0000","last_active":"2024-04-24T23:29:08.218303+0000","last_peered":"2024-04-24T23:29:08.218303+0000","last_clean":"2024-04-24T23:29:08.218303+0000","last_became_active":"2024-04-24T23:27:24.230545+0000","last_became_peered":"2024-04-24T23:27:24.230545+0000","last_unstale":"2024-04-24T23:29:08.218303+0000","last_undegraded":"2024-04-24T23:29:08.218303+0000","last_fullsized":"2024-04-24T23:29:08.218303+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:43:40.460473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.249368+0000","last_change":"2024-04-24T23:27:18.968805+0000","last_active":"2024-04-24T23:29:08.249368+0000","last_peered":"2024-04-24T23:29:08.249368+0000","last_clean":"2024-04-24T23:29:08.249368+0000","last_became_active":"2024-04-24T23:27:18.968423+0000","last_became_peered":"2024-04-24T23:27:18.968423+0000","last_unstale":"2024-04-24T23:29:08.249368+0000","last_undegraded":"2024-04-24T23:29:08.249368+0000","last_fullsized":"2024-04-24T23:29:08.249368+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:14:04.077773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"92'11","reported_seq":49,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.221213+0000","last_change":"2024-04-24T23:28:08.056901+0000","last_active":"2024-04-24T23:29:08.221213+0000","last_peered":"2024-04-24T23:29:08.221213+0000","last_clean":"2024-04-24T23:29:08.221213+0000","last_became_active":"2024-04-24T23:28:08.056422+0000","last_became_peered":"2024-04-24T23:28:08.056422+0000","last_unstale":"2024-04-24T23:29:08.221213+0000","last_undegraded":"2024-04-24T23:29:08.221213+0000","last_fullsized":"2024-04-24T23:29:08.221213+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:10:01.716479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.220583+0000","last_change":"2024-04-24T23:27:24.225481+0000","last_active":"2024-04-24T23:29:08.220583+0000","last_peered":"2024-04-24T23:29:08.220583+0000","last_clean":"2024-04-24T23:29:08.220583+0000","last_became_active":"2024-04-24T23:27:24.224363+0000","last_became_peered":"2024-04-24T23:27:24.224363+0000","last_unstale":"2024-04-24T23:29:08.220583+0000","last_undegraded":"2024-04-24T23:29:08.220583+0000","last_fullsized":"2024-04-24T23:29:08.220583+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:36:17.715596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"94'35","reported_seq":314,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:43.173813+0000","last_change":"2024-04-24T23:27:18.968264+0000","last_active":"2024-04-24T23:29:43.173813+0000","last_peered":"2024-04-24T23:29:43.173813+0000","last_clean":"2024-04-24T23:29:43.173813+0000","last_became_active":"2024-04-24T23:27:18.967856+0000","last_became_peered":"2024-04-24T23:27:18.967856+0000","last_unstale":"2024-04-24T23:29:43.173813+0000","last_undegraded":"2024-04-24T23:29:43.173813+0000","last_fullsized":"2024-04-24T23:29:43.173813+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:48:44.347254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59428,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":245,"num_read_kb":181,"num_write":36,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"92'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.222070+0000","last_change":"2024-04-24T23:27:19.984719+0000","last_active":"2024-04-24T23:29:08.222070+0000","last_peered":"2024-04-24T23:29:08.222070+0000","last_clean":"2024-04-24T23:29:08.222070+0000","last_became_active":"2024-04-24T23:27:19.984386+0000","last_became_peered":"2024-04-24T23:27:19.984386+0000","last_unstale":"2024-04-24T23:29:08.222070+0000","last_undegraded":"2024-04-24T23:29:08.222070+0000","last_fullsized":"2024-04-24T23:29:08.222070+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:02:13.739759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.248640+0000","last_change":"2024-04-24T23:27:24.230795+0000","last_active":"2024-04-24T23:29:08.248640+0000","last_peered":"2024-04-24T23:29:08.248640+0000","last_clean":"2024-04-24T23:29:08.248640+0000","last_became_active":"2024-04-24T23:27:24.230360+0000","last_became_peered":"2024-04-24T23:27:24.230360+0000","last_unstale":"2024-04-24T23:29:08.248640+0000","last_undegraded":"2024-04-24T23:29:08.248640+0000","last_fullsized":"2024-04-24T23:29:08.248640+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:51:24.875668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.214478+0000","last_change":"2024-04-24T23:27:18.964964+0000","last_active":"2024-04-24T23:29:08.214478+0000","last_peered":"2024-04-24T23:29:08.214478+0000","last_clean":"2024-04-24T23:29:08.214478+0000","last_became_active":"2024-04-24T23:27:18.964632+0000","last_became_peered":"2024-04-24T23:27:18.964632+0000","last_unstale":"2024-04-24T23:29:08.214478+0000","last_undegraded":"2024-04-24T23:29:08.214478+0000","last_fullsized":"2024-04-24T23:29:08.214478+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:52:23.144629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"92'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.249122+0000","last_change":"2024-04-24T23:27:19.987258+0000","last_active":"2024-04-24T23:29:08.249122+0000","last_peered":"2024-04-24T23:29:08.249122+0000","last_clean":"2024-04-24T23:29:08.249122+0000","last_became_active":"2024-04-24T23:27:19.986864+0000","last_became_peered":"2024-04-24T23:27:19.986864+0000","last_unstale":"2024-04-24T23:29:08.249122+0000","last_undegraded":"2024-04-24T23:29:08.249122+0000","last_fullsized":"2024-04-24T23:29:08.249122+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:57:40.503506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.251754+0000","last_change":"2024-04-24T23:29:08.251155+0000","last_active":"2024-04-24T23:29:08.251754+0000","last_peered":"2024-04-24T23:29:08.251754+0000","last_clean":"2024-04-24T23:29:08.251754+0000","last_became_active":"2024-04-24T23:29:08.250662+0000","last_became_peered":"2024-04-24T23:29:08.250662+0000","last_unstale":"2024-04-24T23:29:08.251754+0000","last_undegraded":"2024-04-24T23:29:08.251754+0000","last_fullsized":"2024-04-24T23:29:08.251754+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:32:01.403466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.249469+0000","last_change":"2024-04-24T23:27:18.974229+0000","last_active":"2024-04-24T23:29:08.249469+0000","last_peered":"2024-04-24T23:29:08.249469+0000","last_clean":"2024-04-24T23:29:08.249469+0000","last_became_active":"2024-04-24T23:27:18.973893+0000","last_became_peered":"2024-04-24T23:27:18.973893+0000","last_unstale":"2024-04-24T23:29:08.249469+0000","last_undegraded":"2024-04-24T23:29:08.249469+0000","last_fullsized":"2024-04-24T23:29:08.249469+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:41:45.699709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"92'15","reported_seq":47,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.221880+0000","last_change":"2024-04-24T23:27:19.986910+0000","last_active":"2024-04-24T23:29:08.221880+0000","last_peered":"2024-04-24T23:29:08.221880+0000","last_clean":"2024-04-24T23:29:08.221880+0000","last_became_active":"2024-04-24T23:27:19.986556+0000","last_became_peered":"2024-04-24T23:27:19.986556+0000","last_unstale":"2024-04-24T23:29:08.221880+0000","last_undegraded":"2024-04-24T23:29:08.221880+0000","last_fullsized":"2024-04-24T23:29:08.221880+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:06:28.506556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.221054+0000","last_change":"2024-04-24T23:27:24.233115+0000","last_active":"2024-04-24T23:29:08.221054+0000","last_peered":"2024-04-24T23:29:08.221054+0000","last_clean":"2024-04-24T23:29:08.221054+0000","last_became_active":"2024-04-24T23:27:24.232714+0000","last_became_peered":"2024-04-24T23:27:24.232714+0000","last_unstale":"2024-04-24T23:29:08.221054+0000","last_undegraded":"2024-04-24T23:29:08.221054+0000","last_fullsized":"2024-04-24T23:29:08.221054+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:04.365515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"94'6","reported_seq":163,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:42.159532+0000","last_change":"2024-04-24T23:27:18.966426+0000","last_active":"2024-04-24T23:29:42.159532+0000","last_peered":"2024-04-24T23:29:42.159532+0000","last_clean":"2024-04-24T23:29:42.159532+0000","last_became_active":"2024-04-24T23:27:18.966070+0000","last_became_peered":"2024-04-24T23:27:18.966070+0000","last_unstale":"2024-04-24T23:29:42.159532+0000","last_undegraded":"2024-04-24T23:29:42.159532+0000","last_fullsized":"2024-04-24T23:29:42.159532+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:56:21.679639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1428,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":124,"num_read_kb":6,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"92'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.221340+0000","last_change":"2024-04-24T23:27:19.991829+0000","last_active":"2024-04-24T23:29:08.221340+0000","last_peered":"2024-04-24T23:29:08.221340+0000","last_clean":"2024-04-24T23:29:08.221340+0000","last_became_active":"2024-04-24T23:27:19.991492+0000","last_became_peered":"2024-04-24T23:27:19.991492+0000","last_unstale":"2024-04-24T23:29:08.221340+0000","last_undegraded":"2024-04-24T23:29:08.221340+0000","last_fullsized":"2024-04-24T23:29:08.221340+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:23:45.195217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.248500+0000","last_change":"2024-04-24T23:27:24.229794+0000","last_active":"2024-04-24T23:29:08.248500+0000","last_peered":"2024-04-24T23:29:08.248500+0000","last_clean":"2024-04-24T23:29:08.248500+0000","last_became_active":"2024-04-24T23:27:24.229410+0000","last_became_peered":"2024-04-24T23:27:24.229410+0000","last_unstale":"2024-04-24T23:29:08.248500+0000","last_undegraded":"2024-04-24T23:29:08.248500+0000","last_fullsized":"2024-04-24T23:29:08.248500+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:00:53.188307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":74,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.221241+0000","last_change":"2024-04-24T23:28:08.068516+0000","last_active":"2024-04-24T23:29:08.221241+0000","last_peered":"2024-04-24T23:29:08.221241+0000","last_clean":"2024-04-24T23:29:08.221241+0000","last_became_active":"2024-04-24T23:28:08.056740+0000","last_became_peered":"2024-04-24T23:28:08.056740+0000","last_unstale":"2024-04-24T23:29:08.221241+0000","last_undegraded":"2024-04-24T23:29:08.221241+0000","last_fullsized":"2024-04-24T23:29:08.221241+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:47:35.568377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,1],"acting":[8,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"92'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.249623+0000","last_change":"2024-04-24T23:27:19.982608+0000","last_active":"2024-04-24T23:29:08.249623+0000","last_peered":"2024-04-24T23:29:08.249623+0000","last_clean":"2024-04-24T23:29:08.249623+0000","last_became_active":"2024-04-24T23:27:19.982270+0000","last_became_peered":"2024-04-24T23:27:19.982270+0000","last_unstale":"2024-04-24T23:29:08.249623+0000","last_undegraded":"2024-04-24T23:29:08.249623+0000","last_fullsized":"2024-04-24T23:29:08.249623+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:01:10.525164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.248615+0000","last_change":"2024-04-24T23:27:24.236143+0000","last_active":"2024-04-24T23:29:08.248615+0000","last_peered":"2024-04-24T23:29:08.248615+0000","last_clean":"2024-04-24T23:29:08.248615+0000","last_became_active":"2024-04-24T23:27:24.235724+0000","last_became_peered":"2024-04-24T23:27:24.235724+0000","last_unstale":"2024-04-24T23:29:08.248615+0000","last_undegraded":"2024-04-24T23:29:08.248615+0000","last_fullsized":"2024-04-24T23:29:08.248615+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:55:41.317044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"94'3","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:36.689756+0000","last_change":"2024-04-24T23:27:18.971312+0000","last_active":"2024-04-24T23:29:36.689756+0000","last_peered":"2024-04-24T23:29:36.689756+0000","last_clean":"2024-04-24T23:29:36.689756+0000","last_became_active":"2024-04-24T23:27:18.970929+0000","last_became_peered":"2024-04-24T23:27:18.970929+0000","last_unstale":"2024-04-24T23:29:36.689756+0000","last_undegraded":"2024-04-24T23:29:36.689756+0000","last_fullsized":"2024-04-24T23:29:36.689756+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:54:20.318916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"92'5","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.217096+0000","last_change":"2024-04-24T23:28:08.056700+0000","last_active":"2024-04-24T23:29:08.217096+0000","last_peered":"2024-04-24T23:29:08.217096+0000","last_clean":"2024-04-24T23:29:08.217096+0000","last_became_active":"2024-04-24T23:28:08.056271+0000","last_became_peered":"2024-04-24T23:28:08.056271+0000","last_unstale":"2024-04-24T23:29:08.217096+0000","last_undegraded":"2024-04-24T23:29:08.217096+0000","last_fullsized":"2024-04-24T23:29:08.217096+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:03:34.808028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.221630+0000","last_change":"2024-04-24T23:27:24.718948+0000","last_active":"2024-04-24T23:29:08.221630+0000","last_peered":"2024-04-24T23:29:08.221630+0000","last_clean":"2024-04-24T23:29:08.221630+0000","last_became_active":"2024-04-24T23:27:24.718424+0000","last_became_peered":"2024-04-24T23:27:24.718424+0000","last_unstale":"2024-04-24T23:29:08.221630+0000","last_undegraded":"2024-04-24T23:29:08.221630+0000","last_fullsized":"2024-04-24T23:29:08.221630+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:22:01.984976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.220935+0000","last_change":"2024-04-24T23:27:18.957073+0000","last_active":"2024-04-24T23:29:08.220935+0000","last_peered":"2024-04-24T23:29:08.220935+0000","last_clean":"2024-04-24T23:29:08.220935+0000","last_became_active":"2024-04-24T23:27:18.956713+0000","last_became_peered":"2024-04-24T23:27:18.956713+0000","last_unstale":"2024-04-24T23:29:08.220935+0000","last_undegraded":"2024-04-24T23:29:08.220935+0000","last_fullsized":"2024-04-24T23:29:08.220935+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:16:21.015902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"92'4","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.249932+0000","last_change":"2024-04-24T23:27:19.985058+0000","last_active":"2024-04-24T23:29:08.249932+0000","last_peered":"2024-04-24T23:29:08.249932+0000","last_clean":"2024-04-24T23:29:08.249932+0000","last_became_active":"2024-04-24T23:27:19.984675+0000","last_became_peered":"2024-04-24T23:27:19.984675+0000","last_unstale":"2024-04-24T23:29:08.249932+0000","last_undegraded":"2024-04-24T23:29:08.249932+0000","last_fullsized":"2024-04-24T23:29:08.249932+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:22:35.252839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.248925+0000","last_change":"2024-04-24T23:27:24.236117+0000","last_active":"2024-04-24T23:29:08.248925+0000","last_peered":"2024-04-24T23:29:08.248925+0000","last_clean":"2024-04-24T23:29:08.248925+0000","last_became_active":"2024-04-24T23:27:24.235720+0000","last_became_peered":"2024-04-24T23:27:24.235720+0000","last_unstale":"2024-04-24T23:29:08.248925+0000","last_undegraded":"2024-04-24T23:29:08.248925+0000","last_fullsized":"2024-04-24T23:29:08.248925+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:46:40.604988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.221627+0000","last_change":"2024-04-24T23:27:18.973690+0000","last_active":"2024-04-24T23:29:08.221627+0000","last_peered":"2024-04-24T23:29:08.221627+0000","last_clean":"2024-04-24T23:29:08.221627+0000","last_became_active":"2024-04-24T23:27:18.973182+0000","last_became_peered":"2024-04-24T23:27:18.973182+0000","last_unstale":"2024-04-24T23:29:08.221627+0000","last_undegraded":"2024-04-24T23:29:08.221627+0000","last_fullsized":"2024-04-24T23:29:08.221627+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:00:44.809880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"92'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.249631+0000","last_change":"2024-04-24T23:27:19.991252+0000","last_active":"2024-04-24T23:29:08.249631+0000","last_peered":"2024-04-24T23:29:08.249631+0000","last_clean":"2024-04-24T23:29:08.249631+0000","last_became_active":"2024-04-24T23:27:19.990892+0000","last_became_peered":"2024-04-24T23:27:19.990892+0000","last_unstale":"2024-04-24T23:29:08.249631+0000","last_undegraded":"2024-04-24T23:29:08.249631+0000","last_fullsized":"2024-04-24T23:29:08.249631+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:29:37.835521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.249473+0000","last_change":"2024-04-24T23:27:24.718929+0000","last_active":"2024-04-24T23:29:08.249473+0000","last_peered":"2024-04-24T23:29:08.249473+0000","last_clean":"2024-04-24T23:29:08.249473+0000","last_became_active":"2024-04-24T23:27:24.718486+0000","last_became_peered":"2024-04-24T23:27:24.718486+0000","last_unstale":"2024-04-24T23:29:08.249473+0000","last_undegraded":"2024-04-24T23:29:08.249473+0000","last_fullsized":"2024-04-24T23:29:08.249473+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:31:13.359829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"92'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.249896+0000","last_change":"2024-04-24T23:27:19.983181+0000","last_active":"2024-04-24T23:29:08.249896+0000","last_peered":"2024-04-24T23:29:08.249896+0000","last_clean":"2024-04-24T23:29:08.249896+0000","last_became_active":"2024-04-24T23:27:19.981764+0000","last_became_peered":"2024-04-24T23:27:19.981764+0000","last_unstale":"2024-04-24T23:29:08.249896+0000","last_undegraded":"2024-04-24T23:29:08.249896+0000","last_fullsized":"2024-04-24T23:29:08.249896+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:01:20.653572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"94'6","reported_seq":159,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:43.160323+0000","last_change":"2024-04-24T23:27:18.976351+0000","last_active":"2024-04-24T23:29:43.160323+0000","last_peered":"2024-04-24T23:29:43.160323+0000","last_clean":"2024-04-24T23:29:43.160323+0000","last_became_active":"2024-04-24T23:27:18.975552+0000","last_became_peered":"2024-04-24T23:27:18.975552+0000","last_unstale":"2024-04-24T23:29:43.160323+0000","last_undegraded":"2024-04-24T23:29:43.160323+0000","last_fullsized":"2024-04-24T23:29:43.160323+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:26:29.551439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":119,"num_read_kb":119,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":27,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.222548+0000","last_change":"2024-04-24T23:27:24.229528+0000","last_active":"2024-04-24T23:29:08.222548+0000","last_peered":"2024-04-24T23:29:08.222548+0000","last_clean":"2024-04-24T23:29:08.222548+0000","last_became_active":"2024-04-24T23:27:24.229099+0000","last_became_peered":"2024-04-24T23:27:24.229099+0000","last_unstale":"2024-04-24T23:29:08.222548+0000","last_undegraded":"2024-04-24T23:29:08.222548+0000","last_fullsized":"2024-04-24T23:29:08.222548+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:51:51.421912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"92'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.212843+0000","last_change":"2024-04-24T23:27:19.980094+0000","last_active":"2024-04-24T23:29:08.212843+0000","last_peered":"2024-04-24T23:29:08.212843+0000","last_clean":"2024-04-24T23:29:08.212843+0000","last_became_active":"2024-04-24T23:27:19.979710+0000","last_became_peered":"2024-04-24T23:27:19.979710+0000","last_unstale":"2024-04-24T23:29:08.212843+0000","last_undegraded":"2024-04-24T23:29:08.212843+0000","last_fullsized":"2024-04-24T23:29:08.212843+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:50:37.920235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.218213+0000","last_change":"2024-04-24T23:27:18.963224+0000","last_active":"2024-04-24T23:29:08.218213+0000","last_peered":"2024-04-24T23:29:08.218213+0000","last_clean":"2024-04-24T23:29:08.218213+0000","last_became_active":"2024-04-24T23:27:18.962794+0000","last_became_peered":"2024-04-24T23:27:18.962794+0000","last_unstale":"2024-04-24T23:29:08.218213+0000","last_undegraded":"2024-04-24T23:29:08.218213+0000","last_fullsized":"2024-04-24T23:29:08.218213+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:05:36.869062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":358,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.220962+0000","last_change":"2024-04-24T23:26:02.307599+0000","last_active":"2024-04-24T23:29:08.220962+0000","last_peered":"2024-04-24T23:29:08.220962+0000","last_clean":"2024-04-24T23:29:08.220962+0000","last_became_active":"2024-04-24T23:26:02.307186+0000","last_became_peered":"2024-04-24T23:26:02.307186+0000","last_unstale":"2024-04-24T23:29:08.220962+0000","last_undegraded":"2024-04-24T23:29:08.220962+0000","last_fullsized":"2024-04-24T23:29:08.220962+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:22:33.824274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:22:33.824274+0000","last_clean_scrub_stamp":"2024-04-24T23:22:33.824274+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:19:26.789955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.249319+0000","last_change":"2024-04-24T23:27:24.719315+0000","last_active":"2024-04-24T23:29:08.249319+0000","last_peered":"2024-04-24T23:29:08.249319+0000","last_clean":"2024-04-24T23:29:08.249319+0000","last_became_active":"2024-04-24T23:27:24.718868+0000","last_became_peered":"2024-04-24T23:27:24.718868+0000","last_unstale":"2024-04-24T23:29:08.249319+0000","last_undegraded":"2024-04-24T23:29:08.249319+0000","last_fullsized":"2024-04-24T23:29:08.249319+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:43:32.122873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"92'5","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.248863+0000","last_change":"2024-04-24T23:27:19.983711+0000","last_active":"2024-04-24T23:29:08.248863+0000","last_peered":"2024-04-24T23:29:08.248863+0000","last_clean":"2024-04-24T23:29:08.248863+0000","last_became_active":"2024-04-24T23:27:19.983302+0000","last_became_peered":"2024-04-24T23:27:19.983302+0000","last_unstale":"2024-04-24T23:29:08.248863+0000","last_undegraded":"2024-04-24T23:29:08.248863+0000","last_fullsized":"2024-04-24T23:29:08.248863+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:44:49.251679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.221222+0000","last_change":"2024-04-24T23:27:18.976766+0000","last_active":"2024-04-24T23:29:08.221222+0000","last_peered":"2024-04-24T23:29:08.221222+0000","last_clean":"2024-04-24T23:29:08.221222+0000","last_became_active":"2024-04-24T23:27:18.976420+0000","last_became_peered":"2024-04-24T23:27:18.976420+0000","last_unstale":"2024-04-24T23:29:08.221222+0000","last_undegraded":"2024-04-24T23:29:08.221222+0000","last_fullsized":"2024-04-24T23:29:08.221222+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:59:34.262774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.216819+0000","last_change":"2024-04-24T23:27:24.226979+0000","last_active":"2024-04-24T23:29:08.216819+0000","last_peered":"2024-04-24T23:29:08.216819+0000","last_clean":"2024-04-24T23:29:08.216819+0000","last_became_active":"2024-04-24T23:27:24.226596+0000","last_became_peered":"2024-04-24T23:27:24.226596+0000","last_unstale":"2024-04-24T23:29:08.216819+0000","last_undegraded":"2024-04-24T23:29:08.216819+0000","last_fullsized":"2024-04-24T23:29:08.216819+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:50:48.585067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.222082+0000","last_change":"2024-04-24T23:27:18.970811+0000","last_active":"2024-04-24T23:29:08.222082+0000","last_peered":"2024-04-24T23:29:08.222082+0000","last_clean":"2024-04-24T23:29:08.222082+0000","last_became_active":"2024-04-24T23:27:18.970292+0000","last_became_peered":"2024-04-24T23:27:18.970292+0000","last_unstale":"2024-04-24T23:29:08.222082+0000","last_undegraded":"2024-04-24T23:29:08.222082+0000","last_fullsized":"2024-04-24T23:29:08.222082+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:06:37.556527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"92'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.249285+0000","last_change":"2024-04-24T23:27:19.975455+0000","last_active":"2024-04-24T23:29:08.249285+0000","last_peered":"2024-04-24T23:29:08.249285+0000","last_clean":"2024-04-24T23:29:08.249285+0000","last_became_active":"2024-04-24T23:27:19.973504+0000","last_became_peered":"2024-04-24T23:27:19.973504+0000","last_unstale":"2024-04-24T23:29:08.249285+0000","last_undegraded":"2024-04-24T23:29:08.249285+0000","last_fullsized":"2024-04-24T23:29:08.249285+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:50:34.450370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.212709+0000","last_change":"2024-04-24T23:27:24.719253+0000","last_active":"2024-04-24T23:29:08.212709+0000","last_peered":"2024-04-24T23:29:08.212709+0000","last_clean":"2024-04-24T23:29:08.212709+0000","last_became_active":"2024-04-24T23:27:24.718357+0000","last_became_peered":"2024-04-24T23:27:24.718357+0000","last_unstale":"2024-04-24T23:29:08.212709+0000","last_undegraded":"2024-04-24T23:29:08.212709+0000","last_fullsized":"2024-04-24T23:29:08.212709+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:31:03.798037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"94'10","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:28.012182+0000","last_change":"2024-04-24T23:27:18.974310+0000","last_active":"2024-04-24T23:29:28.012182+0000","last_peered":"2024-04-24T23:29:28.012182+0000","last_clean":"2024-04-24T23:29:28.012182+0000","last_became_active":"2024-04-24T23:27:18.973956+0000","last_became_peered":"2024-04-24T23:27:18.973956+0000","last_unstale":"2024-04-24T23:29:28.012182+0000","last_undegraded":"2024-04-24T23:29:28.012182+0000","last_fullsized":"2024-04-24T23:29:28.012182+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:44:27.233155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"92'5","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.213085+0000","last_change":"2024-04-24T23:27:19.985763+0000","last_active":"2024-04-24T23:29:08.213085+0000","last_peered":"2024-04-24T23:29:08.213085+0000","last_clean":"2024-04-24T23:29:08.213085+0000","last_became_active":"2024-04-24T23:27:19.985413+0000","last_became_peered":"2024-04-24T23:27:19.985413+0000","last_unstale":"2024-04-24T23:29:08.213085+0000","last_undegraded":"2024-04-24T23:29:08.213085+0000","last_fullsized":"2024-04-24T23:29:08.213085+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:04:54.910407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.214033+0000","last_change":"2024-04-24T23:27:24.225615+0000","last_active":"2024-04-24T23:29:08.214033+0000","last_peered":"2024-04-24T23:29:08.214033+0000","last_clean":"2024-04-24T23:29:08.214033+0000","last_became_active":"2024-04-24T23:27:24.225029+0000","last_became_peered":"2024-04-24T23:27:24.225029+0000","last_unstale":"2024-04-24T23:29:08.214033+0000","last_undegraded":"2024-04-24T23:29:08.214033+0000","last_fullsized":"2024-04-24T23:29:08.214033+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:56:02.956669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.218561+0000","last_change":"2024-04-24T23:27:18.963235+0000","last_active":"2024-04-24T23:29:08.218561+0000","last_peered":"2024-04-24T23:29:08.218561+0000","last_clean":"2024-04-24T23:29:08.218561+0000","last_became_active":"2024-04-24T23:27:18.962835+0000","last_became_peered":"2024-04-24T23:27:18.962835+0000","last_unstale":"2024-04-24T23:29:08.218561+0000","last_undegraded":"2024-04-24T23:29:08.218561+0000","last_fullsized":"2024-04-24T23:29:08.218561+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:34:29.376856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"92'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.387790+0000","last_change":"2024-04-24T23:27:19.987386+0000","last_active":"2024-04-24T23:29:08.387790+0000","last_peered":"2024-04-24T23:29:08.387790+0000","last_clean":"2024-04-24T23:29:08.387790+0000","last_became_active":"2024-04-24T23:27:19.987003+0000","last_became_peered":"2024-04-24T23:27:19.987003+0000","last_unstale":"2024-04-24T23:29:08.387790+0000","last_undegraded":"2024-04-24T23:29:08.387790+0000","last_fullsized":"2024-04-24T23:29:08.387790+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:20:25.635732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.218484+0000","last_change":"2024-04-24T23:27:24.719291+0000","last_active":"2024-04-24T23:29:08.218484+0000","last_peered":"2024-04-24T23:29:08.218484+0000","last_clean":"2024-04-24T23:29:08.218484+0000","last_became_active":"2024-04-24T23:27:24.718796+0000","last_became_peered":"2024-04-24T23:27:24.718796+0000","last_unstale":"2024-04-24T23:29:08.218484+0000","last_undegraded":"2024-04-24T23:29:08.218484+0000","last_fullsized":"2024-04-24T23:29:08.218484+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:20:25.135902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.220641+0000","last_change":"2024-04-24T23:27:18.967856+0000","last_active":"2024-04-24T23:29:08.220641+0000","last_peered":"2024-04-24T23:29:08.220641+0000","last_clean":"2024-04-24T23:29:08.220641+0000","last_became_active":"2024-04-24T23:27:18.967497+0000","last_became_peered":"2024-04-24T23:27:18.967497+0000","last_unstale":"2024-04-24T23:29:08.220641+0000","last_undegraded":"2024-04-24T23:29:08.220641+0000","last_fullsized":"2024-04-24T23:29:08.220641+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:21:29.841791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"92'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.221174+0000","last_change":"2024-04-24T23:27:19.981302+0000","last_active":"2024-04-24T23:29:08.221174+0000","last_peered":"2024-04-24T23:29:08.221174+0000","last_clean":"2024-04-24T23:29:08.221174+0000","last_became_active":"2024-04-24T23:27:19.980938+0000","last_became_peered":"2024-04-24T23:27:19.980938+0000","last_unstale":"2024-04-24T23:29:08.221174+0000","last_undegraded":"2024-04-24T23:29:08.221174+0000","last_fullsized":"2024-04-24T23:29:08.221174+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:24:53.939882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.221231+0000","last_change":"2024-04-24T23:27:24.224626+0000","last_active":"2024-04-24T23:29:08.221231+0000","last_peered":"2024-04-24T23:29:08.221231+0000","last_clean":"2024-04-24T23:29:08.221231+0000","last_became_active":"2024-04-24T23:27:24.224175+0000","last_became_peered":"2024-04-24T23:27:24.224175+0000","last_unstale":"2024-04-24T23:29:08.221231+0000","last_undegraded":"2024-04-24T23:29:08.221231+0000","last_fullsized":"2024-04-24T23:29:08.221231+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:54:44.155637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"90'3","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.212460+0000","last_change":"2024-04-24T23:27:18.975662+0000","last_active":"2024-04-24T23:29:08.212460+0000","last_peered":"2024-04-24T23:29:08.212460+0000","last_clean":"2024-04-24T23:29:08.212460+0000","last_became_active":"2024-04-24T23:27:18.975326+0000","last_became_peered":"2024-04-24T23:27:18.975326+0000","last_unstale":"2024-04-24T23:29:08.212460+0000","last_undegraded":"2024-04-24T23:29:08.212460+0000","last_fullsized":"2024-04-24T23:29:08.212460+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:39.440866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"92'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.216748+0000","last_change":"2024-04-24T23:27:19.978818+0000","last_active":"2024-04-24T23:29:08.216748+0000","last_peered":"2024-04-24T23:29:08.216748+0000","last_clean":"2024-04-24T23:29:08.216748+0000","last_became_active":"2024-04-24T23:27:19.978107+0000","last_became_peered":"2024-04-24T23:27:19.978107+0000","last_unstale":"2024-04-24T23:29:08.216748+0000","last_undegraded":"2024-04-24T23:29:08.216748+0000","last_fullsized":"2024-04-24T23:29:08.216748+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:22:25.797996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.255331+0000","last_change":"2024-04-24T23:29:08.254854+0000","last_active":"2024-04-24T23:29:08.255331+0000","last_peered":"2024-04-24T23:29:08.255331+0000","last_clean":"2024-04-24T23:29:08.255331+0000","last_became_active":"2024-04-24T23:29:08.250700+0000","last_became_peered":"2024-04-24T23:29:08.250700+0000","last_unstale":"2024-04-24T23:29:08.255331+0000","last_undegraded":"2024-04-24T23:29:08.255331+0000","last_fullsized":"2024-04-24T23:29:08.255331+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:25:31.400400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'1","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.214173+0000","last_change":"2024-04-24T23:27:18.967165+0000","last_active":"2024-04-24T23:29:08.214173+0000","last_peered":"2024-04-24T23:29:08.214173+0000","last_clean":"2024-04-24T23:29:08.214173+0000","last_became_active":"2024-04-24T23:27:18.966628+0000","last_became_peered":"2024-04-24T23:27:18.966628+0000","last_unstale":"2024-04-24T23:29:08.214173+0000","last_undegraded":"2024-04-24T23:29:08.214173+0000","last_fullsized":"2024-04-24T23:29:08.214173+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:12:20.520777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"92'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.212534+0000","last_change":"2024-04-24T23:27:19.988069+0000","last_active":"2024-04-24T23:29:08.212534+0000","last_peered":"2024-04-24T23:29:08.212534+0000","last_clean":"2024-04-24T23:29:08.212534+0000","last_became_active":"2024-04-24T23:27:19.987660+0000","last_became_peered":"2024-04-24T23:27:19.987660+0000","last_unstale":"2024-04-24T23:29:08.212534+0000","last_undegraded":"2024-04-24T23:29:08.212534+0000","last_fullsized":"2024-04-24T23:29:08.212534+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:21:03.505313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.218455+0000","last_change":"2024-04-24T23:27:24.719041+0000","last_active":"2024-04-24T23:29:08.218455+0000","last_peered":"2024-04-24T23:29:08.218455+0000","last_clean":"2024-04-24T23:29:08.218455+0000","last_became_active":"2024-04-24T23:27:24.718523+0000","last_became_peered":"2024-04-24T23:27:24.718523+0000","last_unstale":"2024-04-24T23:29:08.218455+0000","last_undegraded":"2024-04-24T23:29:08.218455+0000","last_fullsized":"2024-04-24T23:29:08.218455+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:27:57.396800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.221356+0000","last_change":"2024-04-24T23:27:18.975034+0000","last_active":"2024-04-24T23:29:08.221356+0000","last_peered":"2024-04-24T23:29:08.221356+0000","last_clean":"2024-04-24T23:29:08.221356+0000","last_became_active":"2024-04-24T23:27:18.973764+0000","last_became_peered":"2024-04-24T23:27:18.973764+0000","last_unstale":"2024-04-24T23:29:08.221356+0000","last_undegraded":"2024-04-24T23:29:08.221356+0000","last_fullsized":"2024-04-24T23:29:08.221356+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:43:23.451904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"92'5","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.221861+0000","last_change":"2024-04-24T23:27:19.984948+0000","last_active":"2024-04-24T23:29:08.221861+0000","last_peered":"2024-04-24T23:29:08.221861+0000","last_clean":"2024-04-24T23:29:08.221861+0000","last_became_active":"2024-04-24T23:27:19.984484+0000","last_became_peered":"2024-04-24T23:27:19.984484+0000","last_unstale":"2024-04-24T23:29:08.221861+0000","last_undegraded":"2024-04-24T23:29:08.221861+0000","last_fullsized":"2024-04-24T23:29:08.221861+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:17:43.335983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.251471+0000","last_change":"2024-04-24T23:29:08.250873+0000","last_active":"2024-04-24T23:29:08.251471+0000","last_peered":"2024-04-24T23:29:08.251471+0000","last_clean":"2024-04-24T23:29:08.251471+0000","last_became_active":"2024-04-24T23:29:08.250147+0000","last_became_peered":"2024-04-24T23:29:08.250147+0000","last_unstale":"2024-04-24T23:29:08.251471+0000","last_undegraded":"2024-04-24T23:29:08.251471+0000","last_fullsized":"2024-04-24T23:29:08.251471+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:14:42.358098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6,7],"acting":[1,8,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.214665+0000","last_change":"2024-04-24T23:27:18.964232+0000","last_active":"2024-04-24T23:29:08.214665+0000","last_peered":"2024-04-24T23:29:08.214665+0000","last_clean":"2024-04-24T23:29:08.214665+0000","last_became_active":"2024-04-24T23:27:18.963855+0000","last_became_peered":"2024-04-24T23:27:18.963855+0000","last_unstale":"2024-04-24T23:29:08.214665+0000","last_undegraded":"2024-04-24T23:29:08.214665+0000","last_fullsized":"2024-04-24T23:29:08.214665+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:02:21.000326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"92'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.222653+0000","last_change":"2024-04-24T23:27:19.984354+0000","last_active":"2024-04-24T23:29:08.222653+0000","last_peered":"2024-04-24T23:29:08.222653+0000","last_clean":"2024-04-24T23:29:08.222653+0000","last_became_active":"2024-04-24T23:27:19.983972+0000","last_became_peered":"2024-04-24T23:27:19.983972+0000","last_unstale":"2024-04-24T23:29:08.222653+0000","last_undegraded":"2024-04-24T23:29:08.222653+0000","last_fullsized":"2024-04-24T23:29:08.222653+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:28:28.390283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.387070+0000","last_change":"2024-04-24T23:27:24.238404+0000","last_active":"2024-04-24T23:29:08.387070+0000","last_peered":"2024-04-24T23:29:08.387070+0000","last_clean":"2024-04-24T23:29:08.387070+0000","last_became_active":"2024-04-24T23:27:24.237976+0000","last_became_peered":"2024-04-24T23:27:24.237976+0000","last_unstale":"2024-04-24T23:29:08.387070+0000","last_undegraded":"2024-04-24T23:29:08.387070+0000","last_fullsized":"2024-04-24T23:29:08.387070+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:19:31.476789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.249641+0000","last_change":"2024-04-24T23:27:18.971259+0000","last_active":"2024-04-24T23:29:08.249641+0000","last_peered":"2024-04-24T23:29:08.249641+0000","last_clean":"2024-04-24T23:29:08.249641+0000","last_became_active":"2024-04-24T23:27:18.970856+0000","last_became_peered":"2024-04-24T23:27:18.970856+0000","last_unstale":"2024-04-24T23:29:08.249641+0000","last_undegraded":"2024-04-24T23:29:08.249641+0000","last_fullsized":"2024-04-24T23:29:08.249641+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:54:58.642528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"92'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.222288+0000","last_change":"2024-04-24T23:27:19.986320+0000","last_active":"2024-04-24T23:29:08.222288+0000","last_peered":"2024-04-24T23:29:08.222288+0000","last_clean":"2024-04-24T23:29:08.222288+0000","last_became_active":"2024-04-24T23:27:19.985951+0000","last_became_peered":"2024-04-24T23:27:19.985951+0000","last_unstale":"2024-04-24T23:29:08.222288+0000","last_undegraded":"2024-04-24T23:29:08.222288+0000","last_fullsized":"2024-04-24T23:29:08.222288+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:05:35.427735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.248604+0000","last_change":"2024-04-24T23:27:24.228149+0000","last_active":"2024-04-24T23:29:08.248604+0000","last_peered":"2024-04-24T23:29:08.248604+0000","last_clean":"2024-04-24T23:29:08.248604+0000","last_became_active":"2024-04-24T23:27:24.227683+0000","last_became_peered":"2024-04-24T23:27:24.227683+0000","last_unstale":"2024-04-24T23:29:08.248604+0000","last_undegraded":"2024-04-24T23:29:08.248604+0000","last_fullsized":"2024-04-24T23:29:08.248604+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:23.196065+0000","last_clean_scrub_stamp":"2024-04-24T23:27:23.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:55:50.408834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":46,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.249519+0000","last_change":"2024-04-24T23:28:08.059374+0000","last_active":"2024-04-24T23:29:08.249519+0000","last_peered":"2024-04-24T23:29:08.249519+0000","last_clean":"2024-04-24T23:29:08.249519+0000","last_became_active":"2024-04-24T23:28:08.056104+0000","last_became_peered":"2024-04-24T23:28:08.056104+0000","last_unstale":"2024-04-24T23:29:08.249519+0000","last_undegraded":"2024-04-24T23:29:08.249519+0000","last_fullsized":"2024-04-24T23:29:08.249519+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:17.942613+0000","last_clean_scrub_stamp":"2024-04-24T23:27:17.942613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:07:17.185948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"92'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.222499+0000","last_change":"2024-04-24T23:27:19.990935+0000","last_active":"2024-04-24T23:29:08.222499+0000","last_peered":"2024-04-24T23:29:08.222499+0000","last_clean":"2024-04-24T23:29:08.222499+0000","last_became_active":"2024-04-24T23:27:19.990524+0000","last_became_peered":"2024-04-24T23:27:19.990524+0000","last_unstale":"2024-04-24T23:29:08.222499+0000","last_undegraded":"2024-04-24T23:29:08.222499+0000","last_fullsized":"2024-04-24T23:29:08.222499+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:54:02.463248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"92'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.216879+0000","last_change":"2024-04-24T23:27:19.979142+0000","last_active":"2024-04-24T23:29:08.216879+0000","last_peered":"2024-04-24T23:29:08.216879+0000","last_clean":"2024-04-24T23:29:08.216879+0000","last_became_active":"2024-04-24T23:27:19.978799+0000","last_became_peered":"2024-04-24T23:27:19.978799+0000","last_unstale":"2024-04-24T23:29:08.216879+0000","last_undegraded":"2024-04-24T23:29:08.216879+0000","last_fullsized":"2024-04-24T23:29:08.216879+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:07:19.046313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"92'12","reported_seq":50,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.214206+0000","last_change":"2024-04-24T23:28:08.109820+0000","last_active":"2024-04-24T23:29:08.214206+0000","last_peered":"2024-04-24T23:29:08.214206+0000","last_clean":"2024-04-24T23:29:08.214206+0000","last_became_active":"2024-04-24T23:28:08.109500+0000","last_became_peered":"2024-04-24T23:28:08.109500+0000","last_unstale":"2024-04-24T23:29:08.214206+0000","last_undegraded":"2024-04-24T23:29:08.214206+0000","last_fullsized":"2024-04-24T23:29:08.214206+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:34:32.814256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"92'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.214320+0000","last_change":"2024-04-24T23:27:19.980008+0000","last_active":"2024-04-24T23:29:08.214320+0000","last_peered":"2024-04-24T23:29:08.214320+0000","last_clean":"2024-04-24T23:29:08.214320+0000","last_became_active":"2024-04-24T23:27:19.979424+0000","last_became_peered":"2024-04-24T23:27:19.979424+0000","last_unstale":"2024-04-24T23:29:08.214320+0000","last_undegraded":"2024-04-24T23:29:08.214320+0000","last_fullsized":"2024-04-24T23:29:08.214320+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:44.449245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"92'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.387685+0000","last_change":"2024-04-24T23:27:19.987574+0000","last_active":"2024-04-24T23:29:08.387685+0000","last_peered":"2024-04-24T23:29:08.387685+0000","last_clean":"2024-04-24T23:29:08.387685+0000","last_became_active":"2024-04-24T23:27:19.987242+0000","last_became_peered":"2024-04-24T23:27:19.987242+0000","last_unstale":"2024-04-24T23:29:08.387685+0000","last_undegraded":"2024-04-24T23:29:08.387685+0000","last_fullsized":"2024-04-24T23:29:08.387685+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:39:10.777152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"92'4","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.249324+0000","last_change":"2024-04-24T23:27:19.980983+0000","last_active":"2024-04-24T23:29:08.249324+0000","last_peered":"2024-04-24T23:29:08.249324+0000","last_clean":"2024-04-24T23:29:08.249324+0000","last_became_active":"2024-04-24T23:27:19.980659+0000","last_became_peered":"2024-04-24T23:27:19.980659+0000","last_unstale":"2024-04-24T23:29:08.249324+0000","last_undegraded":"2024-04-24T23:29:08.249324+0000","last_fullsized":"2024-04-24T23:29:08.249324+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:02:52.647687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"92'6","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.217260+0000","last_change":"2024-04-24T23:28:08.055819+0000","last_active":"2024-04-24T23:29:08.217260+0000","last_peered":"2024-04-24T23:29:08.217260+0000","last_clean":"2024-04-24T23:29:08.217260+0000","last_became_active":"2024-04-24T23:28:08.055386+0000","last_became_peered":"2024-04-24T23:28:08.055386+0000","last_unstale":"2024-04-24T23:29:08.217260+0000","last_undegraded":"2024-04-24T23:29:08.217260+0000","last_fullsized":"2024-04-24T23:29:08.217260+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:59:30.488812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"92'15","reported_seq":47,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.214979+0000","last_change":"2024-04-24T23:27:19.985864+0000","last_active":"2024-04-24T23:29:08.214979+0000","last_peered":"2024-04-24T23:29:08.214979+0000","last_clean":"2024-04-24T23:29:08.214979+0000","last_became_active":"2024-04-24T23:27:19.984877+0000","last_became_peered":"2024-04-24T23:27:19.984877+0000","last_unstale":"2024-04-24T23:29:08.214979+0000","last_undegraded":"2024-04-24T23:29:08.214979+0000","last_fullsized":"2024-04-24T23:29:08.214979+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:14:43.458970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"92'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.213153+0000","last_change":"2024-04-24T23:27:19.988831+0000","last_active":"2024-04-24T23:29:08.213153+0000","last_peered":"2024-04-24T23:29:08.213153+0000","last_clean":"2024-04-24T23:29:08.213153+0000","last_became_active":"2024-04-24T23:27:19.988478+0000","last_became_peered":"2024-04-24T23:27:19.988478+0000","last_unstale":"2024-04-24T23:29:08.213153+0000","last_undegraded":"2024-04-24T23:29:08.213153+0000","last_fullsized":"2024-04-24T23:29:08.213153+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:18:37.488710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"92'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.249789+0000","last_change":"2024-04-24T23:27:19.983553+0000","last_active":"2024-04-24T23:29:08.249789+0000","last_peered":"2024-04-24T23:29:08.249789+0000","last_clean":"2024-04-24T23:29:08.249789+0000","last_became_active":"2024-04-24T23:27:19.981042+0000","last_became_peered":"2024-04-24T23:27:19.981042+0000","last_unstale":"2024-04-24T23:29:08.249789+0000","last_undegraded":"2024-04-24T23:29:08.249789+0000","last_fullsized":"2024-04-24T23:29:08.249789+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:09:42.608238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"92'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.212461+0000","last_change":"2024-04-24T23:27:19.985425+0000","last_active":"2024-04-24T23:29:08.212461+0000","last_peered":"2024-04-24T23:29:08.212461+0000","last_clean":"2024-04-24T23:29:08.212461+0000","last_became_active":"2024-04-24T23:27:19.985094+0000","last_became_peered":"2024-04-24T23:27:19.985094+0000","last_unstale":"2024-04-24T23:29:08.212461+0000","last_undegraded":"2024-04-24T23:29:08.212461+0000","last_fullsized":"2024-04-24T23:29:08.212461+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:07:15.612247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"92'10","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.220939+0000","last_change":"2024-04-24T23:27:19.988224+0000","last_active":"2024-04-24T23:29:08.220939+0000","last_peered":"2024-04-24T23:29:08.220939+0000","last_clean":"2024-04-24T23:29:08.220939+0000","last_became_active":"2024-04-24T23:27:19.987820+0000","last_became_peered":"2024-04-24T23:27:19.987820+0000","last_unstale":"2024-04-24T23:29:08.220939+0000","last_undegraded":"2024-04-24T23:29:08.220939+0000","last_fullsized":"2024-04-24T23:29:08.220939+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:27.653290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"92'11","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.249349+0000","last_change":"2024-04-24T23:27:19.989672+0000","last_active":"2024-04-24T23:29:08.249349+0000","last_peered":"2024-04-24T23:29:08.249349+0000","last_clean":"2024-04-24T23:29:08.249349+0000","last_became_active":"2024-04-24T23:27:19.989337+0000","last_became_peered":"2024-04-24T23:27:19.989337+0000","last_unstale":"2024-04-24T23:29:08.249349+0000","last_undegraded":"2024-04-24T23:29:08.249349+0000","last_fullsized":"2024-04-24T23:29:08.249349+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:07:13.399366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"92'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:29:08.216974+0000","last_change":"2024-04-24T23:27:19.987703+0000","last_active":"2024-04-24T23:29:08.216974+0000","last_peered":"2024-04-24T23:29:08.216974+0000","last_clean":"2024-04-24T23:29:08.216974+0000","last_became_active":"2024-04-24T23:27:19.987355+0000","last_became_peered":"2024-04-24T23:27:19.987355+0000","last_unstale":"2024-04-24T23:29:08.216974+0000","last_undegraded":"2024-04-24T23:29:08.216974+0000","last_fullsized":"2024-04-24T23:29:08.216974+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:27:18.947337+0000","last_clean_scrub_stamp":"2024-04-24T23:27:18.947337+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:03:15.812726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":280134,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":64410,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":741,"num_read_kb":441,"num_write":176,"num_write_kb":124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":694,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":193230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":152,"ondisk_log_size":152,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383724,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1036288,"data_stored":529776,"data_compressed":29657,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645360,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28168,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716984,"statfs":{"total":95995035648,"available":95966191616,"internally_reserved":0,"allocated":1179648,"data_stored":1180826,"data_compressed":65288,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808886,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32208,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712944,"statfs":{"total":95995035648,"available":95962054656,"internally_reserved":0,"allocated":1048576,"data_stored":529430,"data_compressed":29657,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070524,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1048576,"data_stored":529350,"data_compressed":29657,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332160,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28088,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717064,"statfs":{"total":95995035648,"available":95966273536,"internally_reserved":0,"allocated":1093632,"data_stored":1125658,"data_compressed":65288,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495686,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1028096,"data_stored":524606,"data_compressed":29657,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574943,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36412,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708740,"statfs":{"total":95995035648,"available":95957749760,"internally_reserved":0,"allocated":1101824,"data_stored":584056,"data_compressed":29657,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738467,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1052672,"data_stored":525994,"data_compressed":29657,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411418,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32260,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712892,"statfs":{"total":95995035648,"available":95962001408,"internally_reserved":0,"allocated":1110016,"data_stored":585714,"data_compressed":29657,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182485,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1040384,"data_stored":524800,"data_compressed":29657,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920848,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1040384,"data_stored":530096,"data_compressed":29657,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757323,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1105920,"data_stored":1120988,"data_compressed":65288,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33993,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33993,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33993,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35631,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35631,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35631,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35631,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35631,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":59540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":61198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":61846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T23:29:47.422 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:29:47.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:29:47 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/3621144815' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T23:29:47.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:29:47 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/905843497' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:29:47.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:29:47 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/3621144815' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T23:29:47.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:29:47 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/905843497' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:29:47.554 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.551+0000 7f83d9b3b640 1 -- 172.21.15.80:0/1090710782 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f83d4100680 msgr2=0x7f83d4100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:47.555 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.551+0000 7f83d9b3b640 1 --2- 172.21.15.80:0/1090710782 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f83d4100680 0x7f83d4100aa0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f83c4009900 tx=0x7f83c402a410 comp rx=0 tx=0).stop 2024-04-24T23:29:47.555 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.551+0000 7f83d9b3b640 1 -- 172.21.15.80:0/1090710782 shutdown_connections 2024-04-24T23:29:47.555 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.551+0000 7f83d9b3b640 1 --2- 172.21.15.80:0/1090710782 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f83d4102230 0x7f83d4108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:47.555 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.551+0000 7f83d9b3b640 1 --2- 172.21.15.80:0/1090710782 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83d4101870 0x7f83d4101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:47.555 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.551+0000 7f83d9b3b640 1 --2- 172.21.15.80:0/1090710782 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f83d4100680 0x7f83d4100aa0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:47.555 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.551+0000 7f83d9b3b640 1 -- 172.21.15.80:0/1090710782 >> 172.21.15.80:0/1090710782 conn(0x7f83d40fbbf0 msgr2=0x7f83d40fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:47.555 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.551+0000 7f83d9b3b640 1 -- 172.21.15.80:0/1090710782 shutdown_connections 2024-04-24T23:29:47.555 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.551+0000 7f83d9b3b640 1 -- 172.21.15.80:0/1090710782 wait complete. 2024-04-24T23:29:47.555 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.552+0000 7f83d9b3b640 1 Processor -- start 2024-04-24T23:29:47.555 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.552+0000 7f83d9b3b640 1 -- start start 2024-04-24T23:29:47.555 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.552+0000 7f83d9b3b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f83d4100680 0x7f83d4196b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:47.556 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.552+0000 7f83d9b3b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83d4101870 0x7f83d4197040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:47.556 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.552+0000 7f83d9b3b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f83d4102230 0x7f83d419e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:47.556 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.552+0000 7f83d9b3b640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f83d410c3d0 con 0x7f83d4102230 2024-04-24T23:29:47.556 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.552+0000 7f83d9b3b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f83d410c250 con 0x7f83d4100680 2024-04-24T23:29:47.556 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.552+0000 7f83d9b3b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f83d410c550 con 0x7f83d4101870 2024-04-24T23:29:47.556 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.552+0000 7f83d3fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f83d4102230 0x7f83d419e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:47.556 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.552+0000 7f83d3fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f83d4102230 0x7f83d419e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:52074/0 (socket says 172.21.15.80:52074) 2024-04-24T23:29:47.556 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.552+0000 7f83d3fff640 1 -- 172.21.15.80:0/885699762 learned_addr learned my addr 172.21.15.80:0/885699762 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:29:47.556 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.552+0000 7f83caffd640 1 --2- 172.21.15.80:0/885699762 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83d4101870 0x7f83d4197040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:47.556 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.553+0000 7f83d37fe640 1 --2- 172.21.15.80:0/885699762 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f83d4100680 0x7f83d4196b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:47.556 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.553+0000 7f83d3fff640 1 -- 172.21.15.80:0/885699762 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83d4101870 msgr2=0x7f83d4197040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:47.556 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.553+0000 7f83d3fff640 1 --2- 172.21.15.80:0/885699762 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83d4101870 0x7f83d4197040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:47.556 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.553+0000 7f83d3fff640 1 -- 172.21.15.80:0/885699762 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f83d4100680 msgr2=0x7f83d4196b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:47.556 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.553+0000 7f83d3fff640 1 --2- 172.21.15.80:0/885699762 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f83d4100680 0x7f83d4196b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:47.557 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.553+0000 7f83d3fff640 1 -- 172.21.15.80:0/885699762 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83d419e720 con 0x7f83d4102230 2024-04-24T23:29:47.557 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.553+0000 7f83d3fff640 1 --2- 172.21.15.80:0/885699762 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f83d4102230 0x7f83d419e0b0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f83cc00b970 tx=0x7f83cc00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:47.557 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.553+0000 7f83d17fa640 1 -- 172.21.15.80:0/885699762 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83cc019070 con 0x7f83d4102230 2024-04-24T23:29:47.557 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.553+0000 7f83d9b3b640 1 -- 172.21.15.80:0/885699762 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83d419ea10 con 0x7f83d4102230 2024-04-24T23:29:47.557 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.553+0000 7f83d17fa640 1 -- 172.21.15.80:0/885699762 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83cc00cb40 con 0x7f83d4102230 2024-04-24T23:29:47.557 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.553+0000 7f83d17fa640 1 -- 172.21.15.80:0/885699762 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83cc007650 con 0x7f83d4102230 2024-04-24T23:29:47.557 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.553+0000 7f83d9b3b640 1 -- 172.21.15.80:0/885699762 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83d419ef90 con 0x7f83d4102230 2024-04-24T23:29:47.557 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.554+0000 7f83d9b3b640 1 -- 172.21.15.80:0/885699762 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83d4107a70 con 0x7f83d4102230 2024-04-24T23:29:47.558 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.555+0000 7f83d17fa640 1 -- 172.21.15.80:0/885699762 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f83cc01c070 con 0x7f83d4102230 2024-04-24T23:29:47.558 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.556+0000 7f83d17fa640 1 --2- 172.21.15.80:0/885699762 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f83c00a2b90 0x7f83c00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:47.558 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.556+0000 7f83d37fe640 1 --2- 172.21.15.80:0/885699762 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f83c00a2b90 0x7f83c00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:47.558 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.556+0000 7f83d17fa640 1 -- 172.21.15.80:0/885699762 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f83cc0cef20 con 0x7f83d4102230 2024-04-24T23:29:47.558 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.556+0000 7f83d37fe640 1 --2- 172.21.15.80:0/885699762 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f83c00a2b90 0x7f83c00a5050 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f83c4031610 tx=0x7f83c4000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:47.559 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.559+0000 7f83d17fa640 1 -- 172.21.15.80:0/885699762 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f83cc09bab0 con 0x7f83d4102230 2024-04-24T23:29:47.735 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.734+0000 7f83d9b3b640 1 -- 172.21.15.80:0/885699762 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f83d4062340 con 0x7f83d4102230 2024-04-24T23:29:47.737 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.736+0000 7f83d17fa640 1 -- 172.21.15.80:0/885699762 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24839 (secure 0 0 0) 0x7f83cc09b3b0 con 0x7f83d4102230 2024-04-24T23:29:47.737 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-24T23:29:47.737 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":94,"fsid":"ec7885aa-0290-11ef-bc93-c7b262605968","created":"2024-04-24T23:19:05.586407+0000","modified":"2024-04-24T23:29:08.209684+0000","last_up_change":"2024-04-24T23:26:25.938316+0000","last_in_change":"2024-04-24T23:26:04.786699+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:22:33.124428+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:27:17.575788+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:27:18.492762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T23:27:22.438503+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a525741d-b46c-4eef-aa64-3f057cac1497","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":2597718112},{"type":"v1","addr":"172.21.15.80:6803","nonce":2597718112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":2597718112},{"type":"v1","addr":"172.21.15.80:6805","nonce":2597718112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2597718112},{"type":"v1","addr":"172.21.15.80:6809","nonce":2597718112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":2597718112},{"type":"v1","addr":"172.21.15.80:6807","nonce":2597718112}]},"public_addr":"172.21.15.80:6803/2597718112","cluster_addr":"172.21.15.80:6805/2597718112","heartbeat_back_addr":"172.21.15.80:6809/2597718112","heartbeat_front_addr":"172.21.15.80:6807/2597718112","state":["exists","up"]},{"osd":1,"uuid":"2d0f0d67-6db4-40ac-91cc-58604ab6795a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":2395610201},{"type":"v1","addr":"172.21.15.134:6801","nonce":2395610201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":2395610201},{"type":"v1","addr":"172.21.15.134:6803","nonce":2395610201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":2395610201},{"type":"v1","addr":"172.21.15.134:6807","nonce":2395610201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":2395610201},{"type":"v1","addr":"172.21.15.134:6805","nonce":2395610201}]},"public_addr":"172.21.15.134:6801/2395610201","cluster_addr":"172.21.15.134:6803/2395610201","heartbeat_back_addr":"172.21.15.134:6807/2395610201","heartbeat_front_addr":"172.21.15.134:6805/2395610201","state":["exists","up"]},{"osd":2,"uuid":"781fb879-fedc-4c84-82fb-59e073bc4cf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1629152187},{"type":"v1","addr":"172.21.15.148:6801","nonce":1629152187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1629152187},{"type":"v1","addr":"172.21.15.148:6803","nonce":1629152187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1629152187},{"type":"v1","addr":"172.21.15.148:6807","nonce":1629152187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1629152187},{"type":"v1","addr":"172.21.15.148:6805","nonce":1629152187}]},"public_addr":"172.21.15.148:6801/1629152187","cluster_addr":"172.21.15.148:6803/1629152187","heartbeat_back_addr":"172.21.15.148:6807/1629152187","heartbeat_front_addr":"172.21.15.148:6805/1629152187","state":["exists","up"]},{"osd":3,"uuid":"cd982757-f2f6-4296-b3e7-4cd40fbd8d9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":3128298510},{"type":"v1","addr":"172.21.15.80:6811","nonce":3128298510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":3128298510},{"type":"v1","addr":"172.21.15.80:6813","nonce":3128298510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":3128298510},{"type":"v1","addr":"172.21.15.80:6817","nonce":3128298510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":3128298510},{"type":"v1","addr":"172.21.15.80:6815","nonce":3128298510}]},"public_addr":"172.21.15.80:6811/3128298510","cluster_addr":"172.21.15.80:6813/3128298510","heartbeat_back_addr":"172.21.15.80:6817/3128298510","heartbeat_front_addr":"172.21.15.80:6815/3128298510","state":["exists","up"]},{"osd":4,"uuid":"dfc55a79-a7e8-417a-b898-ec44171a54af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1917556060},{"type":"v1","addr":"172.21.15.134:6809","nonce":1917556060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1917556060},{"type":"v1","addr":"172.21.15.134:6811","nonce":1917556060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1917556060},{"type":"v1","addr":"172.21.15.134:6815","nonce":1917556060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1917556060},{"type":"v1","addr":"172.21.15.134:6813","nonce":1917556060}]},"public_addr":"172.21.15.134:6809/1917556060","cluster_addr":"172.21.15.134:6811/1917556060","heartbeat_back_addr":"172.21.15.134:6815/1917556060","heartbeat_front_addr":"172.21.15.134:6813/1917556060","state":["exists","up"]},{"osd":5,"uuid":"b5491337-0694-440c-b679-312f3d14cf65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":2715951281},{"type":"v1","addr":"172.21.15.148:6809","nonce":2715951281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":2715951281},{"type":"v1","addr":"172.21.15.148:6811","nonce":2715951281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":2715951281},{"type":"v1","addr":"172.21.15.148:6815","nonce":2715951281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":2715951281},{"type":"v1","addr":"172.21.15.148:6813","nonce":2715951281}]},"public_addr":"172.21.15.148:6809/2715951281","cluster_addr":"172.21.15.148:6811/2715951281","heartbeat_back_addr":"172.21.15.148:6815/2715951281","heartbeat_front_addr":"172.21.15.148:6813/2715951281","state":["exists","up"]},{"osd":6,"uuid":"ba4b5df7-d86e-47cd-ab67-cf763c4e4d6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":953261724},{"type":"v1","addr":"172.21.15.80:6819","nonce":953261724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":953261724},{"type":"v1","addr":"172.21.15.80:6821","nonce":953261724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":953261724},{"type":"v1","addr":"172.21.15.80:6825","nonce":953261724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":953261724},{"type":"v1","addr":"172.21.15.80:6823","nonce":953261724}]},"public_addr":"172.21.15.80:6819/953261724","cluster_addr":"172.21.15.80:6821/953261724","heartbeat_back_addr":"172.21.15.80:6825/953261724","heartbeat_front_addr":"172.21.15.80:6823/953261724","state":["exists","up"]},{"osd":7,"uuid":"464a503a-75ae-49e7-ab6e-a7f9f2a6a307","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":108354844},{"type":"v1","addr":"172.21.15.134:6817","nonce":108354844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":108354844},{"type":"v1","addr":"172.21.15.134:6819","nonce":108354844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":108354844},{"type":"v1","addr":"172.21.15.134:6823","nonce":108354844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":108354844},{"type":"v1","addr":"172.21.15.134:6821","nonce":108354844}]},"public_addr":"172.21.15.134:6817/108354844","cluster_addr":"172.21.15.134:6819/108354844","heartbeat_back_addr":"172.21.15.134:6823/108354844","heartbeat_front_addr":"172.21.15.134:6821/108354844","state":["exists","up"]},{"osd":8,"uuid":"ce544b2e-4540-47c0-9997-500c45351330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3559655329},{"type":"v1","addr":"172.21.15.148:6817","nonce":3559655329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":3559655329},{"type":"v1","addr":"172.21.15.148:6819","nonce":3559655329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":3559655329},{"type":"v1","addr":"172.21.15.148:6823","nonce":3559655329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3559655329},{"type":"v1","addr":"172.21.15.148:6821","nonce":3559655329}]},"public_addr":"172.21.15.148:6817/3559655329","cluster_addr":"172.21.15.148:6819/3559655329","heartbeat_back_addr":"172.21.15.148:6823/3559655329","heartbeat_front_addr":"172.21.15.148:6821/3559655329","state":["exists","up"]},{"osd":9,"uuid":"06de3732-19b9-496c-a464-55322efa6759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":4019913545},{"type":"v1","addr":"172.21.15.80:6827","nonce":4019913545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":4019913545},{"type":"v1","addr":"172.21.15.80:6829","nonce":4019913545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":4019913545},{"type":"v1","addr":"172.21.15.80:6833","nonce":4019913545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":4019913545},{"type":"v1","addr":"172.21.15.80:6831","nonce":4019913545}]},"public_addr":"172.21.15.80:6827/4019913545","cluster_addr":"172.21.15.80:6829/4019913545","heartbeat_back_addr":"172.21.15.80:6833/4019913545","heartbeat_front_addr":"172.21.15.80:6831/4019913545","state":["exists","up"]},{"osd":10,"uuid":"8d92f0d9-5c64-429a-b750-c33ae0e070c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":647343766},{"type":"v1","addr":"172.21.15.134:6825","nonce":647343766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":647343766},{"type":"v1","addr":"172.21.15.134:6827","nonce":647343766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":647343766},{"type":"v1","addr":"172.21.15.134:6831","nonce":647343766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":647343766},{"type":"v1","addr":"172.21.15.134:6829","nonce":647343766}]},"public_addr":"172.21.15.134:6825/647343766","cluster_addr":"172.21.15.134:6827/647343766","heartbeat_back_addr":"172.21.15.134:6831/647343766","heartbeat_front_addr":"172.21.15.134:6829/647343766","state":["exists","up"]},{"osd":11,"uuid":"4c003a41-f65b-479f-bc80-b27d680cf936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":4068024116},{"type":"v1","addr":"172.21.15.148:6825","nonce":4068024116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":4068024116},{"type":"v1","addr":"172.21.15.148:6827","nonce":4068024116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":4068024116},{"type":"v1","addr":"172.21.15.148:6831","nonce":4068024116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":4068024116},{"type":"v1","addr":"172.21.15.148:6829","nonce":4068024116}]},"public_addr":"172.21.15.148:6825/4068024116","cluster_addr":"172.21.15.148:6827/4068024116","heartbeat_back_addr":"172.21.15.148:6831/4068024116","heartbeat_front_addr":"172.21.15.148:6829/4068024116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:21:42.433631+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:22:06.145933+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:22:30.726143+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:22:55.729961+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:23:21.280913+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:23:46.609983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:24:12.585247+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:24:38.359287+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:25:04.491232+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:25:30.788089+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:25:56.891078+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:26:23.899113+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":7}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"4.a","mappings":[{"from":5,"to":8}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.80:0/2828378749":"2024-04-25T23:20:06.793609+0000","172.21.15.80:6801/233364790":"2024-04-25T23:20:06.793609+0000","172.21.15.80:6800/1614691410":"2024-04-25T23:19:40.490692+0000","172.21.15.80:0/1406187145":"2024-04-25T23:20:06.793609+0000","172.21.15.80:0/92953396":"2024-04-25T23:19:40.490692+0000","172.21.15.80:0/598129432":"2024-04-25T23:20:06.793609+0000","172.21.15.80:0/232193118":"2024-04-25T23:19:40.490692+0000","172.21.15.80:6800/233364790":"2024-04-25T23:20:06.793609+0000","172.21.15.80:6801/1614691410":"2024-04-25T23:19:40.490692+0000","172.21.15.80:0/2081562529":"2024-04-25T23:19:40.490692+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:29:47.739 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.738+0000 7f83d9b3b640 1 -- 172.21.15.80:0/885699762 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f83c00a2b90 msgr2=0x7f83c00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:47.739 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.738+0000 7f83d9b3b640 1 --2- 172.21.15.80:0/885699762 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f83c00a2b90 0x7f83c00a5050 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f83c4031610 tx=0x7f83c4000bc0 comp rx=0 tx=0).stop 2024-04-24T23:29:47.739 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.738+0000 7f83d9b3b640 1 -- 172.21.15.80:0/885699762 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f83d4102230 msgr2=0x7f83d419e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:47.739 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.738+0000 7f83d9b3b640 1 --2- 172.21.15.80:0/885699762 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f83d4102230 0x7f83d419e0b0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f83cc00b970 tx=0x7f83cc00be30 comp rx=0 tx=0).stop 2024-04-24T23:29:47.739 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.738+0000 7f83d9b3b640 1 -- 172.21.15.80:0/885699762 shutdown_connections 2024-04-24T23:29:47.740 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.738+0000 7f83d9b3b640 1 --2- 172.21.15.80:0/885699762 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f83c00a2b90 0x7f83c00a5050 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:47.740 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.738+0000 7f83d9b3b640 1 --2- 172.21.15.80:0/885699762 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f83d4102230 0x7f83d419e0b0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:47.740 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.738+0000 7f83d9b3b640 1 --2- 172.21.15.80:0/885699762 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83d4101870 0x7f83d4197040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:47.740 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.738+0000 7f83d9b3b640 1 --2- 172.21.15.80:0/885699762 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f83d4100680 0x7f83d4196b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:47.740 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.738+0000 7f83d9b3b640 1 -- 172.21.15.80:0/885699762 >> 172.21.15.80:0/885699762 conn(0x7f83d40fbbf0 msgr2=0x7f83d4104c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:47.740 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.738+0000 7f83d9b3b640 1 -- 172.21.15.80:0/885699762 shutdown_connections 2024-04-24T23:29:47.740 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.738+0000 7f83d9b3b640 1 -- 172.21.15.80:0/885699762 wait complete. 2024-04-24T23:29:47.754 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:22:33.124428+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T23:27:17.575788+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T23:27:18.492762+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-24T23:27:22.438503+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-24T23:29:47.755 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:29:47.861 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.855+0000 7f56c415a640 1 -- 172.21.15.80:0/1309803569 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f56bc102250 msgr2=0x7f56bc108ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:47.861 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.855+0000 7f56c415a640 1 --2- 172.21.15.80:0/1309803569 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f56bc102250 0x7f56bc108ae0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f56b4009ca0 tx=0x7f56b402a8c0 comp rx=0 tx=0).stop 2024-04-24T23:29:47.861 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.855+0000 7f56c415a640 1 -- 172.21.15.80:0/1309803569 shutdown_connections 2024-04-24T23:29:47.862 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.855+0000 7f56c415a640 1 --2- 172.21.15.80:0/1309803569 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f56bc102250 0x7f56bc108ae0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:47.862 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.855+0000 7f56c415a640 1 --2- 172.21.15.80:0/1309803569 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56bc101890 0x7f56bc101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:47.862 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.855+0000 7f56c415a640 1 --2- 172.21.15.80:0/1309803569 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56bc1006a0 0x7f56bc100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:47.862 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.855+0000 7f56c415a640 1 -- 172.21.15.80:0/1309803569 >> 172.21.15.80:0/1309803569 conn(0x7f56bc0fbc30 msgr2=0x7f56bc0fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:47.862 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.855+0000 7f56c415a640 1 -- 172.21.15.80:0/1309803569 shutdown_connections 2024-04-24T23:29:47.862 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.855+0000 7f56c415a640 1 -- 172.21.15.80:0/1309803569 wait complete. 2024-04-24T23:29:47.863 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.855+0000 7f56c415a640 1 Processor -- start 2024-04-24T23:29:47.863 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.855+0000 7f56c415a640 1 -- start start 2024-04-24T23:29:47.863 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.855+0000 7f56c415a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56bc1006a0 0x7f56bc1979f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:47.863 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.855+0000 7f56c415a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56bc101890 0x7f56bc197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:47.863 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.855+0000 7f56c415a640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f56bc102250 0x7f56bc19efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:47.863 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.855+0000 7f56c415a640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f56bc10c320 con 0x7f56bc102250 2024-04-24T23:29:47.863 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.855+0000 7f56c415a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f56bc10c1a0 con 0x7f56bc1006a0 2024-04-24T23:29:47.863 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.855+0000 7f56c415a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f56bc10c4a0 con 0x7f56bc101890 2024-04-24T23:29:47.864 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.856+0000 7f56c26f7640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f56bc102250 0x7f56bc19efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:47.864 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.856+0000 7f56c26f7640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f56bc102250 0x7f56bc19efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:52104/0 (socket says 172.21.15.80:52104) 2024-04-24T23:29:47.864 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.856+0000 7f56c26f7640 1 -- 172.21.15.80:0/202659441 learned_addr learned my addr 172.21.15.80:0/202659441 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:29:47.864 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.856+0000 7f56c16f5640 1 --2- 172.21.15.80:0/202659441 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56bc101890 0x7f56bc197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:47.864 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.856+0000 7f56c1ef6640 1 --2- 172.21.15.80:0/202659441 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56bc1006a0 0x7f56bc1979f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:47.864 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.856+0000 7f56c26f7640 1 -- 172.21.15.80:0/202659441 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56bc101890 msgr2=0x7f56bc197f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:47.865 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.856+0000 7f56c26f7640 1 --2- 172.21.15.80:0/202659441 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56bc101890 0x7f56bc197f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:47.865 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.856+0000 7f56c26f7640 1 -- 172.21.15.80:0/202659441 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56bc1006a0 msgr2=0x7f56bc1979f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:47.865 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.856+0000 7f56c26f7640 1 --2- 172.21.15.80:0/202659441 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56bc1006a0 0x7f56bc1979f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:47.865 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.856+0000 7f56c26f7640 1 -- 172.21.15.80:0/202659441 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56bc19f610 con 0x7f56bc102250 2024-04-24T23:29:47.865 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.856+0000 7f56c1ef6640 1 --2- 172.21.15.80:0/202659441 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56bc1006a0 0x7f56bc1979f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:29:47.865 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.856+0000 7f56c26f7640 1 --2- 172.21.15.80:0/202659441 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f56bc102250 0x7f56bc19efa0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f56b4005e30 tx=0x7f56b4031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:47.865 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.856+0000 7f56b2ffd640 1 -- 172.21.15.80:0/202659441 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b4003c30 con 0x7f56bc102250 2024-04-24T23:29:47.866 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.856+0000 7f56b2ffd640 1 -- 172.21.15.80:0/202659441 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56b4003dd0 con 0x7f56bc102250 2024-04-24T23:29:47.866 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.856+0000 7f56b2ffd640 1 -- 172.21.15.80:0/202659441 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b402fde0 con 0x7f56bc102250 2024-04-24T23:29:47.866 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.856+0000 7f56c415a640 1 -- 172.21.15.80:0/202659441 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56bc19f8a0 con 0x7f56bc102250 2024-04-24T23:29:47.866 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.856+0000 7f56c415a640 1 -- 172.21.15.80:0/202659441 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56bc19fe40 con 0x7f56bc102250 2024-04-24T23:29:47.866 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.857+0000 7f56c415a640 1 -- 172.21.15.80:0/202659441 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56bc1a0070 con 0x7f56bc102250 2024-04-24T23:29:47.866 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.858+0000 7f56b2ffd640 1 -- 172.21.15.80:0/202659441 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f56b4004050 con 0x7f56bc102250 2024-04-24T23:29:47.867 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.859+0000 7f56b2ffd640 1 --2- 172.21.15.80:0/202659441 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f56a80a2c10 0x7f56a80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:47.867 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.859+0000 7f56c1ef6640 1 --2- 172.21.15.80:0/202659441 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f56a80a2c10 0x7f56a80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:47.867 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.859+0000 7f56b2ffd640 1 -- 172.21.15.80:0/202659441 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f56b40f0850 con 0x7f56bc102250 2024-04-24T23:29:47.867 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.859+0000 7f56c1ef6640 1 --2- 172.21.15.80:0/202659441 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f56a80a2c10 0x7f56a80a50d0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f56bc1016f0 tx=0x7f56b800a5e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:47.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:47.862+0000 7f56b2ffd640 1 -- 172.21.15.80:0/202659441 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f56b40bd3e0 con 0x7f56bc102250 2024-04-24T23:29:48.041 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.040+0000 7f56c415a640 1 -- 172.21.15.80:0/202659441 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f56bc1a0240 con 0x7f56bc102250 2024-04-24T23:29:48.041 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.040+0000 7f56b2ffd640 1 -- 172.21.15.80:0/202659441 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7f56b40bcce0 con 0x7f56bc102250 2024-04-24T23:29:48.041 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 1 2024-04-24T23:29:48.043 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.042+0000 7f56c415a640 1 -- 172.21.15.80:0/202659441 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f56a80a2c10 msgr2=0x7f56a80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:48.044 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.042+0000 7f56c415a640 1 --2- 172.21.15.80:0/202659441 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f56a80a2c10 0x7f56a80a50d0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f56bc1016f0 tx=0x7f56b800a5e0 comp rx=0 tx=0).stop 2024-04-24T23:29:48.044 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.042+0000 7f56c415a640 1 -- 172.21.15.80:0/202659441 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f56bc102250 msgr2=0x7f56bc19efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:48.044 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.042+0000 7f56c415a640 1 --2- 172.21.15.80:0/202659441 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f56bc102250 0x7f56bc19efa0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f56b4005e30 tx=0x7f56b4031ee0 comp rx=0 tx=0).stop 2024-04-24T23:29:48.044 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.043+0000 7f56c415a640 1 -- 172.21.15.80:0/202659441 shutdown_connections 2024-04-24T23:29:48.044 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.043+0000 7f56c415a640 1 --2- 172.21.15.80:0/202659441 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f56a80a2c10 0x7f56a80a50d0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:48.044 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.043+0000 7f56c415a640 1 --2- 172.21.15.80:0/202659441 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f56bc102250 0x7f56bc19efa0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:48.044 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.043+0000 7f56c415a640 1 --2- 172.21.15.80:0/202659441 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56bc101890 0x7f56bc197f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:48.044 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.043+0000 7f56c415a640 1 --2- 172.21.15.80:0/202659441 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56bc1006a0 0x7f56bc1979f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:48.044 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.043+0000 7f56c415a640 1 -- 172.21.15.80:0/202659441 >> 172.21.15.80:0/202659441 conn(0x7f56bc0fbc30 msgr2=0x7f56bc0fd620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:48.044 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.043+0000 7f56c415a640 1 -- 172.21.15.80:0/202659441 shutdown_connections 2024-04-24T23:29:48.044 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.043+0000 7f56c415a640 1 -- 172.21.15.80:0/202659441 wait complete. 2024-04-24T23:29:48.055 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T23:29:48.158 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.154+0000 7f0f14b7d640 1 -- 172.21.15.80:0/2630997617 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0f10102200 msgr2=0x7f0f101045f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:48.158 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.154+0000 7f0f14b7d640 1 --2- 172.21.15.80:0/2630997617 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0f10102200 0x7f0f101045f0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f0f00009ca0 tx=0x7f0f0002a8c0 comp rx=0 tx=0).stop 2024-04-24T23:29:48.158 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.154+0000 7f0f14b7d640 1 -- 172.21.15.80:0/2630997617 shutdown_connections 2024-04-24T23:29:48.158 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.154+0000 7f0f14b7d640 1 --2- 172.21.15.80:0/2630997617 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0f10107460 0x7f0f101098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:48.158 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.154+0000 7f0f14b7d640 1 --2- 172.21.15.80:0/2630997617 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f10104b30 0x7f0f10106f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:48.158 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.154+0000 7f0f14b7d640 1 --2- 172.21.15.80:0/2630997617 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0f10102200 0x7f0f101045f0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:48.158 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.154+0000 7f0f14b7d640 1 -- 172.21.15.80:0/2630997617 >> 172.21.15.80:0/2630997617 conn(0x7f0f100fbbf0 msgr2=0x7f0f100fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:48.158 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.154+0000 7f0f14b7d640 1 -- 172.21.15.80:0/2630997617 shutdown_connections 2024-04-24T23:29:48.158 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.154+0000 7f0f14b7d640 1 -- 172.21.15.80:0/2630997617 wait complete. 2024-04-24T23:29:48.158 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.155+0000 7f0f14b7d640 1 Processor -- start 2024-04-24T23:29:48.158 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.155+0000 7f0f14b7d640 1 -- start start 2024-04-24T23:29:48.159 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.155+0000 7f0f14b7d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f10102200 0x7f0f10197c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:48.159 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.155+0000 7f0f14b7d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0f10104b30 0x7f0f10198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:48.159 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.155+0000 7f0f14b7d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0f10107460 0x7f0f1019f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:48.159 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.155+0000 7f0f14b7d640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f0f1010d4f0 con 0x7f0f10104b30 2024-04-24T23:29:48.159 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.155+0000 7f0f14b7d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0f1010d370 con 0x7f0f10102200 2024-04-24T23:29:48.162 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.155+0000 7f0f14b7d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f0f1010d670 con 0x7f0f10107460 2024-04-24T23:29:48.162 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.155+0000 7f0f0dd9b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0f10104b30 0x7f0f10198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:48.162 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.155+0000 7f0f0dd9b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0f10104b30 0x7f0f10198180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:52126/0 (socket says 172.21.15.80:52126) 2024-04-24T23:29:48.162 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.155+0000 7f0f0dd9b640 1 -- 172.21.15.80:0/3240928900 learned_addr learned my addr 172.21.15.80:0/3240928900 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:29:48.162 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.156+0000 7f0f0ed9d640 1 --2- 172.21.15.80:0/3240928900 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0f10107460 0x7f0f1019f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:48.162 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.156+0000 7f0f0e59c640 1 --2- 172.21.15.80:0/3240928900 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f10102200 0x7f0f10197c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:48.162 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.156+0000 7f0f0dd9b640 1 -- 172.21.15.80:0/3240928900 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0f10107460 msgr2=0x7f0f1019f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:48.162 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.156+0000 7f0f0dd9b640 1 --2- 172.21.15.80:0/3240928900 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0f10107460 0x7f0f1019f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:48.162 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.156+0000 7f0f0dd9b640 1 -- 172.21.15.80:0/3240928900 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f10102200 msgr2=0x7f0f10197c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:48.162 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.156+0000 7f0f0dd9b640 1 --2- 172.21.15.80:0/3240928900 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f10102200 0x7f0f10197c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:48.162 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.156+0000 7f0f0dd9b640 1 -- 172.21.15.80:0/3240928900 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f1019f850 con 0x7f0f10104b30 2024-04-24T23:29:48.162 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.156+0000 7f0f0ed9d640 1 --2- 172.21.15.80:0/3240928900 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0f10107460 0x7f0f1019f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:29:48.163 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.156+0000 7f0f0dd9b640 1 --2- 172.21.15.80:0/3240928900 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0f10104b30 0x7f0f10198180 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f0f0400b450 tx=0x7f0f0400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:48.163 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.156+0000 7f0eff7fe640 1 -- 172.21.15.80:0/3240928900 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f04015020 con 0x7f0f10104b30 2024-04-24T23:29:48.163 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.156+0000 7f0f14b7d640 1 -- 172.21.15.80:0/3240928900 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f1019fb40 con 0x7f0f10104b30 2024-04-24T23:29:48.163 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.156+0000 7f0eff7fe640 1 -- 172.21.15.80:0/3240928900 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f04011d70 con 0x7f0f10104b30 2024-04-24T23:29:48.163 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.156+0000 7f0eff7fe640 1 -- 172.21.15.80:0/3240928900 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f0400f340 con 0x7f0f10104b30 2024-04-24T23:29:48.163 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.156+0000 7f0f14b7d640 1 -- 172.21.15.80:0/3240928900 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f101a0090 con 0x7f0f10104b30 2024-04-24T23:29:48.163 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.157+0000 7f0f14b7d640 1 -- 172.21.15.80:0/3240928900 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f100fd6e0 con 0x7f0f10104b30 2024-04-24T23:29:48.164 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.158+0000 7f0eff7fe640 1 -- 172.21.15.80:0/3240928900 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0f04018020 con 0x7f0f10104b30 2024-04-24T23:29:48.164 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.159+0000 7f0eff7fe640 1 --2- 172.21.15.80:0/3240928900 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f0ef80a2b90 0x7f0ef80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:48.164 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.159+0000 7f0f0e59c640 1 --2- 172.21.15.80:0/3240928900 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f0ef80a2b90 0x7f0ef80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:48.164 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.159+0000 7f0eff7fe640 1 -- 172.21.15.80:0/3240928900 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f0f040cefb0 con 0x7f0f10104b30 2024-04-24T23:29:48.164 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.159+0000 7f0f0e59c640 1 --2- 172.21.15.80:0/3240928900 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f0ef80a2b90 0x7f0ef80a5050 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f0f00006010 tx=0x7f0f00005f80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:48.164 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.163+0000 7f0eff7fe640 1 -- 172.21.15.80:0/3240928900 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f0f0401f020 con 0x7f0f10104b30 2024-04-24T23:29:48.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:29:48 smithi080 ceph-mon[29152]: from='client.15552 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T23:29:48.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:29:48 smithi080 ceph-mon[29152]: pgmap v374: 129 pgs: 129 active+clean; 640 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 511 B/s wr, 5 op/s 2024-04-24T23:29:48.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:29:48 smithi080 ceph-mon[29152]: from='client.15558 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T23:29:48.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:29:48 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/885699762' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:29:48.343 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.343+0000 7f0f14b7d640 1 -- 172.21.15.80:0/3240928900 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f0f100623e0 con 0x7f0f10104b30 2024-04-24T23:29:48.344 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.343+0000 7f0eff7fe640 1 -- 172.21.15.80:0/3240928900 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) ==== 104+0+11 (secure 0 0 0) 0x7f0f0400d040 con 0x7f0f10104b30 2024-04-24T23:29:48.344 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 32 2024-04-24T23:29:48.346 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.345+0000 7f0f14b7d640 1 -- 172.21.15.80:0/3240928900 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f0ef80a2b90 msgr2=0x7f0ef80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:48.346 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.345+0000 7f0f14b7d640 1 --2- 172.21.15.80:0/3240928900 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f0ef80a2b90 0x7f0ef80a5050 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f0f00006010 tx=0x7f0f00005f80 comp rx=0 tx=0).stop 2024-04-24T23:29:48.346 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.345+0000 7f0f14b7d640 1 -- 172.21.15.80:0/3240928900 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0f10104b30 msgr2=0x7f0f10198180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:48.346 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.345+0000 7f0f14b7d640 1 --2- 172.21.15.80:0/3240928900 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0f10104b30 0x7f0f10198180 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f0f0400b450 tx=0x7f0f0400b910 comp rx=0 tx=0).stop 2024-04-24T23:29:48.346 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.345+0000 7f0f14b7d640 1 -- 172.21.15.80:0/3240928900 shutdown_connections 2024-04-24T23:29:48.346 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.345+0000 7f0f14b7d640 1 --2- 172.21.15.80:0/3240928900 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f0ef80a2b90 0x7f0ef80a5050 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:48.346 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.345+0000 7f0f14b7d640 1 --2- 172.21.15.80:0/3240928900 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0f10107460 0x7f0f1019f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:48.346 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.345+0000 7f0f14b7d640 1 --2- 172.21.15.80:0/3240928900 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0f10104b30 0x7f0f10198180 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:48.346 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.345+0000 7f0f14b7d640 1 --2- 172.21.15.80:0/3240928900 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f10102200 0x7f0f10197c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:48.346 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.345+0000 7f0f14b7d640 1 -- 172.21.15.80:0/3240928900 >> 172.21.15.80:0/3240928900 conn(0x7f0f100fbbf0 msgr2=0x7f0f10102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:48.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.345+0000 7f0f14b7d640 1 -- 172.21.15.80:0/3240928900 shutdown_connections 2024-04-24T23:29:48.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.345+0000 7f0f14b7d640 1 -- 172.21.15.80:0/3240928900 wait complete. 2024-04-24T23:29:48.357 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T23:29:48.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:29:48 smithi134 ceph-mon[32619]: from='client.15552 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T23:29:48.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:29:48 smithi134 ceph-mon[32619]: pgmap v374: 129 pgs: 129 active+clean; 640 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 511 B/s wr, 5 op/s 2024-04-24T23:29:48.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:29:48 smithi134 ceph-mon[32619]: from='client.15558 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T23:29:48.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:29:48 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/885699762' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:29:48.461 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.458+0000 7f3ee97d5640 1 -- 172.21.15.80:0/1713813821 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3ee4100680 msgr2=0x7f3ee4100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:48.462 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.458+0000 7f3ee97d5640 1 --2- 172.21.15.80:0/1713813821 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3ee4100680 0x7f3ee4100aa0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f3ed4009900 tx=0x7f3ed402a410 comp rx=0 tx=0).stop 2024-04-24T23:29:48.462 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.458+0000 7f3ee97d5640 1 -- 172.21.15.80:0/1713813821 shutdown_connections 2024-04-24T23:29:48.462 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.458+0000 7f3ee97d5640 1 --2- 172.21.15.80:0/1713813821 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ee4102230 0x7f3ee4108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:48.462 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.458+0000 7f3ee97d5640 1 --2- 172.21.15.80:0/1713813821 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ee4101870 0x7f3ee4101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:48.462 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.458+0000 7f3ee97d5640 1 --2- 172.21.15.80:0/1713813821 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3ee4100680 0x7f3ee4100aa0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:48.462 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.458+0000 7f3ee97d5640 1 -- 172.21.15.80:0/1713813821 >> 172.21.15.80:0/1713813821 conn(0x7f3ee40fbbf0 msgr2=0x7f3ee40fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:48.462 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.458+0000 7f3ee97d5640 1 -- 172.21.15.80:0/1713813821 shutdown_connections 2024-04-24T23:29:48.462 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.458+0000 7f3ee97d5640 1 -- 172.21.15.80:0/1713813821 wait complete. 2024-04-24T23:29:48.462 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.459+0000 7f3ee97d5640 1 Processor -- start 2024-04-24T23:29:48.462 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.459+0000 7f3ee97d5640 1 -- start start 2024-04-24T23:29:48.462 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.459+0000 7f3ee97d5640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3ee4100680 0x7f3ee406a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:48.463 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.459+0000 7f3ee97d5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ee4101870 0x7f3ee419c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:48.463 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.459+0000 7f3ee97d5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ee4102230 0x7f3ee419ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:48.463 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.459+0000 7f3ee97d5640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f3ee410c300 con 0x7f3ee4100680 2024-04-24T23:29:48.463 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.459+0000 7f3ee97d5640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3ee410c180 con 0x7f3ee4101870 2024-04-24T23:29:48.463 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.459+0000 7f3ee97d5640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3ee410c480 con 0x7f3ee4102230 2024-04-24T23:29:48.463 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.459+0000 7f3ee2ffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3ee4100680 0x7f3ee406a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:48.463 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.459+0000 7f3ee2ffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3ee4100680 0x7f3ee406a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:52148/0 (socket says 172.21.15.80:52148) 2024-04-24T23:29:48.463 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.459+0000 7f3ee2ffd640 1 -- 172.21.15.80:0/1849961220 learned_addr learned my addr 172.21.15.80:0/1849961220 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:29:48.463 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.459+0000 7f3edbfff640 1 --2- 172.21.15.80:0/1849961220 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ee4101870 0x7f3ee419c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:48.463 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.459+0000 7f3ee37fe640 1 --2- 172.21.15.80:0/1849961220 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ee4102230 0x7f3ee419ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:48.463 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.459+0000 7f3ee2ffd640 1 -- 172.21.15.80:0/1849961220 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ee4102230 msgr2=0x7f3ee419ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:48.464 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.459+0000 7f3ee2ffd640 1 --2- 172.21.15.80:0/1849961220 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ee4102230 0x7f3ee419ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:48.464 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.459+0000 7f3ee2ffd640 1 -- 172.21.15.80:0/1849961220 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ee4101870 msgr2=0x7f3ee419c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:48.464 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.459+0000 7f3ee2ffd640 1 --2- 172.21.15.80:0/1849961220 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ee4101870 0x7f3ee419c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:48.464 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.459+0000 7f3ee2ffd640 1 -- 172.21.15.80:0/1849961220 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ee419f750 con 0x7f3ee4100680 2024-04-24T23:29:48.464 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.459+0000 7f3edbfff640 1 --2- 172.21.15.80:0/1849961220 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ee4101870 0x7f3ee419c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:29:48.464 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.459+0000 7f3ee2ffd640 1 --2- 172.21.15.80:0/1849961220 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3ee4100680 0x7f3ee406a8d0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f3ed4031610 tx=0x7f3ed40043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:48.464 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.460+0000 7f3ee0ff9640 1 -- 172.21.15.80:0/1849961220 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ed403f030 con 0x7f3ee4100680 2024-04-24T23:29:48.464 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.460+0000 7f3ee0ff9640 1 -- 172.21.15.80:0/1849961220 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ed4003ce0 con 0x7f3ee4100680 2024-04-24T23:29:48.464 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.460+0000 7f3ee97d5640 1 -- 172.21.15.80:0/1849961220 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ee419f980 con 0x7f3ee4100680 2024-04-24T23:29:48.464 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.460+0000 7f3ee0ff9640 1 -- 172.21.15.80:0/1849961220 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ed4035070 con 0x7f3ee4100680 2024-04-24T23:29:48.464 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.460+0000 7f3ee97d5640 1 -- 172.21.15.80:0/1849961220 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ee419fe70 con 0x7f3ee4100680 2024-04-24T23:29:48.464 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.461+0000 7f3ee97d5640 1 -- 172.21.15.80:0/1849961220 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ee4107a40 con 0x7f3ee4100680 2024-04-24T23:29:48.465 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.462+0000 7f3ee0ff9640 1 -- 172.21.15.80:0/1849961220 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3ed4042030 con 0x7f3ee4100680 2024-04-24T23:29:48.465 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.462+0000 7f3ee0ff9640 1 --2- 172.21.15.80:0/1849961220 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f3ed00a2b90 0x7f3ed00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:48.465 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.462+0000 7f3edbfff640 1 --2- 172.21.15.80:0/1849961220 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f3ed00a2b90 0x7f3ed00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:48.465 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.463+0000 7f3ee0ff9640 1 -- 172.21.15.80:0/1849961220 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f3ed40ef730 con 0x7f3ee4100680 2024-04-24T23:29:48.466 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.463+0000 7f3edbfff640 1 --2- 172.21.15.80:0/1849961220 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f3ed00a2b90 0x7f3ed00a5050 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc0058c0 tx=0x7f3ecc005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:48.468 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.467+0000 7f3ee0ff9640 1 -- 172.21.15.80:0/1849961220 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f3ed40bc2c0 con 0x7f3ee4100680 2024-04-24T23:29:48.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:29:48 smithi148 ceph-mon[32528]: from='client.15552 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T23:29:48.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:29:48 smithi148 ceph-mon[32528]: pgmap v374: 129 pgs: 129 active+clean; 640 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 511 B/s wr, 5 op/s 2024-04-24T23:29:48.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:29:48 smithi148 ceph-mon[32528]: from='client.15558 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T23:29:48.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:29:48 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/885699762' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:29:48.644 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.643+0000 7f3ee97d5640 1 -- 172.21.15.80:0/1849961220 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f3ee4062340 con 0x7f3ee4100680 2024-04-24T23:29:48.644 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.644+0000 7f3ee0ff9640 1 -- 172.21.15.80:0/1849961220 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) ==== 100+0+11 (secure 0 0 0) 0x7f3ed40bbbc0 con 0x7f3ee4100680 2024-04-24T23:29:48.645 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 64 2024-04-24T23:29:48.647 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.646+0000 7f3ee97d5640 1 -- 172.21.15.80:0/1849961220 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f3ed00a2b90 msgr2=0x7f3ed00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:48.647 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.646+0000 7f3ee97d5640 1 --2- 172.21.15.80:0/1849961220 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f3ed00a2b90 0x7f3ed00a5050 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc0058c0 tx=0x7f3ecc005850 comp rx=0 tx=0).stop 2024-04-24T23:29:48.647 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.646+0000 7f3ee97d5640 1 -- 172.21.15.80:0/1849961220 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3ee4100680 msgr2=0x7f3ee406a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:48.647 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.646+0000 7f3ee97d5640 1 --2- 172.21.15.80:0/1849961220 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3ee4100680 0x7f3ee406a8d0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f3ed4031610 tx=0x7f3ed40043b0 comp rx=0 tx=0).stop 2024-04-24T23:29:48.647 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.646+0000 7f3ee97d5640 1 -- 172.21.15.80:0/1849961220 shutdown_connections 2024-04-24T23:29:48.647 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.646+0000 7f3ee97d5640 1 --2- 172.21.15.80:0/1849961220 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f3ed00a2b90 0x7f3ed00a5050 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:48.648 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.646+0000 7f3ee97d5640 1 --2- 172.21.15.80:0/1849961220 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ee4102230 0x7f3ee419ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:48.648 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.646+0000 7f3ee97d5640 1 --2- 172.21.15.80:0/1849961220 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ee4101870 0x7f3ee419c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:48.648 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.646+0000 7f3ee97d5640 1 --2- 172.21.15.80:0/1849961220 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3ee4100680 0x7f3ee406a8d0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:48.648 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.646+0000 7f3ee97d5640 1 -- 172.21.15.80:0/1849961220 >> 172.21.15.80:0/1849961220 conn(0x7f3ee40fbbf0 msgr2=0x7f3ee4104c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:48.648 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.646+0000 7f3ee97d5640 1 -- 172.21.15.80:0/1849961220 shutdown_connections 2024-04-24T23:29:48.648 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.646+0000 7f3ee97d5640 1 -- 172.21.15.80:0/1849961220 wait complete. 2024-04-24T23:29:48.658 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-24T23:29:48.763 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.759+0000 7f77eedf9640 1 -- 172.21.15.80:0/2929282961 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f77e8100680 msgr2=0x7f77e8100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:48.763 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.759+0000 7f77eedf9640 1 --2- 172.21.15.80:0/2929282961 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f77e8100680 0x7f77e8100aa0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f77e0009ca0 tx=0x7f77e002a8c0 comp rx=0 tx=0).stop 2024-04-24T23:29:48.763 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.760+0000 7f77eedf9640 1 -- 172.21.15.80:0/2929282961 shutdown_connections 2024-04-24T23:29:48.763 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.760+0000 7f77eedf9640 1 --2- 172.21.15.80:0/2929282961 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f77e8102230 0x7f77e8108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:48.763 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.760+0000 7f77eedf9640 1 --2- 172.21.15.80:0/2929282961 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f77e8101870 0x7f77e8101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:48.763 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.760+0000 7f77eedf9640 1 --2- 172.21.15.80:0/2929282961 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f77e8100680 0x7f77e8100aa0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:48.763 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.760+0000 7f77eedf9640 1 -- 172.21.15.80:0/2929282961 >> 172.21.15.80:0/2929282961 conn(0x7f77e80fbbf0 msgr2=0x7f77e80fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:48.763 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.760+0000 7f77eedf9640 1 -- 172.21.15.80:0/2929282961 shutdown_connections 2024-04-24T23:29:48.763 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.760+0000 7f77eedf9640 1 -- 172.21.15.80:0/2929282961 wait complete. 2024-04-24T23:29:48.764 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.760+0000 7f77eedf9640 1 Processor -- start 2024-04-24T23:29:48.764 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.760+0000 7f77eedf9640 1 -- start start 2024-04-24T23:29:48.764 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.760+0000 7f77eedf9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f77e8100680 0x7f77e8197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:48.764 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.760+0000 7f77eedf9640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f77e8101870 0x7f77e8197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:48.764 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.760+0000 7f77eedf9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f77e8102230 0x7f77e819f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:48.764 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.760+0000 7f77eedf9640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f77e810c300 con 0x7f77e8101870 2024-04-24T23:29:48.764 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.760+0000 7f77eedf9640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f77e810c180 con 0x7f77e8100680 2024-04-24T23:29:48.764 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.760+0000 7f77eedf9640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f77e810c480 con 0x7f77e8102230 2024-04-24T23:29:48.764 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.760+0000 7f77e7fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f77e8101870 0x7f77e8197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:48.764 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.760+0000 7f77e7fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f77e8101870 0x7f77e8197f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:52166/0 (socket says 172.21.15.80:52166) 2024-04-24T23:29:48.764 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.760+0000 7f77e7fff640 1 -- 172.21.15.80:0/28333951 learned_addr learned my addr 172.21.15.80:0/28333951 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:29:48.764 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.760+0000 7f77ed396640 1 --2- 172.21.15.80:0/28333951 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f77e8102230 0x7f77e819f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:48.764 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.761+0000 7f77ecb95640 1 --2- 172.21.15.80:0/28333951 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f77e8100680 0x7f77e8197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:48.764 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.761+0000 7f77e7fff640 1 -- 172.21.15.80:0/28333951 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f77e8102230 msgr2=0x7f77e819f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:48.765 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.761+0000 7f77e7fff640 1 --2- 172.21.15.80:0/28333951 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f77e8102230 0x7f77e819f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:48.765 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.761+0000 7f77e7fff640 1 -- 172.21.15.80:0/28333951 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f77e8100680 msgr2=0x7f77e8197a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:48.765 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.761+0000 7f77e7fff640 1 --2- 172.21.15.80:0/28333951 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f77e8100680 0x7f77e8197a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:48.765 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.761+0000 7f77e7fff640 1 -- 172.21.15.80:0/28333951 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77e819f670 con 0x7f77e8101870 2024-04-24T23:29:48.765 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.761+0000 7f77e7fff640 1 --2- 172.21.15.80:0/28333951 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f77e8101870 0x7f77e8197f90 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f77d000c920 tx=0x7f77d000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:48.765 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.761+0000 7f77e5ffb640 1 -- 172.21.15.80:0/28333951 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d0007c70 con 0x7f77e8101870 2024-04-24T23:29:48.765 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.761+0000 7f77e5ffb640 1 -- 172.21.15.80:0/28333951 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77d0007e10 con 0x7f77e8101870 2024-04-24T23:29:48.765 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.761+0000 7f77e5ffb640 1 -- 172.21.15.80:0/28333951 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d0018890 con 0x7f77e8101870 2024-04-24T23:29:48.765 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.761+0000 7f77eedf9640 1 -- 172.21.15.80:0/28333951 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77e819f960 con 0x7f77e8101870 2024-04-24T23:29:48.765 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.761+0000 7f77eedf9640 1 -- 172.21.15.80:0/28333951 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77e819feb0 con 0x7f77e8101870 2024-04-24T23:29:48.765 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.762+0000 7f77eedf9640 1 -- 172.21.15.80:0/28333951 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77e81a0140 con 0x7f77e8101870 2024-04-24T23:29:48.766 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.763+0000 7f77e5ffb640 1 -- 172.21.15.80:0/28333951 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f77d0018a70 con 0x7f77e8101870 2024-04-24T23:29:48.766 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.764+0000 7f77e5ffb640 1 --2- 172.21.15.80:0/28333951 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f77d40a2b90 0x7f77d40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:48.766 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.764+0000 7f77ecb95640 1 --2- 172.21.15.80:0/28333951 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f77d40a2b90 0x7f77d40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:48.766 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.764+0000 7f77e5ffb640 1 -- 172.21.15.80:0/28333951 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f77d00cf560 con 0x7f77e8101870 2024-04-24T23:29:48.766 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.764+0000 7f77ecb95640 1 --2- 172.21.15.80:0/28333951 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f77d40a2b90 0x7f77d40a5050 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f77e0006010 tx=0x7f77e0000f40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:48.769 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.768+0000 7f77e5ffb640 1 -- 172.21.15.80:0/28333951 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f77d0014030 con 0x7f77e8101870 2024-04-24T23:29:48.946 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.945+0000 7f77eedf9640 1 -- 172.21.15.80:0/28333951 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f77e81a0310 con 0x7f77e8101870 2024-04-24T23:29:48.946 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.945+0000 7f77e5ffb640 1 -- 172.21.15.80:0/28333951 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) ==== 103+0+11 (secure 0 0 0) 0x7f77d009c0f0 con 0x7f77e8101870 2024-04-24T23:29:48.946 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 32 2024-04-24T23:29:48.948 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.947+0000 7f77eedf9640 1 -- 172.21.15.80:0/28333951 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f77d40a2b90 msgr2=0x7f77d40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:48.948 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.947+0000 7f77eedf9640 1 --2- 172.21.15.80:0/28333951 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f77d40a2b90 0x7f77d40a5050 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f77e0006010 tx=0x7f77e0000f40 comp rx=0 tx=0).stop 2024-04-24T23:29:48.948 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.947+0000 7f77eedf9640 1 -- 172.21.15.80:0/28333951 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f77e8101870 msgr2=0x7f77e8197f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:48.948 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.947+0000 7f77eedf9640 1 --2- 172.21.15.80:0/28333951 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f77e8101870 0x7f77e8197f90 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f77d000c920 tx=0x7f77d000cde0 comp rx=0 tx=0).stop 2024-04-24T23:29:48.948 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.947+0000 7f77eedf9640 1 -- 172.21.15.80:0/28333951 shutdown_connections 2024-04-24T23:29:48.948 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.947+0000 7f77eedf9640 1 --2- 172.21.15.80:0/28333951 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f77d40a2b90 0x7f77d40a5050 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:48.948 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.947+0000 7f77eedf9640 1 --2- 172.21.15.80:0/28333951 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f77e8102230 0x7f77e819f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:48.948 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.947+0000 7f77eedf9640 1 --2- 172.21.15.80:0/28333951 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f77e8101870 0x7f77e8197f90 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:48.949 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.947+0000 7f77eedf9640 1 --2- 172.21.15.80:0/28333951 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f77e8100680 0x7f77e8197a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:48.949 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.947+0000 7f77eedf9640 1 -- 172.21.15.80:0/28333951 >> 172.21.15.80:0/28333951 conn(0x7f77e80fbbf0 msgr2=0x7f77e80688d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:48.949 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.947+0000 7f77eedf9640 1 -- 172.21.15.80:0/28333951 shutdown_connections 2024-04-24T23:29:48.949 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:48.947+0000 7f77eedf9640 1 -- 172.21.15.80:0/28333951 wait complete. 2024-04-24T23:29:48.959 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:29:49.064 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.061+0000 7fbc15963640 1 -- 172.21.15.80:0/3286589231 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbc101022c0 msgr2=0x7fbc101046b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:49.064 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.061+0000 7fbc15963640 1 --2- 172.21.15.80:0/3286589231 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbc101022c0 0x7fbc101046b0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fbc00009ca0 tx=0x7fbc0002a8c0 comp rx=0 tx=0).stop 2024-04-24T23:29:49.064 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.061+0000 7fbc15963640 1 -- 172.21.15.80:0/3286589231 shutdown_connections 2024-04-24T23:29:49.065 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.061+0000 7fbc15963640 1 --2- 172.21.15.80:0/3286589231 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbc10107520 0x7fbc10109910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:49.065 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.061+0000 7fbc15963640 1 --2- 172.21.15.80:0/3286589231 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc10104bf0 0x7fbc10106fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:49.065 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.061+0000 7fbc15963640 1 --2- 172.21.15.80:0/3286589231 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbc101022c0 0x7fbc101046b0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:49.065 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.061+0000 7fbc15963640 1 -- 172.21.15.80:0/3286589231 >> 172.21.15.80:0/3286589231 conn(0x7fbc100fbbf0 msgr2=0x7fbc100fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:49.065 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.061+0000 7fbc15963640 1 -- 172.21.15.80:0/3286589231 shutdown_connections 2024-04-24T23:29:49.065 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.061+0000 7fbc15963640 1 -- 172.21.15.80:0/3286589231 wait complete. 2024-04-24T23:29:49.065 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.061+0000 7fbc15963640 1 Processor -- start 2024-04-24T23:29:49.065 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.061+0000 7fbc15963640 1 -- start start 2024-04-24T23:29:49.065 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.062+0000 7fbc15963640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc101022c0 0x7fbc10197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:49.065 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.062+0000 7fbc15963640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbc10104bf0 0x7fbc10198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:49.065 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.062+0000 7fbc15963640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbc10107520 0x7fbc1019f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:49.065 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.062+0000 7fbc15963640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fbc1010d430 con 0x7fbc10104bf0 2024-04-24T23:29:49.066 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.062+0000 7fbc15963640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbc1010d2b0 con 0x7fbc10107520 2024-04-24T23:29:49.066 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.062+0000 7fbc15963640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fbc1010d5b0 con 0x7fbc101022c0 2024-04-24T23:29:49.066 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.062+0000 7fbc0e7fc640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbc10104bf0 0x7fbc10198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:49.066 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.062+0000 7fbc0e7fc640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbc10104bf0 0x7fbc10198010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:52184/0 (socket says 172.21.15.80:52184) 2024-04-24T23:29:49.066 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.062+0000 7fbc0e7fc640 1 -- 172.21.15.80:0/3795975652 learned_addr learned my addr 172.21.15.80:0/3795975652 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:29:49.066 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.062+0000 7fbc0effd640 1 --2- 172.21.15.80:0/3795975652 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc101022c0 0x7fbc10197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:49.066 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.062+0000 7fbc0f7fe640 1 --2- 172.21.15.80:0/3795975652 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbc10107520 0x7fbc1019f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:49.067 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.062+0000 7fbc0e7fc640 1 -- 172.21.15.80:0/3795975652 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc101022c0 msgr2=0x7fbc10197ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:49.067 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.062+0000 7fbc0e7fc640 1 --2- 172.21.15.80:0/3795975652 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc101022c0 0x7fbc10197ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:49.067 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.062+0000 7fbc0e7fc640 1 -- 172.21.15.80:0/3795975652 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbc10107520 msgr2=0x7fbc1019f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:49.067 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.062+0000 7fbc0e7fc640 1 --2- 172.21.15.80:0/3795975652 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbc10107520 0x7fbc1019f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:49.067 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.062+0000 7fbc0e7fc640 1 -- 172.21.15.80:0/3795975652 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc1019f6f0 con 0x7fbc10104bf0 2024-04-24T23:29:49.067 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.062+0000 7fbc0f7fe640 1 --2- 172.21.15.80:0/3795975652 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbc10107520 0x7fbc1019f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:29:49.067 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.062+0000 7fbc0e7fc640 1 --2- 172.21.15.80:0/3795975652 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbc10104bf0 0x7fbc10198010 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fbc0400d740 tx=0x7fbc0400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:49.067 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.062+0000 7fbc14961640 1 -- 172.21.15.80:0/3795975652 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc04017070 con 0x7fbc10104bf0 2024-04-24T23:29:49.067 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.062+0000 7fbc14961640 1 -- 172.21.15.80:0/3795975652 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc0400de50 con 0x7fbc10104bf0 2024-04-24T23:29:49.067 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.062+0000 7fbc15963640 1 -- 172.21.15.80:0/3795975652 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc1019f9e0 con 0x7fbc10104bf0 2024-04-24T23:29:49.067 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.062+0000 7fbc14961640 1 -- 172.21.15.80:0/3795975652 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc04004e00 con 0x7fbc10104bf0 2024-04-24T23:29:49.067 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.062+0000 7fbc15963640 1 -- 172.21.15.80:0/3795975652 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc1019fee0 con 0x7fbc10104bf0 2024-04-24T23:29:49.067 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.063+0000 7fbc15963640 1 -- 172.21.15.80:0/3795975652 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc100fd6e0 con 0x7fbc10104bf0 2024-04-24T23:29:49.068 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.064+0000 7fbc14961640 1 -- 172.21.15.80:0/3795975652 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fbc04004fa0 con 0x7fbc10104bf0 2024-04-24T23:29:49.068 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.065+0000 7fbc14961640 1 --2- 172.21.15.80:0/3795975652 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fbbf80a2b90 0x7fbbf80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:49.068 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.065+0000 7fbc0effd640 1 --2- 172.21.15.80:0/3795975652 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fbbf80a2b90 0x7fbbf80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:49.068 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.065+0000 7fbc14961640 1 -- 172.21.15.80:0/3795975652 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fbc040cede0 con 0x7fbc10104bf0 2024-04-24T23:29:49.069 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.066+0000 7fbc0effd640 1 --2- 172.21.15.80:0/3795975652 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fbbf80a2b90 0x7fbbf80a5050 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fbc00006010 tx=0x7fbc00000f40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:49.070 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.070+0000 7fbc14961640 1 -- 172.21.15.80:0/3795975652 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fbc0409b970 con 0x7fbc10104bf0 2024-04-24T23:29:49.285 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.284+0000 7fbc15963640 1 -- 172.21.15.80:0/3795975652 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbc10062340 con 0x7fbc10104bf0 2024-04-24T23:29:49.286 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.285+0000 7fbc14961640 1 -- 172.21.15.80:0/3795975652 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11872 (secure 0 0 0) 0x7fbc0409b270 con 0x7fbc10104bf0 2024-04-24T23:29:49.286 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-24T23:29:49.286 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/891591085","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":891591085},{"type":"v1","addr":"172.21.15.80:6835","nonce":891591085}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6837/259529026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6836","nonce":259529026},{"type":"v1","addr":"172.21.15.80:6837","nonce":259529026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14646,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6839/4194284854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6838","nonce":4194284854},{"type":"v1","addr":"172.21.15.80:6839","nonce":4194284854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14658,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6841/3145730958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6840","nonce":3145730958},{"type":"v1","addr":"172.21.15.80:6841","nonce":3145730958}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24395,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/3736784385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3736784385},{"type":"v1","addr":"172.21.15.134:6833","nonce":3736784385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/124777462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":124777462},{"type":"v1","addr":"172.21.15.148:6833","nonce":124777462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T23:27:19.300566+0000","modified":"2024-04-24T23:29:42.158567+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24496,"mds_1":24488,"mds_2":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/1399801120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1399801120},{"type":"v1","addr":"172.21.15.134:6835","nonce":1399801120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/430512704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":430512704},{"type":"v1","addr":"172.21.15.134:6837","nonce":430512704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/3002319310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3002319310},{"type":"v1","addr":"172.21.15.148:6835","nonce":3002319310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.148:6837/856528403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":856528403},{"type":"v1","addr":"172.21.15.148:6837","nonce":856528403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6839/1743815021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":1743815021},{"type":"v1","addr":"172.21.15.134:6839","nonce":1743815021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.148:6839/2952757594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":2952757594},{"type":"v1","addr":"172.21.15.148:6839","nonce":2952757594}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24496,"qdb_cluster":[24496,24472,24488]},"id":1}]} 2024-04-24T23:29:49.286 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 25 2024-04-24T23:29:49.288 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.287+0000 7fbc15963640 1 -- 172.21.15.80:0/3795975652 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fbbf80a2b90 msgr2=0x7fbbf80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:49.288 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.287+0000 7fbc15963640 1 --2- 172.21.15.80:0/3795975652 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fbbf80a2b90 0x7fbbf80a5050 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fbc00006010 tx=0x7fbc00000f40 comp rx=0 tx=0).stop 2024-04-24T23:29:49.288 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.287+0000 7fbc15963640 1 -- 172.21.15.80:0/3795975652 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbc10104bf0 msgr2=0x7fbc10198010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:49.288 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.287+0000 7fbc15963640 1 --2- 172.21.15.80:0/3795975652 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbc10104bf0 0x7fbc10198010 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fbc0400d740 tx=0x7fbc0400dc00 comp rx=0 tx=0).stop 2024-04-24T23:29:49.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:29:49 smithi080 ceph-mon[29152]: from='client.15564 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T23:29:49.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:29:49 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/202659441' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:29:49.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:29:49 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/3240928900' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T23:29:49.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:29:49 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/1849961220' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T23:29:49.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:29:49 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/28333951' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T23:29:49.289 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.287+0000 7fbc15963640 1 -- 172.21.15.80:0/3795975652 shutdown_connections 2024-04-24T23:29:49.289 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.287+0000 7fbc15963640 1 --2- 172.21.15.80:0/3795975652 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fbbf80a2b90 0x7fbbf80a5050 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:49.289 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.287+0000 7fbc15963640 1 --2- 172.21.15.80:0/3795975652 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbc10107520 0x7fbc1019f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:49.289 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.287+0000 7fbc15963640 1 --2- 172.21.15.80:0/3795975652 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbc10104bf0 0x7fbc10198010 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:49.289 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.287+0000 7fbc15963640 1 --2- 172.21.15.80:0/3795975652 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc101022c0 0x7fbc10197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:49.290 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.287+0000 7fbc15963640 1 -- 172.21.15.80:0/3795975652 >> 172.21.15.80:0/3795975652 conn(0x7fbc100fbbf0 msgr2=0x7fbc10102c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:49.290 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.287+0000 7fbc15963640 1 -- 172.21.15.80:0/3795975652 shutdown_connections 2024-04-24T23:29:49.290 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.287+0000 7fbc15963640 1 -- 172.21.15.80:0/3795975652 wait complete. 2024-04-24T23:29:49.300 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:29:49.400 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.397+0000 7fc7a41f4640 1 -- 172.21.15.80:0/128061459 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc79c1074a0 msgr2=0x7fc79c1098f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:49.400 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.397+0000 7fc7a41f4640 1 --2- 172.21.15.80:0/128061459 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc79c1074a0 0x7fc79c1098f0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fc78c009900 tx=0x7fc78c02a410 comp rx=0 tx=0).stop 2024-04-24T23:29:49.400 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.397+0000 7fc7a41f4640 1 -- 172.21.15.80:0/128061459 shutdown_connections 2024-04-24T23:29:49.400 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.397+0000 7fc7a41f4640 1 --2- 172.21.15.80:0/128061459 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc79c1074a0 0x7fc79c1098f0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:49.400 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.397+0000 7fc7a41f4640 1 --2- 172.21.15.80:0/128061459 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc79c104b70 0x7fc79c106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:49.400 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.397+0000 7fc7a41f4640 1 --2- 172.21.15.80:0/128061459 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc79c102240 0x7fc79c104630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:49.400 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.397+0000 7fc7a41f4640 1 -- 172.21.15.80:0/128061459 >> 172.21.15.80:0/128061459 conn(0x7fc79c0fbc30 msgr2=0x7fc79c0fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:49.401 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.397+0000 7fc7a41f4640 1 -- 172.21.15.80:0/128061459 shutdown_connections 2024-04-24T23:29:49.401 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.397+0000 7fc7a41f4640 1 -- 172.21.15.80:0/128061459 wait complete. 2024-04-24T23:29:49.401 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.397+0000 7fc7a41f4640 1 Processor -- start 2024-04-24T23:29:49.401 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.397+0000 7fc7a41f4640 1 -- start start 2024-04-24T23:29:49.401 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.397+0000 7fc7a41f4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc79c102240 0x7fc79c199c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:49.401 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.397+0000 7fc7a41f4640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc79c104b70 0x7fc79c19a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:49.401 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.397+0000 7fc7a41f4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc79c1074a0 0x7fc79c1a11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:49.401 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.397+0000 7fc7a41f4640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fc79c10d440 con 0x7fc79c104b70 2024-04-24T23:29:49.401 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.397+0000 7fc7a41f4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc79c10d2c0 con 0x7fc79c1074a0 2024-04-24T23:29:49.401 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.397+0000 7fc7a41f4640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc79c10d5c0 con 0x7fc79c102240 2024-04-24T23:29:49.401 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.397+0000 7fc7a178f640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc79c104b70 0x7fc79c19a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:49.401 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.397+0000 7fc7a178f640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc79c104b70 0x7fc79c19a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:52210/0 (socket says 172.21.15.80:52210) 2024-04-24T23:29:49.402 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.398+0000 7fc7a178f640 1 -- 172.21.15.80:0/3124221721 learned_addr learned my addr 172.21.15.80:0/3124221721 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:29:49.402 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.398+0000 7fc7a1f90640 1 --2- 172.21.15.80:0/3124221721 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc79c102240 0x7fc79c199c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:49.402 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.398+0000 7fc7a2791640 1 --2- 172.21.15.80:0/3124221721 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc79c1074a0 0x7fc79c1a11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:49.402 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.398+0000 7fc7a178f640 1 -- 172.21.15.80:0/3124221721 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc79c102240 msgr2=0x7fc79c199c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:49.402 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.398+0000 7fc7a178f640 1 --2- 172.21.15.80:0/3124221721 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc79c102240 0x7fc79c199c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:49.402 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.398+0000 7fc7a178f640 1 -- 172.21.15.80:0/3124221721 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc79c1074a0 msgr2=0x7fc79c1a11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:49.403 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.398+0000 7fc7a178f640 1 --2- 172.21.15.80:0/3124221721 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc79c1074a0 0x7fc79c1a11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:49.403 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.398+0000 7fc7a178f640 1 -- 172.21.15.80:0/3124221721 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc79c1a1840 con 0x7fc79c104b70 2024-04-24T23:29:49.403 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.398+0000 7fc7a178f640 1 --2- 172.21.15.80:0/3124221721 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc79c104b70 0x7fc79c19a170 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fc79000c970 tx=0x7fc79000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:49.403 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.398+0000 7fc79affd640 1 -- 172.21.15.80:0/3124221721 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc790014070 con 0x7fc79c104b70 2024-04-24T23:29:49.403 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.398+0000 7fc7a41f4640 1 -- 172.21.15.80:0/3124221721 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc79c1a1b30 con 0x7fc79c104b70 2024-04-24T23:29:49.403 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.398+0000 7fc79affd640 1 -- 172.21.15.80:0/3124221721 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc790004440 con 0x7fc79c104b70 2024-04-24T23:29:49.403 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.398+0000 7fc79affd640 1 -- 172.21.15.80:0/3124221721 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc79000f450 con 0x7fc79c104b70 2024-04-24T23:29:49.403 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.398+0000 7fc7a41f4640 1 -- 172.21.15.80:0/3124221721 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc79c1a2080 con 0x7fc79c104b70 2024-04-24T23:29:49.404 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.399+0000 7fc7a41f4640 1 -- 172.21.15.80:0/3124221721 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc79c108f90 con 0x7fc79c104b70 2024-04-24T23:29:49.404 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.400+0000 7fc79affd640 1 -- 172.21.15.80:0/3124221721 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc790017070 con 0x7fc79c104b70 2024-04-24T23:29:49.404 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.401+0000 7fc79affd640 1 --2- 172.21.15.80:0/3124221721 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fc7840a2b90 0x7fc7840a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:49.405 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.401+0000 7fc79affd640 1 -- 172.21.15.80:0/3124221721 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fc7900cf7e0 con 0x7fc79c104b70 2024-04-24T23:29:49.405 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.401+0000 7fc7a1f90640 1 --2- 172.21.15.80:0/3124221721 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fc7840a2b90 0x7fc7840a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:49.405 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.401+0000 7fc7a1f90640 1 --2- 172.21.15.80:0/3124221721 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fc7840a2b90 0x7fc7840a5050 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc794005c40 tx=0x7fc79400a3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:49.405 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.404+0000 7fc79affd640 1 -- 172.21.15.80:0/3124221721 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fc79001e020 con 0x7fc79c104b70 2024-04-24T23:29:49.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:29:49 smithi134 ceph-mon[32619]: from='client.15564 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T23:29:49.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:29:49 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/202659441' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:29:49.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:29:49 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/3240928900' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T23:29:49.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:29:49 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/1849961220' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T23:29:49.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:29:49 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/28333951' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T23:29:49.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:29:49 smithi148 ceph-mon[32528]: from='client.15564 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T23:29:49.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:29:49 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/202659441' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:29:49.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:29:49 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/3240928900' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T23:29:49.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:29:49 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/1849961220' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T23:29:49.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:29:49 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/28333951' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T23:29:49.581 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.580+0000 7fc7a41f4640 1 -- 172.21.15.80:0/3124221721 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc79c02d330 con 0x7fc79c104b70 2024-04-24T23:29:49.582 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.582+0000 7fc79affd640 1 -- 172.21.15.80:0/3124221721 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24839 (secure 0 0 0) 0x7fc79009c370 con 0x7fc79c104b70 2024-04-24T23:29:49.583 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-24T23:29:49.583 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":94,"fsid":"ec7885aa-0290-11ef-bc93-c7b262605968","created":"2024-04-24T23:19:05.586407+0000","modified":"2024-04-24T23:29:08.209684+0000","last_up_change":"2024-04-24T23:26:25.938316+0000","last_in_change":"2024-04-24T23:26:04.786699+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:22:33.124428+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:27:17.575788+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:27:18.492762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T23:27:22.438503+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a525741d-b46c-4eef-aa64-3f057cac1497","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":2597718112},{"type":"v1","addr":"172.21.15.80:6803","nonce":2597718112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":2597718112},{"type":"v1","addr":"172.21.15.80:6805","nonce":2597718112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2597718112},{"type":"v1","addr":"172.21.15.80:6809","nonce":2597718112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":2597718112},{"type":"v1","addr":"172.21.15.80:6807","nonce":2597718112}]},"public_addr":"172.21.15.80:6803/2597718112","cluster_addr":"172.21.15.80:6805/2597718112","heartbeat_back_addr":"172.21.15.80:6809/2597718112","heartbeat_front_addr":"172.21.15.80:6807/2597718112","state":["exists","up"]},{"osd":1,"uuid":"2d0f0d67-6db4-40ac-91cc-58604ab6795a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":2395610201},{"type":"v1","addr":"172.21.15.134:6801","nonce":2395610201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":2395610201},{"type":"v1","addr":"172.21.15.134:6803","nonce":2395610201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":2395610201},{"type":"v1","addr":"172.21.15.134:6807","nonce":2395610201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":2395610201},{"type":"v1","addr":"172.21.15.134:6805","nonce":2395610201}]},"public_addr":"172.21.15.134:6801/2395610201","cluster_addr":"172.21.15.134:6803/2395610201","heartbeat_back_addr":"172.21.15.134:6807/2395610201","heartbeat_front_addr":"172.21.15.134:6805/2395610201","state":["exists","up"]},{"osd":2,"uuid":"781fb879-fedc-4c84-82fb-59e073bc4cf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1629152187},{"type":"v1","addr":"172.21.15.148:6801","nonce":1629152187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1629152187},{"type":"v1","addr":"172.21.15.148:6803","nonce":1629152187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1629152187},{"type":"v1","addr":"172.21.15.148:6807","nonce":1629152187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1629152187},{"type":"v1","addr":"172.21.15.148:6805","nonce":1629152187}]},"public_addr":"172.21.15.148:6801/1629152187","cluster_addr":"172.21.15.148:6803/1629152187","heartbeat_back_addr":"172.21.15.148:6807/1629152187","heartbeat_front_addr":"172.21.15.148:6805/1629152187","state":["exists","up"]},{"osd":3,"uuid":"cd982757-f2f6-4296-b3e7-4cd40fbd8d9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":3128298510},{"type":"v1","addr":"172.21.15.80:6811","nonce":3128298510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":3128298510},{"type":"v1","addr":"172.21.15.80:6813","nonce":3128298510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":3128298510},{"type":"v1","addr":"172.21.15.80:6817","nonce":3128298510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":3128298510},{"type":"v1","addr":"172.21.15.80:6815","nonce":3128298510}]},"public_addr":"172.21.15.80:6811/3128298510","cluster_addr":"172.21.15.80:6813/3128298510","heartbeat_back_addr":"172.21.15.80:6817/3128298510","heartbeat_front_addr":"172.21.15.80:6815/3128298510","state":["exists","up"]},{"osd":4,"uuid":"dfc55a79-a7e8-417a-b898-ec44171a54af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1917556060},{"type":"v1","addr":"172.21.15.134:6809","nonce":1917556060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1917556060},{"type":"v1","addr":"172.21.15.134:6811","nonce":1917556060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1917556060},{"type":"v1","addr":"172.21.15.134:6815","nonce":1917556060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1917556060},{"type":"v1","addr":"172.21.15.134:6813","nonce":1917556060}]},"public_addr":"172.21.15.134:6809/1917556060","cluster_addr":"172.21.15.134:6811/1917556060","heartbeat_back_addr":"172.21.15.134:6815/1917556060","heartbeat_front_addr":"172.21.15.134:6813/1917556060","state":["exists","up"]},{"osd":5,"uuid":"b5491337-0694-440c-b679-312f3d14cf65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":2715951281},{"type":"v1","addr":"172.21.15.148:6809","nonce":2715951281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":2715951281},{"type":"v1","addr":"172.21.15.148:6811","nonce":2715951281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":2715951281},{"type":"v1","addr":"172.21.15.148:6815","nonce":2715951281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":2715951281},{"type":"v1","addr":"172.21.15.148:6813","nonce":2715951281}]},"public_addr":"172.21.15.148:6809/2715951281","cluster_addr":"172.21.15.148:6811/2715951281","heartbeat_back_addr":"172.21.15.148:6815/2715951281","heartbeat_front_addr":"172.21.15.148:6813/2715951281","state":["exists","up"]},{"osd":6,"uuid":"ba4b5df7-d86e-47cd-ab67-cf763c4e4d6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":953261724},{"type":"v1","addr":"172.21.15.80:6819","nonce":953261724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":953261724},{"type":"v1","addr":"172.21.15.80:6821","nonce":953261724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":953261724},{"type":"v1","addr":"172.21.15.80:6825","nonce":953261724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":953261724},{"type":"v1","addr":"172.21.15.80:6823","nonce":953261724}]},"public_addr":"172.21.15.80:6819/953261724","cluster_addr":"172.21.15.80:6821/953261724","heartbeat_back_addr":"172.21.15.80:6825/953261724","heartbeat_front_addr":"172.21.15.80:6823/953261724","state":["exists","up"]},{"osd":7,"uuid":"464a503a-75ae-49e7-ab6e-a7f9f2a6a307","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":108354844},{"type":"v1","addr":"172.21.15.134:6817","nonce":108354844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":108354844},{"type":"v1","addr":"172.21.15.134:6819","nonce":108354844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":108354844},{"type":"v1","addr":"172.21.15.134:6823","nonce":108354844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":108354844},{"type":"v1","addr":"172.21.15.134:6821","nonce":108354844}]},"public_addr":"172.21.15.134:6817/108354844","cluster_addr":"172.21.15.134:6819/108354844","heartbeat_back_addr":"172.21.15.134:6823/108354844","heartbeat_front_addr":"172.21.15.134:6821/108354844","state":["exists","up"]},{"osd":8,"uuid":"ce544b2e-4540-47c0-9997-500c45351330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3559655329},{"type":"v1","addr":"172.21.15.148:6817","nonce":3559655329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":3559655329},{"type":"v1","addr":"172.21.15.148:6819","nonce":3559655329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":3559655329},{"type":"v1","addr":"172.21.15.148:6823","nonce":3559655329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3559655329},{"type":"v1","addr":"172.21.15.148:6821","nonce":3559655329}]},"public_addr":"172.21.15.148:6817/3559655329","cluster_addr":"172.21.15.148:6819/3559655329","heartbeat_back_addr":"172.21.15.148:6823/3559655329","heartbeat_front_addr":"172.21.15.148:6821/3559655329","state":["exists","up"]},{"osd":9,"uuid":"06de3732-19b9-496c-a464-55322efa6759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":4019913545},{"type":"v1","addr":"172.21.15.80:6827","nonce":4019913545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":4019913545},{"type":"v1","addr":"172.21.15.80:6829","nonce":4019913545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":4019913545},{"type":"v1","addr":"172.21.15.80:6833","nonce":4019913545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":4019913545},{"type":"v1","addr":"172.21.15.80:6831","nonce":4019913545}]},"public_addr":"172.21.15.80:6827/4019913545","cluster_addr":"172.21.15.80:6829/4019913545","heartbeat_back_addr":"172.21.15.80:6833/4019913545","heartbeat_front_addr":"172.21.15.80:6831/4019913545","state":["exists","up"]},{"osd":10,"uuid":"8d92f0d9-5c64-429a-b750-c33ae0e070c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":647343766},{"type":"v1","addr":"172.21.15.134:6825","nonce":647343766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":647343766},{"type":"v1","addr":"172.21.15.134:6827","nonce":647343766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":647343766},{"type":"v1","addr":"172.21.15.134:6831","nonce":647343766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":647343766},{"type":"v1","addr":"172.21.15.134:6829","nonce":647343766}]},"public_addr":"172.21.15.134:6825/647343766","cluster_addr":"172.21.15.134:6827/647343766","heartbeat_back_addr":"172.21.15.134:6831/647343766","heartbeat_front_addr":"172.21.15.134:6829/647343766","state":["exists","up"]},{"osd":11,"uuid":"4c003a41-f65b-479f-bc80-b27d680cf936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":4068024116},{"type":"v1","addr":"172.21.15.148:6825","nonce":4068024116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":4068024116},{"type":"v1","addr":"172.21.15.148:6827","nonce":4068024116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":4068024116},{"type":"v1","addr":"172.21.15.148:6831","nonce":4068024116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":4068024116},{"type":"v1","addr":"172.21.15.148:6829","nonce":4068024116}]},"public_addr":"172.21.15.148:6825/4068024116","cluster_addr":"172.21.15.148:6827/4068024116","heartbeat_back_addr":"172.21.15.148:6831/4068024116","heartbeat_front_addr":"172.21.15.148:6829/4068024116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:21:42.433631+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:22:06.145933+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:22:30.726143+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:22:55.729961+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:23:21.280913+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:23:46.609983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:24:12.585247+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:24:38.359287+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:25:04.491232+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:25:30.788089+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:25:56.891078+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:26:23.899113+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":7}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"4.a","mappings":[{"from":5,"to":8}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.80:0/2828378749":"2024-04-25T23:20:06.793609+0000","172.21.15.80:6801/233364790":"2024-04-25T23:20:06.793609+0000","172.21.15.80:6800/1614691410":"2024-04-25T23:19:40.490692+0000","172.21.15.80:0/1406187145":"2024-04-25T23:20:06.793609+0000","172.21.15.80:0/92953396":"2024-04-25T23:19:40.490692+0000","172.21.15.80:0/598129432":"2024-04-25T23:20:06.793609+0000","172.21.15.80:0/232193118":"2024-04-25T23:19:40.490692+0000","172.21.15.80:6800/233364790":"2024-04-25T23:20:06.793609+0000","172.21.15.80:6801/1614691410":"2024-04-25T23:19:40.490692+0000","172.21.15.80:0/2081562529":"2024-04-25T23:19:40.490692+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:29:49.584 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.584+0000 7fc7a41f4640 1 -- 172.21.15.80:0/3124221721 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fc7840a2b90 msgr2=0x7fc7840a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:49.585 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.584+0000 7fc7a41f4640 1 --2- 172.21.15.80:0/3124221721 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fc7840a2b90 0x7fc7840a5050 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc794005c40 tx=0x7fc79400a3b0 comp rx=0 tx=0).stop 2024-04-24T23:29:49.585 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.584+0000 7fc7a41f4640 1 -- 172.21.15.80:0/3124221721 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc79c104b70 msgr2=0x7fc79c19a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:49.585 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.584+0000 7fc7a41f4640 1 --2- 172.21.15.80:0/3124221721 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc79c104b70 0x7fc79c19a170 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fc79000c970 tx=0x7fc79000ce30 comp rx=0 tx=0).stop 2024-04-24T23:29:49.585 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.584+0000 7fc7a41f4640 1 -- 172.21.15.80:0/3124221721 shutdown_connections 2024-04-24T23:29:49.585 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.584+0000 7fc7a41f4640 1 --2- 172.21.15.80:0/3124221721 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fc7840a2b90 0x7fc7840a5050 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:49.585 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.584+0000 7fc7a41f4640 1 --2- 172.21.15.80:0/3124221721 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc79c1074a0 0x7fc79c1a11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:49.585 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.584+0000 7fc7a41f4640 1 --2- 172.21.15.80:0/3124221721 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc79c104b70 0x7fc79c19a170 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:49.585 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.584+0000 7fc7a41f4640 1 --2- 172.21.15.80:0/3124221721 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc79c102240 0x7fc79c199c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:49.585 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.584+0000 7fc7a41f4640 1 -- 172.21.15.80:0/3124221721 >> 172.21.15.80:0/3124221721 conn(0x7fc79c0fbc30 msgr2=0x7fc79c101860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:49.585 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.584+0000 7fc7a41f4640 1 -- 172.21.15.80:0/3124221721 shutdown_connections 2024-04-24T23:29:49.585 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.584+0000 7fc7a41f4640 1 -- 172.21.15.80:0/3124221721 wait complete. 2024-04-24T23:29:49.598 DEBUG:tasks.fwd_scrub:Yielding 2024-04-24T23:29:49.599 INFO:teuthology.run_tasks:Running task workunit... 2024-04-24T23:29:49.612 INFO:tasks.workunit:Pulling workunits from ref d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T23:29:49.613 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-24T23:29:49.613 INFO:tasks.workunit:timeout=3h 2024-04-24T23:29:49.613 INFO:tasks.workunit:cleanup=True 2024-04-24T23:29:49.613 DEBUG:teuthology.orchestra.run.smithi080:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-24T23:29:49.614 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-04-24T23:29:49.614 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T23:29:49.614 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:29:49.614 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T23:29:49.629 INFO:teuthology.orchestra.run.smithi080.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-04-24T23:29:49.629 INFO:teuthology.orchestra.run.smithi080.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-04-24T23:29:49.629 INFO:teuthology.orchestra.run.smithi080.stdout:Device: 73h/115d Inode: 1099511628283 Links: 2 2024-04-24T23:29:49.629 INFO:teuthology.orchestra.run.smithi080.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T23:29:49.629 INFO:teuthology.orchestra.run.smithi080.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-04-24T23:29:49.630 INFO:teuthology.orchestra.run.smithi080.stdout:Access: 2024-04-24 23:27:47.360842401 +0000 2024-04-24T23:29:49.630 INFO:teuthology.orchestra.run.smithi080.stdout:Modify: 2024-04-24 23:27:47.360842401 +0000 2024-04-24T23:29:49.630 INFO:teuthology.orchestra.run.smithi080.stdout:Change: 2024-04-24 23:29:41.481751045 +0000 2024-04-24T23:29:49.630 INFO:teuthology.orchestra.run.smithi080.stdout: Birth: 2024-04-24 23:27:47.360842401 +0000 2024-04-24T23:29:49.630 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-04-24T23:29:49.630 DEBUG:teuthology.orchestra.run.smithi080:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-04-24T23:29:49.699 DEBUG:teuthology.orchestra.run.smithi080:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T23:29:49.719 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.717+0000 7fca317f9640 1 -- 172.21.15.80:0/3905853862 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fca2c101790 msgr2=0x7fca2c101bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:49.719 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.717+0000 7fca317f9640 1 --2- 172.21.15.80:0/3905853862 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fca2c101790 0x7fca2c101bb0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fca1c009ca0 tx=0x7fca1c02a8c0 comp rx=0 tx=0).stop 2024-04-24T23:29:49.720 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.717+0000 7fca317f9640 1 -- 172.21.15.80:0/3905853862 shutdown_connections 2024-04-24T23:29:49.720 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.717+0000 7fca317f9640 1 --2- 172.21.15.80:0/3905853862 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fca2c103340 0x7fca2c109bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:49.720 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.717+0000 7fca317f9640 1 --2- 172.21.15.80:0/3905853862 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fca2c102980 0x7fca2c102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:49.720 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.717+0000 7fca317f9640 1 --2- 172.21.15.80:0/3905853862 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fca2c101790 0x7fca2c101bb0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:49.720 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.717+0000 7fca317f9640 1 -- 172.21.15.80:0/3905853862 >> 172.21.15.80:0/3905853862 conn(0x7fca2c0fcd20 msgr2=0x7fca2c0ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:49.720 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.717+0000 7fca317f9640 1 -- 172.21.15.80:0/3905853862 shutdown_connections 2024-04-24T23:29:49.720 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.717+0000 7fca317f9640 1 -- 172.21.15.80:0/3905853862 wait complete. 2024-04-24T23:29:49.720 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.718+0000 7fca317f9640 1 Processor -- start 2024-04-24T23:29:49.720 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.718+0000 7fca317f9640 1 -- start start 2024-04-24T23:29:49.720 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.718+0000 7fca317f9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fca2c101790 0x7fca2c197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:49.720 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.718+0000 7fca317f9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fca2c102980 0x7fca2c198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:49.720 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.718+0000 7fca317f9640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fca2c103340 0x7fca2c19f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:49.720 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.718+0000 7fca317f9640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fca2c10d3e0 con 0x7fca2c103340 2024-04-24T23:29:49.720 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.718+0000 7fca317f9640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fca2c10d260 con 0x7fca2c102980 2024-04-24T23:29:49.720 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.718+0000 7fca317f9640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fca2c10d560 con 0x7fca2c101790 2024-04-24T23:29:49.721 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.718+0000 7fca2b7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fca2c103340 0x7fca2c19f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:49.721 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.718+0000 7fca2b7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fca2c103340 0x7fca2c19f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:52236/0 (socket says 172.21.15.80:52236) 2024-04-24T23:29:49.721 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.718+0000 7fca2b7fe640 1 -- 172.21.15.80:0/712995082 learned_addr learned my addr 172.21.15.80:0/712995082 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:29:49.721 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.718+0000 7fca2affd640 1 --2- 172.21.15.80:0/712995082 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fca2c101790 0x7fca2c197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:49.729 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.718+0000 7fca2a7fc640 1 --2- 172.21.15.80:0/712995082 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fca2c102980 0x7fca2c198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:49.729 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.718+0000 7fca2b7fe640 1 -- 172.21.15.80:0/712995082 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fca2c101790 msgr2=0x7fca2c197b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:49.729 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.718+0000 7fca2b7fe640 1 --2- 172.21.15.80:0/712995082 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fca2c101790 0x7fca2c197b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:49.730 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.718+0000 7fca2b7fe640 1 -- 172.21.15.80:0/712995082 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fca2c102980 msgr2=0x7fca2c198050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:49.730 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.718+0000 7fca2b7fe640 1 --2- 172.21.15.80:0/712995082 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fca2c102980 0x7fca2c198050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:49.730 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.719+0000 7fca2b7fe640 1 -- 172.21.15.80:0/712995082 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca2c19f730 con 0x7fca2c103340 2024-04-24T23:29:49.730 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.719+0000 7fca2a7fc640 1 --2- 172.21.15.80:0/712995082 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fca2c102980 0x7fca2c198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:29:49.730 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.719+0000 7fca2b7fe640 1 --2- 172.21.15.80:0/712995082 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fca2c103340 0x7fca2c19f0c0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fca2400d950 tx=0x7fca2400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:49.730 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.719+0000 7fca13fff640 1 -- 172.21.15.80:0/712995082 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca24017030 con 0x7fca2c103340 2024-04-24T23:29:49.730 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.719+0000 7fca317f9640 1 -- 172.21.15.80:0/712995082 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca2c19fa20 con 0x7fca2c103340 2024-04-24T23:29:49.730 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.719+0000 7fca317f9640 1 -- 172.21.15.80:0/712995082 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca2c19ff70 con 0x7fca2c103340 2024-04-24T23:29:49.730 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.719+0000 7fca13fff640 1 -- 172.21.15.80:0/712995082 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca24004020 con 0x7fca2c103340 2024-04-24T23:29:49.730 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.719+0000 7fca13fff640 1 -- 172.21.15.80:0/712995082 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca24003af0 con 0x7fca2c103340 2024-04-24T23:29:49.730 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.721+0000 7fca13fff640 1 -- 172.21.15.80:0/712995082 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fca2401a070 con 0x7fca2c103340 2024-04-24T23:29:49.730 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.721+0000 7fca345e6740 1 Processor -- start 2024-04-24T23:29:49.730 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.721+0000 7fca345e6740 1 -- start start 2024-04-24T23:29:49.731 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.721+0000 7fca345e6740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562caf2e7d90 0x562caf2e81b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:49.731 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.721+0000 7fca13fff640 1 --2- 172.21.15.80:0/712995082 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fca140a2b90 0x7fca140a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:49.731 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.721+0000 7fca345e6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562caf2e8780 0x562caf2ed280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:49.731 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.721+0000 7fca345e6740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562caf2ed8f0 0x562caf2efd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:49.731 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.721+0000 7fca345e6740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x562caf24b2c0 con 0x562caf2e7d90 2024-04-24T23:29:49.731 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.721+0000 7fca345e6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x562caf1f3910 con 0x562caf2e8780 2024-04-24T23:29:49.731 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.722+0000 7fca345e6740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x562caf247280 con 0x562caf2ed8f0 2024-04-24T23:29:49.731 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.722+0000 7fca2a7fc640 1 --2- 172.21.15.80:0/712995082 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fca140a2b90 0x7fca140a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:49.731 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.722+0000 7fca2affd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562caf2e7d90 0x562caf2e81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:49.731 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.722+0000 7fca13fff640 1 -- 172.21.15.80:0/712995082 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fca240cea70 con 0x7fca2c103340 2024-04-24T23:29:49.731 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.722+0000 7fca2affd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562caf2e7d90 0x562caf2e81b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:52242/0 (socket says 172.21.15.80:52242) 2024-04-24T23:29:49.731 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.722+0000 7fca2affd640 1 -- 172.21.15.80:0/1996443955 learned_addr learned my addr 172.21.15.80:0/1996443955 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:29:49.731 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.722+0000 7fca2b7fe640 1 --2- 172.21.15.80:0/1996443955 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562caf2ed8f0 0x562caf2efd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:49.731 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.722+0000 7fca2a7fc640 1 --2- 172.21.15.80:0/1996443955 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562caf2e8780 0x562caf2ed280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:49.732 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.722+0000 7fca2affd640 1 -- 172.21.15.80:0/1996443955 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562caf2ed8f0 msgr2=0x562caf2efd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:49.732 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.722+0000 7fca2affd640 1 --2- 172.21.15.80:0/1996443955 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562caf2ed8f0 0x562caf2efd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:49.732 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.722+0000 7fca2affd640 1 -- 172.21.15.80:0/1996443955 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562caf2e8780 msgr2=0x562caf2ed280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:49.732 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.722+0000 7fca2affd640 1 --2- 172.21.15.80:0/1996443955 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562caf2e8780 0x562caf2ed280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:49.732 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.722+0000 7fca2affd640 1 -- 172.21.15.80:0/1996443955 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562caf204fb0 con 0x562caf2e7d90 2024-04-24T23:29:49.732 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.722+0000 7fca2a7fc640 1 --2- 172.21.15.80:0/712995082 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fca140a2b90 0x7fca140a5050 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fca20005d00 tx=0x7fca20015840 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:49.732 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.722+0000 7fca2affd640 1 --2- 172.21.15.80:0/1996443955 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562caf2e7d90 0x562caf2e81b0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fca1c009ca0 tx=0x7fca1c005f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60b017f48f2ab599 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:49.732 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.722+0000 7fca117fa640 1 -- 172.21.15.80:0/1996443955 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca1c031450 con 0x562caf2e7d90 2024-04-24T23:29:49.732 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.722+0000 7fca117fa640 1 -- 172.21.15.80:0/1996443955 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca1c036050 con 0x562caf2e7d90 2024-04-24T23:29:49.732 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.722+0000 7fca117fa640 1 -- 172.21.15.80:0/1996443955 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca1c004330 con 0x562caf2e7d90 2024-04-24T23:29:49.732 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.723+0000 7fca345e6740 1 -- 172.21.15.80:0/1996443955 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562caf2e7d90 msgr2=0x562caf2e81b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:49.732 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.723+0000 7fca345e6740 1 --2- 172.21.15.80:0/1996443955 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562caf2e7d90 0x562caf2e81b0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fca1c009ca0 tx=0x7fca1c005f70 comp rx=0 tx=0).stop 2024-04-24T23:29:49.732 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.723+0000 7fca345e6740 1 -- 172.21.15.80:0/1996443955 shutdown_connections 2024-04-24T23:29:49.732 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.723+0000 7fca345e6740 1 --2- 172.21.15.80:0/1996443955 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562caf2ed8f0 0x562caf2efd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:49.733 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.723+0000 7fca345e6740 1 --2- 172.21.15.80:0/1996443955 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562caf2e8780 0x562caf2ed280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:49.733 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.723+0000 7fca345e6740 1 --2- 172.21.15.80:0/1996443955 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562caf2e7d90 0x562caf2e81b0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:49.733 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.723+0000 7fca345e6740 1 -- 172.21.15.80:0/1996443955 >> 172.21.15.80:0/1996443955 conn(0x562caf2af950 msgr2=0x562caf2e3810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:49.733 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.723+0000 7fca345e6740 1 -- 172.21.15.80:0/1996443955 shutdown_connections 2024-04-24T23:29:49.733 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.723+0000 7fca345e6740 1 -- 172.21.15.80:0/1996443955 wait complete. 2024-04-24T23:29:49.733 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.724+0000 7fca345e6740 1 -- start start 2024-04-24T23:29:49.733 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.724+0000 7fca345e6740 1 Processor -- start 2024-04-24T23:29:49.733 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.724+0000 7fca345e6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562caf2e7d90 0x562caf2f1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:49.733 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.724+0000 7fca345e6740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562caf2e8780 0x562caf2f2510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:49.733 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.724+0000 7fca345e6740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562caf2ed8f0 0x562caf2f9550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:49.733 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.724+0000 7fca345e6740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x562caf2f06c0 con 0x562caf2e8780 2024-04-24T23:29:49.733 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.724+0000 7fca345e6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x562caf206e90 con 0x562caf2e7d90 2024-04-24T23:29:49.734 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.724+0000 7fca345e6740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x562caf2f0840 con 0x562caf2ed8f0 2024-04-24T23:29:49.734 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.724+0000 7fca345e6740 10 client.0 _ll_register_callbacks cb 0x562caf242970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:29:49.734 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.724+0000 7fca2a7fc640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562caf2e8780 0x562caf2f2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:49.734 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.724+0000 7fca2a7fc640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562caf2e8780 0x562caf2f2510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:52246/0 (socket says 172.21.15.80:52246) 2024-04-24T23:29:49.734 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.724+0000 7fca2a7fc640 1 -- 172.21.15.80:0/3281401380 learned_addr learned my addr 172.21.15.80:0/3281401380 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:29:49.734 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.724+0000 7fca2affd640 1 --2- 172.21.15.80:0/3281401380 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562caf2e7d90 0x562caf2f1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:49.734 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.724+0000 7fca2b7fe640 1 --2- 172.21.15.80:0/3281401380 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562caf2ed8f0 0x562caf2f9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:49.734 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.724+0000 7fca2a7fc640 1 -- 172.21.15.80:0/3281401380 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562caf2ed8f0 msgr2=0x562caf2f9550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:49.734 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.724+0000 7fca2a7fc640 1 --2- 172.21.15.80:0/3281401380 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562caf2ed8f0 0x562caf2f9550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:49.734 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.724+0000 7fca2a7fc640 1 -- 172.21.15.80:0/3281401380 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562caf2e7d90 msgr2=0x562caf2f1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:49.734 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.724+0000 7fca2a7fc640 1 --2- 172.21.15.80:0/3281401380 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562caf2e7d90 0x562caf2f1fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:49.734 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.724+0000 7fca2a7fc640 1 -- 172.21.15.80:0/3281401380 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562caf0277e0 con 0x562caf2e8780 2024-04-24T23:29:49.734 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.724+0000 7fca2b7fe640 1 --2- 172.21.15.80:0/3281401380 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562caf2ed8f0 0x562caf2f9550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:29:49.735 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.724+0000 7fca2a7fc640 1 --2- 172.21.15.80:0/3281401380 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562caf2e8780 0x562caf2f2510 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fca200127b0 tx=0x7fca20004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e549cb790a7d5f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:49.735 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.725+0000 7fca0eff5640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-04-24T23:29:49.735 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.725+0000 7fca0eff5640 1 -- 172.21.15.80:0/3281401380 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca20024020 con 0x562caf2e8780 2024-04-24T23:29:49.735 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.725+0000 7fca0eff5640 1 -- 172.21.15.80:0/3281401380 <== mon.0 v2:172.21.15.80:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fca20013a80 con 0x562caf2e8780 2024-04-24T23:29:49.735 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.725+0000 7fca345e6740 1 -- 172.21.15.80:0/3281401380 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca1c036050 con 0x562caf2e8780 2024-04-24T23:29:49.735 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.725+0000 7fca0eff5640 1 -- 172.21.15.80:0/3281401380 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fca2002f050 con 0x562caf2e8780 2024-04-24T23:29:49.735 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.725+0000 7fca345e6740 10 client.15618 fetch_fsmap learned FSMap version 25 2024-04-24T23:29:49.735 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.725+0000 7fca345e6740 1 -- 172.21.15.80:0/3281401380 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562caf087f70 con 0x562caf2e8780 2024-04-24T23:29:49.735 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.725+0000 7fca0eff5640 1 -- 172.21.15.80:0/3281401380 <== mon.0 v2:172.21.15.80:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca1c036050 con 0x562caf2e8780 2024-04-24T23:29:49.735 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.725+0000 7fca0eff5640 1 -- 172.21.15.80:0/3281401380 <== mon.0 v2:172.21.15.80:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fca2000f040 con 0x562caf2e8780 2024-04-24T23:29:49.735 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.725+0000 7fca345e6740 10 client.15618 fetch_fsmap finished waiting for FSMap version 25 2024-04-24T23:29:49.735 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.725+0000 7fca345e6740 10 client.15618 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:29:49.735 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.725+0000 7fca345e6740 20 client.15618 populate_metadata read hostname 'smithi080' 2024-04-24T23:29:49.736 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.725+0000 7fca345e6740 1 --2- 172.21.15.80:0/3281401380 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x562caf2fdc00 0x562caf3000a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:49.736 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.725+0000 7fca345e6740 4 client.15618 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:29:49.736 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.725+0000 7fca345e6740 1 -- 172.21.15.80:0/3281401380 --> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562caf21f5a0 con 0x562caf2fdc00 2024-04-24T23:29:49.736 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.726+0000 7fca2affd640 1 --2- 172.21.15.80:0/3281401380 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x562caf2fdc00 0x562caf3000a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:49.736 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.726+0000 7fca0eff5640 10 client.15618 ms_handle_connect on v2:172.21.15.148:6838/2952757594 2024-04-24T23:29:49.736 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.726+0000 7fca2affd640 1 --2- 172.21.15.80:0/3281401380 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x562caf2fdc00 0x562caf3000a0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cffe14a52405cc88 server_cookie=6e3bd1d579aca188 in_seq=0 out_seq=0 2024-04-24T23:29:49.736 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.727+0000 7fca0eff5640 1 -- 172.21.15.80:0/3281401380 <== mds.0 v2:172.21.15.148:6838/2952757594 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562caf21f5a0 con 0x562caf2fdc00 2024-04-24T23:29:49.736 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.727+0000 7fca0eff5640 10 client.15618 handle_command_reply: tid=0 2024-04-24T23:29:49.736 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.727+0000 7fca345e6740 1 client.15618 shutdown 2024-04-24T23:29:49.736 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.727+0000 7fca345e6740 1 -- 172.21.15.80:0/3281401380 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562caf2e8780 msgr2=0x562caf2f2510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:49.736 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.727+0000 7fca345e6740 1 --2- 172.21.15.80:0/3281401380 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562caf2e8780 0x562caf2f2510 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fca200127b0 tx=0x7fca20004730 comp rx=0 tx=0).stop 2024-04-24T23:29:49.736 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.727+0000 7fca345e6740 1 -- 172.21.15.80:0/3281401380 shutdown_connections 2024-04-24T23:29:49.736 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.727+0000 7fca345e6740 1 --2- 172.21.15.80:0/3281401380 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x562caf2fdc00 0x562caf3000a0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:49.736 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.727+0000 7fca345e6740 1 --2- 172.21.15.80:0/3281401380 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562caf2ed8f0 0x562caf2f9550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:49.736 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.727+0000 7fca345e6740 1 --2- 172.21.15.80:0/3281401380 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562caf2e8780 0x562caf2f2510 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:49.737 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.727+0000 7fca345e6740 1 --2- 172.21.15.80:0/3281401380 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562caf2e7d90 0x562caf2f1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:49.737 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.727+0000 7fca345e6740 1 -- 172.21.15.80:0/3281401380 >> 172.21.15.80:0/3281401380 conn(0x562caf2af950 msgr2=0x562caf2e2950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:49.737 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.727+0000 7fca0eff5640 0 client.15618 ms_handle_reset on v2:172.21.15.148:6838/2952757594 2024-04-24T23:29:49.737 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.727+0000 7fca345e6740 1 -- 172.21.15.80:0/3281401380 shutdown_connections 2024-04-24T23:29:49.737 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.727+0000 7fca345e6740 1 -- 172.21.15.80:0/3281401380 wait complete. 2024-04-24T23:29:49.737 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.727+0000 7fca345e6740 20 client.15618 trim_cache size 0 max 16384 2024-04-24T23:29:49.737 INFO:tasks.workunit.client.0.smithi080.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-24T23:29:49.750 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.745+0000 7fca345e6740 1 Processor -- start 2024-04-24T23:29:49.750 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.745+0000 7fca345e6740 1 -- start start 2024-04-24T23:29:49.750 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.745+0000 7fca345e6740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562caf2e7d90 0x562caf2e27d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:49.750 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.745+0000 7fca345e6740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562caf2e8780 0x562caf2e2d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:49.750 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.745+0000 7fca345e6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562caf2ed8f0 0x562caf2f0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:49.750 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.745+0000 7fca345e6740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fca18012590 con 0x562caf2e8780 2024-04-24T23:29:49.751 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.745+0000 7fca345e6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fca1800fe60 con 0x562caf2ed8f0 2024-04-24T23:29:49.751 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.745+0000 7fca345e6740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fca1800dcb0 con 0x562caf2e7d90 2024-04-24T23:29:49.751 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.745+0000 7fca2a7fc640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562caf2e8780 0x562caf2e2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:49.751 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.745+0000 7fca2a7fc640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562caf2e8780 0x562caf2e2d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:52254/0 (socket says 172.21.15.80:52254) 2024-04-24T23:29:49.751 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.745+0000 7fca2a7fc640 1 -- 172.21.15.80:0/3547743551 learned_addr learned my addr 172.21.15.80:0/3547743551 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:29:49.751 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.745+0000 7fca2affd640 1 --2- 172.21.15.80:0/3547743551 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562caf2e7d90 0x562caf2e27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:49.751 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.745+0000 7fca2a7fc640 1 -- 172.21.15.80:0/3547743551 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562caf2e7d90 msgr2=0x562caf2e27d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:49.751 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.745+0000 7fca2a7fc640 1 --2- 172.21.15.80:0/3547743551 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562caf2e7d90 0x562caf2e27d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:49.751 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.745+0000 7fca2a7fc640 1 -- 172.21.15.80:0/3547743551 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562caf2ed8f0 msgr2=0x562caf2f0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:49.751 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.745+0000 7fca2a7fc640 1 --2- 172.21.15.80:0/3547743551 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562caf2ed8f0 0x562caf2f0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:49.751 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.745+0000 7fca2a7fc640 1 -- 172.21.15.80:0/3547743551 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562caef1f170 con 0x562caf2e8780 2024-04-24T23:29:49.751 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.745+0000 7fca2affd640 1 --2- 172.21.15.80:0/3547743551 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562caf2e7d90 0x562caf2e27d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:29:49.751 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.746+0000 7fca2a7fc640 1 --2- 172.21.15.80:0/3547743551 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562caf2e8780 0x562caf2e2d10 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fca200153f0 tx=0x7fca200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2179a92ceef4f7c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:49.751 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.746+0000 7fca10ff9640 1 -- 172.21.15.80:0/3547743551 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca20037020 con 0x562caf2e8780 2024-04-24T23:29:49.752 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.746+0000 7fca10ff9640 1 -- 172.21.15.80:0/3547743551 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca20013d60 con 0x562caf2e8780 2024-04-24T23:29:49.752 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.746+0000 7fca10ff9640 1 -- 172.21.15.80:0/3547743551 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca20024050 con 0x562caf2e8780 2024-04-24T23:29:49.752 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.746+0000 7fca345e6740 1 -- 172.21.15.80:0/3547743551 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562caf2e8780 msgr2=0x562caf2e2d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:49.752 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.746+0000 7fca345e6740 1 --2- 172.21.15.80:0/3547743551 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562caf2e8780 0x562caf2e2d10 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fca200153f0 tx=0x7fca200046c0 comp rx=0 tx=0).stop 2024-04-24T23:29:49.752 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.746+0000 7fca345e6740 1 -- 172.21.15.80:0/3547743551 shutdown_connections 2024-04-24T23:29:49.752 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.746+0000 7fca345e6740 1 --2- 172.21.15.80:0/3547743551 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562caf2ed8f0 0x562caf2f0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:49.752 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.746+0000 7fca345e6740 1 --2- 172.21.15.80:0/3547743551 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562caf2e8780 0x562caf2e2d10 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:49.752 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.746+0000 7fca345e6740 1 --2- 172.21.15.80:0/3547743551 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562caf2e7d90 0x562caf2e27d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:49.752 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.746+0000 7fca345e6740 1 -- 172.21.15.80:0/3547743551 >> 172.21.15.80:0/3547743551 conn(0x562caf2af950 msgr2=0x562caf2e20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:49.752 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.746+0000 7fca345e6740 1 -- 172.21.15.80:0/3547743551 shutdown_connections 2024-04-24T23:29:49.752 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.746+0000 7fca345e6740 1 -- 172.21.15.80:0/3547743551 wait complete. 2024-04-24T23:29:49.752 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.747+0000 7fca345e6740 1 -- start start 2024-04-24T23:29:49.752 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.747+0000 7fca345e6740 1 Processor -- start 2024-04-24T23:29:49.752 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.747+0000 7fca345e6740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562caf2e7d90 0x562caf35f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:49.752 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.747+0000 7fca345e6740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562caf2e8780 0x562caf35f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:49.753 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.747+0000 7fca345e6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562caf2ed8f0 0x562caf3668e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:49.753 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.747+0000 7fca345e6740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fca1800e320 con 0x562caf2e8780 2024-04-24T23:29:49.753 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.747+0000 7fca345e6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fca1800f790 con 0x562caf2ed8f0 2024-04-24T23:29:49.753 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.747+0000 7fca345e6740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fca1800e9f0 con 0x562caf2e7d90 2024-04-24T23:29:49.753 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.747+0000 7fca345e6740 10 client.0 _ll_register_callbacks cb 0x7fca200124b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:29:49.753 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.747+0000 7fca2a7fc640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562caf2e8780 0x562caf35f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:49.753 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.747+0000 7fca2a7fc640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562caf2e8780 0x562caf35f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:52270/0 (socket says 172.21.15.80:52270) 2024-04-24T23:29:49.753 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.747+0000 7fca2a7fc640 1 -- 172.21.15.80:0/1924757411 learned_addr learned my addr 172.21.15.80:0/1924757411 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:29:49.753 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.747+0000 7fca2affd640 1 --2- 172.21.15.80:0/1924757411 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562caf2e7d90 0x562caf35f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:49.753 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.747+0000 7fca2b7fe640 1 --2- 172.21.15.80:0/1924757411 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562caf2ed8f0 0x562caf3668e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:49.753 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.747+0000 7fca2a7fc640 1 -- 172.21.15.80:0/1924757411 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562caf2e7d90 msgr2=0x562caf35f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:49.753 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.747+0000 7fca2a7fc640 1 --2- 172.21.15.80:0/1924757411 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562caf2e7d90 0x562caf35f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:49.753 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.747+0000 7fca2a7fc640 1 -- 172.21.15.80:0/1924757411 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562caf2ed8f0 msgr2=0x562caf3668e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:49.753 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.747+0000 7fca2a7fc640 1 --2- 172.21.15.80:0/1924757411 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562caf2ed8f0 0x562caf3668e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:49.754 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.747+0000 7fca2a7fc640 1 -- 172.21.15.80:0/1924757411 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562caf366ed0 con 0x562caf2e8780 2024-04-24T23:29:49.754 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.747+0000 7fca2b7fe640 1 --2- 172.21.15.80:0/1924757411 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562caf2ed8f0 0x562caf3668e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:29:49.754 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.748+0000 7fca2a7fc640 1 --2- 172.21.15.80:0/1924757411 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562caf2e8780 0x562caf35f8a0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fca20005890 tx=0x7fca2001ff60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c54306b9d1d438f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:49.754 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.748+0000 7fca0eff5640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-04-24T23:29:49.754 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.748+0000 7fca0eff5640 1 -- 172.21.15.80:0/1924757411 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca20040020 con 0x562caf2e8780 2024-04-24T23:29:49.754 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.748+0000 7fca0eff5640 1 -- 172.21.15.80:0/1924757411 <== mon.0 v2:172.21.15.80:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fca2001f7a0 con 0x562caf2e8780 2024-04-24T23:29:49.754 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.748+0000 7fca345e6740 1 -- 172.21.15.80:0/1924757411 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca20013d60 con 0x562caf2e8780 2024-04-24T23:29:49.754 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.748+0000 7fca0eff5640 1 -- 172.21.15.80:0/1924757411 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fca20048050 con 0x562caf2e8780 2024-04-24T23:29:49.754 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.748+0000 7fca345e6740 10 client.15624 fetch_fsmap learned FSMap version 25 2024-04-24T23:29:49.754 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.748+0000 7fca345e6740 1 -- 172.21.15.80:0/1924757411 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562caf368180 con 0x562caf2e8780 2024-04-24T23:29:49.754 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.748+0000 7fca0eff5640 1 -- 172.21.15.80:0/1924757411 <== mon.0 v2:172.21.15.80:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca20013d60 con 0x562caf2e8780 2024-04-24T23:29:49.754 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.748+0000 7fca0eff5640 1 -- 172.21.15.80:0/1924757411 <== mon.0 v2:172.21.15.80:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fca20004b30 con 0x562caf2e8780 2024-04-24T23:29:49.754 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.748+0000 7fca345e6740 10 client.15624 fetch_fsmap finished waiting for FSMap version 25 2024-04-24T23:29:49.754 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.748+0000 7fca345e6740 10 client.15624 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:29:49.755 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.748+0000 7fca345e6740 20 client.15624 populate_metadata read hostname 'smithi080' 2024-04-24T23:29:49.755 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.748+0000 7fca345e6740 1 --2- 172.21.15.80:0/1924757411 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x562caf368b50 0x562caf36af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:49.755 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.748+0000 7fca345e6740 4 client.15624 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T23:29:49.755 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.748+0000 7fca345e6740 1 -- 172.21.15.80:0/1924757411 --> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x562caf258800 con 0x562caf368b50 2024-04-24T23:29:49.755 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.749+0000 7fca2affd640 1 --2- 172.21.15.80:0/1924757411 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x562caf368b50 0x562caf36af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:49.755 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.749+0000 7fca0eff5640 10 client.15624 ms_handle_connect on v2:172.21.15.148:6838/2952757594 2024-04-24T23:29:49.755 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.749+0000 7fca2affd640 1 --2- 172.21.15.80:0/1924757411 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x562caf368b50 0x562caf36af70 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d9fa0bf3f66d5e5 server_cookie=76650bd86e46f7c5 in_seq=0 out_seq=0 2024-04-24T23:29:49.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.750+0000 7fca0eff5640 1 -- 172.21.15.80:0/1924757411 <== mds.0 v2:172.21.15.148:6838/2952757594 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x562caf258800 con 0x562caf368b50 2024-04-24T23:29:49.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.750+0000 7fca0eff5640 10 client.15624 handle_command_reply: tid=0 2024-04-24T23:29:49.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.750+0000 7fca345e6740 1 client.15624 shutdown 2024-04-24T23:29:49.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.750+0000 7fca345e6740 1 -- 172.21.15.80:0/1924757411 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562caf2e8780 msgr2=0x562caf35f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:49.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.750+0000 7fca345e6740 1 --2- 172.21.15.80:0/1924757411 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562caf2e8780 0x562caf35f8a0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fca20005890 tx=0x7fca2001ff60 comp rx=0 tx=0).stop 2024-04-24T23:29:49.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.750+0000 7fca345e6740 1 -- 172.21.15.80:0/1924757411 shutdown_connections 2024-04-24T23:29:49.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.750+0000 7fca345e6740 1 --2- 172.21.15.80:0/1924757411 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x562caf368b50 0x562caf36af70 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:49.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.750+0000 7fca345e6740 1 --2- 172.21.15.80:0/1924757411 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562caf2ed8f0 0x562caf3668e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:49.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.750+0000 7fca345e6740 1 --2- 172.21.15.80:0/1924757411 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562caf2e8780 0x562caf35f8a0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:49.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.750+0000 7fca345e6740 1 --2- 172.21.15.80:0/1924757411 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562caf2e7d90 0x562caf35f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:49.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.750+0000 7fca345e6740 1 -- 172.21.15.80:0/1924757411 >> 172.21.15.80:0/1924757411 conn(0x562caf2af950 msgr2=0x562caf2e2950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:49.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.750+0000 7fca0eff5640 0 client.15624 ms_handle_reset on v2:172.21.15.148:6838/2952757594 2024-04-24T23:29:49.758 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.750+0000 7fca345e6740 1 -- 172.21.15.80:0/1924757411 shutdown_connections 2024-04-24T23:29:49.758 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.750+0000 7fca345e6740 1 -- 172.21.15.80:0/1924757411 wait complete. 2024-04-24T23:29:49.758 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.751+0000 7fca345e6740 20 client.15624 trim_cache size 0 max 16384 2024-04-24T23:29:49.758 INFO:teuthology.orchestra.run.smithi080.stdout:{ 2024-04-24T23:29:49.758 INFO:teuthology.orchestra.run.smithi080.stdout: "return_code": 0, 2024-04-24T23:29:49.758 INFO:teuthology.orchestra.run.smithi080.stdout: "scrub_tag": "d1b781f0-123c-4804-87af-ff211786196d", 2024-04-24T23:29:49.758 INFO:teuthology.orchestra.run.smithi080.stdout: "mode": "asynchronous" 2024-04-24T23:29:49.758 INFO:teuthology.orchestra.run.smithi080.stdout:} 2024-04-24T23:29:49.759 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.751+0000 7fca317f9640 1 -- 172.21.15.80:0/712995082 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fca140a2b90 msgr2=0x7fca140a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:49.759 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.751+0000 7fca317f9640 1 --2- 172.21.15.80:0/712995082 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fca140a2b90 0x7fca140a5050 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fca20005d00 tx=0x7fca20015840 comp rx=0 tx=0).stop 2024-04-24T23:29:49.759 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.751+0000 7fca317f9640 1 -- 172.21.15.80:0/712995082 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fca2c103340 msgr2=0x7fca2c19f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:49.759 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.751+0000 7fca317f9640 1 --2- 172.21.15.80:0/712995082 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fca2c103340 0x7fca2c19f0c0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fca2400d950 tx=0x7fca2400de10 comp rx=0 tx=0).stop 2024-04-24T23:29:49.759 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.751+0000 7fca317f9640 1 -- 172.21.15.80:0/712995082 shutdown_connections 2024-04-24T23:29:49.759 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.751+0000 7fca317f9640 1 --2- 172.21.15.80:0/712995082 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fca140a2b90 0x7fca140a5050 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:49.759 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.751+0000 7fca317f9640 1 --2- 172.21.15.80:0/712995082 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fca2c103340 0x7fca2c19f0c0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:49.760 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.751+0000 7fca317f9640 1 --2- 172.21.15.80:0/712995082 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fca2c102980 0x7fca2c198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:49.760 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.751+0000 7fca317f9640 1 --2- 172.21.15.80:0/712995082 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fca2c101790 0x7fca2c197b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:49.760 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.751+0000 7fca317f9640 1 -- 172.21.15.80:0/712995082 >> 172.21.15.80:0/712995082 conn(0x7fca2c0fcd20 msgr2=0x7fca2c105df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:49.760 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.751+0000 7fca317f9640 1 -- 172.21.15.80:0/712995082 shutdown_connections 2024-04-24T23:29:49.760 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.751+0000 7fca317f9640 1 -- 172.21.15.80:0/712995082 wait complete. 2024-04-24T23:29:49.763 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T23:29:49.763 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T23:29:49.906 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.903+0000 7f2951134640 1 -- 172.21.15.80:0/476193465 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f294c101890 msgr2=0x7f294c101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:49.906 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.903+0000 7f2951134640 1 --2- 172.21.15.80:0/476193465 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f294c101890 0x7f294c101d10 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f2934009900 tx=0x7f293402a410 comp rx=0 tx=0).stop 2024-04-24T23:29:49.906 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.903+0000 7f2951134640 1 -- 172.21.15.80:0/476193465 shutdown_connections 2024-04-24T23:29:49.906 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.903+0000 7f2951134640 1 --2- 172.21.15.80:0/476193465 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f294c102250 0x7f294c108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:49.906 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.903+0000 7f2951134640 1 --2- 172.21.15.80:0/476193465 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f294c101890 0x7f294c101d10 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:49.906 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.903+0000 7f2951134640 1 --2- 172.21.15.80:0/476193465 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f294c1006a0 0x7f294c100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:49.906 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.903+0000 7f2951134640 1 -- 172.21.15.80:0/476193465 >> 172.21.15.80:0/476193465 conn(0x7f294c0fbc30 msgr2=0x7f294c0fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:49.907 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.903+0000 7f2951134640 1 -- 172.21.15.80:0/476193465 shutdown_connections 2024-04-24T23:29:49.907 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.903+0000 7f2951134640 1 -- 172.21.15.80:0/476193465 wait complete. 2024-04-24T23:29:49.907 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.904+0000 7f2951134640 1 Processor -- start 2024-04-24T23:29:49.907 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.904+0000 7f2951134640 1 -- start start 2024-04-24T23:29:49.907 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.904+0000 7f2951134640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f294c1006a0 0x7f294c197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:49.907 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.904+0000 7f2951134640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f294c101890 0x7f294c197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:49.907 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.904+0000 7f2951134640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f294c102250 0x7f294c19f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:49.907 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.904+0000 7f2951134640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f294c10d480 con 0x7f294c1006a0 2024-04-24T23:29:49.907 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.904+0000 7f2951134640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f294c10d300 con 0x7f294c102250 2024-04-24T23:29:49.907 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.904+0000 7f2951134640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f294c10d600 con 0x7f294c101890 2024-04-24T23:29:49.907 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.904+0000 7f294ad9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f294c1006a0 0x7f294c197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:49.907 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.904+0000 7f294ad9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f294c1006a0 0x7f294c197a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:52282/0 (socket says 172.21.15.80:52282) 2024-04-24T23:29:49.907 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.904+0000 7f294ad9d640 1 -- 172.21.15.80:0/1558268024 learned_addr learned my addr 172.21.15.80:0/1558268024 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:29:49.907 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.904+0000 7f2943fff640 1 --2- 172.21.15.80:0/1558268024 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f294c101890 0x7f294c197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:49.908 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.904+0000 7f294b59e640 1 --2- 172.21.15.80:0/1558268024 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f294c102250 0x7f294c19f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:49.908 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.904+0000 7f294ad9d640 1 -- 172.21.15.80:0/1558268024 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f294c101890 msgr2=0x7f294c197fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:49.908 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.904+0000 7f294ad9d640 1 --2- 172.21.15.80:0/1558268024 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f294c101890 0x7f294c197fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:49.908 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.904+0000 7f294ad9d640 1 -- 172.21.15.80:0/1558268024 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f294c102250 msgr2=0x7f294c19f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:49.908 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.904+0000 7f294ad9d640 1 --2- 172.21.15.80:0/1558268024 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f294c102250 0x7f294c19f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:49.908 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.904+0000 7f294ad9d640 1 -- 172.21.15.80:0/1558268024 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f294c19f6b0 con 0x7f294c1006a0 2024-04-24T23:29:49.908 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.905+0000 7f294ad9d640 1 --2- 172.21.15.80:0/1558268024 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f294c1006a0 0x7f294c197a90 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f293c00c970 tx=0x7f293c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:49.908 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.905+0000 7f2948d99640 1 -- 172.21.15.80:0/1558268024 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f293c007c10 con 0x7f294c1006a0 2024-04-24T23:29:49.908 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.905+0000 7f2951134640 1 -- 172.21.15.80:0/1558268024 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f294c19f9a0 con 0x7f294c1006a0 2024-04-24T23:29:49.908 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.905+0000 7f2948d99640 1 -- 172.21.15.80:0/1558268024 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f293c007db0 con 0x7f294c1006a0 2024-04-24T23:29:49.908 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.905+0000 7f2948d99640 1 -- 172.21.15.80:0/1558268024 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f293c00f940 con 0x7f294c1006a0 2024-04-24T23:29:49.908 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.905+0000 7f2951134640 1 -- 172.21.15.80:0/1558268024 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f294c19ff40 con 0x7f294c1006a0 2024-04-24T23:29:49.915 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.907+0000 7f2948d99640 1 -- 172.21.15.80:0/1558268024 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f293c00fae0 con 0x7f294c1006a0 2024-04-24T23:29:49.915 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.907+0000 7f2953e41740 1 Processor -- start 2024-04-24T23:29:49.915 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.907+0000 7f2953e41740 1 -- start start 2024-04-24T23:29:49.915 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.908+0000 7f2953e41740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56533815c650 0x56533815ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:49.915 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.908+0000 7f2953e41740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56533815d040 0x565338161b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:49.915 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.908+0000 7f2953e41740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5653381621b0 0x565338164640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:49.915 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.908+0000 7f2953e41740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x5653380bc6b0 con 0x5653381621b0 2024-04-24T23:29:49.916 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.908+0000 7f2953e41740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x565338064880 con 0x56533815d040 2024-04-24T23:29:49.916 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.908+0000 7f2953e41740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5653380b8670 con 0x56533815c650 2024-04-24T23:29:49.916 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.908+0000 7f2948d99640 1 --2- 172.21.15.80:0/1558268024 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f29380a2b90 0x7f29380a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:49.916 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.908+0000 7f2943fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5653381621b0 0x565338164640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:49.916 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.908+0000 7f2943fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5653381621b0 0x565338164640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:52288/0 (socket says 172.21.15.80:52288) 2024-04-24T23:29:49.916 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.908+0000 7f294b59e640 1 --2- 172.21.15.80:0/1558268024 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f29380a2b90 0x7f29380a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:49.916 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.908+0000 7f2943fff640 1 -- 172.21.15.80:0/51682115 learned_addr learned my addr 172.21.15.80:0/51682115 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:29:49.916 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.908+0000 7f294b59e640 1 --2- 172.21.15.80:0/51682115 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56533815c650 0x56533815ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:49.916 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.908+0000 7f2948d99640 1 -- 172.21.15.80:0/1558268024 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f293c0cf800 con 0x7f294c1006a0 2024-04-24T23:29:49.916 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.908+0000 7f294ad9d640 1 --2- 172.21.15.80:0/51682115 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56533815d040 0x565338161b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:49.916 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.908+0000 7f2943fff640 1 -- 172.21.15.80:0/51682115 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56533815c650 msgr2=0x56533815ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:49.916 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.908+0000 7f2943fff640 1 --2- 172.21.15.80:0/51682115 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56533815c650 0x56533815ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:49.916 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.908+0000 7f2943fff640 1 -- 172.21.15.80:0/51682115 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56533815d040 msgr2=0x565338161b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:49.917 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.908+0000 7f2943fff640 1 --2- 172.21.15.80:0/51682115 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56533815d040 0x565338161b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:49.917 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.908+0000 7f2943fff640 1 -- 172.21.15.80:0/51682115 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565337e9a7d0 con 0x5653381621b0 2024-04-24T23:29:49.917 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.908+0000 7f294ad9d640 1 --2- 172.21.15.80:0/51682115 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56533815d040 0x565338161b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:29:49.917 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.908+0000 7f294b59e640 1 --2- 172.21.15.80:0/1558268024 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f29380a2b90 0x7f29380a5050 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x56533815d8a0 tx=0x7f29440015a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:49.917 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.908+0000 7f2943fff640 1 --2- 172.21.15.80:0/51682115 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5653381621b0 0x565338164640 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f2934031610 tx=0x7f2934005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=919418f7dd98808f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:49.917 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.909+0000 7f29417fa640 1 -- 172.21.15.80:0/51682115 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2934040070 con 0x5653381621b0 2024-04-24T23:29:49.917 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.909+0000 7f29417fa640 1 -- 172.21.15.80:0/51682115 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2934005500 con 0x5653381621b0 2024-04-24T23:29:49.917 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.909+0000 7f29417fa640 1 -- 172.21.15.80:0/51682115 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2934035040 con 0x5653381621b0 2024-04-24T23:29:49.917 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.909+0000 7f2953e41740 1 -- 172.21.15.80:0/51682115 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5653381621b0 msgr2=0x565338164640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:49.917 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.909+0000 7f2953e41740 1 --2- 172.21.15.80:0/51682115 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5653381621b0 0x565338164640 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f2934031610 tx=0x7f2934005c00 comp rx=0 tx=0).stop 2024-04-24T23:29:49.917 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.909+0000 7f2953e41740 1 -- 172.21.15.80:0/51682115 shutdown_connections 2024-04-24T23:29:49.917 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.909+0000 7f2953e41740 1 --2- 172.21.15.80:0/51682115 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5653381621b0 0x565338164640 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:49.917 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.909+0000 7f2953e41740 1 --2- 172.21.15.80:0/51682115 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56533815d040 0x565338161b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:49.918 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.909+0000 7f2953e41740 1 --2- 172.21.15.80:0/51682115 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56533815c650 0x56533815ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:49.918 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.909+0000 7f2953e41740 1 -- 172.21.15.80:0/51682115 >> 172.21.15.80:0/51682115 conn(0x5653381538b0 msgr2=0x565338153cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:49.918 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.909+0000 7f2953e41740 1 -- 172.21.15.80:0/51682115 shutdown_connections 2024-04-24T23:29:49.918 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.909+0000 7f2953e41740 1 -- 172.21.15.80:0/51682115 wait complete. 2024-04-24T23:29:49.918 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.910+0000 7f2953e41740 1 -- start start 2024-04-24T23:29:49.918 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.910+0000 7f2953e41740 1 Processor -- start 2024-04-24T23:29:49.918 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.910+0000 7f2953e41740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56533815c650 0x565338166a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:49.918 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.910+0000 7f2953e41740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56533815d040 0x565338166f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:49.918 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.910+0000 7f2953e41740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5653381621b0 0x56533816dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:49.918 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.910+0000 7f2953e41740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x565338165100 con 0x56533815c650 2024-04-24T23:29:49.918 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.910+0000 7f2953e41740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x565338077f50 con 0x56533815d040 2024-04-24T23:29:49.918 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.910+0000 7f2953e41740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x565338165280 con 0x5653381621b0 2024-04-24T23:29:49.918 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.910+0000 7f2953e41740 10 client.0 _ll_register_callbacks cb 0x5653380b3000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:29:49.918 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.910+0000 7f294ad9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56533815c650 0x565338166a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:49.919 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.910+0000 7f294ad9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56533815c650 0x565338166a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:52292/0 (socket says 172.21.15.80:52292) 2024-04-24T23:29:49.919 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.910+0000 7f294ad9d640 1 -- 172.21.15.80:0/1385309749 learned_addr learned my addr 172.21.15.80:0/1385309749 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:29:49.919 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.910+0000 7f294b59e640 1 --2- 172.21.15.80:0/1385309749 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5653381621b0 0x56533816dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:49.919 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.910+0000 7f294ad9d640 1 -- 172.21.15.80:0/1385309749 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5653381621b0 msgr2=0x56533816dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:49.919 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.910+0000 7f2943fff640 1 --2- 172.21.15.80:0/1385309749 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56533815d040 0x565338166f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:49.919 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.910+0000 7f294ad9d640 1 --2- 172.21.15.80:0/1385309749 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5653381621b0 0x56533816dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:49.919 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.910+0000 7f294ad9d640 1 -- 172.21.15.80:0/1385309749 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56533815d040 msgr2=0x565338166f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:49.919 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.910+0000 7f294ad9d640 1 --2- 172.21.15.80:0/1385309749 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56533815d040 0x565338166f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:49.919 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.910+0000 7f294ad9d640 1 -- 172.21.15.80:0/1385309749 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565337dd03f0 con 0x56533815c650 2024-04-24T23:29:49.919 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.910+0000 7f2943fff640 1 --2- 172.21.15.80:0/1385309749 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56533815d040 0x565338166f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:29:49.920 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.910+0000 7f294ad9d640 1 --2- 172.21.15.80:0/1385309749 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56533815c650 0x565338166a50 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f293c007590 tx=0x7f293c09c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87d80c9ad2ea11b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:49.920 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.910+0000 7f29327fc640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-04-24T23:29:49.921 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.910+0000 7f29327fc640 1 -- 172.21.15.80:0/1385309749 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f293c018b80 con 0x56533815c650 2024-04-24T23:29:49.921 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.911+0000 7f29327fc640 1 -- 172.21.15.80:0/1385309749 <== mon.0 v2:172.21.15.80:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f293c0ba5c0 con 0x56533815c650 2024-04-24T23:29:49.921 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.911+0000 7f2953e41740 1 -- 172.21.15.80:0/1385309749 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2934005500 con 0x56533815c650 2024-04-24T23:29:49.921 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.911+0000 7f29327fc640 1 -- 172.21.15.80:0/1385309749 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f293c0a4470 con 0x56533815c650 2024-04-24T23:29:49.921 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.911+0000 7f2953e41740 10 client.15636 fetch_fsmap learned FSMap version 25 2024-04-24T23:29:49.921 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.911+0000 7f2953e41740 1 -- 172.21.15.80:0/1385309749 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x565337d900f0 con 0x56533815c650 2024-04-24T23:29:49.921 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.911+0000 7f29327fc640 1 -- 172.21.15.80:0/1385309749 <== mon.0 v2:172.21.15.80:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2934005500 con 0x56533815c650 2024-04-24T23:29:49.921 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.911+0000 7f29327fc640 1 -- 172.21.15.80:0/1385309749 <== mon.0 v2:172.21.15.80:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f293c0b3070 con 0x56533815c650 2024-04-24T23:29:49.921 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.911+0000 7f2953e41740 10 client.15636 fetch_fsmap finished waiting for FSMap version 25 2024-04-24T23:29:49.921 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.911+0000 7f2953e41740 10 client.15636 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:29:49.921 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.911+0000 7f2953e41740 20 client.15636 populate_metadata read hostname 'smithi080' 2024-04-24T23:29:49.921 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.911+0000 7f2953e41740 1 --2- 172.21.15.80:0/1385309749 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x5653381728c0 0x565338174d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:49.921 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.911+0000 7f2953e41740 4 client.15636 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:29:49.921 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.911+0000 7f2953e41740 1 -- 172.21.15.80:0/1385309749 --> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5653380c9bf0 con 0x5653381728c0 2024-04-24T23:29:49.922 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.911+0000 7f2943fff640 1 --2- 172.21.15.80:0/1385309749 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x5653381728c0 0x565338174d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:49.922 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.912+0000 7f2943fff640 1 --2- 172.21.15.80:0/1385309749 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x5653381728c0 0x565338174d50 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11aa03e71b24eb4 server_cookie=32a0ddddf4097367 in_seq=0 out_seq=0 2024-04-24T23:29:49.922 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.912+0000 7f29327fc640 10 client.15636 ms_handle_connect on v2:172.21.15.148:6838/2952757594 2024-04-24T23:29:49.922 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.912+0000 7f29327fc640 1 -- 172.21.15.80:0/1385309749 <== mds.0 v2:172.21.15.148:6838/2952757594 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5653380c9bf0 con 0x5653381728c0 2024-04-24T23:29:49.922 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.912+0000 7f29327fc640 10 client.15636 handle_command_reply: tid=0 2024-04-24T23:29:49.922 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.912+0000 7f2953e41740 1 client.15636 shutdown 2024-04-24T23:29:49.922 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.913+0000 7f2953e41740 1 -- 172.21.15.80:0/1385309749 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56533815c650 msgr2=0x565338166a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:49.922 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.913+0000 7f2953e41740 1 --2- 172.21.15.80:0/1385309749 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56533815c650 0x565338166a50 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f293c007590 tx=0x7f293c09c560 comp rx=0 tx=0).stop 2024-04-24T23:29:49.922 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.913+0000 7f2953e41740 1 -- 172.21.15.80:0/1385309749 shutdown_connections 2024-04-24T23:29:49.922 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.913+0000 7f2953e41740 1 --2- 172.21.15.80:0/1385309749 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x5653381728c0 0x565338174d50 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:49.922 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.913+0000 7f2953e41740 1 --2- 172.21.15.80:0/1385309749 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5653381621b0 0x56533816dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:49.922 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.913+0000 7f2953e41740 1 --2- 172.21.15.80:0/1385309749 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56533815d040 0x565338166f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:49.922 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.913+0000 7f29327fc640 0 client.15636 ms_handle_reset on v2:172.21.15.148:6838/2952757594 2024-04-24T23:29:49.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.913+0000 7f2953e41740 1 --2- 172.21.15.80:0/1385309749 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56533815c650 0x565338166a50 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:49.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.913+0000 7f2953e41740 1 -- 172.21.15.80:0/1385309749 >> 172.21.15.80:0/1385309749 conn(0x5653381538b0 msgr2=0x565338152af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:49.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.913+0000 7f2953e41740 1 -- 172.21.15.80:0/1385309749 shutdown_connections 2024-04-24T23:29:49.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.913+0000 7f2953e41740 1 -- 172.21.15.80:0/1385309749 wait complete. 2024-04-24T23:29:49.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.913+0000 7f2953e41740 20 client.15636 trim_cache size 0 max 16384 2024-04-24T23:29:49.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.930+0000 7f2953e41740 1 Processor -- start 2024-04-24T23:29:49.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.930+0000 7f2953e41740 1 -- start start 2024-04-24T23:29:49.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.930+0000 7f2953e41740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56533815c650 0x565338166660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:49.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.930+0000 7f2953e41740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56533815d040 0x56533816df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:49.937 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.930+0000 7f2953e41740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5653381621b0 0x565338166ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:49.937 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.930+0000 7f2953e41740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f294c1aab30 con 0x5653381621b0 2024-04-24T23:29:49.937 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.930+0000 7f2953e41740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f294c1a8400 con 0x56533815c650 2024-04-24T23:29:49.937 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.930+0000 7f2953e41740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f294c1a6250 con 0x56533815d040 2024-04-24T23:29:49.937 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.930+0000 7f294b59e640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5653381621b0 0x565338166ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:49.937 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.930+0000 7f294b59e640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5653381621b0 0x565338166ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:52308/0 (socket says 172.21.15.80:52308) 2024-04-24T23:29:49.937 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.930+0000 7f294b59e640 1 -- 172.21.15.80:0/880890120 learned_addr learned my addr 172.21.15.80:0/880890120 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:29:49.937 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.930+0000 7f294ad9d640 1 --2- 172.21.15.80:0/880890120 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56533815c650 0x565338166660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:49.937 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.930+0000 7f294b59e640 1 -- 172.21.15.80:0/880890120 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56533815d040 msgr2=0x56533816df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:49.937 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.931+0000 7f2943fff640 1 --2- 172.21.15.80:0/880890120 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56533815d040 0x56533816df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:49.937 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.931+0000 7f294b59e640 1 --2- 172.21.15.80:0/880890120 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56533815d040 0x56533816df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:49.937 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.931+0000 7f294b59e640 1 -- 172.21.15.80:0/880890120 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56533815c650 msgr2=0x565338166660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:49.937 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.931+0000 7f294b59e640 1 --2- 172.21.15.80:0/880890120 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56533815c650 0x565338166660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:49.938 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.931+0000 7f294b59e640 1 -- 172.21.15.80:0/880890120 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653381533f0 con 0x5653381621b0 2024-04-24T23:29:49.938 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.931+0000 7f294ad9d640 1 --2- 172.21.15.80:0/880890120 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56533815c650 0x565338166660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:29:49.938 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.931+0000 7f2943fff640 1 --2- 172.21.15.80:0/880890120 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56533815d040 0x56533816df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:29:49.938 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.931+0000 7f294b59e640 1 --2- 172.21.15.80:0/880890120 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5653381621b0 0x565338166ba0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f2944001870 tx=0x7f2944001300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57ecb0de08ee1af5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:49.938 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.931+0000 7f2940ff9640 1 -- 172.21.15.80:0/880890120 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2944029070 con 0x5653381621b0 2024-04-24T23:29:49.938 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.931+0000 7f2940ff9640 1 -- 172.21.15.80:0/880890120 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2944018de0 con 0x5653381621b0 2024-04-24T23:29:49.938 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.931+0000 7f2940ff9640 1 -- 172.21.15.80:0/880890120 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2944024e00 con 0x5653381621b0 2024-04-24T23:29:49.938 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.931+0000 7f2953e41740 1 -- 172.21.15.80:0/880890120 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5653381621b0 msgr2=0x565338166ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:49.938 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.931+0000 7f2953e41740 1 --2- 172.21.15.80:0/880890120 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5653381621b0 0x565338166ba0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f2944001870 tx=0x7f2944001300 comp rx=0 tx=0).stop 2024-04-24T23:29:49.938 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.932+0000 7f2953e41740 1 -- 172.21.15.80:0/880890120 shutdown_connections 2024-04-24T23:29:49.938 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.932+0000 7f2953e41740 1 --2- 172.21.15.80:0/880890120 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5653381621b0 0x565338166ba0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:49.938 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.932+0000 7f2953e41740 1 --2- 172.21.15.80:0/880890120 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56533815d040 0x56533816df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:49.938 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.932+0000 7f2953e41740 1 --2- 172.21.15.80:0/880890120 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56533815c650 0x565338166660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:49.939 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.932+0000 7f2953e41740 1 -- 172.21.15.80:0/880890120 >> 172.21.15.80:0/880890120 conn(0x5653381538b0 msgr2=0x565338165650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:49.939 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.932+0000 7f2953e41740 1 -- 172.21.15.80:0/880890120 shutdown_connections 2024-04-24T23:29:49.939 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.932+0000 7f2953e41740 1 -- 172.21.15.80:0/880890120 wait complete. 2024-04-24T23:29:49.939 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.932+0000 7f2953e41740 1 -- start start 2024-04-24T23:29:49.939 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.932+0000 7f2953e41740 1 Processor -- start 2024-04-24T23:29:49.939 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.932+0000 7f2953e41740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56533815c650 0x56533816b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:49.939 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.932+0000 7f2953e41740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56533815d040 0x56533816b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:49.939 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.932+0000 7f2953e41740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5653381621b0 0x56533816c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:49.939 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.932+0000 7f2953e41740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f294c1a68c0 con 0x56533815c650 2024-04-24T23:29:49.939 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.932+0000 7f2953e41740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f294c1a7d30 con 0x5653381621b0 2024-04-24T23:29:49.939 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.932+0000 7f2953e41740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f294c1a6f90 con 0x56533815d040 2024-04-24T23:29:49.939 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.932+0000 7f2953e41740 10 client.0 _ll_register_callbacks cb 0x7f293c0046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:29:49.939 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.932+0000 7f294ad9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56533815c650 0x56533816b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:49.939 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.932+0000 7f294ad9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56533815c650 0x56533816b220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:52310/0 (socket says 172.21.15.80:52310) 2024-04-24T23:29:49.940 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.932+0000 7f294ad9d640 1 -- 172.21.15.80:0/3450880922 learned_addr learned my addr 172.21.15.80:0/3450880922 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:29:49.940 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.932+0000 7f2943fff640 1 --2- 172.21.15.80:0/3450880922 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56533815d040 0x56533816b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:49.940 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.932+0000 7f294ad9d640 1 -- 172.21.15.80:0/3450880922 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56533815d040 msgr2=0x56533816b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:49.940 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.932+0000 7f294ad9d640 1 --2- 172.21.15.80:0/3450880922 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56533815d040 0x56533816b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:49.940 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.932+0000 7f294ad9d640 1 -- 172.21.15.80:0/3450880922 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5653381621b0 msgr2=0x56533816c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:29:49.940 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.932+0000 7f294ad9d640 1 --2- 172.21.15.80:0/3450880922 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5653381621b0 0x56533816c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:49.940 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.932+0000 7f294ad9d640 1 -- 172.21.15.80:0/3450880922 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56533816ccf0 con 0x56533815c650 2024-04-24T23:29:49.940 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.933+0000 7f294ad9d640 1 --2- 172.21.15.80:0/3450880922 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56533815c650 0x56533816b220 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f293c099fd0 tx=0x7f293c098040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c82662497f6e501 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:29:49.940 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.933+0000 7f29327fc640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-04-24T23:29:49.940 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.933+0000 7f29327fc640 1 -- 172.21.15.80:0/3450880922 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f293c07e030 con 0x56533815c650 2024-04-24T23:29:49.940 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.933+0000 7f29327fc640 1 -- 172.21.15.80:0/3450880922 <== mon.0 v2:172.21.15.80:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f293c0ba5c0 con 0x56533815c650 2024-04-24T23:29:49.940 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.933+0000 7f2953e41740 1 -- 172.21.15.80:0/3450880922 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2944018de0 con 0x56533815c650 2024-04-24T23:29:49.940 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.933+0000 7f29327fc640 1 -- 172.21.15.80:0/3450880922 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f293c09b680 con 0x56533815c650 2024-04-24T23:29:49.940 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.933+0000 7f2953e41740 10 client.15642 fetch_fsmap learned FSMap version 25 2024-04-24T23:29:49.941 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.933+0000 7f2953e41740 1 -- 172.21.15.80:0/3450880922 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56533816dfe0 con 0x56533815c650 2024-04-24T23:29:49.941 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.933+0000 7f29327fc640 1 -- 172.21.15.80:0/3450880922 <== mon.0 v2:172.21.15.80:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2944018de0 con 0x56533815c650 2024-04-24T23:29:49.941 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.933+0000 7f29327fc640 1 -- 172.21.15.80:0/3450880922 <== mon.0 v2:172.21.15.80:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f293c0b3070 con 0x56533815c650 2024-04-24T23:29:49.941 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.933+0000 7f2953e41740 10 client.15642 fetch_fsmap finished waiting for FSMap version 25 2024-04-24T23:29:49.941 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.933+0000 7f2953e41740 10 client.15642 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:29:49.941 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.933+0000 7f2953e41740 20 client.15642 populate_metadata read hostname 'smithi080' 2024-04-24T23:29:49.941 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.933+0000 7f2953e41740 1 --2- 172.21.15.80:0/3450880922 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x5653381d92c0 0x5653381db6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:29:49.941 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.933+0000 7f2953e41740 4 client.15642 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:29:49.941 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.933+0000 7f2953e41740 1 -- 172.21.15.80:0/3450880922 --> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5653380c3f40 con 0x5653381d92c0 2024-04-24T23:29:49.941 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.933+0000 7f2943fff640 1 --2- 172.21.15.80:0/3450880922 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x5653381d92c0 0x5653381db6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:29:49.941 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.934+0000 7f29327fc640 10 client.15642 ms_handle_connect on v2:172.21.15.148:6838/2952757594 2024-04-24T23:29:49.941 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.934+0000 7f2943fff640 1 --2- 172.21.15.80:0/3450880922 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x5653381d92c0 0x5653381db6e0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0bee0b1842b2e6b server_cookie=201832c8a8fbebad in_seq=0 out_seq=0 2024-04-24T23:29:49.941 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.934+0000 7f29327fc640 1 -- 172.21.15.80:0/3450880922 <== mds.0 v2:172.21.15.148:6838/2952757594 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5653380c3f40 con 0x5653381d92c0 2024-04-24T23:29:49.941 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.934+0000 7f29327fc640 10 client.15642 handle_command_reply: tid=0 2024-04-24T23:29:49.941 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.934+0000 7f2953e41740 1 client.15642 shutdown 2024-04-24T23:29:49.942 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.934+0000 7f2953e41740 1 -- 172.21.15.80:0/3450880922 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56533815c650 msgr2=0x56533816b220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:49.942 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.934+0000 7f2953e41740 1 --2- 172.21.15.80:0/3450880922 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56533815c650 0x56533816b220 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f293c099fd0 tx=0x7f293c098040 comp rx=0 tx=0).stop 2024-04-24T23:29:49.942 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.934+0000 7f2953e41740 1 -- 172.21.15.80:0/3450880922 shutdown_connections 2024-04-24T23:29:49.942 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.934+0000 7f2953e41740 1 --2- 172.21.15.80:0/3450880922 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x5653381d92c0 0x5653381db6e0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:49.942 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.934+0000 7f2953e41740 1 --2- 172.21.15.80:0/3450880922 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5653381621b0 0x56533816c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:49.942 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.934+0000 7f29327fc640 0 client.15642 ms_handle_reset on v2:172.21.15.148:6838/2952757594 2024-04-24T23:29:49.942 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.934+0000 7f2953e41740 1 --2- 172.21.15.80:0/3450880922 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56533815d040 0x56533816b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:49.942 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.934+0000 7f2953e41740 1 --2- 172.21.15.80:0/3450880922 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56533815c650 0x56533816b220 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:49.942 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.934+0000 7f2953e41740 1 -- 172.21.15.80:0/3450880922 >> 172.21.15.80:0/3450880922 conn(0x5653381538b0 msgr2=0x565338166660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:49.942 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.934+0000 7f2953e41740 1 -- 172.21.15.80:0/3450880922 shutdown_connections 2024-04-24T23:29:49.942 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.934+0000 7f2953e41740 1 -- 172.21.15.80:0/3450880922 wait complete. 2024-04-24T23:29:49.942 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.934+0000 7f2953e41740 20 client.15642 trim_cache size 0 max 16384 2024-04-24T23:29:49.943 INFO:teuthology.orchestra.run.smithi080.stdout:{ 2024-04-24T23:29:49.943 INFO:teuthology.orchestra.run.smithi080.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T23:29:49.943 INFO:teuthology.orchestra.run.smithi080.stdout: "scrubs": { 2024-04-24T23:29:49.943 INFO:teuthology.orchestra.run.smithi080.stdout: "d1b781f0-123c-4804-87af-ff211786196d": { 2024-04-24T23:29:49.943 INFO:teuthology.orchestra.run.smithi080.stdout: "path": "/", 2024-04-24T23:29:49.943 INFO:teuthology.orchestra.run.smithi080.stdout: "tag": "d1b781f0-123c-4804-87af-ff211786196d", 2024-04-24T23:29:49.943 INFO:teuthology.orchestra.run.smithi080.stdout: "options": "recursive,force" 2024-04-24T23:29:49.943 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:29:49.943 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:29:49.943 INFO:teuthology.orchestra.run.smithi080.stdout:} 2024-04-24T23:29:49.944 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.935+0000 7f2951134640 1 -- 172.21.15.80:0/1558268024 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f29380a2b90 msgr2=0x7f29380a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:49.944 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.935+0000 7f2951134640 1 --2- 172.21.15.80:0/1558268024 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f29380a2b90 0x7f29380a5050 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x56533815d8a0 tx=0x7f29440015a0 comp rx=0 tx=0).stop 2024-04-24T23:29:49.944 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.935+0000 7f2951134640 1 -- 172.21.15.80:0/1558268024 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f294c1006a0 msgr2=0x7f294c197a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:29:49.945 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.935+0000 7f2951134640 1 --2- 172.21.15.80:0/1558268024 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f294c1006a0 0x7f294c197a90 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f293c00c970 tx=0x7f293c00ce30 comp rx=0 tx=0).stop 2024-04-24T23:29:49.945 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.935+0000 7f2951134640 1 -- 172.21.15.80:0/1558268024 shutdown_connections 2024-04-24T23:29:49.945 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.935+0000 7f2951134640 1 --2- 172.21.15.80:0/1558268024 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f29380a2b90 0x7f29380a5050 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:49.945 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.935+0000 7f2951134640 1 --2- 172.21.15.80:0/1558268024 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f294c102250 0x7f294c19f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:49.945 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.935+0000 7f2951134640 1 --2- 172.21.15.80:0/1558268024 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f294c101890 0x7f294c197fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:49.945 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.935+0000 7f2951134640 1 --2- 172.21.15.80:0/1558268024 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f294c1006a0 0x7f294c197a90 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:29:49.945 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.935+0000 7f2951134640 1 -- 172.21.15.80:0/1558268024 >> 172.21.15.80:0/1558268024 conn(0x7f294c0fbc30 msgr2=0x7f294c104b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:29:49.945 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.935+0000 7f2951134640 1 -- 172.21.15.80:0/1558268024 shutdown_connections 2024-04-24T23:29:49.945 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:29:49.935+0000 7f2951134640 1 -- 172.21.15.80:0/1558268024 wait complete. 2024-04-24T23:29:49.947 INFO:tasks.cephfs.filesystem:scrub status for tag:d1b781f0-123c-4804-87af-ff211786196d - {'path': '/', 'tag': 'd1b781f0-123c-4804-87af-ff211786196d', 'options': 'recursive,force'} 2024-04-24T23:29:50.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:29:50 smithi080 ceph-mon[29152]: pgmap v375: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 938 B/s wr, 7 op/s 2024-04-24T23:29:50.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:29:50 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/3795975652' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:29:50.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:29:50 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/3124221721' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:29:50.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:29:50 smithi134 ceph-mon[32619]: pgmap v375: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 938 B/s wr, 7 op/s 2024-04-24T23:29:50.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:29:50 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/3795975652' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:29:50.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:29:50 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/3124221721' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:29:50.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:29:50 smithi148 ceph-mon[32528]: pgmap v375: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 938 B/s wr, 7 op/s 2024-04-24T23:29:50.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:29:50 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/3795975652' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:29:50.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:29:50 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/3124221721' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:29:51.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:29:51 smithi134 ceph-mon[32619]: scrub queued for path: / 2024-04-24T23:29:51.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:29:51 smithi134 ceph-mon[32619]: scrub summary: idle+waiting paths [/] 2024-04-24T23:29:51.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:29:51 smithi134 ceph-mon[32619]: scrub summary: active paths [/] 2024-04-24T23:29:51.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:29:51 smithi134 ceph-mon[32619]: scrub summary: idle+waiting paths [/] 2024-04-24T23:29:51.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:29:51 smithi148 ceph-mon[32528]: scrub queued for path: / 2024-04-24T23:29:51.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:29:51 smithi148 ceph-mon[32528]: scrub summary: idle+waiting paths [/] 2024-04-24T23:29:51.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:29:51 smithi148 ceph-mon[32528]: scrub summary: active paths [/] 2024-04-24T23:29:51.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:29:51 smithi148 ceph-mon[32528]: scrub summary: idle+waiting paths [/] 2024-04-24T23:29:51.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:29:51 smithi080 ceph-mon[29152]: scrub queued for path: / 2024-04-24T23:29:51.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:29:51 smithi080 ceph-mon[29152]: scrub summary: idle+waiting paths [/] 2024-04-24T23:29:51.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:29:51 smithi080 ceph-mon[29152]: scrub summary: active paths [/] 2024-04-24T23:29:51.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:29:51 smithi080 ceph-mon[29152]: scrub summary: idle+waiting paths [/] 2024-04-24T23:29:52.347 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:29:52 smithi134 ceph-mon[32619]: pgmap v376: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 938 B/s wr, 6 op/s 2024-04-24T23:29:52.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:29:52 smithi148 ceph-mon[32528]: pgmap v376: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 938 B/s wr, 6 op/s 2024-04-24T23:29:52.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:29:52 smithi080 ceph-mon[29152]: pgmap v376: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 938 B/s wr, 6 op/s 2024-04-24T23:29:54.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:29:54 smithi134 ceph-mon[32619]: pgmap v377: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 1023 B/s wr, 6 op/s 2024-04-24T23:29:54.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:29:54 smithi148 ceph-mon[32528]: pgmap v377: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 1023 B/s wr, 6 op/s 2024-04-24T23:29:54.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:29:54 smithi080 ceph-mon[29152]: pgmap v377: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 1023 B/s wr, 6 op/s 2024-04-24T23:29:56.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:29:56 smithi134 ceph-mon[32619]: pgmap v378: 129 pgs: 129 active+clean; 650 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-04-24T23:29:56.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:29:56 smithi148 ceph-mon[32528]: pgmap v378: 129 pgs: 129 active+clean; 650 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-04-24T23:29:56.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:29:56 smithi080 ceph-mon[29152]: pgmap v378: 129 pgs: 129 active+clean; 650 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-04-24T23:29:57.307 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:29:57 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:29:57.308 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:29:57 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:29:57.308 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:29:57 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:29:57.366 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:29:57 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:29:57.366 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:29:57 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:29:57.366 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:29:57 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:29:57.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:29:57 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:29:57.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:29:57 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:29:57.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:29:57 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:29:58.348 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:29:58 smithi080 ceph-mon[29152]: pgmap v379: 129 pgs: 129 active+clean; 650 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-04-24T23:29:58.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:29:58 smithi134 ceph-mon[32619]: pgmap v379: 129 pgs: 129 active+clean; 650 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-04-24T23:29:58.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:29:58 smithi148 ceph-mon[32528]: pgmap v379: 129 pgs: 129 active+clean; 650 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-04-24T23:29:59.359 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:29:59 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:29:59.360 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:29:59 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:29:59.360 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:29:59 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:29:59.360 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:29:59 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:29:59.360 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:29:59 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:29:59.360 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:29:59 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:29:59.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:29:59 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:29:59.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:29:59 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:29:59.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:29:59 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:29:59.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:29:59 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:29:59.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:29:59 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:29:59.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:29:59 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:29:59.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:29:59 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:29:59.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:29:59 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:29:59.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:29:59 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:29:59.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:29:59 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:29:59.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:29:59 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:29:59.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:29:59 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:30:00.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:30:00 smithi134 ceph-mon[32619]: pgmap v380: 129 pgs: 129 active+clean; 650 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-04-24T23:30:00.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:30:00 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:30:00.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:30:00 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:30:00.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:30:00 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:30:00.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:30:00 smithi134 ceph-mon[32619]: overall HEALTH_OK 2024-04-24T23:30:00.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:30:00 smithi148 ceph-mon[32528]: pgmap v380: 129 pgs: 129 active+clean; 650 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-04-24T23:30:00.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:30:00 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:30:00.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:30:00 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:30:00.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:30:00 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:30:00.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:30:00 smithi148 ceph-mon[32528]: overall HEALTH_OK 2024-04-24T23:30:00.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:30:00 smithi080 ceph-mon[29152]: pgmap v380: 129 pgs: 129 active+clean; 650 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-04-24T23:30:00.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:30:00 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:30:00.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:30:00 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:30:00.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:30:00 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:30:00.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:30:00 smithi080 ceph-mon[29152]: overall HEALTH_OK 2024-04-24T23:30:02.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:30:02 smithi134 ceph-mon[32619]: pgmap v381: 129 pgs: 129 active+clean; 650 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 853 B/s wr, 6 op/s 2024-04-24T23:30:02.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:30:02 smithi148 ceph-mon[32528]: pgmap v381: 129 pgs: 129 active+clean; 650 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 853 B/s wr, 6 op/s 2024-04-24T23:30:02.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:30:02 smithi080 ceph-mon[29152]: pgmap v381: 129 pgs: 129 active+clean; 650 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 853 B/s wr, 6 op/s 2024-04-24T23:30:04.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:30:04 smithi134 ceph-mon[32619]: pgmap v382: 129 pgs: 129 active+clean; 650 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 853 B/s wr, 5 op/s 2024-04-24T23:30:04.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:30:04 smithi148 ceph-mon[32528]: pgmap v382: 129 pgs: 129 active+clean; 650 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 853 B/s wr, 5 op/s 2024-04-24T23:30:04.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:30:04 smithi080 ceph-mon[29152]: pgmap v382: 129 pgs: 129 active+clean; 650 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 853 B/s wr, 5 op/s 2024-04-24T23:30:05.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:30:05 smithi134 ceph-mon[32619]: scrub summary: idle 2024-04-24T23:30:05.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:30:05 smithi148 ceph-mon[32528]: scrub summary: idle 2024-04-24T23:30:05.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:30:05 smithi080 ceph-mon[29152]: scrub summary: idle 2024-04-24T23:30:06.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:30:06 smithi134 ceph-mon[32619]: pgmap v383: 129 pgs: 129 active+clean; 650 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 767 B/s wr, 8 op/s 2024-04-24T23:30:06.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:30:06 smithi148 ceph-mon[32528]: pgmap v383: 129 pgs: 129 active+clean; 650 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 767 B/s wr, 8 op/s 2024-04-24T23:30:06.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:30:06 smithi080 ceph-mon[29152]: pgmap v383: 129 pgs: 129 active+clean; 650 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 767 B/s wr, 8 op/s 2024-04-24T23:30:08.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:30:08 smithi134 ceph-mon[32619]: pgmap v384: 129 pgs: 129 active+clean; 650 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:30:08.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:30:08 smithi148 ceph-mon[32528]: pgmap v384: 129 pgs: 129 active+clean; 650 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:30:08.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:30:08 smithi080 ceph-mon[29152]: pgmap v384: 129 pgs: 129 active+clean; 650 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:30:10.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:30:10 smithi134 ceph-mon[32619]: pgmap v385: 129 pgs: 129 active+clean; 650 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T23:30:10.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:30:10 smithi148 ceph-mon[32528]: pgmap v385: 129 pgs: 129 active+clean; 650 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T23:30:10.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:30:10 smithi080 ceph-mon[29152]: pgmap v385: 129 pgs: 129 active+clean; 650 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T23:30:12.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:30:12 smithi134 ceph-mon[32619]: pgmap v386: 129 pgs: 129 active+clean; 650 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T23:30:12.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:30:12 smithi148 ceph-mon[32528]: pgmap v386: 129 pgs: 129 active+clean; 650 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T23:30:12.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:30:12 smithi080 ceph-mon[29152]: pgmap v386: 129 pgs: 129 active+clean; 650 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T23:30:14.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:30:14 smithi134 ceph-mon[32619]: pgmap v387: 129 pgs: 129 active+clean; 650 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:30:14.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:30:14 smithi148 ceph-mon[32528]: pgmap v387: 129 pgs: 129 active+clean; 650 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:30:14.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:30:14 smithi080 ceph-mon[29152]: pgmap v387: 129 pgs: 129 active+clean; 650 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:30:16.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:30:16 smithi134 ceph-mon[32619]: pgmap v388: 129 pgs: 129 active+clean; 650 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T23:30:16.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:30:16 smithi148 ceph-mon[32528]: pgmap v388: 129 pgs: 129 active+clean; 650 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T23:30:16.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:30:16 smithi080 ceph-mon[29152]: pgmap v388: 129 pgs: 129 active+clean; 650 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T23:30:18.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:30:18 smithi134 ceph-mon[32619]: pgmap v389: 129 pgs: 129 active+clean; 650 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:30:18.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:30:18 smithi148 ceph-mon[32528]: pgmap v389: 129 pgs: 129 active+clean; 650 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:30:18.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:30:18 smithi080 ceph-mon[29152]: pgmap v389: 129 pgs: 129 active+clean; 650 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:30:19.948 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T23:30:19.948 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T23:30:20.070 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.069+0000 7fb068410640 1 -- 172.21.15.80:0/2649828636 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb060102230 msgr2=0x7fb060108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:20.070 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.069+0000 7fb068410640 1 --2- 172.21.15.80:0/2649828636 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb060102230 0x7fb060108ac0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fb058009ca0 tx=0x7fb05802a8c0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.070 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.069+0000 7fb068410640 1 -- 172.21.15.80:0/2649828636 shutdown_connections 2024-04-24T23:30:20.070 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.069+0000 7fb068410640 1 --2- 172.21.15.80:0/2649828636 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb060102230 0x7fb060108ac0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.071 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.069+0000 7fb068410640 1 --2- 172.21.15.80:0/2649828636 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb060101870 0x7fb060101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.071 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.069+0000 7fb068410640 1 --2- 172.21.15.80:0/2649828636 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb060100680 0x7fb060100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.071 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.069+0000 7fb068410640 1 -- 172.21.15.80:0/2649828636 >> 172.21.15.80:0/2649828636 conn(0x7fb0600fbbf0 msgr2=0x7fb0600fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:20.071 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.069+0000 7fb068410640 1 -- 172.21.15.80:0/2649828636 shutdown_connections 2024-04-24T23:30:20.071 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.069+0000 7fb068410640 1 -- 172.21.15.80:0/2649828636 wait complete. 2024-04-24T23:30:20.073 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.070+0000 7fb068410640 1 Processor -- start 2024-04-24T23:30:20.073 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.070+0000 7fb068410640 1 -- start start 2024-04-24T23:30:20.074 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.070+0000 7fb068410640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb060100680 0x7fb060104610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:20.074 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.070+0000 7fb068410640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb060101870 0x7fb060102c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:20.074 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.070+0000 7fb068410640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb060102230 0x7fb060103190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:20.074 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.070+0000 7fb068410640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fb06010c300 con 0x7fb060101870 2024-04-24T23:30:20.074 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.070+0000 7fb068410640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb06010c180 con 0x7fb060102230 2024-04-24T23:30:20.074 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.070+0000 7fb068410640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb06010c480 con 0x7fb060100680 2024-04-24T23:30:20.074 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.070+0000 7fb0659ab640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb060101870 0x7fb060102c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:20.074 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.070+0000 7fb0659ab640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb060101870 0x7fb060102c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:46718/0 (socket says 172.21.15.80:46718) 2024-04-24T23:30:20.074 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.070+0000 7fb0659ab640 1 -- 172.21.15.80:0/2050303318 learned_addr learned my addr 172.21.15.80:0/2050303318 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:30:20.075 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.070+0000 7fb0661ac640 1 --2- 172.21.15.80:0/2050303318 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb060100680 0x7fb060104610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:20.075 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.070+0000 7fb0669ad640 1 --2- 172.21.15.80:0/2050303318 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb060102230 0x7fb060103190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:20.075 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.070+0000 7fb0659ab640 1 -- 172.21.15.80:0/2050303318 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb060100680 msgr2=0x7fb060104610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:20.075 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.070+0000 7fb0659ab640 1 --2- 172.21.15.80:0/2050303318 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb060100680 0x7fb060104610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.075 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.070+0000 7fb0659ab640 1 -- 172.21.15.80:0/2050303318 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb060102230 msgr2=0x7fb060103190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:20.075 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.070+0000 7fb0659ab640 1 --2- 172.21.15.80:0/2050303318 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb060102230 0x7fb060103190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.075 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.070+0000 7fb0659ab640 1 -- 172.21.15.80:0/2050303318 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb060103a20 con 0x7fb060101870 2024-04-24T23:30:20.075 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.070+0000 7fb0661ac640 1 --2- 172.21.15.80:0/2050303318 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb060100680 0x7fb060104610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:30:20.075 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.070+0000 7fb0659ab640 1 --2- 172.21.15.80:0/2050303318 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb060101870 0x7fb060102c50 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fb05000d770 tx=0x7fb05000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:20.075 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.071+0000 7fb0577fe640 1 -- 172.21.15.80:0/2050303318 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb050017070 con 0x7fb060101870 2024-04-24T23:30:20.075 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.071+0000 7fb068410640 1 -- 172.21.15.80:0/2050303318 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0601a1b60 con 0x7fb060101870 2024-04-24T23:30:20.075 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.071+0000 7fb0577fe640 1 -- 172.21.15.80:0/2050303318 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb050005500 con 0x7fb060101870 2024-04-24T23:30:20.075 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.071+0000 7fb068410640 1 -- 172.21.15.80:0/2050303318 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0601a2060 con 0x7fb060101870 2024-04-24T23:30:20.075 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.071+0000 7fb0577fe640 1 -- 172.21.15.80:0/2050303318 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb050020060 con 0x7fb060101870 2024-04-24T23:30:20.075 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.073+0000 7fb0577fe640 1 -- 172.21.15.80:0/2050303318 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb050004da0 con 0x7fb060101870 2024-04-24T23:30:20.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.073+0000 7fb06b1f4740 1 Processor -- start 2024-04-24T23:30:20.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.074+0000 7fb06b1f4740 1 -- start start 2024-04-24T23:30:20.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.074+0000 7fb06b1f4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f2d41f240 0x561f2d41f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:20.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.074+0000 7fb06b1f4740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561f2d41fc30 0x561f2d424730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:20.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.074+0000 7fb06b1f4740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561f2d424da0 0x561f2d427230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:20.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.074+0000 7fb06b1f4740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x561f2d3836b0 con 0x561f2d424da0 2024-04-24T23:30:20.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.074+0000 7fb06b1f4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x561f2d32b880 con 0x561f2d41f240 2024-04-24T23:30:20.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.074+0000 7fb06b1f4740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x561f2d37f670 con 0x561f2d41fc30 2024-04-24T23:30:20.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.074+0000 7fb0577fe640 1 --2- 172.21.15.80:0/2050303318 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fb04c0a2b90 0x7fb04c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:20.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.074+0000 7fb0659ab640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561f2d424da0 0x561f2d427230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:20.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.074+0000 7fb0659ab640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561f2d424da0 0x561f2d427230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:46730/0 (socket says 172.21.15.80:46730) 2024-04-24T23:30:20.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.074+0000 7fb0669ad640 1 --2- 172.21.15.80:0/2050303318 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fb04c0a2b90 0x7fb04c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:20.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.074+0000 7fb0659ab640 1 -- 172.21.15.80:0/3418040782 learned_addr learned my addr 172.21.15.80:0/3418040782 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:30:20.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.074+0000 7fb0661ac640 1 --2- 172.21.15.80:0/3418040782 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561f2d41fc30 0x561f2d424730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:20.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.074+0000 7fb0669ad640 1 --2- 172.21.15.80:0/3418040782 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f2d41f240 0x561f2d41f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:20.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.074+0000 7fb0577fe640 1 -- 172.21.15.80:0/2050303318 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fb0500cf290 con 0x7fb060101870 2024-04-24T23:30:20.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.074+0000 7fb0659ab640 1 -- 172.21.15.80:0/3418040782 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561f2d41fc30 msgr2=0x561f2d424730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:20.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.074+0000 7fb0659ab640 1 --2- 172.21.15.80:0/3418040782 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561f2d41fc30 0x561f2d424730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.074+0000 7fb0659ab640 1 -- 172.21.15.80:0/3418040782 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f2d41f240 msgr2=0x561f2d41f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:20.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.074+0000 7fb0659ab640 1 --2- 172.21.15.80:0/3418040782 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f2d41f240 0x561f2d41f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.074+0000 7fb0659ab640 1 -- 172.21.15.80:0/3418040782 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f2d1617d0 con 0x561f2d424da0 2024-04-24T23:30:20.084 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.074+0000 7fb0661ac640 1 --2- 172.21.15.80:0/3418040782 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561f2d41fc30 0x561f2d424730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:30:20.084 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.075+0000 7fb0659ab640 1 --2- 172.21.15.80:0/3418040782 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561f2d424da0 0x561f2d427230 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fb0500980f0 tx=0x7fb0500048a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc8296a950db180d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:20.084 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.075+0000 7fb0669ad640 1 --2- 172.21.15.80:0/2050303318 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fb04c0a2b90 0x7fb04c0a5050 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fb058006010 tx=0x7fb0580099b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:20.084 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.075+0000 7fb054bfe640 1 -- 172.21.15.80:0/3418040782 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb05009b720 con 0x561f2d424da0 2024-04-24T23:30:20.084 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.075+0000 7fb054bfe640 1 -- 172.21.15.80:0/3418040782 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb05009fad0 con 0x561f2d424da0 2024-04-24T23:30:20.084 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.075+0000 7fb054bfe640 1 -- 172.21.15.80:0/3418040782 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb050097d90 con 0x561f2d424da0 2024-04-24T23:30:20.084 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.076+0000 7fb06b1f4740 1 -- 172.21.15.80:0/3418040782 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561f2d424da0 msgr2=0x561f2d427230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:20.084 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.076+0000 7fb06b1f4740 1 --2- 172.21.15.80:0/3418040782 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561f2d424da0 0x561f2d427230 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fb0500980f0 tx=0x7fb0500048a0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.084 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.076+0000 7fb06b1f4740 1 -- 172.21.15.80:0/3418040782 shutdown_connections 2024-04-24T23:30:20.084 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.076+0000 7fb06b1f4740 1 --2- 172.21.15.80:0/3418040782 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561f2d424da0 0x561f2d427230 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.084 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.076+0000 7fb06b1f4740 1 --2- 172.21.15.80:0/3418040782 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561f2d41fc30 0x561f2d424730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.084 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.076+0000 7fb06b1f4740 1 --2- 172.21.15.80:0/3418040782 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f2d41f240 0x561f2d41f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.084 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.076+0000 7fb06b1f4740 1 -- 172.21.15.80:0/3418040782 >> 172.21.15.80:0/3418040782 conn(0x561f2d41a8b0 msgr2=0x561f2d41acb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:20.084 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.076+0000 7fb06b1f4740 1 -- 172.21.15.80:0/3418040782 shutdown_connections 2024-04-24T23:30:20.084 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.076+0000 7fb06b1f4740 1 -- 172.21.15.80:0/3418040782 wait complete. 2024-04-24T23:30:20.085 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.076+0000 7fb06b1f4740 1 -- start start 2024-04-24T23:30:20.085 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.076+0000 7fb06b1f4740 1 Processor -- start 2024-04-24T23:30:20.085 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.076+0000 7fb06b1f4740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561f2d41f240 0x561f2d429570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:20.085 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.076+0000 7fb06b1f4740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561f2d41fc30 0x561f2d429ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:20.085 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.076+0000 7fb06b1f4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f2d424da0 0x561f2d430af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:20.085 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.076+0000 7fb06b1f4740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x561f2d427bd0 con 0x561f2d41fc30 2024-04-24T23:30:20.085 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.076+0000 7fb06b1f4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x561f2d33ef50 con 0x561f2d424da0 2024-04-24T23:30:20.085 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.076+0000 7fb06b1f4740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x561f2d427d50 con 0x561f2d41f240 2024-04-24T23:30:20.085 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.076+0000 7fb06b1f4740 10 client.0 _ll_register_callbacks cb 0x561f2d37a000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:30:20.085 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.076+0000 7fb0659ab640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561f2d41fc30 0x561f2d429ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:20.085 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.077+0000 7fb0659ab640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561f2d41fc30 0x561f2d429ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:46734/0 (socket says 172.21.15.80:46734) 2024-04-24T23:30:20.085 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.077+0000 7fb0659ab640 1 -- 172.21.15.80:0/2273970185 learned_addr learned my addr 172.21.15.80:0/2273970185 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:30:20.085 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.077+0000 7fb0669ad640 1 --2- 172.21.15.80:0/2273970185 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f2d424da0 0x561f2d430af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:20.085 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.077+0000 7fb0661ac640 1 --2- 172.21.15.80:0/2273970185 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561f2d41f240 0x561f2d429570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:20.086 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.077+0000 7fb0659ab640 1 -- 172.21.15.80:0/2273970185 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561f2d41f240 msgr2=0x561f2d429570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:20.086 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.077+0000 7fb0659ab640 1 --2- 172.21.15.80:0/2273970185 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561f2d41f240 0x561f2d429570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.086 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.077+0000 7fb0659ab640 1 -- 172.21.15.80:0/2273970185 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f2d424da0 msgr2=0x561f2d430af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:20.086 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.077+0000 7fb0659ab640 1 --2- 172.21.15.80:0/2273970185 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f2d424da0 0x561f2d430af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.086 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.077+0000 7fb0659ab640 1 -- 172.21.15.80:0/2273970185 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f2d0973f0 con 0x561f2d41fc30 2024-04-24T23:30:20.086 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.077+0000 7fb0661ac640 1 --2- 172.21.15.80:0/2273970185 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561f2d41f240 0x561f2d429570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:30:20.086 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.077+0000 7fb0659ab640 1 --2- 172.21.15.80:0/2273970185 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561f2d41fc30 0x561f2d429ab0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fb050009c70 tx=0x7fb050099be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5ad1fd0fb55dc8c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:20.086 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.077+0000 7fb049ffb640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-04-24T23:30:20.086 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.077+0000 7fb049ffb640 1 -- 172.21.15.80:0/2273970185 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb05009b720 con 0x561f2d41fc30 2024-04-24T23:30:20.086 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.077+0000 7fb049ffb640 1 -- 172.21.15.80:0/2273970185 <== mon.0 v2:172.21.15.80:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fb0500a3f00 con 0x561f2d41fc30 2024-04-24T23:30:20.086 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.077+0000 7fb06b1f4740 1 -- 172.21.15.80:0/2273970185 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb05009fad0 con 0x561f2d41fc30 2024-04-24T23:30:20.086 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.077+0000 7fb049ffb640 1 -- 172.21.15.80:0/2273970185 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb0500a4100 con 0x561f2d41fc30 2024-04-24T23:30:20.086 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.077+0000 7fb06b1f4740 10 client.15654 fetch_fsmap learned FSMap version 25 2024-04-24T23:30:20.086 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.077+0000 7fb06b1f4740 1 -- 172.21.15.80:0/2273970185 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561f2d0570f0 con 0x561f2d41fc30 2024-04-24T23:30:20.087 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.077+0000 7fb049ffb640 1 -- 172.21.15.80:0/2273970185 <== mon.0 v2:172.21.15.80:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb05009fad0 con 0x561f2d41fc30 2024-04-24T23:30:20.087 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.078+0000 7fb049ffb640 1 -- 172.21.15.80:0/2273970185 <== mon.0 v2:172.21.15.80:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fb05009ab70 con 0x561f2d41fc30 2024-04-24T23:30:20.087 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.078+0000 7fb06b1f4740 10 client.15654 fetch_fsmap finished waiting for FSMap version 25 2024-04-24T23:30:20.087 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.078+0000 7fb06b1f4740 10 client.15654 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:30:20.087 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.078+0000 7fb06b1f4740 20 client.15654 populate_metadata read hostname 'smithi080' 2024-04-24T23:30:20.087 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.078+0000 7fb06b1f4740 1 --2- 172.21.15.80:0/2273970185 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x561f2d435050 0x561f2d437440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:20.087 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.078+0000 7fb06b1f4740 4 client.15654 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:30:20.087 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.078+0000 7fb06b1f4740 1 -- 172.21.15.80:0/2273970185 --> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561f2d390bf0 con 0x561f2d435050 2024-04-24T23:30:20.087 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.078+0000 7fb0661ac640 1 --2- 172.21.15.80:0/2273970185 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x561f2d435050 0x561f2d437440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:20.087 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.079+0000 7fb0661ac640 1 --2- 172.21.15.80:0/2273970185 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x561f2d435050 0x561f2d437440 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c57c753d45fb75b1 server_cookie=2ced22d2d55ebb60 in_seq=0 out_seq=0 2024-04-24T23:30:20.087 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.079+0000 7fb049ffb640 10 client.15654 ms_handle_connect on v2:172.21.15.148:6838/2952757594 2024-04-24T23:30:20.087 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.079+0000 7fb049ffb640 1 -- 172.21.15.80:0/2273970185 <== mds.0 v2:172.21.15.148:6838/2952757594 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561f2d390bf0 con 0x561f2d435050 2024-04-24T23:30:20.087 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.079+0000 7fb049ffb640 10 client.15654 handle_command_reply: tid=0 2024-04-24T23:30:20.087 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.079+0000 7fb06b1f4740 1 client.15654 shutdown 2024-04-24T23:30:20.087 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.079+0000 7fb06b1f4740 1 -- 172.21.15.80:0/2273970185 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561f2d41fc30 msgr2=0x561f2d429ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:20.088 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.079+0000 7fb06b1f4740 1 --2- 172.21.15.80:0/2273970185 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561f2d41fc30 0x561f2d429ab0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fb050009c70 tx=0x7fb050099be0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.088 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.079+0000 7fb06b1f4740 1 -- 172.21.15.80:0/2273970185 shutdown_connections 2024-04-24T23:30:20.088 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.079+0000 7fb06b1f4740 1 --2- 172.21.15.80:0/2273970185 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x561f2d435050 0x561f2d437440 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.088 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.079+0000 7fb06b1f4740 1 --2- 172.21.15.80:0/2273970185 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f2d424da0 0x561f2d430af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.088 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.079+0000 7fb06b1f4740 1 --2- 172.21.15.80:0/2273970185 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561f2d41fc30 0x561f2d429ab0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.088 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.079+0000 7fb049ffb640 0 client.15654 ms_handle_reset on v2:172.21.15.148:6838/2952757594 2024-04-24T23:30:20.088 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.079+0000 7fb06b1f4740 1 --2- 172.21.15.80:0/2273970185 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561f2d41f240 0x561f2d429570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.088 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.079+0000 7fb06b1f4740 1 -- 172.21.15.80:0/2273970185 >> 172.21.15.80:0/2273970185 conn(0x561f2d41a8b0 msgr2=0x561f2d419af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:20.088 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.079+0000 7fb06b1f4740 1 -- 172.21.15.80:0/2273970185 shutdown_connections 2024-04-24T23:30:20.088 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.080+0000 7fb06b1f4740 1 -- 172.21.15.80:0/2273970185 wait complete. 2024-04-24T23:30:20.088 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.080+0000 7fb06b1f4740 20 client.15654 trim_cache size 0 max 16384 2024-04-24T23:30:20.103 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.097+0000 7fb06b1f4740 1 Processor -- start 2024-04-24T23:30:20.103 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.097+0000 7fb06b1f4740 1 -- start start 2024-04-24T23:30:20.103 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.097+0000 7fb06b1f4740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561f2d41f240 0x561f2d41d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:20.103 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.097+0000 7fb06b1f4740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561f2d41fc30 0x561f2d430ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:20.103 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.097+0000 7fb06b1f4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f2d424da0 0x561f2d41db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:20.104 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.097+0000 7fb06b1f4740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fb0601aee40 con 0x561f2d41fc30 2024-04-24T23:30:20.104 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.097+0000 7fb06b1f4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb0601ac710 con 0x561f2d424da0 2024-04-24T23:30:20.104 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.097+0000 7fb06b1f4740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb0601aa560 con 0x561f2d41f240 2024-04-24T23:30:20.104 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.097+0000 7fb0659ab640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561f2d41fc30 0x561f2d430ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:20.104 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.097+0000 7fb0659ab640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561f2d41fc30 0x561f2d430ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:46736/0 (socket says 172.21.15.80:46736) 2024-04-24T23:30:20.104 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.097+0000 7fb0659ab640 1 -- 172.21.15.80:0/740815059 learned_addr learned my addr 172.21.15.80:0/740815059 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:30:20.104 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.098+0000 7fb0661ac640 1 --2- 172.21.15.80:0/740815059 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561f2d41f240 0x561f2d41d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:20.104 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.098+0000 7fb0659ab640 1 -- 172.21.15.80:0/740815059 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561f2d41f240 msgr2=0x561f2d41d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:20.104 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.098+0000 7fb0659ab640 1 --2- 172.21.15.80:0/740815059 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561f2d41f240 0x561f2d41d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.104 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.098+0000 7fb0669ad640 1 --2- 172.21.15.80:0/740815059 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f2d424da0 0x561f2d41db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:20.104 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.098+0000 7fb0659ab640 1 -- 172.21.15.80:0/740815059 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f2d424da0 msgr2=0x561f2d41db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:20.104 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.098+0000 7fb0659ab640 1 --2- 172.21.15.80:0/740815059 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f2d424da0 0x561f2d41db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.104 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.098+0000 7fb0659ab640 1 -- 172.21.15.80:0/740815059 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f2d4377b0 con 0x561f2d41fc30 2024-04-24T23:30:20.104 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.098+0000 7fb0659ab640 1 --2- 172.21.15.80:0/740815059 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561f2d41fc30 0x561f2d430ab0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fb05009b420 tx=0x7fb050009ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cba0fea3bedaa71f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:20.105 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.098+0000 7fb04bfff640 1 -- 172.21.15.80:0/740815059 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb05007f070 con 0x561f2d41fc30 2024-04-24T23:30:20.105 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.098+0000 7fb04bfff640 1 -- 172.21.15.80:0/740815059 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0500b4070 con 0x561f2d41fc30 2024-04-24T23:30:20.105 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.098+0000 7fb04bfff640 1 -- 172.21.15.80:0/740815059 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb05009a950 con 0x561f2d41fc30 2024-04-24T23:30:20.105 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.098+0000 7fb06b1f4740 1 -- 172.21.15.80:0/740815059 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561f2d41fc30 msgr2=0x561f2d430ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:20.105 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.098+0000 7fb06b1f4740 1 --2- 172.21.15.80:0/740815059 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561f2d41fc30 0x561f2d430ab0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fb05009b420 tx=0x7fb050009ed0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.105 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.099+0000 7fb06b1f4740 1 -- 172.21.15.80:0/740815059 shutdown_connections 2024-04-24T23:30:20.105 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.099+0000 7fb06b1f4740 1 --2- 172.21.15.80:0/740815059 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f2d424da0 0x561f2d41db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.105 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.099+0000 7fb06b1f4740 1 --2- 172.21.15.80:0/740815059 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561f2d41fc30 0x561f2d430ab0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.105 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.099+0000 7fb06b1f4740 1 --2- 172.21.15.80:0/740815059 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561f2d41f240 0x561f2d41d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.105 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.099+0000 7fb06b1f4740 1 -- 172.21.15.80:0/740815059 >> 172.21.15.80:0/740815059 conn(0x561f2d41a8b0 msgr2=0x561f2d421090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:20.105 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.099+0000 7fb06b1f4740 1 -- 172.21.15.80:0/740815059 shutdown_connections 2024-04-24T23:30:20.105 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.099+0000 7fb06b1f4740 1 -- 172.21.15.80:0/740815059 wait complete. 2024-04-24T23:30:20.105 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.099+0000 7fb06b1f4740 1 -- start start 2024-04-24T23:30:20.105 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.099+0000 7fb06b1f4740 1 Processor -- start 2024-04-24T23:30:20.105 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.099+0000 7fb06b1f4740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561f2d41f240 0x561f2d42dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:20.105 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.099+0000 7fb06b1f4740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561f2d41fc30 0x561f2d42e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:20.106 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.099+0000 7fb06b1f4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f2d424da0 0x561f2d42f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:20.106 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.099+0000 7fb06b1f4740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fb0601aabd0 con 0x561f2d41fc30 2024-04-24T23:30:20.106 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.099+0000 7fb06b1f4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb0601ac040 con 0x561f2d424da0 2024-04-24T23:30:20.106 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.099+0000 7fb06b1f4740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb0601ab2a0 con 0x561f2d41f240 2024-04-24T23:30:20.106 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.099+0000 7fb06b1f4740 10 client.0 _ll_register_callbacks cb 0x7fb05000df50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:30:20.106 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.099+0000 7fb0659ab640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561f2d41fc30 0x561f2d42e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:20.106 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.099+0000 7fb0659ab640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561f2d41fc30 0x561f2d42e2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:46750/0 (socket says 172.21.15.80:46750) 2024-04-24T23:30:20.106 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.099+0000 7fb0659ab640 1 -- 172.21.15.80:0/2262138437 learned_addr learned my addr 172.21.15.80:0/2262138437 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:30:20.106 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.099+0000 7fb0669ad640 1 --2- 172.21.15.80:0/2262138437 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f2d424da0 0x561f2d42f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:20.106 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.099+0000 7fb0661ac640 1 --2- 172.21.15.80:0/2262138437 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561f2d41f240 0x561f2d42dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:20.106 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.100+0000 7fb0659ab640 1 -- 172.21.15.80:0/2262138437 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561f2d41f240 msgr2=0x561f2d42dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:20.106 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.100+0000 7fb0659ab640 1 --2- 172.21.15.80:0/2262138437 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561f2d41f240 0x561f2d42dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.106 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.100+0000 7fb0659ab640 1 -- 172.21.15.80:0/2262138437 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f2d424da0 msgr2=0x561f2d42f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:20.107 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.100+0000 7fb0659ab640 1 --2- 172.21.15.80:0/2262138437 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f2d424da0 0x561f2d42f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.107 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.100+0000 7fb0659ab640 1 -- 172.21.15.80:0/2262138437 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f2d42f8a0 con 0x561f2d41fc30 2024-04-24T23:30:20.107 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.100+0000 7fb0661ac640 1 --2- 172.21.15.80:0/2262138437 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561f2d41f240 0x561f2d42dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:30:20.107 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.100+0000 7fb0659ab640 1 --2- 172.21.15.80:0/2262138437 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561f2d41fc30 0x561f2d42e2a0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fb0500862e0 tx=0x7fb0500cf170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c12b8feb21403190 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:20.107 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.100+0000 7fb049ffb640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-04-24T23:30:20.107 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.100+0000 7fb049ffb640 1 -- 172.21.15.80:0/2262138437 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb05007f020 con 0x561f2d41fc30 2024-04-24T23:30:20.107 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.100+0000 7fb049ffb640 1 -- 172.21.15.80:0/2262138437 <== mon.0 v2:172.21.15.80:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fb05009ab70 con 0x561f2d41fc30 2024-04-24T23:30:20.107 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.100+0000 7fb06b1f4740 1 -- 172.21.15.80:0/2262138437 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0500b4070 con 0x561f2d41fc30 2024-04-24T23:30:20.107 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.100+0000 7fb049ffb640 1 -- 172.21.15.80:0/2262138437 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb0500979c0 con 0x561f2d41fc30 2024-04-24T23:30:20.107 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.100+0000 7fb06b1f4740 10 client.15660 fetch_fsmap learned FSMap version 25 2024-04-24T23:30:20.107 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.100+0000 7fb06b1f4740 1 -- 172.21.15.80:0/2262138437 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561f2d42f8a0 con 0x561f2d41fc30 2024-04-24T23:30:20.107 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.100+0000 7fb049ffb640 1 -- 172.21.15.80:0/2262138437 <== mon.0 v2:172.21.15.80:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0500b4070 con 0x561f2d41fc30 2024-04-24T23:30:20.107 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.100+0000 7fb049ffb640 1 -- 172.21.15.80:0/2262138437 <== mon.0 v2:172.21.15.80:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fb0500051b0 con 0x561f2d41fc30 2024-04-24T23:30:20.108 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.100+0000 7fb06b1f4740 10 client.15660 fetch_fsmap finished waiting for FSMap version 25 2024-04-24T23:30:20.108 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.100+0000 7fb06b1f4740 10 client.15660 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:30:20.108 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.100+0000 7fb06b1f4740 20 client.15660 populate_metadata read hostname 'smithi080' 2024-04-24T23:30:20.108 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.100+0000 7fb06b1f4740 1 --2- 172.21.15.80:0/2262138437 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x561f2d49b670 0x561f2d49da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:20.108 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.100+0000 7fb06b1f4740 4 client.15660 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:30:20.108 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.100+0000 7fb06b1f4740 1 -- 172.21.15.80:0/2262138437 --> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561f2d429360 con 0x561f2d49b670 2024-04-24T23:30:20.108 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.101+0000 7fb0661ac640 1 --2- 172.21.15.80:0/2262138437 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x561f2d49b670 0x561f2d49da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:20.108 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.101+0000 7fb0661ac640 1 --2- 172.21.15.80:0/2262138437 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x561f2d49b670 0x561f2d49da60 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=acae777d31963f73 server_cookie=efe7477eb2742e48 in_seq=0 out_seq=0 2024-04-24T23:30:20.108 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.101+0000 7fb049ffb640 10 client.15660 ms_handle_connect on v2:172.21.15.148:6838/2952757594 2024-04-24T23:30:20.108 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.101+0000 7fb049ffb640 1 -- 172.21.15.80:0/2262138437 <== mds.0 v2:172.21.15.148:6838/2952757594 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x561f2d429360 con 0x561f2d49b670 2024-04-24T23:30:20.108 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.101+0000 7fb049ffb640 10 client.15660 handle_command_reply: tid=0 2024-04-24T23:30:20.108 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.101+0000 7fb06b1f4740 1 client.15660 shutdown 2024-04-24T23:30:20.108 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.101+0000 7fb06b1f4740 1 -- 172.21.15.80:0/2262138437 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561f2d41fc30 msgr2=0x561f2d42e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:20.108 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.101+0000 7fb06b1f4740 1 --2- 172.21.15.80:0/2262138437 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561f2d41fc30 0x561f2d42e2a0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fb0500862e0 tx=0x7fb0500cf170 comp rx=0 tx=0).stop 2024-04-24T23:30:20.108 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.101+0000 7fb06b1f4740 1 -- 172.21.15.80:0/2262138437 shutdown_connections 2024-04-24T23:30:20.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.101+0000 7fb06b1f4740 1 --2- 172.21.15.80:0/2262138437 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x561f2d49b670 0x561f2d49da60 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.101+0000 7fb06b1f4740 1 --2- 172.21.15.80:0/2262138437 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f2d424da0 0x561f2d42f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.101+0000 7fb06b1f4740 1 --2- 172.21.15.80:0/2262138437 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561f2d41fc30 0x561f2d42e2a0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.101+0000 7fb06b1f4740 1 --2- 172.21.15.80:0/2262138437 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561f2d41f240 0x561f2d42dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.101+0000 7fb06b1f4740 1 -- 172.21.15.80:0/2262138437 >> 172.21.15.80:0/2262138437 conn(0x561f2d41a8b0 msgr2=0x561f2d419260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:20.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.101+0000 7fb049ffb640 0 client.15660 ms_handle_reset on v2:172.21.15.148:6838/2952757594 2024-04-24T23:30:20.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.101+0000 7fb06b1f4740 1 -- 172.21.15.80:0/2262138437 shutdown_connections 2024-04-24T23:30:20.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.102+0000 7fb06b1f4740 1 -- 172.21.15.80:0/2262138437 wait complete. 2024-04-24T23:30:20.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.102+0000 7fb06b1f4740 20 client.15660 trim_cache size 0 max 16384 2024-04-24T23:30:20.109 INFO:teuthology.orchestra.run.smithi080.stdout:{ 2024-04-24T23:30:20.110 INFO:teuthology.orchestra.run.smithi080.stdout: "status": "no active scrubs running", 2024-04-24T23:30:20.110 INFO:teuthology.orchestra.run.smithi080.stdout: "scrubs": {} 2024-04-24T23:30:20.110 INFO:teuthology.orchestra.run.smithi080.stdout:} 2024-04-24T23:30:20.110 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.102+0000 7fb068410640 1 -- 172.21.15.80:0/2050303318 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fb04c0a2b90 msgr2=0x7fb04c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:20.111 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.102+0000 7fb068410640 1 --2- 172.21.15.80:0/2050303318 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fb04c0a2b90 0x7fb04c0a5050 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fb058006010 tx=0x7fb0580099b0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.111 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.102+0000 7fb068410640 1 -- 172.21.15.80:0/2050303318 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb060101870 msgr2=0x7fb060102c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:20.111 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.102+0000 7fb068410640 1 --2- 172.21.15.80:0/2050303318 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb060101870 0x7fb060102c50 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fb05000d770 tx=0x7fb05000dc30 comp rx=0 tx=0).stop 2024-04-24T23:30:20.111 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.102+0000 7fb068410640 1 -- 172.21.15.80:0/2050303318 shutdown_connections 2024-04-24T23:30:20.111 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.102+0000 7fb068410640 1 --2- 172.21.15.80:0/2050303318 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fb04c0a2b90 0x7fb04c0a5050 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.111 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.102+0000 7fb068410640 1 --2- 172.21.15.80:0/2050303318 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb060102230 0x7fb060103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.111 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.102+0000 7fb068410640 1 --2- 172.21.15.80:0/2050303318 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb060101870 0x7fb060102c50 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.111 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.102+0000 7fb068410640 1 --2- 172.21.15.80:0/2050303318 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb060100680 0x7fb060104610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.111 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.102+0000 7fb068410640 1 -- 172.21.15.80:0/2050303318 >> 172.21.15.80:0/2050303318 conn(0x7fb0600fbbf0 msgr2=0x7fb0600fd620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:20.111 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.102+0000 7fb068410640 1 -- 172.21.15.80:0/2050303318 shutdown_connections 2024-04-24T23:30:20.111 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.102+0000 7fb068410640 1 -- 172.21.15.80:0/2050303318 wait complete. 2024-04-24T23:30:20.114 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T23:30:20.114 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:30:20.216 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.214+0000 7f4afacca640 1 -- 172.21.15.80:0/17048302 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4af4102200 msgr2=0x7f4af41045f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:20.216 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.214+0000 7f4afacca640 1 --2- 172.21.15.80:0/17048302 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4af4102200 0x7f4af41045f0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f4ae8009900 tx=0x7f4ae802a410 comp rx=0 tx=0).stop 2024-04-24T23:30:20.216 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.214+0000 7f4afacca640 1 -- 172.21.15.80:0/17048302 shutdown_connections 2024-04-24T23:30:20.216 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.214+0000 7f4afacca640 1 --2- 172.21.15.80:0/17048302 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4af4107460 0x7f4af41098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.217 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.214+0000 7f4afacca640 1 --2- 172.21.15.80:0/17048302 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4af4104b30 0x7f4af4106f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.217 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.214+0000 7f4afacca640 1 --2- 172.21.15.80:0/17048302 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4af4102200 0x7f4af41045f0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.217 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.214+0000 7f4afacca640 1 -- 172.21.15.80:0/17048302 >> 172.21.15.80:0/17048302 conn(0x7f4af40fbbf0 msgr2=0x7f4af40fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:20.217 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.214+0000 7f4afacca640 1 -- 172.21.15.80:0/17048302 shutdown_connections 2024-04-24T23:30:20.217 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.214+0000 7f4afacca640 1 -- 172.21.15.80:0/17048302 wait complete. 2024-04-24T23:30:20.217 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.214+0000 7f4afacca640 1 Processor -- start 2024-04-24T23:30:20.217 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.214+0000 7f4afacca640 1 -- start start 2024-04-24T23:30:20.217 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.214+0000 7f4afacca640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4af4102200 0x7f4af4197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:20.217 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.214+0000 7f4afacca640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4af4104b30 0x7f4af4198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:20.217 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.214+0000 7f4afacca640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4af4107460 0x7f4af419f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:20.217 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.214+0000 7f4afacca640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f4af410d400 con 0x7f4af4104b30 2024-04-24T23:30:20.217 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.214+0000 7f4afacca640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4af410d280 con 0x7f4af4102200 2024-04-24T23:30:20.217 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.214+0000 7f4afacca640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4af410d580 con 0x7f4af4107460 2024-04-24T23:30:20.218 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.214+0000 7f4af3fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4af4104b30 0x7f4af4198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:20.218 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.214+0000 7f4af3fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4af4104b30 0x7f4af4198010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:46774/0 (socket says 172.21.15.80:46774) 2024-04-24T23:30:20.218 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.214+0000 7f4af3fff640 1 -- 172.21.15.80:0/460372372 learned_addr learned my addr 172.21.15.80:0/460372372 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:30:20.218 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.214+0000 7f4af8a66640 1 --2- 172.21.15.80:0/460372372 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4af4102200 0x7f4af4197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:20.218 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.215+0000 7f4af9267640 1 --2- 172.21.15.80:0/460372372 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4af4107460 0x7f4af419f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:20.218 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.215+0000 7f4af3fff640 1 -- 172.21.15.80:0/460372372 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4af4107460 msgr2=0x7f4af419f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:20.218 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.215+0000 7f4af3fff640 1 --2- 172.21.15.80:0/460372372 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4af4107460 0x7f4af419f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.218 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.215+0000 7f4af3fff640 1 -- 172.21.15.80:0/460372372 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4af4102200 msgr2=0x7f4af4197ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:20.218 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.215+0000 7f4af3fff640 1 --2- 172.21.15.80:0/460372372 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4af4102200 0x7f4af4197ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.218 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.215+0000 7f4af3fff640 1 -- 172.21.15.80:0/460372372 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4af419f6f0 con 0x7f4af4104b30 2024-04-24T23:30:20.218 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.215+0000 7f4af3fff640 1 --2- 172.21.15.80:0/460372372 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4af4104b30 0x7f4af4198010 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f4ae000d770 tx=0x7f4ae000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:20.218 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.215+0000 7f4af1ffb640 1 -- 172.21.15.80:0/460372372 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ae0017070 con 0x7f4af4104b30 2024-04-24T23:30:20.218 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.215+0000 7f4afacca640 1 -- 172.21.15.80:0/460372372 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4af419f9e0 con 0x7f4af4104b30 2024-04-24T23:30:20.218 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.215+0000 7f4af1ffb640 1 -- 172.21.15.80:0/460372372 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ae0009c30 con 0x7f4af4104b30 2024-04-24T23:30:20.218 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.215+0000 7f4af1ffb640 1 -- 172.21.15.80:0/460372372 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ae000f620 con 0x7f4af4104b30 2024-04-24T23:30:20.219 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.215+0000 7f4afacca640 1 -- 172.21.15.80:0/460372372 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4af419ff30 con 0x7f4af4104b30 2024-04-24T23:30:20.220 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.217+0000 7f4afacca640 1 -- 172.21.15.80:0/460372372 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4af40fd6e0 con 0x7f4af4104b30 2024-04-24T23:30:20.220 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.218+0000 7f4af1ffb640 1 -- 172.21.15.80:0/460372372 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4ae001a070 con 0x7f4af4104b30 2024-04-24T23:30:20.220 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.218+0000 7f4af1ffb640 1 --2- 172.21.15.80:0/460372372 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f4ae40a2b90 0x7f4ae40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:20.220 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.218+0000 7f4af8a66640 1 --2- 172.21.15.80:0/460372372 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f4ae40a2b90 0x7f4ae40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:20.220 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.218+0000 7f4af1ffb640 1 -- 172.21.15.80:0/460372372 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f4ae00cee70 con 0x7f4af4104b30 2024-04-24T23:30:20.220 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.219+0000 7f4af8a66640 1 --2- 172.21.15.80:0/460372372 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f4ae40a2b90 0x7f4ae40a5050 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f4ae80314e0 tx=0x7f4ae8000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:20.223 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.223+0000 7f4af1ffb640 1 -- 172.21.15.80:0/460372372 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f4ae0009de0 con 0x7f4af4104b30 2024-04-24T23:30:20.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:30:20 smithi080 ceph-mon[29152]: pgmap v390: 129 pgs: 129 active+clean; 650 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T23:30:20.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:30:20 smithi134 ceph-mon[32619]: pgmap v390: 129 pgs: 129 active+clean; 650 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T23:30:20.451 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.450+0000 7f4afacca640 1 -- 172.21.15.80:0/460372372 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4af4062340 con 0x7f4af4104b30 2024-04-24T23:30:20.452 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.451+0000 7f4af1ffb640 1 -- 172.21.15.80:0/460372372 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11872 (secure 0 0 0) 0x7f4ae009b3b0 con 0x7f4af4104b30 2024-04-24T23:30:20.452 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-24T23:30:20.452 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/891591085","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":891591085},{"type":"v1","addr":"172.21.15.80:6835","nonce":891591085}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6837/259529026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6836","nonce":259529026},{"type":"v1","addr":"172.21.15.80:6837","nonce":259529026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14646,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6839/4194284854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6838","nonce":4194284854},{"type":"v1","addr":"172.21.15.80:6839","nonce":4194284854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14658,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6841/3145730958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6840","nonce":3145730958},{"type":"v1","addr":"172.21.15.80:6841","nonce":3145730958}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24395,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/3736784385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3736784385},{"type":"v1","addr":"172.21.15.134:6833","nonce":3736784385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/124777462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":124777462},{"type":"v1","addr":"172.21.15.148:6833","nonce":124777462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T23:27:19.300566+0000","modified":"2024-04-24T23:29:42.158567+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24496,"mds_1":24488,"mds_2":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/1399801120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1399801120},{"type":"v1","addr":"172.21.15.134:6835","nonce":1399801120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/430512704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":430512704},{"type":"v1","addr":"172.21.15.134:6837","nonce":430512704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/3002319310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3002319310},{"type":"v1","addr":"172.21.15.148:6835","nonce":3002319310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.148:6837/856528403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":856528403},{"type":"v1","addr":"172.21.15.148:6837","nonce":856528403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6839/1743815021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":1743815021},{"type":"v1","addr":"172.21.15.134:6839","nonce":1743815021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.148:6839/2952757594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":2952757594},{"type":"v1","addr":"172.21.15.148:6839","nonce":2952757594}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24496,"qdb_cluster":[24496,24472,24488]},"id":1}]} 2024-04-24T23:30:20.452 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 25 2024-04-24T23:30:20.454 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.453+0000 7f4afacca640 1 -- 172.21.15.80:0/460372372 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f4ae40a2b90 msgr2=0x7f4ae40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:20.454 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.453+0000 7f4afacca640 1 --2- 172.21.15.80:0/460372372 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f4ae40a2b90 0x7f4ae40a5050 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f4ae80314e0 tx=0x7f4ae8000bc0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.454 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.453+0000 7f4afacca640 1 -- 172.21.15.80:0/460372372 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4af4104b30 msgr2=0x7f4af4198010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:20.454 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.453+0000 7f4afacca640 1 --2- 172.21.15.80:0/460372372 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4af4104b30 0x7f4af4198010 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f4ae000d770 tx=0x7f4ae000dc30 comp rx=0 tx=0).stop 2024-04-24T23:30:20.454 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.453+0000 7f4afacca640 1 -- 172.21.15.80:0/460372372 shutdown_connections 2024-04-24T23:30:20.454 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.453+0000 7f4afacca640 1 --2- 172.21.15.80:0/460372372 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f4ae40a2b90 0x7f4ae40a5050 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.455 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.453+0000 7f4afacca640 1 --2- 172.21.15.80:0/460372372 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4af4107460 0x7f4af419f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.455 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.453+0000 7f4afacca640 1 --2- 172.21.15.80:0/460372372 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4af4104b30 0x7f4af4198010 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.455 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.453+0000 7f4afacca640 1 --2- 172.21.15.80:0/460372372 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4af4102200 0x7f4af4197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.455 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.453+0000 7f4afacca640 1 -- 172.21.15.80:0/460372372 >> 172.21.15.80:0/460372372 conn(0x7f4af40fbbf0 msgr2=0x7f4af4102c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:20.455 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.453+0000 7f4afacca640 1 -- 172.21.15.80:0/460372372 shutdown_connections 2024-04-24T23:30:20.455 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.453+0000 7f4afacca640 1 -- 172.21.15.80:0/460372372 wait complete. 2024-04-24T23:30:20.466 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T23:30:20.508 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:30:20 smithi148 ceph-mon[32528]: pgmap v390: 129 pgs: 129 active+clean; 650 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T23:30:20.571 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.567+0000 7f8628e5d640 1 -- 172.21.15.80:0/1727625791 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8624101890 msgr2=0x7f8624101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:20.571 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.567+0000 7f8628e5d640 1 --2- 172.21.15.80:0/1727625791 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8624101890 0x7f8624101d10 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f8618009900 tx=0x7f861802a410 comp rx=0 tx=0).stop 2024-04-24T23:30:20.571 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.567+0000 7f8628e5d640 1 -- 172.21.15.80:0/1727625791 shutdown_connections 2024-04-24T23:30:20.571 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.567+0000 7f8628e5d640 1 --2- 172.21.15.80:0/1727625791 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8624102250 0x7f8624108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.571 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.567+0000 7f8628e5d640 1 --2- 172.21.15.80:0/1727625791 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8624101890 0x7f8624101d10 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.571 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.567+0000 7f8628e5d640 1 --2- 172.21.15.80:0/1727625791 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86241006a0 0x7f8624100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.571 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.567+0000 7f8628e5d640 1 -- 172.21.15.80:0/1727625791 >> 172.21.15.80:0/1727625791 conn(0x7f86240fbc30 msgr2=0x7f86240fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:20.571 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.567+0000 7f8628e5d640 1 -- 172.21.15.80:0/1727625791 shutdown_connections 2024-04-24T23:30:20.571 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.568+0000 7f8628e5d640 1 -- 172.21.15.80:0/1727625791 wait complete. 2024-04-24T23:30:20.571 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.568+0000 7f8628e5d640 1 Processor -- start 2024-04-24T23:30:20.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.568+0000 7f8628e5d640 1 -- start start 2024-04-24T23:30:20.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.568+0000 7f8628e5d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f86241006a0 0x7f8624104630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:20.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.568+0000 7f8628e5d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8624101890 0x7f8624102c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:20.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.568+0000 7f8628e5d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8624102250 0x7f86241031b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:20.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.568+0000 7f8628e5d640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f862410c300 con 0x7f86241006a0 2024-04-24T23:30:20.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.568+0000 7f8628e5d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f862410c180 con 0x7f8624101890 2024-04-24T23:30:20.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.568+0000 7f8628e5d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f862410c480 con 0x7f8624102250 2024-04-24T23:30:20.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.568+0000 7f862259c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f86241006a0 0x7f8624104630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:20.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.569+0000 7f862259c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f86241006a0 0x7f8624104630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:46796/0 (socket says 172.21.15.80:46796) 2024-04-24T23:30:20.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.569+0000 7f862259c640 1 -- 172.21.15.80:0/1343060477 learned_addr learned my addr 172.21.15.80:0/1343060477 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:30:20.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.569+0000 7f8621d9b640 1 --2- 172.21.15.80:0/1343060477 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8624101890 0x7f8624102c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:20.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.569+0000 7f8622d9d640 1 --2- 172.21.15.80:0/1343060477 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8624102250 0x7f86241031b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:20.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.569+0000 7f862259c640 1 -- 172.21.15.80:0/1343060477 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8624102250 msgr2=0x7f86241031b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:20.573 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.569+0000 7f862259c640 1 --2- 172.21.15.80:0/1343060477 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8624102250 0x7f86241031b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.573 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.569+0000 7f862259c640 1 -- 172.21.15.80:0/1343060477 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8624101890 msgr2=0x7f8624102c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:20.573 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.569+0000 7f862259c640 1 --2- 172.21.15.80:0/1343060477 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8624101890 0x7f8624102c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.573 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.569+0000 7f862259c640 1 -- 172.21.15.80:0/1343060477 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8624103a40 con 0x7f86241006a0 2024-04-24T23:30:20.573 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.569+0000 7f862259c640 1 --2- 172.21.15.80:0/1343060477 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f86241006a0 0x7f8624104630 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f861400d950 tx=0x7f861400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:20.573 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.569+0000 7f86137fe640 1 -- 172.21.15.80:0/1343060477 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8614017020 con 0x7f86241006a0 2024-04-24T23:30:20.573 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.569+0000 7f8628e5d640 1 -- 172.21.15.80:0/1343060477 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86241a1ca0 con 0x7f86241006a0 2024-04-24T23:30:20.573 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.569+0000 7f86137fe640 1 -- 172.21.15.80:0/1343060477 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8614009c30 con 0x7f86241006a0 2024-04-24T23:30:20.573 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.569+0000 7f86137fe640 1 -- 172.21.15.80:0/1343060477 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f861400f740 con 0x7f86241006a0 2024-04-24T23:30:20.573 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.569+0000 7f8628e5d640 1 -- 172.21.15.80:0/1343060477 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86241a21c0 con 0x7f86241006a0 2024-04-24T23:30:20.580 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.572+0000 7f862bb69740 1 Processor -- start 2024-04-24T23:30:20.580 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.572+0000 7f862bb69740 1 -- start start 2024-04-24T23:30:20.580 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.572+0000 7f862bb69740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5647b80ef230 0x5647b80ef650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:20.580 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.572+0000 7f862bb69740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5647b80efc20 0x5647b80f4720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:20.580 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.572+0000 7f862bb69740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5647b80f4d90 0x5647b80f7220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:20.581 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.572+0000 7f862bb69740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x5647b8053670 con 0x5647b80f4d90 2024-04-24T23:30:20.581 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.572+0000 7f862bb69740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5647b7ffb970 con 0x5647b80efc20 2024-04-24T23:30:20.581 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.572+0000 7f862bb69740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5647b804f630 con 0x5647b80ef230 2024-04-24T23:30:20.581 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.572+0000 7f8621d9b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5647b80f4d90 0x5647b80f7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:20.581 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.572+0000 7f86137fe640 1 -- 172.21.15.80:0/1343060477 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f861401a020 con 0x7f86241006a0 2024-04-24T23:30:20.581 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.572+0000 7f8621d9b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5647b80f4d90 0x5647b80f7220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:46810/0 (socket says 172.21.15.80:46810) 2024-04-24T23:30:20.581 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.572+0000 7f8621d9b640 1 -- 172.21.15.80:0/2884174612 learned_addr learned my addr 172.21.15.80:0/2884174612 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:30:20.581 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.572+0000 7f8622d9d640 1 --2- 172.21.15.80:0/2884174612 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5647b80ef230 0x5647b80ef650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:20.581 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.573+0000 7f8621d9b640 1 -- 172.21.15.80:0/2884174612 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5647b80ef230 msgr2=0x5647b80ef650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:20.581 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.573+0000 7f8621d9b640 1 --2- 172.21.15.80:0/2884174612 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5647b80ef230 0x5647b80ef650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.582 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.573+0000 7f8621d9b640 1 -- 172.21.15.80:0/2884174612 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5647b80efc20 msgr2=0x5647b80f4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:20.582 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.573+0000 7f862259c640 1 --2- 172.21.15.80:0/2884174612 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5647b80efc20 0x5647b80f4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:20.582 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.573+0000 7f8621d9b640 1 --2- 172.21.15.80:0/2884174612 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5647b80efc20 0x5647b80f4720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.582 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.573+0000 7f8621d9b640 1 -- 172.21.15.80:0/2884174612 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647b801f400 con 0x5647b80f4d90 2024-04-24T23:30:20.582 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.573+0000 7f8622d9d640 1 --2- 172.21.15.80:0/2884174612 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5647b80ef230 0x5647b80ef650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:30:20.582 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.573+0000 7f862259c640 1 --2- 172.21.15.80:0/2884174612 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5647b80efc20 0x5647b80f4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:30:20.582 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.573+0000 7f8621d9b640 1 --2- 172.21.15.80:0/2884174612 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5647b80f4d90 0x5647b80f7220 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f8618009900 tx=0x7f8618031730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc11995dde943d46 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:20.582 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.573+0000 7f86137fe640 1 --2- 172.21.15.80:0/1343060477 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f860c0a2b90 0x7f860c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:20.582 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.573+0000 7f8610ff9640 1 -- 172.21.15.80:0/2884174612 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8618040070 con 0x5647b80f4d90 2024-04-24T23:30:20.582 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.573+0000 7f8610ff9640 1 -- 172.21.15.80:0/2884174612 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86180319e0 con 0x5647b80f4d90 2024-04-24T23:30:20.583 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.573+0000 7f8610ff9640 1 -- 172.21.15.80:0/2884174612 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8618035040 con 0x5647b80f4d90 2024-04-24T23:30:20.583 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.573+0000 7f8622d9d640 1 --2- 172.21.15.80:0/1343060477 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f860c0a2b90 0x7f860c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:20.583 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.573+0000 7f86137fe640 1 -- 172.21.15.80:0/1343060477 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f86140ceee0 con 0x7f86241006a0 2024-04-24T23:30:20.583 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.574+0000 7f862bb69740 1 -- 172.21.15.80:0/2884174612 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5647b80f4d90 msgr2=0x5647b80f7220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:20.583 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.574+0000 7f862bb69740 1 --2- 172.21.15.80:0/2884174612 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5647b80f4d90 0x5647b80f7220 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f8618009900 tx=0x7f8618031730 comp rx=0 tx=0).stop 2024-04-24T23:30:20.583 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.574+0000 7f8622d9d640 1 --2- 172.21.15.80:0/1343060477 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f860c0a2b90 0x7f860c0a5050 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x5647b80f0480 tx=0x7f861c00b400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:20.583 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.574+0000 7f862bb69740 1 -- 172.21.15.80:0/2884174612 shutdown_connections 2024-04-24T23:30:20.583 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.574+0000 7f862bb69740 1 --2- 172.21.15.80:0/2884174612 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5647b80f4d90 0x5647b80f7220 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.583 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.574+0000 7f862bb69740 1 --2- 172.21.15.80:0/2884174612 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5647b80efc20 0x5647b80f4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.583 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.574+0000 7f862bb69740 1 --2- 172.21.15.80:0/2884174612 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5647b80ef230 0x5647b80ef650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.583 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.574+0000 7f862bb69740 1 -- 172.21.15.80:0/2884174612 >> 172.21.15.80:0/2884174612 conn(0x5647b80ea8a0 msgr2=0x5647b80eaca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:20.583 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.574+0000 7f862bb69740 1 -- 172.21.15.80:0/2884174612 shutdown_connections 2024-04-24T23:30:20.583 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.574+0000 7f862bb69740 1 -- 172.21.15.80:0/2884174612 wait complete. 2024-04-24T23:30:20.584 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.574+0000 7f862bb69740 1 -- start start 2024-04-24T23:30:20.584 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.574+0000 7f862bb69740 1 Processor -- start 2024-04-24T23:30:20.584 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.575+0000 7f862bb69740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5647b80ef230 0x5647b80f9300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:20.584 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.575+0000 7f862bb69740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5647b80efc20 0x5647b80f9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:20.584 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.575+0000 7f862bb69740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5647b80f4d90 0x5647b8100810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:20.584 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.575+0000 7f862bb69740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x5647b80f7ce0 con 0x5647b80f4d90 2024-04-24T23:30:20.584 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.575+0000 7f862bb69740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5647b800f050 con 0x5647b80efc20 2024-04-24T23:30:20.584 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.575+0000 7f862bb69740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5647b80f7e60 con 0x5647b80ef230 2024-04-24T23:30:20.584 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.575+0000 7f862bb69740 10 client.0 _ll_register_callbacks cb 0x5647b7d66be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:30:20.585 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.575+0000 7f8622d9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5647b80f4d90 0x5647b8100810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:20.585 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.575+0000 7f8622d9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5647b80f4d90 0x5647b8100810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:46826/0 (socket says 172.21.15.80:46826) 2024-04-24T23:30:20.585 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.575+0000 7f8622d9d640 1 -- 172.21.15.80:0/3078123639 learned_addr learned my addr 172.21.15.80:0/3078123639 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:30:20.585 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.575+0000 7f8622d9d640 1 -- 172.21.15.80:0/3078123639 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5647b80ef230 msgr2=0x5647b80f9300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:20.585 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.575+0000 7f8622d9d640 1 --2- 172.21.15.80:0/3078123639 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5647b80ef230 0x5647b80f9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.585 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.575+0000 7f8622d9d640 1 -- 172.21.15.80:0/3078123639 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5647b80efc20 msgr2=0x5647b80f9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:20.585 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.575+0000 7f8622d9d640 1 --2- 172.21.15.80:0/3078123639 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5647b80efc20 0x5647b80f9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.585 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.575+0000 7f8622d9d640 1 -- 172.21.15.80:0/3078123639 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647b7fe0dc0 con 0x5647b80f4d90 2024-04-24T23:30:20.585 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.575+0000 7f8622d9d640 1 --2- 172.21.15.80:0/3078123639 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5647b80f4d90 0x5647b8100810 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f861c00bc10 tx=0x7f861c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6f89addfb1f16a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:20.585 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.575+0000 7f8609ffb640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-04-24T23:30:20.585 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.575+0000 7f8609ffb640 1 -- 172.21.15.80:0/3078123639 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f861c004b20 con 0x5647b80f4d90 2024-04-24T23:30:20.586 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.575+0000 7f862bb69740 1 -- 172.21.15.80:0/3078123639 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86180319e0 con 0x5647b80f4d90 2024-04-24T23:30:20.586 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.575+0000 7f8609ffb640 1 -- 172.21.15.80:0/3078123639 <== mon.0 v2:172.21.15.80:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f861c004cc0 con 0x5647b80f4d90 2024-04-24T23:30:20.586 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.575+0000 7f8609ffb640 1 -- 172.21.15.80:0/3078123639 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f861c02e050 con 0x5647b80f4d90 2024-04-24T23:30:20.586 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.576+0000 7f862bb69740 10 client.15678 fetch_fsmap learned FSMap version 25 2024-04-24T23:30:20.586 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.576+0000 7f862bb69740 1 -- 172.21.15.80:0/3078123639 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5647b7d673f0 con 0x5647b80f4d90 2024-04-24T23:30:20.586 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.576+0000 7f8609ffb640 1 -- 172.21.15.80:0/3078123639 <== mon.0 v2:172.21.15.80:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86180319e0 con 0x5647b80f4d90 2024-04-24T23:30:20.586 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.576+0000 7f8609ffb640 1 -- 172.21.15.80:0/3078123639 <== mon.0 v2:172.21.15.80:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f861c01e520 con 0x5647b80f4d90 2024-04-24T23:30:20.586 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.576+0000 7f862bb69740 10 client.15678 fetch_fsmap finished waiting for FSMap version 25 2024-04-24T23:30:20.586 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.576+0000 7f862bb69740 10 client.15678 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-24T23:30:20.586 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.576+0000 7f862bb69740 20 client.15678 populate_metadata read hostname 'smithi080' 2024-04-24T23:30:20.586 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.576+0000 7f862bb69740 1 --2- 172.21.15.80:0/3078123639 >> [v2:172.21.15.148:6836/856528403,v1:172.21.15.148:6837/856528403] conn(0x5647b8105190 0x5647b8107650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:20.586 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.576+0000 7f862bb69740 4 client.15678 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:30:20.586 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.576+0000 7f862bb69740 1 -- 172.21.15.80:0/3078123639 --> [v2:172.21.15.148:6836/856528403,v1:172.21.15.148:6837/856528403] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5647b8060bb0 con 0x5647b8105190 2024-04-24T23:30:20.586 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.576+0000 7f862259c640 1 --2- 172.21.15.80:0/3078123639 >> [v2:172.21.15.148:6836/856528403,v1:172.21.15.148:6837/856528403] conn(0x5647b8105190 0x5647b8107650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:20.587 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.577+0000 7f8609ffb640 10 client.15678 ms_handle_connect on v2:172.21.15.148:6836/856528403 2024-04-24T23:30:20.587 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.577+0000 7f862259c640 1 --2- 172.21.15.80:0/3078123639 >> [v2:172.21.15.148:6836/856528403,v1:172.21.15.148:6837/856528403] conn(0x5647b8105190 0x5647b8107650 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8e1f3a5c0fe64feb server_cookie=ee56a89b3d5792d in_seq=0 out_seq=0 2024-04-24T23:30:20.587 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.577+0000 7f8609ffb640 1 -- 172.21.15.80:0/3078123639 <== mds.2 v2:172.21.15.148:6836/856528403 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5647b8060bb0 con 0x5647b8105190 2024-04-24T23:30:20.587 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.577+0000 7f8609ffb640 10 client.15678 handle_command_reply: tid=0 2024-04-24T23:30:20.587 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.577+0000 7f862bb69740 1 client.15678 shutdown 2024-04-24T23:30:20.587 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.578+0000 7f862bb69740 1 -- 172.21.15.80:0/3078123639 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5647b80f4d90 msgr2=0x5647b8100810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:20.587 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.578+0000 7f862bb69740 1 --2- 172.21.15.80:0/3078123639 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5647b80f4d90 0x5647b8100810 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f861c00bc10 tx=0x7f861c004a40 comp rx=0 tx=0).stop 2024-04-24T23:30:20.587 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.578+0000 7f862bb69740 1 -- 172.21.15.80:0/3078123639 shutdown_connections 2024-04-24T23:30:20.587 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.578+0000 7f862bb69740 1 --2- 172.21.15.80:0/3078123639 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5647b80f4d90 0x5647b8100810 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.587 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.578+0000 7f862bb69740 1 --2- 172.21.15.80:0/3078123639 >> [v2:172.21.15.148:6836/856528403,v1:172.21.15.148:6837/856528403] conn(0x5647b8105190 0x5647b8107650 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.587 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.578+0000 7f862bb69740 1 --2- 172.21.15.80:0/3078123639 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5647b80efc20 0x5647b80f9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.587 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.578+0000 7f862bb69740 1 --2- 172.21.15.80:0/3078123639 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5647b80ef230 0x5647b80f9300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.587 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.578+0000 7f862bb69740 1 -- 172.21.15.80:0/3078123639 >> 172.21.15.80:0/3078123639 conn(0x5647b80ea8a0 msgr2=0x5647b80e9b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:20.587 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.578+0000 7f8609ffb640 0 client.15678 ms_handle_reset on v2:172.21.15.148:6836/856528403 2024-04-24T23:30:20.588 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.578+0000 7f862bb69740 1 -- 172.21.15.80:0/3078123639 shutdown_connections 2024-04-24T23:30:20.588 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.578+0000 7f862bb69740 1 -- 172.21.15.80:0/3078123639 wait complete. 2024-04-24T23:30:20.588 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.578+0000 7f862bb69740 20 client.15678 trim_cache size 0 max 16384 2024-04-24T23:30:20.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.605+0000 7f862bb69740 1 Processor -- start 2024-04-24T23:30:20.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.605+0000 7f862bb69740 1 -- start start 2024-04-24T23:30:20.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.605+0000 7f862bb69740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5647b80ef230 0x5647b80f06c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:20.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.605+0000 7f862bb69740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5647b80efc20 0x5647b81007d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:20.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.605+0000 7f862bb69740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5647b80f4d90 0x5647b80f0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:20.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.605+0000 7f862bb69740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f86241af0a0 con 0x5647b80f4d90 2024-04-24T23:30:20.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.605+0000 7f862bb69740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f86241ac970 con 0x5647b80efc20 2024-04-24T23:30:20.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.605+0000 7f862bb69740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f86241aa7c0 con 0x5647b80ef230 2024-04-24T23:30:20.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.605+0000 7f8622d9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5647b80f4d90 0x5647b80f0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:20.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.605+0000 7f8622d9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5647b80f4d90 0x5647b80f0c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:46834/0 (socket says 172.21.15.80:46834) 2024-04-24T23:30:20.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.605+0000 7f8622d9d640 1 -- 172.21.15.80:0/646361163 learned_addr learned my addr 172.21.15.80:0/646361163 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:30:20.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.605+0000 7f8621d9b640 1 --2- 172.21.15.80:0/646361163 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5647b80efc20 0x5647b81007d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:20.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.605+0000 7f8622d9d640 1 -- 172.21.15.80:0/646361163 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5647b80ef230 msgr2=0x5647b80f06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:20.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.606+0000 7f8622d9d640 1 --2- 172.21.15.80:0/646361163 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5647b80ef230 0x5647b80f06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.606+0000 7f8622d9d640 1 -- 172.21.15.80:0/646361163 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5647b80efc20 msgr2=0x5647b81007d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:20.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.606+0000 7f8622d9d640 1 --2- 172.21.15.80:0/646361163 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5647b80efc20 0x5647b81007d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.606+0000 7f8622d9d640 1 -- 172.21.15.80:0/646361163 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647b7e317d0 con 0x5647b80f4d90 2024-04-24T23:30:20.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.606+0000 7f8622d9d640 1 --2- 172.21.15.80:0/646361163 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5647b80f4d90 0x5647b80f0c00 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f861c010c30 tx=0x7f861c010e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23f8c77c33fec60a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:20.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.606+0000 7f860bfff640 1 -- 172.21.15.80:0/646361163 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f861c01edd0 con 0x5647b80f4d90 2024-04-24T23:30:20.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.606+0000 7f860bfff640 1 -- 172.21.15.80:0/646361163 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f861c01e420 con 0x5647b80f4d90 2024-04-24T23:30:20.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.606+0000 7f860bfff640 1 -- 172.21.15.80:0/646361163 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f861c005740 con 0x5647b80f4d90 2024-04-24T23:30:20.614 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.607+0000 7f862bb69740 1 -- 172.21.15.80:0/646361163 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5647b80f4d90 msgr2=0x5647b80f0c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:20.614 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.607+0000 7f862bb69740 1 --2- 172.21.15.80:0/646361163 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5647b80f4d90 0x5647b80f0c00 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f861c010c30 tx=0x7f861c010e00 comp rx=0 tx=0).stop 2024-04-24T23:30:20.614 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.607+0000 7f862bb69740 1 -- 172.21.15.80:0/646361163 shutdown_connections 2024-04-24T23:30:20.614 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.607+0000 7f862bb69740 1 --2- 172.21.15.80:0/646361163 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5647b80f4d90 0x5647b80f0c00 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.614 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.607+0000 7f862bb69740 1 --2- 172.21.15.80:0/646361163 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5647b80efc20 0x5647b81007d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.614 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.607+0000 7f862bb69740 1 --2- 172.21.15.80:0/646361163 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5647b80ef230 0x5647b80f06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.614 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.607+0000 7f862bb69740 1 -- 172.21.15.80:0/646361163 >> 172.21.15.80:0/646361163 conn(0x5647b80ea8a0 msgr2=0x5647b80f9300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:20.614 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.607+0000 7f862bb69740 1 -- 172.21.15.80:0/646361163 shutdown_connections 2024-04-24T23:30:20.615 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.607+0000 7f862bb69740 1 -- 172.21.15.80:0/646361163 wait complete. 2024-04-24T23:30:20.615 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.607+0000 7f862bb69740 1 -- start start 2024-04-24T23:30:20.615 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.607+0000 7f862bb69740 1 Processor -- start 2024-04-24T23:30:20.615 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.608+0000 7f862bb69740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5647b80ef230 0x5647b80fd890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:20.615 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.608+0000 7f862bb69740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5647b80efc20 0x5647b80fddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:20.615 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.608+0000 7f862bb69740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5647b80f4d90 0x5647b80fede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:20.615 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.608+0000 7f862bb69740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f86241aae30 con 0x5647b80f4d90 2024-04-24T23:30:20.615 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.608+0000 7f862bb69740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f86241ac2a0 con 0x5647b80ef230 2024-04-24T23:30:20.615 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.608+0000 7f862bb69740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f86241ab500 con 0x5647b80efc20 2024-04-24T23:30:20.615 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.608+0000 7f862bb69740 10 client.0 _ll_register_callbacks cb 0x7f861c00be30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:30:20.615 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.608+0000 7f8622d9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5647b80f4d90 0x5647b80fede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:20.615 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.608+0000 7f8622d9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5647b80f4d90 0x5647b80fede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:46850/0 (socket says 172.21.15.80:46850) 2024-04-24T23:30:20.615 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.608+0000 7f8622d9d640 1 -- 172.21.15.80:0/525053060 learned_addr learned my addr 172.21.15.80:0/525053060 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:30:20.616 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.608+0000 7f862259c640 1 --2- 172.21.15.80:0/525053060 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5647b80ef230 0x5647b80fd890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:20.616 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.608+0000 7f8622d9d640 1 -- 172.21.15.80:0/525053060 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5647b80efc20 msgr2=0x5647b80fddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:20.616 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.608+0000 7f8622d9d640 1 --2- 172.21.15.80:0/525053060 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5647b80efc20 0x5647b80fddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.616 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.608+0000 7f8622d9d640 1 -- 172.21.15.80:0/525053060 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5647b80ef230 msgr2=0x5647b80fd890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:20.616 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.608+0000 7f8622d9d640 1 --2- 172.21.15.80:0/525053060 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5647b80ef230 0x5647b80fd890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.616 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.608+0000 7f8622d9d640 1 -- 172.21.15.80:0/525053060 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647b80ff3d0 con 0x5647b80f4d90 2024-04-24T23:30:20.616 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.608+0000 7f862259c640 1 --2- 172.21.15.80:0/525053060 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5647b80ef230 0x5647b80fd890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:30:20.616 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.608+0000 7f8622d9d640 1 --2- 172.21.15.80:0/525053060 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5647b80f4d90 0x5647b80fede0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f861c01e960 tx=0x7f861c005230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=550aa46e07392fa0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:20.616 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.608+0000 7f8609ffb640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-04-24T23:30:20.616 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.608+0000 7f8609ffb640 1 -- 172.21.15.80:0/525053060 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f861c01ed50 con 0x5647b80f4d90 2024-04-24T23:30:20.616 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.608+0000 7f8609ffb640 1 -- 172.21.15.80:0/525053060 <== mon.0 v2:172.21.15.80:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f861c004a00 con 0x5647b80f4d90 2024-04-24T23:30:20.616 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.608+0000 7f862bb69740 1 -- 172.21.15.80:0/525053060 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f861c01e420 con 0x5647b80f4d90 2024-04-24T23:30:20.616 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.608+0000 7f8609ffb640 1 -- 172.21.15.80:0/525053060 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f861c04f050 con 0x5647b80f4d90 2024-04-24T23:30:20.616 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.608+0000 7f862bb69740 10 client.15684 fetch_fsmap learned FSMap version 25 2024-04-24T23:30:20.616 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.608+0000 7f862bb69740 1 -- 172.21.15.80:0/525053060 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5647b81006d0 con 0x5647b80f4d90 2024-04-24T23:30:20.617 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.609+0000 7f8609ffb640 1 -- 172.21.15.80:0/525053060 <== mon.0 v2:172.21.15.80:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f861c01e420 con 0x5647b80f4d90 2024-04-24T23:30:20.617 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.609+0000 7f8609ffb640 1 -- 172.21.15.80:0/525053060 <== mon.0 v2:172.21.15.80:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f861c044020 con 0x5647b80f4d90 2024-04-24T23:30:20.617 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.609+0000 7f862bb69740 10 client.15684 fetch_fsmap finished waiting for FSMap version 25 2024-04-24T23:30:20.617 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.609+0000 7f862bb69740 10 client.15684 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-24T23:30:20.617 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.609+0000 7f862bb69740 20 client.15684 populate_metadata read hostname 'smithi080' 2024-04-24T23:30:20.617 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.609+0000 7f862bb69740 1 --2- 172.21.15.80:0/525053060 >> [v2:172.21.15.148:6836/856528403,v1:172.21.15.148:6837/856528403] conn(0x5647b816bb40 0x5647b816df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:20.617 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.609+0000 7f862bb69740 4 client.15684 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:30:20.617 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.609+0000 7f862bb69740 1 -- 172.21.15.80:0/525053060 --> [v2:172.21.15.148:6836/856528403,v1:172.21.15.148:6837/856528403] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5647b80f4b00 con 0x5647b816bb40 2024-04-24T23:30:20.617 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.609+0000 7f862259c640 1 --2- 172.21.15.80:0/525053060 >> [v2:172.21.15.148:6836/856528403,v1:172.21.15.148:6837/856528403] conn(0x5647b816bb40 0x5647b816df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:20.617 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.610+0000 7f8609ffb640 10 client.15684 ms_handle_connect on v2:172.21.15.148:6836/856528403 2024-04-24T23:30:20.617 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.610+0000 7f862259c640 1 --2- 172.21.15.80:0/525053060 >> [v2:172.21.15.148:6836/856528403,v1:172.21.15.148:6837/856528403] conn(0x5647b816bb40 0x5647b816df60 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d49533b0cc97eeb6 server_cookie=f7f376f5c2441595 in_seq=0 out_seq=0 2024-04-24T23:30:20.617 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.610+0000 7f8609ffb640 1 -- 172.21.15.80:0/525053060 <== mds.2 v2:172.21.15.148:6836/856528403 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5647b80f4b00 con 0x5647b816bb40 2024-04-24T23:30:20.617 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.610+0000 7f8609ffb640 10 client.15684 handle_command_reply: tid=0 2024-04-24T23:30:20.617 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.610+0000 7f862bb69740 1 client.15684 shutdown 2024-04-24T23:30:20.618 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.610+0000 7f862bb69740 1 -- 172.21.15.80:0/525053060 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5647b80f4d90 msgr2=0x5647b80fede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:20.618 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.610+0000 7f862bb69740 1 --2- 172.21.15.80:0/525053060 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5647b80f4d90 0x5647b80fede0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f861c01e960 tx=0x7f861c005230 comp rx=0 tx=0).stop 2024-04-24T23:30:20.618 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.610+0000 7f862bb69740 1 -- 172.21.15.80:0/525053060 shutdown_connections 2024-04-24T23:30:20.618 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.610+0000 7f862bb69740 1 --2- 172.21.15.80:0/525053060 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5647b80f4d90 0x5647b80fede0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.618 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.610+0000 7f862bb69740 1 --2- 172.21.15.80:0/525053060 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5647b80efc20 0x5647b80fddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.618 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.610+0000 7f862bb69740 1 --2- 172.21.15.80:0/525053060 >> [v2:172.21.15.148:6836/856528403,v1:172.21.15.148:6837/856528403] conn(0x5647b816bb40 0x5647b816df60 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.618 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.610+0000 7f8609ffb640 0 client.15684 ms_handle_reset on v2:172.21.15.148:6836/856528403 2024-04-24T23:30:20.618 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.610+0000 7f862bb69740 1 --2- 172.21.15.80:0/525053060 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5647b80ef230 0x5647b80fd890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.618 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.610+0000 7f862bb69740 1 -- 172.21.15.80:0/525053060 >> 172.21.15.80:0/525053060 conn(0x5647b80ea8a0 msgr2=0x5647b80e92b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:20.618 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.610+0000 7f862bb69740 1 -- 172.21.15.80:0/525053060 shutdown_connections 2024-04-24T23:30:20.618 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.610+0000 7f862bb69740 1 -- 172.21.15.80:0/525053060 wait complete. 2024-04-24T23:30:20.618 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.610+0000 7f862bb69740 20 client.15684 trim_cache size 0 max 16384 2024-04-24T23:30:20.619 INFO:teuthology.orchestra.run.smithi080.stdout:[] 2024-04-24T23:30:20.619 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.610+0000 7f8628e5d640 1 -- 172.21.15.80:0/1343060477 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f860c0a2b90 msgr2=0x7f860c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:20.619 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.610+0000 7f8628e5d640 1 --2- 172.21.15.80:0/1343060477 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f860c0a2b90 0x7f860c0a5050 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x5647b80f0480 tx=0x7f861c00b400 comp rx=0 tx=0).stop 2024-04-24T23:30:20.619 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.611+0000 7f8628e5d640 1 -- 172.21.15.80:0/1343060477 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f86241006a0 msgr2=0x7f8624104630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:20.620 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.611+0000 7f8628e5d640 1 --2- 172.21.15.80:0/1343060477 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f86241006a0 0x7f8624104630 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f861400d950 tx=0x7f861400de10 comp rx=0 tx=0).stop 2024-04-24T23:30:20.620 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.611+0000 7f8628e5d640 1 -- 172.21.15.80:0/1343060477 shutdown_connections 2024-04-24T23:30:20.620 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.611+0000 7f8628e5d640 1 --2- 172.21.15.80:0/1343060477 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f860c0a2b90 0x7f860c0a5050 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.620 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.611+0000 7f8628e5d640 1 --2- 172.21.15.80:0/1343060477 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8624102250 0x7f86241031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.620 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.611+0000 7f8628e5d640 1 --2- 172.21.15.80:0/1343060477 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8624101890 0x7f8624102c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.620 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.611+0000 7f8628e5d640 1 --2- 172.21.15.80:0/1343060477 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f86241006a0 0x7f8624104630 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.620 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.611+0000 7f8628e5d640 1 -- 172.21.15.80:0/1343060477 >> 172.21.15.80:0/1343060477 conn(0x7f86240fbc30 msgr2=0x7f8624069b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:20.620 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.611+0000 7f8628e5d640 1 -- 172.21.15.80:0/1343060477 shutdown_connections 2024-04-24T23:30:20.620 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.611+0000 7f8628e5d640 1 -- 172.21.15.80:0/1343060477 wait complete. 2024-04-24T23:30:20.622 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T23:30:20.730 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.725+0000 7f4a1288c640 1 -- 172.21.15.80:0/2551768121 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4a0c100050 msgr2=0x7f4a0c100470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:20.730 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.725+0000 7f4a1288c640 1 --2- 172.21.15.80:0/2551768121 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4a0c100050 0x7f4a0c100470 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f49fc009900 tx=0x7f49fc02a410 comp rx=0 tx=0).stop 2024-04-24T23:30:20.730 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.725+0000 7f4a1288c640 1 -- 172.21.15.80:0/2551768121 shutdown_connections 2024-04-24T23:30:20.730 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.725+0000 7f4a1288c640 1 --2- 172.21.15.80:0/2551768121 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a0c101350 0x7f4a0c1017d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.730 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.725+0000 7f4a1288c640 1 --2- 172.21.15.80:0/2551768121 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4a0c1009b0 0x7f4a0c100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.730 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.725+0000 7f4a1288c640 1 --2- 172.21.15.80:0/2551768121 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4a0c100050 0x7f4a0c100470 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.730 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.725+0000 7f4a1288c640 1 -- 172.21.15.80:0/2551768121 >> 172.21.15.80:0/2551768121 conn(0x7f4a0c0fbbf0 msgr2=0x7f4a0c0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:20.730 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.725+0000 7f4a1288c640 1 -- 172.21.15.80:0/2551768121 shutdown_connections 2024-04-24T23:30:20.731 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.725+0000 7f4a1288c640 1 -- 172.21.15.80:0/2551768121 wait complete. 2024-04-24T23:30:20.731 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.726+0000 7f4a1288c640 1 Processor -- start 2024-04-24T23:30:20.731 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.726+0000 7f4a1288c640 1 -- start start 2024-04-24T23:30:20.731 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.726+0000 7f4a1288c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4a0c100050 0x7f4a0c199cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:20.731 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.726+0000 7f4a1288c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a0c1009b0 0x7f4a0c19a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:20.731 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.726+0000 7f4a1288c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4a0c101350 0x7f4a0c1a1260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:20.731 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.726+0000 7f4a1288c640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f4a0c10f5c0 con 0x7f4a0c101350 2024-04-24T23:30:20.731 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.726+0000 7f4a1288c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4a0c10f440 con 0x7f4a0c100050 2024-04-24T23:30:20.731 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.726+0000 7f4a1288c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4a0c10f740 con 0x7f4a0c1009b0 2024-04-24T23:30:20.731 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.727+0000 7f4a10e29640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4a0c101350 0x7f4a0c1a1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:20.731 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.727+0000 7f4a10e29640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4a0c101350 0x7f4a0c1a1260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:46858/0 (socket says 172.21.15.80:46858) 2024-04-24T23:30:20.731 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.727+0000 7f4a10e29640 1 -- 172.21.15.80:0/2604464405 learned_addr learned my addr 172.21.15.80:0/2604464405 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:30:20.731 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.727+0000 7f4a0bfff640 1 --2- 172.21.15.80:0/2604464405 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4a0c100050 0x7f4a0c199cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:20.732 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.727+0000 7f4a0b7fe640 1 --2- 172.21.15.80:0/2604464405 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a0c1009b0 0x7f4a0c19a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:20.732 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.727+0000 7f4a10e29640 1 -- 172.21.15.80:0/2604464405 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a0c1009b0 msgr2=0x7f4a0c19a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:20.732 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.727+0000 7f4a10e29640 1 --2- 172.21.15.80:0/2604464405 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a0c1009b0 0x7f4a0c19a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.732 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.727+0000 7f4a10e29640 1 -- 172.21.15.80:0/2604464405 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4a0c100050 msgr2=0x7f4a0c199cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:20.732 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.727+0000 7f4a10e29640 1 --2- 172.21.15.80:0/2604464405 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4a0c100050 0x7f4a0c199cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.732 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.727+0000 7f4a10e29640 1 -- 172.21.15.80:0/2604464405 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a0c1a1900 con 0x7f4a0c101350 2024-04-24T23:30:20.732 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.727+0000 7f4a10e29640 1 --2- 172.21.15.80:0/2604464405 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4a0c101350 0x7f4a0c1a1260 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f4a0400eb10 tx=0x7f4a0400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:20.732 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.727+0000 7f4a097fa640 1 -- 172.21.15.80:0/2604464405 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a04004b20 con 0x7f4a0c101350 2024-04-24T23:30:20.732 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.727+0000 7f4a1288c640 1 -- 172.21.15.80:0/2604464405 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a0c1a1bf0 con 0x7f4a0c101350 2024-04-24T23:30:20.732 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.727+0000 7f4a097fa640 1 -- 172.21.15.80:0/2604464405 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a04004cc0 con 0x7f4a0c101350 2024-04-24T23:30:20.732 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.727+0000 7f4a097fa640 1 -- 172.21.15.80:0/2604464405 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a04010760 con 0x7f4a0c101350 2024-04-24T23:30:20.732 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.727+0000 7f4a1288c640 1 -- 172.21.15.80:0/2604464405 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a0c1a20f0 con 0x7f4a0c101350 2024-04-24T23:30:20.733 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.729+0000 7f4a097fa640 1 -- 172.21.15.80:0/2604464405 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4a04005020 con 0x7f4a0c101350 2024-04-24T23:30:20.740 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.730+0000 7f4a15596740 1 Processor -- start 2024-04-24T23:30:20.740 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.730+0000 7f4a15596740 1 -- start start 2024-04-24T23:30:20.740 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.730+0000 7f4a15596740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b0d19e8230 0x55b0d19e8650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:20.740 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.730+0000 7f4a15596740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b0d19e8c20 0x55b0d19ed720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:20.740 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.730+0000 7f4a15596740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b0d19edd90 0x55b0d19f0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:20.740 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.730+0000 7f4a15596740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x55b0d194c670 con 0x55b0d19e8230 2024-04-24T23:30:20.740 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.730+0000 7f4a15596740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55b0d18f4970 con 0x55b0d19edd90 2024-04-24T23:30:20.740 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.730+0000 7f4a15596740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55b0d1948630 con 0x55b0d19e8c20 2024-04-24T23:30:20.741 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.730+0000 7f4a0b7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b0d19e8230 0x55b0d19e8650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:20.741 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.730+0000 7f4a097fa640 1 --2- 172.21.15.80:0/2604464405 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f49f40a2b90 0x7f49f40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:20.741 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.730+0000 7f4a0b7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b0d19e8230 0x55b0d19e8650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:46864/0 (socket says 172.21.15.80:46864) 2024-04-24T23:30:20.741 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.730+0000 7f4a0b7fe640 1 -- 172.21.15.80:0/3042192189 learned_addr learned my addr 172.21.15.80:0/3042192189 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:30:20.741 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.730+0000 7f4a0bfff640 1 --2- 172.21.15.80:0/3042192189 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b0d19e8c20 0x55b0d19ed720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:20.741 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.731+0000 7f4a0b7fe640 1 --2- 172.21.15.80:0/3042192189 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b0d19edd90 0x55b0d19f0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:20.741 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.731+0000 7f4a10e29640 1 --2- 172.21.15.80:0/2604464405 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f49f40a2b90 0x7f49f40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:20.741 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.731+0000 7f4a0b7fe640 1 -- 172.21.15.80:0/3042192189 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b0d19e8c20 msgr2=0x55b0d19ed720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:20.741 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.731+0000 7f4a0b7fe640 1 --2- 172.21.15.80:0/3042192189 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b0d19e8c20 0x55b0d19ed720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.742 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.731+0000 7f4a0b7fe640 1 -- 172.21.15.80:0/3042192189 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b0d19edd90 msgr2=0x55b0d19f0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:20.742 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.731+0000 7f4a0b7fe640 1 --2- 172.21.15.80:0/3042192189 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b0d19edd90 0x55b0d19f0220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.742 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.731+0000 7f4a0b7fe640 1 -- 172.21.15.80:0/3042192189 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0d1918400 con 0x55b0d19e8230 2024-04-24T23:30:20.742 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.731+0000 7f4a097fa640 1 -- 172.21.15.80:0/2604464405 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f4a040cfab0 con 0x7f4a0c101350 2024-04-24T23:30:20.742 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.731+0000 7f4a0b7fe640 1 --2- 172.21.15.80:0/3042192189 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b0d19e8230 0x55b0d19e8650 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f4a000144c0 tx=0x7f4a000097e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2115c7c83587bd0c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:20.742 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.731+0000 7f49f27fc640 1 -- 172.21.15.80:0/3042192189 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a0001a030 con 0x55b0d19e8230 2024-04-24T23:30:20.742 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.731+0000 7f4a10e29640 1 --2- 172.21.15.80:0/2604464405 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f49f40a2b90 0x7f49f40a5050 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f4a0401c000 tx=0x7f4a040a0040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:20.742 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.731+0000 7f49f27fc640 1 -- 172.21.15.80:0/3042192189 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a000140d0 con 0x55b0d19e8230 2024-04-24T23:30:20.742 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.731+0000 7f49f27fc640 1 -- 172.21.15.80:0/3042192189 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a00012480 con 0x55b0d19e8230 2024-04-24T23:30:20.742 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.732+0000 7f4a15596740 1 -- 172.21.15.80:0/3042192189 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b0d19e8230 msgr2=0x55b0d19e8650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:20.742 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.732+0000 7f4a15596740 1 --2- 172.21.15.80:0/3042192189 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b0d19e8230 0x55b0d19e8650 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f4a000144c0 tx=0x7f4a000097e0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.742 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.732+0000 7f4a15596740 1 -- 172.21.15.80:0/3042192189 shutdown_connections 2024-04-24T23:30:20.743 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.732+0000 7f4a15596740 1 --2- 172.21.15.80:0/3042192189 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b0d19edd90 0x55b0d19f0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.743 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.732+0000 7f4a15596740 1 --2- 172.21.15.80:0/3042192189 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b0d19e8c20 0x55b0d19ed720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.743 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.732+0000 7f4a15596740 1 --2- 172.21.15.80:0/3042192189 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b0d19e8230 0x55b0d19e8650 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.743 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.732+0000 7f4a15596740 1 -- 172.21.15.80:0/3042192189 >> 172.21.15.80:0/3042192189 conn(0x55b0d19e38a0 msgr2=0x55b0d19e3ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:20.743 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.732+0000 7f4a15596740 1 -- 172.21.15.80:0/3042192189 shutdown_connections 2024-04-24T23:30:20.743 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.732+0000 7f4a15596740 1 -- 172.21.15.80:0/3042192189 wait complete. 2024-04-24T23:30:20.743 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.732+0000 7f4a15596740 1 -- start start 2024-04-24T23:30:20.743 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.732+0000 7f4a15596740 1 Processor -- start 2024-04-24T23:30:20.743 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.733+0000 7f4a15596740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b0d19e8230 0x55b0d19f2240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:20.743 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.733+0000 7f4a15596740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b0d19e8c20 0x55b0d19f2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:20.743 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.733+0000 7f4a15596740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b0d19edd90 0x55b0d19f97c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:20.743 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.733+0000 7f4a15596740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x55b0d19f0ce0 con 0x55b0d19e8c20 2024-04-24T23:30:20.743 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.733+0000 7f4a15596740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55b0d1908050 con 0x55b0d19edd90 2024-04-24T23:30:20.743 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.733+0000 7f4a15596740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55b0d19f0e60 con 0x55b0d19e8230 2024-04-24T23:30:20.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.733+0000 7f4a15596740 10 client.0 _ll_register_callbacks cb 0x55b0d165fbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:30:20.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.733+0000 7f4a0bfff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b0d19e8c20 0x55b0d19f2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:20.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.733+0000 7f4a0bfff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b0d19e8c20 0x55b0d19f2780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:46870/0 (socket says 172.21.15.80:46870) 2024-04-24T23:30:20.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.733+0000 7f4a0bfff640 1 -- 172.21.15.80:0/3992861803 learned_addr learned my addr 172.21.15.80:0/3992861803 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:30:20.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.733+0000 7f4a0b7fe640 1 --2- 172.21.15.80:0/3992861803 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b0d19e8230 0x55b0d19f2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:20.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.733+0000 7f4a0b7fe640 1 --2- 172.21.15.80:0/3992861803 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b0d19edd90 0x55b0d19f97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:20.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.733+0000 7f4a0bfff640 1 -- 172.21.15.80:0/3992861803 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b0d19e8230 msgr2=0x55b0d19f2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:20.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.733+0000 7f4a0bfff640 1 --2- 172.21.15.80:0/3992861803 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b0d19e8230 0x55b0d19f2240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.733+0000 7f4a0bfff640 1 -- 172.21.15.80:0/3992861803 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b0d19edd90 msgr2=0x55b0d19f97c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:20.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.733+0000 7f4a0bfff640 1 --2- 172.21.15.80:0/3992861803 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b0d19edd90 0x55b0d19f97c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.733+0000 7f4a0bfff640 1 -- 172.21.15.80:0/3992861803 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0d18d9dc0 con 0x55b0d19e8c20 2024-04-24T23:30:20.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.733+0000 7f4a0b7fe640 1 --2- 172.21.15.80:0/3992861803 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b0d19edd90 0x55b0d19f97c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:30:20.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.733+0000 7f4a0bfff640 1 --2- 172.21.15.80:0/3992861803 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b0d19e8c20 0x55b0d19f2780 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f49fc02a930 tx=0x7f49fc032eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb8f12386db20201 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:20.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.733+0000 7f49eb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-04-24T23:30:20.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.733+0000 7f49eb7fe640 1 -- 172.21.15.80:0/3992861803 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49fc035040 con 0x55b0d19e8c20 2024-04-24T23:30:20.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.733+0000 7f4a15596740 1 -- 172.21.15.80:0/3992861803 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a000140d0 con 0x55b0d19e8c20 2024-04-24T23:30:20.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.733+0000 7f49eb7fe640 1 -- 172.21.15.80:0/3992861803 <== mon.0 v2:172.21.15.80:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f49fc004040 con 0x55b0d19e8c20 2024-04-24T23:30:20.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.734+0000 7f49eb7fe640 1 -- 172.21.15.80:0/3992861803 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f49fc04f050 con 0x55b0d19e8c20 2024-04-24T23:30:20.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.734+0000 7f4a15596740 10 client.15696 fetch_fsmap learned FSMap version 25 2024-04-24T23:30:20.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.734+0000 7f4a15596740 1 -- 172.21.15.80:0/3992861803 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b0d16603f0 con 0x55b0d19e8c20 2024-04-24T23:30:20.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.734+0000 7f49eb7fe640 1 -- 172.21.15.80:0/3992861803 <== mon.0 v2:172.21.15.80:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a000140d0 con 0x55b0d19e8c20 2024-04-24T23:30:20.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.734+0000 7f49eb7fe640 1 -- 172.21.15.80:0/3992861803 <== mon.0 v2:172.21.15.80:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f49fc004780 con 0x55b0d19e8c20 2024-04-24T23:30:20.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.734+0000 7f4a15596740 10 client.15696 fetch_fsmap finished waiting for FSMap version 25 2024-04-24T23:30:20.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.734+0000 7f4a15596740 10 client.15696 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-04-24T23:30:20.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.734+0000 7f4a15596740 20 client.15696 populate_metadata read hostname 'smithi080' 2024-04-24T23:30:20.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.734+0000 7f4a15596740 1 --2- 172.21.15.80:0/3992861803 >> [v2:172.21.15.134:6838/1743815021,v1:172.21.15.134:6839/1743815021] conn(0x55b0d19fe120 0x55b0d1a00580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:20.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.734+0000 7f4a15596740 4 client.15696 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:30:20.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.734+0000 7f4a15596740 1 -- 172.21.15.80:0/3992861803 --> [v2:172.21.15.134:6838/1743815021,v1:172.21.15.134:6839/1743815021] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b0d1959bb0 con 0x55b0d19fe120 2024-04-24T23:30:20.746 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.734+0000 7f4a0b7fe640 1 --2- 172.21.15.80:0/3992861803 >> [v2:172.21.15.134:6838/1743815021,v1:172.21.15.134:6839/1743815021] conn(0x55b0d19fe120 0x55b0d1a00580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:20.746 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.735+0000 7f4a0b7fe640 1 --2- 172.21.15.80:0/3992861803 >> [v2:172.21.15.134:6838/1743815021,v1:172.21.15.134:6839/1743815021] conn(0x55b0d19fe120 0x55b0d1a00580 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5719c84b5ef3e844 server_cookie=b6813fcbe5049d5d in_seq=0 out_seq=0 2024-04-24T23:30:20.746 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.735+0000 7f49eb7fe640 10 client.15696 ms_handle_connect on v2:172.21.15.134:6838/1743815021 2024-04-24T23:30:20.746 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.736+0000 7f49eb7fe640 1 -- 172.21.15.80:0/3992861803 <== mds.1 v2:172.21.15.134:6838/1743815021 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b0d1959bb0 con 0x55b0d19fe120 2024-04-24T23:30:20.746 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.736+0000 7f49eb7fe640 10 client.15696 handle_command_reply: tid=0 2024-04-24T23:30:20.746 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.736+0000 7f4a15596740 1 client.15696 shutdown 2024-04-24T23:30:20.746 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.736+0000 7f4a15596740 1 -- 172.21.15.80:0/3992861803 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b0d19e8c20 msgr2=0x55b0d19f2780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:20.746 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.736+0000 7f4a15596740 1 --2- 172.21.15.80:0/3992861803 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b0d19e8c20 0x55b0d19f2780 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f49fc02a930 tx=0x7f49fc032eb0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.746 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.736+0000 7f4a15596740 1 -- 172.21.15.80:0/3992861803 shutdown_connections 2024-04-24T23:30:20.746 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.736+0000 7f4a15596740 1 --2- 172.21.15.80:0/3992861803 >> [v2:172.21.15.134:6838/1743815021,v1:172.21.15.134:6839/1743815021] conn(0x55b0d19fe120 0x55b0d1a00580 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.746 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.736+0000 7f4a15596740 1 --2- 172.21.15.80:0/3992861803 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b0d19edd90 0x55b0d19f97c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.746 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.736+0000 7f49eb7fe640 0 client.15696 ms_handle_reset on v2:172.21.15.134:6838/1743815021 2024-04-24T23:30:20.746 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.736+0000 7f4a15596740 1 --2- 172.21.15.80:0/3992861803 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b0d19e8c20 0x55b0d19f2780 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.746 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.736+0000 7f4a15596740 1 --2- 172.21.15.80:0/3992861803 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b0d19e8230 0x55b0d19f2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.747 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.736+0000 7f4a15596740 1 -- 172.21.15.80:0/3992861803 >> 172.21.15.80:0/3992861803 conn(0x55b0d19e38a0 msgr2=0x55b0d19e2b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:20.747 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.736+0000 7f4a15596740 1 -- 172.21.15.80:0/3992861803 shutdown_connections 2024-04-24T23:30:20.747 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.736+0000 7f4a15596740 1 -- 172.21.15.80:0/3992861803 wait complete. 2024-04-24T23:30:20.747 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.736+0000 7f4a15596740 20 client.15696 trim_cache size 0 max 16384 2024-04-24T23:30:20.752 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.749+0000 7f4a15596740 1 Processor -- start 2024-04-24T23:30:20.752 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.749+0000 7f4a15596740 1 -- start start 2024-04-24T23:30:20.752 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.749+0000 7f4a15596740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b0d19e8230 0x55b0d19e6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:20.752 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.749+0000 7f4a15596740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b0d19e8c20 0x55b0d19e71d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:20.752 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.749+0000 7f4a15596740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b0d19edd90 0x55b0d19f9780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:20.752 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.749+0000 7f4a15596740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f4a0c1aef90 con 0x55b0d19e8230 2024-04-24T23:30:20.752 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.749+0000 7f4a15596740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4a0c1ac860 con 0x55b0d19e8c20 2024-04-24T23:30:20.752 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.749+0000 7f4a15596740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4a0c1aa6b0 con 0x55b0d19edd90 2024-04-24T23:30:20.752 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.749+0000 7f4a0b7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b0d19e8230 0x55b0d19e6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:20.752 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.750+0000 7f4a0b7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b0d19e8230 0x55b0d19e6c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:46876/0 (socket says 172.21.15.80:46876) 2024-04-24T23:30:20.752 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.750+0000 7f4a0b7fe640 1 -- 172.21.15.80:0/3917864334 learned_addr learned my addr 172.21.15.80:0/3917864334 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:30:20.752 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.750+0000 7f4a0bfff640 1 --2- 172.21.15.80:0/3917864334 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b0d19e8c20 0x55b0d19e71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:20.753 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.750+0000 7f4a0b7fe640 1 --2- 172.21.15.80:0/3917864334 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b0d19edd90 0x55b0d19f9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:20.753 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.750+0000 7f4a0b7fe640 1 -- 172.21.15.80:0/3917864334 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b0d19edd90 msgr2=0x55b0d19f9780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:20.753 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.750+0000 7f4a0b7fe640 1 --2- 172.21.15.80:0/3917864334 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b0d19edd90 0x55b0d19f9780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.753 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.750+0000 7f4a0b7fe640 1 -- 172.21.15.80:0/3917864334 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b0d19e8c20 msgr2=0x55b0d19e71d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:20.753 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.750+0000 7f4a0b7fe640 1 --2- 172.21.15.80:0/3917864334 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b0d19e8c20 0x55b0d19e71d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.753 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.750+0000 7f4a0b7fe640 1 -- 172.21.15.80:0/3917864334 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0d172a7d0 con 0x55b0d19e8230 2024-04-24T23:30:20.753 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.750+0000 7f4a0b7fe640 1 --2- 172.21.15.80:0/3917864334 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b0d19e8230 0x55b0d19e6c90 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f4a000144c0 tx=0x7f4a000097e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f99c9709489d234 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:20.753 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.750+0000 7f49f1ffb640 1 -- 172.21.15.80:0/3917864334 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a00034070 con 0x55b0d19e8230 2024-04-24T23:30:20.753 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.750+0000 7f49f1ffb640 1 -- 172.21.15.80:0/3917864334 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a00005520 con 0x55b0d19e8230 2024-04-24T23:30:20.753 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.750+0000 7f49f1ffb640 1 -- 172.21.15.80:0/3917864334 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a0002e6b0 con 0x55b0d19e8230 2024-04-24T23:30:20.753 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.751+0000 7f4a15596740 1 -- 172.21.15.80:0/3917864334 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b0d19e8230 msgr2=0x55b0d19e6c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:20.753 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.751+0000 7f4a15596740 1 --2- 172.21.15.80:0/3917864334 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b0d19e8230 0x55b0d19e6c90 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f4a000144c0 tx=0x7f4a000097e0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.754 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.751+0000 7f4a15596740 1 -- 172.21.15.80:0/3917864334 shutdown_connections 2024-04-24T23:30:20.754 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.751+0000 7f4a15596740 1 --2- 172.21.15.80:0/3917864334 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b0d19edd90 0x55b0d19f9780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.754 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.751+0000 7f4a15596740 1 --2- 172.21.15.80:0/3917864334 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b0d19e8c20 0x55b0d19e71d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.754 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.751+0000 7f4a15596740 1 --2- 172.21.15.80:0/3917864334 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b0d19e8230 0x55b0d19e6c90 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.754 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.751+0000 7f4a15596740 1 -- 172.21.15.80:0/3917864334 >> 172.21.15.80:0/3917864334 conn(0x55b0d19e38a0 msgr2=0x55b0d19e6580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:20.754 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.751+0000 7f4a15596740 1 -- 172.21.15.80:0/3917864334 shutdown_connections 2024-04-24T23:30:20.754 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.751+0000 7f4a15596740 1 -- 172.21.15.80:0/3917864334 wait complete. 2024-04-24T23:30:20.754 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.751+0000 7f4a15596740 1 -- start start 2024-04-24T23:30:20.758 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.751+0000 7f4a15596740 1 Processor -- start 2024-04-24T23:30:20.758 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.752+0000 7f4a15596740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b0d19e8230 0x55b0d19f7890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:20.758 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.752+0000 7f4a15596740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b0d19e8c20 0x55b0d19f7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:20.758 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.752+0000 7f4a15596740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b0d19edd90 0x55b0d19f8d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:20.759 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.752+0000 7f4a15596740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f4a0c1aad20 con 0x55b0d19edd90 2024-04-24T23:30:20.759 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.752+0000 7f4a15596740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4a0c1ac190 con 0x55b0d19e8230 2024-04-24T23:30:20.759 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.752+0000 7f4a15596740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4a0c1ab3f0 con 0x55b0d19e8c20 2024-04-24T23:30:20.759 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.752+0000 7f4a15596740 10 client.0 _ll_register_callbacks cb 0x7f49fc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:30:20.759 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.752+0000 7f4a0b7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b0d19edd90 0x55b0d19f8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:20.759 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.752+0000 7f4a0b7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b0d19edd90 0x55b0d19f8d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:46882/0 (socket says 172.21.15.80:46882) 2024-04-24T23:30:20.759 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.752+0000 7f4a0b7fe640 1 -- 172.21.15.80:0/2355447530 learned_addr learned my addr 172.21.15.80:0/2355447530 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:30:20.759 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.752+0000 7f4a0b7fe640 1 --2- 172.21.15.80:0/2355447530 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b0d19e8230 0x55b0d19f7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:20.759 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.752+0000 7f4a0bfff640 1 --2- 172.21.15.80:0/2355447530 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b0d19e8c20 0x55b0d19f7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:20.759 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.752+0000 7f4a0b7fe640 1 -- 172.21.15.80:0/2355447530 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b0d19e8c20 msgr2=0x55b0d19f7dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:20.759 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.752+0000 7f4a0b7fe640 1 --2- 172.21.15.80:0/2355447530 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b0d19e8c20 0x55b0d19f7dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.759 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.752+0000 7f4a0b7fe640 1 -- 172.21.15.80:0/2355447530 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b0d19e8230 msgr2=0x55b0d19f7890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:20.759 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.752+0000 7f4a0b7fe640 1 --2- 172.21.15.80:0/2355447530 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b0d19e8230 0x55b0d19f7890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.759 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.752+0000 7f4a0b7fe640 1 -- 172.21.15.80:0/2355447530 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0d19f9360 con 0x55b0d19edd90 2024-04-24T23:30:20.760 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.752+0000 7f4a0bfff640 1 --2- 172.21.15.80:0/2355447530 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b0d19e8c20 0x55b0d19f7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:30:20.760 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.752+0000 7f4a0b7fe640 1 --2- 172.21.15.80:0/2355447530 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b0d19edd90 0x55b0d19f8d70 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f4a00009d20 tx=0x7f4a0002e3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d1b650bcea1af29 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:20.760 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.752+0000 7f49eb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-04-24T23:30:20.760 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.753+0000 7f49eb7fe640 1 -- 172.21.15.80:0/2355447530 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a00034040 con 0x55b0d19edd90 2024-04-24T23:30:20.760 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.753+0000 7f4a15596740 1 -- 172.21.15.80:0/2355447530 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a00005520 con 0x55b0d19edd90 2024-04-24T23:30:20.760 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.753+0000 7f49eb7fe640 1 -- 172.21.15.80:0/2355447530 <== mon.0 v2:172.21.15.80:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f4a0002e830 con 0x55b0d19edd90 2024-04-24T23:30:20.760 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.753+0000 7f49eb7fe640 1 -- 172.21.15.80:0/2355447530 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4a0002ea70 con 0x55b0d19edd90 2024-04-24T23:30:20.760 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.753+0000 7f4a15596740 10 client.15702 fetch_fsmap learned FSMap version 25 2024-04-24T23:30:20.760 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.753+0000 7f4a15596740 1 -- 172.21.15.80:0/2355447530 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b0d1a64fd0 con 0x55b0d19edd90 2024-04-24T23:30:20.760 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.753+0000 7f49eb7fe640 1 -- 172.21.15.80:0/2355447530 <== mon.0 v2:172.21.15.80:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a00005520 con 0x55b0d19edd90 2024-04-24T23:30:20.760 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.753+0000 7f49eb7fe640 1 -- 172.21.15.80:0/2355447530 <== mon.0 v2:172.21.15.80:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f4a0003a040 con 0x55b0d19edd90 2024-04-24T23:30:20.760 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.753+0000 7f4a15596740 10 client.15702 fetch_fsmap finished waiting for FSMap version 25 2024-04-24T23:30:20.760 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.753+0000 7f4a15596740 10 client.15702 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-04-24T23:30:20.760 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.753+0000 7f4a15596740 20 client.15702 populate_metadata read hostname 'smithi080' 2024-04-24T23:30:20.761 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.753+0000 7f4a15596740 1 --2- 172.21.15.80:0/2355447530 >> [v2:172.21.15.134:6838/1743815021,v1:172.21.15.134:6839/1743815021] conn(0x55b0d1a65660 0x55b0d1a67a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:20.761 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.753+0000 7f4a15596740 4 client.15702 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:30:20.761 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.753+0000 7f4a15596740 1 -- 172.21.15.80:0/2355447530 --> [v2:172.21.15.134:6838/1743815021,v1:172.21.15.134:6839/1743815021] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b0d1953f00 con 0x55b0d1a65660 2024-04-24T23:30:20.761 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.753+0000 7f4a0bfff640 1 --2- 172.21.15.80:0/2355447530 >> [v2:172.21.15.134:6838/1743815021,v1:172.21.15.134:6839/1743815021] conn(0x55b0d1a65660 0x55b0d1a67a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:20.761 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.754+0000 7f4a0bfff640 1 --2- 172.21.15.80:0/2355447530 >> [v2:172.21.15.134:6838/1743815021,v1:172.21.15.134:6839/1743815021] conn(0x55b0d1a65660 0x55b0d1a67a80 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d32acf59f27e671c server_cookie=3205aa67dbd730ac in_seq=0 out_seq=0 2024-04-24T23:30:20.761 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.754+0000 7f49eb7fe640 10 client.15702 ms_handle_connect on v2:172.21.15.134:6838/1743815021 2024-04-24T23:30:20.761 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.754+0000 7f49eb7fe640 1 -- 172.21.15.80:0/2355447530 <== mds.1 v2:172.21.15.134:6838/1743815021 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b0d1953f00 con 0x55b0d1a65660 2024-04-24T23:30:20.761 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.754+0000 7f49eb7fe640 10 client.15702 handle_command_reply: tid=0 2024-04-24T23:30:20.761 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.754+0000 7f4a15596740 1 client.15702 shutdown 2024-04-24T23:30:20.761 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.754+0000 7f4a15596740 1 -- 172.21.15.80:0/2355447530 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b0d19edd90 msgr2=0x55b0d19f8d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:20.761 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.754+0000 7f4a15596740 1 --2- 172.21.15.80:0/2355447530 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b0d19edd90 0x55b0d19f8d70 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f4a00009d20 tx=0x7f4a0002e3f0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.761 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.754+0000 7f4a15596740 1 -- 172.21.15.80:0/2355447530 shutdown_connections 2024-04-24T23:30:20.761 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.754+0000 7f4a15596740 1 --2- 172.21.15.80:0/2355447530 >> [v2:172.21.15.134:6838/1743815021,v1:172.21.15.134:6839/1743815021] conn(0x55b0d1a65660 0x55b0d1a67a80 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.761 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.754+0000 7f4a15596740 1 --2- 172.21.15.80:0/2355447530 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b0d19edd90 0x55b0d19f8d70 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.762 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.754+0000 7f4a15596740 1 --2- 172.21.15.80:0/2355447530 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b0d19e8c20 0x55b0d19f7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.762 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.754+0000 7f49eb7fe640 0 client.15702 ms_handle_reset on v2:172.21.15.134:6838/1743815021 2024-04-24T23:30:20.762 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.754+0000 7f4a15596740 1 --2- 172.21.15.80:0/2355447530 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b0d19e8230 0x55b0d19f7890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.762 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.754+0000 7f4a15596740 1 -- 172.21.15.80:0/2355447530 >> 172.21.15.80:0/2355447530 conn(0x55b0d19e38a0 msgr2=0x55b0d19e6e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:20.762 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.755+0000 7f4a15596740 1 -- 172.21.15.80:0/2355447530 shutdown_connections 2024-04-24T23:30:20.762 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.755+0000 7f4a15596740 1 -- 172.21.15.80:0/2355447530 wait complete. 2024-04-24T23:30:20.762 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.755+0000 7f4a15596740 20 client.15702 trim_cache size 0 max 16384 2024-04-24T23:30:20.762 INFO:teuthology.orchestra.run.smithi080.stdout:[] 2024-04-24T23:30:20.763 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.755+0000 7f4a1288c640 1 -- 172.21.15.80:0/2604464405 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f49f40a2b90 msgr2=0x7f49f40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:20.763 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.755+0000 7f4a1288c640 1 --2- 172.21.15.80:0/2604464405 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f49f40a2b90 0x7f49f40a5050 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f4a0401c000 tx=0x7f4a040a0040 comp rx=0 tx=0).stop 2024-04-24T23:30:20.763 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.755+0000 7f4a1288c640 1 -- 172.21.15.80:0/2604464405 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4a0c101350 msgr2=0x7f4a0c1a1260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:20.763 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.755+0000 7f4a1288c640 1 --2- 172.21.15.80:0/2604464405 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4a0c101350 0x7f4a0c1a1260 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f4a0400eb10 tx=0x7f4a0400c2d0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.763 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.755+0000 7f4a1288c640 1 -- 172.21.15.80:0/2604464405 shutdown_connections 2024-04-24T23:30:20.764 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.755+0000 7f4a1288c640 1 --2- 172.21.15.80:0/2604464405 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f49f40a2b90 0x7f49f40a5050 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.764 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.755+0000 7f4a1288c640 1 --2- 172.21.15.80:0/2604464405 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4a0c101350 0x7f4a0c1a1260 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.764 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.755+0000 7f4a1288c640 1 --2- 172.21.15.80:0/2604464405 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a0c1009b0 0x7f4a0c19a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.764 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.755+0000 7f4a1288c640 1 --2- 172.21.15.80:0/2604464405 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4a0c100050 0x7f4a0c199cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.764 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.755+0000 7f4a1288c640 1 -- 172.21.15.80:0/2604464405 >> 172.21.15.80:0/2604464405 conn(0x7f4a0c0fbbf0 msgr2=0x7f4a0c0fd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:20.764 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.755+0000 7f4a1288c640 1 -- 172.21.15.80:0/2604464405 shutdown_connections 2024-04-24T23:30:20.764 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.755+0000 7f4a1288c640 1 -- 172.21.15.80:0/2604464405 wait complete. 2024-04-24T23:30:20.767 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T23:30:20.871 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.868+0000 7f69ec9e9640 1 -- 172.21.15.80:0/3631606848 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f69e8102200 msgr2=0x7f69e81045f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:20.871 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.868+0000 7f69ec9e9640 1 --2- 172.21.15.80:0/3631606848 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f69e8102200 0x7f69e81045f0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f69d8009900 tx=0x7f69d802a410 comp rx=0 tx=0).stop 2024-04-24T23:30:20.872 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.868+0000 7f69ec9e9640 1 -- 172.21.15.80:0/3631606848 shutdown_connections 2024-04-24T23:30:20.872 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.868+0000 7f69ec9e9640 1 --2- 172.21.15.80:0/3631606848 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f69e8107460 0x7f69e81098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.872 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.868+0000 7f69ec9e9640 1 --2- 172.21.15.80:0/3631606848 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f69e8104b30 0x7f69e8106f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.872 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.868+0000 7f69ec9e9640 1 --2- 172.21.15.80:0/3631606848 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f69e8102200 0x7f69e81045f0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.872 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.868+0000 7f69ec9e9640 1 -- 172.21.15.80:0/3631606848 >> 172.21.15.80:0/3631606848 conn(0x7f69e80fbbf0 msgr2=0x7f69e80fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:20.872 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.868+0000 7f69ec9e9640 1 -- 172.21.15.80:0/3631606848 shutdown_connections 2024-04-24T23:30:20.872 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.868+0000 7f69ec9e9640 1 -- 172.21.15.80:0/3631606848 wait complete. 2024-04-24T23:30:20.872 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.869+0000 7f69ec9e9640 1 Processor -- start 2024-04-24T23:30:20.872 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.869+0000 7f69ec9e9640 1 -- start start 2024-04-24T23:30:20.872 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.869+0000 7f69ec9e9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f69e8102200 0x7f69e8197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:20.872 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.869+0000 7f69ec9e9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f69e8104b30 0x7f69e8198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:20.872 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.869+0000 7f69ec9e9640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f69e8107460 0x7f69e819f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:20.872 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.869+0000 7f69ec9e9640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f69e810d420 con 0x7f69e8107460 2024-04-24T23:30:20.873 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.869+0000 7f69ec9e9640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f69e810d2a0 con 0x7f69e8104b30 2024-04-24T23:30:20.873 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.869+0000 7f69ec9e9640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f69e810d5a0 con 0x7f69e8102200 2024-04-24T23:30:20.873 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.869+0000 7f69e6d9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f69e8107460 0x7f69e819f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:20.873 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.869+0000 7f69e6d9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f69e8107460 0x7f69e819f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:46896/0 (socket says 172.21.15.80:46896) 2024-04-24T23:30:20.873 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.869+0000 7f69e6d9d640 1 -- 172.21.15.80:0/2484790016 learned_addr learned my addr 172.21.15.80:0/2484790016 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:30:20.873 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.869+0000 7f69e659c640 1 --2- 172.21.15.80:0/2484790016 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f69e8102200 0x7f69e8197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:20.873 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.869+0000 7f69e5d9b640 1 --2- 172.21.15.80:0/2484790016 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f69e8104b30 0x7f69e8198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:20.873 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.869+0000 7f69e6d9d640 1 -- 172.21.15.80:0/2484790016 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f69e8102200 msgr2=0x7f69e8197af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:20.873 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.869+0000 7f69e6d9d640 1 --2- 172.21.15.80:0/2484790016 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f69e8102200 0x7f69e8197af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.873 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.869+0000 7f69e6d9d640 1 -- 172.21.15.80:0/2484790016 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f69e8104b30 msgr2=0x7f69e8198030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:20.873 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.869+0000 7f69e6d9d640 1 --2- 172.21.15.80:0/2484790016 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f69e8104b30 0x7f69e8198030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.873 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.869+0000 7f69e6d9d640 1 -- 172.21.15.80:0/2484790016 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69e819f710 con 0x7f69e8107460 2024-04-24T23:30:20.873 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.869+0000 7f69e6d9d640 1 --2- 172.21.15.80:0/2484790016 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f69e8107460 0x7f69e819f0a0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f69e000eaa0 tx=0x7f69e000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:20.874 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.870+0000 7f69d77fe640 1 -- 172.21.15.80:0/2484790016 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69e000cdb0 con 0x7f69e8107460 2024-04-24T23:30:20.874 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.870+0000 7f69ec9e9640 1 -- 172.21.15.80:0/2484790016 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69e819fa00 con 0x7f69e8107460 2024-04-24T23:30:20.874 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.870+0000 7f69d77fe640 1 -- 172.21.15.80:0/2484790016 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69e0005020 con 0x7f69e8107460 2024-04-24T23:30:20.874 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.870+0000 7f69d77fe640 1 -- 172.21.15.80:0/2484790016 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69e00116d0 con 0x7f69e8107460 2024-04-24T23:30:20.874 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.870+0000 7f69ec9e9640 1 -- 172.21.15.80:0/2484790016 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69e819ffa0 con 0x7f69e8107460 2024-04-24T23:30:20.881 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.872+0000 7f69d77fe640 1 -- 172.21.15.80:0/2484790016 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f69e00054d0 con 0x7f69e8107460 2024-04-24T23:30:20.881 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.872+0000 7f69ef7cb740 1 Processor -- start 2024-04-24T23:30:20.882 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.872+0000 7f69ef7cb740 1 -- start start 2024-04-24T23:30:20.882 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.873+0000 7f69ef7cb740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55baa1f71640 0x55baa1f71a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:20.882 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.873+0000 7f69ef7cb740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55baa1f72030 0x55baa1f76b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:20.882 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.873+0000 7f69ef7cb740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55baa1f771a0 0x55baa1f79630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:20.882 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.873+0000 7f69ef7cb740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x55baa1ed1670 con 0x55baa1f72030 2024-04-24T23:30:20.882 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.873+0000 7f69ef7cb740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55baa1e79970 con 0x55baa1f71640 2024-04-24T23:30:20.882 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.873+0000 7f69ef7cb740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55baa1ecd630 con 0x55baa1f771a0 2024-04-24T23:30:20.882 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.873+0000 7f69e659c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55baa1f72030 0x55baa1f76b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:20.882 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.873+0000 7f69d77fe640 1 --2- 172.21.15.80:0/2484790016 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f69d00a2b90 0x7f69d00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:20.882 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.873+0000 7f69e659c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55baa1f72030 0x55baa1f76b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:46906/0 (socket says 172.21.15.80:46906) 2024-04-24T23:30:20.882 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.873+0000 7f69e659c640 1 -- 172.21.15.80:0/1078676360 learned_addr learned my addr 172.21.15.80:0/1078676360 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:30:20.882 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.873+0000 7f69e6d9d640 1 --2- 172.21.15.80:0/2484790016 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f69d00a2b90 0x7f69d00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:20.883 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.873+0000 7f69e5d9b640 1 --2- 172.21.15.80:0/1078676360 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55baa1f71640 0x55baa1f71a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:20.883 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.873+0000 7f69e5d9b640 1 --2- 172.21.15.80:0/1078676360 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55baa1f771a0 0x55baa1f79630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:20.883 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.873+0000 7f69d77fe640 1 -- 172.21.15.80:0/2484790016 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f69e00cf6e0 con 0x7f69e8107460 2024-04-24T23:30:20.883 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.873+0000 7f69e659c640 1 -- 172.21.15.80:0/1078676360 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55baa1f771a0 msgr2=0x55baa1f79630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:20.883 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.873+0000 7f69e659c640 1 --2- 172.21.15.80:0/1078676360 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55baa1f771a0 0x55baa1f79630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.883 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.873+0000 7f69e659c640 1 -- 172.21.15.80:0/1078676360 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55baa1f71640 msgr2=0x55baa1f71a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:20.883 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.873+0000 7f69e659c640 1 --2- 172.21.15.80:0/1078676360 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55baa1f71640 0x55baa1f71a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.883 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.873+0000 7f69e659c640 1 -- 172.21.15.80:0/1078676360 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55baa1e9d400 con 0x55baa1f72030 2024-04-24T23:30:20.883 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.873+0000 7f69e5d9b640 1 --2- 172.21.15.80:0/1078676360 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55baa1f71640 0x55baa1f71a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:30:20.883 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.873+0000 7f69e6d9d640 1 --2- 172.21.15.80:0/2484790016 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f69d00a2b90 0x7f69d00a5050 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f69e001c000 tx=0x7f69e006ba20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:20.883 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.873+0000 7f69e659c640 1 --2- 172.21.15.80:0/1078676360 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55baa1f72030 0x55baa1f76b30 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f69d802a930 tx=0x7f69d8032cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af8a5e610c4dac4a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:20.883 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.874+0000 7f69d4ff9640 1 -- 172.21.15.80:0/1078676360 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69d803f070 con 0x55baa1f72030 2024-04-24T23:30:20.883 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.874+0000 7f69d4ff9640 1 -- 172.21.15.80:0/1078676360 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69d80318b0 con 0x55baa1f72030 2024-04-24T23:30:20.883 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.874+0000 7f69d4ff9640 1 -- 172.21.15.80:0/1078676360 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69d8035070 con 0x55baa1f72030 2024-04-24T23:30:20.884 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.874+0000 7f69ef7cb740 1 -- 172.21.15.80:0/1078676360 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55baa1f72030 msgr2=0x55baa1f76b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:20.884 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.874+0000 7f69ef7cb740 1 --2- 172.21.15.80:0/1078676360 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55baa1f72030 0x55baa1f76b30 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f69d802a930 tx=0x7f69d8032cb0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.884 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.874+0000 7f69ef7cb740 1 -- 172.21.15.80:0/1078676360 shutdown_connections 2024-04-24T23:30:20.884 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.874+0000 7f69ef7cb740 1 --2- 172.21.15.80:0/1078676360 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55baa1f771a0 0x55baa1f79630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.884 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.874+0000 7f69ef7cb740 1 --2- 172.21.15.80:0/1078676360 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55baa1f72030 0x55baa1f76b30 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.884 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.874+0000 7f69ef7cb740 1 --2- 172.21.15.80:0/1078676360 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55baa1f71640 0x55baa1f71a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.884 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.874+0000 7f69ef7cb740 1 -- 172.21.15.80:0/1078676360 >> 172.21.15.80:0/1078676360 conn(0x55baa1f688a0 msgr2=0x55baa1f68ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:20.884 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.875+0000 7f69ef7cb740 1 -- 172.21.15.80:0/1078676360 shutdown_connections 2024-04-24T23:30:20.884 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.875+0000 7f69ef7cb740 1 -- 172.21.15.80:0/1078676360 wait complete. 2024-04-24T23:30:20.884 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.875+0000 7f69ef7cb740 1 -- start start 2024-04-24T23:30:20.884 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.875+0000 7f69ef7cb740 1 Processor -- start 2024-04-24T23:30:20.884 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.875+0000 7f69ef7cb740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55baa1f71640 0x55baa1f7b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:20.884 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.875+0000 7f69ef7cb740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55baa1f72030 0x55baa1f7bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:20.884 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.875+0000 7f69ef7cb740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55baa1f771a0 0x55baa1f82cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:20.885 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.875+0000 7f69ef7cb740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x55baa1f7a0f0 con 0x55baa1f771a0 2024-04-24T23:30:20.885 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.875+0000 7f69ef7cb740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55baa1e8d050 con 0x55baa1f71640 2024-04-24T23:30:20.885 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.875+0000 7f69ef7cb740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55baa1f7a270 con 0x55baa1f72030 2024-04-24T23:30:20.885 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.875+0000 7f69ef7cb740 10 client.0 _ll_register_callbacks cb 0x55baa1be4be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:30:20.885 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.875+0000 7f69e5d9b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55baa1f771a0 0x55baa1f82cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:20.885 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.875+0000 7f69e5d9b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55baa1f771a0 0x55baa1f82cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:46910/0 (socket says 172.21.15.80:46910) 2024-04-24T23:30:20.885 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.875+0000 7f69e5d9b640 1 -- 172.21.15.80:0/4216565120 learned_addr learned my addr 172.21.15.80:0/4216565120 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:30:20.885 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.875+0000 7f69e5d9b640 1 --2- 172.21.15.80:0/4216565120 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55baa1f71640 0x55baa1f7b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:20.885 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.875+0000 7f69e659c640 1 --2- 172.21.15.80:0/4216565120 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55baa1f72030 0x55baa1f7bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:20.885 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.875+0000 7f69e5d9b640 1 -- 172.21.15.80:0/4216565120 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55baa1f72030 msgr2=0x55baa1f7bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:20.885 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.875+0000 7f69e5d9b640 1 --2- 172.21.15.80:0/4216565120 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55baa1f72030 0x55baa1f7bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.885 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.875+0000 7f69e5d9b640 1 -- 172.21.15.80:0/4216565120 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55baa1f71640 msgr2=0x55baa1f7b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:20.885 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.875+0000 7f69e5d9b640 1 --2- 172.21.15.80:0/4216565120 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55baa1f71640 0x55baa1f7b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.886 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.875+0000 7f69e5d9b640 1 -- 172.21.15.80:0/4216565120 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55baa1e5edc0 con 0x55baa1f771a0 2024-04-24T23:30:20.886 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.876+0000 7f69e5d9b640 1 --2- 172.21.15.80:0/4216565120 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55baa1f771a0 0x55baa1f82cd0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f69dc014ec0 tx=0x7f69dc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de459c5fdf27c83c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:20.886 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.876+0000 7f69cdffb640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-04-24T23:30:20.886 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.876+0000 7f69cdffb640 1 -- 172.21.15.80:0/4216565120 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69dc00dea0 con 0x55baa1f771a0 2024-04-24T23:30:20.886 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.876+0000 7f69ef7cb740 1 -- 172.21.15.80:0/4216565120 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69d80318b0 con 0x55baa1f771a0 2024-04-24T23:30:20.886 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.876+0000 7f69cdffb640 1 -- 172.21.15.80:0/4216565120 <== mon.0 v2:172.21.15.80:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f69dc009040 con 0x55baa1f771a0 2024-04-24T23:30:20.886 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.876+0000 7f69cdffb640 1 -- 172.21.15.80:0/4216565120 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f69dc029050 con 0x55baa1f771a0 2024-04-24T23:30:20.886 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.876+0000 7f69ef7cb740 10 client.15714 fetch_fsmap learned FSMap version 25 2024-04-24T23:30:20.886 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.876+0000 7f69ef7cb740 1 -- 172.21.15.80:0/4216565120 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55baa1be53f0 con 0x55baa1f771a0 2024-04-24T23:30:20.886 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.876+0000 7f69cdffb640 1 -- 172.21.15.80:0/4216565120 <== mon.0 v2:172.21.15.80:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69d80318b0 con 0x55baa1f771a0 2024-04-24T23:30:20.886 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.876+0000 7f69cdffb640 1 -- 172.21.15.80:0/4216565120 <== mon.0 v2:172.21.15.80:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f69dc020070 con 0x55baa1f771a0 2024-04-24T23:30:20.886 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.876+0000 7f69ef7cb740 10 client.15714 fetch_fsmap finished waiting for FSMap version 25 2024-04-24T23:30:20.886 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.876+0000 7f69ef7cb740 10 client.15714 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:30:20.886 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.876+0000 7f69ef7cb740 20 client.15714 populate_metadata read hostname 'smithi080' 2024-04-24T23:30:20.887 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.876+0000 7f69ef7cb740 1 --2- 172.21.15.80:0/4216565120 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x55baa1f87550 0x55baa1f899c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:20.887 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.876+0000 7f69ef7cb740 4 client.15714 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:30:20.887 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.876+0000 7f69ef7cb740 1 -- 172.21.15.80:0/4216565120 --> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55baa1edebb0 con 0x55baa1f87550 2024-04-24T23:30:20.887 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.877+0000 7f69e659c640 1 --2- 172.21.15.80:0/4216565120 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x55baa1f87550 0x55baa1f899c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:20.887 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.877+0000 7f69cdffb640 10 client.15714 ms_handle_connect on v2:172.21.15.148:6838/2952757594 2024-04-24T23:30:20.887 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.877+0000 7f69e659c640 1 --2- 172.21.15.80:0/4216565120 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x55baa1f87550 0x55baa1f899c0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e28c8b4167e63e4e server_cookie=d3ba051f64680346 in_seq=0 out_seq=0 2024-04-24T23:30:20.887 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.878+0000 7f69cdffb640 1 -- 172.21.15.80:0/4216565120 <== mds.0 v2:172.21.15.148:6838/2952757594 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55baa1edebb0 con 0x55baa1f87550 2024-04-24T23:30:20.887 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.878+0000 7f69cdffb640 10 client.15714 handle_command_reply: tid=0 2024-04-24T23:30:20.887 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.878+0000 7f69ef7cb740 1 client.15714 shutdown 2024-04-24T23:30:20.887 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.878+0000 7f69ef7cb740 1 -- 172.21.15.80:0/4216565120 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55baa1f771a0 msgr2=0x55baa1f82cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:20.887 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.878+0000 7f69ef7cb740 1 --2- 172.21.15.80:0/4216565120 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55baa1f771a0 0x55baa1f82cd0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f69dc014ec0 tx=0x7f69dc004730 comp rx=0 tx=0).stop 2024-04-24T23:30:20.887 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.878+0000 7f69ef7cb740 1 -- 172.21.15.80:0/4216565120 shutdown_connections 2024-04-24T23:30:20.887 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.878+0000 7f69ef7cb740 1 --2- 172.21.15.80:0/4216565120 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x55baa1f87550 0x55baa1f899c0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.887 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.878+0000 7f69ef7cb740 1 --2- 172.21.15.80:0/4216565120 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55baa1f771a0 0x55baa1f82cd0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.888 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.878+0000 7f69ef7cb740 1 --2- 172.21.15.80:0/4216565120 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55baa1f72030 0x55baa1f7bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.888 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.878+0000 7f69ef7cb740 1 --2- 172.21.15.80:0/4216565120 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55baa1f71640 0x55baa1f7b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.888 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.878+0000 7f69ef7cb740 1 -- 172.21.15.80:0/4216565120 >> 172.21.15.80:0/4216565120 conn(0x55baa1f688a0 msgr2=0x55baa1f67b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:20.888 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.878+0000 7f69cdffb640 0 client.15714 ms_handle_reset on v2:172.21.15.148:6838/2952757594 2024-04-24T23:30:20.888 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.878+0000 7f69ef7cb740 1 -- 172.21.15.80:0/4216565120 shutdown_connections 2024-04-24T23:30:20.888 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.878+0000 7f69ef7cb740 1 -- 172.21.15.80:0/4216565120 wait complete. 2024-04-24T23:30:20.888 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.878+0000 7f69ef7cb740 20 client.15714 trim_cache size 0 max 16384 2024-04-24T23:30:20.894 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.893+0000 7f69ef7cb740 1 Processor -- start 2024-04-24T23:30:20.895 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.893+0000 7f69ef7cb740 1 -- start start 2024-04-24T23:30:20.895 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.893+0000 7f69ef7cb740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55baa1f71640 0x55baa1f89560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:20.895 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.893+0000 7f69ef7cb740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55baa1f72030 0x55baa1f82c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:20.895 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.893+0000 7f69ef7cb740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55baa1f771a0 0x55baa1f72ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:20.895 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.893+0000 7f69ef7cb740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f69e81aac20 con 0x55baa1f71640 2024-04-24T23:30:20.895 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.893+0000 7f69ef7cb740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f69e81a84f0 con 0x55baa1f72030 2024-04-24T23:30:20.896 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.893+0000 7f69ef7cb740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f69e81a6340 con 0x55baa1f771a0 2024-04-24T23:30:20.896 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.893+0000 7f69e5d9b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55baa1f71640 0x55baa1f89560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:20.896 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.893+0000 7f69e5d9b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55baa1f71640 0x55baa1f89560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:46918/0 (socket says 172.21.15.80:46918) 2024-04-24T23:30:20.896 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.893+0000 7f69e5d9b640 1 -- 172.21.15.80:0/67683648 learned_addr learned my addr 172.21.15.80:0/67683648 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:30:20.896 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.893+0000 7f69e659c640 1 --2- 172.21.15.80:0/67683648 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55baa1f72030 0x55baa1f82c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:20.896 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.893+0000 7f69e5d9b640 1 -- 172.21.15.80:0/67683648 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55baa1f771a0 msgr2=0x55baa1f72ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:20.896 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.893+0000 7f69e5d9b640 1 --2- 172.21.15.80:0/67683648 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55baa1f771a0 0x55baa1f72ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.896 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.893+0000 7f69e5d9b640 1 -- 172.21.15.80:0/67683648 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55baa1f72030 msgr2=0x55baa1f82c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:20.896 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.893+0000 7f69e5d9b640 1 --2- 172.21.15.80:0/67683648 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55baa1f72030 0x55baa1f82c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.896 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.893+0000 7f69e5d9b640 1 -- 172.21.15.80:0/67683648 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55baa1f89d30 con 0x55baa1f71640 2024-04-24T23:30:20.896 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.893+0000 7f69e659c640 1 --2- 172.21.15.80:0/67683648 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55baa1f72030 0x55baa1f82c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:30:20.896 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.893+0000 7f69e5d9b640 1 --2- 172.21.15.80:0/67683648 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55baa1f71640 0x55baa1f89560 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f69dc004700 tx=0x7f69dc009d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94995246e96e817e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:20.896 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.894+0000 7f69cffff640 1 -- 172.21.15.80:0/67683648 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69dc031030 con 0x55baa1f71640 2024-04-24T23:30:20.897 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.894+0000 7f69cffff640 1 -- 172.21.15.80:0/67683648 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69dc009980 con 0x55baa1f71640 2024-04-24T23:30:20.897 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.894+0000 7f69cffff640 1 -- 172.21.15.80:0/67683648 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69dc012870 con 0x55baa1f71640 2024-04-24T23:30:20.901 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.894+0000 7f69ef7cb740 1 -- 172.21.15.80:0/67683648 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55baa1f71640 msgr2=0x55baa1f89560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:20.901 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.894+0000 7f69ef7cb740 1 --2- 172.21.15.80:0/67683648 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55baa1f71640 0x55baa1f89560 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f69dc004700 tx=0x7f69dc009d20 comp rx=0 tx=0).stop 2024-04-24T23:30:20.901 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.895+0000 7f69ef7cb740 1 -- 172.21.15.80:0/67683648 shutdown_connections 2024-04-24T23:30:20.901 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.895+0000 7f69ef7cb740 1 --2- 172.21.15.80:0/67683648 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55baa1f771a0 0x55baa1f72ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.901 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.895+0000 7f69ef7cb740 1 --2- 172.21.15.80:0/67683648 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55baa1f72030 0x55baa1f82c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.902 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.895+0000 7f69ef7cb740 1 --2- 172.21.15.80:0/67683648 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55baa1f71640 0x55baa1f89560 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.902 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.895+0000 7f69ef7cb740 1 -- 172.21.15.80:0/67683648 >> 172.21.15.80:0/67683648 conn(0x55baa1f688a0 msgr2=0x55baa1f67b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:20.902 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.895+0000 7f69ef7cb740 1 -- 172.21.15.80:0/67683648 shutdown_connections 2024-04-24T23:30:20.902 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.895+0000 7f69ef7cb740 1 -- 172.21.15.80:0/67683648 wait complete. 2024-04-24T23:30:20.902 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.895+0000 7f69ef7cb740 1 -- start start 2024-04-24T23:30:20.902 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.895+0000 7f69ef7cb740 1 Processor -- start 2024-04-24T23:30:20.902 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.895+0000 7f69ef7cb740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55baa1f71640 0x55baa1f805f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:20.902 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.895+0000 7f69ef7cb740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55baa1f72030 0x55baa1f80b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:20.902 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.895+0000 7f69ef7cb740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55baa1f771a0 0x55baa1f81aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:20.902 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.895+0000 7f69ef7cb740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f69e81a69b0 con 0x55baa1f771a0 2024-04-24T23:30:20.902 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.895+0000 7f69ef7cb740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f69e81a7e20 con 0x55baa1f71640 2024-04-24T23:30:20.902 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.895+0000 7f69ef7cb740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f69e81a7080 con 0x55baa1f72030 2024-04-24T23:30:20.903 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.895+0000 7f69ef7cb740 10 client.0 _ll_register_callbacks cb 0x7f69dc014ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:30:20.903 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.895+0000 7f69e5d9b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55baa1f771a0 0x55baa1f81aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:20.903 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.895+0000 7f69e5d9b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55baa1f771a0 0x55baa1f81aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:46922/0 (socket says 172.21.15.80:46922) 2024-04-24T23:30:20.903 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.895+0000 7f69e5d9b640 1 -- 172.21.15.80:0/2570777926 learned_addr learned my addr 172.21.15.80:0/2570777926 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:30:20.903 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.896+0000 7f69e5d9b640 1 --2- 172.21.15.80:0/2570777926 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55baa1f71640 0x55baa1f805f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:20.903 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.896+0000 7f69e659c640 1 --2- 172.21.15.80:0/2570777926 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55baa1f72030 0x55baa1f80b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:20.903 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.896+0000 7f69e5d9b640 1 -- 172.21.15.80:0/2570777926 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55baa1f72030 msgr2=0x55baa1f80b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:20.903 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.896+0000 7f69e5d9b640 1 --2- 172.21.15.80:0/2570777926 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55baa1f72030 0x55baa1f80b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.903 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.896+0000 7f69e5d9b640 1 -- 172.21.15.80:0/2570777926 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55baa1f71640 msgr2=0x55baa1f805f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:20.903 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.896+0000 7f69e5d9b640 1 --2- 172.21.15.80:0/2570777926 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55baa1f71640 0x55baa1f805f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.903 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.896+0000 7f69e5d9b640 1 -- 172.21.15.80:0/2570777926 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55baa1caf7d0 con 0x55baa1f771a0 2024-04-24T23:30:20.903 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.896+0000 7f69e5d9b640 1 --2- 172.21.15.80:0/2570777926 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55baa1f771a0 0x55baa1f81aa0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f69dc006010 tx=0x7f69dc005970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc243a5aac58e19b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:20.903 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.896+0000 7f69cdffb640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-04-24T23:30:20.904 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.896+0000 7f69cdffb640 1 -- 172.21.15.80:0/2570777926 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69dc031040 con 0x55baa1f771a0 2024-04-24T23:30:20.904 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.896+0000 7f69ef7cb740 1 -- 172.21.15.80:0/2570777926 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69dc009980 con 0x55baa1f771a0 2024-04-24T23:30:20.904 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.896+0000 7f69cdffb640 1 -- 172.21.15.80:0/2570777926 <== mon.0 v2:172.21.15.80:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f69dc0127f0 con 0x55baa1f771a0 2024-04-24T23:30:20.904 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.896+0000 7f69cdffb640 1 -- 172.21.15.80:0/2570777926 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f69dc005020 con 0x55baa1f771a0 2024-04-24T23:30:20.904 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.896+0000 7f69ef7cb740 10 client.15720 fetch_fsmap learned FSMap version 25 2024-04-24T23:30:20.904 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.896+0000 7f69ef7cb740 1 -- 172.21.15.80:0/2570777926 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55baa1fedb00 con 0x55baa1f771a0 2024-04-24T23:30:20.904 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.896+0000 7f69cdffb640 1 -- 172.21.15.80:0/2570777926 <== mon.0 v2:172.21.15.80:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69dc009980 con 0x55baa1f771a0 2024-04-24T23:30:20.904 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.896+0000 7f69cdffb640 1 -- 172.21.15.80:0/2570777926 <== mon.0 v2:172.21.15.80:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f69dc03d030 con 0x55baa1f771a0 2024-04-24T23:30:20.904 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.897+0000 7f69ef7cb740 10 client.15720 fetch_fsmap finished waiting for FSMap version 25 2024-04-24T23:30:20.904 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.897+0000 7f69ef7cb740 10 client.15720 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:30:20.904 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.897+0000 7f69ef7cb740 20 client.15720 populate_metadata read hostname 'smithi080' 2024-04-24T23:30:20.904 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.897+0000 7f69ef7cb740 1 --2- 172.21.15.80:0/2570777926 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x55baa1fee440 0x55baa1ff0860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:20.904 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.897+0000 7f69ef7cb740 4 client.15720 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:30:20.904 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.897+0000 7f69ef7cb740 1 -- 172.21.15.80:0/2570777926 --> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55baa1ed8f00 con 0x55baa1fee440 2024-04-24T23:30:20.905 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.897+0000 7f69e659c640 1 --2- 172.21.15.80:0/2570777926 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x55baa1fee440 0x55baa1ff0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:20.905 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.897+0000 7f69cdffb640 10 client.15720 ms_handle_connect on v2:172.21.15.148:6838/2952757594 2024-04-24T23:30:20.905 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.897+0000 7f69e659c640 1 --2- 172.21.15.80:0/2570777926 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x55baa1fee440 0x55baa1ff0860 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61496c11b187dd98 server_cookie=f135b7a23682c462 in_seq=0 out_seq=0 2024-04-24T23:30:20.905 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.897+0000 7f69cdffb640 1 -- 172.21.15.80:0/2570777926 <== mds.0 v2:172.21.15.148:6838/2952757594 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55baa1ed8f00 con 0x55baa1fee440 2024-04-24T23:30:20.905 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.897+0000 7f69cdffb640 10 client.15720 handle_command_reply: tid=0 2024-04-24T23:30:20.905 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.897+0000 7f69ef7cb740 1 client.15720 shutdown 2024-04-24T23:30:20.905 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.898+0000 7f69ef7cb740 1 -- 172.21.15.80:0/2570777926 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55baa1f771a0 msgr2=0x55baa1f81aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:20.905 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.898+0000 7f69ef7cb740 1 --2- 172.21.15.80:0/2570777926 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55baa1f771a0 0x55baa1f81aa0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f69dc006010 tx=0x7f69dc005970 comp rx=0 tx=0).stop 2024-04-24T23:30:20.905 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.898+0000 7f69ef7cb740 1 -- 172.21.15.80:0/2570777926 shutdown_connections 2024-04-24T23:30:20.905 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.898+0000 7f69ef7cb740 1 --2- 172.21.15.80:0/2570777926 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x55baa1fee440 0x55baa1ff0860 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.905 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.898+0000 7f69ef7cb740 1 --2- 172.21.15.80:0/2570777926 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55baa1f771a0 0x55baa1f81aa0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.905 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.898+0000 7f69ef7cb740 1 --2- 172.21.15.80:0/2570777926 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55baa1f72030 0x55baa1f80b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.905 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.898+0000 7f69ef7cb740 1 --2- 172.21.15.80:0/2570777926 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55baa1f71640 0x55baa1f805f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.905 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.898+0000 7f69ef7cb740 1 -- 172.21.15.80:0/2570777926 >> 172.21.15.80:0/2570777926 conn(0x55baa1f688a0 msgr2=0x55baa1f6b7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:20.906 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.898+0000 7f69cdffb640 0 client.15720 ms_handle_reset on v2:172.21.15.148:6838/2952757594 2024-04-24T23:30:20.906 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.898+0000 7f69ef7cb740 1 -- 172.21.15.80:0/2570777926 shutdown_connections 2024-04-24T23:30:20.906 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.898+0000 7f69ef7cb740 1 -- 172.21.15.80:0/2570777926 wait complete. 2024-04-24T23:30:20.906 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.898+0000 7f69ef7cb740 20 client.15720 trim_cache size 0 max 16384 2024-04-24T23:30:20.906 INFO:teuthology.orchestra.run.smithi080.stdout:[] 2024-04-24T23:30:20.907 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.898+0000 7f69ec9e9640 1 -- 172.21.15.80:0/2484790016 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f69d00a2b90 msgr2=0x7f69d00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:20.907 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.898+0000 7f69ec9e9640 1 --2- 172.21.15.80:0/2484790016 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f69d00a2b90 0x7f69d00a5050 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f69e001c000 tx=0x7f69e006ba20 comp rx=0 tx=0).stop 2024-04-24T23:30:20.907 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.898+0000 7f69ec9e9640 1 -- 172.21.15.80:0/2484790016 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f69e8107460 msgr2=0x7f69e819f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:20.907 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.898+0000 7f69ec9e9640 1 --2- 172.21.15.80:0/2484790016 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f69e8107460 0x7f69e819f0a0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f69e000eaa0 tx=0x7f69e000ef60 comp rx=0 tx=0).stop 2024-04-24T23:30:20.907 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.898+0000 7f69ec9e9640 1 -- 172.21.15.80:0/2484790016 shutdown_connections 2024-04-24T23:30:20.907 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.898+0000 7f69ec9e9640 1 --2- 172.21.15.80:0/2484790016 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f69d00a2b90 0x7f69d00a5050 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.907 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.898+0000 7f69ec9e9640 1 --2- 172.21.15.80:0/2484790016 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f69e8107460 0x7f69e819f0a0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.907 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.898+0000 7f69ec9e9640 1 --2- 172.21.15.80:0/2484790016 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f69e8104b30 0x7f69e8198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.908 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.898+0000 7f69ec9e9640 1 --2- 172.21.15.80:0/2484790016 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f69e8102200 0x7f69e8197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:20.908 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.898+0000 7f69ec9e9640 1 -- 172.21.15.80:0/2484790016 >> 172.21.15.80:0/2484790016 conn(0x7f69e80fbbf0 msgr2=0x7f69e8102ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:20.908 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.899+0000 7f69ec9e9640 1 -- 172.21.15.80:0/2484790016 shutdown_connections 2024-04-24T23:30:20.908 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:20.899+0000 7f69ec9e9640 1 -- 172.21.15.80:0/2484790016 wait complete. 2024-04-24T23:30:21.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:30:21 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/460372372' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:30:21.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:30:21 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/460372372' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:30:21.537 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:30:21 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/460372372' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:30:21.914 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T23:30:21.914 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:30:21.914 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T23:30:22.021 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.018+0000 7f6aed4c1640 1 -- 172.21.15.80:0/2920043036 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6ae8101870 msgr2=0x7f6ae8101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:22.021 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.018+0000 7f6aed4c1640 1 --2- 172.21.15.80:0/2920043036 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6ae8101870 0x7f6ae8101cf0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f6adc009900 tx=0x7f6adc02a410 comp rx=0 tx=0).stop 2024-04-24T23:30:22.021 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.018+0000 7f6aed4c1640 1 -- 172.21.15.80:0/2920043036 shutdown_connections 2024-04-24T23:30:22.021 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.018+0000 7f6aed4c1640 1 --2- 172.21.15.80:0/2920043036 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ae8102230 0x7f6ae8108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:22.021 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.018+0000 7f6aed4c1640 1 --2- 172.21.15.80:0/2920043036 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6ae8101870 0x7f6ae8101cf0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:22.021 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.018+0000 7f6aed4c1640 1 --2- 172.21.15.80:0/2920043036 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ae8100680 0x7f6ae8100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:22.021 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.018+0000 7f6aed4c1640 1 -- 172.21.15.80:0/2920043036 >> 172.21.15.80:0/2920043036 conn(0x7f6ae80fbbf0 msgr2=0x7f6ae80fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:22.021 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.018+0000 7f6aed4c1640 1 -- 172.21.15.80:0/2920043036 shutdown_connections 2024-04-24T23:30:22.021 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.018+0000 7f6aed4c1640 1 -- 172.21.15.80:0/2920043036 wait complete. 2024-04-24T23:30:22.022 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.019+0000 7f6aed4c1640 1 Processor -- start 2024-04-24T23:30:22.022 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.019+0000 7f6aed4c1640 1 -- start start 2024-04-24T23:30:22.022 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.019+0000 7f6aed4c1640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6ae8100680 0x7f6ae8197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:22.022 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.019+0000 7f6aed4c1640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ae8101870 0x7f6ae8198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:22.022 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.019+0000 7f6aed4c1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ae8102230 0x7f6ae819f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:22.022 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.019+0000 7f6aed4c1640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f6ae810c370 con 0x7f6ae8100680 2024-04-24T23:30:22.022 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.019+0000 7f6aed4c1640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6ae810c1f0 con 0x7f6ae8101870 2024-04-24T23:30:22.022 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.019+0000 7f6aed4c1640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f6ae810c4f0 con 0x7f6ae8102230 2024-04-24T23:30:22.022 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.019+0000 7f6ae6ffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6ae8100680 0x7f6ae8197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:22.022 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.019+0000 7f6ae6ffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6ae8100680 0x7f6ae8197b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:46948/0 (socket says 172.21.15.80:46948) 2024-04-24T23:30:22.022 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.019+0000 7f6ae6ffd640 1 -- 172.21.15.80:0/991425555 learned_addr learned my addr 172.21.15.80:0/991425555 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:30:22.022 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.019+0000 7f6ae77fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ae8102230 0x7f6ae819f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:22.022 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.019+0000 7f6ae67fc640 1 --2- 172.21.15.80:0/991425555 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ae8101870 0x7f6ae8198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:22.023 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.020+0000 7f6ae6ffd640 1 -- 172.21.15.80:0/991425555 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ae8102230 msgr2=0x7f6ae819f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:22.023 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.020+0000 7f6ae6ffd640 1 --2- 172.21.15.80:0/991425555 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ae8102230 0x7f6ae819f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:22.023 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.020+0000 7f6ae6ffd640 1 -- 172.21.15.80:0/991425555 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ae8101870 msgr2=0x7f6ae8198070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:22.023 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.020+0000 7f6ae6ffd640 1 --2- 172.21.15.80:0/991425555 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ae8101870 0x7f6ae8198070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:22.023 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.020+0000 7f6ae6ffd640 1 -- 172.21.15.80:0/991425555 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ae819f750 con 0x7f6ae8100680 2024-04-24T23:30:22.023 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.020+0000 7f6ae67fc640 1 --2- 172.21.15.80:0/991425555 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ae8101870 0x7f6ae8198070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:30:22.023 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.020+0000 7f6ae6ffd640 1 --2- 172.21.15.80:0/991425555 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6ae8100680 0x7f6ae8197b30 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f6ad800b450 tx=0x7f6ad800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:22.023 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.020+0000 7f6acffff640 1 -- 172.21.15.80:0/991425555 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ad8015020 con 0x7f6ae8100680 2024-04-24T23:30:22.023 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.020+0000 7f6aed4c1640 1 -- 172.21.15.80:0/991425555 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ae819fa40 con 0x7f6ae8100680 2024-04-24T23:30:22.023 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.020+0000 7f6acffff640 1 -- 172.21.15.80:0/991425555 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ad8011d70 con 0x7f6ae8100680 2024-04-24T23:30:22.023 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.020+0000 7f6acffff640 1 -- 172.21.15.80:0/991425555 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ad800f340 con 0x7f6ae8100680 2024-04-24T23:30:22.023 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.020+0000 7f6aed4c1640 1 -- 172.21.15.80:0/991425555 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ae819ffb0 con 0x7f6ae8100680 2024-04-24T23:30:22.031 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.022+0000 7f6af01cd740 1 Processor -- start 2024-04-24T23:30:22.031 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.023+0000 7f6af01cd740 1 -- start start 2024-04-24T23:30:22.031 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.023+0000 7f6af01cd740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dade8af1a0 0x55dade8af5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:22.031 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.023+0000 7f6af01cd740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55dade8afb90 0x55dade8b4690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:22.031 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.023+0000 7f6af01cd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dade8b4d00 0x55dade8b7190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:22.031 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.023+0000 7f6af01cd740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x55dade80e2c0 con 0x55dade8afb90 2024-04-24T23:30:22.031 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.023+0000 7f6af01cd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55dade7b6910 con 0x55dade8b4d00 2024-04-24T23:30:22.031 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.023+0000 7f6af01cd740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55dade80a280 con 0x55dade8af1a0 2024-04-24T23:30:22.031 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.023+0000 7f6ae77fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dade8af1a0 0x55dade8af5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:22.032 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.023+0000 7f6ae67fc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dade8b4d00 0x55dade8b7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:22.032 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.023+0000 7f6ae77fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dade8af1a0 0x55dade8af5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.80:38620/0 (socket says 172.21.15.80:38620) 2024-04-24T23:30:22.032 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.023+0000 7f6ae77fe640 1 -- 172.21.15.80:0/1403171935 learned_addr learned my addr 172.21.15.80:0/1403171935 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:30:22.032 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.023+0000 7f6acffff640 1 -- 172.21.15.80:0/991425555 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6ad8018020 con 0x7f6ae8100680 2024-04-24T23:30:22.032 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.023+0000 7f6ae6ffd640 1 --2- 172.21.15.80:0/1403171935 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55dade8afb90 0x55dade8b4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:22.032 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.023+0000 7f6ae77fe640 1 -- 172.21.15.80:0/1403171935 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dade8b4d00 msgr2=0x55dade8b7190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:22.032 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.023+0000 7f6ae77fe640 1 --2- 172.21.15.80:0/1403171935 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dade8b4d00 0x55dade8b7190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:22.032 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.023+0000 7f6ae77fe640 1 -- 172.21.15.80:0/1403171935 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55dade8afb90 msgr2=0x55dade8b4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:22.032 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.023+0000 7f6ae77fe640 1 --2- 172.21.15.80:0/1403171935 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55dade8afb90 0x55dade8b4690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:22.032 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.023+0000 7f6ae77fe640 1 -- 172.21.15.80:0/1403171935 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dade7c7fb0 con 0x55dade8af1a0 2024-04-24T23:30:22.032 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.023+0000 7f6ae6ffd640 1 --2- 172.21.15.80:0/1403171935 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55dade8afb90 0x55dade8b4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:30:22.032 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.024+0000 7f6acffff640 1 --2- 172.21.15.80:0/991425555 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f6ad00a2b90 0x7f6ad00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:22.032 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.024+0000 7f6ae77fe640 1 --2- 172.21.15.80:0/1403171935 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dade8af1a0 0x55dade8af5c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6ae0010370 tx=0x7f6ae0010830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=210f5ad9e9ee3494 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:22.032 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.024+0000 7f6ae6ffd640 1 --2- 172.21.15.80:0/991425555 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f6ad00a2b90 0x7f6ad00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:22.033 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.024+0000 7f6acd7fa640 1 -- 172.21.15.80:0/1403171935 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae0016070 con 0x55dade8af1a0 2024-04-24T23:30:22.033 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.024+0000 7f6acffff640 1 -- 172.21.15.80:0/991425555 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f6ad80cee90 con 0x7f6ae8100680 2024-04-24T23:30:22.033 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.024+0000 7f6acd7fa640 1 -- 172.21.15.80:0/1403171935 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ae0004d20 con 0x55dade8af1a0 2024-04-24T23:30:22.033 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.024+0000 7f6acd7fa640 1 -- 172.21.15.80:0/1403171935 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae0005480 con 0x55dade8af1a0 2024-04-24T23:30:22.033 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.025+0000 7f6af01cd740 1 -- 172.21.15.80:0/1403171935 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dade8af1a0 msgr2=0x55dade8af5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:22.033 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.025+0000 7f6ae6ffd640 1 --2- 172.21.15.80:0/991425555 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f6ad00a2b90 0x7f6ad00a5050 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f6ad80cc930 tx=0x7f6ad8010970 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:22.033 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.025+0000 7f6af01cd740 1 --2- 172.21.15.80:0/1403171935 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dade8af1a0 0x55dade8af5c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6ae0010370 tx=0x7f6ae0010830 comp rx=0 tx=0).stop 2024-04-24T23:30:22.033 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.025+0000 7f6af01cd740 1 -- 172.21.15.80:0/1403171935 shutdown_connections 2024-04-24T23:30:22.033 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.025+0000 7f6af01cd740 1 --2- 172.21.15.80:0/1403171935 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dade8b4d00 0x55dade8b7190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:22.033 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.025+0000 7f6af01cd740 1 --2- 172.21.15.80:0/1403171935 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55dade8afb90 0x55dade8b4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:22.033 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.025+0000 7f6af01cd740 1 --2- 172.21.15.80:0/1403171935 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dade8af1a0 0x55dade8af5c0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:22.033 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.025+0000 7f6af01cd740 1 -- 172.21.15.80:0/1403171935 >> 172.21.15.80:0/1403171935 conn(0x55dade872950 msgr2=0x55dade8a6810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:22.033 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.025+0000 7f6af01cd740 1 -- 172.21.15.80:0/1403171935 shutdown_connections 2024-04-24T23:30:22.033 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.025+0000 7f6af01cd740 1 -- 172.21.15.80:0/1403171935 wait complete. 2024-04-24T23:30:22.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.025+0000 7f6af01cd740 1 -- start start 2024-04-24T23:30:22.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.025+0000 7f6af01cd740 1 Processor -- start 2024-04-24T23:30:22.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.025+0000 7f6af01cd740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55dade8af1a0 0x55dade8b95f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:22.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.025+0000 7f6af01cd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dade8afb90 0x55dade8b9b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:22.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.025+0000 7f6af01cd740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dade8b4d00 0x55dade8c0b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:22.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.025+0000 7f6af01cd740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x55dade8b7cb0 con 0x55dade8af1a0 2024-04-24T23:30:22.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.025+0000 7f6af01cd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55dade7c9e90 con 0x55dade8afb90 2024-04-24T23:30:22.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.025+0000 7f6af01cd740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55dade8b7e30 con 0x55dade8b4d00 2024-04-24T23:30:22.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.025+0000 7f6af01cd740 10 client.0 _ll_register_callbacks cb 0x55dade805970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:30:22.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.025+0000 7f6ae77fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55dade8af1a0 0x55dade8b95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:22.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.025+0000 7f6ae77fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55dade8af1a0 0x55dade8b95f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:46960/0 (socket says 172.21.15.80:46960) 2024-04-24T23:30:22.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.025+0000 7f6ae77fe640 1 -- 172.21.15.80:0/1137076656 learned_addr learned my addr 172.21.15.80:0/1137076656 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:30:22.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.025+0000 7f6ae67fc640 1 --2- 172.21.15.80:0/1137076656 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dade8afb90 0x55dade8b9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:22.035 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.026+0000 7f6ae77fe640 1 --2- 172.21.15.80:0/1137076656 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dade8b4d00 0x55dade8c0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:22.035 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.026+0000 7f6ae77fe640 1 -- 172.21.15.80:0/1137076656 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dade8b4d00 msgr2=0x55dade8c0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:22.035 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.026+0000 7f6ae77fe640 1 --2- 172.21.15.80:0/1137076656 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dade8b4d00 0x55dade8c0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:22.035 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.026+0000 7f6ae77fe640 1 -- 172.21.15.80:0/1137076656 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dade8afb90 msgr2=0x55dade8b9b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:22.035 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.026+0000 7f6ae77fe640 1 --2- 172.21.15.80:0/1137076656 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dade8afb90 0x55dade8b9b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:22.035 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.026+0000 7f6ae77fe640 1 -- 172.21.15.80:0/1137076656 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dade5ea7e0 con 0x55dade8af1a0 2024-04-24T23:30:22.035 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.026+0000 7f6ae77fe640 1 --2- 172.21.15.80:0/1137076656 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55dade8af1a0 0x55dade8b95f0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f6ae0005cc0 tx=0x7f6ae0005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a53cc607dcfd8a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:22.035 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.026+0000 7f6acaff5640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-04-24T23:30:22.035 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.026+0000 7f6acaff5640 1 -- 172.21.15.80:0/1137076656 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae0016070 con 0x55dade8af1a0 2024-04-24T23:30:22.035 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.026+0000 7f6acaff5640 1 -- 172.21.15.80:0/1137076656 <== mon.0 v2:172.21.15.80:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f6ae0022040 con 0x55dade8af1a0 2024-04-24T23:30:22.035 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.026+0000 7f6af01cd740 1 -- 172.21.15.80:0/1137076656 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ae0004d20 con 0x55dade8af1a0 2024-04-24T23:30:22.035 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.026+0000 7f6acaff5640 1 -- 172.21.15.80:0/1137076656 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6ae0003d90 con 0x55dade8af1a0 2024-04-24T23:30:22.035 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.026+0000 7f6af01cd740 10 client.15729 fetch_fsmap learned FSMap version 25 2024-04-24T23:30:22.035 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.026+0000 7f6af01cd740 1 -- 172.21.15.80:0/1137076656 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55dade64af70 con 0x55dade8af1a0 2024-04-24T23:30:22.036 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.026+0000 7f6acaff5640 1 -- 172.21.15.80:0/1137076656 <== mon.0 v2:172.21.15.80:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ae0004d20 con 0x55dade8af1a0 2024-04-24T23:30:22.036 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.027+0000 7f6acaff5640 1 -- 172.21.15.80:0/1137076656 <== mon.0 v2:172.21.15.80:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f6ae0025070 con 0x55dade8af1a0 2024-04-24T23:30:22.036 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.027+0000 7f6af01cd740 10 client.15729 fetch_fsmap finished waiting for FSMap version 25 2024-04-24T23:30:22.036 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.027+0000 7f6af01cd740 10 client.15729 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:30:22.036 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.027+0000 7f6af01cd740 20 client.15729 populate_metadata read hostname 'smithi080' 2024-04-24T23:30:22.036 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.027+0000 7f6af01cd740 1 --2- 172.21.15.80:0/1137076656 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x55dade8c4d40 0x55dade8c7130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:22.036 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.027+0000 7f6af01cd740 4 client.15729 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:30:22.036 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.027+0000 7f6af01cd740 1 -- 172.21.15.80:0/1137076656 --> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dade7e25a0 con 0x55dade8c4d40 2024-04-24T23:30:22.036 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.027+0000 7f6ae67fc640 1 --2- 172.21.15.80:0/1137076656 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x55dade8c4d40 0x55dade8c7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:22.036 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.027+0000 7f6acaff5640 10 client.15729 ms_handle_connect on v2:172.21.15.148:6838/2952757594 2024-04-24T23:30:22.036 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.027+0000 7f6ae67fc640 1 --2- 172.21.15.80:0/1137076656 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x55dade8c4d40 0x55dade8c7130 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7cc5c349441ad7fa server_cookie=9cec213b31457a35 in_seq=0 out_seq=0 2024-04-24T23:30:22.036 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.028+0000 7f6acaff5640 1 -- 172.21.15.80:0/1137076656 <== mds.0 v2:172.21.15.148:6838/2952757594 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55dade7e25a0 con 0x55dade8c4d40 2024-04-24T23:30:22.036 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.028+0000 7f6acaff5640 10 client.15729 handle_command_reply: tid=0 2024-04-24T23:30:22.036 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.028+0000 7f6af01cd740 1 client.15729 shutdown 2024-04-24T23:30:22.037 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.028+0000 7f6af01cd740 1 -- 172.21.15.80:0/1137076656 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55dade8af1a0 msgr2=0x55dade8b95f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:22.037 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.028+0000 7f6af01cd740 1 --2- 172.21.15.80:0/1137076656 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55dade8af1a0 0x55dade8b95f0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f6ae0005cc0 tx=0x7f6ae0005c50 comp rx=0 tx=0).stop 2024-04-24T23:30:22.037 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.028+0000 7f6af01cd740 1 -- 172.21.15.80:0/1137076656 shutdown_connections 2024-04-24T23:30:22.037 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.028+0000 7f6af01cd740 1 --2- 172.21.15.80:0/1137076656 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x55dade8c4d40 0x55dade8c7130 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:22.037 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.028+0000 7f6af01cd740 1 --2- 172.21.15.80:0/1137076656 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dade8b4d00 0x55dade8c0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:22.037 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.028+0000 7f6af01cd740 1 --2- 172.21.15.80:0/1137076656 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dade8afb90 0x55dade8b9b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:22.037 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.028+0000 7f6af01cd740 1 --2- 172.21.15.80:0/1137076656 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55dade8af1a0 0x55dade8b95f0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:22.037 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.028+0000 7f6af01cd740 1 -- 172.21.15.80:0/1137076656 >> 172.21.15.80:0/1137076656 conn(0x55dade872950 msgr2=0x55dade8a5950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:22.037 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.028+0000 7f6acaff5640 0 client.15729 ms_handle_reset on v2:172.21.15.148:6838/2952757594 2024-04-24T23:30:22.037 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.028+0000 7f6af01cd740 1 -- 172.21.15.80:0/1137076656 shutdown_connections 2024-04-24T23:30:22.037 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.028+0000 7f6af01cd740 1 -- 172.21.15.80:0/1137076656 wait complete. 2024-04-24T23:30:22.037 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.028+0000 7f6af01cd740 20 client.15729 trim_cache size 0 max 16384 2024-04-24T23:30:22.053 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.046+0000 7f6af01cd740 1 Processor -- start 2024-04-24T23:30:22.053 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.046+0000 7f6af01cd740 1 -- start start 2024-04-24T23:30:22.053 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.046+0000 7f6af01cd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dade8af1a0 0x55dade8cb9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:22.053 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.046+0000 7f6af01cd740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55dade8afb90 0x55dade8c0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:22.053 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.046+0000 7f6af01cd740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dade8b4d00 0x55dade8cbf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:22.053 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.046+0000 7f6af01cd740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f6ae81aad10 con 0x55dade8afb90 2024-04-24T23:30:22.053 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.046+0000 7f6af01cd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6ae81a85e0 con 0x55dade8af1a0 2024-04-24T23:30:22.053 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.046+0000 7f6af01cd740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f6ae81a6430 con 0x55dade8b4d00 2024-04-24T23:30:22.053 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.046+0000 7f6ae67fc640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55dade8afb90 0x55dade8c0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:22.053 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.046+0000 7f6ae67fc640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55dade8afb90 0x55dade8c0b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:46972/0 (socket says 172.21.15.80:46972) 2024-04-24T23:30:22.053 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.046+0000 7f6ae67fc640 1 -- 172.21.15.80:0/3711052979 learned_addr learned my addr 172.21.15.80:0/3711052979 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:30:22.053 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.046+0000 7f6ae77fe640 1 --2- 172.21.15.80:0/3711052979 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dade8b4d00 0x55dade8cbf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:22.053 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.046+0000 7f6ae77fe640 1 --2- 172.21.15.80:0/3711052979 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dade8af1a0 0x55dade8cb9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:22.053 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.047+0000 7f6ae67fc640 1 -- 172.21.15.80:0/3711052979 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dade8b4d00 msgr2=0x55dade8cbf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:22.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.047+0000 7f6ae67fc640 1 --2- 172.21.15.80:0/3711052979 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dade8b4d00 0x55dade8cbf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:22.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.047+0000 7f6ae67fc640 1 -- 172.21.15.80:0/3711052979 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dade8af1a0 msgr2=0x55dade8cb9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:22.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.047+0000 7f6ae67fc640 1 --2- 172.21.15.80:0/3711052979 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dade8af1a0 0x55dade8cb9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:22.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.047+0000 7f6ae67fc640 1 -- 172.21.15.80:0/3711052979 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dade4e2170 con 0x55dade8afb90 2024-04-24T23:30:22.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.047+0000 7f6ae77fe640 1 --2- 172.21.15.80:0/3711052979 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dade8af1a0 0x55dade8cb9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:30:22.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.047+0000 7f6ae67fc640 1 --2- 172.21.15.80:0/3711052979 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55dade8afb90 0x55dade8c0b30 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f6adc031440 tx=0x7f6adc0315d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61e93c9ea7724dfe server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:22.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.047+0000 7f6accff9640 1 -- 172.21.15.80:0/3711052979 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6adc035040 con 0x55dade8afb90 2024-04-24T23:30:22.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.047+0000 7f6accff9640 1 -- 172.21.15.80:0/3711052979 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6adc04a070 con 0x55dade8afb90 2024-04-24T23:30:22.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.047+0000 7f6accff9640 1 -- 172.21.15.80:0/3711052979 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6adc005570 con 0x55dade8afb90 2024-04-24T23:30:22.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.048+0000 7f6af01cd740 1 -- 172.21.15.80:0/3711052979 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55dade8afb90 msgr2=0x55dade8c0b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:22.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.048+0000 7f6af01cd740 1 --2- 172.21.15.80:0/3711052979 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55dade8afb90 0x55dade8c0b30 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f6adc031440 tx=0x7f6adc0315d0 comp rx=0 tx=0).stop 2024-04-24T23:30:22.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.048+0000 7f6af01cd740 1 -- 172.21.15.80:0/3711052979 shutdown_connections 2024-04-24T23:30:22.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.048+0000 7f6af01cd740 1 --2- 172.21.15.80:0/3711052979 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dade8b4d00 0x55dade8cbf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:22.055 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.048+0000 7f6af01cd740 1 --2- 172.21.15.80:0/3711052979 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55dade8afb90 0x55dade8c0b30 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:22.055 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.048+0000 7f6af01cd740 1 --2- 172.21.15.80:0/3711052979 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dade8af1a0 0x55dade8cb9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:22.055 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.048+0000 7f6af01cd740 1 -- 172.21.15.80:0/3711052979 >> 172.21.15.80:0/3711052979 conn(0x55dade872950 msgr2=0x55dade8b8a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:22.055 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.048+0000 7f6af01cd740 1 -- 172.21.15.80:0/3711052979 shutdown_connections 2024-04-24T23:30:22.055 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.048+0000 7f6af01cd740 1 -- 172.21.15.80:0/3711052979 wait complete. 2024-04-24T23:30:22.055 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.048+0000 7f6af01cd740 1 -- start start 2024-04-24T23:30:22.055 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.048+0000 7f6af01cd740 1 Processor -- start 2024-04-24T23:30:22.055 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.048+0000 7f6af01cd740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dade8af1a0 0x55dade8bde10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:22.055 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.048+0000 7f6af01cd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dade8afb90 0x55dade8be350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:22.055 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.048+0000 7f6af01cd740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55dade8b4d00 0x55dade8bf360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:22.055 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.048+0000 7f6af01cd740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f6ae81a6aa0 con 0x55dade8b4d00 2024-04-24T23:30:22.055 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.048+0000 7f6af01cd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6ae81a7f10 con 0x55dade8afb90 2024-04-24T23:30:22.055 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.048+0000 7f6af01cd740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f6ae81a7170 con 0x55dade8af1a0 2024-04-24T23:30:22.055 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.048+0000 7f6af01cd740 10 client.0 _ll_register_callbacks cb 0x7f6adc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:30:22.055 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.048+0000 7f6ae77fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55dade8b4d00 0x55dade8bf360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:22.056 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.048+0000 7f6ae77fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55dade8b4d00 0x55dade8bf360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:46984/0 (socket says 172.21.15.80:46984) 2024-04-24T23:30:22.056 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.048+0000 7f6ae77fe640 1 -- 172.21.15.80:0/3989405259 learned_addr learned my addr 172.21.15.80:0/3989405259 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:30:22.056 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.048+0000 7f6ae77fe640 1 --2- 172.21.15.80:0/3989405259 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dade8af1a0 0x55dade8bde10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:22.056 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.048+0000 7f6ae67fc640 1 --2- 172.21.15.80:0/3989405259 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dade8afb90 0x55dade8be350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:22.056 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.049+0000 7f6ae77fe640 1 -- 172.21.15.80:0/3989405259 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dade8af1a0 msgr2=0x55dade8bde10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:22.056 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.049+0000 7f6ae77fe640 1 --2- 172.21.15.80:0/3989405259 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dade8af1a0 0x55dade8bde10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:22.056 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.049+0000 7f6ae77fe640 1 -- 172.21.15.80:0/3989405259 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dade8afb90 msgr2=0x55dade8be350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:22.056 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.049+0000 7f6ae77fe640 1 --2- 172.21.15.80:0/3989405259 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dade8afb90 0x55dade8be350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:22.056 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.049+0000 7f6ae77fe640 1 -- 172.21.15.80:0/3989405259 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dade8bf950 con 0x55dade8b4d00 2024-04-24T23:30:22.056 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.049+0000 7f6ae77fe640 1 --2- 172.21.15.80:0/3989405259 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55dade8b4d00 0x55dade8bf360 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f6ae0009770 tx=0x7f6ae0004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30b6dccea7da3475 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:22.056 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.049+0000 7f6acaff5640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-04-24T23:30:22.056 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.049+0000 7f6acaff5640 1 -- 172.21.15.80:0/3989405259 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae0016040 con 0x55dade8b4d00 2024-04-24T23:30:22.056 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.049+0000 7f6af01cd740 1 -- 172.21.15.80:0/3989405259 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6adc04a070 con 0x55dade8b4d00 2024-04-24T23:30:22.056 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.049+0000 7f6acaff5640 1 -- 172.21.15.80:0/3989405259 <== mon.0 v2:172.21.15.80:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f6ae0002030 con 0x55dade8b4d00 2024-04-24T23:30:22.056 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.049+0000 7f6acaff5640 1 -- 172.21.15.80:0/3989405259 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6ae0004140 con 0x55dade8b4d00 2024-04-24T23:30:22.057 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.049+0000 7f6af01cd740 10 client.15735 fetch_fsmap learned FSMap version 25 2024-04-24T23:30:22.057 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.049+0000 7f6af01cd740 1 -- 172.21.15.80:0/3989405259 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55dade8c0c00 con 0x55dade8b4d00 2024-04-24T23:30:22.057 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.049+0000 7f6acaff5640 1 -- 172.21.15.80:0/3989405259 <== mon.0 v2:172.21.15.80:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6adc04a070 con 0x55dade8b4d00 2024-04-24T23:30:22.057 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.050+0000 7f6acaff5640 1 -- 172.21.15.80:0/3989405259 <== mon.0 v2:172.21.15.80:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f6ae0025070 con 0x55dade8b4d00 2024-04-24T23:30:22.057 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.050+0000 7f6af01cd740 10 client.15735 fetch_fsmap finished waiting for FSMap version 25 2024-04-24T23:30:22.057 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.050+0000 7f6af01cd740 10 client.15735 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:30:22.057 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.050+0000 7f6af01cd740 20 client.15735 populate_metadata read hostname 'smithi080' 2024-04-24T23:30:22.057 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.050+0000 7f6af01cd740 1 --2- 172.21.15.80:0/3989405259 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x55dade92b750 0x55dade92db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:22.057 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.050+0000 7f6af01cd740 4 client.15735 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T23:30:22.057 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.050+0000 7f6af01cd740 1 -- 172.21.15.80:0/3989405259 --> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55dade81b800 con 0x55dade92b750 2024-04-24T23:30:22.057 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.050+0000 7f6ae67fc640 1 --2- 172.21.15.80:0/3989405259 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x55dade92b750 0x55dade92db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:22.057 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.050+0000 7f6acaff5640 10 client.15735 ms_handle_connect on v2:172.21.15.148:6838/2952757594 2024-04-24T23:30:22.057 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.050+0000 7f6ae67fc640 1 --2- 172.21.15.80:0/3989405259 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x55dade92b750 0x55dade92db70 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=390ebb53081e5cd1 server_cookie=519c8d5ab399519 in_seq=0 out_seq=0 2024-04-24T23:30:22.057 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.051+0000 7f6acaff5640 1 -- 172.21.15.80:0/3989405259 <== mds.0 v2:172.21.15.148:6838/2952757594 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55dade81b800 con 0x55dade92b750 2024-04-24T23:30:22.058 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.051+0000 7f6acaff5640 10 client.15735 handle_command_reply: tid=0 2024-04-24T23:30:22.058 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.051+0000 7f6af01cd740 1 client.15735 shutdown 2024-04-24T23:30:22.058 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.051+0000 7f6af01cd740 1 -- 172.21.15.80:0/3989405259 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55dade8b4d00 msgr2=0x55dade8bf360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:22.058 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.051+0000 7f6af01cd740 1 --2- 172.21.15.80:0/3989405259 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55dade8b4d00 0x55dade8bf360 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f6ae0009770 tx=0x7f6ae0004ac0 comp rx=0 tx=0).stop 2024-04-24T23:30:22.058 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.051+0000 7f6af01cd740 1 -- 172.21.15.80:0/3989405259 shutdown_connections 2024-04-24T23:30:22.058 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.051+0000 7f6af01cd740 1 --2- 172.21.15.80:0/3989405259 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x55dade92b750 0x55dade92db70 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:22.058 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.051+0000 7f6acaff5640 0 client.15735 ms_handle_reset on v2:172.21.15.148:6838/2952757594 2024-04-24T23:30:22.058 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.051+0000 7f6af01cd740 1 --2- 172.21.15.80:0/3989405259 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55dade8b4d00 0x55dade8bf360 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:22.058 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.051+0000 7f6af01cd740 1 --2- 172.21.15.80:0/3989405259 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dade8afb90 0x55dade8be350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:22.058 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.051+0000 7f6af01cd740 1 --2- 172.21.15.80:0/3989405259 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dade8af1a0 0x55dade8bde10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:22.058 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.051+0000 7f6af01cd740 1 -- 172.21.15.80:0/3989405259 >> 172.21.15.80:0/3989405259 conn(0x55dade872950 msgr2=0x55dade8a94d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:22.058 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.052+0000 7f6af01cd740 1 -- 172.21.15.80:0/3989405259 shutdown_connections 2024-04-24T23:30:22.058 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.052+0000 7f6af01cd740 1 -- 172.21.15.80:0/3989405259 wait complete. 2024-04-24T23:30:22.058 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.052+0000 7f6af01cd740 20 client.15735 trim_cache size 0 max 16384 2024-04-24T23:30:22.059 INFO:teuthology.orchestra.run.smithi080.stdout:{ 2024-04-24T23:30:22.059 INFO:teuthology.orchestra.run.smithi080.stdout: "return_code": 0, 2024-04-24T23:30:22.059 INFO:teuthology.orchestra.run.smithi080.stdout: "scrub_tag": "5152e04c-8be7-4907-8cb6-9f6c552edbec", 2024-04-24T23:30:22.059 INFO:teuthology.orchestra.run.smithi080.stdout: "mode": "asynchronous" 2024-04-24T23:30:22.059 INFO:teuthology.orchestra.run.smithi080.stdout:} 2024-04-24T23:30:22.060 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.052+0000 7f6aed4c1640 1 -- 172.21.15.80:0/991425555 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f6ad00a2b90 msgr2=0x7f6ad00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:22.060 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.052+0000 7f6aed4c1640 1 --2- 172.21.15.80:0/991425555 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f6ad00a2b90 0x7f6ad00a5050 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f6ad80cc930 tx=0x7f6ad8010970 comp rx=0 tx=0).stop 2024-04-24T23:30:22.060 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.052+0000 7f6aed4c1640 1 -- 172.21.15.80:0/991425555 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6ae8100680 msgr2=0x7f6ae8197b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:22.060 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.052+0000 7f6aed4c1640 1 --2- 172.21.15.80:0/991425555 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6ae8100680 0x7f6ae8197b30 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f6ad800b450 tx=0x7f6ad800b910 comp rx=0 tx=0).stop 2024-04-24T23:30:22.060 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.052+0000 7f6aed4c1640 1 -- 172.21.15.80:0/991425555 shutdown_connections 2024-04-24T23:30:22.060 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.052+0000 7f6aed4c1640 1 --2- 172.21.15.80:0/991425555 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f6ad00a2b90 0x7f6ad00a5050 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:22.060 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.052+0000 7f6aed4c1640 1 --2- 172.21.15.80:0/991425555 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ae8102230 0x7f6ae819f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:22.060 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.052+0000 7f6aed4c1640 1 --2- 172.21.15.80:0/991425555 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ae8101870 0x7f6ae8198070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:22.060 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.052+0000 7f6aed4c1640 1 --2- 172.21.15.80:0/991425555 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6ae8100680 0x7f6ae8197b30 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:22.060 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.052+0000 7f6aed4c1640 1 -- 172.21.15.80:0/991425555 >> 172.21.15.80:0/991425555 conn(0x7f6ae80fbbf0 msgr2=0x7f6ae80688d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:22.061 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.052+0000 7f6aed4c1640 1 -- 172.21.15.80:0/991425555 shutdown_connections 2024-04-24T23:30:22.061 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.052+0000 7f6aed4c1640 1 -- 172.21.15.80:0/991425555 wait complete. 2024-04-24T23:30:22.064 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T23:30:22.064 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T23:30:22.170 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.167+0000 7fd07a89a640 1 -- 172.21.15.80:0/1102108939 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd074102230 msgr2=0x7fd074108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:22.170 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.167+0000 7fd07a89a640 1 --2- 172.21.15.80:0/1102108939 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd074102230 0x7fd074108ac0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fd06c009ca0 tx=0x7fd06c02a8c0 comp rx=0 tx=0).stop 2024-04-24T23:30:22.171 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.167+0000 7fd07a89a640 1 -- 172.21.15.80:0/1102108939 shutdown_connections 2024-04-24T23:30:22.171 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.167+0000 7fd07a89a640 1 --2- 172.21.15.80:0/1102108939 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd074102230 0x7fd074108ac0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:22.171 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.167+0000 7fd07a89a640 1 --2- 172.21.15.80:0/1102108939 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd074101870 0x7fd074101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:22.171 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.167+0000 7fd07a89a640 1 --2- 172.21.15.80:0/1102108939 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd074100680 0x7fd074100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:22.171 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.167+0000 7fd07a89a640 1 -- 172.21.15.80:0/1102108939 >> 172.21.15.80:0/1102108939 conn(0x7fd0740fbbf0 msgr2=0x7fd0740fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:22.171 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.167+0000 7fd07a89a640 1 -- 172.21.15.80:0/1102108939 shutdown_connections 2024-04-24T23:30:22.171 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.167+0000 7fd07a89a640 1 -- 172.21.15.80:0/1102108939 wait complete. 2024-04-24T23:30:22.171 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.168+0000 7fd07a89a640 1 Processor -- start 2024-04-24T23:30:22.171 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.168+0000 7fd07a89a640 1 -- start start 2024-04-24T23:30:22.171 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.168+0000 7fd07a89a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd074100680 0x7fd074069440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:22.171 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.168+0000 7fd07a89a640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd074101870 0x7fd07419c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:22.171 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.168+0000 7fd07a89a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd074102230 0x7fd07419edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:22.171 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.168+0000 7fd07a89a640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fd07410c300 con 0x7fd074101870 2024-04-24T23:30:22.172 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.168+0000 7fd07a89a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd07410c180 con 0x7fd074100680 2024-04-24T23:30:22.172 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.168+0000 7fd07a89a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd07410c480 con 0x7fd074102230 2024-04-24T23:30:22.172 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.168+0000 7fd0737fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd074101870 0x7fd07419c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:22.172 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.168+0000 7fd0737fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd074101870 0x7fd07419c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:46996/0 (socket says 172.21.15.80:46996) 2024-04-24T23:30:22.172 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.169+0000 7fd0737fe640 1 -- 172.21.15.80:0/343565630 learned_addr learned my addr 172.21.15.80:0/343565630 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:30:22.172 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.169+0000 7fd073fff640 1 --2- 172.21.15.80:0/343565630 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd074100680 0x7fd074069440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:22.172 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.169+0000 7fd078e37640 1 --2- 172.21.15.80:0/343565630 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd074102230 0x7fd07419edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:22.172 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.169+0000 7fd0737fe640 1 -- 172.21.15.80:0/343565630 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd074102230 msgr2=0x7fd07419edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:22.172 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.169+0000 7fd0737fe640 1 --2- 172.21.15.80:0/343565630 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd074102230 0x7fd07419edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:22.172 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.169+0000 7fd0737fe640 1 -- 172.21.15.80:0/343565630 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd074100680 msgr2=0x7fd074069440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:22.172 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.169+0000 7fd0737fe640 1 --2- 172.21.15.80:0/343565630 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd074100680 0x7fd074069440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:22.172 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.169+0000 7fd0737fe640 1 -- 172.21.15.80:0/343565630 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd07419f700 con 0x7fd074101870 2024-04-24T23:30:22.172 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.169+0000 7fd0737fe640 1 --2- 172.21.15.80:0/343565630 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd074101870 0x7fd07419c7d0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fd06800d770 tx=0x7fd06800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:22.173 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.169+0000 7fd0717fa640 1 -- 172.21.15.80:0/343565630 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd068017070 con 0x7fd074101870 2024-04-24T23:30:22.173 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.169+0000 7fd0717fa640 1 -- 172.21.15.80:0/343565630 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd068005500 con 0x7fd074101870 2024-04-24T23:30:22.173 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.169+0000 7fd0717fa640 1 -- 172.21.15.80:0/343565630 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd068004e00 con 0x7fd074101870 2024-04-24T23:30:22.173 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.169+0000 7fd07a89a640 1 -- 172.21.15.80:0/343565630 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd07419f990 con 0x7fd074101870 2024-04-24T23:30:22.173 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.169+0000 7fd07a89a640 1 -- 172.21.15.80:0/343565630 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd07419fee0 con 0x7fd074101870 2024-04-24T23:30:22.180 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.171+0000 7fd0717fa640 1 -- 172.21.15.80:0/343565630 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd068004fa0 con 0x7fd074101870 2024-04-24T23:30:22.180 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.172+0000 7fd0717fa640 1 --2- 172.21.15.80:0/343565630 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fd05c0a2c10 0x7fd05c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:22.181 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.172+0000 7fd073fff640 1 --2- 172.21.15.80:0/343565630 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fd05c0a2c10 0x7fd05c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:22.181 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.172+0000 7fd0717fa640 1 -- 172.21.15.80:0/343565630 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fd0680ced30 con 0x7fd074101870 2024-04-24T23:30:22.181 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.172+0000 7fd073fff640 1 --2- 172.21.15.80:0/343565630 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fd05c0a2c10 0x7fd05c0a50d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd0741016d0 tx=0x7fd06400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:22.181 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.173+0000 7fd07d5a4740 1 Processor -- start 2024-04-24T23:30:22.181 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.173+0000 7fd07d5a4740 1 -- start start 2024-04-24T23:30:22.181 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.173+0000 7fd07d5a4740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cf79810240 0x55cf79810660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:22.181 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.173+0000 7fd07d5a4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cf79810c30 0x55cf79815730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:22.181 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.173+0000 7fd07d5a4740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cf79815da0 0x55cf79818230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:22.181 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.173+0000 7fd07d5a4740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x55cf797746b0 con 0x55cf79810240 2024-04-24T23:30:22.181 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.173+0000 7fd07d5a4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55cf7971c880 con 0x55cf79810c30 2024-04-24T23:30:22.181 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.173+0000 7fd07d5a4740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55cf79770670 con 0x55cf79815da0 2024-04-24T23:30:22.181 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.173+0000 7fd073fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cf79810240 0x55cf79810660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:22.182 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.173+0000 7fd073fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cf79810240 0x55cf79810660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:46998/0 (socket says 172.21.15.80:46998) 2024-04-24T23:30:22.182 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.173+0000 7fd073fff640 1 -- 172.21.15.80:0/1198529962 learned_addr learned my addr 172.21.15.80:0/1198529962 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:30:22.182 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.173+0000 7fd0737fe640 1 --2- 172.21.15.80:0/1198529962 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cf79810c30 0x55cf79815730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:22.182 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.173+0000 7fd078e37640 1 --2- 172.21.15.80:0/1198529962 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cf79815da0 0x55cf79818230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:22.182 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.173+0000 7fd073fff640 1 -- 172.21.15.80:0/1198529962 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cf79815da0 msgr2=0x55cf79818230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:22.182 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.173+0000 7fd073fff640 1 --2- 172.21.15.80:0/1198529962 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cf79815da0 0x55cf79818230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:22.182 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.173+0000 7fd073fff640 1 -- 172.21.15.80:0/1198529962 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cf79810c30 msgr2=0x55cf79815730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:22.182 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.173+0000 7fd073fff640 1 --2- 172.21.15.80:0/1198529962 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cf79810c30 0x55cf79815730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:22.182 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.173+0000 7fd073fff640 1 -- 172.21.15.80:0/1198529962 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf795527d0 con 0x55cf79810240 2024-04-24T23:30:22.182 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.173+0000 7fd073fff640 1 --2- 172.21.15.80:0/1198529962 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cf79810240 0x55cf79810660 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fd0640058a0 tx=0x7fd064013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8caf8561fc00102e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:22.182 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.174+0000 7fd05a7fc640 1 -- 172.21.15.80:0/1198529962 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd064004220 con 0x55cf79810240 2024-04-24T23:30:22.182 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.174+0000 7fd05a7fc640 1 -- 172.21.15.80:0/1198529962 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0640043c0 con 0x55cf79810240 2024-04-24T23:30:22.182 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.174+0000 7fd05a7fc640 1 -- 172.21.15.80:0/1198529962 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd06401e930 con 0x55cf79810240 2024-04-24T23:30:22.182 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.175+0000 7fd07d5a4740 1 -- 172.21.15.80:0/1198529962 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cf79810240 msgr2=0x55cf79810660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:22.183 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.175+0000 7fd07d5a4740 1 --2- 172.21.15.80:0/1198529962 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cf79810240 0x55cf79810660 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fd0640058a0 tx=0x7fd064013000 comp rx=0 tx=0).stop 2024-04-24T23:30:22.183 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.175+0000 7fd07d5a4740 1 -- 172.21.15.80:0/1198529962 shutdown_connections 2024-04-24T23:30:22.183 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.175+0000 7fd07d5a4740 1 --2- 172.21.15.80:0/1198529962 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cf79815da0 0x55cf79818230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:22.183 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.175+0000 7fd07d5a4740 1 --2- 172.21.15.80:0/1198529962 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cf79810c30 0x55cf79815730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:22.183 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.175+0000 7fd07d5a4740 1 --2- 172.21.15.80:0/1198529962 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cf79810240 0x55cf79810660 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:22.183 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.175+0000 7fd07d5a4740 1 -- 172.21.15.80:0/1198529962 >> 172.21.15.80:0/1198529962 conn(0x55cf7980b8b0 msgr2=0x55cf7980bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:22.183 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.175+0000 7fd07d5a4740 1 -- 172.21.15.80:0/1198529962 shutdown_connections 2024-04-24T23:30:22.183 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.175+0000 7fd07d5a4740 1 -- 172.21.15.80:0/1198529962 wait complete. 2024-04-24T23:30:22.183 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.175+0000 7fd07d5a4740 1 -- start start 2024-04-24T23:30:22.183 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.175+0000 7fd07d5a4740 1 Processor -- start 2024-04-24T23:30:22.183 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.175+0000 7fd07d5a4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cf79810240 0x55cf7981a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:22.183 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.175+0000 7fd07d5a4740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cf79810c30 0x55cf7981ab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:22.184 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.175+0000 7fd07d5a4740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cf79815da0 0x55cf79821b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:22.184 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.175+0000 7fd07d5a4740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x55cf79818d50 con 0x55cf79815da0 2024-04-24T23:30:22.184 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.175+0000 7fd07d5a4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55cf7972ff50 con 0x55cf79810240 2024-04-24T23:30:22.184 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.175+0000 7fd07d5a4740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55cf79818ed0 con 0x55cf79810c30 2024-04-24T23:30:22.184 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.175+0000 7fd07d5a4740 10 client.0 _ll_register_callbacks cb 0x55cf7976b000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:30:22.184 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.176+0000 7fd078e37640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cf79815da0 0x55cf79821b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:22.184 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.176+0000 7fd078e37640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cf79815da0 0x55cf79821b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:47006/0 (socket says 172.21.15.80:47006) 2024-04-24T23:30:22.184 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.176+0000 7fd078e37640 1 -- 172.21.15.80:0/1162556837 learned_addr learned my addr 172.21.15.80:0/1162556837 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:30:22.184 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.176+0000 7fd073fff640 1 --2- 172.21.15.80:0/1162556837 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cf79810240 0x55cf7981a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:22.184 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.176+0000 7fd0737fe640 1 --2- 172.21.15.80:0/1162556837 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cf79810c30 0x55cf7981ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:22.184 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.176+0000 7fd078e37640 1 -- 172.21.15.80:0/1162556837 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cf79810c30 msgr2=0x55cf7981ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:22.184 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.176+0000 7fd078e37640 1 --2- 172.21.15.80:0/1162556837 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cf79810c30 0x55cf7981ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:22.184 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.176+0000 7fd078e37640 1 -- 172.21.15.80:0/1162556837 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cf79810240 msgr2=0x55cf7981a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:22.185 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.176+0000 7fd078e37640 1 --2- 172.21.15.80:0/1162556837 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cf79810240 0x55cf7981a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:22.185 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.176+0000 7fd078e37640 1 -- 172.21.15.80:0/1162556837 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf794883f0 con 0x55cf79815da0 2024-04-24T23:30:22.185 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.176+0000 7fd078e37640 1 --2- 172.21.15.80:0/1162556837 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cf79815da0 0x55cf79821b90 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fd06c009bf0 tx=0x7fd06c02f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d52470b3c2381b27 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:22.185 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.176+0000 7fd0537fe640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-04-24T23:30:22.185 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.176+0000 7fd0537fe640 1 -- 172.21.15.80:0/1162556837 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd06c04a060 con 0x55cf79815da0 2024-04-24T23:30:22.185 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.176+0000 7fd0537fe640 1 -- 172.21.15.80:0/1162556837 <== mon.0 v2:172.21.15.80:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fd06c003970 con 0x55cf79815da0 2024-04-24T23:30:22.185 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.176+0000 7fd07d5a4740 1 -- 172.21.15.80:0/1162556837 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0640043c0 con 0x55cf79815da0 2024-04-24T23:30:22.185 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.176+0000 7fd0537fe640 1 -- 172.21.15.80:0/1162556837 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd06c003b70 con 0x55cf79815da0 2024-04-24T23:30:22.185 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.176+0000 7fd07d5a4740 10 client.15747 fetch_fsmap learned FSMap version 25 2024-04-24T23:30:22.185 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.176+0000 7fd07d5a4740 1 -- 172.21.15.80:0/1162556837 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55cf794480f0 con 0x55cf79815da0 2024-04-24T23:30:22.185 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.177+0000 7fd0537fe640 1 -- 172.21.15.80:0/1162556837 <== mon.0 v2:172.21.15.80:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0640043c0 con 0x55cf79815da0 2024-04-24T23:30:22.185 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.177+0000 7fd0537fe640 1 -- 172.21.15.80:0/1162556837 <== mon.0 v2:172.21.15.80:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fd06c02f730 con 0x55cf79815da0 2024-04-24T23:30:22.186 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.177+0000 7fd07d5a4740 10 client.15747 fetch_fsmap finished waiting for FSMap version 25 2024-04-24T23:30:22.186 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.177+0000 7fd07d5a4740 10 client.15747 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:30:22.186 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.177+0000 7fd07d5a4740 20 client.15747 populate_metadata read hostname 'smithi080' 2024-04-24T23:30:22.186 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.177+0000 7fd07d5a4740 1 --2- 172.21.15.80:0/1162556837 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x55cf79826480 0x55cf79828920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:22.186 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.177+0000 7fd07d5a4740 4 client.15747 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:30:22.186 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.177+0000 7fd07d5a4740 1 -- 172.21.15.80:0/1162556837 --> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cf79781bf0 con 0x55cf79826480 2024-04-24T23:30:22.186 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.177+0000 7fd073fff640 1 --2- 172.21.15.80:0/1162556837 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x55cf79826480 0x55cf79828920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:22.186 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.177+0000 7fd073fff640 1 --2- 172.21.15.80:0/1162556837 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x55cf79826480 0x55cf79828920 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9e79423bdc310e2 server_cookie=fa9fadbf28150b12 in_seq=0 out_seq=0 2024-04-24T23:30:22.186 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.177+0000 7fd0537fe640 10 client.15747 ms_handle_connect on v2:172.21.15.148:6838/2952757594 2024-04-24T23:30:22.186 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.178+0000 7fd0537fe640 1 -- 172.21.15.80:0/1162556837 <== mds.0 v2:172.21.15.148:6838/2952757594 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cf79781bf0 con 0x55cf79826480 2024-04-24T23:30:22.186 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.178+0000 7fd0537fe640 10 client.15747 handle_command_reply: tid=0 2024-04-24T23:30:22.186 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.178+0000 7fd07d5a4740 1 client.15747 shutdown 2024-04-24T23:30:22.186 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.178+0000 7fd07d5a4740 1 -- 172.21.15.80:0/1162556837 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cf79815da0 msgr2=0x55cf79821b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:22.187 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.178+0000 7fd07d5a4740 1 --2- 172.21.15.80:0/1162556837 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cf79815da0 0x55cf79821b90 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fd06c009bf0 tx=0x7fd06c02f380 comp rx=0 tx=0).stop 2024-04-24T23:30:22.187 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.178+0000 7fd07d5a4740 1 -- 172.21.15.80:0/1162556837 shutdown_connections 2024-04-24T23:30:22.187 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.178+0000 7fd07d5a4740 1 --2- 172.21.15.80:0/1162556837 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x55cf79826480 0x55cf79828920 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:22.187 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.178+0000 7fd07d5a4740 1 --2- 172.21.15.80:0/1162556837 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cf79815da0 0x55cf79821b90 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:22.187 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.178+0000 7fd07d5a4740 1 --2- 172.21.15.80:0/1162556837 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cf79810c30 0x55cf7981ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:22.187 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.178+0000 7fd07d5a4740 1 --2- 172.21.15.80:0/1162556837 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cf79810240 0x55cf7981a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:22.187 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.178+0000 7fd07d5a4740 1 -- 172.21.15.80:0/1162556837 >> 172.21.15.80:0/1162556837 conn(0x55cf7980b8b0 msgr2=0x55cf7980aaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:22.187 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.178+0000 7fd0537fe640 0 client.15747 ms_handle_reset on v2:172.21.15.148:6838/2952757594 2024-04-24T23:30:22.187 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.178+0000 7fd07d5a4740 1 -- 172.21.15.80:0/1162556837 shutdown_connections 2024-04-24T23:30:22.188 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.178+0000 7fd07d5a4740 1 -- 172.21.15.80:0/1162556837 wait complete. 2024-04-24T23:30:22.188 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.179+0000 7fd07d5a4740 20 client.15747 trim_cache size 0 max 16384 2024-04-24T23:30:22.202 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.196+0000 7fd07d5a4740 1 Processor -- start 2024-04-24T23:30:22.202 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.196+0000 7fd07d5a4740 1 -- start start 2024-04-24T23:30:22.203 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.196+0000 7fd07d5a4740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cf79810240 0x55cf7980bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:22.203 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.196+0000 7fd07d5a4740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cf79810c30 0x55cf79821b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:22.203 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.196+0000 7fd07d5a4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cf79815da0 0x55cf7980e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:22.203 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.196+0000 7fd07d5a4740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fd0741acd00 con 0x55cf79810240 2024-04-24T23:30:22.203 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.196+0000 7fd07d5a4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd0741aa5d0 con 0x55cf79815da0 2024-04-24T23:30:22.203 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.196+0000 7fd07d5a4740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd0741a8420 con 0x55cf79810c30 2024-04-24T23:30:22.203 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.196+0000 7fd073fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cf79810240 0x55cf7980bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:22.203 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.196+0000 7fd073fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cf79810240 0x55cf7980bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:47010/0 (socket says 172.21.15.80:47010) 2024-04-24T23:30:22.203 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.196+0000 7fd073fff640 1 -- 172.21.15.80:0/171294433 learned_addr learned my addr 172.21.15.80:0/171294433 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:30:22.203 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.196+0000 7fd078e37640 1 --2- 172.21.15.80:0/171294433 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cf79815da0 0x55cf7980e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:22.203 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.196+0000 7fd073fff640 1 -- 172.21.15.80:0/171294433 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cf79810c30 msgr2=0x55cf79821b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:22.203 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.196+0000 7fd073fff640 1 --2- 172.21.15.80:0/171294433 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cf79810c30 0x55cf79821b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:22.203 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.196+0000 7fd073fff640 1 -- 172.21.15.80:0/171294433 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cf79815da0 msgr2=0x55cf7980e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:22.204 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.196+0000 7fd073fff640 1 --2- 172.21.15.80:0/171294433 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cf79815da0 0x55cf7980e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:22.204 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.196+0000 7fd073fff640 1 -- 172.21.15.80:0/171294433 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf797fd090 con 0x55cf79810240 2024-04-24T23:30:22.204 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.196+0000 7fd078e37640 1 --2- 172.21.15.80:0/171294433 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cf79815da0 0x55cf7980e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:30:22.204 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.196+0000 7fd073fff640 1 --2- 172.21.15.80:0/171294433 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cf79810240 0x55cf7980bc90 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fd064010fd0 tx=0x7fd064004310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ac821d24729f27a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:22.204 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.196+0000 7fd059ffb640 1 -- 172.21.15.80:0/171294433 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd064004480 con 0x55cf79810240 2024-04-24T23:30:22.204 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.197+0000 7fd059ffb640 1 -- 172.21.15.80:0/171294433 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd06403b070 con 0x55cf79810240 2024-04-24T23:30:22.204 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.197+0000 7fd059ffb640 1 -- 172.21.15.80:0/171294433 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd064003850 con 0x55cf79810240 2024-04-24T23:30:22.204 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.197+0000 7fd07d5a4740 1 -- 172.21.15.80:0/171294433 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cf79810240 msgr2=0x55cf7980bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:22.204 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.197+0000 7fd07d5a4740 1 --2- 172.21.15.80:0/171294433 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cf79810240 0x55cf7980bc90 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fd064010fd0 tx=0x7fd064004310 comp rx=0 tx=0).stop 2024-04-24T23:30:22.204 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.197+0000 7fd07d5a4740 1 -- 172.21.15.80:0/171294433 shutdown_connections 2024-04-24T23:30:22.204 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.197+0000 7fd07d5a4740 1 --2- 172.21.15.80:0/171294433 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cf79815da0 0x55cf7980e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:22.204 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.197+0000 7fd07d5a4740 1 --2- 172.21.15.80:0/171294433 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cf79810c30 0x55cf79821b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:22.204 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.197+0000 7fd07d5a4740 1 --2- 172.21.15.80:0/171294433 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cf79810240 0x55cf7980bc90 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:22.205 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.197+0000 7fd07d5a4740 1 -- 172.21.15.80:0/171294433 >> 172.21.15.80:0/171294433 conn(0x55cf7980b8b0 msgr2=0x55cf7981a610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:22.205 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.197+0000 7fd07d5a4740 1 -- 172.21.15.80:0/171294433 shutdown_connections 2024-04-24T23:30:22.205 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.197+0000 7fd07d5a4740 1 -- 172.21.15.80:0/171294433 wait complete. 2024-04-24T23:30:22.205 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.198+0000 7fd07d5a4740 1 -- start start 2024-04-24T23:30:22.205 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.198+0000 7fd07d5a4740 1 Processor -- start 2024-04-24T23:30:22.205 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.198+0000 7fd07d5a4740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cf79810240 0x55cf7981f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:22.205 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.198+0000 7fd07d5a4740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cf79810c30 0x55cf7981f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:22.205 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.198+0000 7fd07d5a4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cf79815da0 0x55cf79820710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:22.205 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.198+0000 7fd07d5a4740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fd0741a8a90 con 0x55cf79810240 2024-04-24T23:30:22.205 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.198+0000 7fd07d5a4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd0741a9f00 con 0x55cf79815da0 2024-04-24T23:30:22.205 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.198+0000 7fd07d5a4740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd0741a9160 con 0x55cf79810c30 2024-04-24T23:30:22.205 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.198+0000 7fd07d5a4740 10 client.0 _ll_register_callbacks cb 0x7fd06c009360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:30:22.206 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.198+0000 7fd073fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cf79810240 0x55cf7981f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:22.206 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.198+0000 7fd073fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cf79810240 0x55cf7981f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:47024/0 (socket says 172.21.15.80:47024) 2024-04-24T23:30:22.206 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.198+0000 7fd073fff640 1 -- 172.21.15.80:0/3610193421 learned_addr learned my addr 172.21.15.80:0/3610193421 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:30:22.206 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.198+0000 7fd0737fe640 1 --2- 172.21.15.80:0/3610193421 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cf79810c30 0x55cf7981f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:22.206 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.198+0000 7fd078e37640 1 --2- 172.21.15.80:0/3610193421 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cf79815da0 0x55cf79820710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:22.206 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.198+0000 7fd073fff640 1 -- 172.21.15.80:0/3610193421 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cf79810c30 msgr2=0x55cf7981f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:22.206 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.198+0000 7fd073fff640 1 --2- 172.21.15.80:0/3610193421 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cf79810c30 0x55cf7981f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:22.206 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.198+0000 7fd073fff640 1 -- 172.21.15.80:0/3610193421 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cf79815da0 msgr2=0x55cf79820710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:22.206 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.198+0000 7fd073fff640 1 --2- 172.21.15.80:0/3610193421 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cf79815da0 0x55cf79820710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:22.206 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.198+0000 7fd073fff640 1 -- 172.21.15.80:0/3610193421 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf79820d00 con 0x55cf79810240 2024-04-24T23:30:22.206 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.198+0000 7fd078e37640 1 --2- 172.21.15.80:0/3610193421 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cf79815da0 0x55cf79820710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:30:22.206 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.199+0000 7fd073fff640 1 --2- 172.21.15.80:0/3610193421 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cf79810240 0x55cf7981f1c0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fd06400b9c0 tx=0x7fd0640109f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a561273c8531e9c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:22.207 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.199+0000 7fd0537fe640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-04-24T23:30:22.207 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.199+0000 7fd0537fe640 1 -- 172.21.15.80:0/3610193421 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0640365c0 con 0x55cf79810240 2024-04-24T23:30:22.207 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.199+0000 7fd0537fe640 1 -- 172.21.15.80:0/3610193421 <== mon.0 v2:172.21.15.80:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fd06403f420 con 0x55cf79810240 2024-04-24T23:30:22.207 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.199+0000 7fd07d5a4740 1 -- 172.21.15.80:0/3610193421 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd06403b070 con 0x55cf79810240 2024-04-24T23:30:22.207 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.199+0000 7fd0537fe640 1 -- 172.21.15.80:0/3610193421 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd06404c050 con 0x55cf79810240 2024-04-24T23:30:22.207 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.199+0000 7fd07d5a4740 10 client.15753 fetch_fsmap learned FSMap version 25 2024-04-24T23:30:22.207 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.199+0000 7fd07d5a4740 1 -- 172.21.15.80:0/3610193421 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55cf7988ca60 con 0x55cf79810240 2024-04-24T23:30:22.207 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.199+0000 7fd0537fe640 1 -- 172.21.15.80:0/3610193421 <== mon.0 v2:172.21.15.80:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd06403b070 con 0x55cf79810240 2024-04-24T23:30:22.207 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.199+0000 7fd0537fe640 1 -- 172.21.15.80:0/3610193421 <== mon.0 v2:172.21.15.80:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fd064047030 con 0x55cf79810240 2024-04-24T23:30:22.207 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.199+0000 7fd07d5a4740 10 client.15753 fetch_fsmap finished waiting for FSMap version 25 2024-04-24T23:30:22.207 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.199+0000 7fd07d5a4740 10 client.15753 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:30:22.207 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.199+0000 7fd07d5a4740 20 client.15753 populate_metadata read hostname 'smithi080' 2024-04-24T23:30:22.207 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.199+0000 7fd07d5a4740 1 --2- 172.21.15.80:0/3610193421 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x55cf7988d1b0 0x55cf7988f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:22.208 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.199+0000 7fd07d5a4740 4 client.15753 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:30:22.208 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.199+0000 7fd07d5a4740 1 -- 172.21.15.80:0/3610193421 --> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55cf7977bf40 con 0x55cf7988d1b0 2024-04-24T23:30:22.208 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.200+0000 7fd0737fe640 1 --2- 172.21.15.80:0/3610193421 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x55cf7988d1b0 0x55cf7988f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:22.208 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.200+0000 7fd0737fe640 1 --2- 172.21.15.80:0/3610193421 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x55cf7988d1b0 0x55cf7988f5d0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5db0f9a624e9c455 server_cookie=4f4d0d0959f2f930 in_seq=0 out_seq=0 2024-04-24T23:30:22.208 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.200+0000 7fd0537fe640 10 client.15753 ms_handle_connect on v2:172.21.15.148:6838/2952757594 2024-04-24T23:30:22.208 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.200+0000 7fd0537fe640 1 -- 172.21.15.80:0/3610193421 <== mds.0 v2:172.21.15.148:6838/2952757594 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55cf7977bf40 con 0x55cf7988d1b0 2024-04-24T23:30:22.208 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.200+0000 7fd0537fe640 10 client.15753 handle_command_reply: tid=0 2024-04-24T23:30:22.208 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.200+0000 7fd07d5a4740 1 client.15753 shutdown 2024-04-24T23:30:22.208 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.200+0000 7fd07d5a4740 1 -- 172.21.15.80:0/3610193421 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cf79810240 msgr2=0x55cf7981f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:22.208 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.200+0000 7fd07d5a4740 1 --2- 172.21.15.80:0/3610193421 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cf79810240 0x55cf7981f1c0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fd06400b9c0 tx=0x7fd0640109f0 comp rx=0 tx=0).stop 2024-04-24T23:30:22.208 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.200+0000 7fd07d5a4740 1 -- 172.21.15.80:0/3610193421 shutdown_connections 2024-04-24T23:30:22.208 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.200+0000 7fd07d5a4740 1 --2- 172.21.15.80:0/3610193421 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x55cf7988d1b0 0x55cf7988f5d0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:22.208 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.200+0000 7fd07d5a4740 1 --2- 172.21.15.80:0/3610193421 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cf79815da0 0x55cf79820710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:22.208 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.200+0000 7fd0537fe640 0 client.15753 ms_handle_reset on v2:172.21.15.148:6838/2952757594 2024-04-24T23:30:22.208 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.200+0000 7fd07d5a4740 1 --2- 172.21.15.80:0/3610193421 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cf79810c30 0x55cf7981f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:22.209 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.200+0000 7fd07d5a4740 1 --2- 172.21.15.80:0/3610193421 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cf79810240 0x55cf7981f1c0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:22.209 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.200+0000 7fd07d5a4740 1 -- 172.21.15.80:0/3610193421 >> 172.21.15.80:0/3610193421 conn(0x55cf7980b8b0 msgr2=0x55cf7980a260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:22.209 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.200+0000 7fd07d5a4740 1 -- 172.21.15.80:0/3610193421 shutdown_connections 2024-04-24T23:30:22.209 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.200+0000 7fd07d5a4740 1 -- 172.21.15.80:0/3610193421 wait complete. 2024-04-24T23:30:22.209 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.200+0000 7fd07d5a4740 20 client.15753 trim_cache size 0 max 16384 2024-04-24T23:30:22.209 INFO:teuthology.orchestra.run.smithi080.stdout:{ 2024-04-24T23:30:22.209 INFO:teuthology.orchestra.run.smithi080.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T23:30:22.210 INFO:teuthology.orchestra.run.smithi080.stdout: "scrubs": { 2024-04-24T23:30:22.210 INFO:teuthology.orchestra.run.smithi080.stdout: "5152e04c-8be7-4907-8cb6-9f6c552edbec": { 2024-04-24T23:30:22.210 INFO:teuthology.orchestra.run.smithi080.stdout: "path": "/", 2024-04-24T23:30:22.210 INFO:teuthology.orchestra.run.smithi080.stdout: "tag": "5152e04c-8be7-4907-8cb6-9f6c552edbec", 2024-04-24T23:30:22.210 INFO:teuthology.orchestra.run.smithi080.stdout: "options": "recursive,force" 2024-04-24T23:30:22.210 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:30:22.210 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:30:22.210 INFO:teuthology.orchestra.run.smithi080.stdout:} 2024-04-24T23:30:22.211 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.201+0000 7fd07a89a640 1 -- 172.21.15.80:0/343565630 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fd05c0a2c10 msgr2=0x7fd05c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:22.211 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.201+0000 7fd07a89a640 1 --2- 172.21.15.80:0/343565630 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fd05c0a2c10 0x7fd05c0a50d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd0741016d0 tx=0x7fd06400b390 comp rx=0 tx=0).stop 2024-04-24T23:30:22.211 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.201+0000 7fd07a89a640 1 -- 172.21.15.80:0/343565630 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd074101870 msgr2=0x7fd07419c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:22.211 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.201+0000 7fd07a89a640 1 --2- 172.21.15.80:0/343565630 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd074101870 0x7fd07419c7d0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fd06800d770 tx=0x7fd06800dc30 comp rx=0 tx=0).stop 2024-04-24T23:30:22.211 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.201+0000 7fd07a89a640 1 -- 172.21.15.80:0/343565630 shutdown_connections 2024-04-24T23:30:22.211 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.201+0000 7fd07a89a640 1 --2- 172.21.15.80:0/343565630 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fd05c0a2c10 0x7fd05c0a50d0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:22.211 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.201+0000 7fd07a89a640 1 --2- 172.21.15.80:0/343565630 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd074102230 0x7fd07419edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:22.211 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.201+0000 7fd07a89a640 1 --2- 172.21.15.80:0/343565630 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd074101870 0x7fd07419c7d0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:22.212 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.201+0000 7fd07a89a640 1 --2- 172.21.15.80:0/343565630 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd074100680 0x7fd074069440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:22.212 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.201+0000 7fd07a89a640 1 -- 172.21.15.80:0/343565630 >> 172.21.15.80:0/343565630 conn(0x7fd0740fbbf0 msgr2=0x7fd074104c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:22.212 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.201+0000 7fd07a89a640 1 -- 172.21.15.80:0/343565630 shutdown_connections 2024-04-24T23:30:22.212 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:22.201+0000 7fd07a89a640 1 -- 172.21.15.80:0/343565630 wait complete. 2024-04-24T23:30:22.212 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:30:22 smithi080 ceph-mon[29152]: pgmap v391: 129 pgs: 129 active+clean; 650 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T23:30:22.213 INFO:tasks.cephfs.filesystem:scrub status for tag:5152e04c-8be7-4907-8cb6-9f6c552edbec - {'path': '/', 'tag': '5152e04c-8be7-4907-8cb6-9f6c552edbec', 'options': 'recursive,force'} 2024-04-24T23:30:22.446 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:30:22 smithi134 ceph-mon[32619]: pgmap v391: 129 pgs: 129 active+clean; 650 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T23:30:22.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:30:22 smithi148 ceph-mon[32528]: pgmap v391: 129 pgs: 129 active+clean; 650 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T23:30:23.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:30:23 smithi134 ceph-mon[32619]: scrub queued for path: / 2024-04-24T23:30:23.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:30:23 smithi134 ceph-mon[32619]: scrub summary: idle+waiting paths [/] 2024-04-24T23:30:23.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:30:23 smithi134 ceph-mon[32619]: scrub summary: active paths [/] 2024-04-24T23:30:23.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:30:23 smithi134 ceph-mon[32619]: scrub summary: idle+waiting paths [/] 2024-04-24T23:30:23.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:30:23 smithi148 ceph-mon[32528]: scrub queued for path: / 2024-04-24T23:30:23.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:30:23 smithi148 ceph-mon[32528]: scrub summary: idle+waiting paths [/] 2024-04-24T23:30:23.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:30:23 smithi148 ceph-mon[32528]: scrub summary: active paths [/] 2024-04-24T23:30:23.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:30:23 smithi148 ceph-mon[32528]: scrub summary: idle+waiting paths [/] 2024-04-24T23:30:23.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:30:23 smithi080 ceph-mon[29152]: scrub queued for path: / 2024-04-24T23:30:23.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:30:23 smithi080 ceph-mon[29152]: scrub summary: idle+waiting paths [/] 2024-04-24T23:30:23.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:30:23 smithi080 ceph-mon[29152]: scrub summary: active paths [/] 2024-04-24T23:30:23.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:30:23 smithi080 ceph-mon[29152]: scrub summary: idle+waiting paths [/] 2024-04-24T23:30:24.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:30:24 smithi134 ceph-mon[32619]: pgmap v392: 129 pgs: 129 active+clean; 650 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:30:24.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:30:24 smithi148 ceph-mon[32528]: pgmap v392: 129 pgs: 129 active+clean; 650 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:30:24.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:30:24 smithi080 ceph-mon[29152]: pgmap v392: 129 pgs: 129 active+clean; 650 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:30:25.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:30:25 smithi134 ceph-mon[32619]: scrub summary: idle 2024-04-24T23:30:25.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:30:25 smithi148 ceph-mon[32528]: scrub summary: idle 2024-04-24T23:30:25.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:30:25 smithi080 ceph-mon[29152]: scrub summary: idle 2024-04-24T23:30:26.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:30:26 smithi134 ceph-mon[32619]: pgmap v393: 129 pgs: 129 active+clean; 650 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-24T23:30:26.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:30:26 smithi148 ceph-mon[32528]: pgmap v393: 129 pgs: 129 active+clean; 650 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-24T23:30:26.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:30:26 smithi080 ceph-mon[29152]: pgmap v393: 129 pgs: 129 active+clean; 650 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-24T23:30:28.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:30:28 smithi134 ceph-mon[32619]: pgmap v394: 129 pgs: 129 active+clean; 650 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-24T23:30:28.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:30:28 smithi148 ceph-mon[32528]: pgmap v394: 129 pgs: 129 active+clean; 650 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-24T23:30:28.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:30:28 smithi080 ceph-mon[29152]: pgmap v394: 129 pgs: 129 active+clean; 650 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-24T23:30:30.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:30:30 smithi134 ceph-mon[32619]: pgmap v395: 129 pgs: 129 active+clean; 656 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 682 B/s wr, 7 op/s 2024-04-24T23:30:30.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:30:30 smithi148 ceph-mon[32528]: pgmap v395: 129 pgs: 129 active+clean; 656 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 682 B/s wr, 7 op/s 2024-04-24T23:30:30.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:30:30 smithi080 ceph-mon[29152]: pgmap v395: 129 pgs: 129 active+clean; 656 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 682 B/s wr, 7 op/s 2024-04-24T23:30:32.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:30:32 smithi134 ceph-mon[32619]: pgmap v396: 129 pgs: 129 active+clean; 656 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 682 B/s wr, 6 op/s 2024-04-24T23:30:32.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:30:32 smithi148 ceph-mon[32528]: pgmap v396: 129 pgs: 129 active+clean; 656 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 682 B/s wr, 6 op/s 2024-04-24T23:30:32.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:30:32 smithi080 ceph-mon[29152]: pgmap v396: 129 pgs: 129 active+clean; 656 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 682 B/s wr, 6 op/s 2024-04-24T23:30:34.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:30:34 smithi134 ceph-mon[32619]: pgmap v397: 129 pgs: 129 active+clean; 656 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 682 B/s wr, 5 op/s 2024-04-24T23:30:34.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:30:34 smithi148 ceph-mon[32528]: pgmap v397: 129 pgs: 129 active+clean; 656 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 682 B/s wr, 5 op/s 2024-04-24T23:30:34.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:30:34 smithi080 ceph-mon[29152]: pgmap v397: 129 pgs: 129 active+clean; 656 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 682 B/s wr, 5 op/s 2024-04-24T23:30:36.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:30:36 smithi134 ceph-mon[32619]: pgmap v398: 129 pgs: 129 active+clean; 656 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 682 B/s wr, 8 op/s 2024-04-24T23:30:36.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:30:36 smithi148 ceph-mon[32528]: pgmap v398: 129 pgs: 129 active+clean; 656 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 682 B/s wr, 8 op/s 2024-04-24T23:30:36.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:30:36 smithi080 ceph-mon[29152]: pgmap v398: 129 pgs: 129 active+clean; 656 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 682 B/s wr, 8 op/s 2024-04-24T23:30:38.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:30:38 smithi134 ceph-mon[32619]: pgmap v399: 129 pgs: 129 active+clean; 656 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 5 op/s 2024-04-24T23:30:38.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:30:38 smithi148 ceph-mon[32528]: pgmap v399: 129 pgs: 129 active+clean; 656 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 5 op/s 2024-04-24T23:30:38.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:30:38 smithi080 ceph-mon[29152]: pgmap v399: 129 pgs: 129 active+clean; 656 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 5 op/s 2024-04-24T23:30:40.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:30:40 smithi134 ceph-mon[32619]: pgmap v400: 129 pgs: 129 active+clean; 656 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 682 B/s wr, 7 op/s 2024-04-24T23:30:40.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:30:40 smithi148 ceph-mon[32528]: pgmap v400: 129 pgs: 129 active+clean; 656 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 682 B/s wr, 7 op/s 2024-04-24T23:30:40.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:30:40 smithi080 ceph-mon[29152]: pgmap v400: 129 pgs: 129 active+clean; 656 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 682 B/s wr, 7 op/s 2024-04-24T23:30:42.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:30:42 smithi134 ceph-mon[32619]: pgmap v401: 129 pgs: 129 active+clean; 656 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T23:30:42.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:30:42 smithi148 ceph-mon[32528]: pgmap v401: 129 pgs: 129 active+clean; 656 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T23:30:42.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:30:42 smithi080 ceph-mon[29152]: pgmap v401: 129 pgs: 129 active+clean; 656 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T23:30:44.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:30:44 smithi134 ceph-mon[32619]: pgmap v402: 129 pgs: 129 active+clean; 656 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:30:44.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:30:44 smithi148 ceph-mon[32528]: pgmap v402: 129 pgs: 129 active+clean; 656 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:30:44.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:30:44 smithi080 ceph-mon[29152]: pgmap v402: 129 pgs: 129 active+clean; 656 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:30:46.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:30:46 smithi134 ceph-mon[32619]: pgmap v403: 129 pgs: 129 active+clean; 656 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T23:30:46.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:30:46 smithi148 ceph-mon[32528]: pgmap v403: 129 pgs: 129 active+clean; 656 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T23:30:46.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:30:46 smithi080 ceph-mon[29152]: pgmap v403: 129 pgs: 129 active+clean; 656 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T23:30:48.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:30:48 smithi134 ceph-mon[32619]: pgmap v404: 129 pgs: 129 active+clean; 656 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:30:48.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:30:48 smithi148 ceph-mon[32528]: pgmap v404: 129 pgs: 129 active+clean; 656 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:30:48.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:30:48 smithi080 ceph-mon[29152]: pgmap v404: 129 pgs: 129 active+clean; 656 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:30:50.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:30:50 smithi134 ceph-mon[32619]: pgmap v405: 129 pgs: 129 active+clean; 656 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T23:30:50.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:30:50 smithi148 ceph-mon[32528]: pgmap v405: 129 pgs: 129 active+clean; 656 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T23:30:50.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:30:50 smithi080 ceph-mon[29152]: pgmap v405: 129 pgs: 129 active+clean; 656 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T23:30:52.214 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T23:30:52.215 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T23:30:52.332 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.330+0000 7f8f4afe7640 1 -- 172.21.15.80:0/2500187973 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8f44107460 msgr2=0x7f8f441098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:52.332 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.330+0000 7f8f4afe7640 1 --2- 172.21.15.80:0/2500187973 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8f44107460 0x7f8f441098b0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c009ca0 tx=0x7f8f3c02a8c0 comp rx=0 tx=0).stop 2024-04-24T23:30:52.332 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.330+0000 7f8f4afe7640 1 -- 172.21.15.80:0/2500187973 shutdown_connections 2024-04-24T23:30:52.332 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.330+0000 7f8f4afe7640 1 --2- 172.21.15.80:0/2500187973 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8f44107460 0x7f8f441098b0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:52.333 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.330+0000 7f8f4afe7640 1 --2- 172.21.15.80:0/2500187973 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8f44104b30 0x7f8f44106f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:52.333 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.330+0000 7f8f4afe7640 1 --2- 172.21.15.80:0/2500187973 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f44102200 0x7f8f441045f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:52.333 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.330+0000 7f8f4afe7640 1 -- 172.21.15.80:0/2500187973 >> 172.21.15.80:0/2500187973 conn(0x7f8f440fbbf0 msgr2=0x7f8f440fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:52.333 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.330+0000 7f8f4afe7640 1 -- 172.21.15.80:0/2500187973 shutdown_connections 2024-04-24T23:30:52.333 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.330+0000 7f8f4afe7640 1 -- 172.21.15.80:0/2500187973 wait complete. 2024-04-24T23:30:52.333 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.331+0000 7f8f4afe7640 1 Processor -- start 2024-04-24T23:30:52.333 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.331+0000 7f8f4afe7640 1 -- start start 2024-04-24T23:30:52.333 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.331+0000 7f8f4afe7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8f44102200 0x7f8f44197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:52.333 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.331+0000 7f8f4afe7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f44104b30 0x7f8f44197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:52.333 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.331+0000 7f8f4afe7640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8f44107460 0x7f8f4419f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:52.334 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.331+0000 7f8f4afe7640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f8f4410d420 con 0x7f8f44107460 2024-04-24T23:30:52.334 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.331+0000 7f8f4afe7640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8f4410d2a0 con 0x7f8f44102200 2024-04-24T23:30:52.334 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.331+0000 7f8f4afe7640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f8f4410d5a0 con 0x7f8f44104b30 2024-04-24T23:30:52.334 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.331+0000 7f8f49584640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8f44107460 0x7f8f4419f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:52.334 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.331+0000 7f8f49584640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8f44107460 0x7f8f4419f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:54966/0 (socket says 172.21.15.80:54966) 2024-04-24T23:30:52.334 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.331+0000 7f8f48d83640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8f44102200 0x7f8f44197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:52.334 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.331+0000 7f8f49584640 1 -- 172.21.15.80:0/1331431428 learned_addr learned my addr 172.21.15.80:0/1331431428 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:30:52.334 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.331+0000 7f8f43fff640 1 --2- 172.21.15.80:0/1331431428 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f44104b30 0x7f8f44197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:52.334 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.332+0000 7f8f48d83640 1 -- 172.21.15.80:0/1331431428 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f44104b30 msgr2=0x7f8f44197fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:52.334 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.332+0000 7f8f48d83640 1 --2- 172.21.15.80:0/1331431428 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f44104b30 0x7f8f44197fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:52.334 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.332+0000 7f8f48d83640 1 -- 172.21.15.80:0/1331431428 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8f44107460 msgr2=0x7f8f4419f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:52.334 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.332+0000 7f8f48d83640 1 --2- 172.21.15.80:0/1331431428 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8f44107460 0x7f8f4419f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:52.334 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.332+0000 7f8f48d83640 1 -- 172.21.15.80:0/1331431428 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f4419f6b0 con 0x7f8f44102200 2024-04-24T23:30:52.334 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.332+0000 7f8f49584640 1 --2- 172.21.15.80:0/1331431428 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8f44107460 0x7f8f4419f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:30:52.335 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.332+0000 7f8f48d83640 1 --2- 172.21.15.80:0/1331431428 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8f44102200 0x7f8f44197a90 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8f3800d720 tx=0x7f8f3800dbe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:52.335 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.332+0000 7f8f41ffb640 1 -- 172.21.15.80:0/1331431428 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f38017070 con 0x7f8f44102200 2024-04-24T23:30:52.335 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.332+0000 7f8f4afe7640 1 -- 172.21.15.80:0/1331431428 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f4419f9a0 con 0x7f8f44102200 2024-04-24T23:30:52.335 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.332+0000 7f8f41ffb640 1 -- 172.21.15.80:0/1331431428 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f3800de30 con 0x7f8f44102200 2024-04-24T23:30:52.335 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.332+0000 7f8f41ffb640 1 -- 172.21.15.80:0/1331431428 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f380047a0 con 0x7f8f44102200 2024-04-24T23:30:52.335 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.332+0000 7f8f4afe7640 1 -- 172.21.15.80:0/1331431428 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f4419fef0 con 0x7f8f44102200 2024-04-24T23:30:52.340 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.334+0000 7f8f41ffb640 1 -- 172.21.15.80:0/1331431428 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8f380099d0 con 0x7f8f44102200 2024-04-24T23:30:52.340 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.335+0000 7f8f41ffb640 1 --2- 172.21.15.80:0/1331431428 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f8f340a2c90 0x7f8f340a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:52.341 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.335+0000 7f8f41ffb640 1 -- 172.21.15.80:0/1331431428 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f8f380cefb0 con 0x7f8f44102200 2024-04-24T23:30:52.341 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.335+0000 7f8f43fff640 1 --2- 172.21.15.80:0/1331431428 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f8f340a2c90 0x7f8f340a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:52.341 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.335+0000 7f8f43fff640 1 --2- 172.21.15.80:0/1331431428 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f8f340a2c90 0x7f8f340a5150 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f8f44198fa0 tx=0x7f8f30005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:52.341 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.336+0000 7f8f4ddd5740 1 Processor -- start 2024-04-24T23:30:52.341 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.336+0000 7f8f4ddd5740 1 -- start start 2024-04-24T23:30:52.341 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.336+0000 7f8f4ddd5740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559802b95240 0x559802b95660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:52.341 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.336+0000 7f8f4ddd5740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559802b95c30 0x559802b9a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:52.341 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.336+0000 7f8f4ddd5740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559802b9ada0 0x559802b9d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:52.341 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.336+0000 7f8f43fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559802b95c30 0x559802b9a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:52.341 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.336+0000 7f8f43fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559802b95c30 0x559802b9a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.80:47624/0 (socket says 172.21.15.80:47624) 2024-04-24T23:30:52.341 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.336+0000 7f8f49584640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559802b9ada0 0x559802b9d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:52.341 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.336+0000 7f8f49584640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559802b9ada0 0x559802b9d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:54972/0 (socket says 172.21.15.80:54972) 2024-04-24T23:30:52.341 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.336+0000 7f8f43fff640 1 -- 172.21.15.80:0/3969006682 learned_addr learned my addr 172.21.15.80:0/3969006682 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:30:52.341 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.336+0000 7f8f4ddd5740 1 -- 172.21.15.80:0/3969006682 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x559802af96b0 con 0x559802b9ada0 2024-04-24T23:30:52.342 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.336+0000 7f8f4ddd5740 1 -- 172.21.15.80:0/3969006682 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x559802aa1880 con 0x559802b95c30 2024-04-24T23:30:52.342 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.336+0000 7f8f4ddd5740 1 -- 172.21.15.80:0/3969006682 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x559802af5670 con 0x559802b95240 2024-04-24T23:30:52.342 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.336+0000 7f8f48d83640 1 --2- 172.21.15.80:0/3969006682 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559802b95240 0x559802b95660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:52.342 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.337+0000 7f8f43fff640 1 -- 172.21.15.80:0/3969006682 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559802b95240 msgr2=0x559802b95660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:52.342 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.337+0000 7f8f43fff640 1 --2- 172.21.15.80:0/3969006682 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559802b95240 0x559802b95660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:52.342 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.337+0000 7f8f43fff640 1 -- 172.21.15.80:0/3969006682 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559802b9ada0 msgr2=0x559802b9d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:52.342 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.337+0000 7f8f43fff640 1 --2- 172.21.15.80:0/3969006682 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559802b9ada0 0x559802b9d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:52.342 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.337+0000 7f8f43fff640 1 -- 172.21.15.80:0/3969006682 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598028d77d0 con 0x559802b95c30 2024-04-24T23:30:52.342 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.337+0000 7f8f49584640 1 --2- 172.21.15.80:0/3969006682 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559802b9ada0 0x559802b9d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:30:52.342 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.337+0000 7f8f43fff640 1 --2- 172.21.15.80:0/3969006682 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559802b95c30 0x559802b9a730 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8f3000e630 tx=0x7f8f30014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8cd40b84b7327dcc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:52.342 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.337+0000 7f8f2effd640 1 -- 172.21.15.80:0/3969006682 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f30026430 con 0x559802b95c30 2024-04-24T23:30:52.342 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.337+0000 7f8f2effd640 1 -- 172.21.15.80:0/3969006682 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f3001c8f0 con 0x559802b95c30 2024-04-24T23:30:52.342 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.337+0000 7f8f2effd640 1 -- 172.21.15.80:0/3969006682 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f3001d9d0 con 0x559802b95c30 2024-04-24T23:30:52.342 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.338+0000 7f8f4ddd5740 1 -- 172.21.15.80:0/3969006682 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559802b95c30 msgr2=0x559802b9a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:52.343 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.338+0000 7f8f4ddd5740 1 --2- 172.21.15.80:0/3969006682 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559802b95c30 0x559802b9a730 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8f3000e630 tx=0x7f8f30014000 comp rx=0 tx=0).stop 2024-04-24T23:30:52.343 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.338+0000 7f8f4ddd5740 1 -- 172.21.15.80:0/3969006682 shutdown_connections 2024-04-24T23:30:52.343 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.338+0000 7f8f4ddd5740 1 --2- 172.21.15.80:0/3969006682 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559802b9ada0 0x559802b9d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:52.343 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.338+0000 7f8f4ddd5740 1 --2- 172.21.15.80:0/3969006682 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559802b95c30 0x559802b9a730 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:52.343 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.338+0000 7f8f4ddd5740 1 --2- 172.21.15.80:0/3969006682 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559802b95240 0x559802b95660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:52.343 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.338+0000 7f8f4ddd5740 1 -- 172.21.15.80:0/3969006682 >> 172.21.15.80:0/3969006682 conn(0x559802b908b0 msgr2=0x559802b90cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:52.343 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.338+0000 7f8f4ddd5740 1 -- 172.21.15.80:0/3969006682 shutdown_connections 2024-04-24T23:30:52.343 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.338+0000 7f8f4ddd5740 1 -- 172.21.15.80:0/3969006682 wait complete. 2024-04-24T23:30:52.343 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.338+0000 7f8f4ddd5740 1 -- start start 2024-04-24T23:30:52.343 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.338+0000 7f8f4ddd5740 1 Processor -- start 2024-04-24T23:30:52.343 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.339+0000 7f8f4ddd5740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559802b95240 0x559802b9f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:52.343 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.339+0000 7f8f4ddd5740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559802b95c30 0x559802b9fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:52.343 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.339+0000 7f8f4ddd5740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559802b9ada0 0x559802ba6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:52.343 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.339+0000 7f8f4ddd5740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x559802b9dd50 con 0x559802b95c30 2024-04-24T23:30:52.343 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.339+0000 7f8f4ddd5740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x559802ab4f50 con 0x559802b95240 2024-04-24T23:30:52.344 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.339+0000 7f8f43fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559802b95c30 0x559802b9fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:52.344 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.339+0000 7f8f43fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559802b95c30 0x559802b9fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:54984/0 (socket says 172.21.15.80:54984) 2024-04-24T23:30:52.344 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.339+0000 7f8f43fff640 1 -- 172.21.15.80:0/3033900381 learned_addr learned my addr 172.21.15.80:0/3033900381 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:30:52.344 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.339+0000 7f8f4ddd5740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x559802b9ded0 con 0x559802b9ada0 2024-04-24T23:30:52.344 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.339+0000 7f8f48d83640 1 --2- 172.21.15.80:0/3033900381 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559802b95240 0x559802b9f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:52.344 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.339+0000 7f8f4ddd5740 10 client.0 _ll_register_callbacks cb 0x559802af0000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:30:52.344 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.339+0000 7f8f48d83640 1 -- 172.21.15.80:0/3033900381 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559802b9ada0 msgr2=0x559802ba6c40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-24T23:30:52.344 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.339+0000 7f8f48d83640 1 --2- 172.21.15.80:0/3033900381 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559802b9ada0 0x559802ba6c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:52.344 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.339+0000 7f8f48d83640 1 -- 172.21.15.80:0/3033900381 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559802b95c30 msgr2=0x559802b9fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:52.344 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.339+0000 7f8f48d83640 1 --2- 172.21.15.80:0/3033900381 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559802b95c30 0x559802b9fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:52.344 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.339+0000 7f8f48d83640 1 -- 172.21.15.80:0/3033900381 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55980280d3f0 con 0x559802b95240 2024-04-24T23:30:52.344 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.339+0000 7f8f43fff640 1 --2- 172.21.15.80:0/3033900381 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559802b95c30 0x559802b9fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:30:52.344 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.339+0000 7f8f48d83640 1 --2- 172.21.15.80:0/3033900381 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559802b95240 0x559802b9f6c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8f38097840 tx=0x7f8f380998f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de6b333a2637330c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:52.344 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.339+0000 7f8f23fff640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-04-24T23:30:52.344 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.339+0000 7f8f23fff640 1 -- 172.21.15.80:0/3033900381 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f380b9060 con 0x559802b95240 2024-04-24T23:30:52.345 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.339+0000 7f8f23fff640 1 -- 172.21.15.80:0/3033900381 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f8f38009cc0 con 0x559802b95240 2024-04-24T23:30:52.345 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.339+0000 7f8f4ddd5740 1 -- 172.21.15.80:0/3033900381 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f3001c8f0 con 0x559802b95240 2024-04-24T23:30:52.345 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.340+0000 7f8f23fff640 1 -- 172.21.15.80:0/3033900381 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8f38081050 con 0x559802b95240 2024-04-24T23:30:52.345 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.340+0000 7f8f4ddd5740 10 client.25091 fetch_fsmap learned FSMap version 25 2024-04-24T23:30:52.345 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.340+0000 7f8f4ddd5740 1 -- 172.21.15.80:0/3033900381 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5598027cd0f0 con 0x559802b95240 2024-04-24T23:30:52.345 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.340+0000 7f8f23fff640 1 -- 172.21.15.80:0/3033900381 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f3001c8f0 con 0x559802b95240 2024-04-24T23:30:52.345 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.340+0000 7f8f23fff640 1 -- 172.21.15.80:0/3033900381 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f8f380ccb40 con 0x559802b95240 2024-04-24T23:30:52.345 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.340+0000 7f8f4ddd5740 10 client.25091 fetch_fsmap finished waiting for FSMap version 25 2024-04-24T23:30:52.345 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.340+0000 7f8f4ddd5740 10 client.25091 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:30:52.345 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.340+0000 7f8f4ddd5740 20 client.25091 populate_metadata read hostname 'smithi080' 2024-04-24T23:30:52.345 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.340+0000 7f8f4ddd5740 1 --2- 172.21.15.80:0/3033900381 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x559802bab540 0x559802bada00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:52.345 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.340+0000 7f8f4ddd5740 4 client.25091 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:30:52.345 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.340+0000 7f8f4ddd5740 1 -- 172.21.15.80:0/3033900381 --> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559802b06bf0 con 0x559802bab540 2024-04-24T23:30:52.345 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.340+0000 7f8f43fff640 1 --2- 172.21.15.80:0/3033900381 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x559802bab540 0x559802bada00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:52.346 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.341+0000 7f8f23fff640 10 client.25091 ms_handle_connect on v2:172.21.15.148:6838/2952757594 2024-04-24T23:30:52.346 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.341+0000 7f8f43fff640 1 --2- 172.21.15.80:0/3033900381 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x559802bab540 0x559802bada00 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f0daf065a8749a9 server_cookie=1c37ddc63ea30d7a in_seq=0 out_seq=0 2024-04-24T23:30:52.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.342+0000 7f8f23fff640 1 -- 172.21.15.80:0/3033900381 <== mds.0 v2:172.21.15.148:6838/2952757594 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559802b06bf0 con 0x559802bab540 2024-04-24T23:30:52.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.342+0000 7f8f23fff640 10 client.25091 handle_command_reply: tid=0 2024-04-24T23:30:52.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.342+0000 7f8f4ddd5740 1 client.25091 shutdown 2024-04-24T23:30:52.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.342+0000 7f8f4ddd5740 1 -- 172.21.15.80:0/3033900381 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559802b95240 msgr2=0x559802b9f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:52.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.342+0000 7f8f4ddd5740 1 --2- 172.21.15.80:0/3033900381 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559802b95240 0x559802b9f6c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8f38097840 tx=0x7f8f380998f0 comp rx=0 tx=0).stop 2024-04-24T23:30:52.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.342+0000 7f8f4ddd5740 1 -- 172.21.15.80:0/3033900381 shutdown_connections 2024-04-24T23:30:52.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.342+0000 7f8f4ddd5740 1 --2- 172.21.15.80:0/3033900381 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x559802bab540 0x559802bada00 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:52.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.342+0000 7f8f23fff640 0 client.25091 ms_handle_reset on v2:172.21.15.148:6838/2952757594 2024-04-24T23:30:52.348 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.342+0000 7f8f4ddd5740 1 --2- 172.21.15.80:0/3033900381 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559802b9ada0 0x559802ba6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:52.348 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.342+0000 7f8f4ddd5740 1 --2- 172.21.15.80:0/3033900381 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559802b95c30 0x559802b9fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:52.348 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.342+0000 7f8f4ddd5740 1 --2- 172.21.15.80:0/3033900381 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559802b95240 0x559802b9f6c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:52.348 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.342+0000 7f8f4ddd5740 1 -- 172.21.15.80:0/3033900381 >> 172.21.15.80:0/3033900381 conn(0x559802b908b0 msgr2=0x559802b8faf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:52.348 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.342+0000 7f8f4ddd5740 1 -- 172.21.15.80:0/3033900381 shutdown_connections 2024-04-24T23:30:52.348 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.342+0000 7f8f4ddd5740 1 -- 172.21.15.80:0/3033900381 wait complete. 2024-04-24T23:30:52.348 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.342+0000 7f8f4ddd5740 20 client.25091 trim_cache size 0 max 16384 2024-04-24T23:30:52.360 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.359+0000 7f8f4ddd5740 1 Processor -- start 2024-04-24T23:30:52.360 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.359+0000 7f8f4ddd5740 1 -- start start 2024-04-24T23:30:52.360 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.360+0000 7f8f4ddd5740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559802b95240 0x559802b9f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:52.360 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.360+0000 7f8f4ddd5740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559802b95c30 0x559802ba6c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:52.361 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.360+0000 7f8f4ddd5740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559802b9ada0 0x559802b93860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:52.361 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.360+0000 7f8f4ddd5740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f8f441aab90 con 0x559802b95240 2024-04-24T23:30:52.361 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.360+0000 7f8f4ddd5740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8f441a8460 con 0x559802b9ada0 2024-04-24T23:30:52.361 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.360+0000 7f8f4ddd5740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f8f441a62b0 con 0x559802b95c30 2024-04-24T23:30:52.361 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.360+0000 7f8f48d83640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559802b95240 0x559802b9f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:52.361 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.360+0000 7f8f48d83640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559802b95240 0x559802b9f9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:55000/0 (socket says 172.21.15.80:55000) 2024-04-24T23:30:52.361 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.360+0000 7f8f48d83640 1 -- 172.21.15.80:0/723886348 learned_addr learned my addr 172.21.15.80:0/723886348 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:30:52.361 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.360+0000 7f8f49584640 1 --2- 172.21.15.80:0/723886348 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559802b9ada0 0x559802b93860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:52.361 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.360+0000 7f8f43fff640 1 --2- 172.21.15.80:0/723886348 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559802b95c30 0x559802ba6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:52.361 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.360+0000 7f8f48d83640 1 -- 172.21.15.80:0/723886348 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559802b95c30 msgr2=0x559802ba6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:52.361 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.360+0000 7f8f48d83640 1 --2- 172.21.15.80:0/723886348 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559802b95c30 0x559802ba6c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:52.361 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.360+0000 7f8f48d83640 1 -- 172.21.15.80:0/723886348 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559802b9ada0 msgr2=0x559802b93860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:52.361 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.360+0000 7f8f48d83640 1 --2- 172.21.15.80:0/723886348 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559802b9ada0 0x559802b93860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:52.361 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.360+0000 7f8f48d83640 1 -- 172.21.15.80:0/723886348 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559802b82090 con 0x559802b95240 2024-04-24T23:30:52.361 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.360+0000 7f8f48d83640 1 --2- 172.21.15.80:0/723886348 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559802b95240 0x559802b9f9e0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f8f38005fd0 tx=0x7f8f3809bc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6f598100fd1f414 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:52.362 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.361+0000 7f8f2e7fc640 1 -- 172.21.15.80:0/723886348 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f380b9040 con 0x559802b95240 2024-04-24T23:30:52.362 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.361+0000 7f8f2e7fc640 1 -- 172.21.15.80:0/723886348 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f3809b3e0 con 0x559802b95240 2024-04-24T23:30:52.362 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.361+0000 7f8f2e7fc640 1 -- 172.21.15.80:0/723886348 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f38097bc0 con 0x559802b95240 2024-04-24T23:30:52.365 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.361+0000 7f8f4ddd5740 1 -- 172.21.15.80:0/723886348 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559802b95240 msgr2=0x559802b9f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:52.365 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.361+0000 7f8f4ddd5740 1 --2- 172.21.15.80:0/723886348 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559802b95240 0x559802b9f9e0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f8f38005fd0 tx=0x7f8f3809bc20 comp rx=0 tx=0).stop 2024-04-24T23:30:52.366 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.361+0000 7f8f4ddd5740 1 -- 172.21.15.80:0/723886348 shutdown_connections 2024-04-24T23:30:52.366 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.361+0000 7f8f4ddd5740 1 --2- 172.21.15.80:0/723886348 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559802b9ada0 0x559802b93860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:52.366 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.361+0000 7f8f4ddd5740 1 --2- 172.21.15.80:0/723886348 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559802b95c30 0x559802ba6c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:52.366 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.361+0000 7f8f4ddd5740 1 --2- 172.21.15.80:0/723886348 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559802b95240 0x559802b9f9e0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:52.366 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.361+0000 7f8f4ddd5740 1 -- 172.21.15.80:0/723886348 >> 172.21.15.80:0/723886348 conn(0x559802b908b0 msgr2=0x559802b9f2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:52.366 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.361+0000 7f8f4ddd5740 1 -- 172.21.15.80:0/723886348 shutdown_connections 2024-04-24T23:30:52.366 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.361+0000 7f8f4ddd5740 1 -- 172.21.15.80:0/723886348 wait complete. 2024-04-24T23:30:52.366 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.362+0000 7f8f4ddd5740 1 -- start start 2024-04-24T23:30:52.366 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.362+0000 7f8f4ddd5740 1 Processor -- start 2024-04-24T23:30:52.366 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.362+0000 7f8f4ddd5740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559802b95240 0x559802ba4440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:52.366 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.362+0000 7f8f4ddd5740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559802b95c30 0x559802ba4980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:52.366 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.362+0000 7f8f4ddd5740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559802b9ada0 0x559802ba5940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:52.366 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.362+0000 7f8f4ddd5740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f8f441a6920 con 0x559802b95c30 2024-04-24T23:30:52.366 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.362+0000 7f8f4ddd5740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8f441a7d90 con 0x559802b95240 2024-04-24T23:30:52.366 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.362+0000 7f8f4ddd5740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f8f441a6ff0 con 0x559802b9ada0 2024-04-24T23:30:52.367 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.362+0000 7f8f43fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559802b95c30 0x559802ba4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:52.367 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.362+0000 7f8f43fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559802b95c30 0x559802ba4980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:55014/0 (socket says 172.21.15.80:55014) 2024-04-24T23:30:52.367 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.362+0000 7f8f43fff640 1 -- 172.21.15.80:0/1362903590 learned_addr learned my addr 172.21.15.80:0/1362903590 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:30:52.367 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.362+0000 7f8f48d83640 1 --2- 172.21.15.80:0/1362903590 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559802b95240 0x559802ba4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:52.367 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.362+0000 7f8f4ddd5740 10 client.0 _ll_register_callbacks cb 0x7f8f3809b1b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:30:52.367 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.363+0000 7f8f49584640 1 --2- 172.21.15.80:0/1362903590 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559802b9ada0 0x559802ba5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:52.367 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.363+0000 7f8f43fff640 1 -- 172.21.15.80:0/1362903590 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559802b9ada0 msgr2=0x559802ba5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:52.367 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.363+0000 7f8f43fff640 1 --2- 172.21.15.80:0/1362903590 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559802b9ada0 0x559802ba5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:52.367 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.363+0000 7f8f43fff640 1 -- 172.21.15.80:0/1362903590 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559802b95240 msgr2=0x559802ba4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:52.367 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.363+0000 7f8f43fff640 1 --2- 172.21.15.80:0/1362903590 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559802b95240 0x559802ba4440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:52.367 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.363+0000 7f8f43fff640 1 -- 172.21.15.80:0/1362903590 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559802ba5f30 con 0x559802b95c30 2024-04-24T23:30:52.367 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.363+0000 7f8f48d83640 1 --2- 172.21.15.80:0/1362903590 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559802b95240 0x559802ba4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:30:52.367 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.363+0000 7f8f43fff640 1 --2- 172.21.15.80:0/1362903590 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559802b95c30 0x559802ba4980 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f8f3000e630 tx=0x7f8f30009370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bc65bd4c8554b2f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:52.367 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.363+0000 7f8f23fff640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-04-24T23:30:52.368 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.363+0000 7f8f23fff640 1 -- 172.21.15.80:0/1362903590 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f3003a060 con 0x559802b95c30 2024-04-24T23:30:52.368 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.363+0000 7f8f23fff640 1 -- 172.21.15.80:0/1362903590 <== mon.0 v2:172.21.15.80:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f8f30005240 con 0x559802b95c30 2024-04-24T23:30:52.368 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.363+0000 7f8f4ddd5740 1 -- 172.21.15.80:0/1362903590 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f3809b3e0 con 0x559802b95c30 2024-04-24T23:30:52.368 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.363+0000 7f8f23fff640 1 -- 172.21.15.80:0/1362903590 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8f30041050 con 0x559802b95c30 2024-04-24T23:30:52.368 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.363+0000 7f8f4ddd5740 10 client.15771 fetch_fsmap learned FSMap version 25 2024-04-24T23:30:52.368 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.363+0000 7f8f4ddd5740 1 -- 172.21.15.80:0/1362903590 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559802c1a4e0 con 0x559802b95c30 2024-04-24T23:30:52.368 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.363+0000 7f8f23fff640 1 -- 172.21.15.80:0/1362903590 <== mon.0 v2:172.21.15.80:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f30035e50 con 0x559802b95c30 2024-04-24T23:30:52.368 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.364+0000 7f8f23fff640 1 -- 172.21.15.80:0/1362903590 <== mon.0 v2:172.21.15.80:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f8f30004680 con 0x559802b95c30 2024-04-24T23:30:52.368 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.364+0000 7f8f4ddd5740 10 client.15771 fetch_fsmap finished waiting for FSMap version 25 2024-04-24T23:30:52.368 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.364+0000 7f8f4ddd5740 10 client.15771 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:30:52.368 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.364+0000 7f8f4ddd5740 20 client.15771 populate_metadata read hostname 'smithi080' 2024-04-24T23:30:52.368 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.364+0000 7f8f4ddd5740 1 --2- 172.21.15.80:0/1362903590 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x559802bab540 0x559802c1cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:52.368 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.364+0000 7f8f4ddd5740 4 client.15771 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:30:52.368 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.364+0000 7f8f4ddd5740 1 -- 172.21.15.80:0/1362903590 --> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f8f3809b3e0 con 0x559802bab540 2024-04-24T23:30:52.368 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.364+0000 7f8f48d83640 1 --2- 172.21.15.80:0/1362903590 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x559802bab540 0x559802c1cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:52.369 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.364+0000 7f8f48d83640 1 --2- 172.21.15.80:0/1362903590 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x559802bab540 0x559802c1cc40 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95e80fa9951aa8ce server_cookie=72f015fb14a789b2 in_seq=0 out_seq=0 2024-04-24T23:30:52.369 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.364+0000 7f8f23fff640 10 client.15771 ms_handle_connect on v2:172.21.15.148:6838/2952757594 2024-04-24T23:30:52.369 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.365+0000 7f8f23fff640 1 -- 172.21.15.80:0/1362903590 <== mds.0 v2:172.21.15.148:6838/2952757594 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7f8f3809b3e0 con 0x559802bab540 2024-04-24T23:30:52.369 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.365+0000 7f8f23fff640 10 client.15771 handle_command_reply: tid=0 2024-04-24T23:30:52.369 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.365+0000 7f8f4ddd5740 1 client.15771 shutdown 2024-04-24T23:30:52.369 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.365+0000 7f8f4ddd5740 1 -- 172.21.15.80:0/1362903590 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559802b95c30 msgr2=0x559802ba4980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:52.369 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.365+0000 7f8f4ddd5740 1 --2- 172.21.15.80:0/1362903590 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559802b95c30 0x559802ba4980 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f8f3000e630 tx=0x7f8f30009370 comp rx=0 tx=0).stop 2024-04-24T23:30:52.369 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.365+0000 7f8f4ddd5740 1 -- 172.21.15.80:0/1362903590 shutdown_connections 2024-04-24T23:30:52.369 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.365+0000 7f8f4ddd5740 1 --2- 172.21.15.80:0/1362903590 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x559802bab540 0x559802c1cc40 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:52.369 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.365+0000 7f8f4ddd5740 1 --2- 172.21.15.80:0/1362903590 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559802b9ada0 0x559802ba5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:52.369 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.365+0000 7f8f4ddd5740 1 --2- 172.21.15.80:0/1362903590 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559802b95c30 0x559802ba4980 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:52.369 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.365+0000 7f8f23fff640 0 client.15771 ms_handle_reset on v2:172.21.15.148:6838/2952757594 2024-04-24T23:30:52.369 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.365+0000 7f8f4ddd5740 1 --2- 172.21.15.80:0/1362903590 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559802b95240 0x559802ba4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:52.369 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.365+0000 7f8f4ddd5740 1 -- 172.21.15.80:0/1362903590 >> 172.21.15.80:0/1362903590 conn(0x559802b908b0 msgr2=0x559802b8f260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:52.369 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.365+0000 7f8f4ddd5740 1 -- 172.21.15.80:0/1362903590 shutdown_connections 2024-04-24T23:30:52.370 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.365+0000 7f8f4ddd5740 1 -- 172.21.15.80:0/1362903590 wait complete. 2024-04-24T23:30:52.370 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.365+0000 7f8f4ddd5740 20 client.15771 trim_cache size 0 max 16384 2024-04-24T23:30:52.370 INFO:teuthology.orchestra.run.smithi080.stdout:{ 2024-04-24T23:30:52.370 INFO:teuthology.orchestra.run.smithi080.stdout: "status": "no active scrubs running", 2024-04-24T23:30:52.370 INFO:teuthology.orchestra.run.smithi080.stdout: "scrubs": {} 2024-04-24T23:30:52.370 INFO:teuthology.orchestra.run.smithi080.stdout:} 2024-04-24T23:30:52.371 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.366+0000 7f8f4afe7640 1 -- 172.21.15.80:0/1331431428 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f8f340a2c90 msgr2=0x7f8f340a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:52.371 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.366+0000 7f8f4afe7640 1 --2- 172.21.15.80:0/1331431428 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f8f340a2c90 0x7f8f340a5150 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f8f44198fa0 tx=0x7f8f30005f90 comp rx=0 tx=0).stop 2024-04-24T23:30:52.371 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.366+0000 7f8f4afe7640 1 -- 172.21.15.80:0/1331431428 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8f44102200 msgr2=0x7f8f44197a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:52.371 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.366+0000 7f8f4afe7640 1 --2- 172.21.15.80:0/1331431428 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8f44102200 0x7f8f44197a90 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8f3800d720 tx=0x7f8f3800dbe0 comp rx=0 tx=0).stop 2024-04-24T23:30:52.371 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.366+0000 7f8f4afe7640 1 -- 172.21.15.80:0/1331431428 shutdown_connections 2024-04-24T23:30:52.371 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.366+0000 7f8f4afe7640 1 --2- 172.21.15.80:0/1331431428 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f8f340a2c90 0x7f8f340a5150 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:52.371 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.366+0000 7f8f4afe7640 1 --2- 172.21.15.80:0/1331431428 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8f44107460 0x7f8f4419f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:52.371 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.366+0000 7f8f4afe7640 1 --2- 172.21.15.80:0/1331431428 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f44104b30 0x7f8f44197fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:52.371 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.366+0000 7f8f4afe7640 1 --2- 172.21.15.80:0/1331431428 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8f44102200 0x7f8f44197a90 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:52.371 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.366+0000 7f8f4afe7640 1 -- 172.21.15.80:0/1331431428 >> 172.21.15.80:0/1331431428 conn(0x7f8f440fbbf0 msgr2=0x7f8f44102b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:52.372 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.366+0000 7f8f4afe7640 1 -- 172.21.15.80:0/1331431428 shutdown_connections 2024-04-24T23:30:52.372 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.366+0000 7f8f4afe7640 1 -- 172.21.15.80:0/1331431428 wait complete. 2024-04-24T23:30:52.376 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T23:30:52.376 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:30:52.453 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:30:52 smithi134 ceph-mon[32619]: pgmap v406: 129 pgs: 129 active+clean; 656 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T23:30:52.502 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.500+0000 7f5a23716640 1 -- 172.21.15.80:0/4110888364 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5a1c101870 msgr2=0x7f5a1c101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:52.502 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.500+0000 7f5a23716640 1 --2- 172.21.15.80:0/4110888364 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5a1c101870 0x7f5a1c101cf0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c009900 tx=0x7f5a0c02a410 comp rx=0 tx=0).stop 2024-04-24T23:30:52.502 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.500+0000 7f5a23716640 1 -- 172.21.15.80:0/4110888364 shutdown_connections 2024-04-24T23:30:52.502 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.500+0000 7f5a23716640 1 --2- 172.21.15.80:0/4110888364 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a1c102230 0x7f5a1c108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:52.502 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.500+0000 7f5a23716640 1 --2- 172.21.15.80:0/4110888364 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5a1c101870 0x7f5a1c101cf0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:52.502 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.500+0000 7f5a23716640 1 --2- 172.21.15.80:0/4110888364 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a1c100680 0x7f5a1c100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:52.502 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.500+0000 7f5a23716640 1 -- 172.21.15.80:0/4110888364 >> 172.21.15.80:0/4110888364 conn(0x7f5a1c0fbbf0 msgr2=0x7f5a1c0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:52.502 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.500+0000 7f5a23716640 1 -- 172.21.15.80:0/4110888364 shutdown_connections 2024-04-24T23:30:52.502 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.500+0000 7f5a23716640 1 -- 172.21.15.80:0/4110888364 wait complete. 2024-04-24T23:30:52.502 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.501+0000 7f5a23716640 1 Processor -- start 2024-04-24T23:30:52.502 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.501+0000 7f5a23716640 1 -- start start 2024-04-24T23:30:52.502 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.501+0000 7f5a23716640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a1c100680 0x7f5a1c069440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:52.503 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.501+0000 7f5a23716640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5a1c101870 0x7f5a1c19ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:52.503 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.501+0000 7f5a23716640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a1c102230 0x7f5a1c1a1160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:52.503 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.501+0000 7f5a23716640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f5a1c10c350 con 0x7f5a1c101870 2024-04-24T23:30:52.503 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.501+0000 7f5a23716640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5a1c10c1d0 con 0x7f5a1c100680 2024-04-24T23:30:52.503 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.501+0000 7f5a23716640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5a1c10c4d0 con 0x7f5a1c102230 2024-04-24T23:30:52.503 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.501+0000 7f5a20cb1640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5a1c101870 0x7f5a1c19ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:52.503 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.501+0000 7f5a214b2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a1c100680 0x7f5a1c069440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:52.503 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.501+0000 7f5a20cb1640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5a1c101870 0x7f5a1c19ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:55186/0 (socket says 172.21.15.80:55186) 2024-04-24T23:30:52.503 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.501+0000 7f5a20cb1640 1 -- 172.21.15.80:0/1547840007 learned_addr learned my addr 172.21.15.80:0/1547840007 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:30:52.503 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.501+0000 7f5a214b2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a1c100680 0x7f5a1c069440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.80:57658/0 (socket says 172.21.15.80:57658) 2024-04-24T23:30:52.503 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.502+0000 7f5a20cb1640 1 -- 172.21.15.80:0/1547840007 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a1c102230 msgr2=0x7f5a1c1a1160 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-24T23:30:52.503 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.502+0000 7f5a20cb1640 1 --2- 172.21.15.80:0/1547840007 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a1c102230 0x7f5a1c1a1160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:52.503 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.502+0000 7f5a20cb1640 1 -- 172.21.15.80:0/1547840007 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a1c100680 msgr2=0x7f5a1c069440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:52.503 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.502+0000 7f5a20cb1640 1 --2- 172.21.15.80:0/1547840007 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a1c100680 0x7f5a1c069440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:52.504 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.502+0000 7f5a20cb1640 1 -- 172.21.15.80:0/1547840007 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a1c1a1950 con 0x7f5a1c101870 2024-04-24T23:30:52.504 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.502+0000 7f5a20cb1640 1 --2- 172.21.15.80:0/1547840007 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5a1c101870 0x7f5a1c19ec10 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c031440 tx=0x7f5a0c032da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:52.504 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.502+0000 7f5a127fc640 1 -- 172.21.15.80:0/1547840007 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a0c03f030 con 0x7f5a1c101870 2024-04-24T23:30:52.504 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.502+0000 7f5a127fc640 1 -- 172.21.15.80:0/1547840007 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a0c003ce0 con 0x7f5a1c101870 2024-04-24T23:30:52.504 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.502+0000 7f5a23716640 1 -- 172.21.15.80:0/1547840007 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a1c1a1b80 con 0x7f5a1c101870 2024-04-24T23:30:52.504 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.502+0000 7f5a127fc640 1 -- 172.21.15.80:0/1547840007 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a0c035070 con 0x7f5a1c101870 2024-04-24T23:30:52.504 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.502+0000 7f5a23716640 1 -- 172.21.15.80:0/1547840007 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a1c1a20c0 con 0x7f5a1c101870 2024-04-24T23:30:52.504 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.503+0000 7f5a23716640 1 -- 172.21.15.80:0/1547840007 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a1c1079a0 con 0x7f5a1c101870 2024-04-24T23:30:52.505 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.505+0000 7f5a127fc640 1 -- 172.21.15.80:0/1547840007 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5a0c042030 con 0x7f5a1c101870 2024-04-24T23:30:52.505 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.506+0000 7f5a127fc640 1 --2- 172.21.15.80:0/1547840007 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f5a080a2cf0 0x7f5a080a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:52.505 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.506+0000 7f5a127fc640 1 -- 172.21.15.80:0/1547840007 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f5a0c0efa70 con 0x7f5a1c101870 2024-04-24T23:30:52.505 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.506+0000 7f5a214b2640 1 --2- 172.21.15.80:0/1547840007 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f5a080a2cf0 0x7f5a080a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:52.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:30:52 smithi148 ceph-mon[32528]: pgmap v406: 129 pgs: 129 active+clean; 656 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T23:30:52.506 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.506+0000 7f5a214b2640 1 --2- 172.21.15.80:0/1547840007 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f5a080a2cf0 0x7f5a080a51b0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f5a18005fd0 tx=0x7f5a1800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:52.510 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.510+0000 7f5a127fc640 1 -- 172.21.15.80:0/1547840007 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f5a0c0bc5a0 con 0x7f5a1c101870 2024-04-24T23:30:52.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:30:52 smithi080 ceph-mon[29152]: pgmap v406: 129 pgs: 129 active+clean; 656 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T23:30:52.777 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.777+0000 7f5a23716640 1 -- 172.21.15.80:0/1547840007 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5a1c062340 con 0x7f5a1c101870 2024-04-24T23:30:52.778 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.778+0000 7f5a127fc640 1 -- 172.21.15.80:0/1547840007 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11872 (secure 0 0 0) 0x7f5a0c0bbea0 con 0x7f5a1c101870 2024-04-24T23:30:52.778 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-24T23:30:52.779 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/891591085","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":891591085},{"type":"v1","addr":"172.21.15.80:6835","nonce":891591085}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6837/259529026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6836","nonce":259529026},{"type":"v1","addr":"172.21.15.80:6837","nonce":259529026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14646,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6839/4194284854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6838","nonce":4194284854},{"type":"v1","addr":"172.21.15.80:6839","nonce":4194284854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14658,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6841/3145730958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6840","nonce":3145730958},{"type":"v1","addr":"172.21.15.80:6841","nonce":3145730958}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24395,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/3736784385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3736784385},{"type":"v1","addr":"172.21.15.134:6833","nonce":3736784385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/124777462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":124777462},{"type":"v1","addr":"172.21.15.148:6833","nonce":124777462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T23:27:19.300566+0000","modified":"2024-04-24T23:29:42.158567+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24496,"mds_1":24488,"mds_2":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/1399801120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1399801120},{"type":"v1","addr":"172.21.15.134:6835","nonce":1399801120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/430512704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":430512704},{"type":"v1","addr":"172.21.15.134:6837","nonce":430512704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/3002319310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3002319310},{"type":"v1","addr":"172.21.15.148:6835","nonce":3002319310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.148:6837/856528403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":856528403},{"type":"v1","addr":"172.21.15.148:6837","nonce":856528403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6839/1743815021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":1743815021},{"type":"v1","addr":"172.21.15.134:6839","nonce":1743815021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.148:6839/2952757594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":2952757594},{"type":"v1","addr":"172.21.15.148:6839","nonce":2952757594}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24496,"qdb_cluster":[24496,24472,24488]},"id":1}]} 2024-04-24T23:30:52.779 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 25 2024-04-24T23:30:52.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.781+0000 7f5a23716640 1 -- 172.21.15.80:0/1547840007 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f5a080a2cf0 msgr2=0x7f5a080a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:52.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.781+0000 7f5a23716640 1 --2- 172.21.15.80:0/1547840007 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f5a080a2cf0 0x7f5a080a51b0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f5a18005fd0 tx=0x7f5a1800b390 comp rx=0 tx=0).stop 2024-04-24T23:30:52.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.781+0000 7f5a23716640 1 -- 172.21.15.80:0/1547840007 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5a1c101870 msgr2=0x7f5a1c19ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:52.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.781+0000 7f5a23716640 1 --2- 172.21.15.80:0/1547840007 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5a1c101870 0x7f5a1c19ec10 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c031440 tx=0x7f5a0c032da0 comp rx=0 tx=0).stop 2024-04-24T23:30:52.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.781+0000 7f5a23716640 1 -- 172.21.15.80:0/1547840007 shutdown_connections 2024-04-24T23:30:52.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.781+0000 7f5a23716640 1 --2- 172.21.15.80:0/1547840007 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f5a080a2cf0 0x7f5a080a51b0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:52.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.781+0000 7f5a23716640 1 --2- 172.21.15.80:0/1547840007 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a1c102230 0x7f5a1c1a1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:52.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.781+0000 7f5a23716640 1 --2- 172.21.15.80:0/1547840007 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5a1c101870 0x7f5a1c19ec10 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:52.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.781+0000 7f5a23716640 1 --2- 172.21.15.80:0/1547840007 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a1c100680 0x7f5a1c069440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:52.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.781+0000 7f5a23716640 1 -- 172.21.15.80:0/1547840007 >> 172.21.15.80:0/1547840007 conn(0x7f5a1c0fbbf0 msgr2=0x7f5a1c104b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:52.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.781+0000 7f5a23716640 1 -- 172.21.15.80:0/1547840007 shutdown_connections 2024-04-24T23:30:52.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.781+0000 7f5a23716640 1 -- 172.21.15.80:0/1547840007 wait complete. 2024-04-24T23:30:52.795 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T23:30:52.917 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.917+0000 7f6fe20a5640 1 -- 172.21.15.80:0/4101240594 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6fdc101790 msgr2=0x7f6fdc101bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:52.917 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.917+0000 7f6fe20a5640 1 --2- 172.21.15.80:0/4101240594 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6fdc101790 0x7f6fdc101bb0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc009900 tx=0x7f6fcc02a410 comp rx=0 tx=0).stop 2024-04-24T23:30:52.917 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.917+0000 7f6fe20a5640 1 -- 172.21.15.80:0/4101240594 shutdown_connections 2024-04-24T23:30:52.917 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.917+0000 7f6fe20a5640 1 --2- 172.21.15.80:0/4101240594 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6fdc103340 0x7f6fdc109bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:52.917 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.917+0000 7f6fe20a5640 1 --2- 172.21.15.80:0/4101240594 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6fdc102980 0x7f6fdc102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:52.917 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.917+0000 7f6fe20a5640 1 --2- 172.21.15.80:0/4101240594 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6fdc101790 0x7f6fdc101bb0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:52.917 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.917+0000 7f6fe20a5640 1 -- 172.21.15.80:0/4101240594 >> 172.21.15.80:0/4101240594 conn(0x7f6fdc0fcd20 msgr2=0x7f6fdc0ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:52.917 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.917+0000 7f6fe20a5640 1 -- 172.21.15.80:0/4101240594 shutdown_connections 2024-04-24T23:30:52.917 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.917+0000 7f6fe20a5640 1 -- 172.21.15.80:0/4101240594 wait complete. 2024-04-24T23:30:52.917 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.917+0000 7f6fe20a5640 1 Processor -- start 2024-04-24T23:30:52.919 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.918+0000 7f6fe20a5640 1 -- start start 2024-04-24T23:30:52.920 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.918+0000 7f6fe20a5640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6fdc101790 0x7f6fdc197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:52.920 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.918+0000 7f6fe20a5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6fdc102980 0x7f6fdc197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:52.920 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.918+0000 7f6fe20a5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6fdc103340 0x7f6fdc19f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:52.920 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.918+0000 7f6fe20a5640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f6fdc10d410 con 0x7f6fdc101790 2024-04-24T23:30:52.920 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.918+0000 7f6fe20a5640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6fdc10d290 con 0x7f6fdc102980 2024-04-24T23:30:52.920 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.918+0000 7f6fe20a5640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f6fdc10d590 con 0x7f6fdc103340 2024-04-24T23:30:52.920 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.918+0000 7f6fdaffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6fdc102980 0x7f6fdc197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:52.920 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.918+0000 7f6fdaffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6fdc102980 0x7f6fdc197fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.80:57676/0 (socket says 172.21.15.80:57676) 2024-04-24T23:30:52.921 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.918+0000 7f6fdaffd640 1 -- 172.21.15.80:0/203237058 learned_addr learned my addr 172.21.15.80:0/203237058 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:30:52.921 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.918+0000 7f6fdb7fe640 1 --2- 172.21.15.80:0/203237058 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6fdc101790 0x7f6fdc197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:52.921 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.918+0000 7f6fdbfff640 1 --2- 172.21.15.80:0/203237058 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6fdc103340 0x7f6fdc19f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:52.921 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.918+0000 7f6fdaffd640 1 -- 172.21.15.80:0/203237058 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6fdc103340 msgr2=0x7f6fdc19f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:52.921 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.918+0000 7f6fdaffd640 1 --2- 172.21.15.80:0/203237058 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6fdc103340 0x7f6fdc19f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:52.921 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.918+0000 7f6fdaffd640 1 -- 172.21.15.80:0/203237058 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6fdc101790 msgr2=0x7f6fdc197a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:52.921 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.918+0000 7f6fdaffd640 1 --2- 172.21.15.80:0/203237058 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6fdc101790 0x7f6fdc197a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:52.921 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.918+0000 7f6fdaffd640 1 -- 172.21.15.80:0/203237058 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fdc19f690 con 0x7f6fdc102980 2024-04-24T23:30:52.921 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.918+0000 7f6fdb7fe640 1 --2- 172.21.15.80:0/203237058 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6fdc101790 0x7f6fdc197a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:30:52.921 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.919+0000 7f6fdaffd640 1 --2- 172.21.15.80:0/203237058 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6fdc102980 0x7f6fdc197fb0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6fd000e9f0 tx=0x7f6fd000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:52.922 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.919+0000 7f6fd8ff9640 1 -- 172.21.15.80:0/203237058 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fd000cd80 con 0x7f6fdc102980 2024-04-24T23:30:52.922 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.919+0000 7f6fe20a5640 1 -- 172.21.15.80:0/203237058 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fdc19f980 con 0x7f6fdc102980 2024-04-24T23:30:52.922 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.919+0000 7f6fe20a5640 1 -- 172.21.15.80:0/203237058 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fdc19ff20 con 0x7f6fdc102980 2024-04-24T23:30:52.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.919+0000 7f6fd8ff9640 1 -- 172.21.15.80:0/203237058 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6fd0010910 con 0x7f6fdc102980 2024-04-24T23:30:52.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.919+0000 7f6fd8ff9640 1 -- 172.21.15.80:0/203237058 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fd00188f0 con 0x7f6fdc102980 2024-04-24T23:30:52.930 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.921+0000 7f6fd8ff9640 1 -- 172.21.15.80:0/203237058 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6fd0018bd0 con 0x7f6fdc102980 2024-04-24T23:30:52.930 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.922+0000 7f6fd8ff9640 1 --2- 172.21.15.80:0/203237058 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f6fc40a2c10 0x7f6fc40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:52.930 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.922+0000 7f6fe4dbb740 1 Processor -- start 2024-04-24T23:30:52.930 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.922+0000 7f6fdb7fe640 1 --2- 172.21.15.80:0/203237058 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f6fc40a2c10 0x7f6fc40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:52.930 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.922+0000 7f6fd8ff9640 1 -- 172.21.15.80:0/203237058 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f6fd00cf7e0 con 0x7f6fdc102980 2024-04-24T23:30:52.930 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.922+0000 7f6fe4dbb740 1 -- start start 2024-04-24T23:30:52.930 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.923+0000 7f6fe4dbb740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55edf0616230 0x55edf0616650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:52.930 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.923+0000 7f6fe4dbb740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55edf0616c20 0x55edf061b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:52.931 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.923+0000 7f6fe4dbb740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55edf061bd90 0x55edf061e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:52.931 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.923+0000 7f6fe4dbb740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x55edf057a670 con 0x55edf0616c20 2024-04-24T23:30:52.931 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.923+0000 7f6fe4dbb740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55edf0522970 con 0x55edf061bd90 2024-04-24T23:30:52.931 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.923+0000 7f6fe4dbb740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55edf0576630 con 0x55edf0616230 2024-04-24T23:30:52.931 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.923+0000 7f6fdb7fe640 1 --2- 172.21.15.80:0/203237058 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f6fc40a2c10 0x7f6fc40a50d0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc006010 tx=0x7f6fcc032cf0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:52.931 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.923+0000 7f6fdbfff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55edf061bd90 0x55edf061e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:52.931 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.923+0000 7f6fdaffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55edf0616c20 0x55edf061b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:52.931 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.923+0000 7f6fdbfff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55edf061bd90 0x55edf061e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.80:57678/0 (socket says 172.21.15.80:57678) 2024-04-24T23:30:52.931 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.923+0000 7f6fdbfff640 1 -- 172.21.15.80:0/2929280829 learned_addr learned my addr 172.21.15.80:0/2929280829 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:30:52.931 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.923+0000 7f6fdb7fe640 1 --2- 172.21.15.80:0/2929280829 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55edf0616230 0x55edf0616650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:52.931 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.923+0000 7f6fdbfff640 1 -- 172.21.15.80:0/2929280829 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55edf0616230 msgr2=0x55edf0616650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:52.931 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.923+0000 7f6fdbfff640 1 --2- 172.21.15.80:0/2929280829 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55edf0616230 0x55edf0616650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:52.931 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.923+0000 7f6fdbfff640 1 -- 172.21.15.80:0/2929280829 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55edf0616c20 msgr2=0x55edf061b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:52.931 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.923+0000 7f6fdbfff640 1 --2- 172.21.15.80:0/2929280829 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55edf0616c20 0x55edf061b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:52.932 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.923+0000 7f6fdbfff640 1 -- 172.21.15.80:0/2929280829 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55edf0546400 con 0x55edf061bd90 2024-04-24T23:30:52.932 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.923+0000 7f6fdb7fe640 1 --2- 172.21.15.80:0/2929280829 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55edf0616230 0x55edf0616650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:30:52.932 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.923+0000 7f6fdaffd640 1 --2- 172.21.15.80:0/2929280829 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55edf0616c20 0x55edf061b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:30:52.932 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.923+0000 7f6fdbfff640 1 --2- 172.21.15.80:0/2929280829 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55edf061bd90 0x55edf061e220 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6fd4009eb0 tx=0x7f6fd400f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=238edb9961ab6cff server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:52.932 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.924+0000 7f6fc1ffb640 1 -- 172.21.15.80:0/2929280829 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fd4011640 con 0x55edf061bd90 2024-04-24T23:30:52.932 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.924+0000 7f6fc1ffb640 1 -- 172.21.15.80:0/2929280829 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6fd4011cc0 con 0x55edf061bd90 2024-04-24T23:30:52.932 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.924+0000 7f6fc1ffb640 1 -- 172.21.15.80:0/2929280829 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fd401bd80 con 0x55edf061bd90 2024-04-24T23:30:52.932 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.924+0000 7f6fe4dbb740 1 -- 172.21.15.80:0/2929280829 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55edf061bd90 msgr2=0x55edf061e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:52.932 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.924+0000 7f6fe4dbb740 1 --2- 172.21.15.80:0/2929280829 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55edf061bd90 0x55edf061e220 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6fd4009eb0 tx=0x7f6fd400f730 comp rx=0 tx=0).stop 2024-04-24T23:30:52.932 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.924+0000 7f6fe4dbb740 1 -- 172.21.15.80:0/2929280829 shutdown_connections 2024-04-24T23:30:52.932 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.924+0000 7f6fe4dbb740 1 --2- 172.21.15.80:0/2929280829 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55edf061bd90 0x55edf061e220 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:52.932 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.924+0000 7f6fe4dbb740 1 --2- 172.21.15.80:0/2929280829 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55edf0616c20 0x55edf061b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:52.932 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.924+0000 7f6fe4dbb740 1 --2- 172.21.15.80:0/2929280829 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55edf0616230 0x55edf0616650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:52.933 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.924+0000 7f6fe4dbb740 1 -- 172.21.15.80:0/2929280829 >> 172.21.15.80:0/2929280829 conn(0x55edf06118a0 msgr2=0x55edf0611ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:52.933 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.924+0000 7f6fe4dbb740 1 -- 172.21.15.80:0/2929280829 shutdown_connections 2024-04-24T23:30:52.933 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.924+0000 7f6fe4dbb740 1 -- 172.21.15.80:0/2929280829 wait complete. 2024-04-24T23:30:52.933 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.925+0000 7f6fe4dbb740 1 -- start start 2024-04-24T23:30:52.933 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.925+0000 7f6fe4dbb740 1 Processor -- start 2024-04-24T23:30:52.933 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.925+0000 7f6fe4dbb740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55edf0616230 0x55edf0619080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:52.934 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.925+0000 7f6fe4dbb740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55edf0616c20 0x55edf06195c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:52.934 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.925+0000 7f6fe4dbb740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55edf061bd90 0x55edf061a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:52.934 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.925+0000 7f6fe4dbb740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x55edf061eb20 con 0x55edf0616230 2024-04-24T23:30:52.934 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.925+0000 7f6fe4dbb740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55edf0536050 con 0x55edf0616c20 2024-04-24T23:30:52.934 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.925+0000 7f6fe4dbb740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55edf061eca0 con 0x55edf061bd90 2024-04-24T23:30:52.934 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.925+0000 7f6fdb7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55edf0616230 0x55edf0619080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:52.934 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.925+0000 7f6fdb7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55edf0616230 0x55edf0619080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:55230/0 (socket says 172.21.15.80:55230) 2024-04-24T23:30:52.934 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.925+0000 7f6fdb7fe640 1 -- 172.21.15.80:0/2855000202 learned_addr learned my addr 172.21.15.80:0/2855000202 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:30:52.934 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.925+0000 7f6fdaffd640 1 --2- 172.21.15.80:0/2855000202 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55edf0616c20 0x55edf06195c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:52.934 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.925+0000 7f6fe4dbb740 10 client.0 _ll_register_callbacks cb 0x55edf028dbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:30:52.934 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.926+0000 7f6fdbfff640 1 --2- 172.21.15.80:0/2855000202 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55edf061bd90 0x55edf061a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:52.934 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.926+0000 7f6fdaffd640 1 -- 172.21.15.80:0/2855000202 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55edf061bd90 msgr2=0x55edf061a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:52.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.926+0000 7f6fdaffd640 1 --2- 172.21.15.80:0/2855000202 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55edf061bd90 0x55edf061a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:52.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.926+0000 7f6fdaffd640 1 -- 172.21.15.80:0/2855000202 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55edf0616230 msgr2=0x55edf0619080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:52.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.926+0000 7f6fdaffd640 1 --2- 172.21.15.80:0/2855000202 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55edf0616230 0x55edf0619080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:52.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.926+0000 7f6fdaffd640 1 -- 172.21.15.80:0/2855000202 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55edf0507dc0 con 0x55edf0616c20 2024-04-24T23:30:52.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.926+0000 7f6fdbfff640 1 --2- 172.21.15.80:0/2855000202 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55edf061bd90 0x55edf061a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:30:52.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.926+0000 7f6fdb7fe640 1 --2- 172.21.15.80:0/2855000202 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55edf0616230 0x55edf0619080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:30:52.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.926+0000 7f6fdaffd640 1 --2- 172.21.15.80:0/2855000202 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55edf0616c20 0x55edf06195c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6fd009b910 tx=0x7f6fd009bd50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f99a5fba25c0c0a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:52.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.926+0000 7f6fbaffd640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-04-24T23:30:52.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.926+0000 7f6fbaffd640 1 -- 172.21.15.80:0/2855000202 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fd00cf480 con 0x55edf0616c20 2024-04-24T23:30:52.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.926+0000 7f6fbaffd640 1 -- 172.21.15.80:0/2855000202 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f6fd009b060 con 0x55edf0616c20 2024-04-24T23:30:52.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.926+0000 7f6fe4dbb740 1 -- 172.21.15.80:0/2855000202 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6fd4011cc0 con 0x55edf0616c20 2024-04-24T23:30:52.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.926+0000 7f6fbaffd640 1 -- 172.21.15.80:0/2855000202 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6fd009b260 con 0x55edf0616c20 2024-04-24T23:30:52.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.926+0000 7f6fe4dbb740 10 client.25115 fetch_fsmap learned FSMap version 25 2024-04-24T23:30:52.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.926+0000 7f6fe4dbb740 1 -- 172.21.15.80:0/2855000202 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55edf028e3f0 con 0x55edf0616c20 2024-04-24T23:30:52.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.927+0000 7f6fbaffd640 1 -- 172.21.15.80:0/2855000202 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6fd0097960 con 0x55edf0616c20 2024-04-24T23:30:52.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.927+0000 7f6fbaffd640 1 -- 172.21.15.80:0/2855000202 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f6fd0010ac0 con 0x55edf0616c20 2024-04-24T23:30:52.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.927+0000 7f6fe4dbb740 10 client.25115 fetch_fsmap finished waiting for FSMap version 25 2024-04-24T23:30:52.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.927+0000 7f6fe4dbb740 10 client.25115 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-24T23:30:52.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.927+0000 7f6fe4dbb740 20 client.25115 populate_metadata read hostname 'smithi080' 2024-04-24T23:30:52.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.927+0000 7f6fe4dbb740 1 --2- 172.21.15.80:0/2855000202 >> [v2:172.21.15.148:6836/856528403,v1:172.21.15.148:6837/856528403] conn(0x55edf0634980 0x55edf0636e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:52.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.927+0000 7f6fe4dbb740 4 client.25115 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:30:52.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.927+0000 7f6fe4dbb740 1 -- 172.21.15.80:0/2855000202 --> [v2:172.21.15.148:6836/856528403,v1:172.21.15.148:6837/856528403] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f6fd4011cc0 con 0x55edf0634980 2024-04-24T23:30:52.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.927+0000 7f6fdb7fe640 1 --2- 172.21.15.80:0/2855000202 >> [v2:172.21.15.148:6836/856528403,v1:172.21.15.148:6837/856528403] conn(0x55edf0634980 0x55edf0636e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:52.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.928+0000 7f6fbaffd640 10 client.25115 ms_handle_connect on v2:172.21.15.148:6836/856528403 2024-04-24T23:30:52.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.928+0000 7f6fdb7fe640 1 --2- 172.21.15.80:0/2855000202 >> [v2:172.21.15.148:6836/856528403,v1:172.21.15.148:6837/856528403] conn(0x55edf0634980 0x55edf0636e20 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f70531b40ff3498c server_cookie=9a73bb505e50c0be in_seq=0 out_seq=0 2024-04-24T23:30:52.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.928+0000 7f6fbaffd640 1 -- 172.21.15.80:0/2855000202 <== mds.2 v2:172.21.15.148:6836/856528403 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f6fd4011cc0 con 0x55edf0634980 2024-04-24T23:30:52.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.928+0000 7f6fbaffd640 10 client.25115 handle_command_reply: tid=0 2024-04-24T23:30:52.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.928+0000 7f6fe4dbb740 1 client.25115 shutdown 2024-04-24T23:30:52.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.929+0000 7f6fe4dbb740 1 -- 172.21.15.80:0/2855000202 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55edf0616c20 msgr2=0x55edf06195c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:52.937 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.929+0000 7f6fe4dbb740 1 --2- 172.21.15.80:0/2855000202 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55edf0616c20 0x55edf06195c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6fd009b910 tx=0x7f6fd009bd50 comp rx=0 tx=0).stop 2024-04-24T23:30:52.937 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.929+0000 7f6fe4dbb740 1 -- 172.21.15.80:0/2855000202 shutdown_connections 2024-04-24T23:30:52.937 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.929+0000 7f6fe4dbb740 1 --2- 172.21.15.80:0/2855000202 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55edf061bd90 0x55edf061a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:52.937 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.929+0000 7f6fe4dbb740 1 --2- 172.21.15.80:0/2855000202 >> [v2:172.21.15.148:6836/856528403,v1:172.21.15.148:6837/856528403] conn(0x55edf0634980 0x55edf0636e20 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:52.937 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.929+0000 7f6fe4dbb740 1 --2- 172.21.15.80:0/2855000202 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55edf0616c20 0x55edf06195c0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:52.937 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.929+0000 7f6fe4dbb740 1 --2- 172.21.15.80:0/2855000202 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55edf0616230 0x55edf0619080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:52.937 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.929+0000 7f6fe4dbb740 1 -- 172.21.15.80:0/2855000202 >> 172.21.15.80:0/2855000202 conn(0x55edf06118a0 msgr2=0x55edf0610b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:52.937 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.929+0000 7f6fbaffd640 0 client.25115 ms_handle_reset on v2:172.21.15.148:6836/856528403 2024-04-24T23:30:52.937 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.929+0000 7f6fe4dbb740 1 -- 172.21.15.80:0/2855000202 shutdown_connections 2024-04-24T23:30:52.937 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.929+0000 7f6fe4dbb740 1 -- 172.21.15.80:0/2855000202 wait complete. 2024-04-24T23:30:52.937 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.929+0000 7f6fe4dbb740 20 client.25115 trim_cache size 0 max 16384 2024-04-24T23:30:52.946 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.942+0000 7f6fe4dbb740 1 Processor -- start 2024-04-24T23:30:52.946 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.942+0000 7f6fe4dbb740 1 -- start start 2024-04-24T23:30:52.947 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.943+0000 7f6fe4dbb740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55edf0616230 0x55edf0617410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:52.947 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.943+0000 7f6fe4dbb740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55edf0616c20 0x55edf0617950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:52.947 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.943+0000 7f6fe4dbb740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55edf061bd90 0x55edf0617e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:52.947 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.943+0000 7f6fe4dbb740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f6fc8012590 con 0x55edf0616230 2024-04-24T23:30:52.947 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.943+0000 7f6fe4dbb740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6fc800fe60 con 0x55edf061bd90 2024-04-24T23:30:52.947 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.943+0000 7f6fe4dbb740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f6fc800dcb0 con 0x55edf0616c20 2024-04-24T23:30:52.947 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.943+0000 7f6fdb7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55edf0616230 0x55edf0617410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:52.948 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.943+0000 7f6fdb7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55edf0616230 0x55edf0617410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:55246/0 (socket says 172.21.15.80:55246) 2024-04-24T23:30:52.948 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.943+0000 7f6fdb7fe640 1 -- 172.21.15.80:0/1224678731 learned_addr learned my addr 172.21.15.80:0/1224678731 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:30:52.948 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.943+0000 7f6fdbfff640 1 --2- 172.21.15.80:0/1224678731 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55edf061bd90 0x55edf0617e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:52.948 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.943+0000 7f6fdaffd640 1 --2- 172.21.15.80:0/1224678731 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55edf0616c20 0x55edf0617950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:52.948 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.943+0000 7f6fdb7fe640 1 -- 172.21.15.80:0/1224678731 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55edf0616c20 msgr2=0x55edf0617950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:52.948 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.943+0000 7f6fdb7fe640 1 --2- 172.21.15.80:0/1224678731 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55edf0616c20 0x55edf0617950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:52.948 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.943+0000 7f6fdb7fe640 1 -- 172.21.15.80:0/1224678731 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55edf061bd90 msgr2=0x55edf0617e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:52.948 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.943+0000 7f6fdb7fe640 1 --2- 172.21.15.80:0/1224678731 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55edf061bd90 0x55edf0617e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:52.949 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.943+0000 7f6fdb7fe640 1 -- 172.21.15.80:0/1224678731 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55edf03587d0 con 0x55edf0616230 2024-04-24T23:30:52.949 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.943+0000 7f6fdbfff640 1 --2- 172.21.15.80:0/1224678731 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55edf061bd90 0x55edf0617e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:30:52.949 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.943+0000 7f6fdb7fe640 1 --2- 172.21.15.80:0/1224678731 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55edf0616230 0x55edf0617410 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc009230 tx=0x7f6fcc03f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f987db82a69d4ef6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:52.949 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.943+0000 7f6fc17fa640 1 -- 172.21.15.80:0/1224678731 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fcc035050 con 0x55edf0616230 2024-04-24T23:30:52.949 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.944+0000 7f6fc17fa640 1 -- 172.21.15.80:0/1224678731 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6fcc005040 con 0x55edf0616230 2024-04-24T23:30:52.950 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.944+0000 7f6fc17fa640 1 -- 172.21.15.80:0/1224678731 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fcc051520 con 0x55edf0616230 2024-04-24T23:30:52.950 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.944+0000 7f6fe4dbb740 1 -- 172.21.15.80:0/1224678731 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55edf0616230 msgr2=0x55edf0617410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:52.950 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.944+0000 7f6fe4dbb740 1 --2- 172.21.15.80:0/1224678731 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55edf0616230 0x55edf0617410 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc009230 tx=0x7f6fcc03f000 comp rx=0 tx=0).stop 2024-04-24T23:30:52.950 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.944+0000 7f6fe4dbb740 1 -- 172.21.15.80:0/1224678731 shutdown_connections 2024-04-24T23:30:52.950 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.944+0000 7f6fe4dbb740 1 --2- 172.21.15.80:0/1224678731 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55edf061bd90 0x55edf0617e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:52.950 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.944+0000 7f6fe4dbb740 1 --2- 172.21.15.80:0/1224678731 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55edf0616c20 0x55edf0617950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:52.950 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.944+0000 7f6fe4dbb740 1 --2- 172.21.15.80:0/1224678731 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55edf0616230 0x55edf0617410 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:52.950 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.944+0000 7f6fe4dbb740 1 -- 172.21.15.80:0/1224678731 >> 172.21.15.80:0/1224678731 conn(0x55edf06118a0 msgr2=0x55edf0610b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:52.950 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.944+0000 7f6fe4dbb740 1 -- 172.21.15.80:0/1224678731 shutdown_connections 2024-04-24T23:30:52.951 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.944+0000 7f6fe4dbb740 1 -- 172.21.15.80:0/1224678731 wait complete. 2024-04-24T23:30:52.951 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.945+0000 7f6fe4dbb740 1 -- start start 2024-04-24T23:30:52.951 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.945+0000 7f6fe4dbb740 1 Processor -- start 2024-04-24T23:30:52.958 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.945+0000 7f6fe4dbb740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55edf0616230 0x55edf062e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:52.958 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.945+0000 7f6fe4dbb740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55edf0616c20 0x55edf062ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:52.958 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.945+0000 7f6fe4dbb740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55edf061bd90 0x55edf062fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:52.958 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.945+0000 7f6fdb7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55edf0616230 0x55edf062e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:52.958 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.945+0000 7f6fe4dbb740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f6fc800e320 con 0x55edf0616230 2024-04-24T23:30:52.958 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.946+0000 7f6fdb7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55edf0616230 0x55edf062e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:55262/0 (socket says 172.21.15.80:55262) 2024-04-24T23:30:52.958 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.946+0000 7f6fdb7fe640 1 -- 172.21.15.80:0/2839427644 learned_addr learned my addr 172.21.15.80:0/2839427644 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:30:52.959 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.946+0000 7f6fe4dbb740 1 -- 172.21.15.80:0/2839427644 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6fc800f790 con 0x55edf061bd90 2024-04-24T23:30:52.959 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.946+0000 7f6fdaffd640 1 --2- 172.21.15.80:0/2839427644 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55edf0616c20 0x55edf062ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:52.959 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.946+0000 7f6fe4dbb740 1 -- 172.21.15.80:0/2839427644 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f6fc800e9f0 con 0x55edf0616c20 2024-04-24T23:30:52.959 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.946+0000 7f6fdbfff640 1 --2- 172.21.15.80:0/2839427644 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55edf061bd90 0x55edf062fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:52.959 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.946+0000 7f6fdb7fe640 1 -- 172.21.15.80:0/2839427644 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55edf0616c20 msgr2=0x55edf062ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:52.959 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.946+0000 7f6fdb7fe640 1 --2- 172.21.15.80:0/2839427644 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55edf0616c20 0x55edf062ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:52.959 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.946+0000 7f6fdb7fe640 1 -- 172.21.15.80:0/2839427644 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55edf061bd90 msgr2=0x55edf062fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:52.959 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.946+0000 7f6fdb7fe640 1 --2- 172.21.15.80:0/2839427644 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55edf061bd90 0x55edf062fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:52.959 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.946+0000 7f6fdb7fe640 1 -- 172.21.15.80:0/2839427644 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55edf0630170 con 0x55edf0616230 2024-04-24T23:30:52.959 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.946+0000 7f6fdaffd640 1 --2- 172.21.15.80:0/2839427644 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55edf0616c20 0x55edf062ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:30:52.959 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.946+0000 7f6fdbfff640 1 --2- 172.21.15.80:0/2839427644 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55edf061bd90 0x55edf062fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:30:52.959 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.946+0000 7f6fe4dbb740 10 client.0 _ll_register_callbacks cb 0x7f6fd009ba70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:30:52.959 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.946+0000 7f6fdb7fe640 1 --2- 172.21.15.80:0/2839427644 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55edf0616230 0x55edf062e720 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc051e00 tx=0x7f6fcc004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acebd484a9edfa5d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:52.959 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.946+0000 7f6fbaffd640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-04-24T23:30:52.959 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.946+0000 7f6fe4dbb740 1 -- 172.21.15.80:0/2839427644 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6fcc005040 con 0x55edf0616230 2024-04-24T23:30:52.960 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.946+0000 7f6fbaffd640 1 -- 172.21.15.80:0/2839427644 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fcc060050 con 0x55edf0616230 2024-04-24T23:30:52.960 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.946+0000 7f6fbaffd640 1 -- 172.21.15.80:0/2839427644 <== mon.0 v2:172.21.15.80:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f6fcc0051f0 con 0x55edf0616230 2024-04-24T23:30:52.960 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.947+0000 7f6fbaffd640 1 -- 172.21.15.80:0/2839427644 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6fcc035070 con 0x55edf0616230 2024-04-24T23:30:52.960 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.947+0000 7f6fe4dbb740 10 client.15795 fetch_fsmap learned FSMap version 25 2024-04-24T23:30:52.960 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.947+0000 7f6fe4dbb740 1 -- 172.21.15.80:0/2839427644 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({fsmap=25}) -- 0x55edf06311b0 con 0x55edf0616230 2024-04-24T23:30:52.960 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.947+0000 7f6fbaffd640 1 -- 172.21.15.80:0/2839427644 <== mon.0 v2:172.21.15.80:3300/0 4 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f6fcc063020 con 0x55edf0616230 2024-04-24T23:30:52.960 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.947+0000 7f6fe4dbb740 10 client.15795 fetch_fsmap finished waiting for FSMap version 25 2024-04-24T23:30:52.960 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.947+0000 7f6fe4dbb740 10 client.15795 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-24T23:30:52.960 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.947+0000 7f6fe4dbb740 20 client.15795 populate_metadata read hostname 'smithi080' 2024-04-24T23:30:52.960 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.947+0000 7f6fe4dbb740 1 --2- 172.21.15.80:0/2839427644 >> [v2:172.21.15.148:6836/856528403,v1:172.21.15.148:6837/856528403] conn(0x55edf069b330 0x55edf069d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:52.960 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.947+0000 7f6fe4dbb740 4 client.15795 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:30:52.960 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.947+0000 7f6fe4dbb740 1 -- 172.21.15.80:0/2839427644 --> [v2:172.21.15.148:6836/856528403,v1:172.21.15.148:6837/856528403] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f6fcc005040 con 0x55edf069b330 2024-04-24T23:30:52.960 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.947+0000 7f6fdaffd640 1 --2- 172.21.15.80:0/2839427644 >> [v2:172.21.15.148:6836/856528403,v1:172.21.15.148:6837/856528403] conn(0x55edf069b330 0x55edf069d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:52.960 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.948+0000 7f6fdaffd640 1 --2- 172.21.15.80:0/2839427644 >> [v2:172.21.15.148:6836/856528403,v1:172.21.15.148:6837/856528403] conn(0x55edf069b330 0x55edf069d7f0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6ed0222a89a759b2 server_cookie=bb6a8ba5b248314 in_seq=0 out_seq=0 2024-04-24T23:30:52.960 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.948+0000 7f6fbaffd640 10 client.15795 ms_handle_connect on v2:172.21.15.148:6836/856528403 2024-04-24T23:30:52.961 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.948+0000 7f6fbaffd640 1 -- 172.21.15.80:0/2839427644 <== mds.2 v2:172.21.15.148:6836/856528403 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f6fcc005040 con 0x55edf069b330 2024-04-24T23:30:52.961 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.948+0000 7f6fbaffd640 10 client.15795 handle_command_reply: tid=0 2024-04-24T23:30:52.961 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.948+0000 7f6fe4dbb740 1 client.15795 shutdown 2024-04-24T23:30:52.961 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.948+0000 7f6fe4dbb740 1 -- 172.21.15.80:0/2839427644 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55edf0616230 msgr2=0x55edf062e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:52.961 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.948+0000 7f6fe4dbb740 1 --2- 172.21.15.80:0/2839427644 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55edf0616230 0x55edf062e720 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc051e00 tx=0x7f6fcc004340 comp rx=0 tx=0).stop 2024-04-24T23:30:52.961 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.948+0000 7f6fe4dbb740 1 -- 172.21.15.80:0/2839427644 shutdown_connections 2024-04-24T23:30:52.961 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.948+0000 7f6fe4dbb740 1 --2- 172.21.15.80:0/2839427644 >> [v2:172.21.15.148:6836/856528403,v1:172.21.15.148:6837/856528403] conn(0x55edf069b330 0x55edf069d7f0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:52.961 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.949+0000 7f6fe4dbb740 1 --2- 172.21.15.80:0/2839427644 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55edf061bd90 0x55edf062fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:52.961 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.949+0000 7f6fbaffd640 0 client.15795 ms_handle_reset on v2:172.21.15.148:6836/856528403 2024-04-24T23:30:52.961 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.949+0000 7f6fe4dbb740 1 --2- 172.21.15.80:0/2839427644 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55edf0616c20 0x55edf062ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:52.961 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.949+0000 7f6fe4dbb740 1 --2- 172.21.15.80:0/2839427644 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55edf0616230 0x55edf062e720 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:52.961 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.949+0000 7f6fe4dbb740 1 -- 172.21.15.80:0/2839427644 >> 172.21.15.80:0/2839427644 conn(0x55edf06118a0 msgr2=0x55edf06147b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:52.961 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.949+0000 7f6fe4dbb740 1 -- 172.21.15.80:0/2839427644 shutdown_connections 2024-04-24T23:30:52.961 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.949+0000 7f6fe4dbb740 1 -- 172.21.15.80:0/2839427644 wait complete. 2024-04-24T23:30:52.961 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.949+0000 7f6fe4dbb740 20 client.15795 trim_cache size 0 max 16384 2024-04-24T23:30:52.962 INFO:teuthology.orchestra.run.smithi080.stdout:[] 2024-04-24T23:30:52.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.949+0000 7f6fe20a5640 1 -- 172.21.15.80:0/203237058 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f6fc40a2c10 msgr2=0x7f6fc40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:52.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.949+0000 7f6fe20a5640 1 --2- 172.21.15.80:0/203237058 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f6fc40a2c10 0x7f6fc40a50d0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc006010 tx=0x7f6fcc032cf0 comp rx=0 tx=0).stop 2024-04-24T23:30:52.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.949+0000 7f6fe20a5640 1 -- 172.21.15.80:0/203237058 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6fdc102980 msgr2=0x7f6fdc197fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:52.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.949+0000 7f6fe20a5640 1 --2- 172.21.15.80:0/203237058 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6fdc102980 0x7f6fdc197fb0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6fd000e9f0 tx=0x7f6fd000eeb0 comp rx=0 tx=0).stop 2024-04-24T23:30:52.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.949+0000 7f6fe20a5640 1 -- 172.21.15.80:0/203237058 shutdown_connections 2024-04-24T23:30:52.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.949+0000 7f6fe20a5640 1 --2- 172.21.15.80:0/203237058 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f6fc40a2c10 0x7f6fc40a50d0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:52.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.949+0000 7f6fe20a5640 1 --2- 172.21.15.80:0/203237058 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6fdc103340 0x7f6fdc19f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:52.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.949+0000 7f6fe20a5640 1 --2- 172.21.15.80:0/203237058 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6fdc102980 0x7f6fdc197fb0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:52.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.949+0000 7f6fe20a5640 1 --2- 172.21.15.80:0/203237058 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6fdc101790 0x7f6fdc197a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:52.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.949+0000 7f6fe20a5640 1 -- 172.21.15.80:0/203237058 >> 172.21.15.80:0/203237058 conn(0x7f6fdc0fcd20 msgr2=0x7f6fdc105ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:52.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.950+0000 7f6fe20a5640 1 -- 172.21.15.80:0/203237058 shutdown_connections 2024-04-24T23:30:52.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:52.950+0000 7f6fe20a5640 1 -- 172.21.15.80:0/203237058 wait complete. 2024-04-24T23:30:52.964 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T23:30:53.070 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.068+0000 7f22e796a640 1 -- 172.21.15.80:0/3973968671 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f22e0101350 msgr2=0x7f22e01017d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:53.070 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.068+0000 7f22e796a640 1 --2- 172.21.15.80:0/3973968671 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f22e0101350 0x7f22e01017d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f22d8009be0 tx=0x7f22d802a6c0 comp rx=0 tx=0).stop 2024-04-24T23:30:53.070 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.068+0000 7f22e796a640 1 -- 172.21.15.80:0/3973968671 shutdown_connections 2024-04-24T23:30:53.070 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.068+0000 7f22e796a640 1 --2- 172.21.15.80:0/3973968671 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f22e0101350 0x7f22e01017d0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:53.070 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.068+0000 7f22e796a640 1 --2- 172.21.15.80:0/3973968671 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f22e01009b0 0x7f22e0100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:53.070 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.068+0000 7f22e796a640 1 --2- 172.21.15.80:0/3973968671 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f22e0100050 0x7f22e0100470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:53.071 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.068+0000 7f22e796a640 1 -- 172.21.15.80:0/3973968671 >> 172.21.15.80:0/3973968671 conn(0x7f22e00fbbf0 msgr2=0x7f22e00fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:53.071 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.068+0000 7f22e796a640 1 -- 172.21.15.80:0/3973968671 shutdown_connections 2024-04-24T23:30:53.071 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.068+0000 7f22e796a640 1 -- 172.21.15.80:0/3973968671 wait complete. 2024-04-24T23:30:53.071 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.069+0000 7f22e796a640 1 Processor -- start 2024-04-24T23:30:53.071 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.069+0000 7f22e796a640 1 -- start start 2024-04-24T23:30:53.071 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.069+0000 7f22e796a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f22e0100050 0x7f22e0199c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:53.071 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.069+0000 7f22e796a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f22e01009b0 0x7f22e019a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:53.071 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.069+0000 7f22e796a640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f22e0101350 0x7f22e01a11c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:53.071 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.069+0000 7f22e796a640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f22e010f580 con 0x7f22e0101350 2024-04-24T23:30:53.071 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.069+0000 7f22e796a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f22e010f400 con 0x7f22e01009b0 2024-04-24T23:30:53.071 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.069+0000 7f22e796a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f22e010f700 con 0x7f22e0100050 2024-04-24T23:30:53.071 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.069+0000 7f22e4f05640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f22e01009b0 0x7f22e019a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:53.071 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.069+0000 7f22e5f07640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f22e0101350 0x7f22e01a11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:53.072 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.069+0000 7f22e4f05640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f22e01009b0 0x7f22e019a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.80:57712/0 (socket says 172.21.15.80:57712) 2024-04-24T23:30:53.072 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.069+0000 7f22e4f05640 1 -- 172.21.15.80:0/46174579 learned_addr learned my addr 172.21.15.80:0/46174579 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:30:53.072 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.070+0000 7f22e5706640 1 --2- 172.21.15.80:0/46174579 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f22e0100050 0x7f22e0199c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:53.072 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.070+0000 7f22e4f05640 1 -- 172.21.15.80:0/46174579 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f22e0100050 msgr2=0x7f22e0199c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:53.072 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.070+0000 7f22e4f05640 1 --2- 172.21.15.80:0/46174579 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f22e0100050 0x7f22e0199c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:53.072 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.070+0000 7f22e4f05640 1 -- 172.21.15.80:0/46174579 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f22e0101350 msgr2=0x7f22e01a11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:53.072 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.070+0000 7f22e4f05640 1 --2- 172.21.15.80:0/46174579 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f22e0101350 0x7f22e01a11c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:53.072 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.070+0000 7f22e4f05640 1 -- 172.21.15.80:0/46174579 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22e01a1860 con 0x7f22e01009b0 2024-04-24T23:30:53.072 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.070+0000 7f22e5f07640 1 --2- 172.21.15.80:0/46174579 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f22e0101350 0x7f22e01a11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:30:53.072 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.070+0000 7f22e4f05640 1 --2- 172.21.15.80:0/46174579 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f22e01009b0 0x7f22e019a190 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f22d000d950 tx=0x7f22d000de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:53.072 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.070+0000 7f22d67fc640 1 -- 172.21.15.80:0/46174579 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22d0017030 con 0x7f22e01009b0 2024-04-24T23:30:53.072 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.070+0000 7f22d67fc640 1 -- 172.21.15.80:0/46174579 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22d0004020 con 0x7f22e01009b0 2024-04-24T23:30:53.072 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.070+0000 7f22e796a640 1 -- 172.21.15.80:0/46174579 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22e01a1b50 con 0x7f22e01009b0 2024-04-24T23:30:53.073 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.070+0000 7f22d67fc640 1 -- 172.21.15.80:0/46174579 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22d00047a0 con 0x7f22e01009b0 2024-04-24T23:30:53.073 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.070+0000 7f22e796a640 1 -- 172.21.15.80:0/46174579 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22e01a2050 con 0x7f22e01009b0 2024-04-24T23:30:53.075 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.073+0000 7f22d67fc640 1 -- 172.21.15.80:0/46174579 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f22d00041d0 con 0x7f22e01009b0 2024-04-24T23:30:53.075 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.073+0000 7f22d67fc640 1 --2- 172.21.15.80:0/46174579 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f22cc0a2b90 0x7f22cc0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:53.075 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.074+0000 7f22ea67f740 1 Processor -- start 2024-04-24T23:30:53.075 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.074+0000 7f22d67fc640 1 -- 172.21.15.80:0/46174579 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f22d00cee50 con 0x7f22e01009b0 2024-04-24T23:30:53.075 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.074+0000 7f22e5706640 1 --2- 172.21.15.80:0/46174579 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f22cc0a2b90 0x7f22cc0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:53.075 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.074+0000 7f22ea67f740 1 -- start start 2024-04-24T23:30:53.075 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.074+0000 7f22ea67f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5636e855aa40 0x5636e855ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:53.075 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.074+0000 7f22ea67f740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5636e855b430 0x5636e855ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:53.075 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.074+0000 7f22ea67f740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5636e85605a0 0x5636e8562a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:53.075 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.074+0000 7f22ea67f740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x5636e84b6670 con 0x5636e85605a0 2024-04-24T23:30:53.075 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.074+0000 7f22ea67f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5636e845e970 con 0x5636e855aa40 2024-04-24T23:30:53.076 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.074+0000 7f22ea67f740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5636e84b2630 con 0x5636e855b430 2024-04-24T23:30:53.076 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.074+0000 7f22e5706640 1 --2- 172.21.15.80:0/46174579 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f22cc0a2b90 0x7f22cc0a5050 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f22dc0096c0 tx=0x7f22dc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:53.076 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.074+0000 7f22e5f07640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5636e85605a0 0x5636e8562a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:53.076 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.074+0000 7f22e5f07640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5636e85605a0 0x5636e8562a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:55282/0 (socket says 172.21.15.80:55282) 2024-04-24T23:30:53.076 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.074+0000 7f22e5f07640 1 -- 172.21.15.80:0/2894749259 learned_addr learned my addr 172.21.15.80:0/2894749259 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:30:53.076 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.074+0000 7f22e5706640 1 --2- 172.21.15.80:0/2894749259 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5636e855aa40 0x5636e855ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:53.076 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.074+0000 7f22e4f05640 1 --2- 172.21.15.80:0/2894749259 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5636e855b430 0x5636e855ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:53.076 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.075+0000 7f22e5f07640 1 -- 172.21.15.80:0/2894749259 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5636e855b430 msgr2=0x5636e855ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:53.076 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.075+0000 7f22e5f07640 1 --2- 172.21.15.80:0/2894749259 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5636e855b430 0x5636e855ff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:53.076 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.075+0000 7f22e5f07640 1 -- 172.21.15.80:0/2894749259 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5636e855aa40 msgr2=0x5636e855ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:53.076 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.075+0000 7f22e5f07640 1 --2- 172.21.15.80:0/2894749259 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5636e855aa40 0x5636e855ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:53.076 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.075+0000 7f22e5f07640 1 -- 172.21.15.80:0/2894749259 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636e8482400 con 0x5636e85605a0 2024-04-24T23:30:53.076 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.075+0000 7f22e5f07640 1 --2- 172.21.15.80:0/2894749259 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5636e85605a0 0x5636e8562a30 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f22d8009be0 tx=0x7f22d8009280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9c6716b3f699753 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:53.076 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.075+0000 7f22cb7fe640 1 -- 172.21.15.80:0/2894749259 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22d8042030 con 0x5636e85605a0 2024-04-24T23:30:53.077 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.075+0000 7f22cb7fe640 1 -- 172.21.15.80:0/2894749259 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22d8037040 con 0x5636e85605a0 2024-04-24T23:30:53.077 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.075+0000 7f22cb7fe640 1 -- 172.21.15.80:0/2894749259 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22d8036070 con 0x5636e85605a0 2024-04-24T23:30:53.081 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.076+0000 7f22ea67f740 1 -- 172.21.15.80:0/2894749259 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5636e85605a0 msgr2=0x5636e8562a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:53.081 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.076+0000 7f22ea67f740 1 --2- 172.21.15.80:0/2894749259 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5636e85605a0 0x5636e8562a30 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f22d8009be0 tx=0x7f22d8009280 comp rx=0 tx=0).stop 2024-04-24T23:30:53.081 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.076+0000 7f22ea67f740 1 -- 172.21.15.80:0/2894749259 shutdown_connections 2024-04-24T23:30:53.081 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.076+0000 7f22ea67f740 1 --2- 172.21.15.80:0/2894749259 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5636e85605a0 0x5636e8562a30 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:53.081 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.076+0000 7f22ea67f740 1 --2- 172.21.15.80:0/2894749259 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5636e855b430 0x5636e855ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:53.081 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.076+0000 7f22ea67f740 1 --2- 172.21.15.80:0/2894749259 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5636e855aa40 0x5636e855ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:53.081 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.076+0000 7f22ea67f740 1 -- 172.21.15.80:0/2894749259 >> 172.21.15.80:0/2894749259 conn(0x5636e854d8a0 msgr2=0x5636e854dca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:53.081 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.076+0000 7f22ea67f740 1 -- 172.21.15.80:0/2894749259 shutdown_connections 2024-04-24T23:30:53.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.076+0000 7f22ea67f740 1 -- 172.21.15.80:0/2894749259 wait complete. 2024-04-24T23:30:53.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.076+0000 7f22ea67f740 1 -- start start 2024-04-24T23:30:53.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.076+0000 7f22ea67f740 1 Processor -- start 2024-04-24T23:30:53.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.077+0000 7f22ea67f740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5636e855aa40 0x5636e8564b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:53.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.077+0000 7f22ea67f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5636e855b430 0x5636e8565050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:53.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.077+0000 7f22ea67f740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5636e85605a0 0x5636e856c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:53.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.077+0000 7f22ea67f740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x5636e85634f0 con 0x5636e855aa40 2024-04-24T23:30:53.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.077+0000 7f22ea67f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5636e8472050 con 0x5636e855b430 2024-04-24T23:30:53.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.077+0000 7f22ea67f740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5636e8563670 con 0x5636e85605a0 2024-04-24T23:30:53.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.077+0000 7f22ea67f740 10 client.0 _ll_register_callbacks cb 0x5636e81c9be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:30:53.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.077+0000 7f22e4f05640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5636e855b430 0x5636e8565050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:53.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.077+0000 7f22e4f05640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5636e855b430 0x5636e8565050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.80:57728/0 (socket says 172.21.15.80:57728) 2024-04-24T23:30:53.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.077+0000 7f22e4f05640 1 -- 172.21.15.80:0/1560888552 learned_addr learned my addr 172.21.15.80:0/1560888552 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:30:53.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.077+0000 7f22e5706640 1 --2- 172.21.15.80:0/1560888552 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5636e855aa40 0x5636e8564b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:53.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.077+0000 7f22e5f07640 1 --2- 172.21.15.80:0/1560888552 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5636e85605a0 0x5636e856c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:53.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.077+0000 7f22e4f05640 1 -- 172.21.15.80:0/1560888552 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5636e85605a0 msgr2=0x5636e856c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:53.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.077+0000 7f22e4f05640 1 --2- 172.21.15.80:0/1560888552 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5636e85605a0 0x5636e856c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:53.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.077+0000 7f22e4f05640 1 -- 172.21.15.80:0/1560888552 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5636e855aa40 msgr2=0x5636e8564b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:53.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.077+0000 7f22e4f05640 1 --2- 172.21.15.80:0/1560888552 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5636e855aa40 0x5636e8564b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:53.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.077+0000 7f22e4f05640 1 -- 172.21.15.80:0/1560888552 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636e8443dc0 con 0x5636e855b430 2024-04-24T23:30:53.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.077+0000 7f22e5f07640 1 --2- 172.21.15.80:0/1560888552 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5636e85605a0 0x5636e856c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:30:53.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.077+0000 7f22e5706640 1 --2- 172.21.15.80:0/1560888552 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5636e855aa40 0x5636e8564b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:30:53.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.078+0000 7f22e4f05640 1 --2- 172.21.15.80:0/1560888552 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5636e855b430 0x5636e8565050 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f22d00a39c0 tx=0x7f22d009b170 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e872c8c06dec49b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:53.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.078+0000 7f22c8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-04-24T23:30:53.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.078+0000 7f22c8ff9640 1 -- 172.21.15.80:0/1560888552 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22d00bb2c0 con 0x5636e855b430 2024-04-24T23:30:53.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.078+0000 7f22c8ff9640 1 -- 172.21.15.80:0/1560888552 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f22d009f120 con 0x5636e855b430 2024-04-24T23:30:53.084 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.078+0000 7f22ea67f740 1 -- 172.21.15.80:0/1560888552 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22d8037040 con 0x5636e855b430 2024-04-24T23:30:53.084 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.078+0000 7f22c8ff9640 1 -- 172.21.15.80:0/1560888552 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f22d00b4240 con 0x5636e855b430 2024-04-24T23:30:53.084 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.078+0000 7f22ea67f740 10 client.25133 fetch_fsmap learned FSMap version 25 2024-04-24T23:30:53.084 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.078+0000 7f22ea67f740 1 -- 172.21.15.80:0/1560888552 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5636e81ca3f0 con 0x5636e855b430 2024-04-24T23:30:53.084 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.078+0000 7f22c8ff9640 1 -- 172.21.15.80:0/1560888552 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22d009b280 con 0x5636e855b430 2024-04-24T23:30:53.084 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.078+0000 7f22c8ff9640 1 -- 172.21.15.80:0/1560888552 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f22d009a860 con 0x5636e855b430 2024-04-24T23:30:53.084 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.078+0000 7f22ea67f740 10 client.25133 fetch_fsmap finished waiting for FSMap version 25 2024-04-24T23:30:53.084 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.078+0000 7f22ea67f740 10 client.25133 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-04-24T23:30:53.084 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.078+0000 7f22ea67f740 20 client.25133 populate_metadata read hostname 'smithi080' 2024-04-24T23:30:53.084 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.078+0000 7f22ea67f740 1 --2- 172.21.15.80:0/1560888552 >> [v2:172.21.15.134:6838/1743815021,v1:172.21.15.134:6839/1743815021] conn(0x5636e8570860 0x5636e8572cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:53.084 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.078+0000 7f22ea67f740 4 client.25133 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:30:53.084 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.078+0000 7f22ea67f740 1 -- 172.21.15.80:0/1560888552 --> [v2:172.21.15.134:6838/1743815021,v1:172.21.15.134:6839/1743815021] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f22d8037040 con 0x5636e8570860 2024-04-24T23:30:53.084 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.079+0000 7f22e5706640 1 --2- 172.21.15.80:0/1560888552 >> [v2:172.21.15.134:6838/1743815021,v1:172.21.15.134:6839/1743815021] conn(0x5636e8570860 0x5636e8572cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:53.085 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.079+0000 7f22c8ff9640 10 client.25133 ms_handle_connect on v2:172.21.15.134:6838/1743815021 2024-04-24T23:30:53.085 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.079+0000 7f22e5706640 1 --2- 172.21.15.80:0/1560888552 >> [v2:172.21.15.134:6838/1743815021,v1:172.21.15.134:6839/1743815021] conn(0x5636e8570860 0x5636e8572cd0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=319ec8bb989eaee5 server_cookie=cc27066f7b964ca1 in_seq=0 out_seq=0 2024-04-24T23:30:53.085 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.080+0000 7f22c8ff9640 1 -- 172.21.15.80:0/1560888552 <== mds.1 v2:172.21.15.134:6838/1743815021 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f22d8037040 con 0x5636e8570860 2024-04-24T23:30:53.085 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.080+0000 7f22c8ff9640 10 client.25133 handle_command_reply: tid=0 2024-04-24T23:30:53.085 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.080+0000 7f22ea67f740 1 client.25133 shutdown 2024-04-24T23:30:53.085 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.080+0000 7f22ea67f740 1 -- 172.21.15.80:0/1560888552 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5636e855b430 msgr2=0x5636e8565050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:53.085 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.080+0000 7f22ea67f740 1 --2- 172.21.15.80:0/1560888552 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5636e855b430 0x5636e8565050 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f22d00a39c0 tx=0x7f22d009b170 comp rx=0 tx=0).stop 2024-04-24T23:30:53.085 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.080+0000 7f22ea67f740 1 -- 172.21.15.80:0/1560888552 shutdown_connections 2024-04-24T23:30:53.085 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.080+0000 7f22ea67f740 1 --2- 172.21.15.80:0/1560888552 >> [v2:172.21.15.134:6838/1743815021,v1:172.21.15.134:6839/1743815021] conn(0x5636e8570860 0x5636e8572cd0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:53.085 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.080+0000 7f22ea67f740 1 --2- 172.21.15.80:0/1560888552 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5636e85605a0 0x5636e856c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:53.085 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.080+0000 7f22c8ff9640 0 client.25133 ms_handle_reset on v2:172.21.15.134:6838/1743815021 2024-04-24T23:30:53.085 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.080+0000 7f22ea67f740 1 --2- 172.21.15.80:0/1560888552 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5636e855b430 0x5636e8565050 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:53.085 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.080+0000 7f22ea67f740 1 --2- 172.21.15.80:0/1560888552 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5636e855aa40 0x5636e8564b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:53.086 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.080+0000 7f22ea67f740 1 -- 172.21.15.80:0/1560888552 >> 172.21.15.80:0/1560888552 conn(0x5636e854d8a0 msgr2=0x5636e854cb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:53.086 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.080+0000 7f22ea67f740 1 -- 172.21.15.80:0/1560888552 shutdown_connections 2024-04-24T23:30:53.086 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.080+0000 7f22ea67f740 1 -- 172.21.15.80:0/1560888552 wait complete. 2024-04-24T23:30:53.086 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.080+0000 7f22ea67f740 20 client.25133 trim_cache size 0 max 16384 2024-04-24T23:30:53.096 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.095+0000 7f22ea67f740 1 Processor -- start 2024-04-24T23:30:53.096 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.096+0000 7f22ea67f740 1 -- start start 2024-04-24T23:30:53.096 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.096+0000 7f22ea67f740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5636e855aa40 0x5636e8550ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:53.096 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.096+0000 7f22ea67f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5636e855b430 0x5636e856c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:53.097 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.096+0000 7f22ea67f740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5636e85605a0 0x5636e8551400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:53.097 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.096+0000 7f22ea67f740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f22e01aabb0 con 0x5636e85605a0 2024-04-24T23:30:53.097 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.096+0000 7f22ea67f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f22e01a8480 con 0x5636e855b430 2024-04-24T23:30:53.097 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.096+0000 7f22ea67f740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f22e01a62d0 con 0x5636e855aa40 2024-04-24T23:30:53.097 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.096+0000 7f22e4f05640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5636e855b430 0x5636e856c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:53.097 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.096+0000 7f22e5f07640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5636e85605a0 0x5636e8551400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:53.097 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.096+0000 7f22e4f05640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5636e855b430 0x5636e856c050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.80:57744/0 (socket says 172.21.15.80:57744) 2024-04-24T23:30:53.097 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.096+0000 7f22e4f05640 1 -- 172.21.15.80:0/2003259360 learned_addr learned my addr 172.21.15.80:0/2003259360 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:30:53.097 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.096+0000 7f22e5706640 1 --2- 172.21.15.80:0/2003259360 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5636e855aa40 0x5636e8550ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:53.097 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.096+0000 7f22e4f05640 1 -- 172.21.15.80:0/2003259360 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5636e855aa40 msgr2=0x5636e8550ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:53.097 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.096+0000 7f22e4f05640 1 --2- 172.21.15.80:0/2003259360 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5636e855aa40 0x5636e8550ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:53.097 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.096+0000 7f22e4f05640 1 -- 172.21.15.80:0/2003259360 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5636e85605a0 msgr2=0x5636e8551400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:53.097 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.096+0000 7f22e4f05640 1 --2- 172.21.15.80:0/2003259360 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5636e85605a0 0x5636e8551400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:53.097 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.096+0000 7f22e4f05640 1 -- 172.21.15.80:0/2003259360 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636e82947d0 con 0x5636e855b430 2024-04-24T23:30:53.098 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.096+0000 7f22e4f05640 1 --2- 172.21.15.80:0/2003259360 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5636e855b430 0x5636e856c050 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f22d009af80 tx=0x7f22d000b510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=682f0548ad4d5391 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:53.098 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.097+0000 7f22caffd640 1 -- 172.21.15.80:0/2003259360 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22d009f320 con 0x5636e855b430 2024-04-24T23:30:53.098 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.097+0000 7f22caffd640 1 -- 172.21.15.80:0/2003259360 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22d009b280 con 0x5636e855b430 2024-04-24T23:30:53.098 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.097+0000 7f22caffd640 1 -- 172.21.15.80:0/2003259360 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22d009a6d0 con 0x5636e855b430 2024-04-24T23:30:53.103 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.098+0000 7f22ea67f740 1 -- 172.21.15.80:0/2003259360 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5636e855b430 msgr2=0x5636e856c050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:53.103 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.098+0000 7f22ea67f740 1 --2- 172.21.15.80:0/2003259360 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5636e855b430 0x5636e856c050 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f22d009af80 tx=0x7f22d000b510 comp rx=0 tx=0).stop 2024-04-24T23:30:53.103 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.098+0000 7f22ea67f740 1 -- 172.21.15.80:0/2003259360 shutdown_connections 2024-04-24T23:30:53.103 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.098+0000 7f22ea67f740 1 --2- 172.21.15.80:0/2003259360 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5636e85605a0 0x5636e8551400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:53.104 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.098+0000 7f22ea67f740 1 --2- 172.21.15.80:0/2003259360 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5636e855b430 0x5636e856c050 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:53.104 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.098+0000 7f22ea67f740 1 --2- 172.21.15.80:0/2003259360 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5636e855aa40 0x5636e8550ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:53.104 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.098+0000 7f22ea67f740 1 -- 172.21.15.80:0/2003259360 >> 172.21.15.80:0/2003259360 conn(0x5636e854d8a0 msgr2=0x5636e85507b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:53.104 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.098+0000 7f22ea67f740 1 -- 172.21.15.80:0/2003259360 shutdown_connections 2024-04-24T23:30:53.104 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.098+0000 7f22ea67f740 1 -- 172.21.15.80:0/2003259360 wait complete. 2024-04-24T23:30:53.104 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.098+0000 7f22ea67f740 1 -- start start 2024-04-24T23:30:53.104 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.098+0000 7f22ea67f740 1 Processor -- start 2024-04-24T23:30:53.104 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.099+0000 7f22ea67f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5636e855aa40 0x5636e856b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:53.104 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.099+0000 7f22ea67f740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5636e855b430 0x5636e856bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:53.104 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.099+0000 7f22ea67f740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5636e85605a0 0x5636e8569e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:53.104 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.099+0000 7f22ea67f740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f22e01a6940 con 0x5636e85605a0 2024-04-24T23:30:53.104 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.099+0000 7f22ea67f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f22e01a7db0 con 0x5636e855aa40 2024-04-24T23:30:53.104 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.099+0000 7f22ea67f740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f22e01a7010 con 0x5636e855b430 2024-04-24T23:30:53.104 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.099+0000 7f22e5706640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5636e855aa40 0x5636e856b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:53.105 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.099+0000 7f22e5f07640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5636e85605a0 0x5636e8569e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:53.105 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.099+0000 7f22e5706640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5636e855aa40 0x5636e856b840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.80:57750/0 (socket says 172.21.15.80:57750) 2024-04-24T23:30:53.105 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.099+0000 7f22e5706640 1 -- 172.21.15.80:0/4050583859 learned_addr learned my addr 172.21.15.80:0/4050583859 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:30:53.105 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.099+0000 7f22e4f05640 1 --2- 172.21.15.80:0/4050583859 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5636e855b430 0x5636e856bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:53.105 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.099+0000 7f22ea67f740 10 client.0 _ll_register_callbacks cb 0x7f22d00046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:30:53.105 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.099+0000 7f22e5706640 1 -- 172.21.15.80:0/4050583859 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5636e855b430 msgr2=0x5636e856bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:53.105 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.099+0000 7f22e5706640 1 --2- 172.21.15.80:0/4050583859 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5636e855b430 0x5636e856bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:53.105 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.099+0000 7f22e5706640 1 -- 172.21.15.80:0/4050583859 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5636e85605a0 msgr2=0x5636e8569e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:53.105 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.099+0000 7f22e5706640 1 --2- 172.21.15.80:0/4050583859 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5636e85605a0 0x5636e8569e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:53.105 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.099+0000 7f22e5706640 1 -- 172.21.15.80:0/4050583859 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636e856a4c0 con 0x5636e855aa40 2024-04-24T23:30:53.105 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.099+0000 7f22e5f07640 1 --2- 172.21.15.80:0/4050583859 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5636e85605a0 0x5636e8569e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:30:53.105 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.100+0000 7f22e5706640 1 --2- 172.21.15.80:0/4050583859 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5636e855aa40 0x5636e856b840 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f22e019b120 tx=0x7f22dc004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1845bc6db3951c59 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:53.105 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.100+0000 7f22c8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-04-24T23:30:53.106 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.100+0000 7f22c8ff9640 1 -- 172.21.15.80:0/4050583859 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22dc032060 con 0x5636e855aa40 2024-04-24T23:30:53.106 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.100+0000 7f22c8ff9640 1 -- 172.21.15.80:0/4050583859 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f22dc02c4e0 con 0x5636e855aa40 2024-04-24T23:30:53.106 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.100+0000 7f22ea67f740 1 -- 172.21.15.80:0/4050583859 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22d009b280 con 0x5636e855aa40 2024-04-24T23:30:53.106 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.100+0000 7f22c8ff9640 1 -- 172.21.15.80:0/4050583859 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f22dc039050 con 0x5636e855aa40 2024-04-24T23:30:53.106 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.100+0000 7f22ea67f740 10 client.25139 fetch_fsmap learned FSMap version 25 2024-04-24T23:30:53.106 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.100+0000 7f22ea67f740 1 -- 172.21.15.80:0/4050583859 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5636e85d77e0 con 0x5636e855aa40 2024-04-24T23:30:53.106 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.100+0000 7f22c8ff9640 1 -- 172.21.15.80:0/4050583859 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22d009b280 con 0x5636e855aa40 2024-04-24T23:30:53.106 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.101+0000 7f22c8ff9640 1 -- 172.21.15.80:0/4050583859 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f22dc0283e0 con 0x5636e855aa40 2024-04-24T23:30:53.106 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.101+0000 7f22ea67f740 10 client.25139 fetch_fsmap finished waiting for FSMap version 25 2024-04-24T23:30:53.106 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.101+0000 7f22ea67f740 10 client.25139 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-04-24T23:30:53.106 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.101+0000 7f22ea67f740 20 client.25139 populate_metadata read hostname 'smithi080' 2024-04-24T23:30:53.106 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.101+0000 7f22ea67f740 1 --2- 172.21.15.80:0/4050583859 >> [v2:172.21.15.134:6838/1743815021,v1:172.21.15.134:6839/1743815021] conn(0x5636e85d8170 0x5636e85da590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:53.106 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.101+0000 7f22ea67f740 4 client.25139 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:30:53.107 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.101+0000 7f22ea67f740 1 -- 172.21.15.80:0/4050583859 --> [v2:172.21.15.134:6838/1743815021,v1:172.21.15.134:6839/1743815021] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5636e84c3bb0 con 0x5636e85d8170 2024-04-24T23:30:53.107 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.101+0000 7f22e4f05640 1 --2- 172.21.15.80:0/4050583859 >> [v2:172.21.15.134:6838/1743815021,v1:172.21.15.134:6839/1743815021] conn(0x5636e85d8170 0x5636e85da590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:53.107 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.101+0000 7f22c8ff9640 10 client.25139 ms_handle_connect on v2:172.21.15.134:6838/1743815021 2024-04-24T23:30:53.107 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.101+0000 7f22e4f05640 1 --2- 172.21.15.80:0/4050583859 >> [v2:172.21.15.134:6838/1743815021,v1:172.21.15.134:6839/1743815021] conn(0x5636e85d8170 0x5636e85da590 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d9cfde00d4b41f33 server_cookie=6cf4770f979910fc in_seq=0 out_seq=0 2024-04-24T23:30:53.107 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.101+0000 7f22c8ff9640 1 -- 172.21.15.80:0/4050583859 <== mds.1 v2:172.21.15.134:6838/1743815021 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5636e84c3bb0 con 0x5636e85d8170 2024-04-24T23:30:53.107 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.101+0000 7f22c8ff9640 10 client.25139 handle_command_reply: tid=0 2024-04-24T23:30:53.107 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.101+0000 7f22ea67f740 1 client.25139 shutdown 2024-04-24T23:30:53.107 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.102+0000 7f22ea67f740 1 -- 172.21.15.80:0/4050583859 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5636e855aa40 msgr2=0x5636e856b840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:53.107 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.102+0000 7f22ea67f740 1 --2- 172.21.15.80:0/4050583859 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5636e855aa40 0x5636e856b840 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f22e019b120 tx=0x7f22dc004730 comp rx=0 tx=0).stop 2024-04-24T23:30:53.107 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.102+0000 7f22ea67f740 1 -- 172.21.15.80:0/4050583859 shutdown_connections 2024-04-24T23:30:53.107 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.102+0000 7f22ea67f740 1 --2- 172.21.15.80:0/4050583859 >> [v2:172.21.15.134:6838/1743815021,v1:172.21.15.134:6839/1743815021] conn(0x5636e85d8170 0x5636e85da590 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:53.107 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.102+0000 7f22ea67f740 1 --2- 172.21.15.80:0/4050583859 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5636e85605a0 0x5636e8569e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:53.107 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.102+0000 7f22c8ff9640 0 client.25139 ms_handle_reset on v2:172.21.15.134:6838/1743815021 2024-04-24T23:30:53.107 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.102+0000 7f22ea67f740 1 --2- 172.21.15.80:0/4050583859 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5636e855b430 0x5636e856bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:53.108 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.102+0000 7f22ea67f740 1 --2- 172.21.15.80:0/4050583859 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5636e855aa40 0x5636e856b840 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:53.108 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.102+0000 7f22ea67f740 1 -- 172.21.15.80:0/4050583859 >> 172.21.15.80:0/4050583859 conn(0x5636e854d8a0 msgr2=0x5636e8551040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:53.108 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.102+0000 7f22ea67f740 1 -- 172.21.15.80:0/4050583859 shutdown_connections 2024-04-24T23:30:53.108 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.102+0000 7f22ea67f740 1 -- 172.21.15.80:0/4050583859 wait complete. 2024-04-24T23:30:53.108 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.102+0000 7f22ea67f740 20 client.25139 trim_cache size 0 max 16384 2024-04-24T23:30:53.108 INFO:teuthology.orchestra.run.smithi080.stdout:[] 2024-04-24T23:30:53.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.103+0000 7f22e796a640 1 -- 172.21.15.80:0/46174579 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f22cc0a2b90 msgr2=0x7f22cc0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:53.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.103+0000 7f22e796a640 1 --2- 172.21.15.80:0/46174579 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f22cc0a2b90 0x7f22cc0a5050 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f22dc0096c0 tx=0x7f22dc009260 comp rx=0 tx=0).stop 2024-04-24T23:30:53.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.103+0000 7f22e796a640 1 -- 172.21.15.80:0/46174579 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f22e01009b0 msgr2=0x7f22e019a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:53.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.103+0000 7f22e796a640 1 --2- 172.21.15.80:0/46174579 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f22e01009b0 0x7f22e019a190 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f22d000d950 tx=0x7f22d000de10 comp rx=0 tx=0).stop 2024-04-24T23:30:53.110 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.103+0000 7f22e796a640 1 -- 172.21.15.80:0/46174579 shutdown_connections 2024-04-24T23:30:53.110 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.103+0000 7f22e796a640 1 --2- 172.21.15.80:0/46174579 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f22cc0a2b90 0x7f22cc0a5050 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:53.110 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.103+0000 7f22e796a640 1 --2- 172.21.15.80:0/46174579 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f22e0101350 0x7f22e01a11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:53.110 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.103+0000 7f22e796a640 1 --2- 172.21.15.80:0/46174579 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f22e01009b0 0x7f22e019a190 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:53.110 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.103+0000 7f22e796a640 1 --2- 172.21.15.80:0/46174579 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f22e0100050 0x7f22e0199c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:53.110 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.103+0000 7f22e796a640 1 -- 172.21.15.80:0/46174579 >> 172.21.15.80:0/46174579 conn(0x7f22e00fbbf0 msgr2=0x7f22e00fd360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:53.110 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.103+0000 7f22e796a640 1 -- 172.21.15.80:0/46174579 shutdown_connections 2024-04-24T23:30:53.110 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.103+0000 7f22e796a640 1 -- 172.21.15.80:0/46174579 wait complete. 2024-04-24T23:30:53.116 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T23:30:53.251 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.249+0000 7fb5544d6640 1 -- 172.21.15.80:0/3837058114 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb54c103340 msgr2=0x7fb54c109bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:53.252 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.249+0000 7fb5544d6640 1 --2- 172.21.15.80:0/3837058114 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb54c103340 0x7fb54c109bd0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fb544009ca0 tx=0x7fb54402a8c0 comp rx=0 tx=0).stop 2024-04-24T23:30:53.252 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.249+0000 7fb5544d6640 1 -- 172.21.15.80:0/3837058114 shutdown_connections 2024-04-24T23:30:53.252 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.249+0000 7fb5544d6640 1 --2- 172.21.15.80:0/3837058114 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb54c103340 0x7fb54c109bd0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:53.252 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.249+0000 7fb5544d6640 1 --2- 172.21.15.80:0/3837058114 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb54c102980 0x7fb54c102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:53.252 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.249+0000 7fb5544d6640 1 --2- 172.21.15.80:0/3837058114 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb54c101790 0x7fb54c101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:53.252 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.249+0000 7fb5544d6640 1 -- 172.21.15.80:0/3837058114 >> 172.21.15.80:0/3837058114 conn(0x7fb54c0fcd20 msgr2=0x7fb54c0ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:53.252 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.249+0000 7fb5544d6640 1 -- 172.21.15.80:0/3837058114 shutdown_connections 2024-04-24T23:30:53.252 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.249+0000 7fb5544d6640 1 -- 172.21.15.80:0/3837058114 wait complete. 2024-04-24T23:30:53.252 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.249+0000 7fb5544d6640 1 Processor -- start 2024-04-24T23:30:53.252 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.249+0000 7fb5544d6640 1 -- start start 2024-04-24T23:30:53.252 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.250+0000 7fb5544d6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb54c101790 0x7fb54c197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:53.253 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.250+0000 7fb5544d6640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb54c102980 0x7fb54c197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:53.253 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.250+0000 7fb5544d6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb54c103340 0x7fb54c19f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:53.253 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.250+0000 7fb5544d6640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fb54c10d410 con 0x7fb54c102980 2024-04-24T23:30:53.253 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.250+0000 7fb5544d6640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb54c10d290 con 0x7fb54c103340 2024-04-24T23:30:53.253 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.250+0000 7fb5544d6640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb54c10d590 con 0x7fb54c101790 2024-04-24T23:30:53.253 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.250+0000 7fb551a71640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb54c102980 0x7fb54c197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:53.253 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.250+0000 7fb551a71640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb54c102980 0x7fb54c197f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:55334/0 (socket says 172.21.15.80:55334) 2024-04-24T23:30:53.253 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.250+0000 7fb551a71640 1 -- 172.21.15.80:0/474162849 learned_addr learned my addr 172.21.15.80:0/474162849 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:30:53.253 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.250+0000 7fb552272640 1 --2- 172.21.15.80:0/474162849 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb54c101790 0x7fb54c197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:53.253 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.250+0000 7fb552a73640 1 --2- 172.21.15.80:0/474162849 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb54c103340 0x7fb54c19f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:53.253 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.250+0000 7fb551a71640 1 -- 172.21.15.80:0/474162849 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb54c101790 msgr2=0x7fb54c197a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:53.253 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.250+0000 7fb551a71640 1 --2- 172.21.15.80:0/474162849 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb54c101790 0x7fb54c197a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:53.253 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.250+0000 7fb551a71640 1 -- 172.21.15.80:0/474162849 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb54c103340 msgr2=0x7fb54c19f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:53.253 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.250+0000 7fb551a71640 1 --2- 172.21.15.80:0/474162849 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb54c103340 0x7fb54c19f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:53.253 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.250+0000 7fb551a71640 1 -- 172.21.15.80:0/474162849 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb54c19f670 con 0x7fb54c102980 2024-04-24T23:30:53.254 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.250+0000 7fb551a71640 1 --2- 172.21.15.80:0/474162849 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb54c102980 0x7fb54c197f90 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fb53c00e9f0 tx=0x7fb53c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:53.254 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.250+0000 7fb5437fe640 1 -- 172.21.15.80:0/474162849 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb53c00cd80 con 0x7fb54c102980 2024-04-24T23:30:53.254 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.251+0000 7fb5544d6640 1 -- 172.21.15.80:0/474162849 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb54c19f960 con 0x7fb54c102980 2024-04-24T23:30:53.254 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.251+0000 7fb5437fe640 1 -- 172.21.15.80:0/474162849 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb53c010910 con 0x7fb54c102980 2024-04-24T23:30:53.254 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.251+0000 7fb5437fe640 1 -- 172.21.15.80:0/474162849 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb53c0188f0 con 0x7fb54c102980 2024-04-24T23:30:53.254 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.251+0000 7fb5544d6640 1 -- 172.21.15.80:0/474162849 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb54c073970 con 0x7fb54c102980 2024-04-24T23:30:53.264 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.253+0000 7fb5437fe640 1 -- 172.21.15.80:0/474162849 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb53c010ac0 con 0x7fb54c102980 2024-04-24T23:30:53.264 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.254+0000 7fb5437fe640 1 --2- 172.21.15.80:0/474162849 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fb5380a2c10 0x7fb5380a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:53.264 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.254+0000 7fb552272640 1 --2- 172.21.15.80:0/474162849 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fb5380a2c10 0x7fb5380a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:53.264 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.254+0000 7fb5437fe640 1 -- 172.21.15.80:0/474162849 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fb53c0cf960 con 0x7fb54c102980 2024-04-24T23:30:53.264 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.254+0000 7fb5571e1740 1 Processor -- start 2024-04-24T23:30:53.264 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.254+0000 7fb552272640 1 --2- 172.21.15.80:0/474162849 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fb5380a2c10 0x7fb5380a50d0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fb548005fd0 tx=0x7fb54800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:53.264 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.255+0000 7fb5571e1740 1 -- start start 2024-04-24T23:30:53.265 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.255+0000 7fb5571e1740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5610df646230 0x5610df646650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:53.265 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.255+0000 7fb5571e1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5610df646c20 0x5610df64b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:53.265 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.255+0000 7fb5571e1740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5610df64bd90 0x5610df64e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:53.265 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.255+0000 7fb5571e1740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x5610df5aa670 con 0x5610df64bd90 2024-04-24T23:30:53.265 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.255+0000 7fb5571e1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5610df552970 con 0x5610df646c20 2024-04-24T23:30:53.265 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.255+0000 7fb5571e1740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5610df5a6630 con 0x5610df646230 2024-04-24T23:30:53.265 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.255+0000 7fb552a73640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5610df64bd90 0x5610df64e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:53.265 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.255+0000 7fb552a73640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5610df64bd90 0x5610df64e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:55342/0 (socket says 172.21.15.80:55342) 2024-04-24T23:30:53.265 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.255+0000 7fb552a73640 1 -- 172.21.15.80:0/1336190952 learned_addr learned my addr 172.21.15.80:0/1336190952 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:30:53.265 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.255+0000 7fb552272640 1 --2- 172.21.15.80:0/1336190952 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5610df646230 0x5610df646650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:53.265 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.255+0000 7fb551a71640 1 --2- 172.21.15.80:0/1336190952 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5610df646c20 0x5610df64b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:53.265 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.255+0000 7fb552a73640 1 -- 172.21.15.80:0/1336190952 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5610df646230 msgr2=0x5610df646650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:53.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.255+0000 7fb552a73640 1 --2- 172.21.15.80:0/1336190952 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5610df646230 0x5610df646650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:53.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.255+0000 7fb552a73640 1 -- 172.21.15.80:0/1336190952 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5610df646c20 msgr2=0x5610df64b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:53.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.255+0000 7fb552a73640 1 --2- 172.21.15.80:0/1336190952 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5610df646c20 0x5610df64b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:53.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.255+0000 7fb552a73640 1 -- 172.21.15.80:0/1336190952 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610df576400 con 0x5610df64bd90 2024-04-24T23:30:53.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.256+0000 7fb552a73640 1 --2- 172.21.15.80:0/1336190952 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5610df64bd90 0x5610df64e220 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fb544009ca0 tx=0x7fb544009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a02b0ea2550e8af server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:53.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.256+0000 7fb540bfe640 1 -- 172.21.15.80:0/1336190952 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5440098c0 con 0x5610df64bd90 2024-04-24T23:30:53.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.256+0000 7fb540bfe640 1 -- 172.21.15.80:0/1336190952 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb544036040 con 0x5610df64bd90 2024-04-24T23:30:53.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.257+0000 7fb540bfe640 1 -- 172.21.15.80:0/1336190952 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb544004290 con 0x5610df64bd90 2024-04-24T23:30:53.267 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.257+0000 7fb5571e1740 1 -- 172.21.15.80:0/1336190952 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5610df64bd90 msgr2=0x5610df64e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:53.267 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.257+0000 7fb5571e1740 1 --2- 172.21.15.80:0/1336190952 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5610df64bd90 0x5610df64e220 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fb544009ca0 tx=0x7fb544009360 comp rx=0 tx=0).stop 2024-04-24T23:30:53.267 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.257+0000 7fb5571e1740 1 -- 172.21.15.80:0/1336190952 shutdown_connections 2024-04-24T23:30:53.267 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.257+0000 7fb5571e1740 1 --2- 172.21.15.80:0/1336190952 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5610df64bd90 0x5610df64e220 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:53.267 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.257+0000 7fb5571e1740 1 --2- 172.21.15.80:0/1336190952 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5610df646c20 0x5610df64b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:53.267 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.257+0000 7fb5571e1740 1 --2- 172.21.15.80:0/1336190952 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5610df646230 0x5610df646650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:53.267 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.257+0000 7fb5571e1740 1 -- 172.21.15.80:0/1336190952 >> 172.21.15.80:0/1336190952 conn(0x5610df6418a0 msgr2=0x5610df641ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:53.268 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.257+0000 7fb5571e1740 1 -- 172.21.15.80:0/1336190952 shutdown_connections 2024-04-24T23:30:53.268 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.257+0000 7fb5571e1740 1 -- 172.21.15.80:0/1336190952 wait complete. 2024-04-24T23:30:53.268 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.258+0000 7fb5571e1740 1 -- start start 2024-04-24T23:30:53.268 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.258+0000 7fb5571e1740 1 Processor -- start 2024-04-24T23:30:53.268 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.258+0000 7fb5571e1740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5610df646230 0x5610df6500d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:53.268 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.258+0000 7fb5571e1740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5610df646c20 0x5610df650610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:53.268 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.258+0000 7fb5571e1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5610df64bd90 0x5610df657650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:53.268 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.258+0000 7fb552272640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5610df646230 0x5610df6500d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:53.268 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.258+0000 7fb552272640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5610df646230 0x5610df6500d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:55348/0 (socket says 172.21.15.80:55348) 2024-04-24T23:30:53.268 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.259+0000 7fb552272640 1 -- 172.21.15.80:0/1330171069 learned_addr learned my addr 172.21.15.80:0/1330171069 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:30:53.268 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.259+0000 7fb5571e1740 1 -- 172.21.15.80:0/1330171069 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x5610df64eb50 con 0x5610df646230 2024-04-24T23:30:53.268 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.259+0000 7fb5571e1740 1 -- 172.21.15.80:0/1330171069 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5610df566050 con 0x5610df64bd90 2024-04-24T23:30:53.268 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.259+0000 7fb551a71640 1 --2- 172.21.15.80:0/1330171069 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5610df646c20 0x5610df650610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:53.269 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.259+0000 7fb5571e1740 1 -- 172.21.15.80:0/1330171069 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5610df64ecd0 con 0x5610df646c20 2024-04-24T23:30:53.269 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.259+0000 7fb552a73640 1 --2- 172.21.15.80:0/1330171069 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5610df64bd90 0x5610df657650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:53.269 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.259+0000 7fb5571e1740 10 client.0 _ll_register_callbacks cb 0x5610df2bdbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:30:53.269 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.259+0000 7fb551a71640 1 -- 172.21.15.80:0/1330171069 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5610df64bd90 msgr2=0x5610df657650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:53.269 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.259+0000 7fb551a71640 1 --2- 172.21.15.80:0/1330171069 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5610df64bd90 0x5610df657650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:53.269 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.259+0000 7fb551a71640 1 -- 172.21.15.80:0/1330171069 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5610df646230 msgr2=0x5610df6500d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:53.269 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.259+0000 7fb551a71640 1 --2- 172.21.15.80:0/1330171069 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5610df646230 0x5610df6500d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:53.269 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.259+0000 7fb551a71640 1 -- 172.21.15.80:0/1330171069 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610df537dc0 con 0x5610df646c20 2024-04-24T23:30:53.269 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.259+0000 7fb551a71640 1 --2- 172.21.15.80:0/1330171069 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5610df646c20 0x5610df650610 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb53c00c6a0 tx=0x7fb53c09bdf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8072d267e2de17b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:53.269 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.259+0000 7fb535ffb640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-24T23:30:53.269 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.260+0000 7fb535ffb640 1 -- 172.21.15.80:0/1330171069 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb53c018d50 con 0x5610df646c20 2024-04-24T23:30:53.269 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.260+0000 7fb5571e1740 1 -- 172.21.15.80:0/1330171069 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb544036040 con 0x5610df646c20 2024-04-24T23:30:53.269 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.260+0000 7fb535ffb640 1 -- 172.21.15.80:0/1330171069 <== mon.1 v2:172.21.15.148:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fb53c09b240 con 0x5610df646c20 2024-04-24T23:30:53.269 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.260+0000 7fb535ffb640 1 -- 172.21.15.80:0/1330171069 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb53c09b440 con 0x5610df646c20 2024-04-24T23:30:53.269 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.260+0000 7fb5571e1740 10 client.25303 fetch_fsmap learned FSMap version 25 2024-04-24T23:30:53.270 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.260+0000 7fb5571e1740 1 -- 172.21.15.80:0/1330171069 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5610df2be3f0 con 0x5610df646c20 2024-04-24T23:30:53.270 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.260+0000 7fb535ffb640 1 -- 172.21.15.80:0/1330171069 <== mon.1 v2:172.21.15.148:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb53c0b9070 con 0x5610df646c20 2024-04-24T23:30:53.270 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.261+0000 7fb535ffb640 1 -- 172.21.15.80:0/1330171069 <== mon.1 v2:172.21.15.148:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fb53c09acb0 con 0x5610df646c20 2024-04-24T23:30:53.270 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.261+0000 7fb5571e1740 10 client.25303 fetch_fsmap finished waiting for FSMap version 25 2024-04-24T23:30:53.270 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.261+0000 7fb5571e1740 10 client.25303 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:30:53.270 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.261+0000 7fb5571e1740 20 client.25303 populate_metadata read hostname 'smithi080' 2024-04-24T23:30:53.271 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.261+0000 7fb5571e1740 1 --2- 172.21.15.80:0/1330171069 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x5610df65beb0 0x5610df65e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:53.271 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.261+0000 7fb5571e1740 4 client.25303 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:30:53.271 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.261+0000 7fb5571e1740 1 -- 172.21.15.80:0/1330171069 --> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fb544036040 con 0x5610df65beb0 2024-04-24T23:30:53.271 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.261+0000 7fb552272640 1 --2- 172.21.15.80:0/1330171069 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x5610df65beb0 0x5610df65e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:53.271 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.262+0000 7fb552272640 1 --2- 172.21.15.80:0/1330171069 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x5610df65beb0 0x5610df65e340 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0eeab7d26c241fe server_cookie=9b5874d02b5ae0a9 in_seq=0 out_seq=0 2024-04-24T23:30:53.271 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.262+0000 7fb535ffb640 10 client.25303 ms_handle_connect on v2:172.21.15.148:6838/2952757594 2024-04-24T23:30:53.271 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.262+0000 7fb535ffb640 1 -- 172.21.15.80:0/1330171069 <== mds.0 v2:172.21.15.148:6838/2952757594 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7fb544036040 con 0x5610df65beb0 2024-04-24T23:30:53.271 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.262+0000 7fb535ffb640 10 client.25303 handle_command_reply: tid=0 2024-04-24T23:30:53.271 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.262+0000 7fb5571e1740 1 client.25303 shutdown 2024-04-24T23:30:53.271 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.263+0000 7fb5571e1740 1 -- 172.21.15.80:0/1330171069 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5610df646c20 msgr2=0x5610df650610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:53.272 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.263+0000 7fb5571e1740 1 --2- 172.21.15.80:0/1330171069 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5610df646c20 0x5610df650610 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb53c00c6a0 tx=0x7fb53c09bdf0 comp rx=0 tx=0).stop 2024-04-24T23:30:53.272 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.263+0000 7fb5571e1740 1 -- 172.21.15.80:0/1330171069 shutdown_connections 2024-04-24T23:30:53.272 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.263+0000 7fb5571e1740 1 --2- 172.21.15.80:0/1330171069 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x5610df65beb0 0x5610df65e340 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:53.272 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.263+0000 7fb5571e1740 1 --2- 172.21.15.80:0/1330171069 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5610df64bd90 0x5610df657650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:53.272 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.263+0000 7fb5571e1740 1 --2- 172.21.15.80:0/1330171069 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5610df646c20 0x5610df650610 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:53.272 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.263+0000 7fb5571e1740 1 --2- 172.21.15.80:0/1330171069 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5610df646230 0x5610df6500d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:53.272 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.263+0000 7fb5571e1740 1 -- 172.21.15.80:0/1330171069 >> 172.21.15.80:0/1330171069 conn(0x5610df6418a0 msgr2=0x5610df640b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:53.272 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.263+0000 7fb535ffb640 0 client.25303 ms_handle_reset on v2:172.21.15.148:6838/2952757594 2024-04-24T23:30:53.272 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.263+0000 7fb5571e1740 1 -- 172.21.15.80:0/1330171069 shutdown_connections 2024-04-24T23:30:53.272 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.263+0000 7fb5571e1740 1 -- 172.21.15.80:0/1330171069 wait complete. 2024-04-24T23:30:53.272 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.263+0000 7fb5571e1740 20 client.25303 trim_cache size 0 max 16384 2024-04-24T23:30:53.279 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.277+0000 7fb5571e1740 1 Processor -- start 2024-04-24T23:30:53.279 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.277+0000 7fb5571e1740 1 -- start start 2024-04-24T23:30:53.280 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.277+0000 7fb5571e1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5610df646230 0x5610df64b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:53.280 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.277+0000 7fb5571e1740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5610df646c20 0x5610df657610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:53.280 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.277+0000 7fb5571e1740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5610df64bd90 0x5610df647410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:53.280 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.277+0000 7fb552a73640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5610df64bd90 0x5610df647410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:53.280 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.277+0000 7fb552a73640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5610df64bd90 0x5610df647410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:55354/0 (socket says 172.21.15.80:55354) 2024-04-24T23:30:53.280 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.277+0000 7fb552272640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5610df646230 0x5610df64b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:53.280 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.277+0000 7fb5571e1740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fb54c1aab20 con 0x5610df64bd90 2024-04-24T23:30:53.280 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.277+0000 7fb552272640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5610df646230 0x5610df64b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.80:57814/0 (socket says 172.21.15.80:57814) 2024-04-24T23:30:53.280 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.277+0000 7fb552a73640 1 -- 172.21.15.80:0/1358551503 learned_addr learned my addr 172.21.15.80:0/1358551503 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:30:53.280 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.277+0000 7fb5571e1740 1 -- 172.21.15.80:0/1358551503 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb54c1a83f0 con 0x5610df646230 2024-04-24T23:30:53.280 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.277+0000 7fb5571e1740 1 -- 172.21.15.80:0/1358551503 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb54c1a6240 con 0x5610df646c20 2024-04-24T23:30:53.280 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.277+0000 7fb551a71640 1 --2- 172.21.15.80:0/1358551503 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5610df646c20 0x5610df657610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:53.280 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.278+0000 7fb552272640 1 -- 172.21.15.80:0/1358551503 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5610df646c20 msgr2=0x5610df657610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:53.280 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.278+0000 7fb552272640 1 --2- 172.21.15.80:0/1358551503 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5610df646c20 0x5610df657610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:53.281 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.278+0000 7fb552272640 1 -- 172.21.15.80:0/1358551503 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5610df64bd90 msgr2=0x5610df647410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:53.281 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.278+0000 7fb552272640 1 --2- 172.21.15.80:0/1358551503 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5610df64bd90 0x5610df647410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:53.281 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.278+0000 7fb552272640 1 -- 172.21.15.80:0/1358551503 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610df3887d0 con 0x5610df646230 2024-04-24T23:30:53.281 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.278+0000 7fb552a73640 1 --2- 172.21.15.80:0/1358551503 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5610df64bd90 0x5610df647410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:30:53.281 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.278+0000 7fb551a71640 1 --2- 172.21.15.80:0/1358551503 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5610df646c20 0x5610df657610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:30:53.281 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.278+0000 7fb552272640 1 --2- 172.21.15.80:0/1358551503 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5610df646230 0x5610df64b270 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb54800ba00 tx=0x7fb548013000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac4519c0a829b10a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:53.281 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.278+0000 7fb537fff640 1 -- 172.21.15.80:0/1358551503 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb548030070 con 0x5610df646230 2024-04-24T23:30:53.281 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.278+0000 7fb537fff640 1 -- 172.21.15.80:0/1358551503 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb548020ac0 con 0x5610df646230 2024-04-24T23:30:53.281 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.278+0000 7fb537fff640 1 -- 172.21.15.80:0/1358551503 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb54802a690 con 0x5610df646230 2024-04-24T23:30:53.281 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.279+0000 7fb5571e1740 1 -- 172.21.15.80:0/1358551503 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5610df646230 msgr2=0x5610df64b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:53.281 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.279+0000 7fb5571e1740 1 --2- 172.21.15.80:0/1358551503 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5610df646230 0x5610df64b270 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb54800ba00 tx=0x7fb548013000 comp rx=0 tx=0).stop 2024-04-24T23:30:53.281 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.279+0000 7fb5571e1740 1 -- 172.21.15.80:0/1358551503 shutdown_connections 2024-04-24T23:30:53.281 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.279+0000 7fb5571e1740 1 --2- 172.21.15.80:0/1358551503 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5610df64bd90 0x5610df647410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:53.281 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.279+0000 7fb5571e1740 1 --2- 172.21.15.80:0/1358551503 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5610df646c20 0x5610df657610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:53.281 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.279+0000 7fb5571e1740 1 --2- 172.21.15.80:0/1358551503 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5610df646230 0x5610df64b270 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:53.282 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.279+0000 7fb5571e1740 1 -- 172.21.15.80:0/1358551503 >> 172.21.15.80:0/1358551503 conn(0x5610df6418a0 msgr2=0x5610df640b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:53.282 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.279+0000 7fb5571e1740 1 -- 172.21.15.80:0/1358551503 shutdown_connections 2024-04-24T23:30:53.282 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.279+0000 7fb5571e1740 1 -- 172.21.15.80:0/1358551503 wait complete. 2024-04-24T23:30:53.282 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.280+0000 7fb5571e1740 1 -- start start 2024-04-24T23:30:53.282 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.280+0000 7fb5571e1740 1 Processor -- start 2024-04-24T23:30:53.282 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.280+0000 7fb5571e1740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5610df646230 0x5610df655120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:53.285 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.280+0000 7fb5571e1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5610df646c20 0x5610df655660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:53.286 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.280+0000 7fb552272640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5610df646230 0x5610df655120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:53.286 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.280+0000 7fb552272640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5610df646230 0x5610df655120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:55358/0 (socket says 172.21.15.80:55358) 2024-04-24T23:30:53.286 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.280+0000 7fb552272640 1 -- 172.21.15.80:0/3503943856 learned_addr learned my addr 172.21.15.80:0/3503943856 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:30:53.286 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.280+0000 7fb5571e1740 1 --2- 172.21.15.80:0/3503943856 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5610df64bd90 0x5610df656600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:53.286 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.280+0000 7fb5571e1740 1 -- 172.21.15.80:0/3503943856 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fb54c1a68b0 con 0x5610df646230 2024-04-24T23:30:53.286 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.280+0000 7fb5571e1740 1 -- 172.21.15.80:0/3503943856 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb54c1a7d20 con 0x5610df646c20 2024-04-24T23:30:53.286 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.280+0000 7fb5571e1740 1 -- 172.21.15.80:0/3503943856 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb54c1a6f80 con 0x5610df64bd90 2024-04-24T23:30:53.286 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.281+0000 7fb551a71640 1 --2- 172.21.15.80:0/3503943856 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5610df646c20 0x5610df655660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:53.286 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.281+0000 7fb552a73640 1 --2- 172.21.15.80:0/3503943856 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5610df64bd90 0x5610df656600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:53.286 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.281+0000 7fb552a73640 1 -- 172.21.15.80:0/3503943856 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5610df646c20 msgr2=0x5610df655660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:53.286 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.281+0000 7fb552a73640 1 --2- 172.21.15.80:0/3503943856 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5610df646c20 0x5610df655660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:53.286 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.281+0000 7fb552a73640 1 -- 172.21.15.80:0/3503943856 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5610df646230 msgr2=0x5610df655120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:53.286 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.281+0000 7fb552a73640 1 --2- 172.21.15.80:0/3503943856 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5610df646230 0x5610df655120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:53.286 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.281+0000 7fb552a73640 1 -- 172.21.15.80:0/3503943856 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610df656bf0 con 0x5610df64bd90 2024-04-24T23:30:53.287 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.281+0000 7fb552272640 1 --2- 172.21.15.80:0/3503943856 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5610df646230 0x5610df655120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:30:53.287 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.281+0000 7fb5571e1740 10 client.0 _ll_register_callbacks cb 0x7fb53c09ba00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:30:53.287 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.281+0000 7fb5571e1740 1 -- 172.21.15.80:0/3503943856 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb548020ac0 con 0x5610df64bd90 2024-04-24T23:30:53.287 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.281+0000 7fb552a73640 1 --2- 172.21.15.80:0/3503943856 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5610df64bd90 0x5610df656600 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb544000f80 tx=0x7fb544003ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f35660078a1f6468 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:53.287 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.281+0000 7fb535ffb640 10 client.25309 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-24T23:30:53.287 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.281+0000 7fb535ffb640 1 -- 172.21.15.80:0/3503943856 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb544052020 con 0x5610df64bd90 2024-04-24T23:30:53.287 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.282+0000 7fb535ffb640 1 -- 172.21.15.80:0/3503943856 <== mon.1 v2:172.21.15.148:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fb54402f610 con 0x5610df64bd90 2024-04-24T23:30:53.287 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.282+0000 7fb535ffb640 1 -- 172.21.15.80:0/3503943856 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb54402f850 con 0x5610df64bd90 2024-04-24T23:30:53.287 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.282+0000 7fb5571e1740 10 client.25309 fetch_fsmap learned FSMap version 25 2024-04-24T23:30:53.287 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.282+0000 7fb5571e1740 1 -- 172.21.15.80:0/3503943856 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({fsmap=25}) -- 0x5610df6c2670 con 0x5610df64bd90 2024-04-24T23:30:53.287 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.282+0000 7fb535ffb640 1 -- 172.21.15.80:0/3503943856 <== mon.1 v2:172.21.15.148:3300/0 4 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fb54402f9f0 con 0x5610df64bd90 2024-04-24T23:30:53.287 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.282+0000 7fb5571e1740 10 client.25309 fetch_fsmap finished waiting for FSMap version 25 2024-04-24T23:30:53.287 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.282+0000 7fb5571e1740 10 client.25309 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:30:53.287 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.282+0000 7fb5571e1740 20 client.25309 populate_metadata read hostname 'smithi080' 2024-04-24T23:30:53.288 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.282+0000 7fb5571e1740 1 --2- 172.21.15.80:0/3503943856 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x5610df6c2c30 0x5610df6c5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:53.288 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.282+0000 7fb5571e1740 4 client.25309 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:30:53.288 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.282+0000 7fb5571e1740 1 -- 172.21.15.80:0/3503943856 --> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5610df5b7bb0 con 0x5610df6c2c30 2024-04-24T23:30:53.288 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.282+0000 7fb552272640 1 --2- 172.21.15.80:0/3503943856 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x5610df6c2c30 0x5610df6c5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:53.288 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.283+0000 7fb535ffb640 10 client.25309 ms_handle_connect on v2:172.21.15.148:6838/2952757594 2024-04-24T23:30:53.288 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.283+0000 7fb552272640 1 --2- 172.21.15.80:0/3503943856 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x5610df6c2c30 0x5610df6c5050 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f30a7d7e9f574ad server_cookie=1566b99040a6ece0 in_seq=0 out_seq=0 2024-04-24T23:30:53.288 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.283+0000 7fb535ffb640 1 -- 172.21.15.80:0/3503943856 <== mds.0 v2:172.21.15.148:6838/2952757594 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5610df5b7bb0 con 0x5610df6c2c30 2024-04-24T23:30:53.288 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.283+0000 7fb535ffb640 10 client.25309 handle_command_reply: tid=0 2024-04-24T23:30:53.288 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.283+0000 7fb5571e1740 1 client.25309 shutdown 2024-04-24T23:30:53.288 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.283+0000 7fb5571e1740 1 -- 172.21.15.80:0/3503943856 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5610df64bd90 msgr2=0x5610df656600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:53.288 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.283+0000 7fb5571e1740 1 --2- 172.21.15.80:0/3503943856 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5610df64bd90 0x5610df656600 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb544000f80 tx=0x7fb544003ba0 comp rx=0 tx=0).stop 2024-04-24T23:30:53.288 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.283+0000 7fb5571e1740 1 -- 172.21.15.80:0/3503943856 shutdown_connections 2024-04-24T23:30:53.288 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.283+0000 7fb5571e1740 1 --2- 172.21.15.80:0/3503943856 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x5610df6c2c30 0x5610df6c5050 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:53.288 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.283+0000 7fb535ffb640 0 client.25309 ms_handle_reset on v2:172.21.15.148:6838/2952757594 2024-04-24T23:30:53.289 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.283+0000 7fb5571e1740 1 --2- 172.21.15.80:0/3503943856 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5610df64bd90 0x5610df656600 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:53.289 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.283+0000 7fb5571e1740 1 --2- 172.21.15.80:0/3503943856 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5610df646c20 0x5610df655660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:53.289 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.283+0000 7fb5571e1740 1 --2- 172.21.15.80:0/3503943856 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5610df646230 0x5610df655120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:53.289 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.283+0000 7fb5571e1740 1 -- 172.21.15.80:0/3503943856 >> 172.21.15.80:0/3503943856 conn(0x5610df6418a0 msgr2=0x5610df6447b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:53.289 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.283+0000 7fb5571e1740 1 -- 172.21.15.80:0/3503943856 shutdown_connections 2024-04-24T23:30:53.289 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.283+0000 7fb5571e1740 1 -- 172.21.15.80:0/3503943856 wait complete. 2024-04-24T23:30:53.289 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.283+0000 7fb5571e1740 20 client.25309 trim_cache size 0 max 16384 2024-04-24T23:30:53.289 INFO:teuthology.orchestra.run.smithi080.stdout:[] 2024-04-24T23:30:53.290 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.284+0000 7fb5544d6640 1 -- 172.21.15.80:0/474162849 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fb5380a2c10 msgr2=0x7fb5380a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:53.290 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.284+0000 7fb5544d6640 1 --2- 172.21.15.80:0/474162849 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fb5380a2c10 0x7fb5380a50d0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fb548005fd0 tx=0x7fb54800b390 comp rx=0 tx=0).stop 2024-04-24T23:30:53.290 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.284+0000 7fb5544d6640 1 -- 172.21.15.80:0/474162849 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb54c102980 msgr2=0x7fb54c197f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:53.290 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.284+0000 7fb5544d6640 1 --2- 172.21.15.80:0/474162849 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb54c102980 0x7fb54c197f90 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fb53c00e9f0 tx=0x7fb53c00eeb0 comp rx=0 tx=0).stop 2024-04-24T23:30:53.290 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.284+0000 7fb5544d6640 1 -- 172.21.15.80:0/474162849 shutdown_connections 2024-04-24T23:30:53.290 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.284+0000 7fb5544d6640 1 --2- 172.21.15.80:0/474162849 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fb5380a2c10 0x7fb5380a50d0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:53.290 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.284+0000 7fb5544d6640 1 --2- 172.21.15.80:0/474162849 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb54c103340 0x7fb54c19f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:53.291 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.284+0000 7fb5544d6640 1 --2- 172.21.15.80:0/474162849 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb54c102980 0x7fb54c197f90 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:53.291 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.284+0000 7fb5544d6640 1 --2- 172.21.15.80:0/474162849 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb54c101790 0x7fb54c197a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:53.291 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.284+0000 7fb5544d6640 1 -- 172.21.15.80:0/474162849 >> 172.21.15.80:0/474162849 conn(0x7fb54c0fcd20 msgr2=0x7fb54c105c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:53.291 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.284+0000 7fb5544d6640 1 -- 172.21.15.80:0/474162849 shutdown_connections 2024-04-24T23:30:53.291 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:53.284+0000 7fb5544d6640 1 -- 172.21.15.80:0/474162849 wait complete. 2024-04-24T23:30:53.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:30:53 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/1547840007' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:30:53.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:30:53 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/1547840007' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:30:53.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:30:53 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/1547840007' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:30:54.299 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T23:30:54.299 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:30:54.299 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T23:30:54.406 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.404+0000 7f3ce409a640 1 -- 172.21.15.80:0/2632836629 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3cdc104b30 msgr2=0x7f3cdc106f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:54.406 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.404+0000 7f3ce409a640 1 --2- 172.21.15.80:0/2632836629 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3cdc104b30 0x7f3cdc106f20 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f3ccc009900 tx=0x7f3ccc02a410 comp rx=0 tx=0).stop 2024-04-24T23:30:54.406 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.404+0000 7f3ce409a640 1 -- 172.21.15.80:0/2632836629 shutdown_connections 2024-04-24T23:30:54.406 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.404+0000 7f3ce409a640 1 --2- 172.21.15.80:0/2632836629 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3cdc107460 0x7f3cdc1098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:54.406 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.404+0000 7f3ce409a640 1 --2- 172.21.15.80:0/2632836629 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3cdc104b30 0x7f3cdc106f20 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:54.406 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.404+0000 7f3ce409a640 1 --2- 172.21.15.80:0/2632836629 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3cdc102200 0x7f3cdc1045f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:54.406 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.404+0000 7f3ce409a640 1 -- 172.21.15.80:0/2632836629 >> 172.21.15.80:0/2632836629 conn(0x7f3cdc0fbbf0 msgr2=0x7f3cdc0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:54.406 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.404+0000 7f3ce409a640 1 -- 172.21.15.80:0/2632836629 shutdown_connections 2024-04-24T23:30:54.407 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.404+0000 7f3ce409a640 1 -- 172.21.15.80:0/2632836629 wait complete. 2024-04-24T23:30:54.407 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.405+0000 7f3ce409a640 1 Processor -- start 2024-04-24T23:30:54.407 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.405+0000 7f3ce409a640 1 -- start start 2024-04-24T23:30:54.407 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.405+0000 7f3ce409a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3cdc102200 0x7f3cdc197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:54.407 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.405+0000 7f3ce409a640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3cdc104b30 0x7f3cdc198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:54.407 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.405+0000 7f3ce409a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3cdc107460 0x7f3cdc19f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:54.407 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.405+0000 7f3ce409a640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f3cdc10d400 con 0x7f3cdc104b30 2024-04-24T23:30:54.407 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.405+0000 7f3ce409a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3cdc10d280 con 0x7f3cdc102200 2024-04-24T23:30:54.407 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.405+0000 7f3ce409a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3cdc10d580 con 0x7f3cdc107460 2024-04-24T23:30:54.407 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.405+0000 7f3ce1635640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3cdc104b30 0x7f3cdc198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:54.407 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.405+0000 7f3ce1635640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3cdc104b30 0x7f3cdc198070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:55372/0 (socket says 172.21.15.80:55372) 2024-04-24T23:30:54.407 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.405+0000 7f3ce1635640 1 -- 172.21.15.80:0/66081561 learned_addr learned my addr 172.21.15.80:0/66081561 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:30:54.407 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.405+0000 7f3ce1e36640 1 --2- 172.21.15.80:0/66081561 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3cdc102200 0x7f3cdc197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:54.407 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.405+0000 7f3ce2637640 1 --2- 172.21.15.80:0/66081561 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3cdc107460 0x7f3cdc19f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:54.408 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.406+0000 7f3ce1635640 1 -- 172.21.15.80:0/66081561 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3cdc107460 msgr2=0x7f3cdc19f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:54.408 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.406+0000 7f3ce1635640 1 --2- 172.21.15.80:0/66081561 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3cdc107460 0x7f3cdc19f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:54.408 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.406+0000 7f3ce1635640 1 -- 172.21.15.80:0/66081561 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3cdc102200 msgr2=0x7f3cdc197b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:54.408 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.406+0000 7f3ce1635640 1 --2- 172.21.15.80:0/66081561 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3cdc102200 0x7f3cdc197b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:54.408 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.406+0000 7f3ce1635640 1 -- 172.21.15.80:0/66081561 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cdc19f750 con 0x7f3cdc104b30 2024-04-24T23:30:54.408 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.406+0000 7f3ce2637640 1 --2- 172.21.15.80:0/66081561 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3cdc107460 0x7f3cdc19f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:30:54.408 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.406+0000 7f3ce1635640 1 --2- 172.21.15.80:0/66081561 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3cdc104b30 0x7f3cdc198070 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f3ccc031610 tx=0x7f3ccc004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:54.408 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.406+0000 7f3cd2ffd640 1 -- 172.21.15.80:0/66081561 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ccc03f030 con 0x7f3cdc104b30 2024-04-24T23:30:54.408 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.406+0000 7f3cd2ffd640 1 -- 172.21.15.80:0/66081561 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ccc003ad0 con 0x7f3cdc104b30 2024-04-24T23:30:54.408 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.406+0000 7f3ce409a640 1 -- 172.21.15.80:0/66081561 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cdc0ffff0 con 0x7f3cdc104b30 2024-04-24T23:30:54.408 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.406+0000 7f3cd2ffd640 1 -- 172.21.15.80:0/66081561 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ccc035070 con 0x7f3cdc104b30 2024-04-24T23:30:54.408 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.406+0000 7f3ce409a640 1 -- 172.21.15.80:0/66081561 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cdc100420 con 0x7f3cdc104b30 2024-04-24T23:30:54.409 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.409+0000 7f3cd2ffd640 1 -- 172.21.15.80:0/66081561 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3ccc042030 con 0x7f3cdc104b30 2024-04-24T23:30:54.409 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.409+0000 7f3cd2ffd640 1 --2- 172.21.15.80:0/66081561 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f3cc80a2b90 0x7f3cc80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:54.409 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.409+0000 7f3ce1e36640 1 --2- 172.21.15.80:0/66081561 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f3cc80a2b90 0x7f3cc80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:54.409 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.409+0000 7f3ce6da3740 1 Processor -- start 2024-04-24T23:30:54.409 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.410+0000 7f3cd2ffd640 1 -- 172.21.15.80:0/66081561 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f3ccc0efef0 con 0x7f3cdc104b30 2024-04-24T23:30:54.409 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.410+0000 7f3ce6da3740 1 -- start start 2024-04-24T23:30:54.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.410+0000 7f3ce6da3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55be3ca0f5a0 0x55be3ca0f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:54.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.410+0000 7f3ce6da3740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55be3ca0ff90 0x55be3ca14a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:54.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.410+0000 7f3ce6da3740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55be3ca15100 0x55be3ca17590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:54.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.410+0000 7f3ce6da3740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x55be3c96a2c0 con 0x55be3ca15100 2024-04-24T23:30:54.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.410+0000 7f3ce6da3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55be3c912910 con 0x55be3ca0f5a0 2024-04-24T23:30:54.417 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.410+0000 7f3ce6da3740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55be3c966280 con 0x55be3ca0ff90 2024-04-24T23:30:54.417 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.410+0000 7f3ce1e36640 1 --2- 172.21.15.80:0/66081561 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f3cc80a2b90 0x7f3cc80a5050 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f3cd8009bd0 tx=0x7f3cd8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:54.417 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.410+0000 7f3ce2637640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55be3ca15100 0x55be3ca17590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:54.417 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.410+0000 7f3ce1e36640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55be3ca0f5a0 0x55be3ca0f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:54.417 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.410+0000 7f3ce2637640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55be3ca15100 0x55be3ca17590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:55384/0 (socket says 172.21.15.80:55384) 2024-04-24T23:30:54.417 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.410+0000 7f3ce2637640 1 -- 172.21.15.80:0/411661248 learned_addr learned my addr 172.21.15.80:0/411661248 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:30:54.417 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.410+0000 7f3ce1635640 1 --2- 172.21.15.80:0/411661248 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55be3ca0ff90 0x55be3ca14a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:54.417 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.411+0000 7f3ce2637640 1 -- 172.21.15.80:0/411661248 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55be3ca0ff90 msgr2=0x55be3ca14a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:54.417 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.411+0000 7f3ce2637640 1 --2- 172.21.15.80:0/411661248 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55be3ca0ff90 0x55be3ca14a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:54.417 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.411+0000 7f3ce2637640 1 -- 172.21.15.80:0/411661248 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55be3ca0f5a0 msgr2=0x55be3ca0f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:54.417 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.411+0000 7f3ce2637640 1 --2- 172.21.15.80:0/411661248 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55be3ca0f5a0 0x55be3ca0f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:54.417 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.411+0000 7f3ce2637640 1 -- 172.21.15.80:0/411661248 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be3c923fb0 con 0x55be3ca15100 2024-04-24T23:30:54.417 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.411+0000 7f3ce1e36640 1 --2- 172.21.15.80:0/411661248 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55be3ca0f5a0 0x55be3ca0f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:30:54.417 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.411+0000 7f3ce2637640 1 --2- 172.21.15.80:0/411661248 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55be3ca15100 0x55be3ca17590 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f3cd4009fb0 tx=0x7f3cd4010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1f4e77ba342d61c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:54.418 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.411+0000 7f3cc7fff640 1 -- 172.21.15.80:0/411661248 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cd4016070 con 0x55be3ca15100 2024-04-24T23:30:54.418 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.411+0000 7f3cc7fff640 1 -- 172.21.15.80:0/411661248 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3cd4012bf0 con 0x55be3ca15100 2024-04-24T23:30:54.418 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.411+0000 7f3cc7fff640 1 -- 172.21.15.80:0/411661248 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cd4011c60 con 0x55be3ca15100 2024-04-24T23:30:54.418 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.412+0000 7f3ce6da3740 1 -- 172.21.15.80:0/411661248 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55be3ca15100 msgr2=0x55be3ca17590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:54.418 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.412+0000 7f3ce6da3740 1 --2- 172.21.15.80:0/411661248 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55be3ca15100 0x55be3ca17590 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f3cd4009fb0 tx=0x7f3cd4010800 comp rx=0 tx=0).stop 2024-04-24T23:30:54.418 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.412+0000 7f3ce6da3740 1 -- 172.21.15.80:0/411661248 shutdown_connections 2024-04-24T23:30:54.418 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.412+0000 7f3ce6da3740 1 --2- 172.21.15.80:0/411661248 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55be3ca15100 0x55be3ca17590 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:54.418 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.412+0000 7f3ce6da3740 1 --2- 172.21.15.80:0/411661248 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55be3ca0ff90 0x55be3ca14a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:54.418 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.412+0000 7f3ce6da3740 1 --2- 172.21.15.80:0/411661248 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55be3ca0f5a0 0x55be3ca0f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:54.418 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.412+0000 7f3ce6da3740 1 -- 172.21.15.80:0/411661248 >> 172.21.15.80:0/411661248 conn(0x55be3c9ce950 msgr2=0x55be3ca0b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:54.418 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.412+0000 7f3ce6da3740 1 -- 172.21.15.80:0/411661248 shutdown_connections 2024-04-24T23:30:54.418 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.412+0000 7f3ce6da3740 1 -- 172.21.15.80:0/411661248 wait complete. 2024-04-24T23:30:54.418 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.412+0000 7f3ce6da3740 1 -- start start 2024-04-24T23:30:54.418 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.412+0000 7f3ce6da3740 1 Processor -- start 2024-04-24T23:30:54.418 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.413+0000 7f3ce6da3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55be3ca0f5a0 0x55be3ca199e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:54.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.413+0000 7f3ce6da3740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55be3ca0ff90 0x55be3ca19f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:54.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.413+0000 7f3ce6da3740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55be3ca15100 0x55be3ca20f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:54.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.413+0000 7f3ce6da3740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x55be3ca180b0 con 0x55be3ca0ff90 2024-04-24T23:30:54.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.413+0000 7f3ce6da3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55be3c925e90 con 0x55be3ca0f5a0 2024-04-24T23:30:54.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.413+0000 7f3ce6da3740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55be3ca18230 con 0x55be3ca15100 2024-04-24T23:30:54.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.413+0000 7f3ce6da3740 10 client.0 _ll_register_callbacks cb 0x55be3c961970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:30:54.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.413+0000 7f3ce1635640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55be3ca0ff90 0x55be3ca19f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:54.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.413+0000 7f3ce1635640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55be3ca0ff90 0x55be3ca19f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:55386/0 (socket says 172.21.15.80:55386) 2024-04-24T23:30:54.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.413+0000 7f3ce1635640 1 -- 172.21.15.80:0/251760698 learned_addr learned my addr 172.21.15.80:0/251760698 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:30:54.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.413+0000 7f3ce1e36640 1 --2- 172.21.15.80:0/251760698 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55be3ca0f5a0 0x55be3ca199e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:54.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.413+0000 7f3ce2637640 1 --2- 172.21.15.80:0/251760698 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55be3ca15100 0x55be3ca20f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:54.420 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.413+0000 7f3ce1635640 1 -- 172.21.15.80:0/251760698 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55be3ca15100 msgr2=0x55be3ca20f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:54.420 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.413+0000 7f3ce1635640 1 --2- 172.21.15.80:0/251760698 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55be3ca15100 0x55be3ca20f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:54.420 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.413+0000 7f3ce1635640 1 -- 172.21.15.80:0/251760698 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55be3ca0f5a0 msgr2=0x55be3ca199e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:54.420 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.413+0000 7f3ce1635640 1 --2- 172.21.15.80:0/251760698 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55be3ca0f5a0 0x55be3ca199e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:54.420 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.413+0000 7f3ce1635640 1 -- 172.21.15.80:0/251760698 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be3c7467e0 con 0x55be3ca0ff90 2024-04-24T23:30:54.420 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.413+0000 7f3ce1e36640 1 --2- 172.21.15.80:0/251760698 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55be3ca0f5a0 0x55be3ca199e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:30:54.420 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.413+0000 7f3ce1635640 1 --2- 172.21.15.80:0/251760698 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55be3ca0ff90 0x55be3ca19f20 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f3ccc0096c0 tx=0x7f3ccc02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a819c7344865eaa6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:54.420 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.413+0000 7f3cc57fa640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-04-24T23:30:54.420 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.414+0000 7f3cc57fa640 1 -- 172.21.15.80:0/251760698 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ccc0d5060 con 0x55be3ca0ff90 2024-04-24T23:30:54.420 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.414+0000 7f3cc57fa640 1 -- 172.21.15.80:0/251760698 <== mon.0 v2:172.21.15.80:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f3ccc0d5200 con 0x55be3ca0ff90 2024-04-24T23:30:54.420 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.414+0000 7f3ce6da3740 1 -- 172.21.15.80:0/251760698 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3cd4012bf0 con 0x55be3ca0ff90 2024-04-24T23:30:54.420 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.414+0000 7f3cc57fa640 1 -- 172.21.15.80:0/251760698 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3ccc049040 con 0x55be3ca0ff90 2024-04-24T23:30:54.421 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.414+0000 7f3ce6da3740 10 client.15843 fetch_fsmap learned FSMap version 25 2024-04-24T23:30:54.421 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.414+0000 7f3ce6da3740 1 -- 172.21.15.80:0/251760698 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55be3c7a6f70 con 0x55be3ca0ff90 2024-04-24T23:30:54.421 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.414+0000 7f3cc57fa640 1 -- 172.21.15.80:0/251760698 <== mon.0 v2:172.21.15.80:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3cd4012bf0 con 0x55be3ca0ff90 2024-04-24T23:30:54.421 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.414+0000 7f3cc57fa640 1 -- 172.21.15.80:0/251760698 <== mon.0 v2:172.21.15.80:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f3ccc03b460 con 0x55be3ca0ff90 2024-04-24T23:30:54.421 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.414+0000 7f3ce6da3740 10 client.15843 fetch_fsmap finished waiting for FSMap version 25 2024-04-24T23:30:54.421 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.414+0000 7f3ce6da3740 10 client.15843 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:30:54.421 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.414+0000 7f3ce6da3740 20 client.15843 populate_metadata read hostname 'smithi080' 2024-04-24T23:30:54.421 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.414+0000 7f3ce6da3740 1 --2- 172.21.15.80:0/251760698 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x55be3ca25440 0x55be3ca278d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:54.421 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.414+0000 7f3ce6da3740 4 client.15843 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:30:54.421 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.414+0000 7f3ce6da3740 1 -- 172.21.15.80:0/251760698 --> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55be3c93e5a0 con 0x55be3ca25440 2024-04-24T23:30:54.421 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.415+0000 7f3ce1e36640 1 --2- 172.21.15.80:0/251760698 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x55be3ca25440 0x55be3ca278d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:54.421 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.415+0000 7f3cc57fa640 10 client.15843 ms_handle_connect on v2:172.21.15.148:6838/2952757594 2024-04-24T23:30:54.421 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.415+0000 7f3ce1e36640 1 --2- 172.21.15.80:0/251760698 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x55be3ca25440 0x55be3ca278d0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d4d5218895daae2 server_cookie=bf9378d3d7783afb in_seq=0 out_seq=0 2024-04-24T23:30:54.421 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.416+0000 7f3cc57fa640 1 -- 172.21.15.80:0/251760698 <== mds.0 v2:172.21.15.148:6838/2952757594 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55be3c93e5a0 con 0x55be3ca25440 2024-04-24T23:30:54.421 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.416+0000 7f3cc57fa640 10 client.15843 handle_command_reply: tid=0 2024-04-24T23:30:54.422 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.416+0000 7f3ce6da3740 1 client.15843 shutdown 2024-04-24T23:30:54.422 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.416+0000 7f3ce6da3740 1 -- 172.21.15.80:0/251760698 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55be3ca0ff90 msgr2=0x55be3ca19f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:54.422 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.416+0000 7f3ce6da3740 1 --2- 172.21.15.80:0/251760698 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55be3ca0ff90 0x55be3ca19f20 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f3ccc0096c0 tx=0x7f3ccc02a410 comp rx=0 tx=0).stop 2024-04-24T23:30:54.422 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.416+0000 7f3ce6da3740 1 -- 172.21.15.80:0/251760698 shutdown_connections 2024-04-24T23:30:54.422 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.416+0000 7f3ce6da3740 1 --2- 172.21.15.80:0/251760698 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x55be3ca25440 0x55be3ca278d0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:54.422 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.416+0000 7f3cc57fa640 0 client.15843 ms_handle_reset on v2:172.21.15.148:6838/2952757594 2024-04-24T23:30:54.422 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.416+0000 7f3ce6da3740 1 --2- 172.21.15.80:0/251760698 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55be3ca15100 0x55be3ca20f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:54.422 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.416+0000 7f3ce6da3740 1 --2- 172.21.15.80:0/251760698 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55be3ca0ff90 0x55be3ca19f20 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:54.422 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.416+0000 7f3ce6da3740 1 --2- 172.21.15.80:0/251760698 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55be3ca0f5a0 0x55be3ca199e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:54.422 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.416+0000 7f3ce6da3740 1 -- 172.21.15.80:0/251760698 >> 172.21.15.80:0/251760698 conn(0x55be3c9ce950 msgr2=0x55be3ca01950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:54.422 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.416+0000 7f3ce6da3740 1 -- 172.21.15.80:0/251760698 shutdown_connections 2024-04-24T23:30:54.422 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.416+0000 7f3ce6da3740 1 -- 172.21.15.80:0/251760698 wait complete. 2024-04-24T23:30:54.422 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.416+0000 7f3ce6da3740 20 client.15843 trim_cache size 0 max 16384 2024-04-24T23:30:54.435 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.434+0000 7f3ce6da3740 1 Processor -- start 2024-04-24T23:30:54.435 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.434+0000 7f3ce6da3740 1 -- start start 2024-04-24T23:30:54.435 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.434+0000 7f3ce6da3740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55be3ca0f5a0 0x55be3ca0e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:54.435 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.434+0000 7f3ce6da3740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55be3ca0ff90 0x55be3ca20f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:54.435 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.434+0000 7f3ce6da3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55be3ca15100 0x55be3ca2c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:54.435 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.434+0000 7f3ce6da3740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f3cdc1aabc0 con 0x55be3ca0f5a0 2024-04-24T23:30:54.435 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.434+0000 7f3ce6da3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3cdc1a8490 con 0x55be3ca15100 2024-04-24T23:30:54.435 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.434+0000 7f3ce6da3740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3cdc1a62e0 con 0x55be3ca0ff90 2024-04-24T23:30:54.435 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.434+0000 7f3ce1e36640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55be3ca0f5a0 0x55be3ca0e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:54.435 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.434+0000 7f3ce1e36640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55be3ca0f5a0 0x55be3ca0e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:55400/0 (socket says 172.21.15.80:55400) 2024-04-24T23:30:54.435 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.434+0000 7f3ce1e36640 1 -- 172.21.15.80:0/1754764335 learned_addr learned my addr 172.21.15.80:0/1754764335 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:30:54.435 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.434+0000 7f3ce2637640 1 --2- 172.21.15.80:0/1754764335 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55be3ca15100 0x55be3ca2c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:54.435 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.434+0000 7f3ce1e36640 1 -- 172.21.15.80:0/1754764335 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55be3ca0ff90 msgr2=0x55be3ca20f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:54.436 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.434+0000 7f3ce1e36640 1 --2- 172.21.15.80:0/1754764335 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55be3ca0ff90 0x55be3ca20f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:54.436 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.434+0000 7f3ce1e36640 1 -- 172.21.15.80:0/1754764335 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55be3ca15100 msgr2=0x55be3ca2c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:54.436 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.434+0000 7f3ce1e36640 1 --2- 172.21.15.80:0/1754764335 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55be3ca15100 0x55be3ca2c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:54.436 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.434+0000 7f3ce1e36640 1 -- 172.21.15.80:0/1754764335 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be3c63e170 con 0x55be3ca0f5a0 2024-04-24T23:30:54.436 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.435+0000 7f3ce1e36640 1 --2- 172.21.15.80:0/1754764335 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55be3ca0f5a0 0x55be3ca0e700 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f3cd800ec30 tx=0x7f3cd8014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a35f9ebe27e1d1b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:54.436 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.435+0000 7f3cc77fe640 1 -- 172.21.15.80:0/1754764335 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cd8030070 con 0x55be3ca0f5a0 2024-04-24T23:30:54.436 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.435+0000 7f3cc77fe640 1 -- 172.21.15.80:0/1754764335 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3cd8005500 con 0x55be3ca0f5a0 2024-04-24T23:30:54.436 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.435+0000 7f3cc77fe640 1 -- 172.21.15.80:0/1754764335 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cd8004c40 con 0x55be3ca0f5a0 2024-04-24T23:30:54.440 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.435+0000 7f3ce6da3740 1 -- 172.21.15.80:0/1754764335 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55be3ca0f5a0 msgr2=0x55be3ca0e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:54.440 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.435+0000 7f3ce6da3740 1 --2- 172.21.15.80:0/1754764335 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55be3ca0f5a0 0x55be3ca0e700 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f3cd800ec30 tx=0x7f3cd8014000 comp rx=0 tx=0).stop 2024-04-24T23:30:54.440 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.436+0000 7f3ce6da3740 1 -- 172.21.15.80:0/1754764335 shutdown_connections 2024-04-24T23:30:54.440 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.436+0000 7f3ce6da3740 1 --2- 172.21.15.80:0/1754764335 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55be3ca15100 0x55be3ca2c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:54.440 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.436+0000 7f3ce6da3740 1 --2- 172.21.15.80:0/1754764335 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55be3ca0ff90 0x55be3ca20f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:54.440 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.436+0000 7f3ce6da3740 1 --2- 172.21.15.80:0/1754764335 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55be3ca0f5a0 0x55be3ca0e700 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:54.440 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.436+0000 7f3ce6da3740 1 -- 172.21.15.80:0/1754764335 >> 172.21.15.80:0/1754764335 conn(0x55be3c9ce950 msgr2=0x55be3ca0dff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:54.440 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.436+0000 7f3ce6da3740 1 -- 172.21.15.80:0/1754764335 shutdown_connections 2024-04-24T23:30:54.440 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.436+0000 7f3ce6da3740 1 -- 172.21.15.80:0/1754764335 wait complete. 2024-04-24T23:30:54.440 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.436+0000 7f3ce6da3740 1 -- start start 2024-04-24T23:30:54.440 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.436+0000 7f3ce6da3740 1 Processor -- start 2024-04-24T23:30:54.440 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.437+0000 7f3ce6da3740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55be3ca0f5a0 0x55be3ca20e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:54.440 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.437+0000 7f3ce6da3740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55be3ca0ff90 0x55be3ca1f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:54.441 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.437+0000 7f3ce6da3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55be3ca15100 0x55be3ca1f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:54.441 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.437+0000 7f3ce6da3740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f3cdc1a6950 con 0x55be3ca0f5a0 2024-04-24T23:30:54.441 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.437+0000 7f3ce6da3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3cdc1a7dc0 con 0x55be3ca15100 2024-04-24T23:30:54.441 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.437+0000 7f3ce6da3740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3cdc1a7020 con 0x55be3ca0ff90 2024-04-24T23:30:54.441 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.437+0000 7f3ce1e36640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55be3ca0f5a0 0x55be3ca20e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:54.441 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.437+0000 7f3ce6da3740 10 client.0 _ll_register_callbacks cb 0x7f3ccc0b7fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:30:54.441 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.437+0000 7f3ce1e36640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55be3ca0f5a0 0x55be3ca20e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:55410/0 (socket says 172.21.15.80:55410) 2024-04-24T23:30:54.441 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.437+0000 7f3ce1e36640 1 -- 172.21.15.80:0/3454465647 learned_addr learned my addr 172.21.15.80:0/3454465647 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:30:54.441 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.437+0000 7f3ce1635640 1 --2- 172.21.15.80:0/3454465647 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55be3ca0ff90 0x55be3ca1f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:54.441 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.437+0000 7f3ce2637640 1 --2- 172.21.15.80:0/3454465647 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55be3ca15100 0x55be3ca1f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:54.441 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.437+0000 7f3ce1e36640 1 -- 172.21.15.80:0/3454465647 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55be3ca0ff90 msgr2=0x55be3ca1f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:54.441 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.437+0000 7f3ce1e36640 1 --2- 172.21.15.80:0/3454465647 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55be3ca0ff90 0x55be3ca1f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:54.441 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.437+0000 7f3ce1e36640 1 -- 172.21.15.80:0/3454465647 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55be3ca15100 msgr2=0x55be3ca1f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:54.441 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.437+0000 7f3ce1e36640 1 --2- 172.21.15.80:0/3454465647 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55be3ca15100 0x55be3ca1f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:54.442 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.437+0000 7f3ce1e36640 1 -- 172.21.15.80:0/3454465647 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be3ca201f0 con 0x55be3ca0f5a0 2024-04-24T23:30:54.442 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.437+0000 7f3ce2637640 1 --2- 172.21.15.80:0/3454465647 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55be3ca15100 0x55be3ca1f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:30:54.442 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.437+0000 7f3ce1e36640 1 --2- 172.21.15.80:0/3454465647 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55be3ca0f5a0 0x55be3ca20e70 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f3cd8009a90 tx=0x7f3cd800ef30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6436e6010f92ae54 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:54.442 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.437+0000 7f3cc57fa640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-04-24T23:30:54.442 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.438+0000 7f3cc57fa640 1 -- 172.21.15.80:0/3454465647 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cd8030050 con 0x55be3ca0f5a0 2024-04-24T23:30:54.442 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.438+0000 7f3cc57fa640 1 -- 172.21.15.80:0/3454465647 <== mon.0 v2:172.21.15.80:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f3cd802b0f0 con 0x55be3ca0f5a0 2024-04-24T23:30:54.442 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.438+0000 7f3ce6da3740 1 -- 172.21.15.80:0/3454465647 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3cd8005500 con 0x55be3ca0f5a0 2024-04-24T23:30:54.442 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.438+0000 7f3cc57fa640 1 -- 172.21.15.80:0/3454465647 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3cd8042050 con 0x55be3ca0f5a0 2024-04-24T23:30:54.442 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.438+0000 7f3ce6da3740 10 client.15849 fetch_fsmap learned FSMap version 25 2024-04-24T23:30:54.442 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.438+0000 7f3ce6da3740 1 -- 172.21.15.80:0/3454465647 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55be3ca8cac0 con 0x55be3ca0f5a0 2024-04-24T23:30:54.442 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.438+0000 7f3cc57fa640 1 -- 172.21.15.80:0/3454465647 <== mon.0 v2:172.21.15.80:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3cd8005500 con 0x55be3ca0f5a0 2024-04-24T23:30:54.442 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.438+0000 7f3cc57fa640 1 -- 172.21.15.80:0/3454465647 <== mon.0 v2:172.21.15.80:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f3cd8004e50 con 0x55be3ca0f5a0 2024-04-24T23:30:54.442 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.438+0000 7f3ce6da3740 10 client.15849 fetch_fsmap finished waiting for FSMap version 25 2024-04-24T23:30:54.442 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.438+0000 7f3ce6da3740 10 client.15849 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:30:54.442 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.438+0000 7f3ce6da3740 20 client.15849 populate_metadata read hostname 'smithi080' 2024-04-24T23:30:54.443 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.438+0000 7f3ce6da3740 1 --2- 172.21.15.80:0/3454465647 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x55be3ca25440 0x55be3ca8f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:54.443 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.438+0000 7f3ce6da3740 4 client.15849 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T23:30:54.443 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.438+0000 7f3ce6da3740 1 -- 172.21.15.80:0/3454465647 --> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55be3c977800 con 0x55be3ca25440 2024-04-24T23:30:54.443 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.439+0000 7f3ce1635640 1 --2- 172.21.15.80:0/3454465647 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x55be3ca25440 0x55be3ca8f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:54.443 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.439+0000 7f3ce1635640 1 --2- 172.21.15.80:0/3454465647 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x55be3ca25440 0x55be3ca8f480 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8cabea30a2b3e72 server_cookie=ac5c113f6f7338d8 in_seq=0 out_seq=0 2024-04-24T23:30:54.443 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.439+0000 7f3cc57fa640 10 client.15849 ms_handle_connect on v2:172.21.15.148:6838/2952757594 2024-04-24T23:30:54.444 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.441+0000 7f3cc57fa640 1 -- 172.21.15.80:0/3454465647 <== mds.0 v2:172.21.15.148:6838/2952757594 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55be3c977800 con 0x55be3ca25440 2024-04-24T23:30:54.444 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.441+0000 7f3cc57fa640 10 client.15849 handle_command_reply: tid=0 2024-04-24T23:30:54.444 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.441+0000 7f3ce6da3740 1 client.15849 shutdown 2024-04-24T23:30:54.444 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.441+0000 7f3ce6da3740 1 -- 172.21.15.80:0/3454465647 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55be3ca0f5a0 msgr2=0x55be3ca20e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:54.444 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.441+0000 7f3ce6da3740 1 --2- 172.21.15.80:0/3454465647 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55be3ca0f5a0 0x55be3ca20e70 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f3cd8009a90 tx=0x7f3cd800ef30 comp rx=0 tx=0).stop 2024-04-24T23:30:54.444 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.441+0000 7f3ce6da3740 1 -- 172.21.15.80:0/3454465647 shutdown_connections 2024-04-24T23:30:54.444 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.441+0000 7f3ce6da3740 1 --2- 172.21.15.80:0/3454465647 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x55be3ca25440 0x55be3ca8f480 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:54.445 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.441+0000 7f3ce6da3740 1 --2- 172.21.15.80:0/3454465647 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55be3ca15100 0x55be3ca1f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:54.445 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.441+0000 7f3ce6da3740 1 --2- 172.21.15.80:0/3454465647 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55be3ca0ff90 0x55be3ca1f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:54.445 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.441+0000 7f3ce6da3740 1 --2- 172.21.15.80:0/3454465647 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55be3ca0f5a0 0x55be3ca20e70 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:54.445 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.441+0000 7f3ce6da3740 1 -- 172.21.15.80:0/3454465647 >> 172.21.15.80:0/3454465647 conn(0x55be3c9ce950 msgr2=0x55be3ca01950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:54.445 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.441+0000 7f3cc57fa640 0 client.15849 ms_handle_reset on v2:172.21.15.148:6838/2952757594 2024-04-24T23:30:54.445 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.441+0000 7f3ce6da3740 1 -- 172.21.15.80:0/3454465647 shutdown_connections 2024-04-24T23:30:54.445 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.441+0000 7f3ce6da3740 1 -- 172.21.15.80:0/3454465647 wait complete. 2024-04-24T23:30:54.445 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.441+0000 7f3ce6da3740 20 client.15849 trim_cache size 0 max 16384 2024-04-24T23:30:54.445 INFO:teuthology.orchestra.run.smithi080.stdout:{ 2024-04-24T23:30:54.445 INFO:teuthology.orchestra.run.smithi080.stdout: "return_code": 0, 2024-04-24T23:30:54.445 INFO:teuthology.orchestra.run.smithi080.stdout: "scrub_tag": "1da3681e-5460-48bb-b660-0edbcbd23b8a", 2024-04-24T23:30:54.446 INFO:teuthology.orchestra.run.smithi080.stdout: "mode": "asynchronous" 2024-04-24T23:30:54.446 INFO:teuthology.orchestra.run.smithi080.stdout:} 2024-04-24T23:30:54.446 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.442+0000 7f3ce409a640 1 -- 172.21.15.80:0/66081561 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f3cc80a2b90 msgr2=0x7f3cc80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:54.446 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.442+0000 7f3ce409a640 1 --2- 172.21.15.80:0/66081561 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f3cc80a2b90 0x7f3cc80a5050 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f3cd8009bd0 tx=0x7f3cd8005f90 comp rx=0 tx=0).stop 2024-04-24T23:30:54.446 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.442+0000 7f3ce409a640 1 -- 172.21.15.80:0/66081561 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3cdc104b30 msgr2=0x7f3cdc198070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:54.447 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.442+0000 7f3ce409a640 1 --2- 172.21.15.80:0/66081561 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3cdc104b30 0x7f3cdc198070 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f3ccc031610 tx=0x7f3ccc004270 comp rx=0 tx=0).stop 2024-04-24T23:30:54.447 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.442+0000 7f3ce409a640 1 -- 172.21.15.80:0/66081561 shutdown_connections 2024-04-24T23:30:54.447 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.442+0000 7f3ce409a640 1 --2- 172.21.15.80:0/66081561 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f3cc80a2b90 0x7f3cc80a5050 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:54.447 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.442+0000 7f3ce409a640 1 --2- 172.21.15.80:0/66081561 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3cdc107460 0x7f3cdc19f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:54.447 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.442+0000 7f3ce409a640 1 --2- 172.21.15.80:0/66081561 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3cdc104b30 0x7f3cdc198070 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:54.447 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.442+0000 7f3ce409a640 1 --2- 172.21.15.80:0/66081561 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3cdc102200 0x7f3cdc197b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:54.447 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.442+0000 7f3ce409a640 1 -- 172.21.15.80:0/66081561 >> 172.21.15.80:0/66081561 conn(0x7f3cdc0fbbf0 msgr2=0x7f3cdc102d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:54.447 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.442+0000 7f3ce409a640 1 -- 172.21.15.80:0/66081561 shutdown_connections 2024-04-24T23:30:54.447 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.442+0000 7f3ce409a640 1 -- 172.21.15.80:0/66081561 wait complete. 2024-04-24T23:30:54.454 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:30:54 smithi080 ceph-mon[29152]: pgmap v407: 129 pgs: 129 active+clean; 656 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:30:54.455 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T23:30:54.455 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T23:30:54.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:30:54 smithi148 ceph-mon[32528]: pgmap v407: 129 pgs: 129 active+clean; 656 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:30:54.559 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.558+0000 7f64eaba9640 1 -- 172.21.15.80:0/2002586565 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f64e4102980 msgr2=0x7f64e4102e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:54.559 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.558+0000 7f64eaba9640 1 --2- 172.21.15.80:0/2002586565 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f64e4102980 0x7f64e4102e00 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f64d0009900 tx=0x7f64d002a410 comp rx=0 tx=0).stop 2024-04-24T23:30:54.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.558+0000 7f64eaba9640 1 -- 172.21.15.80:0/2002586565 shutdown_connections 2024-04-24T23:30:54.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.558+0000 7f64eaba9640 1 --2- 172.21.15.80:0/2002586565 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64e4103340 0x7f64e4109bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:54.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.558+0000 7f64eaba9640 1 --2- 172.21.15.80:0/2002586565 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f64e4102980 0x7f64e4102e00 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:54.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.558+0000 7f64eaba9640 1 --2- 172.21.15.80:0/2002586565 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f64e4101790 0x7f64e4101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:54.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.558+0000 7f64eaba9640 1 -- 172.21.15.80:0/2002586565 >> 172.21.15.80:0/2002586565 conn(0x7f64e40fcd20 msgr2=0x7f64e40ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:54.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.558+0000 7f64eaba9640 1 -- 172.21.15.80:0/2002586565 shutdown_connections 2024-04-24T23:30:54.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.558+0000 7f64eaba9640 1 -- 172.21.15.80:0/2002586565 wait complete. 2024-04-24T23:30:54.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.558+0000 7f64eaba9640 1 Processor -- start 2024-04-24T23:30:54.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.559+0000 7f64eaba9640 1 -- start start 2024-04-24T23:30:54.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.559+0000 7f64eaba9640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f64e4101790 0x7f64e40754f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:54.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.559+0000 7f64eaba9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f64e4102980 0x7f64e4073b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:54.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.559+0000 7f64eaba9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64e4103340 0x7f64e4074070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:54.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.559+0000 7f64eaba9640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f64e410d410 con 0x7f64e4101790 2024-04-24T23:30:54.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.559+0000 7f64eaba9640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f64e410d290 con 0x7f64e4102980 2024-04-24T23:30:54.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.559+0000 7f64eaba9640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f64e410d590 con 0x7f64e4103340 2024-04-24T23:30:54.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.559+0000 7f64e8945640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f64e4101790 0x7f64e40754f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:54.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.559+0000 7f64e8945640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f64e4101790 0x7f64e40754f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:55438/0 (socket says 172.21.15.80:55438) 2024-04-24T23:30:54.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.559+0000 7f64e8945640 1 -- 172.21.15.80:0/677474691 learned_addr learned my addr 172.21.15.80:0/677474691 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:30:54.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.559+0000 7f64e3fff640 1 --2- 172.21.15.80:0/677474691 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f64e4102980 0x7f64e4073b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:54.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.559+0000 7f64e9146640 1 --2- 172.21.15.80:0/677474691 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64e4103340 0x7f64e4074070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:54.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.559+0000 7f64e8945640 1 -- 172.21.15.80:0/677474691 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64e4103340 msgr2=0x7f64e4074070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:54.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.559+0000 7f64e8945640 1 --2- 172.21.15.80:0/677474691 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64e4103340 0x7f64e4074070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:54.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.559+0000 7f64e8945640 1 -- 172.21.15.80:0/677474691 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f64e4102980 msgr2=0x7f64e4073b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:54.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.559+0000 7f64e8945640 1 --2- 172.21.15.80:0/677474691 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f64e4102980 0x7f64e4073b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:54.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.559+0000 7f64e8945640 1 -- 172.21.15.80:0/677474691 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64e4074900 con 0x7f64e4101790 2024-04-24T23:30:54.562 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.559+0000 7f64e8945640 1 --2- 172.21.15.80:0/677474691 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f64e4101790 0x7f64e40754f0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f64d800d770 tx=0x7f64d800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:54.562 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.559+0000 7f64e1ffb640 1 -- 172.21.15.80:0/677474691 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64d8017070 con 0x7f64e4101790 2024-04-24T23:30:54.562 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.560+0000 7f64eaba9640 1 -- 172.21.15.80:0/677474691 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64e41a1ba0 con 0x7f64e4101790 2024-04-24T23:30:54.562 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.560+0000 7f64e1ffb640 1 -- 172.21.15.80:0/677474691 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64d8009c30 con 0x7f64e4101790 2024-04-24T23:30:54.562 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.560+0000 7f64e1ffb640 1 -- 172.21.15.80:0/677474691 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64d8020060 con 0x7f64e4101790 2024-04-24T23:30:54.562 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.560+0000 7f64eaba9640 1 -- 172.21.15.80:0/677474691 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64e41a2110 con 0x7f64e4101790 2024-04-24T23:30:54.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.562+0000 7f64e1ffb640 1 -- 172.21.15.80:0/677474691 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f64d8004020 con 0x7f64e4101790 2024-04-24T23:30:54.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.563+0000 7f64e1ffb640 1 --2- 172.21.15.80:0/677474691 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f64d40a2c10 0x7f64d40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:54.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.563+0000 7f64e3fff640 1 --2- 172.21.15.80:0/677474691 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f64d40a2c10 0x7f64d40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:54.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.563+0000 7f64e1ffb640 1 -- 172.21.15.80:0/677474691 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f64d80ced70 con 0x7f64e4101790 2024-04-24T23:30:54.565 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.563+0000 7f64e3fff640 1 --2- 172.21.15.80:0/677474691 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f64d40a2c10 0x7f64d40a50d0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f64d00314e0 tx=0x7f64d0000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:54.565 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.563+0000 7f64ed999740 1 Processor -- start 2024-04-24T23:30:54.565 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.563+0000 7f64ed999740 1 -- start start 2024-04-24T23:30:54.565 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.564+0000 7f64ed999740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b529a47240 0x55b529a47660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:54.565 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.564+0000 7f64ed999740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b529a47c30 0x55b529a4c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:54.565 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.564+0000 7f64ed999740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b529a4cda0 0x55b529a4f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:54.565 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.564+0000 7f64ed999740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x55b5299ab6b0 con 0x55b529a47c30 2024-04-24T23:30:54.565 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.564+0000 7f64ed999740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55b529953880 con 0x55b529a47240 2024-04-24T23:30:54.565 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.564+0000 7f64ed999740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55b5299a7670 con 0x55b529a4cda0 2024-04-24T23:30:54.565 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.564+0000 7f64e3fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b529a47c30 0x55b529a4c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:54.565 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.564+0000 7f64e3fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b529a47c30 0x55b529a4c730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:55446/0 (socket says 172.21.15.80:55446) 2024-04-24T23:30:54.565 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.564+0000 7f64e3fff640 1 -- 172.21.15.80:0/3820295880 learned_addr learned my addr 172.21.15.80:0/3820295880 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:30:54.565 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.564+0000 7f64e8945640 1 --2- 172.21.15.80:0/3820295880 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b529a47240 0x55b529a47660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:54.565 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.564+0000 7f64e9146640 1 --2- 172.21.15.80:0/3820295880 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b529a4cda0 0x55b529a4f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:54.566 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.564+0000 7f64e3fff640 1 -- 172.21.15.80:0/3820295880 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b529a4cda0 msgr2=0x55b529a4f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:54.566 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.564+0000 7f64e3fff640 1 --2- 172.21.15.80:0/3820295880 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b529a4cda0 0x55b529a4f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:54.566 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.564+0000 7f64e3fff640 1 -- 172.21.15.80:0/3820295880 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b529a47240 msgr2=0x55b529a47660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:54.566 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.564+0000 7f64e3fff640 1 --2- 172.21.15.80:0/3820295880 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b529a47240 0x55b529a47660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:54.566 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.564+0000 7f64e3fff640 1 -- 172.21.15.80:0/3820295880 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5297897d0 con 0x55b529a47c30 2024-04-24T23:30:54.566 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.564+0000 7f64e8945640 1 --2- 172.21.15.80:0/3820295880 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b529a47240 0x55b529a47660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:30:54.566 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.564+0000 7f64e9146640 1 --2- 172.21.15.80:0/3820295880 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b529a4cda0 0x55b529a4f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:30:54.566 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.564+0000 7f64e3fff640 1 --2- 172.21.15.80:0/3820295880 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b529a47c30 0x55b529a4c730 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f64d0033000 tx=0x7f64d003d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=640cf1420b3431f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:54.566 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.565+0000 7f64ceffd640 1 -- 172.21.15.80:0/3820295880 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64d0035040 con 0x55b529a47c30 2024-04-24T23:30:54.566 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.565+0000 7f64ceffd640 1 -- 172.21.15.80:0/3820295880 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64d004d070 con 0x55b529a47c30 2024-04-24T23:30:54.566 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.565+0000 7f64ceffd640 1 -- 172.21.15.80:0/3820295880 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64d00049e0 con 0x55b529a47c30 2024-04-24T23:30:54.571 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.566+0000 7f64ed999740 1 -- 172.21.15.80:0/3820295880 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b529a47c30 msgr2=0x55b529a4c730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:54.571 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.566+0000 7f64ed999740 1 --2- 172.21.15.80:0/3820295880 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b529a47c30 0x55b529a4c730 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f64d0033000 tx=0x7f64d003d000 comp rx=0 tx=0).stop 2024-04-24T23:30:54.571 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.566+0000 7f64ed999740 1 -- 172.21.15.80:0/3820295880 shutdown_connections 2024-04-24T23:30:54.571 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.566+0000 7f64ed999740 1 --2- 172.21.15.80:0/3820295880 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b529a4cda0 0x55b529a4f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:54.571 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.566+0000 7f64ed999740 1 --2- 172.21.15.80:0/3820295880 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b529a47c30 0x55b529a4c730 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:54.571 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.566+0000 7f64ed999740 1 --2- 172.21.15.80:0/3820295880 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b529a47240 0x55b529a47660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:54.571 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.566+0000 7f64ed999740 1 -- 172.21.15.80:0/3820295880 >> 172.21.15.80:0/3820295880 conn(0x55b529a428b0 msgr2=0x55b529a42cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:54.571 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.566+0000 7f64ed999740 1 -- 172.21.15.80:0/3820295880 shutdown_connections 2024-04-24T23:30:54.571 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.566+0000 7f64ed999740 1 -- 172.21.15.80:0/3820295880 wait complete. 2024-04-24T23:30:54.571 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.566+0000 7f64ed999740 1 -- start start 2024-04-24T23:30:54.571 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.566+0000 7f64ed999740 1 Processor -- start 2024-04-24T23:30:54.571 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.567+0000 7f64ed999740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b529a47240 0x55b529a515d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:54.571 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.567+0000 7f64ed999740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b529a47c30 0x55b529a51b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:54.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.567+0000 7f64ed999740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b529a4cda0 0x55b529a58b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:54.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.567+0000 7f64e9146640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b529a4cda0 0x55b529a58b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:54.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.567+0000 7f64ed999740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x55b529a4fcf0 con 0x55b529a4cda0 2024-04-24T23:30:54.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.567+0000 7f64e9146640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b529a4cda0 0x55b529a58b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:55454/0 (socket says 172.21.15.80:55454) 2024-04-24T23:30:54.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.567+0000 7f64e9146640 1 -- 172.21.15.80:0/2629061860 learned_addr learned my addr 172.21.15.80:0/2629061860 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:30:54.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.567+0000 7f64ed999740 1 -- 172.21.15.80:0/2629061860 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55b529966f50 con 0x55b529a47c30 2024-04-24T23:30:54.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.567+0000 7f64ed999740 1 -- 172.21.15.80:0/2629061860 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55b529a4fe70 con 0x55b529a47240 2024-04-24T23:30:54.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.567+0000 7f64e8945640 1 --2- 172.21.15.80:0/2629061860 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b529a47240 0x55b529a515d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:54.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.567+0000 7f64e3fff640 1 --2- 172.21.15.80:0/2629061860 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b529a47c30 0x55b529a51b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:54.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.567+0000 7f64ed999740 10 client.0 _ll_register_callbacks cb 0x55b5299a2000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:30:54.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.567+0000 7f64e9146640 1 -- 172.21.15.80:0/2629061860 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b529a47240 msgr2=0x55b529a515d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:54.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.567+0000 7f64e9146640 1 --2- 172.21.15.80:0/2629061860 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b529a47240 0x55b529a515d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:54.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.567+0000 7f64e9146640 1 -- 172.21.15.80:0/2629061860 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b529a47c30 msgr2=0x55b529a51b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:54.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.567+0000 7f64e9146640 1 --2- 172.21.15.80:0/2629061860 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b529a47c30 0x55b529a51b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:54.573 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.567+0000 7f64e9146640 1 -- 172.21.15.80:0/2629061860 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5296bf3f0 con 0x55b529a4cda0 2024-04-24T23:30:54.573 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.568+0000 7f64e9146640 1 --2- 172.21.15.80:0/2629061860 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b529a4cda0 0x55b529a58b50 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f64dc012a10 tx=0x7f64dc012ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc06505954f4788d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:54.573 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.568+0000 7f64c3fff640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-04-24T23:30:54.573 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.568+0000 7f64ed999740 1 -- 172.21.15.80:0/2629061860 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64d004d070 con 0x55b529a4cda0 2024-04-24T23:30:54.573 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.568+0000 7f64c3fff640 1 -- 172.21.15.80:0/2629061860 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64dc010e20 con 0x55b529a4cda0 2024-04-24T23:30:54.573 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.568+0000 7f64c3fff640 1 -- 172.21.15.80:0/2629061860 <== mon.0 v2:172.21.15.80:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f64dc004b20 con 0x55b529a4cda0 2024-04-24T23:30:54.573 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.568+0000 7f64c3fff640 1 -- 172.21.15.80:0/2629061860 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f64dc025050 con 0x55b529a4cda0 2024-04-24T23:30:54.573 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.568+0000 7f64ed999740 10 client.15861 fetch_fsmap learned FSMap version 25 2024-04-24T23:30:54.573 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.568+0000 7f64ed999740 1 -- 172.21.15.80:0/2629061860 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({fsmap=25}) -- 0x55b52967f0f0 con 0x55b529a4cda0 2024-04-24T23:30:54.573 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.568+0000 7f64c3fff640 1 -- 172.21.15.80:0/2629061860 <== mon.0 v2:172.21.15.80:3300/0 4 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f64dc003970 con 0x55b529a4cda0 2024-04-24T23:30:54.573 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.569+0000 7f64ed999740 10 client.15861 fetch_fsmap finished waiting for FSMap version 25 2024-04-24T23:30:54.573 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.569+0000 7f64ed999740 10 client.15861 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:30:54.573 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.569+0000 7f64ed999740 20 client.15861 populate_metadata read hostname 'smithi080' 2024-04-24T23:30:54.573 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.569+0000 7f64ed999740 1 --2- 172.21.15.80:0/2629061860 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x55b529a5d2c0 0x55b529a5f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:54.574 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.569+0000 7f64ed999740 4 client.15861 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:30:54.574 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.569+0000 7f64ed999740 1 -- 172.21.15.80:0/2629061860 --> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f64d004d070 con 0x55b529a5d2c0 2024-04-24T23:30:54.574 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.569+0000 7f64e8945640 1 --2- 172.21.15.80:0/2629061860 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x55b529a5d2c0 0x55b529a5f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:54.574 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.569+0000 7f64c3fff640 10 client.15861 ms_handle_connect on v2:172.21.15.148:6838/2952757594 2024-04-24T23:30:54.574 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.569+0000 7f64e8945640 1 --2- 172.21.15.80:0/2629061860 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x55b529a5d2c0 0x55b529a5f760 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=733c10368baa51b3 server_cookie=9f20dd86d0de8ccb in_seq=0 out_seq=0 2024-04-24T23:30:54.574 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.570+0000 7f64c3fff640 1 -- 172.21.15.80:0/2629061860 <== mds.0 v2:172.21.15.148:6838/2952757594 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f64d004d070 con 0x55b529a5d2c0 2024-04-24T23:30:54.574 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.570+0000 7f64c3fff640 10 client.15861 handle_command_reply: tid=0 2024-04-24T23:30:54.574 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.570+0000 7f64ed999740 1 client.15861 shutdown 2024-04-24T23:30:54.574 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.570+0000 7f64ed999740 1 -- 172.21.15.80:0/2629061860 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b529a4cda0 msgr2=0x55b529a58b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:54.574 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.570+0000 7f64ed999740 1 --2- 172.21.15.80:0/2629061860 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b529a4cda0 0x55b529a58b50 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f64dc012a10 tx=0x7f64dc012ed0 comp rx=0 tx=0).stop 2024-04-24T23:30:54.574 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.570+0000 7f64ed999740 1 -- 172.21.15.80:0/2629061860 shutdown_connections 2024-04-24T23:30:54.574 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.570+0000 7f64ed999740 1 --2- 172.21.15.80:0/2629061860 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x55b529a5d2c0 0x55b529a5f760 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:54.574 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.570+0000 7f64c3fff640 0 client.15861 ms_handle_reset on v2:172.21.15.148:6838/2952757594 2024-04-24T23:30:54.574 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.570+0000 7f64ed999740 1 --2- 172.21.15.80:0/2629061860 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b529a4cda0 0x55b529a58b50 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:54.574 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.570+0000 7f64ed999740 1 --2- 172.21.15.80:0/2629061860 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b529a47c30 0x55b529a51b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:54.575 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.570+0000 7f64ed999740 1 --2- 172.21.15.80:0/2629061860 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b529a47240 0x55b529a515d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:54.575 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.570+0000 7f64ed999740 1 -- 172.21.15.80:0/2629061860 >> 172.21.15.80:0/2629061860 conn(0x55b529a428b0 msgr2=0x55b529a41af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:54.575 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.571+0000 7f64ed999740 1 -- 172.21.15.80:0/2629061860 shutdown_connections 2024-04-24T23:30:54.575 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.571+0000 7f64ed999740 1 -- 172.21.15.80:0/2629061860 wait complete. 2024-04-24T23:30:54.575 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.571+0000 7f64ed999740 20 client.15861 trim_cache size 0 max 16384 2024-04-24T23:30:54.589 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.588+0000 7f64ed999740 1 Processor -- start 2024-04-24T23:30:54.589 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.588+0000 7f64ed999740 1 -- start start 2024-04-24T23:30:54.589 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.588+0000 7f64ed999740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b529a47240 0x55b529a42c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:54.589 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.588+0000 7f64ed999740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b529a47c30 0x55b529a58b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:54.589 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.588+0000 7f64ed999740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b529a4cda0 0x55b529a45830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:54.589 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.588+0000 7f64ed999740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f64e41aeea0 con 0x55b529a47c30 2024-04-24T23:30:54.589 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.588+0000 7f64ed999740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f64e41ac770 con 0x55b529a4cda0 2024-04-24T23:30:54.590 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.588+0000 7f64ed999740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f64e41aa5c0 con 0x55b529a47240 2024-04-24T23:30:54.590 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.588+0000 7f64e3fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b529a47c30 0x55b529a58b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:54.590 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.588+0000 7f64e3fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b529a47c30 0x55b529a58b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:55458/0 (socket says 172.21.15.80:55458) 2024-04-24T23:30:54.590 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.588+0000 7f64e3fff640 1 -- 172.21.15.80:0/3139392405 learned_addr learned my addr 172.21.15.80:0/3139392405 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:30:54.590 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.589+0000 7f64e9146640 1 --2- 172.21.15.80:0/3139392405 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b529a4cda0 0x55b529a45830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:54.590 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.589+0000 7f64e8945640 1 --2- 172.21.15.80:0/3139392405 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b529a47240 0x55b529a42c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:54.590 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.589+0000 7f64e3fff640 1 -- 172.21.15.80:0/3139392405 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b529a47240 msgr2=0x55b529a42c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:54.590 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.589+0000 7f64e3fff640 1 --2- 172.21.15.80:0/3139392405 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b529a47240 0x55b529a42c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:54.590 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.589+0000 7f64e3fff640 1 -- 172.21.15.80:0/3139392405 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b529a4cda0 msgr2=0x55b529a45830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:54.590 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.589+0000 7f64e3fff640 1 --2- 172.21.15.80:0/3139392405 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b529a4cda0 0x55b529a45830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:54.590 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.589+0000 7f64e3fff640 1 -- 172.21.15.80:0/3139392405 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b529a496d0 con 0x55b529a47c30 2024-04-24T23:30:54.590 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.589+0000 7f64e8945640 1 --2- 172.21.15.80:0/3139392405 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b529a47240 0x55b529a42c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:30:54.590 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.589+0000 7f64e3fff640 1 --2- 172.21.15.80:0/3139392405 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b529a47c30 0x55b529a58b10 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f64d0033000 tx=0x7f64d0032f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5aab00083673bf8f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:54.590 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.589+0000 7f64ce7fc640 1 -- 172.21.15.80:0/3139392405 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64d0035040 con 0x55b529a47c30 2024-04-24T23:30:54.591 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.589+0000 7f64ce7fc640 1 -- 172.21.15.80:0/3139392405 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64d005b070 con 0x55b529a47c30 2024-04-24T23:30:54.591 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.589+0000 7f64ce7fc640 1 -- 172.21.15.80:0/3139392405 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64d0003d50 con 0x55b529a47c30 2024-04-24T23:30:54.596 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.590+0000 7f64ed999740 1 -- 172.21.15.80:0/3139392405 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b529a47c30 msgr2=0x55b529a58b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:54.596 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.590+0000 7f64ed999740 1 --2- 172.21.15.80:0/3139392405 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b529a47c30 0x55b529a58b10 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f64d0033000 tx=0x7f64d0032f80 comp rx=0 tx=0).stop 2024-04-24T23:30:54.596 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.590+0000 7f64ed999740 1 -- 172.21.15.80:0/3139392405 shutdown_connections 2024-04-24T23:30:54.596 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.590+0000 7f64ed999740 1 --2- 172.21.15.80:0/3139392405 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b529a4cda0 0x55b529a45830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:54.596 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.590+0000 7f64ed999740 1 --2- 172.21.15.80:0/3139392405 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b529a47c30 0x55b529a58b10 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:54.596 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.590+0000 7f64ed999740 1 --2- 172.21.15.80:0/3139392405 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b529a47240 0x55b529a42c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:54.596 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.590+0000 7f64ed999740 1 -- 172.21.15.80:0/3139392405 >> 172.21.15.80:0/3139392405 conn(0x55b529a428b0 msgr2=0x55b529a515b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:54.596 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.590+0000 7f64ed999740 1 -- 172.21.15.80:0/3139392405 shutdown_connections 2024-04-24T23:30:54.596 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.590+0000 7f64ed999740 1 -- 172.21.15.80:0/3139392405 wait complete. 2024-04-24T23:30:54.596 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.590+0000 7f64ed999740 1 -- start start 2024-04-24T23:30:54.596 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.591+0000 7f64ed999740 1 Processor -- start 2024-04-24T23:30:54.597 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.591+0000 7f64ed999740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b529a47240 0x55b529a56810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:54.597 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.591+0000 7f64ed999740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b529a4cda0 0x55b529a56d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:54.597 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.591+0000 7f64e8945640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b529a47240 0x55b529a56810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:54.597 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.591+0000 7f64e8945640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b529a47240 0x55b529a56810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:55468/0 (socket says 172.21.15.80:55468) 2024-04-24T23:30:54.597 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.591+0000 7f64ed999740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b529a57d60 0x55b529a581e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:54.597 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.591+0000 7f64e8945640 1 -- 172.21.15.80:0/2979478928 learned_addr learned my addr 172.21.15.80:0/2979478928 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:30:54.597 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.591+0000 7f64ed999740 1 -- 172.21.15.80:0/2979478928 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f64e41aac30 con 0x55b529a47240 2024-04-24T23:30:54.597 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.591+0000 7f64ed999740 1 -- 172.21.15.80:0/2979478928 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f64e41ac0a0 con 0x55b529a57d60 2024-04-24T23:30:54.597 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.591+0000 7f64ed999740 1 -- 172.21.15.80:0/2979478928 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f64e41ab300 con 0x55b529a4cda0 2024-04-24T23:30:54.597 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.591+0000 7f64e9146640 1 --2- 172.21.15.80:0/2979478928 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b529a57d60 0x55b529a581e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:54.597 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.591+0000 7f64e3fff640 1 --2- 172.21.15.80:0/2979478928 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b529a4cda0 0x55b529a56d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:54.597 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.592+0000 7f64ed999740 10 client.0 _ll_register_callbacks cb 0x7f64dc009aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:30:54.597 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.592+0000 7f64e3fff640 1 -- 172.21.15.80:0/2979478928 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b529a57d60 msgr2=0x55b529a581e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:54.597 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.592+0000 7f64e3fff640 1 --2- 172.21.15.80:0/2979478928 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b529a57d60 0x55b529a581e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:54.597 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.592+0000 7f64e3fff640 1 -- 172.21.15.80:0/2979478928 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b529a47240 msgr2=0x55b529a56810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:30:54.598 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.592+0000 7f64e3fff640 1 --2- 172.21.15.80:0/2979478928 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b529a47240 0x55b529a56810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:54.598 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.592+0000 7f64e3fff640 1 -- 172.21.15.80:0/2979478928 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b529a58720 con 0x55b529a4cda0 2024-04-24T23:30:54.598 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.592+0000 7f64e3fff640 1 --2- 172.21.15.80:0/2979478928 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b529a4cda0 0x55b529a56d50 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f64d004d000 tx=0x7f64d0003890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2dba3c3204a3aea8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:30:54.598 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.592+0000 7f64c3fff640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-24T23:30:54.598 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.592+0000 7f64c3fff640 1 -- 172.21.15.80:0/2979478928 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64d0040070 con 0x55b529a4cda0 2024-04-24T23:30:54.598 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.592+0000 7f64ed999740 1 -- 172.21.15.80:0/2979478928 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64d005b070 con 0x55b529a4cda0 2024-04-24T23:30:54.598 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.592+0000 7f64c3fff640 1 -- 172.21.15.80:0/2979478928 <== mon.1 v2:172.21.15.148:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f64d0004980 con 0x55b529a4cda0 2024-04-24T23:30:54.598 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.592+0000 7f64c3fff640 1 -- 172.21.15.80:0/2979478928 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f64d0035040 con 0x55b529a4cda0 2024-04-24T23:30:54.598 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.592+0000 7f64ed999740 10 client.25339 fetch_fsmap learned FSMap version 25 2024-04-24T23:30:54.598 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.592+0000 7f64ed999740 1 -- 172.21.15.80:0/2979478928 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b529ac4170 con 0x55b529a4cda0 2024-04-24T23:30:54.598 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.593+0000 7f64c3fff640 1 -- 172.21.15.80:0/2979478928 <== mon.1 v2:172.21.15.148:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64d005b070 con 0x55b529a4cda0 2024-04-24T23:30:54.598 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.593+0000 7f64c3fff640 1 -- 172.21.15.80:0/2979478928 <== mon.1 v2:172.21.15.148:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f64d00489c0 con 0x55b529a4cda0 2024-04-24T23:30:54.598 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.593+0000 7f64ed999740 10 client.25339 fetch_fsmap finished waiting for FSMap version 25 2024-04-24T23:30:54.598 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.593+0000 7f64ed999740 10 client.25339 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:30:54.598 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.593+0000 7f64ed999740 20 client.25339 populate_metadata read hostname 'smithi080' 2024-04-24T23:30:54.599 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.593+0000 7f64ed999740 1 --2- 172.21.15.80:0/2979478928 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x55b529ac4690 0x55b529ac6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:30:54.599 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.593+0000 7f64ed999740 4 client.25339 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:30:54.599 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.594+0000 7f64ed999740 1 -- 172.21.15.80:0/2979478928 --> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b5299b8bf0 con 0x55b529ac4690 2024-04-24T23:30:54.599 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.594+0000 7f64e8945640 1 --2- 172.21.15.80:0/2979478928 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x55b529ac4690 0x55b529ac6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:30:54.599 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.594+0000 7f64e8945640 1 --2- 172.21.15.80:0/2979478928 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x55b529ac4690 0x55b529ac6ab0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e34e85d4e3322838 server_cookie=fdf356666d9143c1 in_seq=0 out_seq=0 2024-04-24T23:30:54.599 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.594+0000 7f64c3fff640 10 client.25339 ms_handle_connect on v2:172.21.15.148:6838/2952757594 2024-04-24T23:30:54.599 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.594+0000 7f64c3fff640 1 -- 172.21.15.80:0/2979478928 <== mds.0 v2:172.21.15.148:6838/2952757594 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55b5299b8bf0 con 0x55b529ac4690 2024-04-24T23:30:54.599 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.594+0000 7f64c3fff640 10 client.25339 handle_command_reply: tid=0 2024-04-24T23:30:54.599 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.594+0000 7f64ed999740 1 client.25339 shutdown 2024-04-24T23:30:54.599 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.595+0000 7f64ed999740 1 -- 172.21.15.80:0/2979478928 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b529a4cda0 msgr2=0x55b529a56d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:54.599 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.595+0000 7f64ed999740 1 --2- 172.21.15.80:0/2979478928 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b529a4cda0 0x55b529a56d50 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f64d004d000 tx=0x7f64d0003890 comp rx=0 tx=0).stop 2024-04-24T23:30:54.599 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.595+0000 7f64ed999740 1 -- 172.21.15.80:0/2979478928 shutdown_connections 2024-04-24T23:30:54.599 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.595+0000 7f64ed999740 1 --2- 172.21.15.80:0/2979478928 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x55b529ac4690 0x55b529ac6ab0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:54.599 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.595+0000 7f64ed999740 1 --2- 172.21.15.80:0/2979478928 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b529a57d60 0x55b529a581e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:54.600 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.595+0000 7f64c3fff640 0 client.25339 ms_handle_reset on v2:172.21.15.148:6838/2952757594 2024-04-24T23:30:54.600 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.595+0000 7f64ed999740 1 --2- 172.21.15.80:0/2979478928 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b529a4cda0 0x55b529a56d50 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:54.600 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.595+0000 7f64ed999740 1 --2- 172.21.15.80:0/2979478928 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b529a47240 0x55b529a56810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:54.600 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.595+0000 7f64ed999740 1 -- 172.21.15.80:0/2979478928 >> 172.21.15.80:0/2979478928 conn(0x55b529a428b0 msgr2=0x55b529a41260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:54.600 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.595+0000 7f64ed999740 1 -- 172.21.15.80:0/2979478928 shutdown_connections 2024-04-24T23:30:54.600 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.595+0000 7f64ed999740 1 -- 172.21.15.80:0/2979478928 wait complete. 2024-04-24T23:30:54.600 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.595+0000 7f64ed999740 20 client.25339 trim_cache size 0 max 16384 2024-04-24T23:30:54.600 INFO:teuthology.orchestra.run.smithi080.stdout:{ 2024-04-24T23:30:54.600 INFO:teuthology.orchestra.run.smithi080.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T23:30:54.600 INFO:teuthology.orchestra.run.smithi080.stdout: "scrubs": { 2024-04-24T23:30:54.600 INFO:teuthology.orchestra.run.smithi080.stdout: "1da3681e-5460-48bb-b660-0edbcbd23b8a": { 2024-04-24T23:30:54.601 INFO:teuthology.orchestra.run.smithi080.stdout: "path": "/", 2024-04-24T23:30:54.601 INFO:teuthology.orchestra.run.smithi080.stdout: "tag": "1da3681e-5460-48bb-b660-0edbcbd23b8a", 2024-04-24T23:30:54.601 INFO:teuthology.orchestra.run.smithi080.stdout: "options": "recursive,force" 2024-04-24T23:30:54.601 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:30:54.601 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:30:54.601 INFO:teuthology.orchestra.run.smithi080.stdout:} 2024-04-24T23:30:54.601 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.596+0000 7f64eaba9640 1 -- 172.21.15.80:0/677474691 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f64d40a2c10 msgr2=0x7f64d40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:54.602 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.596+0000 7f64eaba9640 1 --2- 172.21.15.80:0/677474691 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f64d40a2c10 0x7f64d40a50d0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f64d00314e0 tx=0x7f64d0000bc0 comp rx=0 tx=0).stop 2024-04-24T23:30:54.602 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.596+0000 7f64eaba9640 1 -- 172.21.15.80:0/677474691 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f64e4101790 msgr2=0x7f64e40754f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:30:54.602 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.596+0000 7f64eaba9640 1 --2- 172.21.15.80:0/677474691 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f64e4101790 0x7f64e40754f0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f64d800d770 tx=0x7f64d800dc30 comp rx=0 tx=0).stop 2024-04-24T23:30:54.602 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.596+0000 7f64eaba9640 1 -- 172.21.15.80:0/677474691 shutdown_connections 2024-04-24T23:30:54.602 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.596+0000 7f64eaba9640 1 --2- 172.21.15.80:0/677474691 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f64d40a2c10 0x7f64d40a50d0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:54.602 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.596+0000 7f64eaba9640 1 --2- 172.21.15.80:0/677474691 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64e4103340 0x7f64e4074070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:54.602 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.596+0000 7f64eaba9640 1 --2- 172.21.15.80:0/677474691 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f64e4102980 0x7f64e4073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:54.602 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.596+0000 7f64eaba9640 1 --2- 172.21.15.80:0/677474691 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f64e4101790 0x7f64e40754f0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:30:54.602 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.596+0000 7f64eaba9640 1 -- 172.21.15.80:0/677474691 >> 172.21.15.80:0/677474691 conn(0x7f64e40fcd20 msgr2=0x7f64e4105d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:30:54.602 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.597+0000 7f64eaba9640 1 -- 172.21.15.80:0/677474691 shutdown_connections 2024-04-24T23:30:54.602 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:30:54.597+0000 7f64eaba9640 1 -- 172.21.15.80:0/677474691 wait complete. 2024-04-24T23:30:54.610 INFO:tasks.cephfs.filesystem:scrub status for tag:1da3681e-5460-48bb-b660-0edbcbd23b8a - {'path': '/', 'tag': '1da3681e-5460-48bb-b660-0edbcbd23b8a', 'options': 'recursive,force'} 2024-04-24T23:30:54.697 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:30:54 smithi134 ceph-mon[32619]: pgmap v407: 129 pgs: 129 active+clean; 656 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:30:55.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:30:55 smithi148 ceph-mon[32528]: scrub queued for path: / 2024-04-24T23:30:55.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:30:55 smithi148 ceph-mon[32528]: scrub summary: idle+waiting paths [/] 2024-04-24T23:30:55.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:30:55 smithi148 ceph-mon[32528]: scrub summary: active paths [/] 2024-04-24T23:30:55.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:30:55 smithi148 ceph-mon[32528]: scrub summary: idle+waiting paths [/] 2024-04-24T23:30:55.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:30:55 smithi148 ceph-mon[32528]: scrub summary: idle 2024-04-24T23:30:55.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:30:55 smithi080 ceph-mon[29152]: scrub queued for path: / 2024-04-24T23:30:55.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:30:55 smithi080 ceph-mon[29152]: scrub summary: idle+waiting paths [/] 2024-04-24T23:30:55.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:30:55 smithi080 ceph-mon[29152]: scrub summary: active paths [/] 2024-04-24T23:30:55.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:30:55 smithi080 ceph-mon[29152]: scrub summary: idle+waiting paths [/] 2024-04-24T23:30:55.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:30:55 smithi080 ceph-mon[29152]: scrub summary: idle 2024-04-24T23:30:55.697 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:30:55 smithi134 ceph-mon[32619]: scrub queued for path: / 2024-04-24T23:30:55.697 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:30:55 smithi134 ceph-mon[32619]: scrub summary: idle+waiting paths [/] 2024-04-24T23:30:55.697 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:30:55 smithi134 ceph-mon[32619]: scrub summary: active paths [/] 2024-04-24T23:30:55.697 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:30:55 smithi134 ceph-mon[32619]: scrub summary: idle+waiting paths [/] 2024-04-24T23:30:55.697 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:30:55 smithi134 ceph-mon[32619]: scrub summary: idle 2024-04-24T23:30:56.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:30:56 smithi148 ceph-mon[32528]: pgmap v408: 129 pgs: 129 active+clean; 656 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T23:30:56.537 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:30:56 smithi080 ceph-mon[29152]: pgmap v408: 129 pgs: 129 active+clean; 656 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T23:30:56.697 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:30:56 smithi134 ceph-mon[32619]: pgmap v408: 129 pgs: 129 active+clean; 656 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T23:30:58.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:30:58 smithi148 ceph-mon[32528]: pgmap v409: 129 pgs: 129 active+clean; 656 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:30:58.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:30:58 smithi080 ceph-mon[29152]: pgmap v409: 129 pgs: 129 active+clean; 656 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:30:58.696 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:30:58 smithi134 ceph-mon[32619]: pgmap v409: 129 pgs: 129 active+clean; 656 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:31:00.386 INFO:tasks.workunit.client.0.smithi080.stderr:Updating files: 96% (12080/12492) Updating files: 97% (12118/12492) Updating files: 98% (12243/12492) Updating files: 99% (12368/12492) Updating files: 100% (12492/12492) Updating files: 100% (12492/12492), done. 2024-04-24T23:31:00.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:31:00 smithi148 ceph-mon[32528]: pgmap v410: 129 pgs: 129 active+clean; 656 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-04-24T23:31:00.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:31:00 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:31:00.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:31:00 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:31:00.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:31:00 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:31:00.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:31:00 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:31:00.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:31:00 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:31:00.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:31:00 smithi080 ceph-mon[29152]: pgmap v410: 129 pgs: 129 active+clean; 656 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-04-24T23:31:00.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:31:00 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:31:00.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:31:00 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:31:00.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:31:00 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:31:00.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:31:00 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:31:00.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:31:00 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:31:00.697 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:31:00 smithi134 ceph-mon[32619]: pgmap v410: 129 pgs: 129 active+clean; 656 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-04-24T23:31:00.697 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:31:00 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:31:00.697 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:31:00 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:31:00.697 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:31:00 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:31:00.697 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:31:00 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:31:00.698 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:31:00 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:31:00.931 INFO:tasks.workunit.client.0.smithi080.stderr:Note: switching to 'd2c45d977142e024a59b68a8f1dce8f0b9840551'. 2024-04-24T23:31:00.932 INFO:tasks.workunit.client.0.smithi080.stderr: 2024-04-24T23:31:00.932 INFO:tasks.workunit.client.0.smithi080.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-24T23:31:00.932 INFO:tasks.workunit.client.0.smithi080.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-24T23:31:00.932 INFO:tasks.workunit.client.0.smithi080.stderr:state without impacting any branches by switching back to a branch. 2024-04-24T23:31:00.932 INFO:tasks.workunit.client.0.smithi080.stderr: 2024-04-24T23:31:00.932 INFO:tasks.workunit.client.0.smithi080.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-24T23:31:00.932 INFO:tasks.workunit.client.0.smithi080.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-24T23:31:00.932 INFO:tasks.workunit.client.0.smithi080.stderr: 2024-04-24T23:31:00.932 INFO:tasks.workunit.client.0.smithi080.stderr: git switch -c 2024-04-24T23:31:00.932 INFO:tasks.workunit.client.0.smithi080.stderr: 2024-04-24T23:31:00.932 INFO:tasks.workunit.client.0.smithi080.stderr:Or undo this operation with: 2024-04-24T23:31:00.932 INFO:tasks.workunit.client.0.smithi080.stderr: 2024-04-24T23:31:00.932 INFO:tasks.workunit.client.0.smithi080.stderr: git switch - 2024-04-24T23:31:00.932 INFO:tasks.workunit.client.0.smithi080.stderr: 2024-04-24T23:31:00.933 INFO:tasks.workunit.client.0.smithi080.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-24T23:31:00.933 INFO:tasks.workunit.client.0.smithi080.stderr: 2024-04-24T23:31:00.933 INFO:tasks.workunit.client.0.smithi080.stderr:HEAD is now at d2c45d97714 Merge pull request #56528 from adk3798/squid-mypy-bump 2024-04-24T23:31:00.935 DEBUG:teuthology.orchestra.run.smithi080:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-24T23:31:00.977 INFO:tasks.workunit.client.0.smithi080.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-24T23:31:00.979 INFO:tasks.workunit.client.0.smithi080.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-24T23:31:00.979 INFO:tasks.workunit.client.0.smithi080.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-24T23:31:01.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:31:01 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:31:01.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:31:01 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:31:01.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:31:01 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:31:01.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:31:01 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:31:01.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:31:01 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:31:01.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:31:01 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:31:01.567 INFO:tasks.workunit.client.0.smithi080.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-24T23:31:01.637 INFO:tasks.workunit.client.0.smithi080.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-24T23:31:01.676 INFO:tasks.workunit.client.0.smithi080.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-24T23:31:01.677 INFO:tasks.workunit.client.0.smithi080.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-24T23:31:01.678 INFO:tasks.workunit.client.0.smithi080.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-24T23:31:01.696 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:31:01 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:31:01.697 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:31:01 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:31:01.697 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:31:01 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:31:01.715 INFO:tasks.workunit.client.0.smithi080.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-24T23:31:01.719 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-24T23:31:01.719 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-24T23:31:01.775 INFO:tasks.workunit:Running workunits matching suites/fsx.sh on client.0... 2024-04-24T23:31:01.776 INFO:tasks.workunit:Running workunit suites/fsx.sh... 2024-04-24T23:31:01.776 DEBUG:teuthology.orchestra.run.smithi080:workunit test suites/fsx.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=d2c45d977142e024a59b68a8f1dce8f0b9840551 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsx.sh 2024-04-24T23:31:01.834 INFO:tasks.workunit.client.0.smithi080.stderr:+ set -e 2024-04-24T23:31:01.835 INFO:tasks.workunit.client.0.smithi080.stderr:+ git clone https://git.ceph.com/xfstests-dev.git 2024-04-24T23:31:01.837 INFO:tasks.workunit.client.0.smithi080.stderr:Cloning into 'xfstests-dev'... 2024-04-24T23:31:02.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:31:02 smithi148 ceph-mon[32528]: pgmap v411: 129 pgs: 129 active+clean; 656 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-04-24T23:31:02.537 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:31:02 smithi080 ceph-mon[29152]: pgmap v411: 129 pgs: 129 active+clean; 656 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-04-24T23:31:02.697 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:31:02 smithi134 ceph-mon[32619]: pgmap v411: 129 pgs: 129 active+clean; 656 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-04-24T23:31:04.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:31:04 smithi148 ceph-mon[32528]: pgmap v412: 129 pgs: 129 active+clean; 656 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-24T23:31:04.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:31:04 smithi080 ceph-mon[29152]: pgmap v412: 129 pgs: 129 active+clean; 656 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-24T23:31:04.697 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:31:04 smithi134 ceph-mon[32619]: pgmap v412: 129 pgs: 129 active+clean; 656 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-24T23:31:06.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:31:06 smithi148 ceph-mon[32528]: pgmap v413: 129 pgs: 129 active+clean; 4.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 324 KiB/s wr, 9 op/s 2024-04-24T23:31:06.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:31:06 smithi080 ceph-mon[29152]: pgmap v413: 129 pgs: 129 active+clean; 4.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 324 KiB/s wr, 9 op/s 2024-04-24T23:31:06.696 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:31:06 smithi134 ceph-mon[32619]: pgmap v413: 129 pgs: 129 active+clean; 4.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 324 KiB/s wr, 9 op/s 2024-04-24T23:31:07.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:31:07 smithi080 ceph-mon[29152]: pgmap v414: 129 pgs: 129 active+clean; 4.4 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 324 KiB/s wr, 6 op/s 2024-04-24T23:31:07.697 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:31:07 smithi134 ceph-mon[32619]: pgmap v414: 129 pgs: 129 active+clean; 4.4 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 324 KiB/s wr, 6 op/s 2024-04-24T23:31:07.756 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:31:07 smithi148 ceph-mon[32528]: pgmap v414: 129 pgs: 129 active+clean; 4.4 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 324 KiB/s wr, 6 op/s 2024-04-24T23:31:09.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:31:09 smithi148 ceph-mon[32528]: pgmap v415: 129 pgs: 129 active+clean; 15 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 11 op/s 2024-04-24T23:31:09.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:31:09 smithi080 ceph-mon[29152]: pgmap v415: 129 pgs: 129 active+clean; 15 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 11 op/s 2024-04-24T23:31:09.696 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:31:09 smithi134 ceph-mon[32619]: pgmap v415: 129 pgs: 129 active+clean; 15 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 11 op/s 2024-04-24T23:31:12.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:31:11 smithi148 ceph-mon[32528]: pgmap v416: 129 pgs: 129 active+clean; 15 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 11 op/s 2024-04-24T23:31:12.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:31:11 smithi080 ceph-mon[29152]: pgmap v416: 129 pgs: 129 active+clean; 15 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 11 op/s 2024-04-24T23:31:12.449 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:31:11 smithi134 ceph-mon[32619]: pgmap v416: 129 pgs: 129 active+clean; 15 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 11 op/s 2024-04-24T23:31:14.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:31:14 smithi148 ceph-mon[32528]: pgmap v417: 129 pgs: 129 active+clean; 15 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 10 op/s 2024-04-24T23:31:14.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:31:14 smithi080 ceph-mon[29152]: pgmap v417: 129 pgs: 129 active+clean; 15 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 10 op/s 2024-04-24T23:31:14.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:31:14 smithi134 ceph-mon[32619]: pgmap v417: 129 pgs: 129 active+clean; 15 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 10 op/s 2024-04-24T23:31:16.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:31:16 smithi080 ceph-mon[29152]: pgmap v418: 129 pgs: 129 active+clean; 15 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 17 op/s 2024-04-24T23:31:16.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:31:16 smithi080 ceph-mon[29152]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T23:31:16.446 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:31:16 smithi134 ceph-mon[32619]: pgmap v418: 129 pgs: 129 active+clean; 15 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 17 op/s 2024-04-24T23:31:16.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:31:16 smithi134 ceph-mon[32619]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T23:31:16.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:31:16 smithi148 ceph-mon[32528]: pgmap v418: 129 pgs: 129 active+clean; 15 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 17 op/s 2024-04-24T23:31:16.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:31:16 smithi148 ceph-mon[32528]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T23:31:18.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:31:18 smithi080 ceph-mon[29152]: pgmap v419: 129 pgs: 129 active+clean; 15 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 789 KiB/s rd, 927 KiB/s wr, 13 op/s 2024-04-24T23:31:18.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:31:18 smithi134 ceph-mon[32619]: pgmap v419: 129 pgs: 129 active+clean; 15 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 789 KiB/s rd, 927 KiB/s wr, 13 op/s 2024-04-24T23:31:18.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:31:18 smithi148 ceph-mon[32528]: pgmap v419: 129 pgs: 129 active+clean; 15 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 789 KiB/s rd, 927 KiB/s wr, 13 op/s 2024-04-24T23:31:20.287 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:31:20 smithi080 ceph-mon[29152]: pgmap v420: 129 pgs: 129 active+clean; 19 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 28 op/s 2024-04-24T23:31:20.446 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:31:20 smithi134 ceph-mon[32619]: pgmap v420: 129 pgs: 129 active+clean; 19 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 28 op/s 2024-04-24T23:31:20.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:31:20 smithi148 ceph-mon[32528]: pgmap v420: 129 pgs: 129 active+clean; 19 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 28 op/s 2024-04-24T23:31:21.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:31:21 smithi080 ceph-mon[29152]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T23:31:21.446 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:31:21 smithi134 ceph-mon[32619]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T23:31:21.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:31:21 smithi148 ceph-mon[32528]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T23:31:22.060 INFO:tasks.workunit.client.0.smithi080.stderr:Updating files: 11% (491/4290) Updating files: 12% (515/4290) Updating files: 13% (558/4290) Updating files: 14% (601/4290) Updating files: 15% (644/4290) Updating files: 16% (687/4290) Updating files: 17% (730/4290) Updating files: 18% (773/4290) Updating files: 18% (791/4290) Updating files: 19% (816/4290) Updating files: 20% (858/4290) Updating files: 21% (901/4290) Updating files: 22% (944/4290) Updating files: 23% (987/4290) Updating files: 24% (1030/4290) Updating files: 25% (1073/4290) Updating files: 26% (1116/4290) Updating files: 27% (1159/4290) Updating files: 28% (1202/4290) Updating files: 29% (1245/4290) Updating files: 30% (1287/4290) Updating files: 30% (1288/4290) Updating files: 31% (1330/4290) Updating files: 32% (1373/4290) Updating files: 32% (1379/4290) Updating files: 33% (1416/4290) Updating files: 34% (1459/4290) Updating files: 35% (1502/4290) Updating files: 36% (1545/4290) Updating files: 37% (1588/4290) Updating files: 38% (1631/4290) Updating files: 39% (1674/4290) Updating files: 40% (1716/4290) Updating files: 41% (1759/4290) Updating files: 42% (1802/4290) Updating files: 43% (1845/4290) Updating files: 44% (1888/4290) Updating files: 45% (1931/4290) Updating files: 46% (1974/4290) Updating files: 47% (2017/4290) Updating files: 48% (2060/4290) Updating files: 49% (2103/4290) Updating files: 49% (2136/4290) Updating files: 50% (2145/4290) Updating files: 51% (2188/4290) Updating files: 52% (2231/4290) Updating files: 53% (2274/4290) Updating files: 54% (2317/4290) Updating files: 55% (2360/4290) Updating files: 56% (2403/4290) Updating files: 57% (2446/4290) Updating files: 58% (2489/4290) Updating files: 59% (2532/4290) Updating files: 60% (2574/4290) Updating files: 61% (2617/4290) Updating files: 62% (2660/4290) Updating files: 62% (2680/4290) Updating files: 63% (2703/4290) Updating files: 64% (2746/4290) Updating files: 64% (2784/4290) Updating files: 65% (2789/4290) Updating files: 66% (2832/4290) Updating files: 67% (2875/4290) Updating files: 68% (2918/4290) Updating files: 69% (2961/4290) Updating files: 70% (3003/4290) Updating files: 70% (3022/4290) Updating files: 71% (3046/4290) Updating files: 72% (3089/4290) Updating files: 73% (3132/4290) Updating files: 74% (3175/4290) Updating files: 75% (3218/4290) Updating files: 76% (3261/4290) Updating files: 77% (3304/4290) Updating files: 78% (3347/4290) Updating files: 79% (3390/4290) Updating files: 80% (3432/4290) Updating files: 81% (3475/4290) Updating files: 82% (3518/4290) Updating files: 83% (3561/4290) Updating files: 84% (3604/4290) Updating files: 85% (3647/4290) Updating files: 86% (3690/4290) Updating files: 87% (3733/4290) Updating files: 88% (3776/4290) Updating files: 88% (3807/4290) Updating files: 89% (3819/4290) Updating files: 90% (3861/4290) Updating files: 91% (3904/4290) Updating files: 92% (3947/4290) Updating files: 93% (3990/4290) Updating files: 94% (4033/4290) Updating files: 95% (4076/4290) Updating files: 96% (4119/4290) Updating files: 97% (4162/4290) Updating files: 98% (4205/4290) Updating files: 99% (4248/4290) Updating files: 99% (4271/4290) Updating files: 100% (4290/4290) Updating files: 100% (4290/4290), done. 2024-04-24T23:31:22.071 INFO:tasks.workunit.client.0.smithi080.stderr:+ cd xfstests-dev 2024-04-24T23:31:22.071 INFO:tasks.workunit.client.0.smithi080.stderr:+ git checkout 12973fc04fd10d4af086901e10ffa8e48866b735 2024-04-24T23:31:22.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:31:22 smithi080 ceph-mon[29152]: pgmap v421: 129 pgs: 129 active+clean; 19 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 511 KiB/s wr, 28 op/s 2024-04-24T23:31:22.348 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:31:22 smithi134 ceph-mon[32619]: pgmap v421: 129 pgs: 129 active+clean; 19 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 511 KiB/s wr, 28 op/s 2024-04-24T23:31:22.505 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:31:22 smithi148 ceph-mon[32528]: pgmap v421: 129 pgs: 129 active+clean; 19 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 511 KiB/s wr, 28 op/s 2024-04-24T23:31:24.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:31:24 smithi080 ceph-mon[29152]: pgmap v422: 129 pgs: 129 active+clean; 20 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 588 KiB/s wr, 29 op/s 2024-04-24T23:31:24.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:31:24 smithi134 ceph-mon[32619]: pgmap v422: 129 pgs: 129 active+clean; 20 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 588 KiB/s wr, 29 op/s 2024-04-24T23:31:24.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:31:24 smithi148 ceph-mon[32528]: pgmap v422: 129 pgs: 129 active+clean; 20 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 588 KiB/s wr, 29 op/s 2024-04-24T23:31:24.611 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T23:31:24.611 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T23:31:24.719 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.718+0000 7f67f22b3640 1 -- 172.21.15.80:0/135584972 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f67ec1034a0 msgr2=0x7f67ec109d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:24.719 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.718+0000 7f67f22b3640 1 --2- 172.21.15.80:0/135584972 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f67ec1034a0 0x7f67ec109d30 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f67e4009cd0 tx=0x7f67e402a8e0 comp rx=0 tx=0).stop 2024-04-24T23:31:24.719 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.718+0000 7f67f22b3640 1 -- 172.21.15.80:0/135584972 shutdown_connections 2024-04-24T23:31:24.719 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.718+0000 7f67f22b3640 1 --2- 172.21.15.80:0/135584972 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f67ec1034a0 0x7f67ec109d30 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:24.719 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.718+0000 7f67f22b3640 1 --2- 172.21.15.80:0/135584972 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f67ec102ae0 0x7f67ec102f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:24.719 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.718+0000 7f67f22b3640 1 --2- 172.21.15.80:0/135584972 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67ec0ffa70 0x7f67ec0ffe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:24.720 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.718+0000 7f67f22b3640 1 -- 172.21.15.80:0/135584972 >> 172.21.15.80:0/135584972 conn(0x7f67ec0fb380 msgr2=0x7f67ec0fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:24.720 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.718+0000 7f67f22b3640 1 -- 172.21.15.80:0/135584972 shutdown_connections 2024-04-24T23:31:24.720 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.718+0000 7f67f22b3640 1 -- 172.21.15.80:0/135584972 wait complete. 2024-04-24T23:31:24.720 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.718+0000 7f67f22b3640 1 Processor -- start 2024-04-24T23:31:24.720 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.719+0000 7f67f22b3640 1 -- start start 2024-04-24T23:31:24.722 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.719+0000 7f67f22b3640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f67ec0ffa70 0x7f67ec197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:24.722 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.719+0000 7f67f22b3640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67ec102ae0 0x7f67ec197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:24.722 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.719+0000 7f67f22b3640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f67ec1034a0 0x7f67ec19f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:24.722 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.719+0000 7f67f22b3640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f67ec10d410 con 0x7f67ec0ffa70 2024-04-24T23:31:24.723 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.719+0000 7f67f22b3640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f67ec10d290 con 0x7f67ec1034a0 2024-04-24T23:31:24.723 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.719+0000 7f67f22b3640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f67ec10d590 con 0x7f67ec102ae0 2024-04-24T23:31:24.723 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.719+0000 7f67ebfff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f67ec0ffa70 0x7f67ec197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:24.723 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.719+0000 7f67ebfff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f67ec0ffa70 0x7f67ec197a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38148/0 (socket says 172.21.15.80:38148) 2024-04-24T23:31:24.723 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.719+0000 7f67ebfff640 1 -- 172.21.15.80:0/1401054113 learned_addr learned my addr 172.21.15.80:0/1401054113 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:31:24.723 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.719+0000 7f67eb7fe640 1 --2- 172.21.15.80:0/1401054113 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67ec102ae0 0x7f67ec197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:24.723 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.719+0000 7f67ebfff640 1 -- 172.21.15.80:0/1401054113 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67ec102ae0 msgr2=0x7f67ec197f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:24.723 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.719+0000 7f67ebfff640 1 --2- 172.21.15.80:0/1401054113 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67ec102ae0 0x7f67ec197f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:24.723 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.719+0000 7f67ebfff640 1 -- 172.21.15.80:0/1401054113 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f67ec1034a0 msgr2=0x7f67ec19f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:24.723 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.719+0000 7f67ebfff640 1 --2- 172.21.15.80:0/1401054113 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f67ec1034a0 0x7f67ec19f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:24.723 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.719+0000 7f67ebfff640 1 -- 172.21.15.80:0/1401054113 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67ec19f670 con 0x7f67ec0ffa70 2024-04-24T23:31:24.723 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.719+0000 7f67ebfff640 1 --2- 172.21.15.80:0/1401054113 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f67ec0ffa70 0x7f67ec197a50 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f67dc00e9c0 tx=0x7f67dc00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:24.724 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.720+0000 7f67e97fa640 1 -- 172.21.15.80:0/1401054113 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67dc00cd80 con 0x7f67ec0ffa70 2024-04-24T23:31:24.724 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.720+0000 7f67f22b3640 1 -- 172.21.15.80:0/1401054113 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67ec19f960 con 0x7f67ec0ffa70 2024-04-24T23:31:24.724 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.720+0000 7f67e97fa640 1 -- 172.21.15.80:0/1401054113 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67dc0035f0 con 0x7f67ec0ffa70 2024-04-24T23:31:24.724 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.720+0000 7f67e97fa640 1 -- 172.21.15.80:0/1401054113 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67dc010690 con 0x7f67ec0ffa70 2024-04-24T23:31:24.724 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.720+0000 7f67f22b3640 1 -- 172.21.15.80:0/1401054113 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67ec19feb0 con 0x7f67ec0ffa70 2024-04-24T23:31:24.729 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.722+0000 7f67e97fa640 1 -- 172.21.15.80:0/1401054113 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f67dc0037a0 con 0x7f67ec0ffa70 2024-04-24T23:31:24.730 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.722+0000 7f67f4fc0740 1 Processor -- start 2024-04-24T23:31:24.730 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.723+0000 7f67f4fc0740 1 -- start start 2024-04-24T23:31:24.730 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.723+0000 7f67f4fc0740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563f57595240 0x563f57595660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:24.730 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.723+0000 7f67f4fc0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f57595c30 0x563f5759a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:24.730 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.723+0000 7f67f4fc0740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563f5759ada0 0x563f5759d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:24.730 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.723+0000 7f67f4fc0740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x563f574f96b0 con 0x563f57595240 2024-04-24T23:31:24.730 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.723+0000 7f67f4fc0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x563f574a1880 con 0x563f57595c30 2024-04-24T23:31:24.730 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.723+0000 7f67f4fc0740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x563f574f5670 con 0x563f5759ada0 2024-04-24T23:31:24.730 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.723+0000 7f67f0850640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563f57595240 0x563f57595660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:24.730 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.723+0000 7f67f0850640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563f57595240 0x563f57595660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38160/0 (socket says 172.21.15.80:38160) 2024-04-24T23:31:24.730 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.723+0000 7f67f0850640 1 -- 172.21.15.80:0/682292166 learned_addr learned my addr 172.21.15.80:0/682292166 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:31:24.730 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.723+0000 7f67eb7fe640 1 --2- 172.21.15.80:0/682292166 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563f5759ada0 0x563f5759d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:24.731 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.723+0000 7f67ebfff640 1 --2- 172.21.15.80:0/682292166 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f57595c30 0x563f5759a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:24.731 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.723+0000 7f67e97fa640 1 --2- 172.21.15.80:0/1401054113 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f67d40a2b90 0x7f67d40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:24.731 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.723+0000 7f67f0850640 1 -- 172.21.15.80:0/682292166 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563f5759ada0 msgr2=0x563f5759d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:24.731 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.723+0000 7f67f0850640 1 --2- 172.21.15.80:0/682292166 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563f5759ada0 0x563f5759d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:24.731 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.723+0000 7f67f0850640 1 -- 172.21.15.80:0/682292166 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f57595c30 msgr2=0x563f5759a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:24.731 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.723+0000 7f67f0850640 1 --2- 172.21.15.80:0/682292166 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f57595c30 0x563f5759a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:24.731 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.723+0000 7f67f0850640 1 -- 172.21.15.80:0/682292166 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f572d77d0 con 0x563f57595240 2024-04-24T23:31:24.731 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.723+0000 7f67ebfff640 1 --2- 172.21.15.80:0/682292166 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f57595c30 0x563f5759a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:31:24.731 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.723+0000 7f67e97fa640 1 -- 172.21.15.80:0/1401054113 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f67dc0cf810 con 0x7f67ec0ffa70 2024-04-24T23:31:24.731 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.723+0000 7f67f0850640 1 --2- 172.21.15.80:0/682292166 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563f57595240 0x563f57595660 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f67e4005cc0 tx=0x7f67e4009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=225679472d0f0a0b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:24.731 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.723+0000 7f67f0850640 1 --2- 172.21.15.80:0/1401054113 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f67d40a2b90 0x7f67d40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:24.731 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.723+0000 7f67d27fc640 1 -- 172.21.15.80:0/682292166 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e4031bc0 con 0x563f57595240 2024-04-24T23:31:24.731 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.724+0000 7f67d27fc640 1 -- 172.21.15.80:0/682292166 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67e4036040 con 0x563f57595240 2024-04-24T23:31:24.732 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.724+0000 7f67d27fc640 1 -- 172.21.15.80:0/682292166 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e4030e90 con 0x563f57595240 2024-04-24T23:31:24.732 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.724+0000 7f67f4fc0740 1 -- 172.21.15.80:0/682292166 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563f57595240 msgr2=0x563f57595660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:24.732 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.724+0000 7f67f0850640 1 --2- 172.21.15.80:0/1401054113 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f67d40a2b90 0x7f67d40a5050 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f67e4004af0 tx=0x7f67e404b000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:24.732 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.724+0000 7f67f4fc0740 1 --2- 172.21.15.80:0/682292166 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563f57595240 0x563f57595660 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f67e4005cc0 tx=0x7f67e4009a20 comp rx=0 tx=0).stop 2024-04-24T23:31:24.732 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.724+0000 7f67f4fc0740 1 -- 172.21.15.80:0/682292166 shutdown_connections 2024-04-24T23:31:24.732 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.724+0000 7f67f4fc0740 1 --2- 172.21.15.80:0/682292166 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563f5759ada0 0x563f5759d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:24.732 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.724+0000 7f67f4fc0740 1 --2- 172.21.15.80:0/682292166 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f57595c30 0x563f5759a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:24.732 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.724+0000 7f67f4fc0740 1 --2- 172.21.15.80:0/682292166 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563f57595240 0x563f57595660 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:24.732 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.724+0000 7f67f4fc0740 1 -- 172.21.15.80:0/682292166 >> 172.21.15.80:0/682292166 conn(0x563f575908b0 msgr2=0x563f57590cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:24.732 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.724+0000 7f67f4fc0740 1 -- 172.21.15.80:0/682292166 shutdown_connections 2024-04-24T23:31:24.732 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.724+0000 7f67f4fc0740 1 -- 172.21.15.80:0/682292166 wait complete. 2024-04-24T23:31:24.732 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.725+0000 7f67f4fc0740 1 -- start start 2024-04-24T23:31:24.732 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.725+0000 7f67f4fc0740 1 Processor -- start 2024-04-24T23:31:24.733 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.725+0000 7f67f4fc0740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563f57595240 0x563f5759f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:24.733 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.725+0000 7f67f4fc0740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563f57595c30 0x563f5759fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:24.733 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.725+0000 7f67f4fc0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f5759ada0 0x563f575a6b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:24.733 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.725+0000 7f67f4fc0740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x563f5759dcf0 con 0x563f57595240 2024-04-24T23:31:24.733 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.725+0000 7f67f4fc0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x563f574b4f50 con 0x563f5759ada0 2024-04-24T23:31:24.733 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.725+0000 7f67f4fc0740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x563f5759de70 con 0x563f57595c30 2024-04-24T23:31:24.733 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.725+0000 7f67f4fc0740 10 client.0 _ll_register_callbacks cb 0x563f574f0000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:31:24.733 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.725+0000 7f67ebfff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563f57595240 0x563f5759f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:24.733 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.726+0000 7f67ebfff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563f57595240 0x563f5759f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38176/0 (socket says 172.21.15.80:38176) 2024-04-24T23:31:24.733 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.726+0000 7f67ebfff640 1 -- 172.21.15.80:0/1413934405 learned_addr learned my addr 172.21.15.80:0/1413934405 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:31:24.733 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.726+0000 7f67eb7fe640 1 --2- 172.21.15.80:0/1413934405 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563f57595c30 0x563f5759fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:24.733 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.726+0000 7f67f0850640 1 --2- 172.21.15.80:0/1413934405 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f5759ada0 0x563f575a6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:24.733 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.726+0000 7f67ebfff640 1 -- 172.21.15.80:0/1413934405 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563f57595c30 msgr2=0x563f5759fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:24.733 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.726+0000 7f67ebfff640 1 --2- 172.21.15.80:0/1413934405 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563f57595c30 0x563f5759fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:24.733 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.726+0000 7f67ebfff640 1 -- 172.21.15.80:0/1413934405 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f5759ada0 msgr2=0x563f575a6b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:24.734 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.726+0000 7f67ebfff640 1 --2- 172.21.15.80:0/1413934405 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f5759ada0 0x563f575a6b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:24.734 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.726+0000 7f67ebfff640 1 -- 172.21.15.80:0/1413934405 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f5720d3f0 con 0x563f57595240 2024-04-24T23:31:24.734 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.726+0000 7f67ebfff640 1 --2- 172.21.15.80:0/1413934405 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563f57595240 0x563f5759f600 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f67dc09b9a0 tx=0x7f67dc09b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a813dea340ae7db6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:24.734 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.726+0000 7f67cb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-04-24T23:31:24.734 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.726+0000 7f67cb7fe640 1 -- 172.21.15.80:0/1413934405 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67dc09c3a0 con 0x563f57595240 2024-04-24T23:31:24.734 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.726+0000 7f67cb7fe640 1 -- 172.21.15.80:0/1413934405 <== mon.0 v2:172.21.15.80:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f67dc099fa0 con 0x563f57595240 2024-04-24T23:31:24.734 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.726+0000 7f67f4fc0740 1 -- 172.21.15.80:0/1413934405 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67e4036040 con 0x563f57595240 2024-04-24T23:31:24.734 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.726+0000 7f67cb7fe640 1 -- 172.21.15.80:0/1413934405 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f67dc09bca0 con 0x563f57595240 2024-04-24T23:31:24.734 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.726+0000 7f67f4fc0740 10 client.15879 fetch_fsmap learned FSMap version 27 2024-04-24T23:31:24.734 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.726+0000 7f67f4fc0740 1 -- 172.21.15.80:0/1413934405 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563f571cd0f0 con 0x563f57595240 2024-04-24T23:31:24.734 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.726+0000 7f67cb7fe640 1 -- 172.21.15.80:0/1413934405 <== mon.0 v2:172.21.15.80:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67e4036040 con 0x563f57595240 2024-04-24T23:31:24.734 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.727+0000 7f67cb7fe640 1 -- 172.21.15.80:0/1413934405 <== mon.0 v2:172.21.15.80:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f67dc0a0d40 con 0x563f57595240 2024-04-24T23:31:24.734 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.727+0000 7f67f4fc0740 10 client.15879 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:31:24.734 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.727+0000 7f67f4fc0740 10 client.15879 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:31:24.735 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.727+0000 7f67f4fc0740 20 client.15879 populate_metadata read hostname 'smithi080' 2024-04-24T23:31:24.735 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.727+0000 7f67f4fc0740 1 --2- 172.21.15.80:0/1413934405 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x563f575ab430 0x563f575ad8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:24.735 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.727+0000 7f67f4fc0740 4 client.15879 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:31:24.735 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.727+0000 7f67f4fc0740 1 -- 172.21.15.80:0/1413934405 --> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563f57506bf0 con 0x563f575ab430 2024-04-24T23:31:24.735 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.727+0000 7f67eb7fe640 1 --2- 172.21.15.80:0/1413934405 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x563f575ab430 0x563f575ad8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:24.735 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.727+0000 7f67cb7fe640 10 client.15879 ms_handle_connect on v2:172.21.15.148:6838/2952757594 2024-04-24T23:31:24.735 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.727+0000 7f67eb7fe640 1 --2- 172.21.15.80:0/1413934405 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x563f575ab430 0x563f575ad8f0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5dffd581167015e server_cookie=486bbb2f0fee7978 in_seq=0 out_seq=0 2024-04-24T23:31:24.736 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.731+0000 7f67cb7fe640 1 -- 172.21.15.80:0/1413934405 <== mds.0 v2:172.21.15.148:6838/2952757594 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563f57506bf0 con 0x563f575ab430 2024-04-24T23:31:24.736 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.731+0000 7f67cb7fe640 10 client.15879 handle_command_reply: tid=0 2024-04-24T23:31:24.736 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.731+0000 7f67f4fc0740 1 client.15879 shutdown 2024-04-24T23:31:24.736 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.731+0000 7f67f4fc0740 1 -- 172.21.15.80:0/1413934405 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563f57595240 msgr2=0x563f5759f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:24.736 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.731+0000 7f67f4fc0740 1 --2- 172.21.15.80:0/1413934405 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563f57595240 0x563f5759f600 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f67dc09b9a0 tx=0x7f67dc09b8a0 comp rx=0 tx=0).stop 2024-04-24T23:31:24.737 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.731+0000 7f67f4fc0740 1 -- 172.21.15.80:0/1413934405 shutdown_connections 2024-04-24T23:31:24.737 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.731+0000 7f67f4fc0740 1 --2- 172.21.15.80:0/1413934405 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x563f575ab430 0x563f575ad8f0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:24.737 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.731+0000 7f67f4fc0740 1 --2- 172.21.15.80:0/1413934405 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f5759ada0 0x563f575a6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:24.737 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.731+0000 7f67cb7fe640 0 client.15879 ms_handle_reset on v2:172.21.15.148:6838/2952757594 2024-04-24T23:31:24.737 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.731+0000 7f67f4fc0740 1 --2- 172.21.15.80:0/1413934405 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563f57595c30 0x563f5759fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:24.737 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.731+0000 7f67f4fc0740 1 --2- 172.21.15.80:0/1413934405 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563f57595240 0x563f5759f600 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:24.737 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.731+0000 7f67f4fc0740 1 -- 172.21.15.80:0/1413934405 >> 172.21.15.80:0/1413934405 conn(0x563f575908b0 msgr2=0x563f5758faf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:24.737 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.731+0000 7f67f4fc0740 1 -- 172.21.15.80:0/1413934405 shutdown_connections 2024-04-24T23:31:24.737 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.731+0000 7f67f4fc0740 1 -- 172.21.15.80:0/1413934405 wait complete. 2024-04-24T23:31:24.737 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.731+0000 7f67f4fc0740 20 client.15879 trim_cache size 0 max 16384 2024-04-24T23:31:24.753 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.748+0000 7f67f4fc0740 1 Processor -- start 2024-04-24T23:31:24.754 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.748+0000 7f67f4fc0740 1 -- start start 2024-04-24T23:31:24.754 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.749+0000 7f67f4fc0740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563f57595240 0x563f5759f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:24.754 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.749+0000 7f67f4fc0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f57595c30 0x563f575a6b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:24.754 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.749+0000 7f67f4fc0740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563f5759ada0 0x563f57593860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:24.754 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.749+0000 7f67f4fc0740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f67ec1ace20 con 0x563f57595240 2024-04-24T23:31:24.754 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.749+0000 7f67f4fc0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f67ec1aa6f0 con 0x563f57595c30 2024-04-24T23:31:24.754 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.749+0000 7f67f4fc0740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f67ec1a8540 con 0x563f5759ada0 2024-04-24T23:31:24.754 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.749+0000 7f67ebfff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563f57595240 0x563f5759f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:24.754 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.749+0000 7f67ebfff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563f57595240 0x563f5759f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38178/0 (socket says 172.21.15.80:38178) 2024-04-24T23:31:24.754 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.749+0000 7f67ebfff640 1 -- 172.21.15.80:0/732311372 learned_addr learned my addr 172.21.15.80:0/732311372 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:31:24.754 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.749+0000 7f67f0850640 1 --2- 172.21.15.80:0/732311372 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563f5759ada0 0x563f57593860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:24.754 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.749+0000 7f67eb7fe640 1 --2- 172.21.15.80:0/732311372 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f57595c30 0x563f575a6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:24.755 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.749+0000 7f67ebfff640 1 -- 172.21.15.80:0/732311372 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563f5759ada0 msgr2=0x563f57593860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:24.755 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.749+0000 7f67ebfff640 1 --2- 172.21.15.80:0/732311372 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563f5759ada0 0x563f57593860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:24.755 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.749+0000 7f67ebfff640 1 -- 172.21.15.80:0/732311372 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f57595c30 msgr2=0x563f575a6b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:24.755 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.749+0000 7f67ebfff640 1 --2- 172.21.15.80:0/732311372 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f57595c30 0x563f575a6b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:24.755 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.749+0000 7f67ebfff640 1 -- 172.21.15.80:0/732311372 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f57582090 con 0x563f57595240 2024-04-24T23:31:24.755 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.749+0000 7f67ebfff640 1 --2- 172.21.15.80:0/732311372 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563f57595240 0x563f5759f920 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f67dc0cfbc0 tx=0x7f67dc003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bd7815ed3816ddc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:24.755 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.749+0000 7f67d1ffb640 1 -- 172.21.15.80:0/732311372 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67dc09c3a0 con 0x563f57595240 2024-04-24T23:31:24.755 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.749+0000 7f67d1ffb640 1 -- 172.21.15.80:0/732311372 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67dc0a4910 con 0x563f57595240 2024-04-24T23:31:24.755 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.749+0000 7f67d1ffb640 1 -- 172.21.15.80:0/732311372 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67dc09b350 con 0x563f57595240 2024-04-24T23:31:24.755 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.750+0000 7f67f4fc0740 1 -- 172.21.15.80:0/732311372 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563f57595240 msgr2=0x563f5759f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:24.755 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.750+0000 7f67f4fc0740 1 --2- 172.21.15.80:0/732311372 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563f57595240 0x563f5759f920 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f67dc0cfbc0 tx=0x7f67dc003f10 comp rx=0 tx=0).stop 2024-04-24T23:31:24.755 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.750+0000 7f67f4fc0740 1 -- 172.21.15.80:0/732311372 shutdown_connections 2024-04-24T23:31:24.755 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.750+0000 7f67f4fc0740 1 --2- 172.21.15.80:0/732311372 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563f5759ada0 0x563f57593860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:24.755 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.750+0000 7f67f4fc0740 1 --2- 172.21.15.80:0/732311372 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f57595c30 0x563f575a6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:24.755 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.750+0000 7f67f4fc0740 1 --2- 172.21.15.80:0/732311372 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563f57595240 0x563f5759f920 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:24.756 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.750+0000 7f67f4fc0740 1 -- 172.21.15.80:0/732311372 >> 172.21.15.80:0/732311372 conn(0x563f575908b0 msgr2=0x563f5759f210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:24.756 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.750+0000 7f67f4fc0740 1 -- 172.21.15.80:0/732311372 shutdown_connections 2024-04-24T23:31:24.756 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.750+0000 7f67f4fc0740 1 -- 172.21.15.80:0/732311372 wait complete. 2024-04-24T23:31:24.756 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.750+0000 7f67f4fc0740 1 -- start start 2024-04-24T23:31:24.756 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.751+0000 7f67f4fc0740 1 Processor -- start 2024-04-24T23:31:24.756 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.751+0000 7f67f4fc0740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563f57595240 0x563f575a4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:24.756 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.751+0000 7f67f4fc0740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563f57595c30 0x563f575a4640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:24.756 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.751+0000 7f67f4fc0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f5759ada0 0x563f575a5650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:24.756 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.751+0000 7f67f4fc0740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f67ec1a8bb0 con 0x563f57595240 2024-04-24T23:31:24.756 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.751+0000 7f67f4fc0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f67ec1aa020 con 0x563f5759ada0 2024-04-24T23:31:24.756 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.751+0000 7f67f4fc0740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f67ec1a9280 con 0x563f57595c30 2024-04-24T23:31:24.756 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.751+0000 7f67f4fc0740 10 client.0 _ll_register_callbacks cb 0x7f67dc003e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:31:24.756 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.751+0000 7f67ebfff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563f57595240 0x563f575a4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:24.756 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.751+0000 7f67ebfff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563f57595240 0x563f575a4100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38190/0 (socket says 172.21.15.80:38190) 2024-04-24T23:31:24.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.751+0000 7f67ebfff640 1 -- 172.21.15.80:0/3074211994 learned_addr learned my addr 172.21.15.80:0/3074211994 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:31:24.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.751+0000 7f67eb7fe640 1 --2- 172.21.15.80:0/3074211994 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563f57595c30 0x563f575a4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:24.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.751+0000 7f67f0850640 1 --2- 172.21.15.80:0/3074211994 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f5759ada0 0x563f575a5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:24.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.751+0000 7f67ebfff640 1 -- 172.21.15.80:0/3074211994 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563f57595c30 msgr2=0x563f575a4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:24.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.751+0000 7f67ebfff640 1 --2- 172.21.15.80:0/3074211994 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563f57595c30 0x563f575a4640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:24.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.751+0000 7f67ebfff640 1 -- 172.21.15.80:0/3074211994 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f5759ada0 msgr2=0x563f575a5650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:24.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.751+0000 7f67ebfff640 1 --2- 172.21.15.80:0/3074211994 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f5759ada0 0x563f575a5650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:24.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.751+0000 7f67ebfff640 1 -- 172.21.15.80:0/3074211994 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f575a5c40 con 0x563f57595240 2024-04-24T23:31:24.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.751+0000 7f67eb7fe640 1 --2- 172.21.15.80:0/3074211994 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563f57595c30 0x563f575a4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:31:24.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.751+0000 7f67ebfff640 1 --2- 172.21.15.80:0/3074211994 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563f57595240 0x563f575a4100 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f67dc0cd2b0 tx=0x7f67dc09c2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4f972b2f5d08e29 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:24.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.751+0000 7f67cb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-04-24T23:31:24.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.751+0000 7f67cb7fe640 1 -- 172.21.15.80:0/3074211994 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67dc004200 con 0x563f57595240 2024-04-24T23:31:24.758 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.751+0000 7f67cb7fe640 1 -- 172.21.15.80:0/3074211994 <== mon.0 v2:172.21.15.80:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f67dc0979f0 con 0x563f57595240 2024-04-24T23:31:24.758 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.751+0000 7f67f4fc0740 1 -- 172.21.15.80:0/3074211994 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67dc0a4910 con 0x563f57595240 2024-04-24T23:31:24.758 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.751+0000 7f67cb7fe640 1 -- 172.21.15.80:0/3074211994 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f67dc09c3a0 con 0x563f57595240 2024-04-24T23:31:24.758 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.751+0000 7f67f4fc0740 10 client.15885 fetch_fsmap learned FSMap version 27 2024-04-24T23:31:24.758 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.751+0000 7f67f4fc0740 1 -- 172.21.15.80:0/3074211994 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563f576119e0 con 0x563f57595240 2024-04-24T23:31:24.758 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.752+0000 7f67cb7fe640 1 -- 172.21.15.80:0/3074211994 <== mon.0 v2:172.21.15.80:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67dc0a4910 con 0x563f57595240 2024-04-24T23:31:24.758 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.752+0000 7f67cb7fe640 1 -- 172.21.15.80:0/3074211994 <== mon.0 v2:172.21.15.80:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f67dc09ab60 con 0x563f57595240 2024-04-24T23:31:24.758 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.752+0000 7f67f4fc0740 10 client.15885 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:31:24.758 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.752+0000 7f67f4fc0740 10 client.15885 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:31:24.758 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.752+0000 7f67f4fc0740 20 client.15885 populate_metadata read hostname 'smithi080' 2024-04-24T23:31:24.758 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.752+0000 7f67f4fc0740 1 --2- 172.21.15.80:0/3074211994 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x563f57611ec0 0x563f576142e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:24.758 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.752+0000 7f67f4fc0740 4 client.15885 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:31:24.758 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.752+0000 7f67f4fc0740 1 -- 172.21.15.80:0/3074211994 --> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563f57500f40 con 0x563f57611ec0 2024-04-24T23:31:24.758 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.752+0000 7f67eb7fe640 1 --2- 172.21.15.80:0/3074211994 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x563f57611ec0 0x563f576142e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:24.759 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.752+0000 7f67cb7fe640 10 client.15885 ms_handle_connect on v2:172.21.15.148:6838/2952757594 2024-04-24T23:31:24.759 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.752+0000 7f67eb7fe640 1 --2- 172.21.15.80:0/3074211994 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x563f57611ec0 0x563f576142e0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4a0f0e891d069c5 server_cookie=ebd650b7d745413c in_seq=0 out_seq=0 2024-04-24T23:31:24.760 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.756+0000 7f67cb7fe640 1 -- 172.21.15.80:0/3074211994 <== mds.0 v2:172.21.15.148:6838/2952757594 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x563f57500f40 con 0x563f57611ec0 2024-04-24T23:31:24.760 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.756+0000 7f67cb7fe640 10 client.15885 handle_command_reply: tid=0 2024-04-24T23:31:24.760 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.756+0000 7f67f4fc0740 1 client.15885 shutdown 2024-04-24T23:31:24.760 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.756+0000 7f67f4fc0740 1 -- 172.21.15.80:0/3074211994 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563f57595240 msgr2=0x563f575a4100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:24.760 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.756+0000 7f67f4fc0740 1 --2- 172.21.15.80:0/3074211994 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563f57595240 0x563f575a4100 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f67dc0cd2b0 tx=0x7f67dc09c2a0 comp rx=0 tx=0).stop 2024-04-24T23:31:24.760 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.756+0000 7f67f4fc0740 1 -- 172.21.15.80:0/3074211994 shutdown_connections 2024-04-24T23:31:24.760 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.756+0000 7f67f4fc0740 1 --2- 172.21.15.80:0/3074211994 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x563f57611ec0 0x563f576142e0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:24.760 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.756+0000 7f67f4fc0740 1 --2- 172.21.15.80:0/3074211994 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f5759ada0 0x563f575a5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:24.760 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.756+0000 7f67cb7fe640 0 client.15885 ms_handle_reset on v2:172.21.15.148:6838/2952757594 2024-04-24T23:31:24.761 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.756+0000 7f67f4fc0740 1 --2- 172.21.15.80:0/3074211994 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563f57595c30 0x563f575a4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:24.761 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.756+0000 7f67f4fc0740 1 --2- 172.21.15.80:0/3074211994 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563f57595240 0x563f575a4100 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:24.761 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.756+0000 7f67f4fc0740 1 -- 172.21.15.80:0/3074211994 >> 172.21.15.80:0/3074211994 conn(0x563f575908b0 msgr2=0x563f5758f260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:24.761 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.756+0000 7f67f4fc0740 1 -- 172.21.15.80:0/3074211994 shutdown_connections 2024-04-24T23:31:24.761 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.756+0000 7f67f4fc0740 1 -- 172.21.15.80:0/3074211994 wait complete. 2024-04-24T23:31:24.761 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.756+0000 7f67f4fc0740 20 client.15885 trim_cache size 0 max 16384 2024-04-24T23:31:24.761 INFO:teuthology.orchestra.run.smithi080.stdout:{ 2024-04-24T23:31:24.761 INFO:teuthology.orchestra.run.smithi080.stdout: "status": "no active scrubs running", 2024-04-24T23:31:24.761 INFO:teuthology.orchestra.run.smithi080.stdout: "scrubs": {} 2024-04-24T23:31:24.761 INFO:teuthology.orchestra.run.smithi080.stdout:} 2024-04-24T23:31:24.762 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.757+0000 7f67f22b3640 1 -- 172.21.15.80:0/1401054113 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f67d40a2b90 msgr2=0x7f67d40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:24.762 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.757+0000 7f67f22b3640 1 --2- 172.21.15.80:0/1401054113 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f67d40a2b90 0x7f67d40a5050 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f67e4004af0 tx=0x7f67e404b000 comp rx=0 tx=0).stop 2024-04-24T23:31:24.762 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.757+0000 7f67f22b3640 1 -- 172.21.15.80:0/1401054113 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f67ec0ffa70 msgr2=0x7f67ec197a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:24.763 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.757+0000 7f67f22b3640 1 --2- 172.21.15.80:0/1401054113 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f67ec0ffa70 0x7f67ec197a50 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f67dc00e9c0 tx=0x7f67dc00ee80 comp rx=0 tx=0).stop 2024-04-24T23:31:24.763 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.757+0000 7f67f22b3640 1 -- 172.21.15.80:0/1401054113 shutdown_connections 2024-04-24T23:31:24.763 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.757+0000 7f67f22b3640 1 --2- 172.21.15.80:0/1401054113 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f67d40a2b90 0x7f67d40a5050 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:24.763 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.757+0000 7f67f22b3640 1 --2- 172.21.15.80:0/1401054113 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f67ec1034a0 0x7f67ec19f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:24.763 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.757+0000 7f67f22b3640 1 --2- 172.21.15.80:0/1401054113 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67ec102ae0 0x7f67ec197f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:24.763 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.757+0000 7f67f22b3640 1 --2- 172.21.15.80:0/1401054113 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f67ec0ffa70 0x7f67ec197a50 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:24.763 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.757+0000 7f67f22b3640 1 -- 172.21.15.80:0/1401054113 >> 172.21.15.80:0/1401054113 conn(0x7f67ec0fb380 msgr2=0x7f67ec105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:24.763 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.757+0000 7f67f22b3640 1 -- 172.21.15.80:0/1401054113 shutdown_connections 2024-04-24T23:31:24.763 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.757+0000 7f67f22b3640 1 -- 172.21.15.80:0/1401054113 wait complete. 2024-04-24T23:31:24.769 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T23:31:24.769 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:31:24.876 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.872+0000 7f9e939c1640 1 -- 172.21.15.80:0/3945787822 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9e8c100680 msgr2=0x7f9e8c100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:24.876 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.872+0000 7f9e939c1640 1 --2- 172.21.15.80:0/3945787822 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9e8c100680 0x7f9e8c100aa0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f9e88009900 tx=0x7f9e8802a410 comp rx=0 tx=0).stop 2024-04-24T23:31:24.876 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.872+0000 7f9e939c1640 1 -- 172.21.15.80:0/3945787822 shutdown_connections 2024-04-24T23:31:24.876 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.872+0000 7f9e939c1640 1 --2- 172.21.15.80:0/3945787822 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9e8c102230 0x7f9e8c108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:24.876 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.872+0000 7f9e939c1640 1 --2- 172.21.15.80:0/3945787822 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9e8c101870 0x7f9e8c101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:24.876 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.872+0000 7f9e939c1640 1 --2- 172.21.15.80:0/3945787822 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9e8c100680 0x7f9e8c100aa0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:24.877 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.872+0000 7f9e939c1640 1 -- 172.21.15.80:0/3945787822 >> 172.21.15.80:0/3945787822 conn(0x7f9e8c0fbbf0 msgr2=0x7f9e8c0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:24.877 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.872+0000 7f9e939c1640 1 -- 172.21.15.80:0/3945787822 shutdown_connections 2024-04-24T23:31:24.877 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.872+0000 7f9e939c1640 1 -- 172.21.15.80:0/3945787822 wait complete. 2024-04-24T23:31:24.877 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.872+0000 7f9e939c1640 1 Processor -- start 2024-04-24T23:31:24.877 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.872+0000 7f9e939c1640 1 -- start start 2024-04-24T23:31:24.877 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.873+0000 7f9e939c1640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9e8c100680 0x7f9e8c196a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:24.877 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.873+0000 7f9e939c1640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9e8c101870 0x7f9e8c196f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:24.877 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.873+0000 7f9e939c1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9e8c102230 0x7f9e8c19dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:24.877 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.873+0000 7f9e939c1640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f9e8c10c2d0 con 0x7f9e8c101870 2024-04-24T23:31:24.877 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.873+0000 7f9e939c1640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9e8c10c150 con 0x7f9e8c100680 2024-04-24T23:31:24.877 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.873+0000 7f9e939c1640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f9e8c10c450 con 0x7f9e8c102230 2024-04-24T23:31:24.877 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.873+0000 7f9e90f5c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9e8c101870 0x7f9e8c196f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:24.878 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.873+0000 7f9e90f5c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9e8c101870 0x7f9e8c196f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38210/0 (socket says 172.21.15.80:38210) 2024-04-24T23:31:24.878 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.873+0000 7f9e90f5c640 1 -- 172.21.15.80:0/3244216091 learned_addr learned my addr 172.21.15.80:0/3244216091 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:31:24.878 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.873+0000 7f9e91f5e640 1 --2- 172.21.15.80:0/3244216091 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9e8c102230 0x7f9e8c19dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:24.878 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.873+0000 7f9e9175d640 1 --2- 172.21.15.80:0/3244216091 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9e8c100680 0x7f9e8c196a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:24.878 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.873+0000 7f9e90f5c640 1 -- 172.21.15.80:0/3244216091 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9e8c102230 msgr2=0x7f9e8c19dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:24.878 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.873+0000 7f9e90f5c640 1 --2- 172.21.15.80:0/3244216091 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9e8c102230 0x7f9e8c19dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:24.878 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.873+0000 7f9e90f5c640 1 -- 172.21.15.80:0/3244216091 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9e8c100680 msgr2=0x7f9e8c196a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:24.878 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.873+0000 7f9e90f5c640 1 --2- 172.21.15.80:0/3244216091 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9e8c100680 0x7f9e8c196a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:24.878 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.873+0000 7f9e90f5c640 1 -- 172.21.15.80:0/3244216091 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e8c19e620 con 0x7f9e8c101870 2024-04-24T23:31:24.878 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.873+0000 7f9e9175d640 1 --2- 172.21.15.80:0/3244216091 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9e8c100680 0x7f9e8c196a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:31:24.878 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.873+0000 7f9e90f5c640 1 --2- 172.21.15.80:0/3244216091 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9e8c101870 0x7f9e8c196f40 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f9e8000d950 tx=0x7f9e8000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:24.878 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.873+0000 7f9e7e7fc640 1 -- 172.21.15.80:0/3244216091 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e80017070 con 0x7f9e8c101870 2024-04-24T23:31:24.878 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.873+0000 7f9e939c1640 1 -- 172.21.15.80:0/3244216091 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e8c19e910 con 0x7f9e8c101870 2024-04-24T23:31:24.879 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.873+0000 7f9e7e7fc640 1 -- 172.21.15.80:0/3244216091 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e80009c30 con 0x7f9e8c101870 2024-04-24T23:31:24.879 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.873+0000 7f9e7e7fc640 1 -- 172.21.15.80:0/3244216091 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e8000f6b0 con 0x7f9e8c101870 2024-04-24T23:31:24.879 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.873+0000 7f9e939c1640 1 -- 172.21.15.80:0/3244216091 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e8c19ee60 con 0x7f9e8c101870 2024-04-24T23:31:24.879 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.874+0000 7f9e939c1640 1 -- 172.21.15.80:0/3244216091 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e8c107a90 con 0x7f9e8c101870 2024-04-24T23:31:24.879 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.875+0000 7f9e7e7fc640 1 -- 172.21.15.80:0/3244216091 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9e800109d0 con 0x7f9e8c101870 2024-04-24T23:31:24.880 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.876+0000 7f9e7e7fc640 1 --2- 172.21.15.80:0/3244216091 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f9e780a2b90 0x7f9e780a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:24.880 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.876+0000 7f9e9175d640 1 --2- 172.21.15.80:0/3244216091 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f9e780a2b90 0x7f9e780a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:24.880 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.876+0000 7f9e7e7fc640 1 -- 172.21.15.80:0/3244216091 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f9e800cf080 con 0x7f9e8c101870 2024-04-24T23:31:24.880 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.876+0000 7f9e9175d640 1 --2- 172.21.15.80:0/3244216091 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f9e780a2b90 0x7f9e780a5050 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f9e88006010 tx=0x7f9e88032cf0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:24.881 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:24.881+0000 7f9e7e7fc640 1 -- 172.21.15.80:0/3244216091 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f9e80009de0 con 0x7f9e8c101870 2024-04-24T23:31:25.111 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.110+0000 7f9e939c1640 1 -- 172.21.15.80:0/3244216091 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9e8c02d170 con 0x7f9e8c101870 2024-04-24T23:31:25.111 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.111+0000 7f9e7e7fc640 1 -- 172.21.15.80:0/3244216091 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11875 (secure 0 0 0) 0x7f9e8009bc10 con 0x7f9e8c101870 2024-04-24T23:31:25.112 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-24T23:31:25.112 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/891591085","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":891591085},{"type":"v1","addr":"172.21.15.80:6835","nonce":891591085}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6837/259529026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6836","nonce":259529026},{"type":"v1","addr":"172.21.15.80:6837","nonce":259529026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14646,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6839/4194284854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6838","nonce":4194284854},{"type":"v1","addr":"172.21.15.80:6839","nonce":4194284854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14658,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6841/3145730958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6840","nonce":3145730958},{"type":"v1","addr":"172.21.15.80:6841","nonce":3145730958}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24395,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/3736784385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3736784385},{"type":"v1","addr":"172.21.15.134:6833","nonce":3736784385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/124777462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":124777462},{"type":"v1","addr":"172.21.15.148:6833","nonce":124777462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T23:27:19.300566+0000","modified":"2024-04-24T23:31:19.604599+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24496,"mds_1":24488,"mds_2":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/1399801120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1399801120},{"type":"v1","addr":"172.21.15.134:6835","nonce":1399801120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/430512704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":430512704},{"type":"v1","addr":"172.21.15.134:6837","nonce":430512704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/3002319310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3002319310},{"type":"v1","addr":"172.21.15.148:6835","nonce":3002319310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.148:6837/856528403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":856528403},{"type":"v1","addr":"172.21.15.148:6837","nonce":856528403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6839/1743815021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":1743815021},{"type":"v1","addr":"172.21.15.134:6839","nonce":1743815021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.148:6839/2952757594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":2952757594},{"type":"v1","addr":"172.21.15.148:6839","nonce":2952757594}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24496,"qdb_cluster":[24472,24488,24496]},"id":1}]} 2024-04-24T23:31:25.112 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 27 2024-04-24T23:31:25.114 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.113+0000 7f9e939c1640 1 -- 172.21.15.80:0/3244216091 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f9e780a2b90 msgr2=0x7f9e780a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:25.114 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.113+0000 7f9e939c1640 1 --2- 172.21.15.80:0/3244216091 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f9e780a2b90 0x7f9e780a5050 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f9e88006010 tx=0x7f9e88032cf0 comp rx=0 tx=0).stop 2024-04-24T23:31:25.115 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.113+0000 7f9e939c1640 1 -- 172.21.15.80:0/3244216091 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9e8c101870 msgr2=0x7f9e8c196f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:25.115 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.113+0000 7f9e939c1640 1 --2- 172.21.15.80:0/3244216091 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9e8c101870 0x7f9e8c196f40 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f9e8000d950 tx=0x7f9e8000de10 comp rx=0 tx=0).stop 2024-04-24T23:31:25.115 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.113+0000 7f9e939c1640 1 -- 172.21.15.80:0/3244216091 shutdown_connections 2024-04-24T23:31:25.115 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.113+0000 7f9e939c1640 1 --2- 172.21.15.80:0/3244216091 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f9e780a2b90 0x7f9e780a5050 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:25.115 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.113+0000 7f9e939c1640 1 --2- 172.21.15.80:0/3244216091 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9e8c102230 0x7f9e8c19dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:25.115 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.113+0000 7f9e939c1640 1 --2- 172.21.15.80:0/3244216091 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9e8c101870 0x7f9e8c196f40 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:25.115 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.113+0000 7f9e939c1640 1 --2- 172.21.15.80:0/3244216091 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9e8c100680 0x7f9e8c196a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:25.115 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.113+0000 7f9e939c1640 1 -- 172.21.15.80:0/3244216091 >> 172.21.15.80:0/3244216091 conn(0x7f9e8c0fbbf0 msgr2=0x7f9e8c104c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:25.116 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.113+0000 7f9e939c1640 1 -- 172.21.15.80:0/3244216091 shutdown_connections 2024-04-24T23:31:25.116 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.113+0000 7f9e939c1640 1 -- 172.21.15.80:0/3244216091 wait complete. 2024-04-24T23:31:25.127 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T23:31:25.235 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.232+0000 7f910c59e640 1 -- 172.21.15.80:0/3217180449 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9104102200 msgr2=0x7f91041045f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:25.235 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.232+0000 7f910c59e640 1 --2- 172.21.15.80:0/3217180449 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9104102200 0x7f91041045f0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f9100009900 tx=0x7f910002a410 comp rx=0 tx=0).stop 2024-04-24T23:31:25.235 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.232+0000 7f910c59e640 1 -- 172.21.15.80:0/3217180449 shutdown_connections 2024-04-24T23:31:25.235 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.232+0000 7f910c59e640 1 --2- 172.21.15.80:0/3217180449 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9104107460 0x7f91041098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:25.236 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.232+0000 7f910c59e640 1 --2- 172.21.15.80:0/3217180449 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9104104b30 0x7f9104106f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:25.236 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.232+0000 7f910c59e640 1 --2- 172.21.15.80:0/3217180449 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9104102200 0x7f91041045f0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:25.236 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.232+0000 7f910c59e640 1 -- 172.21.15.80:0/3217180449 >> 172.21.15.80:0/3217180449 conn(0x7f91040fbbf0 msgr2=0x7f91040fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:25.236 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.232+0000 7f910c59e640 1 -- 172.21.15.80:0/3217180449 shutdown_connections 2024-04-24T23:31:25.236 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.233+0000 7f910c59e640 1 -- 172.21.15.80:0/3217180449 wait complete. 2024-04-24T23:31:25.236 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.233+0000 7f910c59e640 1 Processor -- start 2024-04-24T23:31:25.236 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.233+0000 7f910c59e640 1 -- start start 2024-04-24T23:31:25.236 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.233+0000 7f910c59e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9104102200 0x7f9104101b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:25.236 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.233+0000 7f910c59e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9104104b30 0x7f91041001b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:25.236 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.233+0000 7f910c59e640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9104107460 0x7f91041006f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:25.236 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.233+0000 7f910c59e640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f910410d430 con 0x7f9104107460 2024-04-24T23:31:25.236 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.233+0000 7f910c59e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f910410d2b0 con 0x7f9104102200 2024-04-24T23:31:25.236 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.233+0000 7f910c59e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f910410d5b0 con 0x7f9104104b30 2024-04-24T23:31:25.236 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.233+0000 7f9109b39640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9104104b30 0x7f91041001b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:25.237 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.233+0000 7f910ab3b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9104107460 0x7f91041006f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:25.237 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.233+0000 7f910a33a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9104102200 0x7f9104101b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:25.237 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.233+0000 7f9109b39640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9104104b30 0x7f91041001b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.80:38568/0 (socket says 172.21.15.80:38568) 2024-04-24T23:31:25.237 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.233+0000 7f9109b39640 1 -- 172.21.15.80:0/2561034746 learned_addr learned my addr 172.21.15.80:0/2561034746 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:31:25.237 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.233+0000 7f910a33a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9104102200 0x7f9104101b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.80:56308/0 (socket says 172.21.15.80:56308) 2024-04-24T23:31:25.237 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.233+0000 7f910ab3b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9104107460 0x7f91041006f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38234/0 (socket says 172.21.15.80:38234) 2024-04-24T23:31:25.237 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.234+0000 7f9109b39640 1 -- 172.21.15.80:0/2561034746 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9104102200 msgr2=0x7f9104101b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:25.237 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.234+0000 7f9109b39640 1 --2- 172.21.15.80:0/2561034746 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9104102200 0x7f9104101b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:25.237 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.234+0000 7f9109b39640 1 -- 172.21.15.80:0/2561034746 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9104107460 msgr2=0x7f91041006f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:25.237 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.234+0000 7f9109b39640 1 --2- 172.21.15.80:0/2561034746 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9104107460 0x7f91041006f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:25.237 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.234+0000 7f9109b39640 1 -- 172.21.15.80:0/2561034746 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9104100f80 con 0x7f9104104b30 2024-04-24T23:31:25.237 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.234+0000 7f910a33a640 1 --2- 172.21.15.80:0/2561034746 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9104102200 0x7f9104101b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:31:25.238 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.234+0000 7f910ab3b640 1 --2- 172.21.15.80:0/2561034746 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9104107460 0x7f91041006f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:31:25.238 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.234+0000 7f9109b39640 1 --2- 172.21.15.80:0/2561034746 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9104104b30 0x7f91041001b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f90f400e9f0 tx=0x7f90f400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:25.238 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.234+0000 7f90fb7fe640 1 -- 172.21.15.80:0/2561034746 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f400ccc0 con 0x7f9104104b30 2024-04-24T23:31:25.238 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.234+0000 7f910c59e640 1 -- 172.21.15.80:0/2561034746 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91041a1c60 con 0x7f9104104b30 2024-04-24T23:31:25.238 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.234+0000 7f90fb7fe640 1 -- 172.21.15.80:0/2561034746 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90f4005500 con 0x7f9104104b30 2024-04-24T23:31:25.238 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.234+0000 7f910c59e640 1 -- 172.21.15.80:0/2561034746 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91041a21b0 con 0x7f9104104b30 2024-04-24T23:31:25.238 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.234+0000 7f90fb7fe640 1 -- 172.21.15.80:0/2561034746 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f4018630 con 0x7f9104104b30 2024-04-24T23:31:25.244 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.236+0000 7f90fb7fe640 1 -- 172.21.15.80:0/2561034746 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f90f40046c0 con 0x7f9104104b30 2024-04-24T23:31:25.244 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.237+0000 7f910f385740 1 Processor -- start 2024-04-24T23:31:25.244 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.237+0000 7f910f385740 1 -- start start 2024-04-24T23:31:25.244 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.237+0000 7f910f385740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557498d2c230 0x557498d2c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:25.244 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.237+0000 7f910f385740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557498d2cc20 0x557498d31720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:25.244 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.237+0000 7f910f385740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557498d31d90 0x557498d34220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:25.244 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.237+0000 7f910f385740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x557498c90670 con 0x557498d2cc20 2024-04-24T23:31:25.244 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.237+0000 7f910f385740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x557498c38970 con 0x557498d2c230 2024-04-24T23:31:25.244 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.237+0000 7f910f385740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x557498c8c630 con 0x557498d31d90 2024-04-24T23:31:25.244 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.237+0000 7f910a33a640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557498d2cc20 0x557498d31720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:25.244 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.237+0000 7f90fb7fe640 1 --2- 172.21.15.80:0/2561034746 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f90f00a2b90 0x7f90f00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:25.244 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.237+0000 7f910a33a640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557498d2cc20 0x557498d31720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38236/0 (socket says 172.21.15.80:38236) 2024-04-24T23:31:25.244 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.237+0000 7f910a33a640 1 -- 172.21.15.80:0/2794256164 learned_addr learned my addr 172.21.15.80:0/2794256164 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:31:25.245 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.237+0000 7f9109b39640 1 --2- 172.21.15.80:0/2794256164 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557498d31d90 0x557498d34220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:25.245 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.237+0000 7f910ab3b640 1 --2- 172.21.15.80:0/2794256164 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557498d2c230 0x557498d2c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:25.245 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.237+0000 7f910ab3b640 1 --2- 172.21.15.80:0/2561034746 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f90f00a2b90 0x7f90f00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:25.245 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.237+0000 7f90fb7fe640 1 -- 172.21.15.80:0/2561034746 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f90f40cf900 con 0x7f9104104b30 2024-04-24T23:31:25.245 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.238+0000 7f910a33a640 1 -- 172.21.15.80:0/2794256164 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557498d31d90 msgr2=0x557498d34220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:25.245 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.238+0000 7f910a33a640 1 --2- 172.21.15.80:0/2794256164 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557498d31d90 0x557498d34220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:25.245 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.238+0000 7f910a33a640 1 -- 172.21.15.80:0/2794256164 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557498d2c230 msgr2=0x557498d2c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:25.245 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.238+0000 7f910a33a640 1 --2- 172.21.15.80:0/2794256164 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557498d2c230 0x557498d2c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:25.245 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.238+0000 7f910a33a640 1 -- 172.21.15.80:0/2794256164 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557498c5c400 con 0x557498d2cc20 2024-04-24T23:31:25.245 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.238+0000 7f910a33a640 1 --2- 172.21.15.80:0/2794256164 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557498d2cc20 0x557498d31720 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f910002a930 tx=0x7f9100009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abdd56f8f17f4c75 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:25.245 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.238+0000 7f910ab3b640 1 --2- 172.21.15.80:0/2561034746 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f90f00a2b90 0x7f90f00a5050 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f90fc0099e0 tx=0x7f90fc009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:25.245 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.238+0000 7f90f8bfe640 1 -- 172.21.15.80:0/2794256164 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9100040070 con 0x557498d2cc20 2024-04-24T23:31:25.245 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.238+0000 7f90f8bfe640 1 -- 172.21.15.80:0/2794256164 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91000319b0 con 0x557498d2cc20 2024-04-24T23:31:25.245 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.238+0000 7f90f8bfe640 1 -- 172.21.15.80:0/2794256164 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9100035040 con 0x557498d2cc20 2024-04-24T23:31:25.246 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.239+0000 7f910f385740 1 -- 172.21.15.80:0/2794256164 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557498d2cc20 msgr2=0x557498d31720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:25.246 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.239+0000 7f910f385740 1 --2- 172.21.15.80:0/2794256164 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557498d2cc20 0x557498d31720 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f910002a930 tx=0x7f9100009680 comp rx=0 tx=0).stop 2024-04-24T23:31:25.246 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.239+0000 7f910f385740 1 -- 172.21.15.80:0/2794256164 shutdown_connections 2024-04-24T23:31:25.246 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.239+0000 7f910f385740 1 --2- 172.21.15.80:0/2794256164 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557498d31d90 0x557498d34220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:25.246 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.239+0000 7f910f385740 1 --2- 172.21.15.80:0/2794256164 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557498d2cc20 0x557498d31720 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:25.246 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.239+0000 7f910f385740 1 --2- 172.21.15.80:0/2794256164 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557498d2c230 0x557498d2c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:25.246 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.239+0000 7f910f385740 1 -- 172.21.15.80:0/2794256164 >> 172.21.15.80:0/2794256164 conn(0x557498d278a0 msgr2=0x557498d27ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:25.246 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.240+0000 7f910f385740 1 -- 172.21.15.80:0/2794256164 shutdown_connections 2024-04-24T23:31:25.246 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.240+0000 7f910f385740 1 -- 172.21.15.80:0/2794256164 wait complete. 2024-04-24T23:31:25.246 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.240+0000 7f910f385740 1 -- start start 2024-04-24T23:31:25.246 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.240+0000 7f910f385740 1 Processor -- start 2024-04-24T23:31:25.246 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.241+0000 7f910f385740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557498d2c230 0x557498d3eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:25.246 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.241+0000 7f910f385740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557498d2cc20 0x557498d3f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:25.247 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.241+0000 7f910f385740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557498d31d90 0x557498d460e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:25.247 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.241+0000 7f910f385740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x557498d34ce0 con 0x557498d2cc20 2024-04-24T23:31:25.247 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.241+0000 7f910f385740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x557498c4c050 con 0x557498d2c230 2024-04-24T23:31:25.247 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.241+0000 7f910f385740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x557498d34e60 con 0x557498d31d90 2024-04-24T23:31:25.247 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.241+0000 7f910f385740 10 client.0 _ll_register_callbacks cb 0x5574989a3be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:31:25.247 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.241+0000 7f9109b39640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557498d2cc20 0x557498d3f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:25.247 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.241+0000 7f9109b39640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557498d2cc20 0x557498d3f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38252/0 (socket says 172.21.15.80:38252) 2024-04-24T23:31:25.247 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.241+0000 7f9109b39640 1 -- 172.21.15.80:0/2254503562 learned_addr learned my addr 172.21.15.80:0/2254503562 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:31:25.247 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.241+0000 7f910ab3b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557498d31d90 0x557498d460e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:25.247 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.241+0000 7f910a33a640 1 --2- 172.21.15.80:0/2254503562 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557498d2c230 0x557498d3eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:25.247 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.241+0000 7f9109b39640 1 -- 172.21.15.80:0/2254503562 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557498d31d90 msgr2=0x557498d460e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:25.247 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.241+0000 7f9109b39640 1 --2- 172.21.15.80:0/2254503562 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557498d31d90 0x557498d460e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:25.247 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.241+0000 7f9109b39640 1 -- 172.21.15.80:0/2254503562 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557498d2c230 msgr2=0x557498d3eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:25.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.241+0000 7f9109b39640 1 --2- 172.21.15.80:0/2254503562 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557498d2c230 0x557498d3eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:25.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.241+0000 7f9109b39640 1 -- 172.21.15.80:0/2254503562 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557498c1ddc0 con 0x557498d2cc20 2024-04-24T23:31:25.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.241+0000 7f9109b39640 1 --2- 172.21.15.80:0/2254503562 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557498d2cc20 0x557498d3f0a0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f90f409bd20 tx=0x7f90f406ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36d913b1efd48303 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:25.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.241+0000 7f90edffb640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-04-24T23:31:25.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.242+0000 7f90edffb640 1 -- 172.21.15.80:0/2254503562 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f409b6f0 con 0x557498d2cc20 2024-04-24T23:31:25.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.242+0000 7f90edffb640 1 -- 172.21.15.80:0/2254503562 <== mon.0 v2:172.21.15.80:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f90f409b3e0 con 0x557498d2cc20 2024-04-24T23:31:25.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.242+0000 7f910f385740 1 -- 172.21.15.80:0/2254503562 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f91000319b0 con 0x557498d2cc20 2024-04-24T23:31:25.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.242+0000 7f90edffb640 1 -- 172.21.15.80:0/2254503562 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f90f409c490 con 0x557498d2cc20 2024-04-24T23:31:25.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.242+0000 7f910f385740 10 client.15903 fetch_fsmap learned FSMap version 27 2024-04-24T23:31:25.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.242+0000 7f910f385740 1 -- 172.21.15.80:0/2254503562 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5574989a43f0 con 0x557498d2cc20 2024-04-24T23:31:25.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.242+0000 7f90edffb640 1 -- 172.21.15.80:0/2254503562 <== mon.0 v2:172.21.15.80:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91000319b0 con 0x557498d2cc20 2024-04-24T23:31:25.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.242+0000 7f90edffb640 1 -- 172.21.15.80:0/2254503562 <== mon.0 v2:172.21.15.80:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f90f40a12d0 con 0x557498d2cc20 2024-04-24T23:31:25.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.242+0000 7f910f385740 10 client.15903 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:31:25.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.242+0000 7f910f385740 10 client.15903 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-24T23:31:25.249 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.242+0000 7f910f385740 20 client.15903 populate_metadata read hostname 'smithi080' 2024-04-24T23:31:25.249 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.242+0000 7f910f385740 1 --2- 172.21.15.80:0/2254503562 >> [v2:172.21.15.148:6836/856528403,v1:172.21.15.148:6837/856528403] conn(0x557498d3a380 0x557498d3c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:25.249 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.242+0000 7f910f385740 4 client.15903 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:31:25.249 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.242+0000 7f910f385740 1 -- 172.21.15.80:0/2254503562 --> [v2:172.21.15.148:6836/856528403,v1:172.21.15.148:6837/856528403] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557498c9dbb0 con 0x557498d3a380 2024-04-24T23:31:25.249 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.242+0000 7f910a33a640 1 --2- 172.21.15.80:0/2254503562 >> [v2:172.21.15.148:6836/856528403,v1:172.21.15.148:6837/856528403] conn(0x557498d3a380 0x557498d3c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:25.249 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.243+0000 7f90edffb640 10 client.15903 ms_handle_connect on v2:172.21.15.148:6836/856528403 2024-04-24T23:31:25.249 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.243+0000 7f910a33a640 1 --2- 172.21.15.80:0/2254503562 >> [v2:172.21.15.148:6836/856528403,v1:172.21.15.148:6837/856528403] conn(0x557498d3a380 0x557498d3c810 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d0a871da82f71f2e server_cookie=edb977fea007b10c in_seq=0 out_seq=0 2024-04-24T23:31:25.250 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.244+0000 7f90edffb640 1 -- 172.21.15.80:0/2254503562 <== mds.2 v2:172.21.15.148:6836/856528403 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557498c9dbb0 con 0x557498d3a380 2024-04-24T23:31:25.250 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.244+0000 7f90edffb640 10 client.15903 handle_command_reply: tid=0 2024-04-24T23:31:25.250 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.244+0000 7f910f385740 1 client.15903 shutdown 2024-04-24T23:31:25.250 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.244+0000 7f910f385740 1 -- 172.21.15.80:0/2254503562 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557498d2cc20 msgr2=0x557498d3f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:25.250 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.244+0000 7f910f385740 1 --2- 172.21.15.80:0/2254503562 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557498d2cc20 0x557498d3f0a0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f90f409bd20 tx=0x7f90f406ad10 comp rx=0 tx=0).stop 2024-04-24T23:31:25.251 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.244+0000 7f910f385740 1 -- 172.21.15.80:0/2254503562 shutdown_connections 2024-04-24T23:31:25.251 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.244+0000 7f910f385740 1 --2- 172.21.15.80:0/2254503562 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557498d31d90 0x557498d460e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:25.251 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.244+0000 7f910f385740 1 --2- 172.21.15.80:0/2254503562 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557498d2cc20 0x557498d3f0a0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:25.251 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.244+0000 7f910f385740 1 --2- 172.21.15.80:0/2254503562 >> [v2:172.21.15.148:6836/856528403,v1:172.21.15.148:6837/856528403] conn(0x557498d3a380 0x557498d3c810 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:25.251 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.244+0000 7f910f385740 1 --2- 172.21.15.80:0/2254503562 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557498d2c230 0x557498d3eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:25.251 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.244+0000 7f90edffb640 0 client.15903 ms_handle_reset on v2:172.21.15.148:6836/856528403 2024-04-24T23:31:25.251 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.244+0000 7f910f385740 1 -- 172.21.15.80:0/2254503562 >> 172.21.15.80:0/2254503562 conn(0x557498d278a0 msgr2=0x557498d26b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:25.251 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.244+0000 7f910f385740 1 -- 172.21.15.80:0/2254503562 shutdown_connections 2024-04-24T23:31:25.251 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.244+0000 7f910f385740 1 -- 172.21.15.80:0/2254503562 wait complete. 2024-04-24T23:31:25.251 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.244+0000 7f910f385740 20 client.15903 trim_cache size 0 max 16384 2024-04-24T23:31:25.265 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.258+0000 7f910f385740 1 Processor -- start 2024-04-24T23:31:25.265 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.259+0000 7f910f385740 1 -- start start 2024-04-24T23:31:25.265 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.259+0000 7f910f385740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557498d2c230 0x557498d2a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:25.265 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.259+0000 7f910f385740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557498d2cc20 0x557498d460a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:25.265 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.259+0000 7f910f385740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557498d31d90 0x557498d2acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:25.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.259+0000 7f910f385740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f91041aac50 con 0x557498d2c230 2024-04-24T23:31:25.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.259+0000 7f910f385740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f91041a8520 con 0x557498d2cc20 2024-04-24T23:31:25.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.259+0000 7f910f385740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f91041a6370 con 0x557498d31d90 2024-04-24T23:31:25.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.259+0000 7f910a33a640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557498d2c230 0x557498d2a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:25.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.259+0000 7f910a33a640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557498d2c230 0x557498d2a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38268/0 (socket says 172.21.15.80:38268) 2024-04-24T23:31:25.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.259+0000 7f910a33a640 1 -- 172.21.15.80:0/2813241155 learned_addr learned my addr 172.21.15.80:0/2813241155 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:31:25.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.259+0000 7f910ab3b640 1 --2- 172.21.15.80:0/2813241155 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557498d31d90 0x557498d2acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:25.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.259+0000 7f9109b39640 1 --2- 172.21.15.80:0/2813241155 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557498d2cc20 0x557498d460a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:25.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.259+0000 7f910a33a640 1 -- 172.21.15.80:0/2813241155 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557498d31d90 msgr2=0x557498d2acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:25.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.259+0000 7f910a33a640 1 --2- 172.21.15.80:0/2813241155 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557498d31d90 0x557498d2acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:25.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.259+0000 7f910a33a640 1 -- 172.21.15.80:0/2813241155 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557498d2cc20 msgr2=0x557498d460a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:25.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.259+0000 7f910a33a640 1 --2- 172.21.15.80:0/2813241155 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557498d2cc20 0x557498d460a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:25.267 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.259+0000 7f910a33a640 1 -- 172.21.15.80:0/2813241155 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557498a6e7d0 con 0x557498d2c230 2024-04-24T23:31:25.267 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.259+0000 7f9109b39640 1 --2- 172.21.15.80:0/2813241155 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557498d2cc20 0x557498d460a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:31:25.267 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.259+0000 7f910a33a640 1 --2- 172.21.15.80:0/2813241155 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557498d2c230 0x557498d2a7b0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f9100005ab0 tx=0x7f9100009610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f27b6eb9416684d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:25.267 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.259+0000 7f90effff640 1 -- 172.21.15.80:0/2813241155 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9100052070 con 0x557498d2c230 2024-04-24T23:31:25.267 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.259+0000 7f90effff640 1 -- 172.21.15.80:0/2813241155 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f910004d8e0 con 0x557498d2c230 2024-04-24T23:31:25.267 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.259+0000 7f90effff640 1 -- 172.21.15.80:0/2813241155 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9100035040 con 0x557498d2c230 2024-04-24T23:31:25.267 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.260+0000 7f910f385740 1 -- 172.21.15.80:0/2813241155 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557498d2c230 msgr2=0x557498d2a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:25.267 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.260+0000 7f910f385740 1 --2- 172.21.15.80:0/2813241155 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557498d2c230 0x557498d2a7b0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f9100005ab0 tx=0x7f9100009610 comp rx=0 tx=0).stop 2024-04-24T23:31:25.267 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.260+0000 7f910f385740 1 -- 172.21.15.80:0/2813241155 shutdown_connections 2024-04-24T23:31:25.267 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.260+0000 7f910f385740 1 --2- 172.21.15.80:0/2813241155 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557498d31d90 0x557498d2acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:25.267 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.260+0000 7f910f385740 1 --2- 172.21.15.80:0/2813241155 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557498d2cc20 0x557498d460a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:25.268 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.260+0000 7f910f385740 1 --2- 172.21.15.80:0/2813241155 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557498d2c230 0x557498d2a7b0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:25.268 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.260+0000 7f910f385740 1 -- 172.21.15.80:0/2813241155 >> 172.21.15.80:0/2813241155 conn(0x557498d278a0 msgr2=0x557498d26b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:25.268 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.260+0000 7f910f385740 1 -- 172.21.15.80:0/2813241155 shutdown_connections 2024-04-24T23:31:25.268 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.260+0000 7f910f385740 1 -- 172.21.15.80:0/2813241155 wait complete. 2024-04-24T23:31:25.268 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.261+0000 7f910f385740 1 -- start start 2024-04-24T23:31:25.268 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.261+0000 7f910f385740 1 Processor -- start 2024-04-24T23:31:25.268 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.261+0000 7f910f385740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557498d2c230 0x557498d41fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:25.268 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.261+0000 7f910f385740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557498d2cc20 0x557498d424e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:25.268 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.261+0000 7f910f385740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557498d31d90 0x557498d45500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:25.268 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.261+0000 7f910f385740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f91041a69e0 con 0x557498d31d90 2024-04-24T23:31:25.268 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.261+0000 7f910f385740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f91041a7e50 con 0x557498d2c230 2024-04-24T23:31:25.268 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.261+0000 7f910f385740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f91041a70b0 con 0x557498d2cc20 2024-04-24T23:31:25.268 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.261+0000 7f910f385740 10 client.0 _ll_register_callbacks cb 0x7f90f4004aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:31:25.269 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.261+0000 7f9109b39640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557498d2cc20 0x557498d424e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:25.269 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.261+0000 7f910ab3b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557498d31d90 0x557498d45500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:25.269 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.261+0000 7f910ab3b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557498d31d90 0x557498d45500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38272/0 (socket says 172.21.15.80:38272) 2024-04-24T23:31:25.269 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.261+0000 7f910ab3b640 1 -- 172.21.15.80:0/3774477182 learned_addr learned my addr 172.21.15.80:0/3774477182 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:31:25.269 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.261+0000 7f910a33a640 1 --2- 172.21.15.80:0/3774477182 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557498d2c230 0x557498d41fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:25.269 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.261+0000 7f910ab3b640 1 -- 172.21.15.80:0/3774477182 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557498d2cc20 msgr2=0x557498d424e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:25.269 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.261+0000 7f910ab3b640 1 --2- 172.21.15.80:0/3774477182 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557498d2cc20 0x557498d424e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:25.269 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.261+0000 7f910ab3b640 1 -- 172.21.15.80:0/3774477182 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557498d2c230 msgr2=0x557498d41fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:25.269 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.261+0000 7f910ab3b640 1 --2- 172.21.15.80:0/3774477182 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557498d2c230 0x557498d41fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:25.269 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.261+0000 7f910ab3b640 1 -- 172.21.15.80:0/3774477182 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557498d45af0 con 0x557498d31d90 2024-04-24T23:31:25.269 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.261+0000 7f910ab3b640 1 --2- 172.21.15.80:0/3774477182 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557498d31d90 0x557498d45500 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f90fc003e90 tx=0x7f90fc004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7903fac6847bd541 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:25.269 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.261+0000 7f90edffb640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-04-24T23:31:25.270 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.262+0000 7f90edffb640 1 -- 172.21.15.80:0/3774477182 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90fc011220 con 0x557498d31d90 2024-04-24T23:31:25.270 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.262+0000 7f90edffb640 1 -- 172.21.15.80:0/3774477182 <== mon.0 v2:172.21.15.80:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f90fc004b20 con 0x557498d31d90 2024-04-24T23:31:25.270 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.262+0000 7f910f385740 1 -- 172.21.15.80:0/3774477182 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f910004d8e0 con 0x557498d31d90 2024-04-24T23:31:25.270 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.262+0000 7f90edffb640 1 -- 172.21.15.80:0/3774477182 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f90fc035050 con 0x557498d31d90 2024-04-24T23:31:25.270 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.262+0000 7f910f385740 10 client.15909 fetch_fsmap learned FSMap version 27 2024-04-24T23:31:25.270 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.262+0000 7f910f385740 1 -- 172.21.15.80:0/3774477182 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557498db9750 con 0x557498d31d90 2024-04-24T23:31:25.270 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.262+0000 7f90edffb640 1 -- 172.21.15.80:0/3774477182 <== mon.0 v2:172.21.15.80:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f910004d8e0 con 0x557498d31d90 2024-04-24T23:31:25.270 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.262+0000 7f90edffb640 1 -- 172.21.15.80:0/3774477182 <== mon.0 v2:172.21.15.80:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f90fc01d040 con 0x557498d31d90 2024-04-24T23:31:25.270 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.262+0000 7f910f385740 10 client.15909 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:31:25.270 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.262+0000 7f910f385740 10 client.15909 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-24T23:31:25.270 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.262+0000 7f910f385740 20 client.15909 populate_metadata read hostname 'smithi080' 2024-04-24T23:31:25.270 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.262+0000 7f910f385740 1 --2- 172.21.15.80:0/3774477182 >> [v2:172.21.15.148:6836/856528403,v1:172.21.15.148:6837/856528403] conn(0x557498dba200 0x557498dbc620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:25.270 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.262+0000 7f910f385740 4 client.15909 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:31:25.270 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.262+0000 7f910f385740 1 -- 172.21.15.80:0/3774477182 --> [v2:172.21.15.148:6836/856528403,v1:172.21.15.148:6837/856528403] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557498c97f00 con 0x557498dba200 2024-04-24T23:31:25.271 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.262+0000 7f910a33a640 1 --2- 172.21.15.80:0/3774477182 >> [v2:172.21.15.148:6836/856528403,v1:172.21.15.148:6837/856528403] conn(0x557498dba200 0x557498dbc620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:25.271 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.263+0000 7f90edffb640 10 client.15909 ms_handle_connect on v2:172.21.15.148:6836/856528403 2024-04-24T23:31:25.271 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.263+0000 7f910a33a640 1 --2- 172.21.15.80:0/3774477182 >> [v2:172.21.15.148:6836/856528403,v1:172.21.15.148:6837/856528403] conn(0x557498dba200 0x557498dbc620 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1ca90f404bacc932 server_cookie=1d55a5954e7dbd75 in_seq=0 out_seq=0 2024-04-24T23:31:25.271 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.263+0000 7f90edffb640 1 -- 172.21.15.80:0/3774477182 <== mds.2 v2:172.21.15.148:6836/856528403 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557498c97f00 con 0x557498dba200 2024-04-24T23:31:25.271 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.263+0000 7f90edffb640 10 client.15909 handle_command_reply: tid=0 2024-04-24T23:31:25.271 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.263+0000 7f910f385740 1 client.15909 shutdown 2024-04-24T23:31:25.271 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.263+0000 7f910f385740 1 -- 172.21.15.80:0/3774477182 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557498d31d90 msgr2=0x557498d45500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:25.271 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.263+0000 7f910f385740 1 --2- 172.21.15.80:0/3774477182 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557498d31d90 0x557498d45500 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f90fc003e90 tx=0x7f90fc004ab0 comp rx=0 tx=0).stop 2024-04-24T23:31:25.271 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.263+0000 7f910f385740 1 -- 172.21.15.80:0/3774477182 shutdown_connections 2024-04-24T23:31:25.271 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.263+0000 7f910f385740 1 --2- 172.21.15.80:0/3774477182 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557498d31d90 0x557498d45500 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:25.271 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.263+0000 7f910f385740 1 --2- 172.21.15.80:0/3774477182 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557498d2cc20 0x557498d424e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:25.271 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.263+0000 7f910f385740 1 --2- 172.21.15.80:0/3774477182 >> [v2:172.21.15.148:6836/856528403,v1:172.21.15.148:6837/856528403] conn(0x557498dba200 0x557498dbc620 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:25.271 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.263+0000 7f910f385740 1 --2- 172.21.15.80:0/3774477182 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557498d2c230 0x557498d41fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:25.271 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.263+0000 7f910f385740 1 -- 172.21.15.80:0/3774477182 >> 172.21.15.80:0/3774477182 conn(0x557498d278a0 msgr2=0x557498d2a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:25.272 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.263+0000 7f90edffb640 0 client.15909 ms_handle_reset on v2:172.21.15.148:6836/856528403 2024-04-24T23:31:25.272 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.263+0000 7f910f385740 1 -- 172.21.15.80:0/3774477182 shutdown_connections 2024-04-24T23:31:25.272 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.263+0000 7f910f385740 1 -- 172.21.15.80:0/3774477182 wait complete. 2024-04-24T23:31:25.272 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.263+0000 7f910f385740 20 client.15909 trim_cache size 0 max 16384 2024-04-24T23:31:25.272 INFO:teuthology.orchestra.run.smithi080.stdout:[] 2024-04-24T23:31:25.273 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.264+0000 7f910c59e640 1 -- 172.21.15.80:0/2561034746 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f90f00a2b90 msgr2=0x7f90f00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:25.273 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.264+0000 7f910c59e640 1 --2- 172.21.15.80:0/2561034746 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f90f00a2b90 0x7f90f00a5050 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f90fc0099e0 tx=0x7f90fc009500 comp rx=0 tx=0).stop 2024-04-24T23:31:25.273 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.264+0000 7f910c59e640 1 -- 172.21.15.80:0/2561034746 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9104104b30 msgr2=0x7f91041001b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:25.273 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.264+0000 7f910c59e640 1 --2- 172.21.15.80:0/2561034746 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9104104b30 0x7f91041001b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f90f400e9f0 tx=0x7f90f400eeb0 comp rx=0 tx=0).stop 2024-04-24T23:31:25.273 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.264+0000 7f910c59e640 1 -- 172.21.15.80:0/2561034746 shutdown_connections 2024-04-24T23:31:25.274 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.264+0000 7f910c59e640 1 --2- 172.21.15.80:0/2561034746 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f90f00a2b90 0x7f90f00a5050 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:25.274 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.264+0000 7f910c59e640 1 --2- 172.21.15.80:0/2561034746 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9104107460 0x7f91041006f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:25.274 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.264+0000 7f910c59e640 1 --2- 172.21.15.80:0/2561034746 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9104104b30 0x7f91041001b0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:25.274 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.264+0000 7f910c59e640 1 --2- 172.21.15.80:0/2561034746 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9104102200 0x7f9104101b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:25.274 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.264+0000 7f910c59e640 1 -- 172.21.15.80:0/2561034746 >> 172.21.15.80:0/2561034746 conn(0x7f91040fbbf0 msgr2=0x7f9104102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:25.274 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.264+0000 7f910c59e640 1 -- 172.21.15.80:0/2561034746 shutdown_connections 2024-04-24T23:31:25.274 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.264+0000 7f910c59e640 1 -- 172.21.15.80:0/2561034746 wait complete. 2024-04-24T23:31:25.276 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T23:31:25.387 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.385+0000 7f9173264640 1 -- 172.21.15.80:0/1173472939 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f916c104bd0 msgr2=0x7f916c106fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:25.388 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.385+0000 7f9173264640 1 --2- 172.21.15.80:0/1173472939 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f916c104bd0 0x7f916c106fc0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f9168009900 tx=0x7f916802a410 comp rx=0 tx=0).stop 2024-04-24T23:31:25.388 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.385+0000 7f9173264640 1 -- 172.21.15.80:0/1173472939 shutdown_connections 2024-04-24T23:31:25.388 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.385+0000 7f9173264640 1 --2- 172.21.15.80:0/1173472939 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f916c107500 0x7f916c1098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:25.388 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.385+0000 7f9173264640 1 --2- 172.21.15.80:0/1173472939 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f916c104bd0 0x7f916c106fc0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:25.388 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.385+0000 7f9173264640 1 --2- 172.21.15.80:0/1173472939 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f916c1022a0 0x7f916c104690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:25.388 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.385+0000 7f9173264640 1 -- 172.21.15.80:0/1173472939 >> 172.21.15.80:0/1173472939 conn(0x7f916c0fbbf0 msgr2=0x7f916c0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:25.388 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.385+0000 7f9173264640 1 -- 172.21.15.80:0/1173472939 shutdown_connections 2024-04-24T23:31:25.388 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.385+0000 7f9173264640 1 -- 172.21.15.80:0/1173472939 wait complete. 2024-04-24T23:31:25.388 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.385+0000 7f9173264640 1 Processor -- start 2024-04-24T23:31:25.388 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.385+0000 7f9173264640 1 -- start start 2024-04-24T23:31:25.388 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.385+0000 7f9173264640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f916c1022a0 0x7f916c197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:25.388 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.385+0000 7f9173264640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f916c104bd0 0x7f916c198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:25.389 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.385+0000 7f9173264640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f916c107500 0x7f916c19f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:25.389 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.385+0000 7f9173264640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f916c10d440 con 0x7f916c107500 2024-04-24T23:31:25.389 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.385+0000 7f9173264640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f916c10d2c0 con 0x7f916c1022a0 2024-04-24T23:31:25.389 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.385+0000 7f9173264640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f916c10d5c0 con 0x7f916c104bd0 2024-04-24T23:31:25.389 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.385+0000 7f9171801640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f916c107500 0x7f916c19f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:25.389 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.386+0000 7f9171801640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f916c107500 0x7f916c19f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38292/0 (socket says 172.21.15.80:38292) 2024-04-24T23:31:25.389 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.386+0000 7f9171801640 1 -- 172.21.15.80:0/1927281554 learned_addr learned my addr 172.21.15.80:0/1927281554 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:31:25.390 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.386+0000 7f9163fff640 1 --2- 172.21.15.80:0/1927281554 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f916c104bd0 0x7f916c198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:25.390 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.386+0000 7f9171000640 1 --2- 172.21.15.80:0/1927281554 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f916c1022a0 0x7f916c197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:25.390 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.386+0000 7f9171801640 1 -- 172.21.15.80:0/1927281554 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f916c104bd0 msgr2=0x7f916c198050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:25.390 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.386+0000 7f9171801640 1 --2- 172.21.15.80:0/1927281554 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f916c104bd0 0x7f916c198050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:25.391 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.386+0000 7f9171801640 1 -- 172.21.15.80:0/1927281554 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f916c1022a0 msgr2=0x7f916c197b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:25.391 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.386+0000 7f9171801640 1 --2- 172.21.15.80:0/1927281554 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f916c1022a0 0x7f916c197b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:25.391 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.386+0000 7f9171801640 1 -- 172.21.15.80:0/1927281554 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f916c19f730 con 0x7f916c107500 2024-04-24T23:31:25.391 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.386+0000 7f9171801640 1 --2- 172.21.15.80:0/1927281554 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f916c107500 0x7f916c19f0c0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f916400eb10 tx=0x7f916400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:25.391 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.386+0000 7f9161ffb640 1 -- 172.21.15.80:0/1927281554 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9164004b20 con 0x7f916c107500 2024-04-24T23:31:25.391 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.386+0000 7f9161ffb640 1 -- 172.21.15.80:0/1927281554 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9164004cc0 con 0x7f916c107500 2024-04-24T23:31:25.391 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.386+0000 7f9173264640 1 -- 172.21.15.80:0/1927281554 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f916c19fa20 con 0x7f916c107500 2024-04-24T23:31:25.391 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.386+0000 7f9161ffb640 1 -- 172.21.15.80:0/1927281554 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9164010760 con 0x7f916c107500 2024-04-24T23:31:25.391 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.387+0000 7f9173264640 1 -- 172.21.15.80:0/1927281554 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f916c19fef0 con 0x7f916c107500 2024-04-24T23:31:25.398 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.389+0000 7f9161ffb640 1 -- 172.21.15.80:0/1927281554 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9164005020 con 0x7f916c107500 2024-04-24T23:31:25.398 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.389+0000 7f9175f75740 1 Processor -- start 2024-04-24T23:31:25.399 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.389+0000 7f9175f75740 1 -- start start 2024-04-24T23:31:25.399 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.389+0000 7f9161ffb640 1 --2- 172.21.15.80:0/1927281554 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f91540a2b90 0x7f91540a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:25.399 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.390+0000 7f9163fff640 1 --2- 172.21.15.80:0/1927281554 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f91540a2b90 0x7f91540a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:25.399 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.390+0000 7f9161ffb640 1 -- 172.21.15.80:0/1927281554 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f91640cf9c0 con 0x7f916c107500 2024-04-24T23:31:25.399 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.390+0000 7f9175f75740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557d7d73a230 0x557d7d73a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:25.399 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.390+0000 7f9175f75740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d7d73ac20 0x557d7d73f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:25.399 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.390+0000 7f9175f75740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557d7d73fd90 0x557d7d742220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:25.399 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.390+0000 7f9175f75740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x557d7d69e670 con 0x557d7d73fd90 2024-04-24T23:31:25.399 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.390+0000 7f9175f75740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x557d7d646970 con 0x557d7d73ac20 2024-04-24T23:31:25.399 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.390+0000 7f9175f75740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x557d7d69a630 con 0x557d7d73a230 2024-04-24T23:31:25.399 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.390+0000 7f9171801640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557d7d73fd90 0x557d7d742220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:25.399 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.390+0000 7f9171000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557d7d73a230 0x557d7d73a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:25.399 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.390+0000 7f9171801640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557d7d73fd90 0x557d7d742220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38302/0 (socket says 172.21.15.80:38302) 2024-04-24T23:31:25.400 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.390+0000 7f9171801640 1 -- 172.21.15.80:0/1338431906 learned_addr learned my addr 172.21.15.80:0/1338431906 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:31:25.400 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.390+0000 7f9163fff640 1 --2- 172.21.15.80:0/1338431906 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d7d73ac20 0x557d7d73f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:25.400 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.390+0000 7f9163fff640 1 --2- 172.21.15.80:0/1927281554 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f91540a2b90 0x7f91540a5050 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f916802aa00 tx=0x7f9168004080 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:25.400 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.390+0000 7f9171801640 1 -- 172.21.15.80:0/1338431906 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557d7d73a230 msgr2=0x557d7d73a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:25.400 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.390+0000 7f9171801640 1 --2- 172.21.15.80:0/1338431906 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557d7d73a230 0x557d7d73a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:25.400 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.390+0000 7f9171801640 1 -- 172.21.15.80:0/1338431906 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d7d73ac20 msgr2=0x557d7d73f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:25.400 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.390+0000 7f9171801640 1 --2- 172.21.15.80:0/1338431906 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d7d73ac20 0x557d7d73f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:25.400 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.390+0000 7f9171801640 1 -- 172.21.15.80:0/1338431906 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d7d66a400 con 0x557d7d73fd90 2024-04-24T23:31:25.400 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.390+0000 7f9171000640 1 --2- 172.21.15.80:0/1338431906 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557d7d73a230 0x557d7d73a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:31:25.400 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.390+0000 7f9171801640 1 --2- 172.21.15.80:0/1338431906 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557d7d73fd90 0x557d7d742220 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f91640a4590 tx=0x7f916409b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d94d4f335fe3aec6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:25.400 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.391+0000 7f9152ffd640 1 -- 172.21.15.80:0/1338431906 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f916409be50 con 0x557d7d73fd90 2024-04-24T23:31:25.400 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.391+0000 7f9152ffd640 1 -- 172.21.15.80:0/1338431906 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f916409e070 con 0x557d7d73fd90 2024-04-24T23:31:25.400 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.391+0000 7f9152ffd640 1 -- 172.21.15.80:0/1338431906 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91640a4890 con 0x557d7d73fd90 2024-04-24T23:31:25.401 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.391+0000 7f9175f75740 1 -- 172.21.15.80:0/1338431906 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557d7d73fd90 msgr2=0x557d7d742220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:25.401 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.391+0000 7f9175f75740 1 --2- 172.21.15.80:0/1338431906 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557d7d73fd90 0x557d7d742220 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f91640a4590 tx=0x7f916409b990 comp rx=0 tx=0).stop 2024-04-24T23:31:25.401 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.391+0000 7f9175f75740 1 -- 172.21.15.80:0/1338431906 shutdown_connections 2024-04-24T23:31:25.401 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.391+0000 7f9175f75740 1 --2- 172.21.15.80:0/1338431906 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557d7d73fd90 0x557d7d742220 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:25.401 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.391+0000 7f9175f75740 1 --2- 172.21.15.80:0/1338431906 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d7d73ac20 0x557d7d73f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:25.401 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.391+0000 7f9175f75740 1 --2- 172.21.15.80:0/1338431906 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557d7d73a230 0x557d7d73a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:25.401 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.391+0000 7f9175f75740 1 -- 172.21.15.80:0/1338431906 >> 172.21.15.80:0/1338431906 conn(0x557d7d7358a0 msgr2=0x557d7d735ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:25.401 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.391+0000 7f9175f75740 1 -- 172.21.15.80:0/1338431906 shutdown_connections 2024-04-24T23:31:25.401 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.391+0000 7f9175f75740 1 -- 172.21.15.80:0/1338431906 wait complete. 2024-04-24T23:31:25.401 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.392+0000 7f9175f75740 1 -- start start 2024-04-24T23:31:25.401 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.392+0000 7f9175f75740 1 Processor -- start 2024-04-24T23:31:25.401 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.392+0000 7f9175f75740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557d7d73a230 0x557d7d7443e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:25.401 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.392+0000 7f9175f75740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557d7d73ac20 0x557d7d744920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:25.401 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.392+0000 7f9175f75740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d7d73fd90 0x557d7d74b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:25.402 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.392+0000 7f9175f75740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x557d7d742d40 con 0x557d7d73ac20 2024-04-24T23:31:25.402 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.392+0000 7f9175f75740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x557d7d65a050 con 0x557d7d73fd90 2024-04-24T23:31:25.402 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.392+0000 7f9175f75740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x557d7d742ec0 con 0x557d7d73a230 2024-04-24T23:31:25.402 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.392+0000 7f9175f75740 10 client.0 _ll_register_callbacks cb 0x557d7d3b1be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:31:25.402 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.392+0000 7f9163fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557d7d73ac20 0x557d7d744920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:25.402 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.392+0000 7f9163fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557d7d73ac20 0x557d7d744920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38316/0 (socket says 172.21.15.80:38316) 2024-04-24T23:31:25.402 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.392+0000 7f9163fff640 1 -- 172.21.15.80:0/2700640823 learned_addr learned my addr 172.21.15.80:0/2700640823 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:31:25.402 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.392+0000 7f9171000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557d7d73a230 0x557d7d7443e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:25.402 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.392+0000 7f9171801640 1 --2- 172.21.15.80:0/2700640823 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d7d73fd90 0x557d7d74b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:25.402 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.393+0000 7f9163fff640 1 -- 172.21.15.80:0/2700640823 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557d7d73a230 msgr2=0x557d7d7443e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:25.402 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.393+0000 7f9163fff640 1 --2- 172.21.15.80:0/2700640823 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557d7d73a230 0x557d7d7443e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:25.403 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.393+0000 7f9163fff640 1 -- 172.21.15.80:0/2700640823 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d7d73fd90 msgr2=0x557d7d74b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:25.403 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.393+0000 7f9163fff640 1 --2- 172.21.15.80:0/2700640823 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d7d73fd90 0x557d7d74b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:25.403 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.393+0000 7f9163fff640 1 -- 172.21.15.80:0/2700640823 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d7d62bdc0 con 0x557d7d73ac20 2024-04-24T23:31:25.403 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.393+0000 7f9171801640 1 --2- 172.21.15.80:0/2700640823 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d7d73fd90 0x557d7d74b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:31:25.403 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.393+0000 7f9163fff640 1 --2- 172.21.15.80:0/2700640823 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557d7d73ac20 0x557d7d744920 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f9168031440 tx=0x7f916802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d214bbbd5616777 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:25.403 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.393+0000 7f914bfff640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-04-24T23:31:25.403 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.393+0000 7f914bfff640 1 -- 172.21.15.80:0/2700640823 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9168035040 con 0x557d7d73ac20 2024-04-24T23:31:25.403 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.393+0000 7f914bfff640 1 -- 172.21.15.80:0/2700640823 <== mon.0 v2:172.21.15.80:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f9168004970 con 0x557d7d73ac20 2024-04-24T23:31:25.403 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.393+0000 7f9175f75740 1 -- 172.21.15.80:0/2700640823 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f916409e070 con 0x557d7d73ac20 2024-04-24T23:31:25.403 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.393+0000 7f914bfff640 1 -- 172.21.15.80:0/2700640823 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9168004b70 con 0x557d7d73ac20 2024-04-24T23:31:25.403 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.393+0000 7f9175f75740 10 client.15921 fetch_fsmap learned FSMap version 27 2024-04-24T23:31:25.404 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.393+0000 7f9175f75740 1 -- 172.21.15.80:0/2700640823 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557d7d3b23f0 con 0x557d7d73ac20 2024-04-24T23:31:25.404 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.393+0000 7f914bfff640 1 -- 172.21.15.80:0/2700640823 <== mon.0 v2:172.21.15.80:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f916409e070 con 0x557d7d73ac20 2024-04-24T23:31:25.404 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.393+0000 7f914bfff640 1 -- 172.21.15.80:0/2700640823 <== mon.0 v2:172.21.15.80:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f9168050020 con 0x557d7d73ac20 2024-04-24T23:31:25.404 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.393+0000 7f9175f75740 10 client.15921 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:31:25.404 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.393+0000 7f9175f75740 10 client.15921 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-04-24T23:31:25.404 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.393+0000 7f9175f75740 20 client.15921 populate_metadata read hostname 'smithi080' 2024-04-24T23:31:25.404 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.393+0000 7f9175f75740 1 --2- 172.21.15.80:0/2700640823 >> [v2:172.21.15.134:6838/1743815021,v1:172.21.15.134:6839/1743815021] conn(0x557d7d750230 0x557d7d752690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:25.404 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.393+0000 7f9175f75740 4 client.15921 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:31:25.404 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.393+0000 7f9175f75740 1 -- 172.21.15.80:0/2700640823 --> [v2:172.21.15.134:6838/1743815021,v1:172.21.15.134:6839/1743815021] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557d7d6abbb0 con 0x557d7d750230 2024-04-24T23:31:25.404 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.394+0000 7f9171000640 1 --2- 172.21.15.80:0/2700640823 >> [v2:172.21.15.134:6838/1743815021,v1:172.21.15.134:6839/1743815021] conn(0x557d7d750230 0x557d7d752690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:25.404 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.394+0000 7f914bfff640 10 client.15921 ms_handle_connect on v2:172.21.15.134:6838/1743815021 2024-04-24T23:31:25.404 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.394+0000 7f9171000640 1 --2- 172.21.15.80:0/2700640823 >> [v2:172.21.15.134:6838/1743815021,v1:172.21.15.134:6839/1743815021] conn(0x557d7d750230 0x557d7d752690 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a05969689acba360 server_cookie=e57079fce0f335f8 in_seq=0 out_seq=0 2024-04-24T23:31:25.405 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.396+0000 7f914bfff640 1 -- 172.21.15.80:0/2700640823 <== mds.1 v2:172.21.15.134:6838/1743815021 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557d7d6abbb0 con 0x557d7d750230 2024-04-24T23:31:25.405 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.396+0000 7f914bfff640 10 client.15921 handle_command_reply: tid=0 2024-04-24T23:31:25.405 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.396+0000 7f9175f75740 1 client.15921 shutdown 2024-04-24T23:31:25.405 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.396+0000 7f9175f75740 1 -- 172.21.15.80:0/2700640823 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557d7d73ac20 msgr2=0x557d7d744920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:25.405 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.396+0000 7f9175f75740 1 --2- 172.21.15.80:0/2700640823 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557d7d73ac20 0x557d7d744920 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f9168031440 tx=0x7f916802a410 comp rx=0 tx=0).stop 2024-04-24T23:31:25.405 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.396+0000 7f9175f75740 1 -- 172.21.15.80:0/2700640823 shutdown_connections 2024-04-24T23:31:25.405 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.396+0000 7f9175f75740 1 --2- 172.21.15.80:0/2700640823 >> [v2:172.21.15.134:6838/1743815021,v1:172.21.15.134:6839/1743815021] conn(0x557d7d750230 0x557d7d752690 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:25.405 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.396+0000 7f9175f75740 1 --2- 172.21.15.80:0/2700640823 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d7d73fd90 0x557d7d74b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:25.405 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.396+0000 7f9175f75740 1 --2- 172.21.15.80:0/2700640823 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557d7d73ac20 0x557d7d744920 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:25.405 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.396+0000 7f9175f75740 1 --2- 172.21.15.80:0/2700640823 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557d7d73a230 0x557d7d7443e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:25.405 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.396+0000 7f914bfff640 0 client.15921 ms_handle_reset on v2:172.21.15.134:6838/1743815021 2024-04-24T23:31:25.405 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.396+0000 7f9175f75740 1 -- 172.21.15.80:0/2700640823 >> 172.21.15.80:0/2700640823 conn(0x557d7d7358a0 msgr2=0x557d7d734b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:25.405 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.396+0000 7f9175f75740 1 -- 172.21.15.80:0/2700640823 shutdown_connections 2024-04-24T23:31:25.406 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.396+0000 7f9175f75740 1 -- 172.21.15.80:0/2700640823 wait complete. 2024-04-24T23:31:25.406 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.396+0000 7f9175f75740 20 client.15921 trim_cache size 0 max 16384 2024-04-24T23:31:25.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.410+0000 7f9175f75740 1 Processor -- start 2024-04-24T23:31:25.417 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.410+0000 7f9175f75740 1 -- start start 2024-04-24T23:31:25.417 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.410+0000 7f9175f75740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557d7d73a230 0x557d7d743a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:25.417 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.410+0000 7f9175f75740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d7d73ac20 0x557d7d74b920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:25.417 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.410+0000 7f9175f75740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557d7d73fd90 0x557d7d73b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:25.417 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.410+0000 7f9175f75740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f916c1ace20 con 0x557d7d73a230 2024-04-24T23:31:25.417 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.410+0000 7f9175f75740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f916c1aa6f0 con 0x557d7d73ac20 2024-04-24T23:31:25.417 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.410+0000 7f9175f75740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f916c1a8540 con 0x557d7d73fd90 2024-04-24T23:31:25.417 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.410+0000 7f9171000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557d7d73a230 0x557d7d743a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:25.417 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.410+0000 7f9171000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557d7d73a230 0x557d7d743a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38328/0 (socket says 172.21.15.80:38328) 2024-04-24T23:31:25.417 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.410+0000 7f9171000640 1 -- 172.21.15.80:0/3106770188 learned_addr learned my addr 172.21.15.80:0/3106770188 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:31:25.417 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.410+0000 7f9171801640 1 --2- 172.21.15.80:0/3106770188 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557d7d73fd90 0x557d7d73b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:25.417 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.410+0000 7f9163fff640 1 --2- 172.21.15.80:0/3106770188 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d7d73ac20 0x557d7d74b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:25.418 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.410+0000 7f9171000640 1 -- 172.21.15.80:0/3106770188 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557d7d73fd90 msgr2=0x557d7d73b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:25.418 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.410+0000 7f9171000640 1 --2- 172.21.15.80:0/3106770188 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557d7d73fd90 0x557d7d73b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:25.418 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.410+0000 7f9171000640 1 -- 172.21.15.80:0/3106770188 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d7d73ac20 msgr2=0x557d7d74b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:25.418 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.410+0000 7f9171000640 1 --2- 172.21.15.80:0/3106770188 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d7d73ac20 0x557d7d74b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:25.418 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.410+0000 7f9171000640 1 -- 172.21.15.80:0/3106770188 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d7d47c7d0 con 0x557d7d73a230 2024-04-24T23:31:25.418 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.410+0000 7f9171000640 1 --2- 172.21.15.80:0/3106770188 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557d7d73a230 0x557d7d743a00 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f915c009d10 tx=0x7f915c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b54e9767a03cf80 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:25.418 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.410+0000 7f91527fc640 1 -- 172.21.15.80:0/3106770188 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f915c01f250 con 0x557d7d73a230 2024-04-24T23:31:25.418 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.410+0000 7f91527fc640 1 -- 172.21.15.80:0/3106770188 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f915c004490 con 0x557d7d73a230 2024-04-24T23:31:25.418 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.410+0000 7f91527fc640 1 -- 172.21.15.80:0/3106770188 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f915c010c90 con 0x557d7d73a230 2024-04-24T23:31:25.418 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.411+0000 7f9175f75740 1 -- 172.21.15.80:0/3106770188 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557d7d73a230 msgr2=0x557d7d743a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:25.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.411+0000 7f9175f75740 1 --2- 172.21.15.80:0/3106770188 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557d7d73a230 0x557d7d743a00 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f915c009d10 tx=0x7f915c005f90 comp rx=0 tx=0).stop 2024-04-24T23:31:25.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.411+0000 7f9175f75740 1 -- 172.21.15.80:0/3106770188 shutdown_connections 2024-04-24T23:31:25.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.411+0000 7f9175f75740 1 --2- 172.21.15.80:0/3106770188 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557d7d73fd90 0x557d7d73b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:25.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.411+0000 7f9175f75740 1 --2- 172.21.15.80:0/3106770188 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d7d73ac20 0x557d7d74b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:25.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.411+0000 7f9175f75740 1 --2- 172.21.15.80:0/3106770188 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557d7d73a230 0x557d7d743a00 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:25.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.411+0000 7f9175f75740 1 -- 172.21.15.80:0/3106770188 >> 172.21.15.80:0/3106770188 conn(0x557d7d7358a0 msgr2=0x557d7d734b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:25.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.411+0000 7f9175f75740 1 -- 172.21.15.80:0/3106770188 shutdown_connections 2024-04-24T23:31:25.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.411+0000 7f9175f75740 1 -- 172.21.15.80:0/3106770188 wait complete. 2024-04-24T23:31:25.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.411+0000 7f9175f75740 1 -- start start 2024-04-24T23:31:25.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.411+0000 7f9175f75740 1 Processor -- start 2024-04-24T23:31:25.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.412+0000 7f9175f75740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d7d73a230 0x557d7d748ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:25.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.412+0000 7f9175f75740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557d7d73ac20 0x557d7d7493e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:25.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.412+0000 7f9175f75740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557d7d73fd90 0x557d7d74a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:25.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.412+0000 7f9175f75740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f916c1a8bb0 con 0x557d7d73ac20 2024-04-24T23:31:25.420 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.412+0000 7f9175f75740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f916c1aa020 con 0x557d7d73a230 2024-04-24T23:31:25.420 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.412+0000 7f9175f75740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f916c1a9280 con 0x557d7d73fd90 2024-04-24T23:31:25.420 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.412+0000 7f9175f75740 10 client.0 _ll_register_callbacks cb 0x7f9168009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:31:25.420 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.412+0000 7f9163fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557d7d73ac20 0x557d7d7493e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:25.420 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.412+0000 7f9163fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557d7d73ac20 0x557d7d7493e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38338/0 (socket says 172.21.15.80:38338) 2024-04-24T23:31:25.420 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.412+0000 7f9163fff640 1 -- 172.21.15.80:0/2815757890 learned_addr learned my addr 172.21.15.80:0/2815757890 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:31:25.420 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.412+0000 7f9171000640 1 --2- 172.21.15.80:0/2815757890 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d7d73a230 0x557d7d748ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:25.420 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.412+0000 7f9171801640 1 --2- 172.21.15.80:0/2815757890 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557d7d73fd90 0x557d7d74a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:25.420 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.412+0000 7f9163fff640 1 -- 172.21.15.80:0/2815757890 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557d7d73fd90 msgr2=0x557d7d74a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:25.420 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.412+0000 7f9163fff640 1 --2- 172.21.15.80:0/2815757890 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557d7d73fd90 0x557d7d74a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:25.420 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.412+0000 7f9163fff640 1 -- 172.21.15.80:0/2815757890 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d7d73a230 msgr2=0x557d7d748ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:25.420 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.412+0000 7f9163fff640 1 --2- 172.21.15.80:0/2815757890 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d7d73a230 0x557d7d748ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:25.421 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.412+0000 7f9163fff640 1 -- 172.21.15.80:0/2815757890 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d7d74a9e0 con 0x557d7d73ac20 2024-04-24T23:31:25.421 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.412+0000 7f9163fff640 1 --2- 172.21.15.80:0/2815757890 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557d7d73ac20 0x557d7d7493e0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f9168003980 tx=0x7f9168005990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c773ec49f980755 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:25.421 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.413+0000 7f914bfff640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-04-24T23:31:25.421 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.413+0000 7f914bfff640 1 -- 172.21.15.80:0/2815757890 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9168003c00 con 0x557d7d73ac20 2024-04-24T23:31:25.421 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.413+0000 7f914bfff640 1 -- 172.21.15.80:0/2815757890 <== mon.0 v2:172.21.15.80:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f9168003da0 con 0x557d7d73ac20 2024-04-24T23:31:25.421 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.413+0000 7f9175f75740 1 -- 172.21.15.80:0/2815757890 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f915c004490 con 0x557d7d73ac20 2024-04-24T23:31:25.421 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.413+0000 7f914bfff640 1 -- 172.21.15.80:0/2815757890 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9168004980 con 0x557d7d73ac20 2024-04-24T23:31:25.421 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.413+0000 7f9175f75740 10 client.15927 fetch_fsmap learned FSMap version 27 2024-04-24T23:31:25.421 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.413+0000 7f9175f75740 1 -- 172.21.15.80:0/2815757890 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557d7d74bd80 con 0x557d7d73ac20 2024-04-24T23:31:25.421 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.413+0000 7f914bfff640 1 -- 172.21.15.80:0/2815757890 <== mon.0 v2:172.21.15.80:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f915c004490 con 0x557d7d73ac20 2024-04-24T23:31:25.421 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.413+0000 7f914bfff640 1 -- 172.21.15.80:0/2815757890 <== mon.0 v2:172.21.15.80:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f9168050040 con 0x557d7d73ac20 2024-04-24T23:31:25.422 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.413+0000 7f9175f75740 10 client.15927 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:31:25.422 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.413+0000 7f9175f75740 10 client.15927 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-04-24T23:31:25.422 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.413+0000 7f9175f75740 20 client.15927 populate_metadata read hostname 'smithi080' 2024-04-24T23:31:25.422 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.413+0000 7f9175f75740 1 --2- 172.21.15.80:0/2815757890 >> [v2:172.21.15.134:6838/1743815021,v1:172.21.15.134:6839/1743815021] conn(0x557d7d7b6ed0 0x557d7d7b92f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:25.422 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.413+0000 7f9175f75740 4 client.15927 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:31:25.422 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.413+0000 7f9175f75740 1 -- 172.21.15.80:0/2815757890 --> [v2:172.21.15.134:6838/1743815021,v1:172.21.15.134:6839/1743815021] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557d7d7441d0 con 0x557d7d7b6ed0 2024-04-24T23:31:25.422 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.413+0000 7f9171000640 1 --2- 172.21.15.80:0/2815757890 >> [v2:172.21.15.134:6838/1743815021,v1:172.21.15.134:6839/1743815021] conn(0x557d7d7b6ed0 0x557d7d7b92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:25.422 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.414+0000 7f914bfff640 10 client.15927 ms_handle_connect on v2:172.21.15.134:6838/1743815021 2024-04-24T23:31:25.422 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.414+0000 7f9171000640 1 --2- 172.21.15.80:0/2815757890 >> [v2:172.21.15.134:6838/1743815021,v1:172.21.15.134:6839/1743815021] conn(0x557d7d7b6ed0 0x557d7d7b92f0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=14c4ee3846d41cca server_cookie=6f441d4004acc11 in_seq=0 out_seq=0 2024-04-24T23:31:25.422 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.414+0000 7f914bfff640 1 -- 172.21.15.80:0/2815757890 <== mds.1 v2:172.21.15.134:6838/1743815021 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557d7d7441d0 con 0x557d7d7b6ed0 2024-04-24T23:31:25.422 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.414+0000 7f914bfff640 10 client.15927 handle_command_reply: tid=0 2024-04-24T23:31:25.422 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.414+0000 7f9175f75740 1 client.15927 shutdown 2024-04-24T23:31:25.422 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.414+0000 7f9175f75740 1 -- 172.21.15.80:0/2815757890 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557d7d73ac20 msgr2=0x557d7d7493e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:25.423 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.414+0000 7f9175f75740 1 --2- 172.21.15.80:0/2815757890 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557d7d73ac20 0x557d7d7493e0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f9168003980 tx=0x7f9168005990 comp rx=0 tx=0).stop 2024-04-24T23:31:25.423 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.414+0000 7f9175f75740 1 -- 172.21.15.80:0/2815757890 shutdown_connections 2024-04-24T23:31:25.423 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.414+0000 7f9175f75740 1 --2- 172.21.15.80:0/2815757890 >> [v2:172.21.15.134:6838/1743815021,v1:172.21.15.134:6839/1743815021] conn(0x557d7d7b6ed0 0x557d7d7b92f0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:25.423 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.414+0000 7f9175f75740 1 --2- 172.21.15.80:0/2815757890 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557d7d73fd90 0x557d7d74a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:25.423 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.414+0000 7f9175f75740 1 --2- 172.21.15.80:0/2815757890 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557d7d73ac20 0x557d7d7493e0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:25.423 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.414+0000 7f914bfff640 0 client.15927 ms_handle_reset on v2:172.21.15.134:6838/1743815021 2024-04-24T23:31:25.423 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.414+0000 7f9175f75740 1 --2- 172.21.15.80:0/2815757890 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d7d73a230 0x557d7d748ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:25.423 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.414+0000 7f9175f75740 1 -- 172.21.15.80:0/2815757890 >> 172.21.15.80:0/2815757890 conn(0x557d7d7358a0 msgr2=0x557d7d734b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:25.423 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.414+0000 7f9175f75740 1 -- 172.21.15.80:0/2815757890 shutdown_connections 2024-04-24T23:31:25.423 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.414+0000 7f9175f75740 1 -- 172.21.15.80:0/2815757890 wait complete. 2024-04-24T23:31:25.423 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.414+0000 7f9175f75740 20 client.15927 trim_cache size 0 max 16384 2024-04-24T23:31:25.424 INFO:teuthology.orchestra.run.smithi080.stdout:[] 2024-04-24T23:31:25.424 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.415+0000 7f9173264640 1 -- 172.21.15.80:0/1927281554 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f91540a2b90 msgr2=0x7f91540a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:25.425 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.415+0000 7f9173264640 1 --2- 172.21.15.80:0/1927281554 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f91540a2b90 0x7f91540a5050 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f916802aa00 tx=0x7f9168004080 comp rx=0 tx=0).stop 2024-04-24T23:31:25.425 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.415+0000 7f9173264640 1 -- 172.21.15.80:0/1927281554 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f916c107500 msgr2=0x7f916c19f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:25.425 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.415+0000 7f9173264640 1 --2- 172.21.15.80:0/1927281554 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f916c107500 0x7f916c19f0c0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f916400eb10 tx=0x7f916400c2d0 comp rx=0 tx=0).stop 2024-04-24T23:31:25.425 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.415+0000 7f9173264640 1 -- 172.21.15.80:0/1927281554 shutdown_connections 2024-04-24T23:31:25.425 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.415+0000 7f9173264640 1 --2- 172.21.15.80:0/1927281554 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f91540a2b90 0x7f91540a5050 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:25.425 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.415+0000 7f9173264640 1 --2- 172.21.15.80:0/1927281554 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f916c107500 0x7f916c19f0c0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:25.425 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.415+0000 7f9173264640 1 --2- 172.21.15.80:0/1927281554 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f916c104bd0 0x7f916c198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:25.425 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.415+0000 7f9173264640 1 --2- 172.21.15.80:0/1927281554 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f916c1022a0 0x7f916c197b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:25.425 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.415+0000 7f9173264640 1 -- 172.21.15.80:0/1927281554 >> 172.21.15.80:0/1927281554 conn(0x7f916c0fbbf0 msgr2=0x7f916c102d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:25.425 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.415+0000 7f9173264640 1 -- 172.21.15.80:0/1927281554 shutdown_connections 2024-04-24T23:31:25.426 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.415+0000 7f9173264640 1 -- 172.21.15.80:0/1927281554 wait complete. 2024-04-24T23:31:25.427 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T23:31:25.536 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.533+0000 7fed50446640 1 -- 172.21.15.80:0/1960987179 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fed48102250 msgr2=0x7fed48108ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:25.536 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.533+0000 7fed50446640 1 --2- 172.21.15.80:0/1960987179 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fed48102250 0x7fed48108ae0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fed44009900 tx=0x7fed4402a410 comp rx=0 tx=0).stop 2024-04-24T23:31:25.536 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.533+0000 7fed50446640 1 -- 172.21.15.80:0/1960987179 shutdown_connections 2024-04-24T23:31:25.536 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.533+0000 7fed50446640 1 --2- 172.21.15.80:0/1960987179 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fed48102250 0x7fed48108ae0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:25.536 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.533+0000 7fed50446640 1 --2- 172.21.15.80:0/1960987179 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fed48101890 0x7fed48101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:25.536 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.533+0000 7fed50446640 1 --2- 172.21.15.80:0/1960987179 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed481006a0 0x7fed48100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:25.536 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.533+0000 7fed50446640 1 -- 172.21.15.80:0/1960987179 >> 172.21.15.80:0/1960987179 conn(0x7fed480fbc10 msgr2=0x7fed480fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:25.536 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.533+0000 7fed50446640 1 -- 172.21.15.80:0/1960987179 shutdown_connections 2024-04-24T23:31:25.536 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.533+0000 7fed50446640 1 -- 172.21.15.80:0/1960987179 wait complete. 2024-04-24T23:31:25.536 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.533+0000 7fed50446640 1 Processor -- start 2024-04-24T23:31:25.536 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.534+0000 7fed50446640 1 -- start start 2024-04-24T23:31:25.537 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.534+0000 7fed50446640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed481006a0 0x7fed48197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:25.537 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.534+0000 7fed50446640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fed48101890 0x7fed48197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:25.537 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.534+0000 7fed50446640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fed48102250 0x7fed4819f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:25.537 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.534+0000 7fed50446640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fed4810c320 con 0x7fed48102250 2024-04-24T23:31:25.537 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.534+0000 7fed50446640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fed4810c1a0 con 0x7fed48101890 2024-04-24T23:31:25.537 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.534+0000 7fed50446640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fed4810c4a0 con 0x7fed481006a0 2024-04-24T23:31:25.537 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.534+0000 7fed4e9e3640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fed48102250 0x7fed4819f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:25.537 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.534+0000 7fed4e9e3640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fed48102250 0x7fed4819f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38346/0 (socket says 172.21.15.80:38346) 2024-04-24T23:31:25.537 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.534+0000 7fed4e9e3640 1 -- 172.21.15.80:0/4170001648 learned_addr learned my addr 172.21.15.80:0/4170001648 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:31:25.537 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.534+0000 7fed4e1e2640 1 --2- 172.21.15.80:0/4170001648 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed481006a0 0x7fed48197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:25.537 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.534+0000 7fed4d9e1640 1 --2- 172.21.15.80:0/4170001648 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fed48101890 0x7fed48197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:25.537 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.534+0000 7fed4e9e3640 1 -- 172.21.15.80:0/4170001648 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed481006a0 msgr2=0x7fed48197ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:25.538 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.534+0000 7fed4e9e3640 1 --2- 172.21.15.80:0/4170001648 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed481006a0 0x7fed48197ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:25.538 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.534+0000 7fed4e9e3640 1 -- 172.21.15.80:0/4170001648 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fed48101890 msgr2=0x7fed48197ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:25.538 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.534+0000 7fed4e9e3640 1 --2- 172.21.15.80:0/4170001648 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fed48101890 0x7fed48197ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:25.538 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.534+0000 7fed4e9e3640 1 -- 172.21.15.80:0/4170001648 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed4819f6d0 con 0x7fed48102250 2024-04-24T23:31:25.538 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.534+0000 7fed4e9e3640 1 --2- 172.21.15.80:0/4170001648 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fed48102250 0x7fed4819f060 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fed440314e0 tx=0x7fed440042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:25.538 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.534+0000 7fed3f7fe640 1 -- 172.21.15.80:0/4170001648 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed4403f030 con 0x7fed48102250 2024-04-24T23:31:25.538 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.534+0000 7fed50446640 1 -- 172.21.15.80:0/4170001648 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed4819f960 con 0x7fed48102250 2024-04-24T23:31:25.538 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.534+0000 7fed3f7fe640 1 -- 172.21.15.80:0/4170001648 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed44003ce0 con 0x7fed48102250 2024-04-24T23:31:25.538 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.534+0000 7fed3f7fe640 1 -- 172.21.15.80:0/4170001648 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed44035070 con 0x7fed48102250 2024-04-24T23:31:25.538 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.535+0000 7fed50446640 1 -- 172.21.15.80:0/4170001648 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed4819fea0 con 0x7fed48102250 2024-04-24T23:31:25.545 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.537+0000 7fed3f7fe640 1 -- 172.21.15.80:0/4170001648 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fed44042030 con 0x7fed48102250 2024-04-24T23:31:25.545 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.537+0000 7fed3f7fe640 1 --2- 172.21.15.80:0/4170001648 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fed340a2b90 0x7fed340a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:25.545 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.537+0000 7fed53232740 1 Processor -- start 2024-04-24T23:31:25.545 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.537+0000 7fed4e1e2640 1 --2- 172.21.15.80:0/4170001648 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fed340a2b90 0x7fed340a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:25.545 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.537+0000 7fed3f7fe640 1 -- 172.21.15.80:0/4170001648 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fed440ef850 con 0x7fed48102250 2024-04-24T23:31:25.545 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.538+0000 7fed53232740 1 -- start start 2024-04-24T23:31:25.545 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.538+0000 7fed53232740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56128600c640 0x56128600ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:25.546 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.538+0000 7fed53232740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56128600d030 0x561286011b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:25.546 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.538+0000 7fed53232740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5612860121a0 0x561286014630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:25.546 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.538+0000 7fed53232740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x561285f6c670 con 0x56128600c640 2024-04-24T23:31:25.546 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.538+0000 7fed53232740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x561285f14970 con 0x56128600d030 2024-04-24T23:31:25.546 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.538+0000 7fed53232740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x561285f68630 con 0x5612860121a0 2024-04-24T23:31:25.546 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.538+0000 7fed4e1e2640 1 --2- 172.21.15.80:0/4170001648 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fed340a2b90 0x7fed340a5050 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fed481016f0 tx=0x7fed40009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:25.546 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.538+0000 7fed4e1e2640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56128600c640 0x56128600ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:25.546 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.538+0000 7fed4e1e2640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56128600c640 0x56128600ca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38348/0 (socket says 172.21.15.80:38348) 2024-04-24T23:31:25.546 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.538+0000 7fed4e1e2640 1 -- 172.21.15.80:0/1338294475 learned_addr learned my addr 172.21.15.80:0/1338294475 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:31:25.546 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.538+0000 7fed4e9e3640 1 --2- 172.21.15.80:0/1338294475 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5612860121a0 0x561286014630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:25.546 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.538+0000 7fed4d9e1640 1 --2- 172.21.15.80:0/1338294475 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56128600d030 0x561286011b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:25.546 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.538+0000 7fed4e1e2640 1 -- 172.21.15.80:0/1338294475 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5612860121a0 msgr2=0x561286014630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:25.547 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.538+0000 7fed4e1e2640 1 --2- 172.21.15.80:0/1338294475 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5612860121a0 0x561286014630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:25.547 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.538+0000 7fed4e1e2640 1 -- 172.21.15.80:0/1338294475 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56128600d030 msgr2=0x561286011b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:25.547 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.538+0000 7fed4e1e2640 1 --2- 172.21.15.80:0/1338294475 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56128600d030 0x561286011b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:25.547 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.538+0000 7fed4e1e2640 1 -- 172.21.15.80:0/1338294475 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561285f38400 con 0x56128600c640 2024-04-24T23:31:25.547 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.538+0000 7fed4e1e2640 1 --2- 172.21.15.80:0/1338294475 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56128600c640 0x56128600ca60 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fed4001d5f0 tx=0x7fed40014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dce283d942f5dc6f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:25.547 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.539+0000 7fed3cbfe640 1 -- 172.21.15.80:0/1338294475 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed4001e5b0 con 0x56128600c640 2024-04-24T23:31:25.547 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.539+0000 7fed3cbfe640 1 -- 172.21.15.80:0/1338294475 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed40022070 con 0x56128600c640 2024-04-24T23:31:25.547 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.539+0000 7fed3cbfe640 1 -- 172.21.15.80:0/1338294475 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed4000eb40 con 0x56128600c640 2024-04-24T23:31:25.547 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.539+0000 7fed53232740 1 -- 172.21.15.80:0/1338294475 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56128600c640 msgr2=0x56128600ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:25.547 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.539+0000 7fed53232740 1 --2- 172.21.15.80:0/1338294475 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56128600c640 0x56128600ca60 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fed4001d5f0 tx=0x7fed40014000 comp rx=0 tx=0).stop 2024-04-24T23:31:25.547 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.539+0000 7fed53232740 1 -- 172.21.15.80:0/1338294475 shutdown_connections 2024-04-24T23:31:25.547 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.539+0000 7fed53232740 1 --2- 172.21.15.80:0/1338294475 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5612860121a0 0x561286014630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:25.547 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.539+0000 7fed53232740 1 --2- 172.21.15.80:0/1338294475 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56128600d030 0x561286011b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:25.548 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.539+0000 7fed53232740 1 --2- 172.21.15.80:0/1338294475 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56128600c640 0x56128600ca60 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:25.548 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.539+0000 7fed53232740 1 -- 172.21.15.80:0/1338294475 >> 172.21.15.80:0/1338294475 conn(0x5612860038a0 msgr2=0x561286003ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:25.548 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.539+0000 7fed53232740 1 -- 172.21.15.80:0/1338294475 shutdown_connections 2024-04-24T23:31:25.548 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.539+0000 7fed53232740 1 -- 172.21.15.80:0/1338294475 wait complete. 2024-04-24T23:31:25.548 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.540+0000 7fed53232740 1 -- start start 2024-04-24T23:31:25.548 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.540+0000 7fed53232740 1 Processor -- start 2024-04-24T23:31:25.548 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.540+0000 7fed53232740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56128600c640 0x56128600ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:25.548 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.540+0000 7fed53232740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56128600d030 0x56128600f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:25.548 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.540+0000 7fed53232740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5612860121a0 0x561286010280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:25.548 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.540+0000 7fed53232740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x561286015150 con 0x5612860121a0 2024-04-24T23:31:25.548 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.540+0000 7fed53232740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x561285f28050 con 0x56128600d030 2024-04-24T23:31:25.548 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.540+0000 7fed53232740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5612860152d0 con 0x56128600c640 2024-04-24T23:31:25.548 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.540+0000 7fed53232740 10 client.0 _ll_register_callbacks cb 0x561285c7fbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:31:25.548 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.540+0000 7fed4e9e3640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5612860121a0 0x561286010280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:25.549 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.540+0000 7fed4e1e2640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56128600c640 0x56128600ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:25.549 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.540+0000 7fed4e9e3640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5612860121a0 0x561286010280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38362/0 (socket says 172.21.15.80:38362) 2024-04-24T23:31:25.549 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.540+0000 7fed4e9e3640 1 -- 172.21.15.80:0/2065980196 learned_addr learned my addr 172.21.15.80:0/2065980196 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:31:25.549 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.540+0000 7fed4d9e1640 1 --2- 172.21.15.80:0/2065980196 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56128600d030 0x56128600f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:25.549 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.540+0000 7fed4e9e3640 1 -- 172.21.15.80:0/2065980196 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56128600c640 msgr2=0x56128600ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:25.549 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.540+0000 7fed4e9e3640 1 --2- 172.21.15.80:0/2065980196 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56128600c640 0x56128600ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:25.549 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.540+0000 7fed4e9e3640 1 -- 172.21.15.80:0/2065980196 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56128600d030 msgr2=0x56128600f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:25.549 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.540+0000 7fed4e9e3640 1 --2- 172.21.15.80:0/2065980196 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56128600d030 0x56128600f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:25.549 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.540+0000 7fed4e9e3640 1 -- 172.21.15.80:0/2065980196 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561285ef9dc0 con 0x5612860121a0 2024-04-24T23:31:25.550 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.540+0000 7fed4e1e2640 1 --2- 172.21.15.80:0/2065980196 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56128600c640 0x56128600ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:31:25.550 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.541+0000 7fed4e9e3640 1 --2- 172.21.15.80:0/2065980196 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5612860121a0 0x561286010280 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fed44005ab0 tx=0x7fed440bb8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=679a2227520986c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:25.550 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.541+0000 7fed31ffb640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-04-24T23:31:25.550 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.541+0000 7fed31ffb640 1 -- 172.21.15.80:0/2065980196 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed44049040 con 0x5612860121a0 2024-04-24T23:31:25.550 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.541+0000 7fed31ffb640 1 -- 172.21.15.80:0/2065980196 <== mon.0 v2:172.21.15.80:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fed440b8400 con 0x5612860121a0 2024-04-24T23:31:25.550 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.541+0000 7fed53232740 1 -- 172.21.15.80:0/2065980196 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed40022070 con 0x5612860121a0 2024-04-24T23:31:25.550 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.541+0000 7fed31ffb640 1 -- 172.21.15.80:0/2065980196 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fed440bbd90 con 0x5612860121a0 2024-04-24T23:31:25.550 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.541+0000 7fed53232740 10 client.15939 fetch_fsmap learned FSMap version 27 2024-04-24T23:31:25.550 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.541+0000 7fed53232740 1 -- 172.21.15.80:0/2065980196 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561285c803f0 con 0x5612860121a0 2024-04-24T23:31:25.550 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.541+0000 7fed31ffb640 1 -- 172.21.15.80:0/2065980196 <== mon.0 v2:172.21.15.80:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed40022070 con 0x5612860121a0 2024-04-24T23:31:25.550 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.541+0000 7fed31ffb640 1 -- 172.21.15.80:0/2065980196 <== mon.0 v2:172.21.15.80:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fed4403bc10 con 0x5612860121a0 2024-04-24T23:31:25.550 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.541+0000 7fed53232740 10 client.15939 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:31:25.550 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.541+0000 7fed53232740 10 client.15939 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:31:25.550 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.541+0000 7fed53232740 20 client.15939 populate_metadata read hostname 'smithi080' 2024-04-24T23:31:25.551 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.541+0000 7fed53232740 1 --2- 172.21.15.80:0/2065980196 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x561286026a30 0x561286028e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:25.551 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.541+0000 7fed53232740 4 client.15939 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:31:25.551 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.541+0000 7fed53232740 1 -- 172.21.15.80:0/2065980196 --> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561285f79bb0 con 0x561286026a30 2024-04-24T23:31:25.551 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.541+0000 7fed4e1e2640 1 --2- 172.21.15.80:0/2065980196 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x561286026a30 0x561286028e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:25.551 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.542+0000 7fed4e1e2640 1 --2- 172.21.15.80:0/2065980196 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x561286026a30 0x561286028e90 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92d6dfd166a74939 server_cookie=20936a0f59e2fa0c in_seq=0 out_seq=0 2024-04-24T23:31:25.551 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.542+0000 7fed31ffb640 10 client.15939 ms_handle_connect on v2:172.21.15.148:6838/2952757594 2024-04-24T23:31:25.551 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.543+0000 7fed31ffb640 1 -- 172.21.15.80:0/2065980196 <== mds.0 v2:172.21.15.148:6838/2952757594 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561285f79bb0 con 0x561286026a30 2024-04-24T23:31:25.551 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.543+0000 7fed31ffb640 10 client.15939 handle_command_reply: tid=0 2024-04-24T23:31:25.551 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.543+0000 7fed53232740 1 client.15939 shutdown 2024-04-24T23:31:25.551 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.543+0000 7fed53232740 1 -- 172.21.15.80:0/2065980196 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5612860121a0 msgr2=0x561286010280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:25.551 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.543+0000 7fed53232740 1 --2- 172.21.15.80:0/2065980196 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5612860121a0 0x561286010280 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fed44005ab0 tx=0x7fed440bb8d0 comp rx=0 tx=0).stop 2024-04-24T23:31:25.551 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.543+0000 7fed53232740 1 -- 172.21.15.80:0/2065980196 shutdown_connections 2024-04-24T23:31:25.551 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.543+0000 7fed53232740 1 --2- 172.21.15.80:0/2065980196 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x561286026a30 0x561286028e90 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:25.551 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.543+0000 7fed53232740 1 --2- 172.21.15.80:0/2065980196 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5612860121a0 0x561286010280 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:25.552 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.543+0000 7fed53232740 1 --2- 172.21.15.80:0/2065980196 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56128600d030 0x56128600f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:25.552 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.543+0000 7fed31ffb640 0 client.15939 ms_handle_reset on v2:172.21.15.148:6838/2952757594 2024-04-24T23:31:25.552 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.543+0000 7fed53232740 1 --2- 172.21.15.80:0/2065980196 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56128600c640 0x56128600ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:25.552 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.543+0000 7fed53232740 1 -- 172.21.15.80:0/2065980196 >> 172.21.15.80:0/2065980196 conn(0x5612860038a0 msgr2=0x561286002b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:25.552 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.543+0000 7fed53232740 1 -- 172.21.15.80:0/2065980196 shutdown_connections 2024-04-24T23:31:25.552 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.543+0000 7fed53232740 1 -- 172.21.15.80:0/2065980196 wait complete. 2024-04-24T23:31:25.552 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.543+0000 7fed53232740 20 client.15939 trim_cache size 0 max 16384 2024-04-24T23:31:25.563 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.557+0000 7fed53232740 1 Processor -- start 2024-04-24T23:31:25.563 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.557+0000 7fed53232740 1 -- start start 2024-04-24T23:31:25.563 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.557+0000 7fed53232740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56128600c640 0x561285ffe9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:25.563 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.557+0000 7fed53232740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56128600d030 0x56128600bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:25.563 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.557+0000 7fed53232740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5612860121a0 0x561286007d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:25.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.557+0000 7fed53232740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fed481aab70 con 0x5612860121a0 2024-04-24T23:31:25.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.557+0000 7fed53232740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fed481a8440 con 0x56128600d030 2024-04-24T23:31:25.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.557+0000 7fed53232740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fed481a6290 con 0x56128600c640 2024-04-24T23:31:25.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.557+0000 7fed4e9e3640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5612860121a0 0x561286007d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:25.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.557+0000 7fed4e1e2640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56128600c640 0x561285ffe9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:25.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.557+0000 7fed4e9e3640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5612860121a0 0x561286007d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38378/0 (socket says 172.21.15.80:38378) 2024-04-24T23:31:25.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.557+0000 7fed4e9e3640 1 -- 172.21.15.80:0/2294937156 learned_addr learned my addr 172.21.15.80:0/2294937156 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:31:25.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.557+0000 7fed4d9e1640 1 --2- 172.21.15.80:0/2294937156 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56128600d030 0x56128600bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:25.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.558+0000 7fed4e9e3640 1 -- 172.21.15.80:0/2294937156 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56128600c640 msgr2=0x561285ffe9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:25.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.558+0000 7fed4e9e3640 1 --2- 172.21.15.80:0/2294937156 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56128600c640 0x561285ffe9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:25.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.558+0000 7fed4e9e3640 1 -- 172.21.15.80:0/2294937156 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56128600d030 msgr2=0x56128600bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:25.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.558+0000 7fed4e9e3640 1 --2- 172.21.15.80:0/2294937156 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56128600d030 0x56128600bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:25.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.558+0000 7fed4e9e3640 1 -- 172.21.15.80:0/2294937156 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561285d4a7d0 con 0x5612860121a0 2024-04-24T23:31:25.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.558+0000 7fed4d9e1640 1 --2- 172.21.15.80:0/2294937156 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56128600d030 0x56128600bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:31:25.565 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.558+0000 7fed4e9e3640 1 --2- 172.21.15.80:0/2294937156 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5612860121a0 0x561286007d90 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fed440bba90 tx=0x7fed440ba2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f2a4b3026b4f5bc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:25.565 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.558+0000 7fed33fff640 1 -- 172.21.15.80:0/2294937156 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed44049040 con 0x5612860121a0 2024-04-24T23:31:25.565 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.558+0000 7fed33fff640 1 -- 172.21.15.80:0/2294937156 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed440c0090 con 0x5612860121a0 2024-04-24T23:31:25.565 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.558+0000 7fed33fff640 1 -- 172.21.15.80:0/2294937156 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed440b8570 con 0x5612860121a0 2024-04-24T23:31:25.565 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.558+0000 7fed53232740 1 -- 172.21.15.80:0/2294937156 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5612860121a0 msgr2=0x561286007d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:25.565 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.558+0000 7fed53232740 1 --2- 172.21.15.80:0/2294937156 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5612860121a0 0x561286007d90 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fed440bba90 tx=0x7fed440ba2a0 comp rx=0 tx=0).stop 2024-04-24T23:31:25.565 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.558+0000 7fed53232740 1 -- 172.21.15.80:0/2294937156 shutdown_connections 2024-04-24T23:31:25.565 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.558+0000 7fed53232740 1 --2- 172.21.15.80:0/2294937156 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5612860121a0 0x561286007d90 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:25.565 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.558+0000 7fed53232740 1 --2- 172.21.15.80:0/2294937156 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56128600d030 0x56128600bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:25.565 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.558+0000 7fed53232740 1 --2- 172.21.15.80:0/2294937156 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56128600c640 0x561285ffe9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:25.565 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.558+0000 7fed53232740 1 -- 172.21.15.80:0/2294937156 >> 172.21.15.80:0/2294937156 conn(0x5612860038a0 msgr2=0x561286002b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:25.565 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.559+0000 7fed53232740 1 -- 172.21.15.80:0/2294937156 shutdown_connections 2024-04-24T23:31:25.565 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.559+0000 7fed53232740 1 -- 172.21.15.80:0/2294937156 wait complete. 2024-04-24T23:31:25.566 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.559+0000 7fed53232740 1 -- start start 2024-04-24T23:31:25.566 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.559+0000 7fed53232740 1 Processor -- start 2024-04-24T23:31:25.566 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.559+0000 7fed53232740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56128600c640 0x561286020db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:25.566 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.559+0000 7fed53232740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56128600d030 0x5612860212f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:25.566 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.559+0000 7fed53232740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5612860121a0 0x5612860222b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:25.566 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.559+0000 7fed53232740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fed481a6900 con 0x5612860121a0 2024-04-24T23:31:25.566 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.559+0000 7fed53232740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fed481a7d70 con 0x56128600d030 2024-04-24T23:31:25.566 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.559+0000 7fed53232740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fed481a6fd0 con 0x56128600c640 2024-04-24T23:31:25.566 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.559+0000 7fed53232740 10 client.0 _ll_register_callbacks cb 0x7fed440b8030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:31:25.566 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.559+0000 7fed4e9e3640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5612860121a0 0x5612860222b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:25.566 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.559+0000 7fed4e9e3640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5612860121a0 0x5612860222b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38384/0 (socket says 172.21.15.80:38384) 2024-04-24T23:31:25.566 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.559+0000 7fed4e9e3640 1 -- 172.21.15.80:0/2811030457 learned_addr learned my addr 172.21.15.80:0/2811030457 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:31:25.566 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.559+0000 7fed4e1e2640 1 --2- 172.21.15.80:0/2811030457 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56128600c640 0x561286020db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:25.566 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.559+0000 7fed4d9e1640 1 --2- 172.21.15.80:0/2811030457 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56128600d030 0x5612860212f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:25.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.559+0000 7fed4e9e3640 1 -- 172.21.15.80:0/2811030457 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56128600c640 msgr2=0x561286020db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:25.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.559+0000 7fed4e9e3640 1 --2- 172.21.15.80:0/2811030457 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56128600c640 0x561286020db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:25.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.559+0000 7fed4e9e3640 1 -- 172.21.15.80:0/2811030457 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56128600d030 msgr2=0x5612860212f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:25.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.559+0000 7fed4e9e3640 1 --2- 172.21.15.80:0/2811030457 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56128600d030 0x5612860212f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:25.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.560+0000 7fed4e9e3640 1 -- 172.21.15.80:0/2811030457 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612860228a0 con 0x5612860121a0 2024-04-24T23:31:25.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.560+0000 7fed4d9e1640 1 --2- 172.21.15.80:0/2811030457 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56128600d030 0x5612860212f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:31:25.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.560+0000 7fed4e1e2640 1 --2- 172.21.15.80:0/2811030457 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56128600c640 0x561286020db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:31:25.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.560+0000 7fed4e9e3640 1 --2- 172.21.15.80:0/2811030457 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5612860121a0 0x5612860222b0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fed440ba040 tx=0x7fed440bb8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4629632775efe80 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:25.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.560+0000 7fed31ffb640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-04-24T23:31:25.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.560+0000 7fed31ffb640 1 -- 172.21.15.80:0/2811030457 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed4409f060 con 0x5612860121a0 2024-04-24T23:31:25.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.560+0000 7fed31ffb640 1 -- 172.21.15.80:0/2811030457 <== mon.0 v2:172.21.15.80:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fed440c46d0 con 0x5612860121a0 2024-04-24T23:31:25.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.560+0000 7fed53232740 1 -- 172.21.15.80:0/2811030457 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed440c0090 con 0x5612860121a0 2024-04-24T23:31:25.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.560+0000 7fed31ffb640 1 -- 172.21.15.80:0/2811030457 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fed44049070 con 0x5612860121a0 2024-04-24T23:31:25.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.560+0000 7fed53232740 10 client.15945 fetch_fsmap learned FSMap version 27 2024-04-24T23:31:25.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.560+0000 7fed53232740 1 -- 172.21.15.80:0/2811030457 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5612860228a0 con 0x5612860121a0 2024-04-24T23:31:25.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.560+0000 7fed31ffb640 1 -- 172.21.15.80:0/2811030457 <== mon.0 v2:172.21.15.80:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed440c0090 con 0x5612860121a0 2024-04-24T23:31:25.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.560+0000 7fed31ffb640 1 -- 172.21.15.80:0/2811030457 <== mon.0 v2:172.21.15.80:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fed440bc080 con 0x5612860121a0 2024-04-24T23:31:25.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.560+0000 7fed53232740 10 client.15945 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:31:25.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.560+0000 7fed53232740 10 client.15945 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:31:25.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.560+0000 7fed53232740 20 client.15945 populate_metadata read hostname 'smithi080' 2024-04-24T23:31:25.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.560+0000 7fed53232740 1 --2- 172.21.15.80:0/2811030457 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x56128608d1b0 0x56128608f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:25.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.560+0000 7fed53232740 4 client.15945 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:31:25.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.560+0000 7fed53232740 1 -- 172.21.15.80:0/2811030457 --> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561285f73f00 con 0x56128608d1b0 2024-04-24T23:31:25.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.560+0000 7fed4e1e2640 1 --2- 172.21.15.80:0/2811030457 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x56128608d1b0 0x56128608f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:25.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.561+0000 7fed4e1e2640 1 --2- 172.21.15.80:0/2811030457 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x56128608d1b0 0x56128608f5a0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca430841449d05bf server_cookie=ca184394d59be636 in_seq=0 out_seq=0 2024-04-24T23:31:25.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.561+0000 7fed31ffb640 10 client.15945 ms_handle_connect on v2:172.21.15.148:6838/2952757594 2024-04-24T23:31:25.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.561+0000 7fed31ffb640 1 -- 172.21.15.80:0/2811030457 <== mds.0 v2:172.21.15.148:6838/2952757594 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561285f73f00 con 0x56128608d1b0 2024-04-24T23:31:25.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.561+0000 7fed31ffb640 10 client.15945 handle_command_reply: tid=0 2024-04-24T23:31:25.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.561+0000 7fed53232740 1 client.15945 shutdown 2024-04-24T23:31:25.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.561+0000 7fed53232740 1 -- 172.21.15.80:0/2811030457 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5612860121a0 msgr2=0x5612860222b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:25.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.561+0000 7fed53232740 1 --2- 172.21.15.80:0/2811030457 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5612860121a0 0x5612860222b0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fed440ba040 tx=0x7fed440bb8d0 comp rx=0 tx=0).stop 2024-04-24T23:31:25.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.561+0000 7fed53232740 1 -- 172.21.15.80:0/2811030457 shutdown_connections 2024-04-24T23:31:25.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.561+0000 7fed53232740 1 --2- 172.21.15.80:0/2811030457 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x56128608d1b0 0x56128608f5a0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:25.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.561+0000 7fed53232740 1 --2- 172.21.15.80:0/2811030457 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5612860121a0 0x5612860222b0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:25.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.561+0000 7fed53232740 1 --2- 172.21.15.80:0/2811030457 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56128600d030 0x5612860212f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:25.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.561+0000 7fed53232740 1 --2- 172.21.15.80:0/2811030457 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56128600c640 0x561286020db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:25.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.561+0000 7fed31ffb640 0 client.15945 ms_handle_reset on v2:172.21.15.148:6838/2952757594 2024-04-24T23:31:25.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.561+0000 7fed53232740 1 -- 172.21.15.80:0/2811030457 >> 172.21.15.80:0/2811030457 conn(0x5612860038a0 msgr2=0x561286006890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:25.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.561+0000 7fed53232740 1 -- 172.21.15.80:0/2811030457 shutdown_connections 2024-04-24T23:31:25.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.561+0000 7fed53232740 1 -- 172.21.15.80:0/2811030457 wait complete. 2024-04-24T23:31:25.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.561+0000 7fed53232740 20 client.15945 trim_cache size 0 max 16384 2024-04-24T23:31:25.570 INFO:teuthology.orchestra.run.smithi080.stdout:[] 2024-04-24T23:31:25.570 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.562+0000 7fed50446640 1 -- 172.21.15.80:0/4170001648 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fed340a2b90 msgr2=0x7fed340a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:25.571 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.562+0000 7fed50446640 1 --2- 172.21.15.80:0/4170001648 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fed340a2b90 0x7fed340a5050 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fed481016f0 tx=0x7fed40009500 comp rx=0 tx=0).stop 2024-04-24T23:31:25.571 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.562+0000 7fed50446640 1 -- 172.21.15.80:0/4170001648 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fed48102250 msgr2=0x7fed4819f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:25.571 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.562+0000 7fed50446640 1 --2- 172.21.15.80:0/4170001648 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fed48102250 0x7fed4819f060 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fed440314e0 tx=0x7fed440042f0 comp rx=0 tx=0).stop 2024-04-24T23:31:25.571 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.562+0000 7fed50446640 1 -- 172.21.15.80:0/4170001648 shutdown_connections 2024-04-24T23:31:25.571 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.562+0000 7fed50446640 1 --2- 172.21.15.80:0/4170001648 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fed340a2b90 0x7fed340a5050 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:25.571 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.562+0000 7fed50446640 1 --2- 172.21.15.80:0/4170001648 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fed48102250 0x7fed4819f060 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:25.571 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.562+0000 7fed50446640 1 --2- 172.21.15.80:0/4170001648 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fed48101890 0x7fed48197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:25.571 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.562+0000 7fed50446640 1 --2- 172.21.15.80:0/4170001648 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed481006a0 0x7fed48197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:25.571 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.562+0000 7fed50446640 1 -- 172.21.15.80:0/4170001648 >> 172.21.15.80:0/4170001648 conn(0x7fed480fbc10 msgr2=0x7fed480fd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:25.571 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.562+0000 7fed50446640 1 -- 172.21.15.80:0/4170001648 shutdown_connections 2024-04-24T23:31:25.571 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:25.562+0000 7fed50446640 1 -- 172.21.15.80:0/4170001648 wait complete. 2024-04-24T23:31:26.287 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:31:26 smithi080 ceph-mon[29152]: pgmap v423: 129 pgs: 129 active+clean; 23 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 693 KiB/s rd, 897 KiB/s wr, 48 op/s 2024-04-24T23:31:26.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:31:26 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/3244216091' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:31:26.446 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:31:26 smithi134 ceph-mon[32619]: pgmap v423: 129 pgs: 129 active+clean; 23 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 693 KiB/s rd, 897 KiB/s wr, 48 op/s 2024-04-24T23:31:26.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:31:26 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/3244216091' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:31:26.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:31:26 smithi148 ceph-mon[32528]: pgmap v423: 129 pgs: 129 active+clean; 23 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 693 KiB/s rd, 897 KiB/s wr, 48 op/s 2024-04-24T23:31:26.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:31:26 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/3244216091' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:31:26.575 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T23:31:26.575 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:31:26.575 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T23:31:26.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.681+0000 7f2707e37640 1 -- 172.21.15.80:0/3600135762 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2700101870 msgr2=0x7f2700101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:26.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.681+0000 7f2707e37640 1 --2- 172.21.15.80:0/3600135762 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2700101870 0x7f2700101cf0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f26f0009900 tx=0x7f26f002a410 comp rx=0 tx=0).stop 2024-04-24T23:31:26.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.681+0000 7f2707e37640 1 -- 172.21.15.80:0/3600135762 shutdown_connections 2024-04-24T23:31:26.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.681+0000 7f2707e37640 1 --2- 172.21.15.80:0/3600135762 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2700102230 0x7f2700108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:26.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.681+0000 7f2707e37640 1 --2- 172.21.15.80:0/3600135762 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2700101870 0x7f2700101cf0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:26.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.681+0000 7f2707e37640 1 --2- 172.21.15.80:0/3600135762 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2700100680 0x7f2700100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:26.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.681+0000 7f2707e37640 1 -- 172.21.15.80:0/3600135762 >> 172.21.15.80:0/3600135762 conn(0x7f27000fbbf0 msgr2=0x7f27000fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:26.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.682+0000 7f2707e37640 1 -- 172.21.15.80:0/3600135762 shutdown_connections 2024-04-24T23:31:26.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.682+0000 7f2707e37640 1 -- 172.21.15.80:0/3600135762 wait complete. 2024-04-24T23:31:26.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.682+0000 7f2707e37640 1 Processor -- start 2024-04-24T23:31:26.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.682+0000 7f2707e37640 1 -- start start 2024-04-24T23:31:26.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.682+0000 7f2707e37640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2700100680 0x7f2700197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:26.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.682+0000 7f2707e37640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2700101870 0x7f27001980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:26.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.682+0000 7f2707e37640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2700102230 0x7f270019f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:26.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.682+0000 7f2707e37640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f270010c300 con 0x7f2700100680 2024-04-24T23:31:26.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.682+0000 7f2707e37640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f270010c180 con 0x7f2700102230 2024-04-24T23:31:26.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.682+0000 7f2707e37640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f270010c480 con 0x7f2700101870 2024-04-24T23:31:26.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.682+0000 7f2705bd3640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2700100680 0x7f2700197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:26.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.682+0000 7f2705bd3640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2700100680 0x7f2700197b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38396/0 (socket says 172.21.15.80:38396) 2024-04-24T23:31:26.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.682+0000 7f2705bd3640 1 -- 172.21.15.80:0/2892722512 learned_addr learned my addr 172.21.15.80:0/2892722512 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:31:26.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.682+0000 7f27063d4640 1 --2- 172.21.15.80:0/2892722512 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2700102230 0x7f270019f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:26.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.682+0000 7f27053d2640 1 --2- 172.21.15.80:0/2892722512 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2700101870 0x7f27001980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:26.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.683+0000 7f2705bd3640 1 -- 172.21.15.80:0/2892722512 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2700101870 msgr2=0x7f27001980d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:26.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.683+0000 7f2705bd3640 1 --2- 172.21.15.80:0/2892722512 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2700101870 0x7f27001980d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:26.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.683+0000 7f2705bd3640 1 -- 172.21.15.80:0/2892722512 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2700102230 msgr2=0x7f270019f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:26.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.683+0000 7f2705bd3640 1 --2- 172.21.15.80:0/2892722512 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2700102230 0x7f270019f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:26.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.683+0000 7f2705bd3640 1 -- 172.21.15.80:0/2892722512 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f270019f7b0 con 0x7f2700100680 2024-04-24T23:31:26.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.683+0000 7f27063d4640 1 --2- 172.21.15.80:0/2892722512 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2700102230 0x7f270019f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:31:26.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.683+0000 7f2705bd3640 1 --2- 172.21.15.80:0/2892722512 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2700100680 0x7f2700197b90 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f26fc00e9c0 tx=0x7f26fc00edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:26.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.683+0000 7f26f6ffd640 1 -- 172.21.15.80:0/2892722512 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26fc00ccc0 con 0x7f2700100680 2024-04-24T23:31:26.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.683+0000 7f26f6ffd640 1 -- 172.21.15.80:0/2892722512 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26fc0035f0 con 0x7f2700100680 2024-04-24T23:31:26.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.683+0000 7f2707e37640 1 -- 172.21.15.80:0/2892722512 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f270019faa0 con 0x7f2700100680 2024-04-24T23:31:26.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.683+0000 7f26f6ffd640 1 -- 172.21.15.80:0/2892722512 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26fc00ce60 con 0x7f2700100680 2024-04-24T23:31:26.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.683+0000 7f2707e37640 1 -- 172.21.15.80:0/2892722512 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f270019ff70 con 0x7f2700100680 2024-04-24T23:31:26.694 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.685+0000 7f26f6ffd640 1 -- 172.21.15.80:0/2892722512 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f26fc0037a0 con 0x7f2700100680 2024-04-24T23:31:26.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.686+0000 7f270ac17740 1 Processor -- start 2024-04-24T23:31:26.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.686+0000 7f270ac17740 1 -- start start 2024-04-24T23:31:26.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.686+0000 7f270ac17740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a039f77d90 0x55a039f781b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:26.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.686+0000 7f270ac17740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a039f78780 0x55a039f7d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:26.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.686+0000 7f270ac17740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a039f7d8f0 0x55a039f7fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:26.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.686+0000 7f270ac17740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x55a039edb2c0 con 0x55a039f7d8f0 2024-04-24T23:31:26.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.686+0000 7f270ac17740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55a039e83910 con 0x55a039f78780 2024-04-24T23:31:26.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.686+0000 7f270ac17740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55a039ed7280 con 0x55a039f77d90 2024-04-24T23:31:26.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.686+0000 7f26f6ffd640 1 --2- 172.21.15.80:0/2892722512 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f26ec0a2b90 0x7f26ec0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:26.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.686+0000 7f27053d2640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a039f7d8f0 0x55a039f7fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:26.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.686+0000 7f27063d4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a039f77d90 0x55a039f781b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:26.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.686+0000 7f27063d4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a039f77d90 0x55a039f781b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.80:38748/0 (socket says 172.21.15.80:38748) 2024-04-24T23:31:26.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.686+0000 7f27063d4640 1 -- 172.21.15.80:0/964809223 learned_addr learned my addr 172.21.15.80:0/964809223 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:31:26.696 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.686+0000 7f27063d4640 1 --2- 172.21.15.80:0/2892722512 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f26ec0a2b90 0x7f26ec0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:26.696 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.686+0000 7f2705bd3640 1 --2- 172.21.15.80:0/964809223 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a039f78780 0x55a039f7d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:26.696 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.686+0000 7f26f6ffd640 1 -- 172.21.15.80:0/2892722512 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f26fc0cf770 con 0x7f2700100680 2024-04-24T23:31:26.696 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.687+0000 7f27063d4640 1 -- 172.21.15.80:0/964809223 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a039f78780 msgr2=0x55a039f7d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:26.696 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.687+0000 7f27063d4640 1 --2- 172.21.15.80:0/964809223 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a039f78780 0x55a039f7d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:26.696 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.687+0000 7f27063d4640 1 -- 172.21.15.80:0/964809223 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a039f7d8f0 msgr2=0x55a039f7fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:26.696 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.687+0000 7f27063d4640 1 --2- 172.21.15.80:0/964809223 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a039f7d8f0 0x55a039f7fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:26.696 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.687+0000 7f27063d4640 1 -- 172.21.15.80:0/964809223 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a039e94fb0 con 0x55a039f77d90 2024-04-24T23:31:26.696 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.687+0000 7f2705bd3640 1 --2- 172.21.15.80:0/964809223 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a039f78780 0x55a039f7d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:31:26.696 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.687+0000 7f27053d2640 1 --2- 172.21.15.80:0/964809223 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a039f7d8f0 0x55a039f7fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:31:26.696 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.687+0000 7f27063d4640 1 --2- 172.21.15.80:0/2892722512 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f26ec0a2b90 0x7f26ec0a5050 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f26f800ffb0 tx=0x7f26f801e000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:26.696 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.687+0000 7f27063d4640 1 --2- 172.21.15.80:0/964809223 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a039f77d90 0x55a039f781b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f26f800f5f0 tx=0x7f26f800fab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d019bdd353bd31f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:26.696 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.687+0000 7f26ebfff640 1 -- 172.21.15.80:0/964809223 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26f8022070 con 0x55a039f77d90 2024-04-24T23:31:26.696 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.687+0000 7f26ebfff640 1 -- 172.21.15.80:0/964809223 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26f8003e50 con 0x55a039f77d90 2024-04-24T23:31:26.697 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.687+0000 7f26ebfff640 1 -- 172.21.15.80:0/964809223 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26f8026410 con 0x55a039f77d90 2024-04-24T23:31:26.697 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.688+0000 7f270ac17740 1 -- 172.21.15.80:0/964809223 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a039f77d90 msgr2=0x55a039f781b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:26.697 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.688+0000 7f270ac17740 1 --2- 172.21.15.80:0/964809223 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a039f77d90 0x55a039f781b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f26f800f5f0 tx=0x7f26f800fab0 comp rx=0 tx=0).stop 2024-04-24T23:31:26.697 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.688+0000 7f270ac17740 1 -- 172.21.15.80:0/964809223 shutdown_connections 2024-04-24T23:31:26.697 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.688+0000 7f270ac17740 1 --2- 172.21.15.80:0/964809223 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a039f7d8f0 0x55a039f7fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:26.697 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.688+0000 7f270ac17740 1 --2- 172.21.15.80:0/964809223 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a039f78780 0x55a039f7d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:26.697 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.688+0000 7f270ac17740 1 --2- 172.21.15.80:0/964809223 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a039f77d90 0x55a039f781b0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:26.697 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.688+0000 7f270ac17740 1 -- 172.21.15.80:0/964809223 >> 172.21.15.80:0/964809223 conn(0x55a039f3f950 msgr2=0x55a039f73810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:26.697 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.688+0000 7f270ac17740 1 -- 172.21.15.80:0/964809223 shutdown_connections 2024-04-24T23:31:26.697 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.688+0000 7f270ac17740 1 -- 172.21.15.80:0/964809223 wait complete. 2024-04-24T23:31:26.697 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.688+0000 7f270ac17740 1 -- start start 2024-04-24T23:31:26.697 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.688+0000 7f270ac17740 1 Processor -- start 2024-04-24T23:31:26.697 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.689+0000 7f270ac17740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a039f77d90 0x55a039f821c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:26.697 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.689+0000 7f270ac17740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a039f78780 0x55a039f82700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:26.697 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.689+0000 7f270ac17740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a039f7d8f0 0x55a039f896d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:26.698 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.689+0000 7f270ac17740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x55a039f807e0 con 0x55a039f78780 2024-04-24T23:31:26.698 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.689+0000 7f270ac17740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55a039e96e90 con 0x55a039f7d8f0 2024-04-24T23:31:26.698 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.689+0000 7f270ac17740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55a039f80960 con 0x55a039f77d90 2024-04-24T23:31:26.698 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.689+0000 7f270ac17740 10 client.0 _ll_register_callbacks cb 0x55a039ed2970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:31:26.698 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.689+0000 7f27053d2640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a039f78780 0x55a039f82700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:26.698 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.689+0000 7f27053d2640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a039f78780 0x55a039f82700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38422/0 (socket says 172.21.15.80:38422) 2024-04-24T23:31:26.698 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.689+0000 7f27053d2640 1 -- 172.21.15.80:0/1792556753 learned_addr learned my addr 172.21.15.80:0/1792556753 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:31:26.698 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.689+0000 7f2705bd3640 1 --2- 172.21.15.80:0/1792556753 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a039f77d90 0x55a039f821c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:26.698 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.689+0000 7f27063d4640 1 --2- 172.21.15.80:0/1792556753 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a039f7d8f0 0x55a039f896d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:26.698 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.689+0000 7f27053d2640 1 -- 172.21.15.80:0/1792556753 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a039f77d90 msgr2=0x55a039f821c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:26.698 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.689+0000 7f27053d2640 1 --2- 172.21.15.80:0/1792556753 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a039f77d90 0x55a039f821c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:26.698 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.689+0000 7f27053d2640 1 -- 172.21.15.80:0/1792556753 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a039f7d8f0 msgr2=0x55a039f896d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:26.698 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.689+0000 7f27053d2640 1 --2- 172.21.15.80:0/1792556753 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a039f7d8f0 0x55a039f896d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:26.699 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.689+0000 7f27053d2640 1 -- 172.21.15.80:0/1792556753 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a039cb77e0 con 0x55a039f78780 2024-04-24T23:31:26.699 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.689+0000 7f27063d4640 1 --2- 172.21.15.80:0/1792556753 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a039f7d8f0 0x55a039f896d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:31:26.699 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.689+0000 7f27053d2640 1 --2- 172.21.15.80:0/1792556753 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a039f78780 0x55a039f82700 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f26f0031610 tx=0x7f26f0032eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19d86bfbb7d18702 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:26.699 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.689+0000 7f26e97fa640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-04-24T23:31:26.699 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.689+0000 7f26e97fa640 1 -- 172.21.15.80:0/1792556753 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26f0035040 con 0x55a039f78780 2024-04-24T23:31:26.699 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.689+0000 7f26e97fa640 1 -- 172.21.15.80:0/1792556753 <== mon.0 v2:172.21.15.80:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f26f0031930 con 0x55a039f78780 2024-04-24T23:31:26.699 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.689+0000 7f270ac17740 1 -- 172.21.15.80:0/1792556753 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26f8003e50 con 0x55a039f78780 2024-04-24T23:31:26.699 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.690+0000 7f26e97fa640 1 -- 172.21.15.80:0/1792556753 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f26f0004040 con 0x55a039f78780 2024-04-24T23:31:26.699 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.690+0000 7f270ac17740 10 client.15957 fetch_fsmap learned FSMap version 27 2024-04-24T23:31:26.699 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.690+0000 7f270ac17740 1 -- 172.21.15.80:0/1792556753 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a039d17f70 con 0x55a039f78780 2024-04-24T23:31:26.699 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.690+0000 7f26e97fa640 1 -- 172.21.15.80:0/1792556753 <== mon.0 v2:172.21.15.80:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26f8003e50 con 0x55a039f78780 2024-04-24T23:31:26.699 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.690+0000 7f26e97fa640 1 -- 172.21.15.80:0/1792556753 <== mon.0 v2:172.21.15.80:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f26f0004900 con 0x55a039f78780 2024-04-24T23:31:26.699 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.690+0000 7f270ac17740 10 client.15957 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:31:26.699 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.690+0000 7f270ac17740 10 client.15957 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:31:26.700 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.690+0000 7f270ac17740 20 client.15957 populate_metadata read hostname 'smithi080' 2024-04-24T23:31:26.700 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.690+0000 7f270ac17740 1 --2- 172.21.15.80:0/1792556753 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x55a039f8dcc0 0x55a039f900f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:26.700 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.690+0000 7f270ac17740 4 client.15957 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:31:26.700 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.690+0000 7f270ac17740 1 -- 172.21.15.80:0/1792556753 --> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a039eaf5a0 con 0x55a039f8dcc0 2024-04-24T23:31:26.700 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.690+0000 7f2705bd3640 1 --2- 172.21.15.80:0/1792556753 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x55a039f8dcc0 0x55a039f900f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:26.700 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.690+0000 7f2705bd3640 1 --2- 172.21.15.80:0/1792556753 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x55a039f8dcc0 0x55a039f900f0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dafe31b9e0effee3 server_cookie=6e799826a8036831 in_seq=0 out_seq=0 2024-04-24T23:31:26.700 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.690+0000 7f26e97fa640 10 client.15957 ms_handle_connect on v2:172.21.15.148:6838/2952757594 2024-04-24T23:31:26.700 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.692+0000 7f26e97fa640 1 -- 172.21.15.80:0/1792556753 <== mds.0 v2:172.21.15.148:6838/2952757594 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a039eaf5a0 con 0x55a039f8dcc0 2024-04-24T23:31:26.700 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.692+0000 7f26e97fa640 10 client.15957 handle_command_reply: tid=0 2024-04-24T23:31:26.700 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.692+0000 7f270ac17740 1 client.15957 shutdown 2024-04-24T23:31:26.700 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.692+0000 7f270ac17740 1 -- 172.21.15.80:0/1792556753 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a039f78780 msgr2=0x55a039f82700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:26.700 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.692+0000 7f270ac17740 1 --2- 172.21.15.80:0/1792556753 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a039f78780 0x55a039f82700 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f26f0031610 tx=0x7f26f0032eb0 comp rx=0 tx=0).stop 2024-04-24T23:31:26.700 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.692+0000 7f270ac17740 1 -- 172.21.15.80:0/1792556753 shutdown_connections 2024-04-24T23:31:26.700 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.692+0000 7f270ac17740 1 --2- 172.21.15.80:0/1792556753 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x55a039f8dcc0 0x55a039f900f0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:26.700 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.692+0000 7f270ac17740 1 --2- 172.21.15.80:0/1792556753 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a039f7d8f0 0x55a039f896d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:26.701 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.692+0000 7f270ac17740 1 --2- 172.21.15.80:0/1792556753 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a039f78780 0x55a039f82700 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:26.701 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.692+0000 7f270ac17740 1 --2- 172.21.15.80:0/1792556753 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a039f77d90 0x55a039f821c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:26.701 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.692+0000 7f270ac17740 1 -- 172.21.15.80:0/1792556753 >> 172.21.15.80:0/1792556753 conn(0x55a039f3f950 msgr2=0x55a039f72950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:26.701 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.692+0000 7f26e97fa640 0 client.15957 ms_handle_reset on v2:172.21.15.148:6838/2952757594 2024-04-24T23:31:26.701 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.692+0000 7f270ac17740 1 -- 172.21.15.80:0/1792556753 shutdown_connections 2024-04-24T23:31:26.701 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.692+0000 7f270ac17740 1 -- 172.21.15.80:0/1792556753 wait complete. 2024-04-24T23:31:26.701 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.692+0000 7f270ac17740 20 client.15957 trim_cache size 0 max 16384 2024-04-24T23:31:26.715 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.710+0000 7f270ac17740 1 Processor -- start 2024-04-24T23:31:26.715 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.710+0000 7f270ac17740 1 -- start start 2024-04-24T23:31:26.715 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.710+0000 7f270ac17740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a039f77d90 0x55a039f720c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:26.715 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.710+0000 7f270ac17740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a039f78780 0x55a039f89690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:26.715 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.710+0000 7f270ac17740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a039f7d8f0 0x55a039f72600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:26.715 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.710+0000 7f270ac17740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f27001acde0 con 0x55a039f78780 2024-04-24T23:31:26.716 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.710+0000 7f270ac17740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f27001aa6b0 con 0x55a039f7d8f0 2024-04-24T23:31:26.716 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.710+0000 7f270ac17740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f27001a8500 con 0x55a039f77d90 2024-04-24T23:31:26.716 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.710+0000 7f27053d2640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a039f78780 0x55a039f89690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:26.716 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.710+0000 7f27053d2640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a039f78780 0x55a039f89690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38430/0 (socket says 172.21.15.80:38430) 2024-04-24T23:31:26.716 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.710+0000 7f27053d2640 1 -- 172.21.15.80:0/697874393 learned_addr learned my addr 172.21.15.80:0/697874393 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:31:26.716 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.710+0000 7f2705bd3640 1 --2- 172.21.15.80:0/697874393 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a039f77d90 0x55a039f720c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:26.716 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.710+0000 7f27063d4640 1 --2- 172.21.15.80:0/697874393 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a039f7d8f0 0x55a039f72600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:26.716 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.711+0000 7f27053d2640 1 -- 172.21.15.80:0/697874393 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a039f77d90 msgr2=0x55a039f720c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:26.716 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.711+0000 7f27053d2640 1 --2- 172.21.15.80:0/697874393 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a039f77d90 0x55a039f720c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:26.716 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.711+0000 7f27053d2640 1 -- 172.21.15.80:0/697874393 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a039f7d8f0 msgr2=0x55a039f72600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:26.716 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.711+0000 7f27053d2640 1 --2- 172.21.15.80:0/697874393 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a039f7d8f0 0x55a039f72600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:26.716 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.711+0000 7f27053d2640 1 -- 172.21.15.80:0/697874393 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a039baf170 con 0x55a039f78780 2024-04-24T23:31:26.716 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.711+0000 7f27053d2640 1 --2- 172.21.15.80:0/697874393 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a039f78780 0x55a039f89690 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f26f002a930 tx=0x7f26f0005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a89d1c3022d97005 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:26.716 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.711+0000 7f26eb7fe640 1 -- 172.21.15.80:0/697874393 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26f0059070 con 0x55a039f78780 2024-04-24T23:31:26.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.711+0000 7f26eb7fe640 1 -- 172.21.15.80:0/697874393 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26f00041e0 con 0x55a039f78780 2024-04-24T23:31:26.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.711+0000 7f26eb7fe640 1 -- 172.21.15.80:0/697874393 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26f0035030 con 0x55a039f78780 2024-04-24T23:31:26.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.711+0000 7f270ac17740 1 -- 172.21.15.80:0/697874393 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a039f78780 msgr2=0x55a039f89690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:26.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.711+0000 7f270ac17740 1 --2- 172.21.15.80:0/697874393 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a039f78780 0x55a039f89690 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f26f002a930 tx=0x7f26f0005c90 comp rx=0 tx=0).stop 2024-04-24T23:31:26.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.711+0000 7f270ac17740 1 -- 172.21.15.80:0/697874393 shutdown_connections 2024-04-24T23:31:26.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.711+0000 7f270ac17740 1 --2- 172.21.15.80:0/697874393 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a039f7d8f0 0x55a039f72600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:26.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.712+0000 7f270ac17740 1 --2- 172.21.15.80:0/697874393 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a039f78780 0x55a039f89690 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:26.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.712+0000 7f270ac17740 1 --2- 172.21.15.80:0/697874393 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a039f77d90 0x55a039f720c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:26.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.712+0000 7f270ac17740 1 -- 172.21.15.80:0/697874393 >> 172.21.15.80:0/697874393 conn(0x55a039f3f950 msgr2=0x55a039f79be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:26.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.712+0000 7f270ac17740 1 -- 172.21.15.80:0/697874393 shutdown_connections 2024-04-24T23:31:26.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.712+0000 7f270ac17740 1 -- 172.21.15.80:0/697874393 wait complete. 2024-04-24T23:31:26.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.712+0000 7f270ac17740 1 -- start start 2024-04-24T23:31:26.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.712+0000 7f270ac17740 1 Processor -- start 2024-04-24T23:31:26.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.712+0000 7f270ac17740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a039f77d90 0x55a039f875a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:26.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.712+0000 7f270ac17740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a039f78780 0x55a039f87ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:26.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.712+0000 7f270ac17740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a039f7d8f0 0x55a039f88af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:26.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.712+0000 7f270ac17740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f27001a8b70 con 0x55a039f77d90 2024-04-24T23:31:26.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.712+0000 7f270ac17740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f27001a9fe0 con 0x55a039f7d8f0 2024-04-24T23:31:26.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.712+0000 7f270ac17740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f27001a9240 con 0x55a039f78780 2024-04-24T23:31:26.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.712+0000 7f270ac17740 10 client.0 _ll_register_callbacks cb 0x7f26f0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:31:26.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.712+0000 7f2705bd3640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a039f77d90 0x55a039f875a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:26.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.712+0000 7f2705bd3640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a039f77d90 0x55a039f875a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38440/0 (socket says 172.21.15.80:38440) 2024-04-24T23:31:26.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.712+0000 7f2705bd3640 1 -- 172.21.15.80:0/39460066 learned_addr learned my addr 172.21.15.80:0/39460066 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:31:26.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.712+0000 7f27053d2640 1 --2- 172.21.15.80:0/39460066 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a039f78780 0x55a039f87ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:26.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.712+0000 7f27063d4640 1 --2- 172.21.15.80:0/39460066 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a039f7d8f0 0x55a039f88af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:26.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.713+0000 7f2705bd3640 1 -- 172.21.15.80:0/39460066 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a039f78780 msgr2=0x55a039f87ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:26.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.713+0000 7f2705bd3640 1 --2- 172.21.15.80:0/39460066 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a039f78780 0x55a039f87ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:26.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.713+0000 7f2705bd3640 1 -- 172.21.15.80:0/39460066 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a039f7d8f0 msgr2=0x55a039f88af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:26.719 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.713+0000 7f2705bd3640 1 --2- 172.21.15.80:0/39460066 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a039f7d8f0 0x55a039f88af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:26.719 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.713+0000 7f2705bd3640 1 -- 172.21.15.80:0/39460066 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a039f890e0 con 0x55a039f77d90 2024-04-24T23:31:26.719 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.713+0000 7f2705bd3640 1 --2- 172.21.15.80:0/39460066 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a039f77d90 0x55a039f875a0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f26fc004630 tx=0x7f26fc01c590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ddce9352d322a40 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:26.719 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.713+0000 7f26e97fa640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-04-24T23:31:26.719 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.713+0000 7f26e97fa640 1 -- 172.21.15.80:0/39460066 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26fc0a4870 con 0x55a039f77d90 2024-04-24T23:31:26.719 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.713+0000 7f26e97fa640 1 -- 172.21.15.80:0/39460066 <== mon.0 v2:172.21.15.80:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f26fc003b10 con 0x55a039f77d90 2024-04-24T23:31:26.719 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.713+0000 7f270ac17740 1 -- 172.21.15.80:0/39460066 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26f00041e0 con 0x55a039f77d90 2024-04-24T23:31:26.719 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.713+0000 7f26e97fa640 1 -- 172.21.15.80:0/39460066 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f26fc00a250 con 0x55a039f77d90 2024-04-24T23:31:26.719 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.713+0000 7f270ac17740 10 client.15963 fetch_fsmap learned FSMap version 27 2024-04-24T23:31:26.719 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.713+0000 7f270ac17740 1 -- 172.21.15.80:0/39460066 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a039ff4b40 con 0x55a039f77d90 2024-04-24T23:31:26.719 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.713+0000 7f26e97fa640 1 -- 172.21.15.80:0/39460066 <== mon.0 v2:172.21.15.80:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26f00041e0 con 0x55a039f77d90 2024-04-24T23:31:26.719 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.713+0000 7f26e97fa640 1 -- 172.21.15.80:0/39460066 <== mon.0 v2:172.21.15.80:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f26fc0a43e0 con 0x55a039f77d90 2024-04-24T23:31:26.719 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.713+0000 7f270ac17740 10 client.15963 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:31:26.719 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.714+0000 7f270ac17740 10 client.15963 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:31:26.719 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.714+0000 7f270ac17740 20 client.15963 populate_metadata read hostname 'smithi080' 2024-04-24T23:31:26.720 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.714+0000 7f270ac17740 1 --2- 172.21.15.80:0/39460066 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x55a039ff5520 0x55a039ff7940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:26.722 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.714+0000 7f270ac17740 4 client.15963 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T23:31:26.722 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.714+0000 7f270ac17740 1 -- 172.21.15.80:0/39460066 --> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55a039ee8800 con 0x55a039ff5520 2024-04-24T23:31:26.722 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.714+0000 7f27053d2640 1 --2- 172.21.15.80:0/39460066 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x55a039ff5520 0x55a039ff7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:26.722 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.714+0000 7f26e97fa640 10 client.15963 ms_handle_connect on v2:172.21.15.148:6838/2952757594 2024-04-24T23:31:26.722 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.714+0000 7f27053d2640 1 --2- 172.21.15.80:0/39460066 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x55a039ff5520 0x55a039ff7940 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c69c6acd73e1641e server_cookie=90163fe209217cab in_seq=0 out_seq=0 2024-04-24T23:31:26.724 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.715+0000 7f26e97fa640 1 -- 172.21.15.80:0/39460066 <== mds.0 v2:172.21.15.148:6838/2952757594 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a039ee8800 con 0x55a039ff5520 2024-04-24T23:31:26.724 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.715+0000 7f26e97fa640 10 client.15963 handle_command_reply: tid=0 2024-04-24T23:31:26.724 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.715+0000 7f270ac17740 1 client.15963 shutdown 2024-04-24T23:31:26.724 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.715+0000 7f270ac17740 1 -- 172.21.15.80:0/39460066 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a039f77d90 msgr2=0x55a039f875a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:26.724 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.715+0000 7f270ac17740 1 --2- 172.21.15.80:0/39460066 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a039f77d90 0x55a039f875a0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f26fc004630 tx=0x7f26fc01c590 comp rx=0 tx=0).stop 2024-04-24T23:31:26.724 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.716+0000 7f270ac17740 1 -- 172.21.15.80:0/39460066 shutdown_connections 2024-04-24T23:31:26.724 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.716+0000 7f270ac17740 1 --2- 172.21.15.80:0/39460066 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x55a039ff5520 0x55a039ff7940 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:26.724 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.716+0000 7f270ac17740 1 --2- 172.21.15.80:0/39460066 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a039f7d8f0 0x55a039f88af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:26.724 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.716+0000 7f270ac17740 1 --2- 172.21.15.80:0/39460066 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a039f78780 0x55a039f87ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:26.725 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.716+0000 7f270ac17740 1 --2- 172.21.15.80:0/39460066 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a039f77d90 0x55a039f875a0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:26.725 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.716+0000 7f26e97fa640 0 client.15963 ms_handle_reset on v2:172.21.15.148:6838/2952757594 2024-04-24T23:31:26.725 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.716+0000 7f270ac17740 1 -- 172.21.15.80:0/39460066 >> 172.21.15.80:0/39460066 conn(0x55a039f3f950 msgr2=0x55a039f764d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:26.725 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.716+0000 7f270ac17740 1 -- 172.21.15.80:0/39460066 shutdown_connections 2024-04-24T23:31:26.725 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.716+0000 7f270ac17740 1 -- 172.21.15.80:0/39460066 wait complete. 2024-04-24T23:31:26.725 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.716+0000 7f270ac17740 20 client.15963 trim_cache size 0 max 16384 2024-04-24T23:31:26.725 INFO:teuthology.orchestra.run.smithi080.stdout:{ 2024-04-24T23:31:26.725 INFO:teuthology.orchestra.run.smithi080.stdout: "return_code": 0, 2024-04-24T23:31:26.725 INFO:teuthology.orchestra.run.smithi080.stdout: "scrub_tag": "be057515-fa5a-4f78-b988-4a14f1292aa7", 2024-04-24T23:31:26.726 INFO:teuthology.orchestra.run.smithi080.stdout: "mode": "asynchronous" 2024-04-24T23:31:26.726 INFO:teuthology.orchestra.run.smithi080.stdout:} 2024-04-24T23:31:26.726 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.716+0000 7f2707e37640 1 -- 172.21.15.80:0/2892722512 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f26ec0a2b90 msgr2=0x7f26ec0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:26.727 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.716+0000 7f2707e37640 1 --2- 172.21.15.80:0/2892722512 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f26ec0a2b90 0x7f26ec0a5050 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f26f800ffb0 tx=0x7f26f801e000 comp rx=0 tx=0).stop 2024-04-24T23:31:26.727 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.716+0000 7f2707e37640 1 -- 172.21.15.80:0/2892722512 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2700100680 msgr2=0x7f2700197b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:26.727 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.716+0000 7f2707e37640 1 --2- 172.21.15.80:0/2892722512 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2700100680 0x7f2700197b90 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f26fc00e9c0 tx=0x7f26fc00edc0 comp rx=0 tx=0).stop 2024-04-24T23:31:26.727 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.716+0000 7f2707e37640 1 -- 172.21.15.80:0/2892722512 shutdown_connections 2024-04-24T23:31:26.727 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.716+0000 7f2707e37640 1 --2- 172.21.15.80:0/2892722512 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f26ec0a2b90 0x7f26ec0a5050 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:26.727 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.716+0000 7f2707e37640 1 --2- 172.21.15.80:0/2892722512 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2700102230 0x7f270019f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:26.727 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.716+0000 7f2707e37640 1 --2- 172.21.15.80:0/2892722512 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2700101870 0x7f27001980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:26.727 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.716+0000 7f2707e37640 1 --2- 172.21.15.80:0/2892722512 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2700100680 0x7f2700197b90 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:26.727 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.716+0000 7f2707e37640 1 -- 172.21.15.80:0/2892722512 >> 172.21.15.80:0/2892722512 conn(0x7f27000fbbf0 msgr2=0x7f27000fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:26.727 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.716+0000 7f2707e37640 1 -- 172.21.15.80:0/2892722512 shutdown_connections 2024-04-24T23:31:26.727 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.716+0000 7f2707e37640 1 -- 172.21.15.80:0/2892722512 wait complete. 2024-04-24T23:31:26.728 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T23:31:26.729 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T23:31:26.835 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.833+0000 7fa329d9e640 1 -- 172.21.15.80:0/2911562077 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa324102230 msgr2=0x7fa324108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:26.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.833+0000 7fa329d9e640 1 --2- 172.21.15.80:0/2911562077 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa324102230 0x7fa324108ac0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fa31c009ca0 tx=0x7fa31c02a8c0 comp rx=0 tx=0).stop 2024-04-24T23:31:26.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.833+0000 7fa329d9e640 1 -- 172.21.15.80:0/2911562077 shutdown_connections 2024-04-24T23:31:26.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.833+0000 7fa329d9e640 1 --2- 172.21.15.80:0/2911562077 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa324102230 0x7fa324108ac0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:26.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.833+0000 7fa329d9e640 1 --2- 172.21.15.80:0/2911562077 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa324101870 0x7fa324101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:26.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.833+0000 7fa329d9e640 1 --2- 172.21.15.80:0/2911562077 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa324100680 0x7fa324100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:26.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.833+0000 7fa329d9e640 1 -- 172.21.15.80:0/2911562077 >> 172.21.15.80:0/2911562077 conn(0x7fa3240fbbf0 msgr2=0x7fa3240fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:26.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.833+0000 7fa329d9e640 1 -- 172.21.15.80:0/2911562077 shutdown_connections 2024-04-24T23:31:26.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.833+0000 7fa329d9e640 1 -- 172.21.15.80:0/2911562077 wait complete. 2024-04-24T23:31:26.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.833+0000 7fa329d9e640 1 Processor -- start 2024-04-24T23:31:26.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.833+0000 7fa329d9e640 1 -- start start 2024-04-24T23:31:26.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.834+0000 7fa329d9e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa324100680 0x7fa324069440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:26.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.834+0000 7fa329d9e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa324101870 0x7fa32419c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:26.837 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.834+0000 7fa329d9e640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa324102230 0x7fa32419ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:26.837 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.834+0000 7fa329d9e640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fa32410c300 con 0x7fa324102230 2024-04-24T23:31:26.837 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.834+0000 7fa329d9e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa32410c180 con 0x7fa324100680 2024-04-24T23:31:26.837 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.834+0000 7fa329d9e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa32410c480 con 0x7fa324101870 2024-04-24T23:31:26.837 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.834+0000 7fa323fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa324102230 0x7fa32419ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:26.837 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.834+0000 7fa323fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa324102230 0x7fa32419ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38466/0 (socket says 172.21.15.80:38466) 2024-04-24T23:31:26.837 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.834+0000 7fa323fff640 1 -- 172.21.15.80:0/3128642967 learned_addr learned my addr 172.21.15.80:0/3128642967 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:31:26.837 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.834+0000 7fa322ffd640 1 --2- 172.21.15.80:0/3128642967 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa324101870 0x7fa32419c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:26.837 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.834+0000 7fa3237fe640 1 --2- 172.21.15.80:0/3128642967 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa324100680 0x7fa324069440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:26.837 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.834+0000 7fa323fff640 1 -- 172.21.15.80:0/3128642967 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa324101870 msgr2=0x7fa32419c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:26.837 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.834+0000 7fa323fff640 1 --2- 172.21.15.80:0/3128642967 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa324101870 0x7fa32419c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:26.837 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.834+0000 7fa323fff640 1 -- 172.21.15.80:0/3128642967 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa324100680 msgr2=0x7fa324069440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:26.838 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.834+0000 7fa323fff640 1 --2- 172.21.15.80:0/3128642967 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa324100680 0x7fa324069440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:26.838 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.834+0000 7fa323fff640 1 -- 172.21.15.80:0/3128642967 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa32419f6f0 con 0x7fa324102230 2024-04-24T23:31:26.838 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.834+0000 7fa323fff640 1 --2- 172.21.15.80:0/3128642967 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa324102230 0x7fa32419ede0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fa31c031ae0 tx=0x7fa31c031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:26.838 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.834+0000 7fa320ff9640 1 -- 172.21.15.80:0/3128642967 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa31c003c30 con 0x7fa324102230 2024-04-24T23:31:26.838 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.834+0000 7fa329d9e640 1 -- 172.21.15.80:0/3128642967 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa32419f920 con 0x7fa324102230 2024-04-24T23:31:26.838 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.834+0000 7fa320ff9640 1 -- 172.21.15.80:0/3128642967 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa31c003dd0 con 0x7fa324102230 2024-04-24T23:31:26.838 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.834+0000 7fa329d9e640 1 -- 172.21.15.80:0/3128642967 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa32419fec0 con 0x7fa324102230 2024-04-24T23:31:26.838 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.834+0000 7fa320ff9640 1 -- 172.21.15.80:0/3128642967 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa31c02fc50 con 0x7fa324102230 2024-04-24T23:31:26.845 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.837+0000 7fa320ff9640 1 -- 172.21.15.80:0/3128642967 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa31c004000 con 0x7fa324102230 2024-04-24T23:31:26.845 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.837+0000 7fa32cb85740 1 Processor -- start 2024-04-24T23:31:26.845 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.837+0000 7fa32cb85740 1 -- start start 2024-04-24T23:31:26.845 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.837+0000 7fa32cb85740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560eeb3ad650 0x560eeb3ada70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:26.846 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.837+0000 7fa32cb85740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560eeb3ae040 0x560eeb3b2b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:26.846 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.837+0000 7fa32cb85740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560eeb3b31b0 0x560eeb3b5640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:26.846 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.837+0000 7fa32cb85740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x560eeb30d6b0 con 0x560eeb3ad650 2024-04-24T23:31:26.846 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.837+0000 7fa32cb85740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x560eeb2b5880 con 0x560eeb3b31b0 2024-04-24T23:31:26.846 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.837+0000 7fa32cb85740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x560eeb309670 con 0x560eeb3ae040 2024-04-24T23:31:26.846 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.837+0000 7fa322ffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560eeb3ad650 0x560eeb3ada70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:26.846 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.837+0000 7fa322ffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560eeb3ad650 0x560eeb3ada70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38478/0 (socket says 172.21.15.80:38478) 2024-04-24T23:31:26.846 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.837+0000 7fa3237fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560eeb3ae040 0x560eeb3b2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:26.846 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.837+0000 7fa322ffd640 1 -- 172.21.15.80:0/1509070782 learned_addr learned my addr 172.21.15.80:0/1509070782 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:31:26.846 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.837+0000 7fa320ff9640 1 --2- 172.21.15.80:0/3128642967 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fa30c0a2b90 0x7fa30c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:26.846 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.837+0000 7fa3237fe640 1 -- 172.21.15.80:0/1509070782 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560eeb3b31b0 msgr2=0x560eeb3b5640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:26.846 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.837+0000 7fa3237fe640 1 --2- 172.21.15.80:0/1509070782 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560eeb3b31b0 0x560eeb3b5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:26.847 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.837+0000 7fa323fff640 1 --2- 172.21.15.80:0/3128642967 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fa30c0a2b90 0x7fa30c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:26.847 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.837+0000 7fa3237fe640 1 -- 172.21.15.80:0/1509070782 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560eeb3ad650 msgr2=0x560eeb3ada70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:26.847 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.837+0000 7fa3237fe640 1 --2- 172.21.15.80:0/1509070782 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560eeb3ad650 0x560eeb3ada70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:26.847 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.837+0000 7fa3237fe640 1 -- 172.21.15.80:0/1509070782 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560eeb0eb7d0 con 0x560eeb3ae040 2024-04-24T23:31:26.847 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.837+0000 7fa322ffd640 1 --2- 172.21.15.80:0/1509070782 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560eeb3ad650 0x560eeb3ada70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:31:26.847 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.838+0000 7fa320ff9640 1 -- 172.21.15.80:0/3128642967 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fa31c0f0760 con 0x7fa324102230 2024-04-24T23:31:26.847 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.838+0000 7fa3237fe640 1 --2- 172.21.15.80:0/1509070782 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560eeb3ae040 0x560eeb3b2b40 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa314009bd0 tx=0x7fa314011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b39f25f1dc6e9364 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:26.847 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.838+0000 7fa309ffb640 1 -- 172.21.15.80:0/1509070782 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa314004230 con 0x560eeb3ae040 2024-04-24T23:31:26.847 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.838+0000 7fa323fff640 1 --2- 172.21.15.80:0/3128642967 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fa30c0a2b90 0x7fa30c0a5050 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fa31c0b9050 tx=0x7fa31c08fcb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:26.847 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.838+0000 7fa309ffb640 1 -- 172.21.15.80:0/1509070782 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa314009e40 con 0x560eeb3ae040 2024-04-24T23:31:26.847 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.838+0000 7fa309ffb640 1 -- 172.21.15.80:0/1509070782 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3140197e0 con 0x560eeb3ae040 2024-04-24T23:31:26.847 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.838+0000 7fa32cb85740 1 -- 172.21.15.80:0/1509070782 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560eeb3ae040 msgr2=0x560eeb3b2b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:26.847 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.838+0000 7fa32cb85740 1 --2- 172.21.15.80:0/1509070782 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560eeb3ae040 0x560eeb3b2b40 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa314009bd0 tx=0x7fa314011810 comp rx=0 tx=0).stop 2024-04-24T23:31:26.848 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.839+0000 7fa32cb85740 1 -- 172.21.15.80:0/1509070782 shutdown_connections 2024-04-24T23:31:26.848 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.839+0000 7fa32cb85740 1 --2- 172.21.15.80:0/1509070782 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560eeb3b31b0 0x560eeb3b5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:26.848 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.839+0000 7fa32cb85740 1 --2- 172.21.15.80:0/1509070782 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560eeb3ae040 0x560eeb3b2b40 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:26.848 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.839+0000 7fa32cb85740 1 --2- 172.21.15.80:0/1509070782 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560eeb3ad650 0x560eeb3ada70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:26.848 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.839+0000 7fa32cb85740 1 -- 172.21.15.80:0/1509070782 >> 172.21.15.80:0/1509070782 conn(0x560eeb3a48b0 msgr2=0x560eeb3a4cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:26.848 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.839+0000 7fa32cb85740 1 -- 172.21.15.80:0/1509070782 shutdown_connections 2024-04-24T23:31:26.848 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.839+0000 7fa32cb85740 1 -- 172.21.15.80:0/1509070782 wait complete. 2024-04-24T23:31:26.848 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.839+0000 7fa32cb85740 1 -- start start 2024-04-24T23:31:26.848 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.839+0000 7fa32cb85740 1 Processor -- start 2024-04-24T23:31:26.848 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.839+0000 7fa32cb85740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560eeb3ad650 0x560eeb3b7a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:26.848 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.839+0000 7fa32cb85740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560eeb3ae040 0x560eeb3b7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:26.848 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.839+0000 7fa32cb85740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560eeb3b31b0 0x560eeb3bf000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:26.848 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.839+0000 7fa32cb85740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x560eeb3b6160 con 0x560eeb3b31b0 2024-04-24T23:31:26.848 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.839+0000 7fa32cb85740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x560eeb2c8f50 con 0x560eeb3ad650 2024-04-24T23:31:26.849 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.839+0000 7fa32cb85740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x560eeb3b62e0 con 0x560eeb3ae040 2024-04-24T23:31:26.849 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.839+0000 7fa32cb85740 10 client.0 _ll_register_callbacks cb 0x560eeb304000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:31:26.849 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.839+0000 7fa322ffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560eeb3b31b0 0x560eeb3bf000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:26.849 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.839+0000 7fa322ffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560eeb3b31b0 0x560eeb3bf000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38482/0 (socket says 172.21.15.80:38482) 2024-04-24T23:31:26.849 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.839+0000 7fa322ffd640 1 -- 172.21.15.80:0/1342932255 learned_addr learned my addr 172.21.15.80:0/1342932255 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:31:26.849 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.840+0000 7fa3237fe640 1 --2- 172.21.15.80:0/1342932255 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560eeb3ae040 0x560eeb3b7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:26.849 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.840+0000 7fa322ffd640 1 --2- 172.21.15.80:0/1342932255 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560eeb3ad650 0x560eeb3b7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:26.849 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.840+0000 7fa322ffd640 1 -- 172.21.15.80:0/1342932255 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560eeb3ae040 msgr2=0x560eeb3b7fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:26.849 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.840+0000 7fa322ffd640 1 --2- 172.21.15.80:0/1342932255 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560eeb3ae040 0x560eeb3b7fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:26.849 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.840+0000 7fa322ffd640 1 -- 172.21.15.80:0/1342932255 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560eeb3ad650 msgr2=0x560eeb3b7a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:26.849 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.840+0000 7fa322ffd640 1 --2- 172.21.15.80:0/1342932255 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560eeb3ad650 0x560eeb3b7a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:26.849 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.840+0000 7fa322ffd640 1 -- 172.21.15.80:0/1342932255 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560eeb0213f0 con 0x560eeb3b31b0 2024-04-24T23:31:26.849 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.840+0000 7fa3237fe640 1 --2- 172.21.15.80:0/1342932255 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560eeb3ae040 0x560eeb3b7fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:31:26.850 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.840+0000 7fa322ffd640 1 --2- 172.21.15.80:0/1342932255 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560eeb3b31b0 0x560eeb3bf000 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fa318012b00 tx=0x7fa3180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ef7915862f110f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:26.850 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.840+0000 7fa302ffd640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-04-24T23:31:26.850 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.840+0000 7fa302ffd640 1 -- 172.21.15.80:0/1342932255 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa318012070 con 0x560eeb3b31b0 2024-04-24T23:31:26.850 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.840+0000 7fa302ffd640 1 -- 172.21.15.80:0/1342932255 <== mon.0 v2:172.21.15.80:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fa318012210 con 0x560eeb3b31b0 2024-04-24T23:31:26.850 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.840+0000 7fa32cb85740 1 -- 172.21.15.80:0/1342932255 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560eeb31abf0 con 0x560eeb3b31b0 2024-04-24T23:31:26.850 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.840+0000 7fa302ffd640 1 -- 172.21.15.80:0/1342932255 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa318028050 con 0x560eeb3b31b0 2024-04-24T23:31:26.850 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.840+0000 7fa32cb85740 10 client.15975 fetch_fsmap learned FSMap version 27 2024-04-24T23:31:26.850 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.840+0000 7fa32cb85740 1 -- 172.21.15.80:0/1342932255 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560eeafe10f0 con 0x560eeb3b31b0 2024-04-24T23:31:26.850 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.840+0000 7fa302ffd640 1 -- 172.21.15.80:0/1342932255 <== mon.0 v2:172.21.15.80:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x560eeb31abf0 con 0x560eeb3b31b0 2024-04-24T23:31:26.850 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.841+0000 7fa302ffd640 1 -- 172.21.15.80:0/1342932255 <== mon.0 v2:172.21.15.80:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fa318003780 con 0x560eeb3b31b0 2024-04-24T23:31:26.850 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.841+0000 7fa32cb85740 10 client.15975 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:31:26.851 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.841+0000 7fa32cb85740 10 client.15975 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:31:26.851 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.841+0000 7fa32cb85740 20 client.15975 populate_metadata read hostname 'smithi080' 2024-04-24T23:31:26.851 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.841+0000 7fa32cb85740 1 --2- 172.21.15.80:0/1342932255 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x560eeb3c3910 0x560eeb3c5d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:26.851 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.841+0000 7fa32cb85740 4 client.15975 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:31:26.851 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.841+0000 7fa32cb85740 1 -- 172.21.15.80:0/1342932255 --> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560eeb314f40 con 0x560eeb3c3910 2024-04-24T23:31:26.851 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.841+0000 7fa3237fe640 1 --2- 172.21.15.80:0/1342932255 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x560eeb3c3910 0x560eeb3c5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:26.851 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.841+0000 7fa302ffd640 10 client.15975 ms_handle_connect on v2:172.21.15.148:6838/2952757594 2024-04-24T23:31:26.851 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.841+0000 7fa3237fe640 1 --2- 172.21.15.80:0/1342932255 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x560eeb3c3910 0x560eeb3c5d90 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a33ba650b55bf965 server_cookie=3fb84dd63b1c082a in_seq=0 out_seq=0 2024-04-24T23:31:26.851 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.843+0000 7fa302ffd640 1 -- 172.21.15.80:0/1342932255 <== mds.0 v2:172.21.15.148:6838/2952757594 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560eeb314f40 con 0x560eeb3c3910 2024-04-24T23:31:26.851 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.843+0000 7fa302ffd640 10 client.15975 handle_command_reply: tid=0 2024-04-24T23:31:26.851 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.844+0000 7fa32cb85740 1 client.15975 shutdown 2024-04-24T23:31:26.851 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.844+0000 7fa32cb85740 1 -- 172.21.15.80:0/1342932255 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560eeb3b31b0 msgr2=0x560eeb3bf000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:26.851 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.844+0000 7fa32cb85740 1 --2- 172.21.15.80:0/1342932255 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560eeb3b31b0 0x560eeb3bf000 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fa318012b00 tx=0x7fa3180046c0 comp rx=0 tx=0).stop 2024-04-24T23:31:26.851 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.844+0000 7fa32cb85740 1 -- 172.21.15.80:0/1342932255 shutdown_connections 2024-04-24T23:31:26.852 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.844+0000 7fa32cb85740 1 --2- 172.21.15.80:0/1342932255 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x560eeb3c3910 0x560eeb3c5d90 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:26.852 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.844+0000 7fa32cb85740 1 --2- 172.21.15.80:0/1342932255 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560eeb3b31b0 0x560eeb3bf000 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:26.852 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.844+0000 7fa32cb85740 1 --2- 172.21.15.80:0/1342932255 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560eeb3ae040 0x560eeb3b7fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:26.852 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.844+0000 7fa32cb85740 1 --2- 172.21.15.80:0/1342932255 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560eeb3ad650 0x560eeb3b7a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:26.852 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.844+0000 7fa302ffd640 0 client.15975 ms_handle_reset on v2:172.21.15.148:6838/2952757594 2024-04-24T23:31:26.852 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.844+0000 7fa32cb85740 1 -- 172.21.15.80:0/1342932255 >> 172.21.15.80:0/1342932255 conn(0x560eeb3a48b0 msgr2=0x560eeb3a3af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:26.852 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.844+0000 7fa32cb85740 1 -- 172.21.15.80:0/1342932255 shutdown_connections 2024-04-24T23:31:26.852 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.844+0000 7fa32cb85740 1 -- 172.21.15.80:0/1342932255 wait complete. 2024-04-24T23:31:26.852 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.844+0000 7fa32cb85740 20 client.15975 trim_cache size 0 max 16384 2024-04-24T23:31:26.867 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.862+0000 7fa32cb85740 1 Processor -- start 2024-04-24T23:31:26.867 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.862+0000 7fa32cb85740 1 -- start start 2024-04-24T23:31:26.867 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.862+0000 7fa32cb85740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560eeb3ad650 0x560eeb3a7860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:26.867 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.862+0000 7fa32cb85740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560eeb3ae040 0x560eeb3befc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:26.867 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.862+0000 7fa32cb85740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560eeb3b31b0 0x560eeb3a7da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:26.867 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.862+0000 7fa32cb85740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fa3241aab40 con 0x560eeb3ae040 2024-04-24T23:31:26.867 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.862+0000 7fa32cb85740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa3241a8410 con 0x560eeb3b31b0 2024-04-24T23:31:26.867 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.862+0000 7fa32cb85740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa3241a6260 con 0x560eeb3ad650 2024-04-24T23:31:26.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.862+0000 7fa322ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560eeb3ad650 0x560eeb3a7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:26.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.862+0000 7fa3237fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560eeb3ae040 0x560eeb3befc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:26.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.862+0000 7fa322ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560eeb3ad650 0x560eeb3a7860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.80:38826/0 (socket says 172.21.15.80:38826) 2024-04-24T23:31:26.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.862+0000 7fa322ffd640 1 -- 172.21.15.80:0/1485512354 learned_addr learned my addr 172.21.15.80:0/1485512354 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:31:26.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.862+0000 7fa322ffd640 1 --2- 172.21.15.80:0/1485512354 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560eeb3b31b0 0x560eeb3a7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:26.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.862+0000 7fa3237fe640 1 -- 172.21.15.80:0/1485512354 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560eeb3ad650 msgr2=0x560eeb3a7860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:26.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.862+0000 7fa3237fe640 1 --2- 172.21.15.80:0/1485512354 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560eeb3ad650 0x560eeb3a7860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:26.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.862+0000 7fa3237fe640 1 -- 172.21.15.80:0/1485512354 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560eeb3b31b0 msgr2=0x560eeb3a7da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:26.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.862+0000 7fa3237fe640 1 --2- 172.21.15.80:0/1485512354 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560eeb3b31b0 0x560eeb3a7da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:26.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.862+0000 7fa3237fe640 1 -- 172.21.15.80:0/1485512354 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560eeb3a43f0 con 0x560eeb3ae040 2024-04-24T23:31:26.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.862+0000 7fa3237fe640 1 --2- 172.21.15.80:0/1485512354 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560eeb3ae040 0x560eeb3befc0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x560eeb3a5820 tx=0x7fa3140051d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9f6b12c3dfc6c6e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:26.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.863+0000 7fa3097fa640 1 -- 172.21.15.80:0/1485512354 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa314036070 con 0x560eeb3ae040 2024-04-24T23:31:26.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.863+0000 7fa3097fa640 1 -- 172.21.15.80:0/1485512354 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa314004660 con 0x560eeb3ae040 2024-04-24T23:31:26.869 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.863+0000 7fa3097fa640 1 -- 172.21.15.80:0/1485512354 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3140035f0 con 0x560eeb3ae040 2024-04-24T23:31:26.869 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.863+0000 7fa32cb85740 1 -- 172.21.15.80:0/1485512354 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560eeb3ae040 msgr2=0x560eeb3befc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:26.869 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.863+0000 7fa32cb85740 1 --2- 172.21.15.80:0/1485512354 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560eeb3ae040 0x560eeb3befc0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x560eeb3a5820 tx=0x7fa3140051d0 comp rx=0 tx=0).stop 2024-04-24T23:31:26.869 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.863+0000 7fa32cb85740 1 -- 172.21.15.80:0/1485512354 shutdown_connections 2024-04-24T23:31:26.869 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.863+0000 7fa32cb85740 1 --2- 172.21.15.80:0/1485512354 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560eeb3b31b0 0x560eeb3a7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:26.869 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.863+0000 7fa32cb85740 1 --2- 172.21.15.80:0/1485512354 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560eeb3ae040 0x560eeb3befc0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:26.869 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.863+0000 7fa32cb85740 1 --2- 172.21.15.80:0/1485512354 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560eeb3ad650 0x560eeb3a7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:26.869 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.863+0000 7fa32cb85740 1 -- 172.21.15.80:0/1485512354 >> 172.21.15.80:0/1485512354 conn(0x560eeb3a48b0 msgr2=0x560eeb3af400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:26.869 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.863+0000 7fa32cb85740 1 -- 172.21.15.80:0/1485512354 shutdown_connections 2024-04-24T23:31:26.869 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.863+0000 7fa32cb85740 1 -- 172.21.15.80:0/1485512354 wait complete. 2024-04-24T23:31:26.870 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.864+0000 7fa32cb85740 1 -- start start 2024-04-24T23:31:26.870 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.864+0000 7fa32cb85740 1 Processor -- start 2024-04-24T23:31:26.870 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.864+0000 7fa32cb85740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560eeb3ad650 0x560eeb3bc540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:26.870 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.864+0000 7fa32cb85740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560eeb3ae040 0x560eeb3bca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:26.870 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.864+0000 7fa32cb85740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560eeb3b31b0 0x560eeb3bd9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:26.870 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.864+0000 7fa32cb85740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fa3241a68d0 con 0x560eeb3b31b0 2024-04-24T23:31:26.870 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.864+0000 7fa32cb85740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa3241a7d40 con 0x560eeb3ad650 2024-04-24T23:31:26.870 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.864+0000 7fa32cb85740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa3241a6fa0 con 0x560eeb3ae040 2024-04-24T23:31:26.870 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.864+0000 7fa32cb85740 10 client.0 _ll_register_callbacks cb 0x7fa31800bc10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:31:26.870 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.864+0000 7fa322ffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560eeb3b31b0 0x560eeb3bd9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:26.870 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.864+0000 7fa322ffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560eeb3b31b0 0x560eeb3bd9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38512/0 (socket says 172.21.15.80:38512) 2024-04-24T23:31:26.870 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.864+0000 7fa322ffd640 1 -- 172.21.15.80:0/1148457993 learned_addr learned my addr 172.21.15.80:0/1148457993 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:31:26.870 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.864+0000 7fa3237fe640 1 --2- 172.21.15.80:0/1148457993 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560eeb3ae040 0x560eeb3bca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:26.871 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.864+0000 7fa322ffd640 1 --2- 172.21.15.80:0/1148457993 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560eeb3ad650 0x560eeb3bc540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:26.871 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.864+0000 7fa322ffd640 1 -- 172.21.15.80:0/1148457993 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560eeb3ae040 msgr2=0x560eeb3bca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:26.871 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.864+0000 7fa322ffd640 1 --2- 172.21.15.80:0/1148457993 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560eeb3ae040 0x560eeb3bca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:26.871 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.864+0000 7fa322ffd640 1 -- 172.21.15.80:0/1148457993 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560eeb3ad650 msgr2=0x560eeb3bc540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:26.871 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.864+0000 7fa322ffd640 1 --2- 172.21.15.80:0/1148457993 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560eeb3ad650 0x560eeb3bc540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:26.871 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.864+0000 7fa322ffd640 1 -- 172.21.15.80:0/1148457993 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560eeb3bdfe0 con 0x560eeb3b31b0 2024-04-24T23:31:26.871 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.865+0000 7fa322ffd640 1 --2- 172.21.15.80:0/1148457993 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560eeb3b31b0 0x560eeb3bd9f0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fa318003ba0 tx=0x7fa3180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62799fda33f67045 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:26.871 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.865+0000 7fa302ffd640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-04-24T23:31:26.871 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.865+0000 7fa302ffd640 1 -- 172.21.15.80:0/1148457993 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa31800fb50 con 0x560eeb3b31b0 2024-04-24T23:31:26.871 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.865+0000 7fa302ffd640 1 -- 172.21.15.80:0/1148457993 <== mon.0 v2:172.21.15.80:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fa31800fcf0 con 0x560eeb3b31b0 2024-04-24T23:31:26.871 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.865+0000 7fa32cb85740 1 -- 172.21.15.80:0/1148457993 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa314004660 con 0x560eeb3b31b0 2024-04-24T23:31:26.871 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.865+0000 7fa302ffd640 1 -- 172.21.15.80:0/1148457993 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa3180035f0 con 0x560eeb3b31b0 2024-04-24T23:31:26.871 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.865+0000 7fa32cb85740 10 client.15981 fetch_fsmap learned FSMap version 27 2024-04-24T23:31:26.871 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.865+0000 7fa32cb85740 1 -- 172.21.15.80:0/1148457993 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560eeb429ed0 con 0x560eeb3b31b0 2024-04-24T23:31:26.872 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.865+0000 7fa302ffd640 1 -- 172.21.15.80:0/1148457993 <== mon.0 v2:172.21.15.80:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa314004660 con 0x560eeb3b31b0 2024-04-24T23:31:26.872 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.865+0000 7fa302ffd640 1 -- 172.21.15.80:0/1148457993 <== mon.0 v2:172.21.15.80:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fa31802b040 con 0x560eeb3b31b0 2024-04-24T23:31:26.872 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.865+0000 7fa32cb85740 10 client.15981 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:31:26.872 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.865+0000 7fa32cb85740 10 client.15981 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:31:26.872 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.865+0000 7fa32cb85740 20 client.15981 populate_metadata read hostname 'smithi080' 2024-04-24T23:31:26.872 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.865+0000 7fa32cb85740 1 --2- 172.21.15.80:0/1148457993 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x560eeb42a7c0 0x560eeb42cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:26.872 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.865+0000 7fa32cb85740 4 client.15981 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:31:26.872 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.865+0000 7fa32cb85740 1 -- 172.21.15.80:0/1148457993 --> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560eeb2926f0 con 0x560eeb42a7c0 2024-04-24T23:31:26.872 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.865+0000 7fa3237fe640 1 --2- 172.21.15.80:0/1148457993 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x560eeb42a7c0 0x560eeb42cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:26.872 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.866+0000 7fa302ffd640 10 client.15981 ms_handle_connect on v2:172.21.15.148:6838/2952757594 2024-04-24T23:31:26.872 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.866+0000 7fa3237fe640 1 --2- 172.21.15.80:0/1148457993 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x560eeb42a7c0 0x560eeb42cbb0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c81388b5d4a19b7 server_cookie=7cb8ef94b0104840 in_seq=0 out_seq=0 2024-04-24T23:31:26.874 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.868+0000 7fa302ffd640 1 -- 172.21.15.80:0/1148457993 <== mds.0 v2:172.21.15.148:6838/2952757594 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x560eeb2926f0 con 0x560eeb42a7c0 2024-04-24T23:31:26.874 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.868+0000 7fa302ffd640 10 client.15981 handle_command_reply: tid=0 2024-04-24T23:31:26.874 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.869+0000 7fa32cb85740 1 client.15981 shutdown 2024-04-24T23:31:26.874 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.869+0000 7fa32cb85740 1 -- 172.21.15.80:0/1148457993 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560eeb3b31b0 msgr2=0x560eeb3bd9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:26.874 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.869+0000 7fa32cb85740 1 --2- 172.21.15.80:0/1148457993 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560eeb3b31b0 0x560eeb3bd9f0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fa318003ba0 tx=0x7fa3180046c0 comp rx=0 tx=0).stop 2024-04-24T23:31:26.874 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.869+0000 7fa32cb85740 1 -- 172.21.15.80:0/1148457993 shutdown_connections 2024-04-24T23:31:26.874 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.869+0000 7fa32cb85740 1 --2- 172.21.15.80:0/1148457993 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x560eeb42a7c0 0x560eeb42cbb0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:26.874 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.869+0000 7fa32cb85740 1 --2- 172.21.15.80:0/1148457993 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560eeb3b31b0 0x560eeb3bd9f0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:26.874 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.869+0000 7fa32cb85740 1 --2- 172.21.15.80:0/1148457993 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560eeb3ae040 0x560eeb3bca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:26.875 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.869+0000 7fa32cb85740 1 --2- 172.21.15.80:0/1148457993 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560eeb3ad650 0x560eeb3bc540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:26.875 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.869+0000 7fa302ffd640 0 client.15981 ms_handle_reset on v2:172.21.15.148:6838/2952757594 2024-04-24T23:31:26.875 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.869+0000 7fa32cb85740 1 -- 172.21.15.80:0/1148457993 >> 172.21.15.80:0/1148457993 conn(0x560eeb3a48b0 msgr2=0x560eeb3a3260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:26.875 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.869+0000 7fa32cb85740 1 -- 172.21.15.80:0/1148457993 shutdown_connections 2024-04-24T23:31:26.875 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.869+0000 7fa32cb85740 1 -- 172.21.15.80:0/1148457993 wait complete. 2024-04-24T23:31:26.875 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.869+0000 7fa32cb85740 20 client.15981 trim_cache size 0 max 16384 2024-04-24T23:31:26.875 INFO:teuthology.orchestra.run.smithi080.stdout:{ 2024-04-24T23:31:26.876 INFO:teuthology.orchestra.run.smithi080.stdout: "status": "scrub active (150 inodes in the stack)", 2024-04-24T23:31:26.876 INFO:teuthology.orchestra.run.smithi080.stdout: "scrubs": { 2024-04-24T23:31:26.876 INFO:teuthology.orchestra.run.smithi080.stdout: "be057515-fa5a-4f78-b988-4a14f1292aa7": { 2024-04-24T23:31:26.876 INFO:teuthology.orchestra.run.smithi080.stdout: "path": "/", 2024-04-24T23:31:26.876 INFO:teuthology.orchestra.run.smithi080.stdout: "tag": "be057515-fa5a-4f78-b988-4a14f1292aa7", 2024-04-24T23:31:26.876 INFO:teuthology.orchestra.run.smithi080.stdout: "options": "recursive,force" 2024-04-24T23:31:26.876 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:31:26.876 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:31:26.876 INFO:teuthology.orchestra.run.smithi080.stdout:} 2024-04-24T23:31:26.877 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.869+0000 7fa329d9e640 1 -- 172.21.15.80:0/3128642967 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fa30c0a2b90 msgr2=0x7fa30c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:26.877 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.869+0000 7fa329d9e640 1 --2- 172.21.15.80:0/3128642967 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fa30c0a2b90 0x7fa30c0a5050 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fa31c0b9050 tx=0x7fa31c08fcb0 comp rx=0 tx=0).stop 2024-04-24T23:31:26.877 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.869+0000 7fa329d9e640 1 -- 172.21.15.80:0/3128642967 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa324102230 msgr2=0x7fa32419ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:26.877 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.869+0000 7fa329d9e640 1 --2- 172.21.15.80:0/3128642967 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa324102230 0x7fa32419ede0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fa31c031ae0 tx=0x7fa31c031ee0 comp rx=0 tx=0).stop 2024-04-24T23:31:26.877 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.870+0000 7fa329d9e640 1 -- 172.21.15.80:0/3128642967 shutdown_connections 2024-04-24T23:31:26.878 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.870+0000 7fa329d9e640 1 --2- 172.21.15.80:0/3128642967 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fa30c0a2b90 0x7fa30c0a5050 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:26.878 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.870+0000 7fa329d9e640 1 --2- 172.21.15.80:0/3128642967 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa324102230 0x7fa32419ede0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:26.878 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.870+0000 7fa329d9e640 1 --2- 172.21.15.80:0/3128642967 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa324101870 0x7fa32419c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:26.878 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.870+0000 7fa329d9e640 1 --2- 172.21.15.80:0/3128642967 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa324100680 0x7fa324069440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:26.878 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.870+0000 7fa329d9e640 1 -- 172.21.15.80:0/3128642967 >> 172.21.15.80:0/3128642967 conn(0x7fa3240fbbf0 msgr2=0x7fa324104bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:26.878 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.870+0000 7fa329d9e640 1 -- 172.21.15.80:0/3128642967 shutdown_connections 2024-04-24T23:31:26.878 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:26.870+0000 7fa329d9e640 1 -- 172.21.15.80:0/3128642967 wait complete. 2024-04-24T23:31:26.881 INFO:tasks.cephfs.filesystem:scrub status for tag:be057515-fa5a-4f78-b988-4a14f1292aa7 - {'path': '/', 'tag': 'be057515-fa5a-4f78-b988-4a14f1292aa7', 'options': 'recursive,force'} 2024-04-24T23:31:27.287 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:31:27 smithi080 ceph-mon[29152]: scrub summary: active paths [/] 2024-04-24T23:31:27.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:31:27 smithi080 ceph-mon[29152]: scrub summary: idle+waiting paths [/] 2024-04-24T23:31:27.446 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:31:27 smithi134 ceph-mon[32619]: scrub summary: active paths [/] 2024-04-24T23:31:27.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:31:27 smithi134 ceph-mon[32619]: scrub summary: idle+waiting paths [/] 2024-04-24T23:31:27.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:31:27 smithi148 ceph-mon[32528]: scrub summary: active paths [/] 2024-04-24T23:31:27.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:31:27 smithi148 ceph-mon[32528]: scrub summary: idle+waiting paths [/] 2024-04-24T23:31:28.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:31:28 smithi080 ceph-mon[29152]: scrub queued for path: / 2024-04-24T23:31:28.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:31:28 smithi080 ceph-mon[29152]: scrub summary: idle+waiting paths [/] 2024-04-24T23:31:28.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:31:28 smithi080 ceph-mon[29152]: scrub summary: active paths [/] 2024-04-24T23:31:28.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:31:28 smithi080 ceph-mon[29152]: scrub summary: active paths [/] 2024-04-24T23:31:28.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:31:28 smithi080 ceph-mon[29152]: scrub summary: idle+waiting paths [/] 2024-04-24T23:31:28.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:31:28 smithi080 ceph-mon[29152]: scrub summary: active paths [/] 2024-04-24T23:31:28.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:31:28 smithi080 ceph-mon[29152]: scrub summary: idle+waiting paths [/] 2024-04-24T23:31:28.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:31:28 smithi080 ceph-mon[29152]: pgmap v424: 129 pgs: 129 active+clean; 26 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 939 KiB/s rd, 1.1 MiB/s wr, 45 op/s 2024-04-24T23:31:28.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:31:28 smithi080 ceph-mon[29152]: scrub summary: active paths [/] 2024-04-24T23:31:28.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:31:28 smithi080 ceph-mon[29152]: scrub summary: active paths [/] 2024-04-24T23:31:28.288 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:31:28 smithi080 ceph-mon[29152]: scrub summary: idle+waiting paths [/] 2024-04-24T23:31:28.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:31:28 smithi080 ceph-mon[29152]: scrub summary: active paths [/] 2024-04-24T23:31:28.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:31:28 smithi080 ceph-mon[29152]: scrub summary: idle+waiting paths [/] 2024-04-24T23:31:28.289 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:31:28 smithi080 ceph-mon[29152]: scrub summary: idle+waiting paths [/] 2024-04-24T23:31:28.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:31:28 smithi134 ceph-mon[32619]: scrub queued for path: / 2024-04-24T23:31:28.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:31:28 smithi134 ceph-mon[32619]: scrub summary: idle+waiting paths [/] 2024-04-24T23:31:28.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:31:28 smithi134 ceph-mon[32619]: scrub summary: active paths [/] 2024-04-24T23:31:28.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:31:28 smithi134 ceph-mon[32619]: scrub summary: active paths [/] 2024-04-24T23:31:28.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:31:28 smithi134 ceph-mon[32619]: scrub summary: idle+waiting paths [/] 2024-04-24T23:31:28.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:31:28 smithi134 ceph-mon[32619]: scrub summary: active paths [/] 2024-04-24T23:31:28.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:31:28 smithi134 ceph-mon[32619]: scrub summary: idle+waiting paths [/] 2024-04-24T23:31:28.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:31:28 smithi134 ceph-mon[32619]: pgmap v424: 129 pgs: 129 active+clean; 26 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 939 KiB/s rd, 1.1 MiB/s wr, 45 op/s 2024-04-24T23:31:28.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:31:28 smithi134 ceph-mon[32619]: scrub summary: active paths [/] 2024-04-24T23:31:28.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:31:28 smithi134 ceph-mon[32619]: scrub summary: active paths [/] 2024-04-24T23:31:28.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:31:28 smithi134 ceph-mon[32619]: scrub summary: idle+waiting paths [/] 2024-04-24T23:31:28.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:31:28 smithi134 ceph-mon[32619]: scrub summary: active paths [/] 2024-04-24T23:31:28.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:31:28 smithi134 ceph-mon[32619]: scrub summary: idle+waiting paths [/] 2024-04-24T23:31:28.449 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:31:28 smithi134 ceph-mon[32619]: scrub summary: idle+waiting paths [/] 2024-04-24T23:31:28.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:31:28 smithi148 ceph-mon[32528]: scrub queued for path: / 2024-04-24T23:31:28.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:31:28 smithi148 ceph-mon[32528]: scrub summary: idle+waiting paths [/] 2024-04-24T23:31:28.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:31:28 smithi148 ceph-mon[32528]: scrub summary: active paths [/] 2024-04-24T23:31:28.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:31:28 smithi148 ceph-mon[32528]: scrub summary: active paths [/] 2024-04-24T23:31:28.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:31:28 smithi148 ceph-mon[32528]: scrub summary: idle+waiting paths [/] 2024-04-24T23:31:28.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:31:28 smithi148 ceph-mon[32528]: scrub summary: active paths [/] 2024-04-24T23:31:28.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:31:28 smithi148 ceph-mon[32528]: scrub summary: idle+waiting paths [/] 2024-04-24T23:31:28.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:31:28 smithi148 ceph-mon[32528]: pgmap v424: 129 pgs: 129 active+clean; 26 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 939 KiB/s rd, 1.1 MiB/s wr, 45 op/s 2024-04-24T23:31:28.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:31:28 smithi148 ceph-mon[32528]: scrub summary: active paths [/] 2024-04-24T23:31:28.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:31:28 smithi148 ceph-mon[32528]: scrub summary: active paths [/] 2024-04-24T23:31:28.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:31:28 smithi148 ceph-mon[32528]: scrub summary: idle+waiting paths [/] 2024-04-24T23:31:28.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:31:28 smithi148 ceph-mon[32528]: scrub summary: active paths [/] 2024-04-24T23:31:28.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:31:28 smithi148 ceph-mon[32528]: scrub summary: idle+waiting paths [/] 2024-04-24T23:31:28.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:31:28 smithi148 ceph-mon[32528]: scrub summary: idle+waiting paths [/] 2024-04-24T23:31:29.287 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:31:29 smithi080 ceph-mon[29152]: scrub summary: idle+waiting paths [/] 2024-04-24T23:31:29.446 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:31:29 smithi134 ceph-mon[32619]: scrub summary: idle+waiting paths [/] 2024-04-24T23:31:29.505 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:31:29 smithi148 ceph-mon[32528]: scrub summary: idle+waiting paths [/] 2024-04-24T23:31:30.446 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:31:30 smithi134 ceph-mon[32619]: pgmap v425: 129 pgs: 129 active+clean; 33 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 63 op/s 2024-04-24T23:31:30.505 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:31:30 smithi148 ceph-mon[32528]: pgmap v425: 129 pgs: 129 active+clean; 33 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 63 op/s 2024-04-24T23:31:30.537 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:31:30 smithi080 ceph-mon[29152]: pgmap v425: 129 pgs: 129 active+clean; 33 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 63 op/s 2024-04-24T23:31:32.331 INFO:tasks.workunit.client.0.smithi080.stderr:Note: switching to '12973fc04fd10d4af086901e10ffa8e48866b735'. 2024-04-24T23:31:32.331 INFO:tasks.workunit.client.0.smithi080.stderr: 2024-04-24T23:31:32.331 INFO:tasks.workunit.client.0.smithi080.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-24T23:31:32.331 INFO:tasks.workunit.client.0.smithi080.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-24T23:31:32.332 INFO:tasks.workunit.client.0.smithi080.stderr:state without impacting any branches by switching back to a branch. 2024-04-24T23:31:32.332 INFO:tasks.workunit.client.0.smithi080.stderr: 2024-04-24T23:31:32.332 INFO:tasks.workunit.client.0.smithi080.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-24T23:31:32.332 INFO:tasks.workunit.client.0.smithi080.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-24T23:31:32.332 INFO:tasks.workunit.client.0.smithi080.stderr: 2024-04-24T23:31:32.332 INFO:tasks.workunit.client.0.smithi080.stderr: git switch -c 2024-04-24T23:31:32.332 INFO:tasks.workunit.client.0.smithi080.stderr: 2024-04-24T23:31:32.332 INFO:tasks.workunit.client.0.smithi080.stderr:Or undo this operation with: 2024-04-24T23:31:32.332 INFO:tasks.workunit.client.0.smithi080.stderr: 2024-04-24T23:31:32.332 INFO:tasks.workunit.client.0.smithi080.stderr: git switch - 2024-04-24T23:31:32.332 INFO:tasks.workunit.client.0.smithi080.stderr: 2024-04-24T23:31:32.332 INFO:tasks.workunit.client.0.smithi080.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-24T23:31:32.332 INFO:tasks.workunit.client.0.smithi080.stderr: 2024-04-24T23:31:32.333 INFO:tasks.workunit.client.0.smithi080.stderr:HEAD is now at 12973fc0 btrfs/022: Correct seq number in golden output 2024-04-24T23:31:32.337 INFO:tasks.workunit.client.0.smithi080.stderr:+ make -j4 2024-04-24T23:31:32.345 INFO:tasks.workunit.client.0.smithi080.stdout:libtoolize -cfi 2024-04-24T23:31:32.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:31:32 smithi134 ceph-mon[32619]: pgmap v426: 129 pgs: 129 active+clean; 33 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 49 op/s 2024-04-24T23:31:32.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:31:32 smithi148 ceph-mon[32528]: pgmap v426: 129 pgs: 129 active+clean; 33 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 49 op/s 2024-04-24T23:31:32.537 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:31:32 smithi080 ceph-mon[29152]: pgmap v426: 129 pgs: 129 active+clean; 33 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 49 op/s 2024-04-24T23:31:32.689 INFO:tasks.workunit.client.0.smithi080.stdout:libtoolize: putting auxiliary files in '.'. 2024-04-24T23:31:32.690 INFO:tasks.workunit.client.0.smithi080.stdout:libtoolize: copying file './config.guess' 2024-04-24T23:31:32.694 INFO:tasks.workunit.client.0.smithi080.stdout:libtoolize: copying file './config.sub' 2024-04-24T23:31:32.700 INFO:tasks.workunit.client.0.smithi080.stdout:libtoolize: copying file './install-sh' 2024-04-24T23:31:32.704 INFO:tasks.workunit.client.0.smithi080.stdout:libtoolize: copying file './ltmain.sh' 2024-04-24T23:31:32.728 INFO:tasks.workunit.client.0.smithi080.stdout:libtoolize: putting macros in AC_CONFIG_MACRO_DIRS, 'm4'. 2024-04-24T23:31:32.728 INFO:tasks.workunit.client.0.smithi080.stdout:libtoolize: copying file 'm4/libtool.m4' 2024-04-24T23:31:32.733 INFO:tasks.workunit.client.0.smithi080.stdout:libtoolize: copying file 'm4/ltoptions.m4' 2024-04-24T23:31:32.739 INFO:tasks.workunit.client.0.smithi080.stdout:libtoolize: copying file 'm4/ltsugar.m4' 2024-04-24T23:31:32.745 INFO:tasks.workunit.client.0.smithi080.stdout:libtoolize: copying file 'm4/ltversion.m4' 2024-04-24T23:31:32.751 INFO:tasks.workunit.client.0.smithi080.stdout:libtoolize: copying file 'm4/lt~obsolete.m4' 2024-04-24T23:31:32.758 INFO:tasks.workunit.client.0.smithi080.stdout:libtoolize: Consider adding '-I m4' to ACLOCAL_AMFLAGS in Makefile.am. 2024-04-24T23:31:32.758 INFO:tasks.workunit.client.0.smithi080.stdout:cp include/install-sh . 2024-04-24T23:31:32.769 INFO:tasks.workunit.client.0.smithi080.stdout:aclocal -I m4 2024-04-24T23:31:34.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:31:34 smithi134 ceph-mon[32619]: pgmap v427: 129 pgs: 129 active+clean; 33 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 45 op/s 2024-04-24T23:31:34.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:31:34 smithi148 ceph-mon[32528]: pgmap v427: 129 pgs: 129 active+clean; 33 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 45 op/s 2024-04-24T23:31:34.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:31:34 smithi080 ceph-mon[29152]: pgmap v427: 129 pgs: 129 active+clean; 33 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 45 op/s 2024-04-24T23:31:35.446 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:31:35 smithi134 ceph-mon[32619]: scrub summary: idle 2024-04-24T23:31:35.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:31:35 smithi134 ceph-mon[32619]: scrub summary: idle 2024-04-24T23:31:35.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:31:35 smithi148 ceph-mon[32528]: scrub summary: idle 2024-04-24T23:31:35.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:31:35 smithi148 ceph-mon[32528]: scrub summary: idle 2024-04-24T23:31:35.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:31:35 smithi080 ceph-mon[29152]: scrub summary: idle 2024-04-24T23:31:35.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:31:35 smithi080 ceph-mon[29152]: scrub summary: idle 2024-04-24T23:31:36.446 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:31:36 smithi134 ceph-mon[32619]: scrub summary: idle 2024-04-24T23:31:36.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:31:36 smithi134 ceph-mon[32619]: pgmap v428: 129 pgs: 129 active+clean; 52 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.0 MiB/s wr, 341 op/s 2024-04-24T23:31:36.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:31:36 smithi148 ceph-mon[32528]: scrub summary: idle 2024-04-24T23:31:36.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:31:36 smithi148 ceph-mon[32528]: pgmap v428: 129 pgs: 129 active+clean; 52 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.0 MiB/s wr, 341 op/s 2024-04-24T23:31:36.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:31:36 smithi080 ceph-mon[29152]: scrub summary: idle 2024-04-24T23:31:36.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:31:36 smithi080 ceph-mon[29152]: pgmap v428: 129 pgs: 129 active+clean; 52 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.0 MiB/s wr, 341 op/s 2024-04-24T23:31:37.745 INFO:tasks.workunit.client.0.smithi080.stdout:autoheader 2024-04-24T23:31:38.051 INFO:tasks.workunit.client.0.smithi080.stdout:autoconf 2024-04-24T23:31:38.246 INFO:tasks.workunit.client.0.smithi080.stdout:./configure \ 2024-04-24T23:31:38.382 INFO:tasks.workunit.client.0.smithi080.stdout: --libexecdir=/usr/lib \ 2024-04-24T23:31:38.382 INFO:tasks.workunit.client.0.smithi080.stdout: --exec_prefix=/var/lib 2024-04-24T23:31:38.383 INFO:tasks.workunit.client.0.smithi080.stdout:checking build system type... x86_64-pc-linux-gnu 2024-04-24T23:31:38.383 INFO:tasks.workunit.client.0.smithi080.stdout:checking host system type... x86_64-pc-linux-gnu 2024-04-24T23:31:38.383 INFO:tasks.workunit.client.0.smithi080.stdout:checking how to print strings... printf 2024-04-24T23:31:38.384 INFO:tasks.workunit.client.0.smithi080.stdout:checking for gcc... gcc 2024-04-24T23:31:38.439 INFO:tasks.workunit.client.0.smithi080.stdout:checking whether the C compiler works... yes 2024-04-24T23:31:38.439 INFO:tasks.workunit.client.0.smithi080.stdout:checking for C compiler default output file name... a.out 2024-04-24T23:31:38.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:31:38 smithi134 ceph-mon[32619]: pgmap v429: 129 pgs: 129 active+clean; 57 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.2 MiB/s wr, 414 op/s 2024-04-24T23:31:38.475 INFO:tasks.workunit.client.0.smithi080.stdout:checking for suffix of executables... 2024-04-24T23:31:38.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:31:38 smithi148 ceph-mon[32528]: pgmap v429: 129 pgs: 129 active+clean; 57 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.2 MiB/s wr, 414 op/s 2024-04-24T23:31:38.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:31:38 smithi080 ceph-mon[29152]: pgmap v429: 129 pgs: 129 active+clean; 57 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.2 MiB/s wr, 414 op/s 2024-04-24T23:31:38.600 INFO:tasks.workunit.client.0.smithi080.stdout:checking whether we are cross compiling... no 2024-04-24T23:31:38.674 INFO:tasks.workunit.client.0.smithi080.stdout:checking for suffix of object files... o 2024-04-24T23:31:38.727 INFO:tasks.workunit.client.0.smithi080.stdout:checking whether we are using the GNU C compiler... yes 2024-04-24T23:31:38.796 INFO:tasks.workunit.client.0.smithi080.stdout:checking whether gcc accepts -g... yes 2024-04-24T23:31:39.136 INFO:tasks.workunit.client.0.smithi080.stdout:checking for gcc option to accept ISO C89... none needed 2024-04-24T23:31:39.142 INFO:tasks.workunit.client.0.smithi080.stdout:checking for a sed that does not truncate output... /usr/bin/sed 2024-04-24T23:31:39.147 INFO:tasks.workunit.client.0.smithi080.stdout:checking for grep that handles long lines and -e... /usr/bin/grep 2024-04-24T23:31:39.148 INFO:tasks.workunit.client.0.smithi080.stdout:checking for egrep... /usr/bin/grep -E 2024-04-24T23:31:39.148 INFO:tasks.workunit.client.0.smithi080.stdout:checking for fgrep... /usr/bin/grep -F 2024-04-24T23:31:39.150 INFO:tasks.workunit.client.0.smithi080.stdout:checking for ld used by gcc... /usr/bin/ld 2024-04-24T23:31:39.151 INFO:tasks.workunit.client.0.smithi080.stdout:checking if the linker (/usr/bin/ld) is GNU ld... yes 2024-04-24T23:31:39.182 INFO:tasks.workunit.client.0.smithi080.stdout:checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B 2024-04-24T23:31:40.446 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:31:40 smithi134 ceph-mon[32619]: pgmap v430: 129 pgs: 129 active+clean; 61 MiB data, 511 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.6 MiB/s wr, 674 op/s 2024-04-24T23:31:40.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:31:40 smithi148 ceph-mon[32528]: pgmap v430: 129 pgs: 129 active+clean; 61 MiB data, 511 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.6 MiB/s wr, 674 op/s 2024-04-24T23:31:40.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:31:40 smithi080 ceph-mon[29152]: pgmap v430: 129 pgs: 129 active+clean; 61 MiB data, 511 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.6 MiB/s wr, 674 op/s 2024-04-24T23:31:42.446 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:31:42 smithi134 ceph-mon[32619]: pgmap v431: 129 pgs: 129 active+clean; 61 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.2 MiB/s wr, 766 op/s 2024-04-24T23:31:42.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:31:42 smithi148 ceph-mon[32528]: pgmap v431: 129 pgs: 129 active+clean; 61 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.2 MiB/s wr, 766 op/s 2024-04-24T23:31:42.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:31:42 smithi080 ceph-mon[29152]: pgmap v431: 129 pgs: 129 active+clean; 61 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.2 MiB/s wr, 766 op/s 2024-04-24T23:31:42.803 INFO:tasks.workunit.client.0.smithi080.stdout:checking the name lister (/usr/bin/nm -B) interface... BSD nm 2024-04-24T23:31:42.803 INFO:tasks.workunit.client.0.smithi080.stdout:checking whether ln -s works... yes 2024-04-24T23:31:42.811 INFO:tasks.workunit.client.0.smithi080.stdout:checking the maximum length of command line arguments... 1572864 2024-04-24T23:31:42.814 INFO:tasks.workunit.client.0.smithi080.stdout:checking how to convert x86_64-pc-linux-gnu file names to x86_64-pc-linux-gnu format... func_convert_file_noop 2024-04-24T23:31:42.814 INFO:tasks.workunit.client.0.smithi080.stdout:checking how to convert x86_64-pc-linux-gnu file names to toolchain format... func_convert_file_noop 2024-04-24T23:31:42.814 INFO:tasks.workunit.client.0.smithi080.stdout:checking for /usr/bin/ld option to reload object files... -r 2024-04-24T23:31:42.814 INFO:tasks.workunit.client.0.smithi080.stdout:checking for objdump... objdump 2024-04-24T23:31:42.814 INFO:tasks.workunit.client.0.smithi080.stdout:checking how to recognize dependent libraries... pass_all 2024-04-24T23:31:42.815 INFO:tasks.workunit.client.0.smithi080.stdout:checking for dlltool... no 2024-04-24T23:31:42.815 INFO:tasks.workunit.client.0.smithi080.stdout:checking how to associate runtime and link libraries... printf %s\n 2024-04-24T23:31:42.815 INFO:tasks.workunit.client.0.smithi080.stdout:checking for ar... ar 2024-04-24T23:31:42.903 INFO:tasks.workunit.client.0.smithi080.stdout:checking for archiver @FILE support... @ 2024-04-24T23:31:42.903 INFO:tasks.workunit.client.0.smithi080.stdout:checking for strip... strip 2024-04-24T23:31:42.903 INFO:tasks.workunit.client.0.smithi080.stdout:checking for ranlib... ranlib 2024-04-24T23:31:42.903 INFO:tasks.workunit.client.0.smithi080.stdout:checking for gawk... gawk 2024-04-24T23:31:43.007 INFO:tasks.workunit.client.0.smithi080.stdout:checking command to parse /usr/bin/nm -B output from gcc object... ok 2024-04-24T23:31:43.008 INFO:tasks.workunit.client.0.smithi080.stdout:checking for sysroot... no 2024-04-24T23:31:43.020 INFO:tasks.workunit.client.0.smithi080.stdout:checking for a working dd... /usr/bin/dd 2024-04-24T23:31:43.032 INFO:tasks.workunit.client.0.smithi080.stdout:checking how to truncate binary pipes... /usr/bin/dd bs=4096 count=1 2024-04-24T23:31:43.069 INFO:tasks.workunit.client.0.smithi080.stdout:checking for mt... no 2024-04-24T23:31:43.082 INFO:tasks.workunit.client.0.smithi080.stdout:checking if : is a manifest tool... no 2024-04-24T23:31:43.174 INFO:tasks.workunit.client.0.smithi080.stdout:checking how to run the C preprocessor... gcc -E 2024-04-24T23:31:43.406 INFO:tasks.workunit.client.0.smithi080.stdout:checking for ANSI C header files... yes 2024-04-24T23:31:43.443 INFO:tasks.workunit.client.0.smithi080.stdout:checking for sys/types.h... yes 2024-04-24T23:31:43.482 INFO:tasks.workunit.client.0.smithi080.stdout:checking for sys/stat.h... yes 2024-04-24T23:31:43.521 INFO:tasks.workunit.client.0.smithi080.stdout:checking for stdlib.h... yes 2024-04-24T23:31:43.560 INFO:tasks.workunit.client.0.smithi080.stdout:checking for string.h... yes 2024-04-24T23:31:43.600 INFO:tasks.workunit.client.0.smithi080.stdout:checking for memory.h... yes 2024-04-24T23:31:43.640 INFO:tasks.workunit.client.0.smithi080.stdout:checking for strings.h... yes 2024-04-24T23:31:43.680 INFO:tasks.workunit.client.0.smithi080.stdout:checking for inttypes.h... yes 2024-04-24T23:31:43.720 INFO:tasks.workunit.client.0.smithi080.stdout:checking for stdint.h... yes 2024-04-24T23:31:43.761 INFO:tasks.workunit.client.0.smithi080.stdout:checking for unistd.h... yes 2024-04-24T23:31:43.807 INFO:tasks.workunit.client.0.smithi080.stdout:checking for dlfcn.h... yes 2024-04-24T23:31:43.812 INFO:tasks.workunit.client.0.smithi080.stdout:checking for objdir... .libs 2024-04-24T23:31:43.926 INFO:tasks.workunit.client.0.smithi080.stdout:checking if gcc supports -fno-rtti -fno-exceptions... no 2024-04-24T23:31:43.926 INFO:tasks.workunit.client.0.smithi080.stdout:checking for gcc option to produce PIC... -fPIC -DPIC 2024-04-24T23:31:43.962 INFO:tasks.workunit.client.0.smithi080.stdout:checking if gcc PIC flag -fPIC -DPIC works... yes 2024-04-24T23:31:44.004 INFO:tasks.workunit.client.0.smithi080.stdout:checking if gcc static flag -static works... no 2024-04-24T23:31:44.062 INFO:tasks.workunit.client.0.smithi080.stdout:checking if gcc supports -c -o file.o... yes 2024-04-24T23:31:44.062 INFO:tasks.workunit.client.0.smithi080.stdout:checking if gcc supports -c -o file.o... (cached) yes 2024-04-24T23:31:44.069 INFO:tasks.workunit.client.0.smithi080.stdout:checking whether the gcc linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes 2024-04-24T23:31:44.126 INFO:tasks.workunit.client.0.smithi080.stdout:checking whether -lc should be explicitly linked in... no 2024-04-24T23:31:44.182 INFO:tasks.workunit.client.0.smithi080.stdout:checking dynamic linker characteristics... GNU/Linux ld.so 2024-04-24T23:31:44.183 INFO:tasks.workunit.client.0.smithi080.stdout:checking how to hardcode library paths into programs... immediate 2024-04-24T23:31:44.184 INFO:tasks.workunit.client.0.smithi080.stdout:checking whether stripping libraries is possible... yes 2024-04-24T23:31:44.184 INFO:tasks.workunit.client.0.smithi080.stdout:checking if libtool supports shared libraries... yes 2024-04-24T23:31:44.184 INFO:tasks.workunit.client.0.smithi080.stdout:checking whether to build shared libraries... yes 2024-04-24T23:31:44.184 INFO:tasks.workunit.client.0.smithi080.stdout:checking whether to build static libraries... yes 2024-04-24T23:31:44.189 INFO:tasks.workunit.client.0.smithi080.stdout:checking for gcc... (cached) gcc 2024-04-24T23:31:44.225 INFO:tasks.workunit.client.0.smithi080.stdout:checking whether we are using the GNU C compiler... (cached) yes 2024-04-24T23:31:44.225 INFO:tasks.workunit.client.0.smithi080.stdout:checking whether gcc accepts -g... (cached) yes 2024-04-24T23:31:44.225 INFO:tasks.workunit.client.0.smithi080.stdout:checking for gcc option to accept ISO C89... (cached) none needed 2024-04-24T23:31:44.225 INFO:tasks.workunit.client.0.smithi080.stdout:checking for gmake... /usr/bin/gmake 2024-04-24T23:31:44.225 INFO:tasks.workunit.client.0.smithi080.stderr:./configure: line 11968: test: $(SHELL): binary operator expected 2024-04-24T23:31:44.226 INFO:tasks.workunit.client.0.smithi080.stdout:checking for tar... /bin/tar 2024-04-24T23:31:44.226 INFO:tasks.workunit.client.0.smithi080.stdout:checking for gzip... /bin/gzip 2024-04-24T23:31:44.246 INFO:tasks.workunit.client.0.smithi080.stdout:checking whether gcc -MM is supported... yes 2024-04-24T23:31:44.246 INFO:tasks.workunit.client.0.smithi080.stdout:checking for sort... /bin/sort 2024-04-24T23:31:44.246 INFO:tasks.workunit.client.0.smithi080.stdout:checking whether ln -s works... yes 2024-04-24T23:31:44.247 INFO:tasks.workunit.client.0.smithi080.stdout:checking for rpm... /bin/rpm 2024-04-24T23:31:44.250 INFO:tasks.workunit.client.0.smithi080.stdout:checking for rpmbuild... no 2024-04-24T23:31:44.250 INFO:tasks.workunit.client.0.smithi080.stdout:checking for ANSI C header files... (cached) yes 2024-04-24T23:31:44.313 INFO:tasks.workunit.client.0.smithi080.stdout:checking assert.h usability... yes 2024-04-24T23:31:44.329 INFO:tasks.workunit.client.0.smithi080.stdout:checking assert.h presence... yes 2024-04-24T23:31:44.330 INFO:tasks.workunit.client.0.smithi080.stdout:checking for assert.h... yes 2024-04-24T23:31:44.379 INFO:tasks.workunit.client.0.smithi080.stdout:checking bstring.h usability... no 2024-04-24T23:31:44.408 INFO:tasks.workunit.client.0.smithi080.stdout:checking bstring.h presence... no 2024-04-24T23:31:44.408 INFO:tasks.workunit.client.0.smithi080.stdout:checking for bstring.h... no 2024-04-24T23:31:44.446 INFO:tasks.workunit.client.0.smithi080.stdout:checking libgen.h usability... yes 2024-04-24T23:31:44.446 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:31:44 smithi134 ceph-mon[32619]: pgmap v432: 129 pgs: 129 active+clean; 65 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.5 MiB/s wr, 783 op/s 2024-04-24T23:31:44.462 INFO:tasks.workunit.client.0.smithi080.stdout:checking libgen.h presence... yes 2024-04-24T23:31:44.462 INFO:tasks.workunit.client.0.smithi080.stdout:checking for libgen.h... yes 2024-04-24T23:31:44.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:31:44 smithi148 ceph-mon[32528]: pgmap v432: 129 pgs: 129 active+clean; 65 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.5 MiB/s wr, 783 op/s 2024-04-24T23:31:44.514 INFO:tasks.workunit.client.0.smithi080.stdout:checking dirent.h usability... yes 2024-04-24T23:31:44.532 INFO:tasks.workunit.client.0.smithi080.stdout:checking dirent.h presence... yes 2024-04-24T23:31:44.532 INFO:tasks.workunit.client.0.smithi080.stdout:checking for dirent.h... yes 2024-04-24T23:31:44.537 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:31:44 smithi080 ceph-mon[29152]: pgmap v432: 129 pgs: 129 active+clean; 65 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.5 MiB/s wr, 783 op/s 2024-04-24T23:31:44.577 INFO:tasks.workunit.client.0.smithi080.stdout:checking errno.h usability... yes 2024-04-24T23:31:44.594 INFO:tasks.workunit.client.0.smithi080.stdout:checking errno.h presence... yes 2024-04-24T23:31:44.594 INFO:tasks.workunit.client.0.smithi080.stdout:checking for errno.h... yes 2024-04-24T23:31:44.646 INFO:tasks.workunit.client.0.smithi080.stdout:checking malloc.h usability... yes 2024-04-24T23:31:44.664 INFO:tasks.workunit.client.0.smithi080.stdout:checking malloc.h presence... yes 2024-04-24T23:31:44.664 INFO:tasks.workunit.client.0.smithi080.stdout:checking for malloc.h... yes 2024-04-24T23:31:44.720 INFO:tasks.workunit.client.0.smithi080.stdout:checking uuid.h usability... no 2024-04-24T23:31:44.757 INFO:tasks.workunit.client.0.smithi080.stdout:checking uuid.h presence... no 2024-04-24T23:31:44.757 INFO:tasks.workunit.client.0.smithi080.stdout:checking for uuid.h... no 2024-04-24T23:31:44.819 INFO:tasks.workunit.client.0.smithi080.stdout:checking uuid/uuid.h usability... yes 2024-04-24T23:31:44.838 INFO:tasks.workunit.client.0.smithi080.stdout:checking uuid/uuid.h presence... yes 2024-04-24T23:31:44.838 INFO:tasks.workunit.client.0.smithi080.stdout:checking for uuid/uuid.h... yes 2024-04-24T23:31:44.898 INFO:tasks.workunit.client.0.smithi080.stdout:checking sys/uuid.h usability... no 2024-04-24T23:31:44.934 INFO:tasks.workunit.client.0.smithi080.stdout:checking sys/uuid.h presence... no 2024-04-24T23:31:44.935 INFO:tasks.workunit.client.0.smithi080.stdout:checking for sys/uuid.h... no 2024-04-24T23:31:44.983 INFO:tasks.workunit.client.0.smithi080.stdout:checking sys/file.h usability... yes 2024-04-24T23:31:45.000 INFO:tasks.workunit.client.0.smithi080.stdout:checking sys/file.h presence... yes 2024-04-24T23:31:45.000 INFO:tasks.workunit.client.0.smithi080.stdout:checking for sys/file.h... yes 2024-04-24T23:31:45.057 INFO:tasks.workunit.client.0.smithi080.stdout:checking sys/fcntl.h usability... yes 2024-04-24T23:31:45.075 INFO:tasks.workunit.client.0.smithi080.stdout:checking sys/fcntl.h presence... yes 2024-04-24T23:31:45.075 INFO:tasks.workunit.client.0.smithi080.stdout:checking for sys/fcntl.h... yes 2024-04-24T23:31:45.132 INFO:tasks.workunit.client.0.smithi080.stdout:checking sys/syssgi.h usability... no 2024-04-24T23:31:45.166 INFO:tasks.workunit.client.0.smithi080.stdout:checking sys/syssgi.h presence... no 2024-04-24T23:31:45.166 INFO:tasks.workunit.client.0.smithi080.stdout:checking for sys/syssgi.h... no 2024-04-24T23:31:45.314 INFO:tasks.workunit.client.0.smithi080.stdout:checking sys/param.h usability... yes 2024-04-24T23:31:45.334 INFO:tasks.workunit.client.0.smithi080.stdout:checking sys/param.h presence... yes 2024-04-24T23:31:45.334 INFO:tasks.workunit.client.0.smithi080.stdout:checking for sys/param.h... yes 2024-04-24T23:31:45.353 INFO:tasks.workunit.client.0.smithi080.stdout:checking for sys/stat.h... (cached) yes 2024-04-24T23:31:45.390 INFO:tasks.workunit.client.0.smithi080.stdout:checking sys/statvfs.h usability... yes 2024-04-24T23:31:45.407 INFO:tasks.workunit.client.0.smithi080.stdout:checking sys/statvfs.h presence... yes 2024-04-24T23:31:45.407 INFO:tasks.workunit.client.0.smithi080.stdout:checking for sys/statvfs.h... yes 2024-04-24T23:31:45.457 INFO:tasks.workunit.client.0.smithi080.stdout:checking sys/time.h usability... yes 2024-04-24T23:31:45.475 INFO:tasks.workunit.client.0.smithi080.stdout:checking sys/time.h presence... yes 2024-04-24T23:31:45.475 INFO:tasks.workunit.client.0.smithi080.stdout:checking for sys/time.h... yes 2024-04-24T23:31:45.525 INFO:tasks.workunit.client.0.smithi080.stdout:checking sys/ioctl.h usability... yes 2024-04-24T23:31:45.542 INFO:tasks.workunit.client.0.smithi080.stdout:checking sys/ioctl.h presence... yes 2024-04-24T23:31:45.542 INFO:tasks.workunit.client.0.smithi080.stdout:checking for sys/ioctl.h... yes 2024-04-24T23:31:45.605 INFO:tasks.workunit.client.0.smithi080.stdout:checking sys/wait.h usability... yes 2024-04-24T23:31:45.625 INFO:tasks.workunit.client.0.smithi080.stdout:checking sys/wait.h presence... yes 2024-04-24T23:31:45.625 INFO:tasks.workunit.client.0.smithi080.stdout:checking for sys/wait.h... yes 2024-04-24T23:31:45.643 INFO:tasks.workunit.client.0.smithi080.stdout:checking for sys/types.h... (cached) yes 2024-04-24T23:31:45.647 INFO:tasks.workunit.client.0.smithi080.stdout:checking for strings.h... (cached) yes 2024-04-24T23:31:45.686 INFO:tasks.workunit.client.0.smithi080.stdout:checking err.h usability... yes 2024-04-24T23:31:45.707 INFO:tasks.workunit.client.0.smithi080.stdout:checking err.h presence... yes 2024-04-24T23:31:45.707 INFO:tasks.workunit.client.0.smithi080.stdout:checking for err.h... yes 2024-04-24T23:31:45.755 INFO:tasks.workunit.client.0.smithi080.stdout:checking linux/falloc.h usability... yes 2024-04-24T23:31:45.771 INFO:tasks.workunit.client.0.smithi080.stdout:checking linux/falloc.h presence... yes 2024-04-24T23:31:45.771 INFO:tasks.workunit.client.0.smithi080.stdout:checking for linux/falloc.h... yes 2024-04-24T23:31:45.829 INFO:tasks.workunit.client.0.smithi080.stdout:checking sys/fs/xfs_fsops.h usability... no 2024-04-24T23:31:45.861 INFO:tasks.workunit.client.0.smithi080.stdout:checking sys/fs/xfs_fsops.h presence... no 2024-04-24T23:31:45.861 INFO:tasks.workunit.client.0.smithi080.stdout:checking for sys/fs/xfs_fsops.h... no 2024-04-24T23:31:45.908 INFO:tasks.workunit.client.0.smithi080.stdout:checking sys/fs/xfs_itable.h usability... no 2024-04-24T23:31:45.939 INFO:tasks.workunit.client.0.smithi080.stdout:checking sys/fs/xfs_itable.h presence... no 2024-04-24T23:31:45.939 INFO:tasks.workunit.client.0.smithi080.stdout:checking for sys/fs/xfs_itable.h... no 2024-04-24T23:31:45.987 INFO:tasks.workunit.client.0.smithi080.stdout:checking xfs/platform_defs.h usability... no 2024-04-24T23:31:46.019 INFO:tasks.workunit.client.0.smithi080.stdout:checking xfs/platform_defs.h presence... no 2024-04-24T23:31:46.020 INFO:tasks.workunit.client.0.smithi080.stdout:checking for xfs/platform_defs.h... no 2024-04-24T23:31:46.068 INFO:tasks.workunit.client.0.smithi080.stdout:checking btrfs/ioctl.h usability... no 2024-04-24T23:31:46.096 INFO:tasks.workunit.client.0.smithi080.stdout:checking btrfs/ioctl.h presence... no 2024-04-24T23:31:46.096 INFO:tasks.workunit.client.0.smithi080.stdout:checking for btrfs/ioctl.h... no 2024-04-24T23:31:46.139 INFO:tasks.workunit.client.0.smithi080.stdout:checking cifs/ioctl.h usability... no 2024-04-24T23:31:46.167 INFO:tasks.workunit.client.0.smithi080.stdout:checking cifs/ioctl.h presence... no 2024-04-24T23:31:46.167 INFO:tasks.workunit.client.0.smithi080.stdout:checking for cifs/ioctl.h... no 2024-04-24T23:31:46.212 INFO:tasks.workunit.client.0.smithi080.stdout:checking sys/mman.h usability... yes 2024-04-24T23:31:46.229 INFO:tasks.workunit.client.0.smithi080.stdout:checking sys/mman.h presence... yes 2024-04-24T23:31:46.229 INFO:tasks.workunit.client.0.smithi080.stdout:checking for sys/mman.h... yes 2024-04-24T23:31:46.266 INFO:tasks.workunit.client.0.smithi080.stdout:checking for xfs/xfs_log_format.h... no 2024-04-24T23:31:46.320 INFO:tasks.workunit.client.0.smithi080.stdout:checking for uuid_compare... no 2024-04-24T23:31:46.362 INFO:tasks.workunit.client.0.smithi080.stdout:checking for uuid_compare in -luuid... yes 2024-04-24T23:31:46.430 INFO:tasks.workunit.client.0.smithi080.stdout:checking for xfs/xfs.h... yes 2024-04-24T23:31:46.449 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:31:46 smithi134 ceph-mon[32619]: pgmap v433: 129 pgs: 129 active+clean; 66 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.6 MiB/s wr, 792 op/s 2024-04-24T23:31:46.465 INFO:tasks.workunit.client.0.smithi080.stdout:checking for xfs/libxfs.h... no 2024-04-24T23:31:46.496 INFO:tasks.workunit.client.0.smithi080.stdout:checking whether xlog_assign_lsn is declared... no 2024-04-24T23:31:46.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:31:46 smithi148 ceph-mon[32528]: pgmap v433: 129 pgs: 129 active+clean; 66 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.6 MiB/s wr, 792 op/s 2024-04-24T23:31:46.534 INFO:tasks.workunit.client.0.smithi080.stdout:checking for xfs/xqm.h... yes 2024-04-24T23:31:46.537 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:31:46 smithi080 ceph-mon[29152]: pgmap v433: 129 pgs: 129 active+clean; 66 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.6 MiB/s wr, 792 op/s 2024-04-24T23:31:46.585 INFO:tasks.workunit.client.0.smithi080.stdout:checking xfsctl from xfs/xfs.h... ok 2024-04-24T23:31:46.622 INFO:tasks.workunit.client.0.smithi080.stdout:checking xfs/handle.h usability... yes 2024-04-24T23:31:46.638 INFO:tasks.workunit.client.0.smithi080.stdout:checking xfs/handle.h presence... yes 2024-04-24T23:31:46.638 INFO:tasks.workunit.client.0.smithi080.stdout:checking for xfs/handle.h... yes 2024-04-24T23:31:46.678 INFO:tasks.workunit.client.0.smithi080.stdout:checking for attr_list_by_handle in -lhandle... yes 2024-04-24T23:31:46.738 INFO:tasks.workunit.client.0.smithi080.stdout:checking attr/attributes.h usability... yes 2024-04-24T23:31:46.755 INFO:tasks.workunit.client.0.smithi080.stdout:checking attr/attributes.h presence... yes 2024-04-24T23:31:46.756 INFO:tasks.workunit.client.0.smithi080.stdout:checking for attr/attributes.h... yes 2024-04-24T23:31:46.806 INFO:tasks.workunit.client.0.smithi080.stdout:checking sys/attributes.h usability... no 2024-04-24T23:31:46.837 INFO:tasks.workunit.client.0.smithi080.stdout:checking sys/attributes.h presence... no 2024-04-24T23:31:46.837 INFO:tasks.workunit.client.0.smithi080.stdout:checking for sys/attributes.h... no 2024-04-24T23:31:46.877 INFO:tasks.workunit.client.0.smithi080.stdout:checking for attr_list in -lattr... yes 2024-04-24T23:31:46.916 INFO:tasks.workunit.client.0.smithi080.stdout:checking for attr_set in -lattr... yes 2024-04-24T23:31:46.958 INFO:tasks.workunit.client.0.smithi080.stdout:checking sys/acl.h usability... yes 2024-04-24T23:31:46.976 INFO:tasks.workunit.client.0.smithi080.stdout:checking sys/acl.h presence... yes 2024-04-24T23:31:46.976 INFO:tasks.workunit.client.0.smithi080.stdout:checking for sys/acl.h... yes 2024-04-24T23:31:47.016 INFO:tasks.workunit.client.0.smithi080.stdout:checking acl/libacl.h usability... yes 2024-04-24T23:31:47.035 INFO:tasks.workunit.client.0.smithi080.stdout:checking acl/libacl.h presence... yes 2024-04-24T23:31:47.035 INFO:tasks.workunit.client.0.smithi080.stdout:checking for acl/libacl.h... yes 2024-04-24T23:31:47.076 INFO:tasks.workunit.client.0.smithi080.stdout:checking for acl_init in -lacl... yes 2024-04-24T23:31:47.139 INFO:tasks.workunit.client.0.smithi080.stdout:checking gdbm-ndbm.h usability... no 2024-04-24T23:31:47.168 INFO:tasks.workunit.client.0.smithi080.stdout:checking gdbm-ndbm.h presence... no 2024-04-24T23:31:47.169 INFO:tasks.workunit.client.0.smithi080.stdout:checking for gdbm-ndbm.h... no 2024-04-24T23:31:47.206 INFO:tasks.workunit.client.0.smithi080.stdout:checking gdbm/ndbm.h usability... no 2024-04-24T23:31:47.235 INFO:tasks.workunit.client.0.smithi080.stdout:checking gdbm/ndbm.h presence... no 2024-04-24T23:31:47.235 INFO:tasks.workunit.client.0.smithi080.stdout:checking for gdbm/ndbm.h... no 2024-04-24T23:31:47.272 INFO:tasks.workunit.client.0.smithi080.stdout:checking ndbm.h usability... no 2024-04-24T23:31:47.300 INFO:tasks.workunit.client.0.smithi080.stdout:checking ndbm.h presence... no 2024-04-24T23:31:47.300 INFO:tasks.workunit.client.0.smithi080.stdout:checking for ndbm.h... no 2024-04-24T23:31:47.334 INFO:tasks.workunit.client.0.smithi080.stdout:checking libaio.h usability... yes 2024-04-24T23:31:47.353 INFO:tasks.workunit.client.0.smithi080.stdout:checking libaio.h presence... yes 2024-04-24T23:31:47.353 INFO:tasks.workunit.client.0.smithi080.stdout:checking for libaio.h... yes 2024-04-24T23:31:47.404 INFO:tasks.workunit.client.0.smithi080.stdout:checking for dm_make_handle in -ldm... no 2024-04-24T23:31:47.405 INFO:tasks.workunit.client.0.smithi080.stdout: 2024-04-24T23:31:47.405 INFO:tasks.workunit.client.0.smithi080.stdout:WARNING: could not find a valid DMAPI base library. 2024-04-24T23:31:47.405 INFO:tasks.workunit.client.0.smithi080.stdout:If you want DMAPI support please install the data migration 2024-04-24T23:31:47.405 INFO:tasks.workunit.client.0.smithi080.stdout:API (dmapi) library package. Alternatively, run "make install" 2024-04-24T23:31:47.405 INFO:tasks.workunit.client.0.smithi080.stdout:from the dmapi source. 2024-04-24T23:31:47.405 INFO:tasks.workunit.client.0.smithi080.stdout: 2024-04-24T23:31:47.435 INFO:tasks.workunit.client.0.smithi080.stdout:checking linux/fiemap.h usability... yes 2024-04-24T23:31:47.451 INFO:tasks.workunit.client.0.smithi080.stdout:checking linux/fiemap.h presence... yes 2024-04-24T23:31:47.451 INFO:tasks.workunit.client.0.smithi080.stdout:checking for linux/fiemap.h... yes 2024-04-24T23:31:47.488 INFO:tasks.workunit.client.0.smithi080.stdout:checking for fallocate... yes 2024-04-24T23:31:47.537 INFO:tasks.workunit.client.0.smithi080.stdout:checking for open_by_handle_at... yes 2024-04-24T23:31:47.576 INFO:tasks.workunit.client.0.smithi080.stdout:checking sys/prctl.h usability... yes 2024-04-24T23:31:47.593 INFO:tasks.workunit.client.0.smithi080.stdout:checking sys/prctl.h presence... yes 2024-04-24T23:31:47.593 INFO:tasks.workunit.client.0.smithi080.stdout:checking for sys/prctl.h... yes 2024-04-24T23:31:47.626 INFO:tasks.workunit.client.0.smithi080.stdout:checking linux/fs.h usability... yes 2024-04-24T23:31:47.643 INFO:tasks.workunit.client.0.smithi080.stdout:checking linux/fs.h presence... yes 2024-04-24T23:31:47.643 INFO:tasks.workunit.client.0.smithi080.stdout:checking for linux/fs.h... yes 2024-04-24T23:31:47.709 INFO:tasks.workunit.client.0.smithi080.stdout:checking btrfsutil.h usability... no 2024-04-24T23:31:47.741 INFO:tasks.workunit.client.0.smithi080.stdout:checking btrfsutil.h presence... no 2024-04-24T23:31:47.742 INFO:tasks.workunit.client.0.smithi080.stdout:checking for btrfsutil.h... no 2024-04-24T23:31:47.781 INFO:tasks.workunit.client.0.smithi080.stdout:checking for copy_file_range... yes 2024-04-24T23:31:47.829 INFO:tasks.workunit.client.0.smithi080.stdout:checking for renameat2... yes 2024-04-24T23:31:47.848 INFO:tasks.workunit.client.0.smithi080.stdout:configure: creating ./config.status 2024-04-24T23:31:48.175 INFO:tasks.workunit.client.0.smithi080.stdout:config.status: creating include/builddefs 2024-04-24T23:31:48.189 INFO:tasks.workunit.client.0.smithi080.stdout:config.status: creating include/config.h 2024-04-24T23:31:48.200 INFO:tasks.workunit.client.0.smithi080.stdout:config.status: executing libtool commands 2024-04-24T23:31:48.280 INFO:tasks.workunit.client.0.smithi080.stdout:Building include 2024-04-24T23:31:48.280 INFO:tasks.workunit.client.0.smithi080.stdout:Building lib 2024-04-24T23:31:48.280 INFO:tasks.workunit.client.0.smithi080.stdout:Building tests 2024-04-24T23:31:48.285 INFO:tasks.workunit.client.0.smithi080.stdout: [LTDEP] 2024-04-24T23:31:48.446 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:31:48 smithi134 ceph-mon[32619]: pgmap v434: 129 pgs: 129 active+clean; 66 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 494 op/s 2024-04-24T23:31:48.457 INFO:tasks.workunit.client.0.smithi080.stdout: [CC] dataascii.lo 2024-04-24T23:31:48.458 INFO:tasks.workunit.client.0.smithi080.stdout: [CC] databin.lo 2024-04-24T23:31:48.458 INFO:tasks.workunit.client.0.smithi080.stdout: [CC] datapid.lo 2024-04-24T23:31:48.460 INFO:tasks.workunit.client.0.smithi080.stdout: [CC] file_lock.lo 2024-04-24T23:31:48.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:31:48 smithi148 ceph-mon[32528]: pgmap v434: 129 pgs: 129 active+clean; 66 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 494 op/s 2024-04-24T23:31:48.537 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:31:48 smithi080 ceph-mon[29152]: pgmap v434: 129 pgs: 129 active+clean; 66 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 494 op/s 2024-04-24T23:31:48.575 INFO:tasks.workunit.client.0.smithi080.stdout: [CC] forker.lo 2024-04-24T23:31:48.590 INFO:tasks.workunit.client.0.smithi080.stdout: [CC] pattern.lo 2024-04-24T23:31:48.629 INFO:tasks.workunit.client.0.smithi080.stdout: [CC] open_flags.lo 2024-04-24T23:31:48.641 INFO:tasks.workunit.client.0.smithi080.stdout: [CC] random_range.lo 2024-04-24T23:31:48.714 INFO:tasks.workunit.client.0.smithi080.stdout: [CC] string_to_tokens.lo 2024-04-24T23:31:48.730 INFO:tasks.workunit.client.0.smithi080.stdout: [CC] str_to_bytes.lo 2024-04-24T23:31:48.784 INFO:tasks.workunit.client.0.smithi080.stdout: [CC] tlibio.lo 2024-04-24T23:31:48.827 INFO:tasks.workunit.client.0.smithi080.stdout: [CC] write_log.lo 2024-04-24T23:31:48.880 INFO:tasks.workunit.client.0.smithi080.stdout: [CC] random.lo 2024-04-24T23:31:49.063 INFO:tasks.workunit.client.0.smithi080.stdout: [LD] 2024-04-24T23:31:49.183 INFO:tasks.workunit.client.0.smithi080.stdout:Building ltp 2024-04-24T23:31:49.183 INFO:tasks.workunit.client.0.smithi080.stdout:Building src 2024-04-24T23:31:49.183 INFO:tasks.workunit.client.0.smithi080.stdout:Building m4 2024-04-24T23:31:49.183 INFO:tasks.workunit.client.0.smithi080.stdout:Building common 2024-04-24T23:31:49.188 INFO:tasks.workunit.client.0.smithi080.stdout: [DEP] 2024-04-24T23:31:49.198 INFO:tasks.workunit.client.0.smithi080.stdout: [DEP] 2024-04-24T23:31:49.366 INFO:tasks.workunit.client.0.smithi080.stdout: [CC] doio 2024-04-24T23:31:49.366 INFO:tasks.workunit.client.0.smithi080.stdout: [CC] fsstress 2024-04-24T23:31:49.366 INFO:tasks.workunit.client.0.smithi080.stdout: [CC] fsx 2024-04-24T23:31:49.510 INFO:tasks.workunit.client.0.smithi080.stderr:fsstress.c: In function 'attr_list_path': 2024-04-24T23:31:49.510 INFO:tasks.workunit.client.0.smithi080.stderr:fsstress.c:859:9: warning: 'attr_list' is deprecated: Use listxattr or llistxattr instead [-Wdeprecated-declarations] 2024-04-24T23:31:49.510 INFO:tasks.workunit.client.0.smithi080.stderr: 859 | rval = attr_list(name->path, buffer, buffersize, flags, cursor); 2024-04-24T23:31:49.510 INFO:tasks.workunit.client.0.smithi080.stderr: | ^~~~ 2024-04-24T23:31:49.510 INFO:tasks.workunit.client.0.smithi080.stderr:In file included from ../src/global.h:25, 2024-04-24T23:31:49.510 INFO:tasks.workunit.client.0.smithi080.stderr: from fsstress.c:12: 2024-04-24T23:31:49.510 INFO:tasks.workunit.client.0.smithi080.stderr:/usr/include/attr/attributes.h:165:12: note: declared here 2024-04-24T23:31:49.510 INFO:tasks.workunit.client.0.smithi080.stderr: 165 | extern int attr_list(const char *__path, char *__buffer, const int __buffersize, 2024-04-24T23:31:49.511 INFO:tasks.workunit.client.0.smithi080.stderr: | ^~~~~~~~~ 2024-04-24T23:31:49.511 INFO:tasks.workunit.client.0.smithi080.stderr:fsstress.c: In function 'attr_remove_path': 2024-04-24T23:31:49.511 INFO:tasks.workunit.client.0.smithi080.stderr:fsstress.c:878:9: warning: 'attr_remove' is deprecated: Use removexattr or lremovexattr instead [-Wdeprecated-declarations] 2024-04-24T23:31:49.511 INFO:tasks.workunit.client.0.smithi080.stderr: 878 | rval = attr_remove(name->path, attrname, flags); 2024-04-24T23:31:49.511 INFO:tasks.workunit.client.0.smithi080.stderr: | ^~~~ 2024-04-24T23:31:49.511 INFO:tasks.workunit.client.0.smithi080.stderr:In file included from ../src/global.h:25, 2024-04-24T23:31:49.511 INFO:tasks.workunit.client.0.smithi080.stderr: from fsstress.c:12: 2024-04-24T23:31:49.511 INFO:tasks.workunit.client.0.smithi080.stderr:/usr/include/attr/attributes.h:152:12: note: declared here 2024-04-24T23:31:49.512 INFO:tasks.workunit.client.0.smithi080.stderr: 152 | extern int attr_remove (const char *__path, const char *__attrname, 2024-04-24T23:31:49.512 INFO:tasks.workunit.client.0.smithi080.stderr: | ^~~~~~~~~~~ 2024-04-24T23:31:49.512 INFO:tasks.workunit.client.0.smithi080.stderr:fsstress.c: In function 'attr_set_path': 2024-04-24T23:31:49.512 INFO:tasks.workunit.client.0.smithi080.stderr:fsstress.c:898:9: warning: 'attr_set' is deprecated: Use setxattr or lsetxattr instead [-Wdeprecated-declarations] 2024-04-24T23:31:49.512 INFO:tasks.workunit.client.0.smithi080.stderr: 898 | rval = attr_set(name->path, attrname, attrvalue, valuelength, flags); 2024-04-24T23:31:49.512 INFO:tasks.workunit.client.0.smithi080.stderr: | ^~~~ 2024-04-24T23:31:49.512 INFO:tasks.workunit.client.0.smithi080.stderr:In file included from ../src/global.h:25, 2024-04-24T23:31:49.512 INFO:tasks.workunit.client.0.smithi080.stderr: from fsstress.c:12: 2024-04-24T23:31:49.512 INFO:tasks.workunit.client.0.smithi080.stderr:/usr/include/attr/attributes.h:139:12: note: declared here 2024-04-24T23:31:49.512 INFO:tasks.workunit.client.0.smithi080.stderr: 139 | extern int attr_set (const char *__path, const char *__attrname, 2024-04-24T23:31:49.512 INFO:tasks.workunit.client.0.smithi080.stderr: | ^~~~~~~~ 2024-04-24T23:31:49.520 INFO:tasks.workunit.client.0.smithi080.stderr:fsstress.c: In function 'allocsp_f': 2024-04-24T23:31:49.520 INFO:tasks.workunit.client.0.smithi080.stderr:fsstress.c:2047:32: error: 'XFS_IOC_ALLOCSP64' undeclared (first use in this function); did you mean 'XFS_IOC_RESVSP64'? 2024-04-24T23:31:49.520 INFO:tasks.workunit.client.0.smithi080.stderr: 2047 | e = xfsctl(f.path, fd, XFS_IOC_ALLOCSP64, &fl) < 0 ? errno : 0; 2024-04-24T23:31:49.520 INFO:tasks.workunit.client.0.smithi080.stderr: | ^~~~~~~~~~~~~~~~~ 2024-04-24T23:31:49.520 INFO:tasks.workunit.client.0.smithi080.stderr: | XFS_IOC_RESVSP64 2024-04-24T23:31:49.520 INFO:tasks.workunit.client.0.smithi080.stderr:fsstress.c:2047:32: note: each undeclared identifier is reported only once for each function it appears in 2024-04-24T23:31:49.537 INFO:tasks.workunit.client.0.smithi080.stderr:fsstress.c: In function 'freesp_f': 2024-04-24T23:31:49.537 INFO:tasks.workunit.client.0.smithi080.stderr:fsstress.c:3640:32: error: 'XFS_IOC_FREESP64' undeclared (first use in this function); did you mean 'XFS_IOC_RESVSP64'? 2024-04-24T23:31:49.538 INFO:tasks.workunit.client.0.smithi080.stderr: 3640 | e = xfsctl(f.path, fd, XFS_IOC_FREESP64, &fl) < 0 ? errno : 0; 2024-04-24T23:31:49.538 INFO:tasks.workunit.client.0.smithi080.stderr: | ^~~~~~~~~~~~~~~~ 2024-04-24T23:31:49.538 INFO:tasks.workunit.client.0.smithi080.stderr: | XFS_IOC_RESVSP64 2024-04-24T23:31:49.571 INFO:tasks.workunit.client.0.smithi080.stderr:gmake[3]: *** [Makefile:47: fsstress] Error 1 2024-04-24T23:31:49.571 INFO:tasks.workunit.client.0.smithi080.stderr:gmake[3]: *** Waiting for unfinished jobs.... 2024-04-24T23:31:49.939 INFO:tasks.workunit.client.0.smithi080.stderr:doio.c: In function 'parse_cmdline': 2024-04-24T23:31:49.939 INFO:tasks.workunit.client.0.smithi080.stderr:doio.c:3113:33: warning: '%s' directive argument is null [-Wformat-overflow=] 2024-04-24T23:31:49.940 INFO:tasks.workunit.client.0.smithi080.stderr: 3113 | fprintf(stderr, 2024-04-24T23:31:49.940 INFO:tasks.workunit.client.0.smithi080.stderr: | ^~~~~~~~~~~~~~~ 2024-04-24T23:31:49.940 INFO:tasks.workunit.client.0.smithi080.stderr: 3114 | "%s%s: Illegal -C arg (%s). Must be one of: ", 2024-04-24T23:31:49.940 INFO:tasks.workunit.client.0.smithi080.stderr: | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-24T23:31:49.940 INFO:tasks.workunit.client.0.smithi080.stderr: 3115 | Prog, TagName, tok); 2024-04-24T23:31:49.940 INFO:tasks.workunit.client.0.smithi080.stderr: | ~~~~~~~~~~~~~~~~~~~ 2024-04-24T23:31:49.982 INFO:tasks.workunit.client.0.smithi080.stderr:fsx.c: In function 'main': 2024-04-24T23:31:49.982 INFO:tasks.workunit.client.0.smithi080.stderr:fsx.c:2662:33: warning: 'strncpy' specified bound 4096 equals destination size [-Wstringop-truncation] 2024-04-24T23:31:49.982 INFO:tasks.workunit.client.0.smithi080.stderr: 2662 | strncpy(opsfile, optarg, sizeof(opsfile)); 2024-04-24T23:31:49.982 INFO:tasks.workunit.client.0.smithi080.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-24T23:31:49.982 INFO:tasks.workunit.client.0.smithi080.stderr:fsx.c:2632:25: warning: 'strncpy' specified bound 1024 equals destination size [-Wstringop-truncation] 2024-04-24T23:31:49.982 INFO:tasks.workunit.client.0.smithi080.stderr: 2632 | strncpy(dname, optarg, sizeof(dname)); 2024-04-24T23:31:49.982 INFO:tasks.workunit.client.0.smithi080.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-24T23:31:50.084 INFO:tasks.workunit.client.0.smithi080.stderr:gmake[2]: *** [include/buildrules:30: ltp] Error 2 2024-04-24T23:31:50.084 INFO:tasks.workunit.client.0.smithi080.stderr:gmake[2]: *** Waiting for unfinished jobs.... 2024-04-24T23:31:50.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:31:50 smithi134 ceph-mon[32619]: pgmap v435: 129 pgs: 129 active+clean; 66 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 834 KiB/s rd, 1.4 MiB/s wr, 405 op/s 2024-04-24T23:31:50.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:31:50 smithi148 ceph-mon[32528]: pgmap v435: 129 pgs: 129 active+clean; 66 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 834 KiB/s rd, 1.4 MiB/s wr, 405 op/s 2024-04-24T23:31:50.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:31:50 smithi080 ceph-mon[29152]: pgmap v435: 129 pgs: 129 active+clean; 66 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 834 KiB/s rd, 1.4 MiB/s wr, 405 op/s 2024-04-24T23:31:50.772 INFO:tasks.workunit.client.0.smithi080.stdout: [DEP] 2024-04-24T23:31:50.773 INFO:tasks.workunit.client.0.smithi080.stdout: [DEP] 2024-04-24T23:31:50.773 INFO:tasks.workunit.client.0.smithi080.stdout: [DEP] 2024-04-24T23:31:50.997 INFO:tasks.workunit.client.0.smithi080.stdout: [CC] dirstress 2024-04-24T23:31:50.997 INFO:tasks.workunit.client.0.smithi080.stdout: [CC] fill 2024-04-24T23:31:50.997 INFO:tasks.workunit.client.0.smithi080.stdout: [CC] fill2 2024-04-24T23:31:50.997 INFO:tasks.workunit.client.0.smithi080.stdout: [CC] getpagesize 2024-04-24T23:31:51.162 INFO:tasks.workunit.client.0.smithi080.stdout: [CC] holes 2024-04-24T23:31:51.181 INFO:tasks.workunit.client.0.smithi080.stdout: [CC] lstat64 2024-04-24T23:31:51.255 INFO:tasks.workunit.client.0.smithi080.stdout: [CC] nametest 2024-04-24T23:31:51.256 INFO:tasks.workunit.client.0.smithi080.stdout: [CC] permname 2024-04-24T23:31:51.355 INFO:tasks.workunit.client.0.smithi080.stdout: [CC] randholes 2024-04-24T23:31:51.404 INFO:tasks.workunit.client.0.smithi080.stdout: [CC] runas 2024-04-24T23:31:51.470 INFO:tasks.workunit.client.0.smithi080.stdout: [CC] truncfile 2024-04-24T23:31:51.513 INFO:tasks.workunit.client.0.smithi080.stdout: [CC] usemem 2024-04-24T23:31:51.591 INFO:tasks.workunit.client.0.smithi080.stdout: [CC] mmapcat 2024-04-24T23:31:51.620 INFO:tasks.workunit.client.0.smithi080.stdout: [CC] append_reader 2024-04-24T23:31:51.640 INFO:tasks.workunit.client.0.smithi080.stdout: [CC] append_writer 2024-04-24T23:31:51.674 INFO:tasks.workunit.client.0.smithi080.stdout: [CC] dirperf 2024-04-24T23:31:51.750 INFO:tasks.workunit.client.0.smithi080.stdout: [CC] metaperf 2024-04-24T23:31:51.769 INFO:tasks.workunit.client.0.smithi080.stdout: [CC] devzero 2024-04-24T23:31:51.789 INFO:tasks.workunit.client.0.smithi080.stdout: [CC] feature 2024-04-24T23:31:51.942 INFO:tasks.workunit.client.0.smithi080.stdout: [CC] alloc 2024-04-24T23:31:51.942 INFO:tasks.workunit.client.0.smithi080.stdout: [CC] fault 2024-04-24T23:31:51.999 INFO:tasks.workunit.client.0.smithi080.stdout: [CC] fstest 2024-04-24T23:31:52.059 INFO:tasks.workunit.client.0.smithi080.stdout: [CC] t_access_root 2024-04-24T23:31:52.081 INFO:tasks.workunit.client.0.smithi080.stderr:alloc.c: In function 'main': 2024-04-24T23:31:52.081 INFO:tasks.workunit.client.0.smithi080.stderr:alloc.c:154:1: error: 'bozo' undeclared (first use in this function) 2024-04-24T23:31:52.081 INFO:tasks.workunit.client.0.smithi080.stderr: 154 | bozo! 2024-04-24T23:31:52.081 INFO:tasks.workunit.client.0.smithi080.stderr: | ^~~~ 2024-04-24T23:31:52.081 INFO:tasks.workunit.client.0.smithi080.stderr:alloc.c:154:1: note: each undeclared identifier is reported only once for each function it appears in 2024-04-24T23:31:52.081 INFO:tasks.workunit.client.0.smithi080.stderr:alloc.c:154:5: error: expected ';' before '!' token 2024-04-24T23:31:52.081 INFO:tasks.workunit.client.0.smithi080.stderr: 154 | bozo! 2024-04-24T23:31:52.082 INFO:tasks.workunit.client.0.smithi080.stderr: | ^ 2024-04-24T23:31:52.082 INFO:tasks.workunit.client.0.smithi080.stderr: | ; 2024-04-24T23:31:52.082 INFO:tasks.workunit.client.0.smithi080.stderr:alloc.c:184:25: error: 'rflag' undeclared (first use in this function); did you mean 'nflag'? 2024-04-24T23:31:52.082 INFO:tasks.workunit.client.0.smithi080.stderr: 184 | rflag = 1; 2024-04-24T23:31:52.082 INFO:tasks.workunit.client.0.smithi080.stderr: | ^~~~~ 2024-04-24T23:31:52.082 INFO:tasks.workunit.client.0.smithi080.stderr: | nflag 2024-04-24T23:31:52.082 INFO:tasks.workunit.client.0.smithi080.stderr:alloc.c:320:5: error: expected ';' before '!' token 2024-04-24T23:31:52.083 INFO:tasks.workunit.client.0.smithi080.stderr: 320 | bozo! 2024-04-24T23:31:52.083 INFO:tasks.workunit.client.0.smithi080.stderr: | ^ 2024-04-24T23:31:52.083 INFO:tasks.workunit.client.0.smithi080.stderr: | ; 2024-04-24T23:31:52.083 INFO:tasks.workunit.client.0.smithi080.stderr:alloc.c:128:25: warning: variable 'f' set but not used [-Wunused-but-set-variable] 2024-04-24T23:31:52.083 INFO:tasks.workunit.client.0.smithi080.stderr: 128 | struct flock64 f; 2024-04-24T23:31:52.083 INFO:tasks.workunit.client.0.smithi080.stderr: | ^ 2024-04-24T23:31:52.086 INFO:tasks.workunit.client.0.smithi080.stderr:gmake[3]: *** [Makefile:82: alloc] Error 1 2024-04-24T23:31:52.086 INFO:tasks.workunit.client.0.smithi080.stderr:gmake[3]: *** Waiting for unfinished jobs.... 2024-04-24T23:31:52.163 INFO:tasks.workunit.client.0.smithi080.stderr:fstest.c: In function 'run_child': 2024-04-24T23:31:52.164 INFO:tasks.workunit.client.0.smithi080.stderr:fstest.c:95:27: warning: '/file' directive writing 5 bytes into a region of size between 1 and 1024 [-Wformat-overflow=] 2024-04-24T23:31:52.164 INFO:tasks.workunit.client.0.smithi080.stderr: 95 | sprintf(fname, "%s/file%d", dir, fnum); 2024-04-24T23:31:52.164 INFO:tasks.workunit.client.0.smithi080.stderr: | ^~~~~ 2024-04-24T23:31:52.164 INFO:tasks.workunit.client.0.smithi080.stderr:fstest.c:95:24: note: directive argument in the range [0, 2147483647] 2024-04-24T23:31:52.164 INFO:tasks.workunit.client.0.smithi080.stderr: 95 | sprintf(fname, "%s/file%d", dir, fnum); 2024-04-24T23:31:52.164 INFO:tasks.workunit.client.0.smithi080.stderr: | ^~~~~~~~~~~ 2024-04-24T23:31:52.164 INFO:tasks.workunit.client.0.smithi080.stderr:fstest.c:95:9: note: 'sprintf' output between 7 and 1039 bytes into a destination of size 1024 2024-04-24T23:31:52.164 INFO:tasks.workunit.client.0.smithi080.stderr: 95 | sprintf(fname, "%s/file%d", dir, fnum); 2024-04-24T23:31:52.164 INFO:tasks.workunit.client.0.smithi080.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-24T23:31:52.164 INFO:tasks.workunit.client.0.smithi080.stderr:fstest.c:166:27: warning: '/file' directive writing 5 bytes into a region of size between 1 and 1024 [-Wformat-overflow=] 2024-04-24T23:31:52.164 INFO:tasks.workunit.client.0.smithi080.stderr: 166 | sprintf(fname, "%s/file%d", dir, fnum); 2024-04-24T23:31:52.164 INFO:tasks.workunit.client.0.smithi080.stderr: | ^~~~~ 2024-04-24T23:31:52.165 INFO:tasks.workunit.client.0.smithi080.stderr:fstest.c:166:24: note: directive argument in the range [0, 2147483647] 2024-04-24T23:31:52.165 INFO:tasks.workunit.client.0.smithi080.stderr: 166 | sprintf(fname, "%s/file%d", dir, fnum); 2024-04-24T23:31:52.165 INFO:tasks.workunit.client.0.smithi080.stderr: | ^~~~~~~~~~~ 2024-04-24T23:31:52.165 INFO:tasks.workunit.client.0.smithi080.stderr:fstest.c:166:9: note: 'sprintf' output between 7 and 1039 bytes into a destination of size 1024 2024-04-24T23:31:52.165 INFO:tasks.workunit.client.0.smithi080.stderr: 166 | sprintf(fname, "%s/file%d", dir, fnum); 2024-04-24T23:31:52.165 INFO:tasks.workunit.client.0.smithi080.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-24T23:31:52.235 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-24T23:31:52.235 INFO:tasks.workunit.client.0.smithi080.stderr:gmake[2]: *** [include/buildrules:30: src] Error 2 2024-04-24T23:31:52.235 INFO:tasks.workunit.client.0.smithi080.stderr:make[1]: *** [Makefile:53: default] Error 2 2024-04-24T23:31:52.236 INFO:tasks.workunit.client.0.smithi080.stderr:make: *** [Makefile:51: default] Error 2 2024-04-24T23:31:52.236 INFO:tasks.workunit:Stopping ['suites/fsx.sh'] on client.0... 2024-04-24T23:31:52.236 DEBUG:teuthology.orchestra.run.smithi080:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-24T23:31:52.446 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:31:52 smithi134 ceph-mon[32619]: pgmap v436: 129 pgs: 129 active+clean; 68 MiB data, 527 MiB used, 1.0 TiB / 1.0 TiB avail; 815 KiB/s rd, 901 KiB/s wr, 150 op/s 2024-04-24T23:31:52.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:31:52 smithi148 ceph-mon[32528]: pgmap v436: 129 pgs: 129 active+clean; 68 MiB data, 527 MiB used, 1.0 TiB / 1.0 TiB avail; 815 KiB/s rd, 901 KiB/s wr, 150 op/s 2024-04-24T23:31:52.520 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:31:52 smithi080 ceph-mon[29152]: pgmap v436: 129 pgs: 129 active+clean; 68 MiB data, 527 MiB used, 1.0 TiB / 1.0 TiB avail; 815 KiB/s rd, 901 KiB/s wr, 150 op/s 2024-04-24T23:31:52.521 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/fsx.sh) on smithi080 with status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=d2c45d977142e024a59b68a8f1dce8f0b9840551 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsx.sh' 2024-04-24T23:31:52.763 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=0d5992db68f441259d405d210fc7c8f7 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/fsx.sh) on smithi080 with status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=d2c45d977142e024a59b68a8f1dce8f0b9840551 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsx.sh' 2024-04-24T23:31:52.765 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-04-24T23:31:52.775 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-04-24T23:31:54.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:31:54 smithi134 ceph-mon[32619]: pgmap v437: 129 pgs: 129 active+clean; 68 MiB data, 531 MiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 659 KiB/s wr, 41 op/s 2024-04-24T23:31:54.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:31:54 smithi148 ceph-mon[32528]: pgmap v437: 129 pgs: 129 active+clean; 68 MiB data, 531 MiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 659 KiB/s wr, 41 op/s 2024-04-24T23:31:54.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:31:54 smithi080 ceph-mon[29152]: pgmap v437: 129 pgs: 129 active+clean; 68 MiB data, 531 MiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 659 KiB/s wr, 41 op/s 2024-04-24T23:31:56.446 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:31:56 smithi134 ceph-mon[32619]: pgmap v438: 129 pgs: 129 active+clean; 69 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 407 KiB/s wr, 33 op/s 2024-04-24T23:31:56.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:31:56 smithi148 ceph-mon[32528]: pgmap v438: 129 pgs: 129 active+clean; 69 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 407 KiB/s wr, 33 op/s 2024-04-24T23:31:56.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:31:56 smithi080 ceph-mon[29152]: pgmap v438: 129 pgs: 129 active+clean; 69 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 407 KiB/s wr, 33 op/s 2024-04-24T23:31:56.882 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T23:31:56.883 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T23:31:57.005 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:56.992+0000 7fdc11f12640 1 -- 172.21.15.80:0/1936765896 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdc0c103340 msgr2=0x7fdc0c109bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:57.006 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:56.992+0000 7fdc11f12640 1 --2- 172.21.15.80:0/1936765896 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdc0c103340 0x7fdc0c109bd0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fdc04009ca0 tx=0x7fdc0402a8c0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.006 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:56.992+0000 7fdc11f12640 1 -- 172.21.15.80:0/1936765896 shutdown_connections 2024-04-24T23:31:57.006 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:56.992+0000 7fdc11f12640 1 --2- 172.21.15.80:0/1936765896 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdc0c103340 0x7fdc0c109bd0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.006 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:56.992+0000 7fdc11f12640 1 --2- 172.21.15.80:0/1936765896 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdc0c102980 0x7fdc0c102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.007 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:56.992+0000 7fdc11f12640 1 --2- 172.21.15.80:0/1936765896 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc0c101790 0x7fdc0c101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.007 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:56.992+0000 7fdc11f12640 1 -- 172.21.15.80:0/1936765896 >> 172.21.15.80:0/1936765896 conn(0x7fdc0c0fcd20 msgr2=0x7fdc0c0ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:57.007 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:56.992+0000 7fdc11f12640 1 -- 172.21.15.80:0/1936765896 shutdown_connections 2024-04-24T23:31:57.007 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:56.992+0000 7fdc11f12640 1 -- 172.21.15.80:0/1936765896 wait complete. 2024-04-24T23:31:57.007 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:56.992+0000 7fdc11f12640 1 Processor -- start 2024-04-24T23:31:57.007 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:56.993+0000 7fdc11f12640 1 -- start start 2024-04-24T23:31:57.008 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:56.993+0000 7fdc11f12640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdc0c101790 0x7fdc0c0754f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:57.008 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:56.993+0000 7fdc11f12640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdc0c102980 0x7fdc0c073b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:57.008 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:56.993+0000 7fdc11f12640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc0c103340 0x7fdc0c074070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:57.008 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:56.993+0000 7fdc11f12640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fdc0c10d500 con 0x7fdc0c102980 2024-04-24T23:31:57.008 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:56.993+0000 7fdc11f12640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fdc0c10d380 con 0x7fdc0c103340 2024-04-24T23:31:57.008 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:56.993+0000 7fdc11f12640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fdc0c10d680 con 0x7fdc0c101790 2024-04-24T23:31:57.008 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:56.993+0000 7fdc0affd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdc0c102980 0x7fdc0c073b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:57.009 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:56.993+0000 7fdc0affd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdc0c102980 0x7fdc0c073b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53422/0 (socket says 172.21.15.80:53422) 2024-04-24T23:31:57.009 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:56.993+0000 7fdc0affd640 1 -- 172.21.15.80:0/1233074288 learned_addr learned my addr 172.21.15.80:0/1233074288 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:31:57.009 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:56.993+0000 7fdc0b7fe640 1 --2- 172.21.15.80:0/1233074288 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdc0c101790 0x7fdc0c0754f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:57.009 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:56.993+0000 7fdc0affd640 1 -- 172.21.15.80:0/1233074288 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdc0c101790 msgr2=0x7fdc0c0754f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:57.009 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:56.993+0000 7fdc0affd640 1 --2- 172.21.15.80:0/1233074288 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdc0c101790 0x7fdc0c0754f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.009 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:56.993+0000 7fdc0affd640 1 -- 172.21.15.80:0/1233074288 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc0c103340 msgr2=0x7fdc0c074070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:57.009 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:56.993+0000 7fdc0affd640 1 --2- 172.21.15.80:0/1233074288 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc0c103340 0x7fdc0c074070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.010 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:56.993+0000 7fdc0affd640 1 -- 172.21.15.80:0/1233074288 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc0c074900 con 0x7fdc0c102980 2024-04-24T23:31:57.010 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:56.993+0000 7fdc0affd640 1 --2- 172.21.15.80:0/1233074288 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdc0c102980 0x7fdc0c073b30 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fdc0000b4a0 tx=0x7fdc0000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:57.010 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:56.993+0000 7fdc08ff9640 1 -- 172.21.15.80:0/1233074288 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc00014070 con 0x7fdc0c102980 2024-04-24T23:31:57.010 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:56.993+0000 7fdc08ff9640 1 -- 172.21.15.80:0/1233074288 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc00010af0 con 0x7fdc0c102980 2024-04-24T23:31:57.010 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:56.993+0000 7fdc11f12640 1 -- 172.21.15.80:0/1233074288 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc0c1a1d00 con 0x7fdc0c102980 2024-04-24T23:31:57.011 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:56.993+0000 7fdc08ff9640 1 -- 172.21.15.80:0/1233074288 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc00017d10 con 0x7fdc0c102980 2024-04-24T23:31:57.011 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:56.994+0000 7fdc11f12640 1 -- 172.21.15.80:0/1233074288 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc0c1a2220 con 0x7fdc0c102980 2024-04-24T23:31:57.012 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:56.996+0000 7fdc08ff9640 1 -- 172.21.15.80:0/1233074288 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fdc00017070 con 0x7fdc0c102980 2024-04-24T23:31:57.012 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:56.996+0000 7fdc14c17740 1 Processor -- start 2024-04-24T23:31:57.012 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:56.996+0000 7fdc14c17740 1 -- start start 2024-04-24T23:31:57.012 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:56.996+0000 7fdc14c17740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561562bbe240 0x561562bbe660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:57.013 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:56.996+0000 7fdc14c17740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561562bbec30 0x561562bc3730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:57.013 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:56.996+0000 7fdc14c17740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561562bc3da0 0x561562bc6230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:57.013 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:56.996+0000 7fdc14c17740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x561562b226b0 con 0x561562bbec30 2024-04-24T23:31:57.013 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:56.996+0000 7fdc14c17740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x561562aca880 con 0x561562bc3da0 2024-04-24T23:31:57.014 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:56.996+0000 7fdc14c17740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x561562b1e670 con 0x561562bbe240 2024-04-24T23:31:57.014 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:56.996+0000 7fdc08ff9640 1 --2- 172.21.15.80:0/1233074288 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fdbf40a2b90 0x7fdbf40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:57.014 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:56.996+0000 7fdc0b7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561562bbec30 0x561562bc3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:57.015 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:56.996+0000 7fdc0b7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561562bbec30 0x561562bc3730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53432/0 (socket says 172.21.15.80:53432) 2024-04-24T23:31:57.015 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:56.996+0000 7fdc0b7fe640 1 -- 172.21.15.80:0/3894506902 learned_addr learned my addr 172.21.15.80:0/3894506902 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:31:57.015 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:56.996+0000 7fdc0bfff640 1 --2- 172.21.15.80:0/1233074288 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fdbf40a2b90 0x7fdbf40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:57.016 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:56.996+0000 7fdc0affd640 1 --2- 172.21.15.80:0/3894506902 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561562bc3da0 0x561562bc6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:57.016 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:56.996+0000 7fdc0bfff640 1 --2- 172.21.15.80:0/3894506902 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561562bbe240 0x561562bbe660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:57.016 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:56.996+0000 7fdc08ff9640 1 -- 172.21.15.80:0/1233074288 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fdc000ce910 con 0x7fdc0c102980 2024-04-24T23:31:57.016 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:56.997+0000 7fdc0b7fe640 1 -- 172.21.15.80:0/3894506902 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561562bbe240 msgr2=0x561562bbe660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:57.017 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:56.997+0000 7fdc0b7fe640 1 --2- 172.21.15.80:0/3894506902 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561562bbe240 0x561562bbe660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.017 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:56.997+0000 7fdc0b7fe640 1 -- 172.21.15.80:0/3894506902 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561562bc3da0 msgr2=0x561562bc6230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:57.017 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:56.997+0000 7fdc0b7fe640 1 --2- 172.21.15.80:0/3894506902 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561562bc3da0 0x561562bc6230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.018 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:56.997+0000 7fdc0b7fe640 1 -- 172.21.15.80:0/3894506902 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615629007d0 con 0x561562bbec30 2024-04-24T23:31:57.018 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:56.997+0000 7fdc0affd640 1 --2- 172.21.15.80:0/3894506902 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561562bc3da0 0x561562bc6230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:31:57.018 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:56.997+0000 7fdc0b7fe640 1 --2- 172.21.15.80:0/3894506902 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561562bbec30 0x561562bc3730 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fdbfc00b7f0 tx=0x7fdbfc00bcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44c09398eb3300b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:57.018 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:56.997+0000 7fdc0bfff640 1 --2- 172.21.15.80:0/1233074288 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fdbf40a2b90 0x7fdbf40a5050 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x561562bbf490 tx=0x7fdc040095c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:57.019 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:56.997+0000 7fdbf1ffb640 1 -- 172.21.15.80:0/3894506902 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbfc00d950 con 0x561562bbec30 2024-04-24T23:31:57.019 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:56.997+0000 7fdbf1ffb640 1 -- 172.21.15.80:0/3894506902 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbfc018070 con 0x561562bbec30 2024-04-24T23:31:57.019 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:56.997+0000 7fdbf1ffb640 1 -- 172.21.15.80:0/3894506902 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbfc0123e0 con 0x561562bbec30 2024-04-24T23:31:57.019 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:56.998+0000 7fdc14c17740 1 -- 172.21.15.80:0/3894506902 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561562bbec30 msgr2=0x561562bc3730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:57.020 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:56.998+0000 7fdc14c17740 1 --2- 172.21.15.80:0/3894506902 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561562bbec30 0x561562bc3730 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fdbfc00b7f0 tx=0x7fdbfc00bcb0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.020 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:56.998+0000 7fdc14c17740 1 -- 172.21.15.80:0/3894506902 shutdown_connections 2024-04-24T23:31:57.020 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:56.998+0000 7fdc14c17740 1 --2- 172.21.15.80:0/3894506902 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561562bc3da0 0x561562bc6230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.021 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:56.998+0000 7fdc14c17740 1 --2- 172.21.15.80:0/3894506902 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561562bbec30 0x561562bc3730 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.021 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:56.998+0000 7fdc14c17740 1 --2- 172.21.15.80:0/3894506902 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561562bbe240 0x561562bbe660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.021 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:56.998+0000 7fdc14c17740 1 -- 172.21.15.80:0/3894506902 >> 172.21.15.80:0/3894506902 conn(0x561562bb98b0 msgr2=0x561562bb9cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:57.021 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:56.998+0000 7fdc14c17740 1 -- 172.21.15.80:0/3894506902 shutdown_connections 2024-04-24T23:31:57.022 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:56.998+0000 7fdc14c17740 1 -- 172.21.15.80:0/3894506902 wait complete. 2024-04-24T23:31:57.022 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:56.998+0000 7fdc14c17740 1 -- start start 2024-04-24T23:31:57.022 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:56.998+0000 7fdc14c17740 1 Processor -- start 2024-04-24T23:31:57.023 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:56.998+0000 7fdc14c17740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561562bbe240 0x561562bc8720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:57.023 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:56.998+0000 7fdc14c17740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561562bbec30 0x561562bc8c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:57.023 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:56.998+0000 7fdc14c17740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561562bc3da0 0x561562bcfc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:57.023 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:56.998+0000 7fdc14c17740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x561562bc6d50 con 0x561562bbe240 2024-04-24T23:31:57.023 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:56.998+0000 7fdc14c17740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x561562addf50 con 0x561562bbec30 2024-04-24T23:31:57.023 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:56.998+0000 7fdc14c17740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x561562bc6ed0 con 0x561562bc3da0 2024-04-24T23:31:57.024 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:56.998+0000 7fdc14c17740 10 client.0 _ll_register_callbacks cb 0x561562b19000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:31:57.024 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:56.998+0000 7fdc0b7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561562bbe240 0x561562bc8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:57.024 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:56.999+0000 7fdc0b7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561562bbe240 0x561562bc8720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53444/0 (socket says 172.21.15.80:53444) 2024-04-24T23:31:57.024 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:56.999+0000 7fdc0b7fe640 1 -- 172.21.15.80:0/450273052 learned_addr learned my addr 172.21.15.80:0/450273052 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:31:57.024 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:56.999+0000 7fdc0affd640 1 --2- 172.21.15.80:0/450273052 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561562bbec30 0x561562bc8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:57.024 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:56.999+0000 7fdc0bfff640 1 --2- 172.21.15.80:0/450273052 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561562bc3da0 0x561562bcfc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:57.024 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:56.999+0000 7fdc0b7fe640 1 -- 172.21.15.80:0/450273052 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561562bc3da0 msgr2=0x561562bcfc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:57.025 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:56.999+0000 7fdc0b7fe640 1 --2- 172.21.15.80:0/450273052 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561562bc3da0 0x561562bcfc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.025 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:56.999+0000 7fdc0b7fe640 1 -- 172.21.15.80:0/450273052 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561562bbec30 msgr2=0x561562bc8c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:57.025 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:56.999+0000 7fdc0b7fe640 1 --2- 172.21.15.80:0/450273052 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561562bbec30 0x561562bc8c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.025 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:56.999+0000 7fdc0b7fe640 1 -- 172.21.15.80:0/450273052 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615628363f0 con 0x561562bbe240 2024-04-24T23:31:57.025 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:56.999+0000 7fdc0b7fe640 1 --2- 172.21.15.80:0/450273052 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561562bbe240 0x561562bc8720 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fdbfc000c00 tx=0x7fdbfc0136b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ed1b08735f2beef server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:57.025 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:56.999+0000 7fdbeaffd640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-04-24T23:31:57.026 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:56.999+0000 7fdbeaffd640 1 -- 172.21.15.80:0/450273052 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbfc022070 con 0x561562bbe240 2024-04-24T23:31:57.026 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:56.999+0000 7fdbeaffd640 1 -- 172.21.15.80:0/450273052 <== mon.0 v2:172.21.15.80:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fdbfc00dd60 con 0x561562bbe240 2024-04-24T23:31:57.026 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:56.999+0000 7fdc14c17740 1 -- 172.21.15.80:0/450273052 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdbfc018070 con 0x561562bbe240 2024-04-24T23:31:57.026 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:56.999+0000 7fdbeaffd640 1 -- 172.21.15.80:0/450273052 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdbfc02e050 con 0x561562bbe240 2024-04-24T23:31:57.026 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:56.999+0000 7fdc14c17740 10 client.15993 fetch_fsmap learned FSMap version 27 2024-04-24T23:31:57.027 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:56.999+0000 7fdc14c17740 1 -- 172.21.15.80:0/450273052 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5615627f60f0 con 0x561562bbe240 2024-04-24T23:31:57.027 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:56.999+0000 7fdbeaffd640 1 -- 172.21.15.80:0/450273052 <== mon.0 v2:172.21.15.80:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbfc018070 con 0x561562bbe240 2024-04-24T23:31:57.027 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.000+0000 7fdbeaffd640 1 -- 172.21.15.80:0/450273052 <== mon.0 v2:172.21.15.80:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fdbfc01c520 con 0x561562bbe240 2024-04-24T23:31:57.027 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.000+0000 7fdc14c17740 10 client.15993 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:31:57.027 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.000+0000 7fdc14c17740 10 client.15993 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:31:57.027 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.000+0000 7fdc14c17740 20 client.15993 populate_metadata read hostname 'smithi080' 2024-04-24T23:31:57.028 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.000+0000 7fdc14c17740 1 --2- 172.21.15.80:0/450273052 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x561562bd4180 0x561562bd6590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:57.028 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.000+0000 7fdc14c17740 4 client.15993 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:31:57.028 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.000+0000 7fdc14c17740 1 -- 172.21.15.80:0/450273052 --> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561562b2fbf0 con 0x561562bd4180 2024-04-24T23:31:57.028 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.000+0000 7fdc0affd640 1 --2- 172.21.15.80:0/450273052 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x561562bd4180 0x561562bd6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:57.029 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.000+0000 7fdbeaffd640 10 client.15993 ms_handle_connect on v2:172.21.15.148:6838/2952757594 2024-04-24T23:31:57.029 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.000+0000 7fdc0affd640 1 --2- 172.21.15.80:0/450273052 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x561562bd4180 0x561562bd6590 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ac0c0dc9e699673 server_cookie=ad470deaf2ac05dc in_seq=0 out_seq=0 2024-04-24T23:31:57.029 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.001+0000 7fdbeaffd640 1 -- 172.21.15.80:0/450273052 <== mds.0 v2:172.21.15.148:6838/2952757594 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561562b2fbf0 con 0x561562bd4180 2024-04-24T23:31:57.029 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.001+0000 7fdbeaffd640 10 client.15993 handle_command_reply: tid=0 2024-04-24T23:31:57.029 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.001+0000 7fdc14c17740 1 client.15993 shutdown 2024-04-24T23:31:57.029 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.001+0000 7fdc14c17740 1 -- 172.21.15.80:0/450273052 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561562bbe240 msgr2=0x561562bc8720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:57.030 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.001+0000 7fdc14c17740 1 --2- 172.21.15.80:0/450273052 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561562bbe240 0x561562bc8720 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fdbfc000c00 tx=0x7fdbfc0136b0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.030 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.001+0000 7fdc14c17740 1 -- 172.21.15.80:0/450273052 shutdown_connections 2024-04-24T23:31:57.030 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.001+0000 7fdc14c17740 1 --2- 172.21.15.80:0/450273052 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x561562bd4180 0x561562bd6590 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.030 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.001+0000 7fdc14c17740 1 --2- 172.21.15.80:0/450273052 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561562bc3da0 0x561562bcfc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.030 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.001+0000 7fdc14c17740 1 --2- 172.21.15.80:0/450273052 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561562bbec30 0x561562bc8c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.030 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.001+0000 7fdbeaffd640 0 client.15993 ms_handle_reset on v2:172.21.15.148:6838/2952757594 2024-04-24T23:31:57.031 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.001+0000 7fdc14c17740 1 --2- 172.21.15.80:0/450273052 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561562bbe240 0x561562bc8720 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.031 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.001+0000 7fdc14c17740 1 -- 172.21.15.80:0/450273052 >> 172.21.15.80:0/450273052 conn(0x561562bb98b0 msgr2=0x561562bb8af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:57.031 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.001+0000 7fdc14c17740 1 -- 172.21.15.80:0/450273052 shutdown_connections 2024-04-24T23:31:57.031 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.001+0000 7fdc14c17740 1 -- 172.21.15.80:0/450273052 wait complete. 2024-04-24T23:31:57.031 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.001+0000 7fdc14c17740 20 client.15993 trim_cache size 0 max 16384 2024-04-24T23:31:57.040 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.019+0000 7fdc14c17740 1 Processor -- start 2024-04-24T23:31:57.040 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.019+0000 7fdc14c17740 1 -- start start 2024-04-24T23:31:57.040 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.019+0000 7fdc14c17740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561562bbe240 0x561562bb8970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:57.040 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.019+0000 7fdc14c17740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561562bbec30 0x561562bcfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:57.040 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.019+0000 7fdc14c17740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561562bc3da0 0x561562bc7d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:57.041 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.019+0000 7fdc14c17740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fdc0c1af160 con 0x561562bbec30 2024-04-24T23:31:57.041 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.019+0000 7fdc14c17740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fdc0c1aca30 con 0x561562bc3da0 2024-04-24T23:31:57.041 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.019+0000 7fdc14c17740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fdc0c1aa880 con 0x561562bbe240 2024-04-24T23:31:57.041 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.019+0000 7fdc0affd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561562bbec30 0x561562bcfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:57.041 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.020+0000 7fdc0affd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561562bbec30 0x561562bcfbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53456/0 (socket says 172.21.15.80:53456) 2024-04-24T23:31:57.042 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.020+0000 7fdc0affd640 1 -- 172.21.15.80:0/1619577071 learned_addr learned my addr 172.21.15.80:0/1619577071 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:31:57.042 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.020+0000 7fdc0affd640 1 -- 172.21.15.80:0/1619577071 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561562bbe240 msgr2=0x561562bb8970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:57.042 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.020+0000 7fdc0affd640 1 --2- 172.21.15.80:0/1619577071 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561562bbe240 0x561562bb8970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.042 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.020+0000 7fdc0affd640 1 -- 172.21.15.80:0/1619577071 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561562bc3da0 msgr2=0x561562bc7d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:57.042 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.020+0000 7fdc0affd640 1 --2- 172.21.15.80:0/1619577071 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561562bc3da0 0x561562bc7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.043 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.020+0000 7fdc0affd640 1 -- 172.21.15.80:0/1619577071 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561562bba620 con 0x561562bbec30 2024-04-24T23:31:57.043 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.020+0000 7fdc0affd640 1 --2- 172.21.15.80:0/1619577071 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561562bbec30 0x561562bcfbf0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fdc000cc040 tx=0x7fdc00012040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4e50910dddac699 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:57.043 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.020+0000 7fdbf17fa640 1 -- 172.21.15.80:0/1619577071 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc0001e040 con 0x561562bbec30 2024-04-24T23:31:57.043 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.020+0000 7fdbf17fa640 1 -- 172.21.15.80:0/1619577071 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc000b8070 con 0x561562bbec30 2024-04-24T23:31:57.044 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.020+0000 7fdbf17fa640 1 -- 172.21.15.80:0/1619577071 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc00010ca0 con 0x561562bbec30 2024-04-24T23:31:57.044 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.021+0000 7fdc14c17740 1 -- 172.21.15.80:0/1619577071 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561562bbec30 msgr2=0x561562bcfbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:57.044 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.021+0000 7fdc14c17740 1 --2- 172.21.15.80:0/1619577071 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561562bbec30 0x561562bcfbf0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fdc000cc040 tx=0x7fdc00012040 comp rx=0 tx=0).stop 2024-04-24T23:31:57.044 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.021+0000 7fdc14c17740 1 -- 172.21.15.80:0/1619577071 shutdown_connections 2024-04-24T23:31:57.044 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.021+0000 7fdc14c17740 1 --2- 172.21.15.80:0/1619577071 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561562bc3da0 0x561562bc7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.044 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.021+0000 7fdc14c17740 1 --2- 172.21.15.80:0/1619577071 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561562bbec30 0x561562bcfbf0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.044 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.021+0000 7fdc14c17740 1 --2- 172.21.15.80:0/1619577071 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561562bbe240 0x561562bb8970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.045 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.021+0000 7fdc14c17740 1 -- 172.21.15.80:0/1619577071 >> 172.21.15.80:0/1619577071 conn(0x561562bb98b0 msgr2=0x561562bb8260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:57.045 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.021+0000 7fdc14c17740 1 -- 172.21.15.80:0/1619577071 shutdown_connections 2024-04-24T23:31:57.045 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.021+0000 7fdc14c17740 1 -- 172.21.15.80:0/1619577071 wait complete. 2024-04-24T23:31:57.045 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.021+0000 7fdc14c17740 1 -- start start 2024-04-24T23:31:57.045 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.021+0000 7fdc14c17740 1 Processor -- start 2024-04-24T23:31:57.045 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.021+0000 7fdc14c17740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561562bbe240 0x561562bccde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:57.046 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.021+0000 7fdc14c17740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561562bbec30 0x561562bcd320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:57.046 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.021+0000 7fdc14c17740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561562bc3da0 0x561562bce330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:57.046 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.021+0000 7fdc14c17740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fdc0c1aaef0 con 0x561562bc3da0 2024-04-24T23:31:57.046 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.021+0000 7fdc14c17740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fdc0c1ac360 con 0x561562bbec30 2024-04-24T23:31:57.046 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.021+0000 7fdc14c17740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fdc0c1ab5c0 con 0x561562bbe240 2024-04-24T23:31:57.046 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.021+0000 7fdc14c17740 10 client.0 _ll_register_callbacks cb 0x7fdbfc00b670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:31:57.046 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.021+0000 7fdc0bfff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561562bc3da0 0x561562bce330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:57.047 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.022+0000 7fdc0bfff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561562bc3da0 0x561562bce330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53472/0 (socket says 172.21.15.80:53472) 2024-04-24T23:31:57.047 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.022+0000 7fdc0bfff640 1 -- 172.21.15.80:0/1213522133 learned_addr learned my addr 172.21.15.80:0/1213522133 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:31:57.047 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.022+0000 7fdc0affd640 1 --2- 172.21.15.80:0/1213522133 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561562bbec30 0x561562bcd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:57.047 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.022+0000 7fdc0b7fe640 1 --2- 172.21.15.80:0/1213522133 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561562bbe240 0x561562bccde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:57.047 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.022+0000 7fdc0bfff640 1 -- 172.21.15.80:0/1213522133 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561562bbe240 msgr2=0x561562bccde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:57.047 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.022+0000 7fdc0bfff640 1 --2- 172.21.15.80:0/1213522133 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561562bbe240 0x561562bccde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.047 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.022+0000 7fdc0bfff640 1 -- 172.21.15.80:0/1213522133 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561562bbec30 msgr2=0x561562bcd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:57.048 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.022+0000 7fdc0bfff640 1 --2- 172.21.15.80:0/1213522133 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561562bbec30 0x561562bcd320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.048 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.022+0000 7fdc0bfff640 1 -- 172.21.15.80:0/1213522133 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561562bba320 con 0x561562bc3da0 2024-04-24T23:31:57.048 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.022+0000 7fdc0bfff640 1 --2- 172.21.15.80:0/1213522133 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561562bc3da0 0x561562bce330 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fdc04035000 tx=0x7fdc0402afc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=388d78bebdaf6185 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:57.048 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.022+0000 7fdbeaffd640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-04-24T23:31:57.048 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.022+0000 7fdbeaffd640 1 -- 172.21.15.80:0/1213522133 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc0404f030 con 0x561562bc3da0 2024-04-24T23:31:57.049 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.022+0000 7fdbeaffd640 1 -- 172.21.15.80:0/1213522133 <== mon.0 v2:172.21.15.80:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fdc04004320 con 0x561562bc3da0 2024-04-24T23:31:57.049 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.022+0000 7fdc14c17740 1 -- 172.21.15.80:0/1213522133 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc000b8070 con 0x561562bc3da0 2024-04-24T23:31:57.049 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.022+0000 7fdbeaffd640 1 -- 172.21.15.80:0/1213522133 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdc04004b40 con 0x561562bc3da0 2024-04-24T23:31:57.049 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.022+0000 7fdc14c17740 10 client.15999 fetch_fsmap learned FSMap version 27 2024-04-24T23:31:57.049 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.022+0000 7fdc14c17740 1 -- 172.21.15.80:0/1213522133 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561562bcfad0 con 0x561562bc3da0 2024-04-24T23:31:57.049 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.022+0000 7fdbeaffd640 1 -- 172.21.15.80:0/1213522133 <== mon.0 v2:172.21.15.80:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc000b8070 con 0x561562bc3da0 2024-04-24T23:31:57.049 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.023+0000 7fdbeaffd640 1 -- 172.21.15.80:0/1213522133 <== mon.0 v2:172.21.15.80:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fdc0404a040 con 0x561562bc3da0 2024-04-24T23:31:57.050 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.023+0000 7fdc14c17740 10 client.15999 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:31:57.050 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.023+0000 7fdc14c17740 10 client.15999 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:31:57.050 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.023+0000 7fdc14c17740 20 client.15999 populate_metadata read hostname 'smithi080' 2024-04-24T23:31:57.050 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.023+0000 7fdc14c17740 1 --2- 172.21.15.80:0/1213522133 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x561562bd4180 0x561562c3cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:57.050 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.023+0000 7fdc14c17740 4 client.15999 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:31:57.050 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.023+0000 7fdc14c17740 1 -- 172.21.15.80:0/1213522133 --> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561562b29f40 con 0x561562bd4180 2024-04-24T23:31:57.051 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.023+0000 7fdc0b7fe640 1 --2- 172.21.15.80:0/1213522133 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x561562bd4180 0x561562c3cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:57.051 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.023+0000 7fdc0b7fe640 1 --2- 172.21.15.80:0/1213522133 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x561562bd4180 0x561562c3cbb0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e153f5e31921dbd server_cookie=b75e5398c60e3e82 in_seq=0 out_seq=0 2024-04-24T23:31:57.051 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.023+0000 7fdbeaffd640 10 client.15999 ms_handle_connect on v2:172.21.15.148:6838/2952757594 2024-04-24T23:31:57.051 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.024+0000 7fdbeaffd640 1 -- 172.21.15.80:0/1213522133 <== mds.0 v2:172.21.15.148:6838/2952757594 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x561562b29f40 con 0x561562bd4180 2024-04-24T23:31:57.051 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.024+0000 7fdbeaffd640 10 client.15999 handle_command_reply: tid=0 2024-04-24T23:31:57.052 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.024+0000 7fdc14c17740 1 client.15999 shutdown 2024-04-24T23:31:57.052 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.024+0000 7fdc14c17740 1 -- 172.21.15.80:0/1213522133 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561562bc3da0 msgr2=0x561562bce330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:57.052 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.024+0000 7fdc14c17740 1 --2- 172.21.15.80:0/1213522133 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561562bc3da0 0x561562bce330 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fdc04035000 tx=0x7fdc0402afc0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.053 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.024+0000 7fdc14c17740 1 -- 172.21.15.80:0/1213522133 shutdown_connections 2024-04-24T23:31:57.053 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.024+0000 7fdc14c17740 1 --2- 172.21.15.80:0/1213522133 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x561562bd4180 0x561562c3cbb0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.053 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.024+0000 7fdc14c17740 1 --2- 172.21.15.80:0/1213522133 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561562bc3da0 0x561562bce330 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.053 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.024+0000 7fdc14c17740 1 --2- 172.21.15.80:0/1213522133 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561562bbec30 0x561562bcd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.024+0000 7fdbeaffd640 0 client.15999 ms_handle_reset on v2:172.21.15.148:6838/2952757594 2024-04-24T23:31:57.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.024+0000 7fdc14c17740 1 --2- 172.21.15.80:0/1213522133 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561562bbe240 0x561562bccde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.024+0000 7fdc14c17740 1 -- 172.21.15.80:0/1213522133 >> 172.21.15.80:0/1213522133 conn(0x561562bb98b0 msgr2=0x561562bb8af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:57.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.024+0000 7fdc14c17740 1 -- 172.21.15.80:0/1213522133 shutdown_connections 2024-04-24T23:31:57.055 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.024+0000 7fdc14c17740 1 -- 172.21.15.80:0/1213522133 wait complete. 2024-04-24T23:31:57.055 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.024+0000 7fdc14c17740 20 client.15999 trim_cache size 0 max 16384 2024-04-24T23:31:57.056 INFO:teuthology.orchestra.run.smithi080.stdout:{ 2024-04-24T23:31:57.056 INFO:teuthology.orchestra.run.smithi080.stdout: "status": "no active scrubs running", 2024-04-24T23:31:57.056 INFO:teuthology.orchestra.run.smithi080.stdout: "scrubs": {} 2024-04-24T23:31:57.056 INFO:teuthology.orchestra.run.smithi080.stdout:} 2024-04-24T23:31:57.059 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.024+0000 7fdc11f12640 1 -- 172.21.15.80:0/1233074288 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fdbf40a2b90 msgr2=0x7fdbf40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:57.059 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.024+0000 7fdc11f12640 1 --2- 172.21.15.80:0/1233074288 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fdbf40a2b90 0x7fdbf40a5050 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x561562bbf490 tx=0x7fdc040095c0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.060 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.024+0000 7fdc11f12640 1 -- 172.21.15.80:0/1233074288 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdc0c102980 msgr2=0x7fdc0c073b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:57.060 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.024+0000 7fdc11f12640 1 --2- 172.21.15.80:0/1233074288 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdc0c102980 0x7fdc0c073b30 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fdc0000b4a0 tx=0x7fdc0000b960 comp rx=0 tx=0).stop 2024-04-24T23:31:57.060 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.024+0000 7fdc11f12640 1 -- 172.21.15.80:0/1233074288 shutdown_connections 2024-04-24T23:31:57.060 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.025+0000 7fdc11f12640 1 --2- 172.21.15.80:0/1233074288 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fdbf40a2b90 0x7fdbf40a5050 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.060 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.025+0000 7fdc11f12640 1 --2- 172.21.15.80:0/1233074288 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc0c103340 0x7fdc0c074070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.061 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.025+0000 7fdc11f12640 1 --2- 172.21.15.80:0/1233074288 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdc0c102980 0x7fdc0c073b30 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.061 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.025+0000 7fdc11f12640 1 --2- 172.21.15.80:0/1233074288 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdc0c101790 0x7fdc0c0754f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.061 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.025+0000 7fdc11f12640 1 -- 172.21.15.80:0/1233074288 >> 172.21.15.80:0/1233074288 conn(0x7fdc0c0fcd20 msgr2=0x7fdc0c105d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:57.061 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.025+0000 7fdc11f12640 1 -- 172.21.15.80:0/1233074288 shutdown_connections 2024-04-24T23:31:57.062 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.025+0000 7fdc11f12640 1 -- 172.21.15.80:0/1233074288 wait complete. 2024-04-24T23:31:57.062 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T23:31:57.063 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:31:57.164 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.161+0000 7fc3db091640 1 -- 172.21.15.80:0/2707871669 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc3d41017b0 msgr2=0x7fc3d4101bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:57.164 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.161+0000 7fc3db091640 1 --2- 172.21.15.80:0/2707871669 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc3d41017b0 0x7fc3d4101bd0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fc3c8009900 tx=0x7fc3c802a410 comp rx=0 tx=0).stop 2024-04-24T23:31:57.164 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.161+0000 7fc3db091640 1 -- 172.21.15.80:0/2707871669 shutdown_connections 2024-04-24T23:31:57.165 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.161+0000 7fc3db091640 1 --2- 172.21.15.80:0/2707871669 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc3d4103360 0x7fc3d4109bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.165 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.161+0000 7fc3db091640 1 --2- 172.21.15.80:0/2707871669 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc3d41029a0 0x7fc3d4102e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.165 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.161+0000 7fc3db091640 1 --2- 172.21.15.80:0/2707871669 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc3d41017b0 0x7fc3d4101bd0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.165 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.161+0000 7fc3db091640 1 -- 172.21.15.80:0/2707871669 >> 172.21.15.80:0/2707871669 conn(0x7fc3d40fcd40 msgr2=0x7fc3d40ff160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:57.165 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.161+0000 7fc3db091640 1 -- 172.21.15.80:0/2707871669 shutdown_connections 2024-04-24T23:31:57.165 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.161+0000 7fc3db091640 1 -- 172.21.15.80:0/2707871669 wait complete. 2024-04-24T23:31:57.166 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.161+0000 7fc3db091640 1 Processor -- start 2024-04-24T23:31:57.166 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.161+0000 7fc3db091640 1 -- start start 2024-04-24T23:31:57.166 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.162+0000 7fc3db091640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc3d41017b0 0x7fc3d4197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:57.166 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.162+0000 7fc3db091640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc3d41029a0 0x7fc3d4198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:57.166 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.162+0000 7fc3db091640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc3d4103360 0x7fc3d419f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:57.166 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.162+0000 7fc3db091640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fc3d410d400 con 0x7fc3d41017b0 2024-04-24T23:31:57.167 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.162+0000 7fc3db091640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc3d410d280 con 0x7fc3d41029a0 2024-04-24T23:31:57.167 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.162+0000 7fc3db091640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc3d410d580 con 0x7fc3d4103360 2024-04-24T23:31:57.167 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.162+0000 7fc3d8e2d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc3d41017b0 0x7fc3d4197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:57.167 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.162+0000 7fc3d8e2d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc3d41017b0 0x7fc3d4197ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53494/0 (socket says 172.21.15.80:53494) 2024-04-24T23:31:57.167 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.162+0000 7fc3d8e2d640 1 -- 172.21.15.80:0/1349285704 learned_addr learned my addr 172.21.15.80:0/1349285704 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:31:57.167 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.162+0000 7fc3d3fff640 1 --2- 172.21.15.80:0/1349285704 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc3d41029a0 0x7fc3d4198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:57.167 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.162+0000 7fc3d8e2d640 1 -- 172.21.15.80:0/1349285704 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc3d4103360 msgr2=0x7fc3d419f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:57.167 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.162+0000 7fc3d8e2d640 1 --2- 172.21.15.80:0/1349285704 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc3d4103360 0x7fc3d419f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.168 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.162+0000 7fc3d8e2d640 1 -- 172.21.15.80:0/1349285704 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc3d41029a0 msgr2=0x7fc3d4198010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:57.168 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.162+0000 7fc3d8e2d640 1 --2- 172.21.15.80:0/1349285704 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc3d41029a0 0x7fc3d4198010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.168 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.162+0000 7fc3d8e2d640 1 -- 172.21.15.80:0/1349285704 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3d419f6f0 con 0x7fc3d41017b0 2024-04-24T23:31:57.168 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.162+0000 7fc3d3fff640 1 --2- 172.21.15.80:0/1349285704 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc3d41029a0 0x7fc3d4198010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:31:57.168 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.162+0000 7fc3d8e2d640 1 --2- 172.21.15.80:0/1349285704 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc3d41017b0 0x7fc3d4197ad0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fc3c8031610 tx=0x7fc3c80043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:57.168 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.162+0000 7fc3d1ffb640 1 -- 172.21.15.80:0/1349285704 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3c803f030 con 0x7fc3d41017b0 2024-04-24T23:31:57.169 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.162+0000 7fc3db091640 1 -- 172.21.15.80:0/1349285704 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3d419f980 con 0x7fc3d41017b0 2024-04-24T23:31:57.169 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.162+0000 7fc3d1ffb640 1 -- 172.21.15.80:0/1349285704 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3c8003ce0 con 0x7fc3d41017b0 2024-04-24T23:31:57.169 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.162+0000 7fc3d1ffb640 1 -- 172.21.15.80:0/1349285704 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3c8035070 con 0x7fc3d41017b0 2024-04-24T23:31:57.169 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.163+0000 7fc3db091640 1 -- 172.21.15.80:0/1349285704 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3d419fec0 con 0x7fc3d41017b0 2024-04-24T23:31:57.169 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.164+0000 7fc3db091640 1 -- 172.21.15.80:0/1349285704 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3d4108af0 con 0x7fc3d41017b0 2024-04-24T23:31:57.170 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.165+0000 7fc3d1ffb640 1 -- 172.21.15.80:0/1349285704 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc3c8042030 con 0x7fc3d41017b0 2024-04-24T23:31:57.170 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.165+0000 7fc3d1ffb640 1 --2- 172.21.15.80:0/1349285704 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fc3c40a2c10 0x7fc3c40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:57.171 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.165+0000 7fc3d3fff640 1 --2- 172.21.15.80:0/1349285704 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fc3c40a2c10 0x7fc3c40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:57.171 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.165+0000 7fc3d1ffb640 1 -- 172.21.15.80:0/1349285704 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fc3c80efce0 con 0x7fc3d41017b0 2024-04-24T23:31:57.171 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.166+0000 7fc3d3fff640 1 --2- 172.21.15.80:0/1349285704 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fc3c40a2c10 0x7fc3c40a50d0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fc3d4198fe0 tx=0x7fc3c0005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:57.171 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.169+0000 7fc3d1ffb640 1 -- 172.21.15.80:0/1349285704 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fc3c80bc870 con 0x7fc3d41017b0 2024-04-24T23:31:57.386 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.386+0000 7fc3db091640 1 -- 172.21.15.80:0/1349285704 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc3d402d170 con 0x7fc3d41017b0 2024-04-24T23:31:57.388 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.387+0000 7fc3d1ffb640 1 -- 172.21.15.80:0/1349285704 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11875 (secure 0 0 0) 0x7fc3c80bc170 con 0x7fc3d41017b0 2024-04-24T23:31:57.388 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-24T23:31:57.389 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/891591085","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":891591085},{"type":"v1","addr":"172.21.15.80:6835","nonce":891591085}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6837/259529026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6836","nonce":259529026},{"type":"v1","addr":"172.21.15.80:6837","nonce":259529026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14646,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6839/4194284854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6838","nonce":4194284854},{"type":"v1","addr":"172.21.15.80:6839","nonce":4194284854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14658,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6841/3145730958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6840","nonce":3145730958},{"type":"v1","addr":"172.21.15.80:6841","nonce":3145730958}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24395,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/3736784385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3736784385},{"type":"v1","addr":"172.21.15.134:6833","nonce":3736784385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/124777462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":124777462},{"type":"v1","addr":"172.21.15.148:6833","nonce":124777462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T23:27:19.300566+0000","modified":"2024-04-24T23:31:19.604599+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24496,"mds_1":24488,"mds_2":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/1399801120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1399801120},{"type":"v1","addr":"172.21.15.134:6835","nonce":1399801120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/430512704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":430512704},{"type":"v1","addr":"172.21.15.134:6837","nonce":430512704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/3002319310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3002319310},{"type":"v1","addr":"172.21.15.148:6835","nonce":3002319310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.148:6837/856528403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":856528403},{"type":"v1","addr":"172.21.15.148:6837","nonce":856528403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6839/1743815021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":1743815021},{"type":"v1","addr":"172.21.15.134:6839","nonce":1743815021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.148:6839/2952757594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":2952757594},{"type":"v1","addr":"172.21.15.148:6839","nonce":2952757594}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24496,"qdb_cluster":[24472,24488,24496]},"id":1}]} 2024-04-24T23:31:57.390 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 27 2024-04-24T23:31:57.391 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.389+0000 7fc3db091640 1 -- 172.21.15.80:0/1349285704 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fc3c40a2c10 msgr2=0x7fc3c40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:57.391 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.389+0000 7fc3db091640 1 --2- 172.21.15.80:0/1349285704 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fc3c40a2c10 0x7fc3c40a50d0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fc3d4198fe0 tx=0x7fc3c0005e50 comp rx=0 tx=0).stop 2024-04-24T23:31:57.391 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.389+0000 7fc3db091640 1 -- 172.21.15.80:0/1349285704 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc3d41017b0 msgr2=0x7fc3d4197ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:57.392 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.389+0000 7fc3db091640 1 --2- 172.21.15.80:0/1349285704 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc3d41017b0 0x7fc3d4197ad0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fc3c8031610 tx=0x7fc3c80043b0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.392 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.389+0000 7fc3db091640 1 -- 172.21.15.80:0/1349285704 shutdown_connections 2024-04-24T23:31:57.392 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.389+0000 7fc3db091640 1 --2- 172.21.15.80:0/1349285704 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fc3c40a2c10 0x7fc3c40a50d0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.392 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.389+0000 7fc3db091640 1 --2- 172.21.15.80:0/1349285704 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc3d4103360 0x7fc3d419f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.393 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.389+0000 7fc3db091640 1 --2- 172.21.15.80:0/1349285704 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc3d41029a0 0x7fc3d4198010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.393 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.389+0000 7fc3db091640 1 --2- 172.21.15.80:0/1349285704 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc3d41017b0 0x7fc3d4197ad0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.393 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.389+0000 7fc3db091640 1 -- 172.21.15.80:0/1349285704 >> 172.21.15.80:0/1349285704 conn(0x7fc3d40fcd40 msgr2=0x7fc3d4105cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:57.393 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.389+0000 7fc3db091640 1 -- 172.21.15.80:0/1349285704 shutdown_connections 2024-04-24T23:31:57.394 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.389+0000 7fc3db091640 1 -- 172.21.15.80:0/1349285704 wait complete. 2024-04-24T23:31:57.402 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T23:31:57.504 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.501+0000 7f8ff7bc8640 1 -- 172.21.15.80:0/3751827244 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8ff0075780 msgr2=0x7f8ff0073bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:57.504 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.501+0000 7f8ff7bc8640 1 --2- 172.21.15.80:0/3751827244 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8ff0075780 0x7f8ff0073bd0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f8fec009930 tx=0x7f8fec02a420 comp rx=0 tx=0).stop 2024-04-24T23:31:57.504 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.501+0000 7f8ff7bc8640 1 -- 172.21.15.80:0/3751827244 shutdown_connections 2024-04-24T23:31:57.505 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.501+0000 7f8ff7bc8640 1 --2- 172.21.15.80:0/3751827244 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ff010dd60 0x7f8ff0110150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.505 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.501+0000 7f8ff7bc8640 1 --2- 172.21.15.80:0/3751827244 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8ff00741a0 0x7f8ff0074620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.505 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.501+0000 7f8ff7bc8640 1 --2- 172.21.15.80:0/3751827244 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8ff0075780 0x7f8ff0073bd0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.505 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.501+0000 7f8ff7bc8640 1 -- 172.21.15.80:0/3751827244 >> 172.21.15.80:0/3751827244 conn(0x7f8ff00fcd20 msgr2=0x7f8ff00ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:57.505 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.501+0000 7f8ff7bc8640 1 -- 172.21.15.80:0/3751827244 shutdown_connections 2024-04-24T23:31:57.505 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.501+0000 7f8ff7bc8640 1 -- 172.21.15.80:0/3751827244 wait complete. 2024-04-24T23:31:57.505 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.502+0000 7f8ff7bc8640 1 Processor -- start 2024-04-24T23:31:57.506 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.502+0000 7f8ff7bc8640 1 -- start start 2024-04-24T23:31:57.506 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.502+0000 7f8ff7bc8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ff00741a0 0x7f8ff019e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:57.506 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.502+0000 7f8ff7bc8640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8ff0075780 0x7f8ff019e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:57.506 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.502+0000 7f8ff7bc8640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8ff010dd60 0x7f8ff01a56b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:57.506 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.502+0000 7f8ff7bc8640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff0113a00 con 0x7f8ff0075780 2024-04-24T23:31:57.506 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.502+0000 7f8ff7bc8640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff0113880 con 0x7f8ff010dd60 2024-04-24T23:31:57.507 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.502+0000 7f8ff7bc8640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff0113b80 con 0x7f8ff00741a0 2024-04-24T23:31:57.507 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.502+0000 7f8ff5163640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8ff0075780 0x7f8ff019e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:57.507 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.502+0000 7f8ff5163640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8ff0075780 0x7f8ff019e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53510/0 (socket says 172.21.15.80:53510) 2024-04-24T23:31:57.507 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.502+0000 7f8ff5163640 1 -- 172.21.15.80:0/2600742292 learned_addr learned my addr 172.21.15.80:0/2600742292 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:31:57.507 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.502+0000 7f8ff5964640 1 --2- 172.21.15.80:0/2600742292 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ff00741a0 0x7f8ff019e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:57.507 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.502+0000 7f8ff5163640 1 -- 172.21.15.80:0/2600742292 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ff00741a0 msgr2=0x7f8ff019e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:57.507 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.502+0000 7f8ff5163640 1 --2- 172.21.15.80:0/2600742292 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ff00741a0 0x7f8ff019e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.507 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.502+0000 7f8ff5163640 1 -- 172.21.15.80:0/2600742292 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8ff010dd60 msgr2=0x7f8ff01a56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:57.508 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.502+0000 7f8ff5163640 1 --2- 172.21.15.80:0/2600742292 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8ff010dd60 0x7f8ff01a56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.508 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.502+0000 7f8ff5163640 1 -- 172.21.15.80:0/2600742292 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ff01a5d20 con 0x7f8ff0075780 2024-04-24T23:31:57.508 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.503+0000 7f8ff5163640 1 --2- 172.21.15.80:0/2600742292 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8ff0075780 0x7f8ff019e640 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f8fe000c890 tx=0x7f8fe000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:57.508 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.503+0000 7f8fe6ffd640 1 -- 172.21.15.80:0/2600742292 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe0004220 con 0x7f8ff0075780 2024-04-24T23:31:57.508 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.503+0000 7f8ff7bc8640 1 -- 172.21.15.80:0/2600742292 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ff01a5fb0 con 0x7f8ff0075780 2024-04-24T23:31:57.508 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.503+0000 7f8fe6ffd640 1 -- 172.21.15.80:0/2600742292 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fe00043c0 con 0x7f8ff0075780 2024-04-24T23:31:57.508 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.503+0000 7f8fe6ffd640 1 -- 172.21.15.80:0/2600742292 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe0004a00 con 0x7f8ff0075780 2024-04-24T23:31:57.508 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.503+0000 7f8ff7bc8640 1 -- 172.21.15.80:0/2600742292 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ff01a64d0 con 0x7f8ff0075780 2024-04-24T23:31:57.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.505+0000 7f8fe6ffd640 1 -- 172.21.15.80:0/2600742292 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8fe00079b0 con 0x7f8ff0075780 2024-04-24T23:31:57.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.505+0000 7f8ffa9b4740 1 Processor -- start 2024-04-24T23:31:57.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.505+0000 7f8ffa9b4740 1 -- start start 2024-04-24T23:31:57.516 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.506+0000 7f8ffa9b4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564ce2d35230 0x564ce2d35650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:57.516 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.506+0000 7f8ffa9b4740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564ce2d35c20 0x564ce2d3a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:57.516 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.506+0000 7f8ffa9b4740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564ce2d3ad90 0x564ce2d3d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:57.516 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.506+0000 7f8ffa9b4740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x564ce2c99670 con 0x564ce2d3ad90 2024-04-24T23:31:57.516 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.506+0000 7f8ffa9b4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x564ce2c41970 con 0x564ce2d35230 2024-04-24T23:31:57.516 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.506+0000 7f8ffa9b4740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x564ce2c95630 con 0x564ce2d35c20 2024-04-24T23:31:57.516 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.506+0000 7f8ff5163640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564ce2d3ad90 0x564ce2d3d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:57.516 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.506+0000 7f8ff5163640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564ce2d3ad90 0x564ce2d3d220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53516/0 (socket says 172.21.15.80:53516) 2024-04-24T23:31:57.516 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.506+0000 7f8ff5163640 1 -- 172.21.15.80:0/3216452605 learned_addr learned my addr 172.21.15.80:0/3216452605 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:31:57.517 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.506+0000 7f8fe6ffd640 1 --2- 172.21.15.80:0/2600742292 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f8fdc0a2b90 0x7f8fdc0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:57.517 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.506+0000 7f8ff5964640 1 --2- 172.21.15.80:0/3216452605 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564ce2d35c20 0x564ce2d3a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:57.517 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.506+0000 7f8ff6165640 1 --2- 172.21.15.80:0/3216452605 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564ce2d35230 0x564ce2d35650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:57.517 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.506+0000 7f8ff6165640 1 --2- 172.21.15.80:0/2600742292 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f8fdc0a2b90 0x7f8fdc0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:57.517 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.506+0000 7f8fe6ffd640 1 -- 172.21.15.80:0/2600742292 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f8fe00cf1c0 con 0x7f8ff0075780 2024-04-24T23:31:57.517 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.506+0000 7f8ff5163640 1 -- 172.21.15.80:0/3216452605 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564ce2d35c20 msgr2=0x564ce2d3a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:57.517 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.506+0000 7f8ff5163640 1 --2- 172.21.15.80:0/3216452605 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564ce2d35c20 0x564ce2d3a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.518 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.506+0000 7f8ff5163640 1 -- 172.21.15.80:0/3216452605 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564ce2d35230 msgr2=0x564ce2d35650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:57.518 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.506+0000 7f8ff5163640 1 --2- 172.21.15.80:0/3216452605 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564ce2d35230 0x564ce2d35650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.518 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.506+0000 7f8ff5163640 1 -- 172.21.15.80:0/3216452605 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ce2c65400 con 0x564ce2d3ad90 2024-04-24T23:31:57.518 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.506+0000 7f8ff5163640 1 --2- 172.21.15.80:0/3216452605 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564ce2d3ad90 0x564ce2d3d220 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f8fe00cc830 tx=0x7f8fe0005300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0533f1c915e95cb server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:57.518 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.506+0000 7f8ff6165640 1 --2- 172.21.15.80:0/2600742292 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f8fdc0a2b90 0x7f8fdc0a5050 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x564ce2d36480 tx=0x7f8fe8005e00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:57.518 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.506+0000 7f8fdbfff640 1 -- 172.21.15.80:0/3216452605 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe001d070 con 0x564ce2d3ad90 2024-04-24T23:31:57.518 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.507+0000 7f8fdbfff640 1 -- 172.21.15.80:0/3216452605 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fe009bd50 con 0x564ce2d3ad90 2024-04-24T23:31:57.518 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.507+0000 7f8fdbfff640 1 -- 172.21.15.80:0/3216452605 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe0097cc0 con 0x564ce2d3ad90 2024-04-24T23:31:57.519 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.507+0000 7f8ffa9b4740 1 -- 172.21.15.80:0/3216452605 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564ce2d3ad90 msgr2=0x564ce2d3d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:57.519 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.507+0000 7f8ffa9b4740 1 --2- 172.21.15.80:0/3216452605 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564ce2d3ad90 0x564ce2d3d220 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f8fe00cc830 tx=0x7f8fe0005300 comp rx=0 tx=0).stop 2024-04-24T23:31:57.519 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.507+0000 7f8ffa9b4740 1 -- 172.21.15.80:0/3216452605 shutdown_connections 2024-04-24T23:31:57.519 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.507+0000 7f8ffa9b4740 1 --2- 172.21.15.80:0/3216452605 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564ce2d3ad90 0x564ce2d3d220 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.519 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.507+0000 7f8ffa9b4740 1 --2- 172.21.15.80:0/3216452605 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564ce2d35c20 0x564ce2d3a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.519 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.507+0000 7f8ffa9b4740 1 --2- 172.21.15.80:0/3216452605 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564ce2d35230 0x564ce2d35650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.519 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.507+0000 7f8ffa9b4740 1 -- 172.21.15.80:0/3216452605 >> 172.21.15.80:0/3216452605 conn(0x564ce2d308a0 msgr2=0x564ce2d30ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:57.519 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.507+0000 7f8ffa9b4740 1 -- 172.21.15.80:0/3216452605 shutdown_connections 2024-04-24T23:31:57.519 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.507+0000 7f8ffa9b4740 1 -- 172.21.15.80:0/3216452605 wait complete. 2024-04-24T23:31:57.520 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.508+0000 7f8ffa9b4740 1 -- start start 2024-04-24T23:31:57.520 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.508+0000 7f8ffa9b4740 1 Processor -- start 2024-04-24T23:31:57.520 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.508+0000 7f8ffa9b4740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564ce2d35230 0x564ce2d3f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:57.520 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.508+0000 7f8ffa9b4740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564ce2d35c20 0x564ce2d3f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:57.520 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.508+0000 7f8ffa9b4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564ce2d3ad90 0x564ce2d46810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:57.520 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.508+0000 7f8ffa9b4740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x564ce2d3dc80 con 0x564ce2d35c20 2024-04-24T23:31:57.520 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.508+0000 7f8ffa9b4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x564ce2c55050 con 0x564ce2d3ad90 2024-04-24T23:31:57.521 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.508+0000 7f8ffa9b4740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x564ce2d3de00 con 0x564ce2d35230 2024-04-24T23:31:57.521 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.508+0000 7f8ffa9b4740 10 client.0 _ll_register_callbacks cb 0x564ce29acbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:31:57.521 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.508+0000 7f8ff5163640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564ce2d35c20 0x564ce2d3f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:57.521 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.508+0000 7f8ff5163640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564ce2d35c20 0x564ce2d3f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53522/0 (socket says 172.21.15.80:53522) 2024-04-24T23:31:57.521 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.508+0000 7f8ff5163640 1 -- 172.21.15.80:0/3563240115 learned_addr learned my addr 172.21.15.80:0/3563240115 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:31:57.521 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.508+0000 7f8ff5964640 1 --2- 172.21.15.80:0/3563240115 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564ce2d35230 0x564ce2d3f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:57.521 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.508+0000 7f8ff6165640 1 --2- 172.21.15.80:0/3563240115 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564ce2d3ad90 0x564ce2d46810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:57.521 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.508+0000 7f8ff5163640 1 -- 172.21.15.80:0/3563240115 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564ce2d35230 msgr2=0x564ce2d3f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:57.522 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.508+0000 7f8ff5163640 1 --2- 172.21.15.80:0/3563240115 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564ce2d35230 0x564ce2d3f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.522 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.508+0000 7f8ff5163640 1 -- 172.21.15.80:0/3563240115 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564ce2d3ad90 msgr2=0x564ce2d46810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:57.522 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.508+0000 7f8ff5163640 1 --2- 172.21.15.80:0/3563240115 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564ce2d3ad90 0x564ce2d46810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.522 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.508+0000 7f8ff5163640 1 -- 172.21.15.80:0/3563240115 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ce2c26dc0 con 0x564ce2d35c20 2024-04-24T23:31:57.522 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.509+0000 7f8ff5163640 1 --2- 172.21.15.80:0/3563240115 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564ce2d35c20 0x564ce2d3f7d0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f8fe00ccc60 tx=0x7f8fe0086150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ec3a418818659ca server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:57.522 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.509+0000 7f8fd97fa640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-04-24T23:31:57.522 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.509+0000 7f8fd97fa640 1 -- 172.21.15.80:0/3563240115 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe001d040 con 0x564ce2d35c20 2024-04-24T23:31:57.522 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.509+0000 7f8fd97fa640 1 -- 172.21.15.80:0/3563240115 <== mon.0 v2:172.21.15.80:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f8fe0007ca0 con 0x564ce2d35c20 2024-04-24T23:31:57.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.509+0000 7f8ffa9b4740 1 -- 172.21.15.80:0/3563240115 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8fe009bd50 con 0x564ce2d35c20 2024-04-24T23:31:57.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.509+0000 7f8fd97fa640 1 -- 172.21.15.80:0/3563240115 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8fe009b650 con 0x564ce2d35c20 2024-04-24T23:31:57.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.509+0000 7f8ffa9b4740 10 client.16017 fetch_fsmap learned FSMap version 27 2024-04-24T23:31:57.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.509+0000 7f8ffa9b4740 1 -- 172.21.15.80:0/3563240115 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564ce29ad3f0 con 0x564ce2d35c20 2024-04-24T23:31:57.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.509+0000 7f8fd97fa640 1 -- 172.21.15.80:0/3563240115 <== mon.0 v2:172.21.15.80:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fe009bd50 con 0x564ce2d35c20 2024-04-24T23:31:57.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.509+0000 7f8fd97fa640 1 -- 172.21.15.80:0/3563240115 <== mon.0 v2:172.21.15.80:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f8fe009b9b0 con 0x564ce2d35c20 2024-04-24T23:31:57.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.509+0000 7f8ffa9b4740 10 client.16017 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:31:57.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.509+0000 7f8ffa9b4740 10 client.16017 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-24T23:31:57.524 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.509+0000 7f8ffa9b4740 20 client.16017 populate_metadata read hostname 'smithi080' 2024-04-24T23:31:57.524 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.509+0000 7f8ffa9b4740 1 --2- 172.21.15.80:0/3563240115 >> [v2:172.21.15.148:6836/856528403,v1:172.21.15.148:6837/856528403] conn(0x564ce2d4ad00 0x564ce2d4d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:57.524 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.509+0000 7f8ffa9b4740 4 client.16017 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:31:57.524 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.509+0000 7f8ffa9b4740 1 -- 172.21.15.80:0/3563240115 --> [v2:172.21.15.148:6836/856528403,v1:172.21.15.148:6837/856528403] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564ce2ca6bb0 con 0x564ce2d4ad00 2024-04-24T23:31:57.524 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.510+0000 7f8ff5964640 1 --2- 172.21.15.80:0/3563240115 >> [v2:172.21.15.148:6836/856528403,v1:172.21.15.148:6837/856528403] conn(0x564ce2d4ad00 0x564ce2d4d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:57.524 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.510+0000 7f8fd97fa640 10 client.16017 ms_handle_connect on v2:172.21.15.148:6836/856528403 2024-04-24T23:31:57.524 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.510+0000 7f8ff5964640 1 --2- 172.21.15.80:0/3563240115 >> [v2:172.21.15.148:6836/856528403,v1:172.21.15.148:6837/856528403] conn(0x564ce2d4ad00 0x564ce2d4d0f0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=25683ef7285b4211 server_cookie=39230d2673dd3e89 in_seq=0 out_seq=0 2024-04-24T23:31:57.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.511+0000 7f8fd97fa640 1 -- 172.21.15.80:0/3563240115 <== mds.2 v2:172.21.15.148:6836/856528403 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564ce2ca6bb0 con 0x564ce2d4ad00 2024-04-24T23:31:57.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.511+0000 7f8fd97fa640 10 client.16017 handle_command_reply: tid=0 2024-04-24T23:31:57.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.511+0000 7f8ffa9b4740 1 client.16017 shutdown 2024-04-24T23:31:57.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.511+0000 7f8ffa9b4740 1 -- 172.21.15.80:0/3563240115 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564ce2d35c20 msgr2=0x564ce2d3f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:57.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.511+0000 7f8ffa9b4740 1 --2- 172.21.15.80:0/3563240115 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564ce2d35c20 0x564ce2d3f7d0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f8fe00ccc60 tx=0x7f8fe0086150 comp rx=0 tx=0).stop 2024-04-24T23:31:57.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.511+0000 7f8ffa9b4740 1 -- 172.21.15.80:0/3563240115 shutdown_connections 2024-04-24T23:31:57.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.511+0000 7f8ffa9b4740 1 --2- 172.21.15.80:0/3563240115 >> [v2:172.21.15.148:6836/856528403,v1:172.21.15.148:6837/856528403] conn(0x564ce2d4ad00 0x564ce2d4d0f0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.511+0000 7f8ffa9b4740 1 --2- 172.21.15.80:0/3563240115 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564ce2d3ad90 0x564ce2d46810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.511+0000 7f8fd97fa640 0 client.16017 ms_handle_reset on v2:172.21.15.148:6836/856528403 2024-04-24T23:31:57.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.511+0000 7f8ffa9b4740 1 --2- 172.21.15.80:0/3563240115 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564ce2d35c20 0x564ce2d3f7d0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.511+0000 7f8ffa9b4740 1 --2- 172.21.15.80:0/3563240115 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564ce2d35230 0x564ce2d3f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.511+0000 7f8ffa9b4740 1 -- 172.21.15.80:0/3563240115 >> 172.21.15.80:0/3563240115 conn(0x564ce2d308a0 msgr2=0x564ce2d2fb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:57.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.511+0000 7f8ffa9b4740 1 -- 172.21.15.80:0/3563240115 shutdown_connections 2024-04-24T23:31:57.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.511+0000 7f8ffa9b4740 1 -- 172.21.15.80:0/3563240115 wait complete. 2024-04-24T23:31:57.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.511+0000 7f8ffa9b4740 20 client.16017 trim_cache size 0 max 16384 2024-04-24T23:31:57.533 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.524+0000 7f8ffa9b4740 1 Processor -- start 2024-04-24T23:31:57.533 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.524+0000 7f8ffa9b4740 1 -- start start 2024-04-24T23:31:57.533 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.524+0000 7f8ffa9b4740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564ce2d35230 0x564ce2d3e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:57.533 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.524+0000 7f8ffa9b4740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564ce2d35c20 0x564ce2d467d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:57.533 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.524+0000 7f8ffa9b4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564ce2d3ad90 0x564ce2d36640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:57.533 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.524+0000 7f8ffa9b4740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff01b34b0 con 0x564ce2d35c20 2024-04-24T23:31:57.533 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.524+0000 7f8ffa9b4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff01b0d80 con 0x564ce2d3ad90 2024-04-24T23:31:57.534 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.524+0000 7f8ffa9b4740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff01aebd0 con 0x564ce2d35230 2024-04-24T23:31:57.534 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.524+0000 7f8ff5163640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564ce2d35c20 0x564ce2d467d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:57.534 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.524+0000 7f8ff5163640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564ce2d35c20 0x564ce2d467d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53526/0 (socket says 172.21.15.80:53526) 2024-04-24T23:31:57.534 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.524+0000 7f8ff5163640 1 -- 172.21.15.80:0/1281594690 learned_addr learned my addr 172.21.15.80:0/1281594690 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:31:57.534 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.524+0000 7f8ff6165640 1 --2- 172.21.15.80:0/1281594690 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564ce2d3ad90 0x564ce2d36640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:57.534 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.524+0000 7f8ff5964640 1 --2- 172.21.15.80:0/1281594690 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564ce2d35230 0x564ce2d3e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:57.534 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.525+0000 7f8ff5163640 1 -- 172.21.15.80:0/1281594690 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564ce2d35230 msgr2=0x564ce2d3e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:57.534 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.525+0000 7f8ff5163640 1 --2- 172.21.15.80:0/1281594690 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564ce2d35230 0x564ce2d3e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.535 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.525+0000 7f8ff5163640 1 -- 172.21.15.80:0/1281594690 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564ce2d3ad90 msgr2=0x564ce2d36640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:57.535 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.525+0000 7f8ff5163640 1 --2- 172.21.15.80:0/1281594690 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564ce2d3ad90 0x564ce2d36640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.535 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.525+0000 7f8ff5163640 1 -- 172.21.15.80:0/1281594690 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ce2a777d0 con 0x564ce2d35c20 2024-04-24T23:31:57.535 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.525+0000 7f8ff6165640 1 --2- 172.21.15.80:0/1281594690 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564ce2d3ad90 0x564ce2d36640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:31:57.535 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.525+0000 7f8ff5163640 1 --2- 172.21.15.80:0/1281594690 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564ce2d35c20 0x564ce2d467d0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f8fe0098020 tx=0x7f8fe0099b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7de7ef2a946fecd3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:57.535 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.525+0000 7f8fdb7fe640 1 -- 172.21.15.80:0/1281594690 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe001d070 con 0x564ce2d35c20 2024-04-24T23:31:57.535 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.525+0000 7f8fdb7fe640 1 -- 172.21.15.80:0/1281594690 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fe009bd50 con 0x564ce2d35c20 2024-04-24T23:31:57.535 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.525+0000 7f8fdb7fe640 1 -- 172.21.15.80:0/1281594690 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe0084e40 con 0x564ce2d35c20 2024-04-24T23:31:57.535 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.525+0000 7f8ffa9b4740 1 -- 172.21.15.80:0/1281594690 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564ce2d35c20 msgr2=0x564ce2d467d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:57.536 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.525+0000 7f8ffa9b4740 1 --2- 172.21.15.80:0/1281594690 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564ce2d35c20 0x564ce2d467d0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f8fe0098020 tx=0x7f8fe0099b60 comp rx=0 tx=0).stop 2024-04-24T23:31:57.536 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.525+0000 7f8ffa9b4740 1 -- 172.21.15.80:0/1281594690 shutdown_connections 2024-04-24T23:31:57.536 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.525+0000 7f8ffa9b4740 1 --2- 172.21.15.80:0/1281594690 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564ce2d3ad90 0x564ce2d36640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.536 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.525+0000 7f8ffa9b4740 1 --2- 172.21.15.80:0/1281594690 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564ce2d35c20 0x564ce2d467d0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.536 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.525+0000 7f8ffa9b4740 1 --2- 172.21.15.80:0/1281594690 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564ce2d35230 0x564ce2d3e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.536 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.525+0000 7f8ffa9b4740 1 -- 172.21.15.80:0/1281594690 >> 172.21.15.80:0/1281594690 conn(0x564ce2d308a0 msgr2=0x564ce2d2fb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:57.536 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.525+0000 7f8ffa9b4740 1 -- 172.21.15.80:0/1281594690 shutdown_connections 2024-04-24T23:31:57.536 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.525+0000 7f8ffa9b4740 1 -- 172.21.15.80:0/1281594690 wait complete. 2024-04-24T23:31:57.536 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.526+0000 7f8ffa9b4740 1 -- start start 2024-04-24T23:31:57.537 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.526+0000 7f8ffa9b4740 1 Processor -- start 2024-04-24T23:31:57.537 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.526+0000 7f8ffa9b4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564ce2d35230 0x564ce2d441e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:57.537 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.526+0000 7f8ffa9b4740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564ce2d35c20 0x564ce2d44720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:57.537 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.526+0000 7f8ffa9b4740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564ce2d3ad90 0x564ce2d45670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:57.537 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.526+0000 7f8ffa9b4740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff01af240 con 0x564ce2d3ad90 2024-04-24T23:31:57.537 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.526+0000 7f8ffa9b4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff01b06b0 con 0x564ce2d35230 2024-04-24T23:31:57.537 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.526+0000 7f8ffa9b4740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff01af910 con 0x564ce2d35c20 2024-04-24T23:31:57.537 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.526+0000 7f8ffa9b4740 10 client.0 _ll_register_callbacks cb 0x7f8fe8009bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:31:57.538 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.526+0000 7f8ff6165640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564ce2d3ad90 0x564ce2d45670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:57.538 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.526+0000 7f8ff6165640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564ce2d3ad90 0x564ce2d45670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53532/0 (socket says 172.21.15.80:53532) 2024-04-24T23:31:57.538 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.526+0000 7f8ff6165640 1 -- 172.21.15.80:0/2324477583 learned_addr learned my addr 172.21.15.80:0/2324477583 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:31:57.538 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.526+0000 7f8ff6165640 1 -- 172.21.15.80:0/2324477583 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564ce2d35c20 msgr2=0x564ce2d44720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:57.538 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.526+0000 7f8ff5163640 1 --2- 172.21.15.80:0/2324477583 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564ce2d35c20 0x564ce2d44720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:57.538 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.526+0000 7f8ff5964640 1 --2- 172.21.15.80:0/2324477583 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564ce2d35230 0x564ce2d441e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:57.538 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.526+0000 7f8ff6165640 1 --2- 172.21.15.80:0/2324477583 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564ce2d35c20 0x564ce2d44720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.538 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.526+0000 7f8ff6165640 1 -- 172.21.15.80:0/2324477583 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564ce2d35230 msgr2=0x564ce2d441e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:57.538 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.527+0000 7f8ff6165640 1 --2- 172.21.15.80:0/2324477583 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564ce2d35230 0x564ce2d441e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.539 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.527+0000 7f8ff6165640 1 -- 172.21.15.80:0/2324477583 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ce2d45c60 con 0x564ce2d3ad90 2024-04-24T23:31:57.539 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.527+0000 7f8ff6165640 1 --2- 172.21.15.80:0/2324477583 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564ce2d3ad90 0x564ce2d45670 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f8fe8011ad0 tx=0x7f8fe8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a7a9510a1b78933 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:57.539 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.527+0000 7f8fd97fa640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-04-24T23:31:57.539 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.527+0000 7f8fd97fa640 1 -- 172.21.15.80:0/2324477583 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe8025070 con 0x564ce2d3ad90 2024-04-24T23:31:57.539 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.527+0000 7f8fd97fa640 1 -- 172.21.15.80:0/2324477583 <== mon.0 v2:172.21.15.80:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f8fe801eb60 con 0x564ce2d3ad90 2024-04-24T23:31:57.539 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.527+0000 7f8ffa9b4740 1 -- 172.21.15.80:0/2324477583 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8fe009bd50 con 0x564ce2d3ad90 2024-04-24T23:31:57.539 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.527+0000 7f8fd97fa640 1 -- 172.21.15.80:0/2324477583 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8fe801ed60 con 0x564ce2d3ad90 2024-04-24T23:31:57.539 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.527+0000 7f8ffa9b4740 10 client.16023 fetch_fsmap learned FSMap version 27 2024-04-24T23:31:57.539 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.527+0000 7f8ffa9b4740 1 -- 172.21.15.80:0/2324477583 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564ce2db13c0 con 0x564ce2d3ad90 2024-04-24T23:31:57.540 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.527+0000 7f8fd97fa640 1 -- 172.21.15.80:0/2324477583 <== mon.0 v2:172.21.15.80:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fe009bd50 con 0x564ce2d3ad90 2024-04-24T23:31:57.540 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.527+0000 7f8fd97fa640 1 -- 172.21.15.80:0/2324477583 <== mon.0 v2:172.21.15.80:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f8fe801d040 con 0x564ce2d3ad90 2024-04-24T23:31:57.540 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.527+0000 7f8ffa9b4740 10 client.16023 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:31:57.540 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.527+0000 7f8ffa9b4740 10 client.16023 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-24T23:31:57.540 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.527+0000 7f8ffa9b4740 20 client.16023 populate_metadata read hostname 'smithi080' 2024-04-24T23:31:57.540 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.527+0000 7f8ffa9b4740 1 --2- 172.21.15.80:0/2324477583 >> [v2:172.21.15.148:6836/856528403,v1:172.21.15.148:6837/856528403] conn(0x564ce2db1e70 0x564ce2db4290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:57.540 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.527+0000 7f8ffa9b4740 4 client.16023 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:31:57.540 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.527+0000 7f8ffa9b4740 1 -- 172.21.15.80:0/2324477583 --> [v2:172.21.15.148:6836/856528403,v1:172.21.15.148:6837/856528403] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564ce2d3f080 con 0x564ce2db1e70 2024-04-24T23:31:57.541 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.528+0000 7f8ff5964640 1 --2- 172.21.15.80:0/2324477583 >> [v2:172.21.15.148:6836/856528403,v1:172.21.15.148:6837/856528403] conn(0x564ce2db1e70 0x564ce2db4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:57.541 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.528+0000 7f8fd97fa640 10 client.16023 ms_handle_connect on v2:172.21.15.148:6836/856528403 2024-04-24T23:31:57.541 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.528+0000 7f8ff5964640 1 --2- 172.21.15.80:0/2324477583 >> [v2:172.21.15.148:6836/856528403,v1:172.21.15.148:6837/856528403] conn(0x564ce2db1e70 0x564ce2db4290 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5592aecacbdb500 server_cookie=b3343adc073a5c9a in_seq=0 out_seq=0 2024-04-24T23:31:57.541 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.528+0000 7f8fd97fa640 1 -- 172.21.15.80:0/2324477583 <== mds.2 v2:172.21.15.148:6836/856528403 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564ce2d3f080 con 0x564ce2db1e70 2024-04-24T23:31:57.541 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.528+0000 7f8fd97fa640 10 client.16023 handle_command_reply: tid=0 2024-04-24T23:31:57.541 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.528+0000 7f8ffa9b4740 1 client.16023 shutdown 2024-04-24T23:31:57.541 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.528+0000 7f8ffa9b4740 1 -- 172.21.15.80:0/2324477583 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564ce2d3ad90 msgr2=0x564ce2d45670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:57.541 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.528+0000 7f8ffa9b4740 1 --2- 172.21.15.80:0/2324477583 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564ce2d3ad90 0x564ce2d45670 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f8fe8011ad0 tx=0x7f8fe8004a40 comp rx=0 tx=0).stop 2024-04-24T23:31:57.542 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.528+0000 7f8ffa9b4740 1 -- 172.21.15.80:0/2324477583 shutdown_connections 2024-04-24T23:31:57.542 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.528+0000 7f8ffa9b4740 1 --2- 172.21.15.80:0/2324477583 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564ce2d3ad90 0x564ce2d45670 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.542 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.528+0000 7f8ffa9b4740 1 --2- 172.21.15.80:0/2324477583 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564ce2d35c20 0x564ce2d44720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.542 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.528+0000 7f8ffa9b4740 1 --2- 172.21.15.80:0/2324477583 >> [v2:172.21.15.148:6836/856528403,v1:172.21.15.148:6837/856528403] conn(0x564ce2db1e70 0x564ce2db4290 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.542 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.528+0000 7f8ffa9b4740 1 --2- 172.21.15.80:0/2324477583 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564ce2d35230 0x564ce2d441e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.542 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.528+0000 7f8fd97fa640 0 client.16023 ms_handle_reset on v2:172.21.15.148:6836/856528403 2024-04-24T23:31:57.542 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.528+0000 7f8ffa9b4740 1 -- 172.21.15.80:0/2324477583 >> 172.21.15.80:0/2324477583 conn(0x564ce2d308a0 msgr2=0x564ce2d2fb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:57.543 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.528+0000 7f8ffa9b4740 1 -- 172.21.15.80:0/2324477583 shutdown_connections 2024-04-24T23:31:57.543 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.528+0000 7f8ffa9b4740 1 -- 172.21.15.80:0/2324477583 wait complete. 2024-04-24T23:31:57.543 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.528+0000 7f8ffa9b4740 20 client.16023 trim_cache size 0 max 16384 2024-04-24T23:31:57.543 INFO:teuthology.orchestra.run.smithi080.stdout:[] 2024-04-24T23:31:57.545 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.529+0000 7f8ff7bc8640 1 -- 172.21.15.80:0/2600742292 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f8fdc0a2b90 msgr2=0x7f8fdc0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:57.545 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.529+0000 7f8ff7bc8640 1 --2- 172.21.15.80:0/2600742292 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f8fdc0a2b90 0x7f8fdc0a5050 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x564ce2d36480 tx=0x7f8fe8005e00 comp rx=0 tx=0).stop 2024-04-24T23:31:57.545 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.529+0000 7f8ff7bc8640 1 -- 172.21.15.80:0/2600742292 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8ff0075780 msgr2=0x7f8ff019e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:57.545 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.529+0000 7f8ff7bc8640 1 --2- 172.21.15.80:0/2600742292 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8ff0075780 0x7f8ff019e640 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f8fe000c890 tx=0x7f8fe000cd50 comp rx=0 tx=0).stop 2024-04-24T23:31:57.545 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.529+0000 7f8ff7bc8640 1 -- 172.21.15.80:0/2600742292 shutdown_connections 2024-04-24T23:31:57.545 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.529+0000 7f8ff7bc8640 1 --2- 172.21.15.80:0/2600742292 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f8fdc0a2b90 0x7f8fdc0a5050 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.546 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.529+0000 7f8ff7bc8640 1 --2- 172.21.15.80:0/2600742292 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8ff010dd60 0x7f8ff01a56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.546 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.529+0000 7f8ff7bc8640 1 --2- 172.21.15.80:0/2600742292 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8ff0075780 0x7f8ff019e640 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.546 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.529+0000 7f8ff7bc8640 1 --2- 172.21.15.80:0/2600742292 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ff00741a0 0x7f8ff019e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.546 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.529+0000 7f8ff7bc8640 1 -- 172.21.15.80:0/2600742292 >> 172.21.15.80:0/2600742292 conn(0x7f8ff00fcd20 msgr2=0x7f8ff0101100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:57.546 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.529+0000 7f8ff7bc8640 1 -- 172.21.15.80:0/2600742292 shutdown_connections 2024-04-24T23:31:57.546 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.529+0000 7f8ff7bc8640 1 -- 172.21.15.80:0/2600742292 wait complete. 2024-04-24T23:31:57.547 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T23:31:57.648 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.645+0000 7f8c66b22640 1 -- 172.21.15.80:0/1934091565 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8c60102230 msgr2=0x7f8c60108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:57.648 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.645+0000 7f8c66b22640 1 --2- 172.21.15.80:0/1934091565 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8c60102230 0x7f8c60108ac0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f8c48009900 tx=0x7f8c4802a410 comp rx=0 tx=0).stop 2024-04-24T23:31:57.649 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.645+0000 7f8c66b22640 1 -- 172.21.15.80:0/1934091565 shutdown_connections 2024-04-24T23:31:57.649 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.645+0000 7f8c66b22640 1 --2- 172.21.15.80:0/1934091565 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8c60102230 0x7f8c60108ac0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.649 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.645+0000 7f8c66b22640 1 --2- 172.21.15.80:0/1934091565 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c60101870 0x7f8c60101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.649 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.645+0000 7f8c66b22640 1 --2- 172.21.15.80:0/1934091565 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8c60100680 0x7f8c60100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.649 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.645+0000 7f8c66b22640 1 -- 172.21.15.80:0/1934091565 >> 172.21.15.80:0/1934091565 conn(0x7f8c600fbbf0 msgr2=0x7f8c600fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:57.649 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.646+0000 7f8c66b22640 1 -- 172.21.15.80:0/1934091565 shutdown_connections 2024-04-24T23:31:57.649 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.646+0000 7f8c66b22640 1 -- 172.21.15.80:0/1934091565 wait complete. 2024-04-24T23:31:57.649 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.646+0000 7f8c66b22640 1 Processor -- start 2024-04-24T23:31:57.649 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.646+0000 7f8c66b22640 1 -- start start 2024-04-24T23:31:57.650 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.646+0000 7f8c66b22640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8c60100680 0x7f8c60196940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:57.650 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.646+0000 7f8c66b22640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c60101870 0x7f8c60196e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:57.650 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.646+0000 7f8c66b22640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8c60102230 0x7f8c6019def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:57.650 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.646+0000 7f8c66b22640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f8c6010c2e0 con 0x7f8c60100680 2024-04-24T23:31:57.650 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.646+0000 7f8c66b22640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8c6010c160 con 0x7f8c60102230 2024-04-24T23:31:57.650 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.646+0000 7f8c66b22640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f8c6010c460 con 0x7f8c60101870 2024-04-24T23:31:57.650 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.646+0000 7f8c648be640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8c60100680 0x7f8c60196940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:57.650 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.646+0000 7f8c648be640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8c60100680 0x7f8c60196940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53556/0 (socket says 172.21.15.80:53556) 2024-04-24T23:31:57.651 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.646+0000 7f8c648be640 1 -- 172.21.15.80:0/1176920993 learned_addr learned my addr 172.21.15.80:0/1176920993 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:31:57.651 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.646+0000 7f8c5ffff640 1 --2- 172.21.15.80:0/1176920993 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c60101870 0x7f8c60196e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:57.651 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.647+0000 7f8c650bf640 1 --2- 172.21.15.80:0/1176920993 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8c60102230 0x7f8c6019def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:57.651 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.647+0000 7f8c648be640 1 -- 172.21.15.80:0/1176920993 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c60101870 msgr2=0x7f8c60196e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:57.651 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.647+0000 7f8c648be640 1 --2- 172.21.15.80:0/1176920993 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c60101870 0x7f8c60196e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.651 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.647+0000 7f8c648be640 1 -- 172.21.15.80:0/1176920993 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8c60102230 msgr2=0x7f8c6019def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:57.651 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.647+0000 7f8c648be640 1 --2- 172.21.15.80:0/1176920993 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8c60102230 0x7f8c6019def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.651 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.647+0000 7f8c648be640 1 -- 172.21.15.80:0/1176920993 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c6019e560 con 0x7f8c60100680 2024-04-24T23:31:57.651 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.647+0000 7f8c648be640 1 --2- 172.21.15.80:0/1176920993 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8c60100680 0x7f8c60196940 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f8c5800ccb0 tx=0x7f8c5800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:57.652 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.647+0000 7f8c5dffb640 1 -- 172.21.15.80:0/1176920993 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c58003b70 con 0x7f8c60100680 2024-04-24T23:31:57.652 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.647+0000 7f8c5dffb640 1 -- 172.21.15.80:0/1176920993 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c5800ae50 con 0x7f8c60100680 2024-04-24T23:31:57.652 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.647+0000 7f8c66b22640 1 -- 172.21.15.80:0/1176920993 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c6019e850 con 0x7f8c60100680 2024-04-24T23:31:57.652 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.647+0000 7f8c5dffb640 1 -- 172.21.15.80:0/1176920993 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c5800f810 con 0x7f8c60100680 2024-04-24T23:31:57.652 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.647+0000 7f8c66b22640 1 -- 172.21.15.80:0/1176920993 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c6019eda0 con 0x7f8c60100680 2024-04-24T23:31:57.659 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.649+0000 7f8c5dffb640 1 -- 172.21.15.80:0/1176920993 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8c5800f9b0 con 0x7f8c60100680 2024-04-24T23:31:57.659 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.650+0000 7f8c69834740 1 Processor -- start 2024-04-24T23:31:57.659 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.650+0000 7f8c5dffb640 1 --2- 172.21.15.80:0/1176920993 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f8c4c0a2b90 0x7f8c4c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:57.659 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.650+0000 7f8c69834740 1 -- start start 2024-04-24T23:31:57.660 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.650+0000 7f8c650bf640 1 --2- 172.21.15.80:0/1176920993 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f8c4c0a2b90 0x7f8c4c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:57.660 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.650+0000 7f8c69834740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56239fba9440 0x56239fba9860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:57.660 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.650+0000 7f8c5dffb640 1 -- 172.21.15.80:0/1176920993 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f8c580cf5b0 con 0x7f8c60100680 2024-04-24T23:31:57.660 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.650+0000 7f8c69834740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56239fba9e30 0x56239fbae930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:57.660 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.650+0000 7f8c69834740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56239fbaefa0 0x56239fbb1430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:57.660 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.650+0000 7f8c69834740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x56239fb0b670 con 0x56239fbaefa0 2024-04-24T23:31:57.660 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.650+0000 7f8c69834740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56239fab3970 con 0x56239fba9e30 2024-04-24T23:31:57.660 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.650+0000 7f8c69834740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x56239fb07630 con 0x56239fba9440 2024-04-24T23:31:57.660 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.650+0000 7f8c650bf640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56239fbaefa0 0x56239fbb1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:57.661 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.650+0000 7f8c650bf640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56239fbaefa0 0x56239fbb1430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53572/0 (socket says 172.21.15.80:53572) 2024-04-24T23:31:57.661 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.650+0000 7f8c650bf640 1 -- 172.21.15.80:0/3335890131 learned_addr learned my addr 172.21.15.80:0/3335890131 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:31:57.661 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.650+0000 7f8c648be640 1 --2- 172.21.15.80:0/3335890131 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56239fba9440 0x56239fba9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:57.661 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.650+0000 7f8c5ffff640 1 --2- 172.21.15.80:0/3335890131 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56239fba9e30 0x56239fbae930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:57.661 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.650+0000 7f8c650bf640 1 --2- 172.21.15.80:0/1176920993 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f8c4c0a2b90 0x7f8c4c0a5050 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f8c48009900 tx=0x7f8c480316b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:57.661 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.651+0000 7f8c650bf640 1 -- 172.21.15.80:0/3335890131 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56239fba9440 msgr2=0x56239fba9860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:57.661 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.651+0000 7f8c650bf640 1 --2- 172.21.15.80:0/3335890131 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56239fba9440 0x56239fba9860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.661 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.651+0000 7f8c650bf640 1 -- 172.21.15.80:0/3335890131 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56239fba9e30 msgr2=0x56239fbae930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:57.661 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.651+0000 7f8c650bf640 1 --2- 172.21.15.80:0/3335890131 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56239fba9e30 0x56239fbae930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.662 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.651+0000 7f8c650bf640 1 -- 172.21.15.80:0/3335890131 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56239fad7400 con 0x56239fbaefa0 2024-04-24T23:31:57.662 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.651+0000 7f8c648be640 1 --2- 172.21.15.80:0/3335890131 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56239fba9440 0x56239fba9860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:31:57.662 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.651+0000 7f8c5ffff640 1 --2- 172.21.15.80:0/3335890131 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56239fba9e30 0x56239fbae930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:31:57.662 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.651+0000 7f8c650bf640 1 --2- 172.21.15.80:0/3335890131 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56239fbaefa0 0x56239fbb1430 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f8c48033000 tx=0x7f8c48046000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d0ebec3b1e3392e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:57.662 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.651+0000 7f8c56ffd640 1 -- 172.21.15.80:0/3335890131 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c48035050 con 0x56239fbaefa0 2024-04-24T23:31:57.662 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.651+0000 7f8c56ffd640 1 -- 172.21.15.80:0/3335890131 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c48005520 con 0x56239fbaefa0 2024-04-24T23:31:57.662 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.651+0000 7f8c56ffd640 1 -- 172.21.15.80:0/3335890131 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c48004ce0 con 0x56239fbaefa0 2024-04-24T23:31:57.662 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.652+0000 7f8c69834740 1 -- 172.21.15.80:0/3335890131 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56239fbaefa0 msgr2=0x56239fbb1430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:57.662 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.652+0000 7f8c69834740 1 --2- 172.21.15.80:0/3335890131 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56239fbaefa0 0x56239fbb1430 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f8c48033000 tx=0x7f8c48046000 comp rx=0 tx=0).stop 2024-04-24T23:31:57.663 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.652+0000 7f8c69834740 1 -- 172.21.15.80:0/3335890131 shutdown_connections 2024-04-24T23:31:57.663 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.652+0000 7f8c69834740 1 --2- 172.21.15.80:0/3335890131 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56239fbaefa0 0x56239fbb1430 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.663 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.652+0000 7f8c69834740 1 --2- 172.21.15.80:0/3335890131 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56239fba9e30 0x56239fbae930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.663 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.652+0000 7f8c69834740 1 --2- 172.21.15.80:0/3335890131 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56239fba9440 0x56239fba9860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.663 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.652+0000 7f8c69834740 1 -- 172.21.15.80:0/3335890131 >> 172.21.15.80:0/3335890131 conn(0x56239fba28a0 msgr2=0x56239fba2ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:57.663 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.652+0000 7f8c69834740 1 -- 172.21.15.80:0/3335890131 shutdown_connections 2024-04-24T23:31:57.663 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.652+0000 7f8c69834740 1 -- 172.21.15.80:0/3335890131 wait complete. 2024-04-24T23:31:57.663 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.652+0000 7f8c69834740 1 -- start start 2024-04-24T23:31:57.664 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.652+0000 7f8c69834740 1 Processor -- start 2024-04-24T23:31:57.664 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.652+0000 7f8c69834740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56239fba9440 0x56239fbb3470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:57.664 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.652+0000 7f8c69834740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56239fba9e30 0x56239fbb39b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:57.664 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.652+0000 7f8c69834740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56239fbaefa0 0x56239fbba9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:57.664 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.652+0000 7f8c69834740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x56239fbb1e90 con 0x56239fbaefa0 2024-04-24T23:31:57.664 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.652+0000 7f8c69834740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56239fac7050 con 0x56239fba9440 2024-04-24T23:31:57.664 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.652+0000 7f8c69834740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x56239fbb2010 con 0x56239fba9e30 2024-04-24T23:31:57.664 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.652+0000 7f8c69834740 10 client.0 _ll_register_callbacks cb 0x56239f81ebe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:31:57.664 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.652+0000 7f8c650bf640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56239fbaefa0 0x56239fbba9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:57.665 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.653+0000 7f8c650bf640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56239fbaefa0 0x56239fbba9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53588/0 (socket says 172.21.15.80:53588) 2024-04-24T23:31:57.665 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.653+0000 7f8c650bf640 1 -- 172.21.15.80:0/1520106331 learned_addr learned my addr 172.21.15.80:0/1520106331 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:31:57.665 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.653+0000 7f8c648be640 1 --2- 172.21.15.80:0/1520106331 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56239fba9440 0x56239fbb3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:57.665 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.653+0000 7f8c5ffff640 1 --2- 172.21.15.80:0/1520106331 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56239fba9e30 0x56239fbb39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:57.665 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.653+0000 7f8c650bf640 1 -- 172.21.15.80:0/1520106331 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56239fba9e30 msgr2=0x56239fbb39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:57.665 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.653+0000 7f8c650bf640 1 --2- 172.21.15.80:0/1520106331 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56239fba9e30 0x56239fbb39b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.665 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.653+0000 7f8c650bf640 1 -- 172.21.15.80:0/1520106331 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56239fba9440 msgr2=0x56239fbb3470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:57.665 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.653+0000 7f8c650bf640 1 --2- 172.21.15.80:0/1520106331 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56239fba9440 0x56239fbb3470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.665 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.653+0000 7f8c650bf640 1 -- 172.21.15.80:0/1520106331 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56239fa98dc0 con 0x56239fbaefa0 2024-04-24T23:31:57.666 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.653+0000 7f8c5ffff640 1 --2- 172.21.15.80:0/1520106331 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56239fba9e30 0x56239fbb39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:31:57.666 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.653+0000 7f8c648be640 1 --2- 172.21.15.80:0/1520106331 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56239fba9440 0x56239fbb3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:31:57.666 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.653+0000 7f8c650bf640 1 --2- 172.21.15.80:0/1520106331 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56239fbaefa0 0x56239fbba9f0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f8c48031ab0 tx=0x7f8c480038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c03004081ff621e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:57.666 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.653+0000 7f8c3ffff640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-04-24T23:31:57.666 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.653+0000 7f8c3ffff640 1 -- 172.21.15.80:0/1520106331 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c48056020 con 0x56239fbaefa0 2024-04-24T23:31:57.667 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.653+0000 7f8c3ffff640 1 -- 172.21.15.80:0/1520106331 <== mon.0 v2:172.21.15.80:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f8c48004a60 con 0x56239fbaefa0 2024-04-24T23:31:57.667 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.653+0000 7f8c69834740 1 -- 172.21.15.80:0/1520106331 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c48005520 con 0x56239fbaefa0 2024-04-24T23:31:57.667 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.653+0000 7f8c3ffff640 1 -- 172.21.15.80:0/1520106331 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8c48061050 con 0x56239fbaefa0 2024-04-24T23:31:57.667 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.653+0000 7f8c69834740 10 client.16035 fetch_fsmap learned FSMap version 27 2024-04-24T23:31:57.668 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.653+0000 7f8c69834740 1 -- 172.21.15.80:0/1520106331 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56239f81f3f0 con 0x56239fbaefa0 2024-04-24T23:31:57.668 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.653+0000 7f8c3ffff640 1 -- 172.21.15.80:0/1520106331 <== mon.0 v2:172.21.15.80:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c48005520 con 0x56239fbaefa0 2024-04-24T23:31:57.668 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.654+0000 7f8c3ffff640 1 -- 172.21.15.80:0/1520106331 <== mon.0 v2:172.21.15.80:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f8c48052770 con 0x56239fbaefa0 2024-04-24T23:31:57.668 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.654+0000 7f8c69834740 10 client.16035 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:31:57.668 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.654+0000 7f8c69834740 10 client.16035 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-04-24T23:31:57.668 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.654+0000 7f8c69834740 20 client.16035 populate_metadata read hostname 'smithi080' 2024-04-24T23:31:57.668 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.654+0000 7f8c69834740 1 --2- 172.21.15.80:0/1520106331 >> [v2:172.21.15.134:6838/1743815021,v1:172.21.15.134:6839/1743815021] conn(0x56239fbbeee0 0x56239fbc12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:57.668 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.654+0000 7f8c69834740 4 client.16035 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:31:57.668 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.654+0000 7f8c69834740 1 -- 172.21.15.80:0/1520106331 --> [v2:172.21.15.134:6838/1743815021,v1:172.21.15.134:6839/1743815021] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56239fb18bb0 con 0x56239fbbeee0 2024-04-24T23:31:57.669 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.654+0000 7f8c648be640 1 --2- 172.21.15.80:0/1520106331 >> [v2:172.21.15.134:6838/1743815021,v1:172.21.15.134:6839/1743815021] conn(0x56239fbbeee0 0x56239fbc12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:57.669 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.654+0000 7f8c3ffff640 10 client.16035 ms_handle_connect on v2:172.21.15.134:6838/1743815021 2024-04-24T23:31:57.669 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.654+0000 7f8c648be640 1 --2- 172.21.15.80:0/1520106331 >> [v2:172.21.15.134:6838/1743815021,v1:172.21.15.134:6839/1743815021] conn(0x56239fbbeee0 0x56239fbc12d0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=97f4a5959d75c520 server_cookie=32794bda28e13216 in_seq=0 out_seq=0 2024-04-24T23:31:57.669 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.655+0000 7f8c3ffff640 1 -- 172.21.15.80:0/1520106331 <== mds.1 v2:172.21.15.134:6838/1743815021 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56239fb18bb0 con 0x56239fbbeee0 2024-04-24T23:31:57.669 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.655+0000 7f8c3ffff640 10 client.16035 handle_command_reply: tid=0 2024-04-24T23:31:57.669 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.655+0000 7f8c69834740 1 client.16035 shutdown 2024-04-24T23:31:57.669 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.655+0000 7f8c69834740 1 -- 172.21.15.80:0/1520106331 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56239fbaefa0 msgr2=0x56239fbba9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:57.669 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.655+0000 7f8c69834740 1 --2- 172.21.15.80:0/1520106331 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56239fbaefa0 0x56239fbba9f0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f8c48031ab0 tx=0x7f8c480038a0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.670 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.655+0000 7f8c69834740 1 -- 172.21.15.80:0/1520106331 shutdown_connections 2024-04-24T23:31:57.670 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.655+0000 7f8c69834740 1 --2- 172.21.15.80:0/1520106331 >> [v2:172.21.15.134:6838/1743815021,v1:172.21.15.134:6839/1743815021] conn(0x56239fbbeee0 0x56239fbc12d0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.670 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.655+0000 7f8c69834740 1 --2- 172.21.15.80:0/1520106331 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56239fbaefa0 0x56239fbba9f0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.670 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.655+0000 7f8c3ffff640 0 client.16035 ms_handle_reset on v2:172.21.15.134:6838/1743815021 2024-04-24T23:31:57.670 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.655+0000 7f8c69834740 1 --2- 172.21.15.80:0/1520106331 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56239fba9e30 0x56239fbb39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.670 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.655+0000 7f8c69834740 1 --2- 172.21.15.80:0/1520106331 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56239fba9440 0x56239fbb3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.670 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.655+0000 7f8c69834740 1 -- 172.21.15.80:0/1520106331 >> 172.21.15.80:0/1520106331 conn(0x56239fba28a0 msgr2=0x56239fba1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:57.670 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.655+0000 7f8c69834740 1 -- 172.21.15.80:0/1520106331 shutdown_connections 2024-04-24T23:31:57.670 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.655+0000 7f8c69834740 1 -- 172.21.15.80:0/1520106331 wait complete. 2024-04-24T23:31:57.671 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.655+0000 7f8c69834740 20 client.16035 trim_cache size 0 max 16384 2024-04-24T23:31:57.677 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.668+0000 7f8c69834740 1 Processor -- start 2024-04-24T23:31:57.677 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.668+0000 7f8c69834740 1 -- start start 2024-04-24T23:31:57.677 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.669+0000 7f8c69834740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56239fba9440 0x56239fba12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:57.677 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.669+0000 7f8c69834740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56239fba9e30 0x56239fbba9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:57.678 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.669+0000 7f8c69834740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56239fbaefa0 0x56239fba17f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:57.678 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.669+0000 7f8c69834740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f8c601aab90 con 0x56239fba9e30 2024-04-24T23:31:57.678 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.669+0000 7f8c69834740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8c601a8460 con 0x56239fbaefa0 2024-04-24T23:31:57.678 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.669+0000 7f8c69834740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f8c601a62b0 con 0x56239fba9440 2024-04-24T23:31:57.678 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.669+0000 7f8c5ffff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56239fba9e30 0x56239fbba9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:57.678 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.669+0000 7f8c5ffff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56239fba9e30 0x56239fbba9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53590/0 (socket says 172.21.15.80:53590) 2024-04-24T23:31:57.678 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.669+0000 7f8c5ffff640 1 -- 172.21.15.80:0/966177813 learned_addr learned my addr 172.21.15.80:0/966177813 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:31:57.678 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.669+0000 7f8c648be640 1 --2- 172.21.15.80:0/966177813 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56239fba9440 0x56239fba12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:57.679 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.669+0000 7f8c5ffff640 1 -- 172.21.15.80:0/966177813 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56239fba9440 msgr2=0x56239fba12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:57.679 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.669+0000 7f8c5ffff640 1 --2- 172.21.15.80:0/966177813 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56239fba9440 0x56239fba12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.679 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.669+0000 7f8c5ffff640 1 -- 172.21.15.80:0/966177813 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56239fbaefa0 msgr2=0x56239fba17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:57.679 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.669+0000 7f8c650bf640 1 --2- 172.21.15.80:0/966177813 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56239fbaefa0 0x56239fba17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:57.679 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.669+0000 7f8c5ffff640 1 --2- 172.21.15.80:0/966177813 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56239fbaefa0 0x56239fba17f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.679 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.669+0000 7f8c5ffff640 1 -- 172.21.15.80:0/966177813 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56239f8e97d0 con 0x56239fba9e30 2024-04-24T23:31:57.679 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.669+0000 7f8c648be640 1 --2- 172.21.15.80:0/966177813 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56239fba9440 0x56239fba12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:31:57.679 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.669+0000 7f8c650bf640 1 --2- 172.21.15.80:0/966177813 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56239fbaefa0 0x56239fba17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:31:57.680 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.669+0000 7f8c5ffff640 1 --2- 172.21.15.80:0/966177813 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56239fba9e30 0x56239fbba9b0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f8c500058c0 tx=0x7f8c5000af80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a3552857ca4d9af server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:57.680 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.669+0000 7f8c567fc640 1 -- 172.21.15.80:0/966177813 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c50011b80 con 0x56239fba9e30 2024-04-24T23:31:57.680 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.669+0000 7f8c567fc640 1 -- 172.21.15.80:0/966177813 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c50011d20 con 0x56239fba9e30 2024-04-24T23:31:57.680 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.669+0000 7f8c567fc640 1 -- 172.21.15.80:0/966177813 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c5000f420 con 0x56239fba9e30 2024-04-24T23:31:57.680 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.670+0000 7f8c69834740 1 -- 172.21.15.80:0/966177813 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56239fba9e30 msgr2=0x56239fbba9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:57.680 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.670+0000 7f8c69834740 1 --2- 172.21.15.80:0/966177813 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56239fba9e30 0x56239fbba9b0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f8c500058c0 tx=0x7f8c5000af80 comp rx=0 tx=0).stop 2024-04-24T23:31:57.680 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.670+0000 7f8c69834740 1 -- 172.21.15.80:0/966177813 shutdown_connections 2024-04-24T23:31:57.680 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.670+0000 7f8c69834740 1 --2- 172.21.15.80:0/966177813 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56239fbaefa0 0x56239fba17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.680 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.670+0000 7f8c69834740 1 --2- 172.21.15.80:0/966177813 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56239fba9e30 0x56239fbba9b0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.681 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.670+0000 7f8c69834740 1 --2- 172.21.15.80:0/966177813 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56239fba9440 0x56239fba12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.681 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.670+0000 7f8c69834740 1 -- 172.21.15.80:0/966177813 >> 172.21.15.80:0/966177813 conn(0x56239fba28a0 msgr2=0x56239fbab160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:57.681 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.670+0000 7f8c69834740 1 -- 172.21.15.80:0/966177813 shutdown_connections 2024-04-24T23:31:57.681 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.670+0000 7f8c69834740 1 -- 172.21.15.80:0/966177813 wait complete. 2024-04-24T23:31:57.681 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.670+0000 7f8c69834740 1 -- start start 2024-04-24T23:31:57.681 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.670+0000 7f8c69834740 1 Processor -- start 2024-04-24T23:31:57.681 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.670+0000 7f8c69834740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56239fba9440 0x56239fbb8890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:57.681 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.670+0000 7f8c69834740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56239fba9e30 0x56239fbb8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:57.681 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.671+0000 7f8c69834740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56239fbaefa0 0x56239fbb9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:57.682 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.671+0000 7f8c69834740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f8c601a6920 con 0x56239fba9440 2024-04-24T23:31:57.682 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.671+0000 7f8c69834740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8c601a7d90 con 0x56239fba9e30 2024-04-24T23:31:57.682 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.671+0000 7f8c69834740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f8c601a6ff0 con 0x56239fbaefa0 2024-04-24T23:31:57.682 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.671+0000 7f8c69834740 10 client.0 _ll_register_callbacks cb 0x7f8c48031940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:31:57.682 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.671+0000 7f8c648be640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56239fba9440 0x56239fbb8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:57.682 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.671+0000 7f8c648be640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56239fba9440 0x56239fbb8890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53594/0 (socket says 172.21.15.80:53594) 2024-04-24T23:31:57.682 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.671+0000 7f8c648be640 1 -- 172.21.15.80:0/2530480745 learned_addr learned my addr 172.21.15.80:0/2530480745 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:31:57.682 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.671+0000 7f8c650bf640 1 --2- 172.21.15.80:0/2530480745 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56239fbaefa0 0x56239fbb9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:57.683 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.671+0000 7f8c5ffff640 1 --2- 172.21.15.80:0/2530480745 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56239fba9e30 0x56239fbb8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:57.683 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.671+0000 7f8c648be640 1 -- 172.21.15.80:0/2530480745 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56239fbaefa0 msgr2=0x56239fbb9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:57.683 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.671+0000 7f8c648be640 1 --2- 172.21.15.80:0/2530480745 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56239fbaefa0 0x56239fbb9cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.683 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.671+0000 7f8c648be640 1 -- 172.21.15.80:0/2530480745 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56239fba9e30 msgr2=0x56239fbb8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:57.683 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.671+0000 7f8c648be640 1 --2- 172.21.15.80:0/2530480745 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56239fba9e30 0x56239fbb8dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.683 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.671+0000 7f8c648be640 1 -- 172.21.15.80:0/2530480745 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56239fbba2c0 con 0x56239fba9440 2024-04-24T23:31:57.683 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.671+0000 7f8c5ffff640 1 --2- 172.21.15.80:0/2530480745 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56239fba9e30 0x56239fbb8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:31:57.683 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.671+0000 7f8c648be640 1 --2- 172.21.15.80:0/2530480745 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56239fba9440 0x56239fbb8890 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f8c580cd050 tx=0x7f8c5809c280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d93f59e23752940 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:57.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.671+0000 7f8c3ffff640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-04-24T23:31:57.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.671+0000 7f8c3ffff640 1 -- 172.21.15.80:0/2530480745 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c580b9060 con 0x56239fba9440 2024-04-24T23:31:57.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.671+0000 7f8c3ffff640 1 -- 172.21.15.80:0/2530480745 <== mon.0 v2:172.21.15.80:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f8c580cf280 con 0x56239fba9440 2024-04-24T23:31:57.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.671+0000 7f8c69834740 1 -- 172.21.15.80:0/2530480745 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c50011d20 con 0x56239fba9440 2024-04-24T23:31:57.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.671+0000 7f8c3ffff640 1 -- 172.21.15.80:0/2530480745 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8c5801d040 con 0x56239fba9440 2024-04-24T23:31:57.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.671+0000 7f8c69834740 10 client.16041 fetch_fsmap learned FSMap version 27 2024-04-24T23:31:57.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.671+0000 7f8c69834740 1 -- 172.21.15.80:0/2530480745 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56239fc259d0 con 0x56239fba9440 2024-04-24T23:31:57.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.671+0000 7f8c3ffff640 1 -- 172.21.15.80:0/2530480745 <== mon.0 v2:172.21.15.80:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c50011d20 con 0x56239fba9440 2024-04-24T23:31:57.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.672+0000 7f8c3ffff640 1 -- 172.21.15.80:0/2530480745 <== mon.0 v2:172.21.15.80:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f8c580a4220 con 0x56239fba9440 2024-04-24T23:31:57.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.672+0000 7f8c69834740 10 client.16041 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:31:57.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.672+0000 7f8c69834740 10 client.16041 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-04-24T23:31:57.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.672+0000 7f8c69834740 20 client.16041 populate_metadata read hostname 'smithi080' 2024-04-24T23:31:57.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.672+0000 7f8c69834740 1 --2- 172.21.15.80:0/2530480745 >> [v2:172.21.15.134:6838/1743815021,v1:172.21.15.134:6839/1743815021] conn(0x56239fc26330 0x56239fc28750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:57.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.672+0000 7f8c69834740 4 client.16041 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:31:57.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.672+0000 7f8c69834740 1 -- 172.21.15.80:0/2530480745 --> [v2:172.21.15.134:6838/1743815021,v1:172.21.15.134:6839/1743815021] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56239fb12f00 con 0x56239fc26330 2024-04-24T23:31:57.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.672+0000 7f8c5ffff640 1 --2- 172.21.15.80:0/2530480745 >> [v2:172.21.15.134:6838/1743815021,v1:172.21.15.134:6839/1743815021] conn(0x56239fc26330 0x56239fc28750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:57.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.672+0000 7f8c5ffff640 1 --2- 172.21.15.80:0/2530480745 >> [v2:172.21.15.134:6838/1743815021,v1:172.21.15.134:6839/1743815021] conn(0x56239fc26330 0x56239fc28750 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=694d468dfb72c34e server_cookie=8778d715ff00c4dd in_seq=0 out_seq=0 2024-04-24T23:31:57.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.672+0000 7f8c3ffff640 10 client.16041 ms_handle_connect on v2:172.21.15.134:6838/1743815021 2024-04-24T23:31:57.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.672+0000 7f8c3ffff640 1 -- 172.21.15.80:0/2530480745 <== mds.1 v2:172.21.15.134:6838/1743815021 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56239fb12f00 con 0x56239fc26330 2024-04-24T23:31:57.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.672+0000 7f8c3ffff640 10 client.16041 handle_command_reply: tid=0 2024-04-24T23:31:57.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.672+0000 7f8c69834740 1 client.16041 shutdown 2024-04-24T23:31:57.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.673+0000 7f8c69834740 1 -- 172.21.15.80:0/2530480745 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56239fba9440 msgr2=0x56239fbb8890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:57.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.673+0000 7f8c69834740 1 --2- 172.21.15.80:0/2530480745 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56239fba9440 0x56239fbb8890 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f8c580cd050 tx=0x7f8c5809c280 comp rx=0 tx=0).stop 2024-04-24T23:31:57.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.673+0000 7f8c69834740 1 -- 172.21.15.80:0/2530480745 shutdown_connections 2024-04-24T23:31:57.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.673+0000 7f8c69834740 1 --2- 172.21.15.80:0/2530480745 >> [v2:172.21.15.134:6838/1743815021,v1:172.21.15.134:6839/1743815021] conn(0x56239fc26330 0x56239fc28750 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.673+0000 7f8c69834740 1 --2- 172.21.15.80:0/2530480745 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56239fbaefa0 0x56239fbb9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.673+0000 7f8c69834740 1 --2- 172.21.15.80:0/2530480745 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56239fba9e30 0x56239fbb8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.673+0000 7f8c69834740 1 --2- 172.21.15.80:0/2530480745 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56239fba9440 0x56239fbb8890 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.673+0000 7f8c3ffff640 0 client.16041 ms_handle_reset on v2:172.21.15.134:6838/1743815021 2024-04-24T23:31:57.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.673+0000 7f8c69834740 1 -- 172.21.15.80:0/2530480745 >> 172.21.15.80:0/2530480745 conn(0x56239fba28a0 msgr2=0x56239fba57b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:57.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.673+0000 7f8c69834740 1 -- 172.21.15.80:0/2530480745 shutdown_connections 2024-04-24T23:31:57.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.673+0000 7f8c69834740 1 -- 172.21.15.80:0/2530480745 wait complete. 2024-04-24T23:31:57.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.673+0000 7f8c69834740 20 client.16041 trim_cache size 0 max 16384 2024-04-24T23:31:57.688 INFO:teuthology.orchestra.run.smithi080.stdout:[] 2024-04-24T23:31:57.689 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.673+0000 7f8c66b22640 1 -- 172.21.15.80:0/1176920993 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f8c4c0a2b90 msgr2=0x7f8c4c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:57.689 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.673+0000 7f8c66b22640 1 --2- 172.21.15.80:0/1176920993 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f8c4c0a2b90 0x7f8c4c0a5050 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f8c48009900 tx=0x7f8c480316b0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.689 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.673+0000 7f8c66b22640 1 -- 172.21.15.80:0/1176920993 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8c60100680 msgr2=0x7f8c60196940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:57.689 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.673+0000 7f8c66b22640 1 --2- 172.21.15.80:0/1176920993 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8c60100680 0x7f8c60196940 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f8c5800ccb0 tx=0x7f8c5800a620 comp rx=0 tx=0).stop 2024-04-24T23:31:57.690 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.673+0000 7f8c66b22640 1 -- 172.21.15.80:0/1176920993 shutdown_connections 2024-04-24T23:31:57.690 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.673+0000 7f8c66b22640 1 --2- 172.21.15.80:0/1176920993 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f8c4c0a2b90 0x7f8c4c0a5050 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.690 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.673+0000 7f8c66b22640 1 --2- 172.21.15.80:0/1176920993 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8c60102230 0x7f8c6019def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.690 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.673+0000 7f8c66b22640 1 --2- 172.21.15.80:0/1176920993 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c60101870 0x7f8c60196e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.690 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.673+0000 7f8c66b22640 1 --2- 172.21.15.80:0/1176920993 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8c60100680 0x7f8c60196940 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.690 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.673+0000 7f8c66b22640 1 -- 172.21.15.80:0/1176920993 >> 172.21.15.80:0/1176920993 conn(0x7f8c600fbbf0 msgr2=0x7f8c60104b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:57.690 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.673+0000 7f8c66b22640 1 -- 172.21.15.80:0/1176920993 shutdown_connections 2024-04-24T23:31:57.690 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.673+0000 7f8c66b22640 1 -- 172.21.15.80:0/1176920993 wait complete. 2024-04-24T23:31:57.691 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T23:31:57.794 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.790+0000 7f8f2d770640 1 -- 172.21.15.80:0/4279831331 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8f28100680 msgr2=0x7f8f28100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:57.795 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.790+0000 7f8f2d770640 1 --2- 172.21.15.80:0/4279831331 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8f28100680 0x7f8f28100aa0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f8f18009900 tx=0x7f8f1802a410 comp rx=0 tx=0).stop 2024-04-24T23:31:57.795 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.790+0000 7f8f2d770640 1 -- 172.21.15.80:0/4279831331 shutdown_connections 2024-04-24T23:31:57.795 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.790+0000 7f8f2d770640 1 --2- 172.21.15.80:0/4279831331 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f28102230 0x7f8f28108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.795 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.790+0000 7f8f2d770640 1 --2- 172.21.15.80:0/4279831331 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8f28101870 0x7f8f28101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.795 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.790+0000 7f8f2d770640 1 --2- 172.21.15.80:0/4279831331 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8f28100680 0x7f8f28100aa0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.795 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.790+0000 7f8f2d770640 1 -- 172.21.15.80:0/4279831331 >> 172.21.15.80:0/4279831331 conn(0x7f8f280fbbf0 msgr2=0x7f8f280fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:57.796 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.790+0000 7f8f2d770640 1 -- 172.21.15.80:0/4279831331 shutdown_connections 2024-04-24T23:31:57.796 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.790+0000 7f8f2d770640 1 -- 172.21.15.80:0/4279831331 wait complete. 2024-04-24T23:31:57.796 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.791+0000 7f8f2d770640 1 Processor -- start 2024-04-24T23:31:57.796 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.791+0000 7f8f2d770640 1 -- start start 2024-04-24T23:31:57.796 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.791+0000 7f8f2d770640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f28100680 0x7f8f280694a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:57.796 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.791+0000 7f8f2d770640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8f28101870 0x7f8f2819c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:57.796 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.791+0000 7f8f2d770640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8f28102230 0x7f8f2819ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:57.797 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.791+0000 7f8f2d770640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f8f2810c2d0 con 0x7f8f28102230 2024-04-24T23:31:57.797 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.791+0000 7f8f2d770640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8f2810c150 con 0x7f8f28101870 2024-04-24T23:31:57.797 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.791+0000 7f8f2d770640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f8f2810c450 con 0x7f8f28100680 2024-04-24T23:31:57.797 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.791+0000 7f8f277fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8f28102230 0x7f8f2819ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:57.797 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.791+0000 7f8f277fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8f28102230 0x7f8f2819ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53614/0 (socket says 172.21.15.80:53614) 2024-04-24T23:31:57.797 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.791+0000 7f8f277fe640 1 -- 172.21.15.80:0/1526791608 learned_addr learned my addr 172.21.15.80:0/1526791608 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:31:57.797 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.792+0000 7f8f267fc640 1 --2- 172.21.15.80:0/1526791608 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8f28101870 0x7f8f2819c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:57.797 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.792+0000 7f8f26ffd640 1 --2- 172.21.15.80:0/1526791608 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f28100680 0x7f8f280694a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:57.798 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.792+0000 7f8f277fe640 1 -- 172.21.15.80:0/1526791608 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f28100680 msgr2=0x7f8f280694a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:57.798 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.792+0000 7f8f277fe640 1 --2- 172.21.15.80:0/1526791608 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f28100680 0x7f8f280694a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.798 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.792+0000 7f8f277fe640 1 -- 172.21.15.80:0/1526791608 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8f28101870 msgr2=0x7f8f2819c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:57.798 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.792+0000 7f8f277fe640 1 --2- 172.21.15.80:0/1526791608 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8f28101870 0x7f8f2819c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.798 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.792+0000 7f8f277fe640 1 -- 172.21.15.80:0/1526791608 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f2819f750 con 0x7f8f28102230 2024-04-24T23:31:57.798 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.792+0000 7f8f267fc640 1 --2- 172.21.15.80:0/1526791608 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8f28101870 0x7f8f2819c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:31:57.798 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.792+0000 7f8f26ffd640 1 --2- 172.21.15.80:0/1526791608 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f28100680 0x7f8f280694a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:31:57.798 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.792+0000 7f8f277fe640 1 --2- 172.21.15.80:0/1526791608 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8f28102230 0x7f8f2819ee90 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f8f20009fd0 tx=0x7f8f2000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:57.798 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.792+0000 7f8f0ffff640 1 -- 172.21.15.80:0/1526791608 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f20017070 con 0x7f8f28102230 2024-04-24T23:31:57.799 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.792+0000 7f8f0ffff640 1 -- 172.21.15.80:0/1526791608 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f20003e50 con 0x7f8f28102230 2024-04-24T23:31:57.799 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.792+0000 7f8f2d770640 1 -- 172.21.15.80:0/1526791608 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f2819fa40 con 0x7f8f28102230 2024-04-24T23:31:57.799 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.792+0000 7f8f0ffff640 1 -- 172.21.15.80:0/1526791608 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f200106c0 con 0x7f8f28102230 2024-04-24T23:31:57.799 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.792+0000 7f8f2d770640 1 -- 172.21.15.80:0/1526791608 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f2819ff40 con 0x7f8f28102230 2024-04-24T23:31:57.799 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.794+0000 7f8f0ffff640 1 -- 172.21.15.80:0/1526791608 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8f20003970 con 0x7f8f28102230 2024-04-24T23:31:57.806 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.794+0000 7f8f3047d740 1 Processor -- start 2024-04-24T23:31:57.806 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.794+0000 7f8f3047d740 1 -- start start 2024-04-24T23:31:57.806 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.795+0000 7f8f3047d740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ecc7f68230 0x55ecc7f68650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:57.806 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.795+0000 7f8f3047d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ecc7f68c20 0x55ecc7f6d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:57.806 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.795+0000 7f8f3047d740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ecc7f6dd90 0x55ecc7f70220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:57.807 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.795+0000 7f8f3047d740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x55ecc7ecc670 con 0x55ecc7f68230 2024-04-24T23:31:57.807 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.795+0000 7f8f3047d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55ecc7e74970 con 0x55ecc7f68c20 2024-04-24T23:31:57.807 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.795+0000 7f8f3047d740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55ecc7ec8630 con 0x55ecc7f6dd90 2024-04-24T23:31:57.807 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.795+0000 7f8f0ffff640 1 --2- 172.21.15.80:0/1526791608 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f8f100a2b90 0x7f8f100a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:57.807 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.795+0000 7f8f267fc640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ecc7f68230 0x55ecc7f68650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:57.807 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.795+0000 7f8f267fc640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ecc7f68230 0x55ecc7f68650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53616/0 (socket says 172.21.15.80:53616) 2024-04-24T23:31:57.807 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.795+0000 7f8f267fc640 1 -- 172.21.15.80:0/2728101063 learned_addr learned my addr 172.21.15.80:0/2728101063 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:31:57.807 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.795+0000 7f8f26ffd640 1 --2- 172.21.15.80:0/2728101063 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ecc7f68c20 0x55ecc7f6d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:57.808 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.795+0000 7f8f277fe640 1 --2- 172.21.15.80:0/1526791608 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f8f100a2b90 0x7f8f100a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:57.808 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.795+0000 7f8f267fc640 1 -- 172.21.15.80:0/2728101063 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ecc7f6dd90 msgr2=0x55ecc7f70220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:57.808 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.795+0000 7f8f0ffff640 1 -- 172.21.15.80:0/1526791608 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f8f200cf280 con 0x7f8f28102230 2024-04-24T23:31:57.808 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.795+0000 7f8f267fc640 1 --2- 172.21.15.80:0/2728101063 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ecc7f6dd90 0x55ecc7f70220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.808 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.795+0000 7f8f267fc640 1 -- 172.21.15.80:0/2728101063 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ecc7f68c20 msgr2=0x55ecc7f6d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:57.808 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.795+0000 7f8f267fc640 1 --2- 172.21.15.80:0/2728101063 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ecc7f68c20 0x55ecc7f6d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.808 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.795+0000 7f8f267fc640 1 -- 172.21.15.80:0/2728101063 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ecc7e98400 con 0x55ecc7f68230 2024-04-24T23:31:57.808 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.795+0000 7f8f26ffd640 1 --2- 172.21.15.80:0/2728101063 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ecc7f68c20 0x55ecc7f6d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:31:57.808 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.795+0000 7f8f267fc640 1 --2- 172.21.15.80:0/2728101063 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ecc7f68230 0x55ecc7f68650 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c00bdb0 tx=0x7f8f1c011750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1be5ee50b96ab4af server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:57.809 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.795+0000 7f8f277fe640 1 --2- 172.21.15.80:0/1526791608 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f8f100a2b90 0x7f8f100a5050 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f8f200980e0 tx=0x7f8f2006a690 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:57.809 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.795+0000 7f8f0d7fa640 1 -- 172.21.15.80:0/2728101063 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f1c011e60 con 0x55ecc7f68230 2024-04-24T23:31:57.809 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.795+0000 7f8f0d7fa640 1 -- 172.21.15.80:0/2728101063 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f1c01aae0 con 0x55ecc7f68230 2024-04-24T23:31:57.809 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.795+0000 7f8f0d7fa640 1 -- 172.21.15.80:0/2728101063 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f1c012be0 con 0x55ecc7f68230 2024-04-24T23:31:57.809 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.796+0000 7f8f3047d740 1 -- 172.21.15.80:0/2728101063 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ecc7f68230 msgr2=0x55ecc7f68650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:57.809 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.796+0000 7f8f3047d740 1 --2- 172.21.15.80:0/2728101063 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ecc7f68230 0x55ecc7f68650 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c00bdb0 tx=0x7f8f1c011750 comp rx=0 tx=0).stop 2024-04-24T23:31:57.809 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.796+0000 7f8f3047d740 1 -- 172.21.15.80:0/2728101063 shutdown_connections 2024-04-24T23:31:57.809 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.796+0000 7f8f3047d740 1 --2- 172.21.15.80:0/2728101063 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ecc7f6dd90 0x55ecc7f70220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.809 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.796+0000 7f8f3047d740 1 --2- 172.21.15.80:0/2728101063 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ecc7f68c20 0x55ecc7f6d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.810 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.796+0000 7f8f3047d740 1 --2- 172.21.15.80:0/2728101063 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ecc7f68230 0x55ecc7f68650 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.810 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.796+0000 7f8f3047d740 1 -- 172.21.15.80:0/2728101063 >> 172.21.15.80:0/2728101063 conn(0x55ecc7f638a0 msgr2=0x55ecc7f63ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:57.810 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.796+0000 7f8f3047d740 1 -- 172.21.15.80:0/2728101063 shutdown_connections 2024-04-24T23:31:57.810 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.796+0000 7f8f3047d740 1 -- 172.21.15.80:0/2728101063 wait complete. 2024-04-24T23:31:57.810 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.796+0000 7f8f3047d740 1 -- start start 2024-04-24T23:31:57.810 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.796+0000 7f8f3047d740 1 Processor -- start 2024-04-24T23:31:57.810 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.797+0000 7f8f3047d740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ecc7f68230 0x55ecc7f72350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:57.810 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.797+0000 7f8f3047d740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ecc7f68c20 0x55ecc7f72890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:57.811 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.797+0000 7f8f3047d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ecc7f6dd90 0x55ecc7f798d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:57.811 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.797+0000 7f8f3047d740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x55ecc7f70d40 con 0x55ecc7f68c20 2024-04-24T23:31:57.811 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.797+0000 7f8f3047d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55ecc7e88050 con 0x55ecc7f6dd90 2024-04-24T23:31:57.811 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.797+0000 7f8f3047d740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55ecc7f70ec0 con 0x55ecc7f68230 2024-04-24T23:31:57.811 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.797+0000 7f8f3047d740 10 client.0 _ll_register_callbacks cb 0x55ecc7bdfbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:31:57.811 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.797+0000 7f8f26ffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ecc7f68c20 0x55ecc7f72890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:57.811 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.797+0000 7f8f26ffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ecc7f68c20 0x55ecc7f72890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53618/0 (socket says 172.21.15.80:53618) 2024-04-24T23:31:57.811 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.797+0000 7f8f26ffd640 1 -- 172.21.15.80:0/381214848 learned_addr learned my addr 172.21.15.80:0/381214848 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:31:57.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.797+0000 7f8f267fc640 1 --2- 172.21.15.80:0/381214848 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ecc7f68230 0x55ecc7f72350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:57.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.797+0000 7f8f26ffd640 1 -- 172.21.15.80:0/381214848 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ecc7f68230 msgr2=0x55ecc7f72350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:57.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.797+0000 7f8f26ffd640 1 --2- 172.21.15.80:0/381214848 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ecc7f68230 0x55ecc7f72350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.797+0000 7f8f26ffd640 1 -- 172.21.15.80:0/381214848 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ecc7f6dd90 msgr2=0x55ecc7f798d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:57.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.797+0000 7f8f26ffd640 1 --2- 172.21.15.80:0/381214848 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ecc7f6dd90 0x55ecc7f798d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.797+0000 7f8f26ffd640 1 -- 172.21.15.80:0/381214848 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ecc7e59dc0 con 0x55ecc7f68c20 2024-04-24T23:31:57.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.797+0000 7f8f267fc640 1 --2- 172.21.15.80:0/381214848 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ecc7f68230 0x55ecc7f72350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:31:57.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.797+0000 7f8f26ffd640 1 --2- 172.21.15.80:0/381214848 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ecc7f68c20 0x55ecc7f72890 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f8f18000c00 tx=0x7f8f18031aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38d55715993bea4f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:57.813 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.797+0000 7f8f0aff5640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-04-24T23:31:57.813 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.797+0000 7f8f0aff5640 1 -- 172.21.15.80:0/381214848 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f18035070 con 0x55ecc7f68c20 2024-04-24T23:31:57.813 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.797+0000 7f8f0aff5640 1 -- 172.21.15.80:0/381214848 <== mon.0 v2:172.21.15.80:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f8f18004340 con 0x55ecc7f68c20 2024-04-24T23:31:57.813 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.797+0000 7f8f3047d740 1 -- 172.21.15.80:0/381214848 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f1c01aae0 con 0x55ecc7f68c20 2024-04-24T23:31:57.813 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.797+0000 7f8f0aff5640 1 -- 172.21.15.80:0/381214848 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8f1804d050 con 0x55ecc7f68c20 2024-04-24T23:31:57.813 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.797+0000 7f8f3047d740 10 client.16053 fetch_fsmap learned FSMap version 27 2024-04-24T23:31:57.813 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.797+0000 7f8f3047d740 1 -- 172.21.15.80:0/381214848 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ecc7be03f0 con 0x55ecc7f68c20 2024-04-24T23:31:57.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.798+0000 7f8f0aff5640 1 -- 172.21.15.80:0/381214848 <== mon.0 v2:172.21.15.80:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f1c01aae0 con 0x55ecc7f68c20 2024-04-24T23:31:57.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.798+0000 7f8f0aff5640 1 -- 172.21.15.80:0/381214848 <== mon.0 v2:172.21.15.80:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f8f18004b80 con 0x55ecc7f68c20 2024-04-24T23:31:57.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.798+0000 7f8f3047d740 10 client.16053 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:31:57.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.798+0000 7f8f3047d740 10 client.16053 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:31:57.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.798+0000 7f8f3047d740 20 client.16053 populate_metadata read hostname 'smithi080' 2024-04-24T23:31:57.815 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.798+0000 7f8f3047d740 1 --2- 172.21.15.80:0/381214848 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x55ecc7f7e250 0x55ecc7f806b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:57.815 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.798+0000 7f8f3047d740 4 client.16053 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:31:57.815 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.798+0000 7f8f3047d740 1 -- 172.21.15.80:0/381214848 --> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ecc7ed9bb0 con 0x55ecc7f7e250 2024-04-24T23:31:57.815 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.798+0000 7f8f267fc640 1 --2- 172.21.15.80:0/381214848 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x55ecc7f7e250 0x55ecc7f806b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:57.815 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.798+0000 7f8f267fc640 1 --2- 172.21.15.80:0/381214848 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x55ecc7f7e250 0x55ecc7f806b0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e35cc032af2d2173 server_cookie=3eeb3b430f536fda in_seq=0 out_seq=0 2024-04-24T23:31:57.815 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.798+0000 7f8f0aff5640 10 client.16053 ms_handle_connect on v2:172.21.15.148:6838/2952757594 2024-04-24T23:31:57.816 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.799+0000 7f8f0aff5640 1 -- 172.21.15.80:0/381214848 <== mds.0 v2:172.21.15.148:6838/2952757594 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ecc7ed9bb0 con 0x55ecc7f7e250 2024-04-24T23:31:57.816 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.799+0000 7f8f0aff5640 10 client.16053 handle_command_reply: tid=0 2024-04-24T23:31:57.816 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.799+0000 7f8f3047d740 1 client.16053 shutdown 2024-04-24T23:31:57.816 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.799+0000 7f8f3047d740 1 -- 172.21.15.80:0/381214848 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ecc7f68c20 msgr2=0x55ecc7f72890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:57.816 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.799+0000 7f8f3047d740 1 --2- 172.21.15.80:0/381214848 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ecc7f68c20 0x55ecc7f72890 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f8f18000c00 tx=0x7f8f18031aa0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.816 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.799+0000 7f8f3047d740 1 -- 172.21.15.80:0/381214848 shutdown_connections 2024-04-24T23:31:57.816 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.799+0000 7f8f3047d740 1 --2- 172.21.15.80:0/381214848 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x55ecc7f7e250 0x55ecc7f806b0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.817 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.799+0000 7f8f3047d740 1 --2- 172.21.15.80:0/381214848 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ecc7f6dd90 0x55ecc7f798d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.817 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.799+0000 7f8f0aff5640 0 client.16053 ms_handle_reset on v2:172.21.15.148:6838/2952757594 2024-04-24T23:31:57.817 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.799+0000 7f8f3047d740 1 --2- 172.21.15.80:0/381214848 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ecc7f68c20 0x55ecc7f72890 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.817 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.799+0000 7f8f3047d740 1 --2- 172.21.15.80:0/381214848 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ecc7f68230 0x55ecc7f72350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.817 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.799+0000 7f8f3047d740 1 -- 172.21.15.80:0/381214848 >> 172.21.15.80:0/381214848 conn(0x55ecc7f638a0 msgr2=0x55ecc7f62b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:57.817 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.799+0000 7f8f3047d740 1 -- 172.21.15.80:0/381214848 shutdown_connections 2024-04-24T23:31:57.818 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.799+0000 7f8f3047d740 1 -- 172.21.15.80:0/381214848 wait complete. 2024-04-24T23:31:57.818 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.799+0000 7f8f3047d740 20 client.16053 trim_cache size 0 max 16384 2024-04-24T23:31:57.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.812+0000 7f8f3047d740 1 Processor -- start 2024-04-24T23:31:57.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.812+0000 7f8f3047d740 1 -- start start 2024-04-24T23:31:57.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.813+0000 7f8f3047d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ecc7f68230 0x55ecc7f69670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:57.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.813+0000 7f8f3047d740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ecc7f68c20 0x55ecc7f69bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:57.827 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.813+0000 7f8f3047d740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ecc7f6dd90 0x55ecc7f6a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:57.827 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.813+0000 7f8f3047d740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f8f281acd70 con 0x55ecc7f68c20 2024-04-24T23:31:57.827 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.813+0000 7f8f3047d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8f281aa640 con 0x55ecc7f68230 2024-04-24T23:31:57.827 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.813+0000 7f8f3047d740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f8f281a8490 con 0x55ecc7f6dd90 2024-04-24T23:31:57.827 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.813+0000 7f8f26ffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ecc7f68c20 0x55ecc7f69bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:57.827 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.813+0000 7f8f26ffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ecc7f68c20 0x55ecc7f69bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53630/0 (socket says 172.21.15.80:53630) 2024-04-24T23:31:57.827 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.813+0000 7f8f26ffd640 1 -- 172.21.15.80:0/3301455415 learned_addr learned my addr 172.21.15.80:0/3301455415 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:31:57.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.813+0000 7f8f26ffd640 1 -- 172.21.15.80:0/3301455415 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ecc7f6dd90 msgr2=0x55ecc7f6a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:57.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.813+0000 7f8f267fc640 1 --2- 172.21.15.80:0/3301455415 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ecc7f6dd90 0x55ecc7f6a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:57.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.813+0000 7f8f26ffd640 1 --2- 172.21.15.80:0/3301455415 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ecc7f6dd90 0x55ecc7f6a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.813+0000 7f8f267fc640 1 --2- 172.21.15.80:0/3301455415 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ecc7f68230 0x55ecc7f69670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:57.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.813+0000 7f8f26ffd640 1 -- 172.21.15.80:0/3301455415 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ecc7f68230 msgr2=0x55ecc7f69670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:57.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.813+0000 7f8f26ffd640 1 --2- 172.21.15.80:0/3301455415 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ecc7f68230 0x55ecc7f69670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.813+0000 7f8f26ffd640 1 -- 172.21.15.80:0/3301455415 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ecc7caa7d0 con 0x55ecc7f68c20 2024-04-24T23:31:57.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.813+0000 7f8f267fc640 1 --2- 172.21.15.80:0/3301455415 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ecc7f68230 0x55ecc7f69670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:31:57.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.813+0000 7f8f26ffd640 1 --2- 172.21.15.80:0/3301455415 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ecc7f68c20 0x55ecc7f69bb0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f8f1802a930 tx=0x7f8f180039f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fcde8c00fa2389c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:57.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.813+0000 7f8f0cff9640 1 -- 172.21.15.80:0/3301455415 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f18035070 con 0x55ecc7f68c20 2024-04-24T23:31:57.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.813+0000 7f8f0cff9640 1 -- 172.21.15.80:0/3301455415 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f18031d10 con 0x55ecc7f68c20 2024-04-24T23:31:57.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.813+0000 7f8f0cff9640 1 -- 172.21.15.80:0/3301455415 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f18004a80 con 0x55ecc7f68c20 2024-04-24T23:31:57.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.814+0000 7f8f3047d740 1 -- 172.21.15.80:0/3301455415 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ecc7f68c20 msgr2=0x55ecc7f69bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:57.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.814+0000 7f8f3047d740 1 --2- 172.21.15.80:0/3301455415 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ecc7f68c20 0x55ecc7f69bb0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f8f1802a930 tx=0x7f8f180039f0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.830 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.814+0000 7f8f3047d740 1 -- 172.21.15.80:0/3301455415 shutdown_connections 2024-04-24T23:31:57.830 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.814+0000 7f8f3047d740 1 --2- 172.21.15.80:0/3301455415 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ecc7f6dd90 0x55ecc7f6a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.830 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.814+0000 7f8f3047d740 1 --2- 172.21.15.80:0/3301455415 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ecc7f68c20 0x55ecc7f69bb0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.830 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.814+0000 7f8f3047d740 1 --2- 172.21.15.80:0/3301455415 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ecc7f68230 0x55ecc7f69670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.830 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.814+0000 7f8f3047d740 1 -- 172.21.15.80:0/3301455415 >> 172.21.15.80:0/3301455415 conn(0x55ecc7f638a0 msgr2=0x55ecc7f62b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:57.830 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.814+0000 7f8f3047d740 1 -- 172.21.15.80:0/3301455415 shutdown_connections 2024-04-24T23:31:57.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.814+0000 7f8f3047d740 1 -- 172.21.15.80:0/3301455415 wait complete. 2024-04-24T23:31:57.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.814+0000 7f8f3047d740 1 -- start start 2024-04-24T23:31:57.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.814+0000 7f8f3047d740 1 Processor -- start 2024-04-24T23:31:57.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.814+0000 7f8f3047d740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ecc7f68230 0x55ecc7fdfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:57.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.814+0000 7f8f3047d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ecc7f68c20 0x55ecc7fe0130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:57.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.814+0000 7f8f3047d740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ecc7f6dd90 0x55ecc7fe7100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:57.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.814+0000 7f8f3047d740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f8f281a8b00 con 0x55ecc7f6dd90 2024-04-24T23:31:57.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.814+0000 7f8f3047d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8f281a9f70 con 0x55ecc7f68c20 2024-04-24T23:31:57.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.814+0000 7f8f3047d740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f8f281a91d0 con 0x55ecc7f68230 2024-04-24T23:31:57.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.814+0000 7f8f3047d740 10 client.0 _ll_register_callbacks cb 0x7f8f18009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:31:57.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.814+0000 7f8f267fc640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ecc7f6dd90 0x55ecc7fe7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:57.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.814+0000 7f8f267fc640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ecc7f6dd90 0x55ecc7fe7100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53636/0 (socket says 172.21.15.80:53636) 2024-04-24T23:31:57.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.814+0000 7f8f267fc640 1 -- 172.21.15.80:0/3213383446 learned_addr learned my addr 172.21.15.80:0/3213383446 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:31:57.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.814+0000 7f8f267fc640 1 --2- 172.21.15.80:0/3213383446 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ecc7f68230 0x55ecc7fdfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:57.833 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.815+0000 7f8f26ffd640 1 --2- 172.21.15.80:0/3213383446 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ecc7f68c20 0x55ecc7fe0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:57.833 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.815+0000 7f8f267fc640 1 -- 172.21.15.80:0/3213383446 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ecc7f68230 msgr2=0x55ecc7fdfbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:57.833 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.815+0000 7f8f267fc640 1 --2- 172.21.15.80:0/3213383446 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ecc7f68230 0x55ecc7fdfbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.833 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.815+0000 7f8f267fc640 1 -- 172.21.15.80:0/3213383446 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ecc7f68c20 msgr2=0x55ecc7fe0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:31:57.833 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.815+0000 7f8f267fc640 1 --2- 172.21.15.80:0/3213383446 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ecc7f68c20 0x55ecc7fe0130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.833 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.815+0000 7f8f267fc640 1 -- 172.21.15.80:0/3213383446 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ecc7fe76f0 con 0x55ecc7f6dd90 2024-04-24T23:31:57.833 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.815+0000 7f8f26ffd640 1 --2- 172.21.15.80:0/3213383446 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ecc7f68c20 0x55ecc7fe0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:31:57.833 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.815+0000 7f8f267fc640 1 --2- 172.21.15.80:0/3213383446 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ecc7f6dd90 0x55ecc7fe7100 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c005890 tx=0x7f8f1c011750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=653505d0634ece2e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:57.834 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.815+0000 7f8f0aff5640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-04-24T23:31:57.834 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.815+0000 7f8f0aff5640 1 -- 172.21.15.80:0/3213383446 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f1c029770 con 0x55ecc7f6dd90 2024-04-24T23:31:57.834 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.815+0000 7f8f0aff5640 1 -- 172.21.15.80:0/3213383446 <== mon.0 v2:172.21.15.80:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f8f1c004030 con 0x55ecc7f6dd90 2024-04-24T23:31:57.834 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.815+0000 7f8f3047d740 1 -- 172.21.15.80:0/3213383446 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f18031d10 con 0x55ecc7f6dd90 2024-04-24T23:31:57.834 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.815+0000 7f8f0aff5640 1 -- 172.21.15.80:0/3213383446 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8f1c029910 con 0x55ecc7f6dd90 2024-04-24T23:31:57.834 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.815+0000 7f8f3047d740 10 client.16059 fetch_fsmap learned FSMap version 27 2024-04-24T23:31:57.834 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.815+0000 7f8f3047d740 1 -- 172.21.15.80:0/3213383446 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ecc7fe8a00 con 0x55ecc7f6dd90 2024-04-24T23:31:57.834 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.815+0000 7f8f0aff5640 1 -- 172.21.15.80:0/3213383446 <== mon.0 v2:172.21.15.80:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f18031d10 con 0x55ecc7f6dd90 2024-04-24T23:31:57.835 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.815+0000 7f8f0aff5640 1 -- 172.21.15.80:0/3213383446 <== mon.0 v2:172.21.15.80:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f8f1c01a610 con 0x55ecc7f6dd90 2024-04-24T23:31:57.835 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.815+0000 7f8f3047d740 10 client.16059 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:31:57.835 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.815+0000 7f8f3047d740 10 client.16059 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-04-24T23:31:57.835 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.815+0000 7f8f3047d740 20 client.16059 populate_metadata read hostname 'smithi080' 2024-04-24T23:31:57.835 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.815+0000 7f8f3047d740 1 --2- 172.21.15.80:0/3213383446 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x55ecc7fe9220 0x55ecc7feb640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:57.835 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.815+0000 7f8f3047d740 4 client.16059 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:31:57.835 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.815+0000 7f8f3047d740 1 -- 172.21.15.80:0/3213383446 --> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ecc7f72140 con 0x55ecc7fe9220 2024-04-24T23:31:57.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.816+0000 7f8f26ffd640 1 --2- 172.21.15.80:0/3213383446 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x55ecc7fe9220 0x55ecc7feb640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:57.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.816+0000 7f8f26ffd640 1 --2- 172.21.15.80:0/3213383446 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x55ecc7fe9220 0x55ecc7feb640 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ac9bce519fa7862 server_cookie=9eb978c4ca6c264d in_seq=0 out_seq=0 2024-04-24T23:31:57.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.816+0000 7f8f0aff5640 10 client.16059 ms_handle_connect on v2:172.21.15.148:6838/2952757594 2024-04-24T23:31:57.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.816+0000 7f8f0aff5640 1 -- 172.21.15.80:0/3213383446 <== mds.0 v2:172.21.15.148:6838/2952757594 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ecc7f72140 con 0x55ecc7fe9220 2024-04-24T23:31:57.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.816+0000 7f8f0aff5640 10 client.16059 handle_command_reply: tid=0 2024-04-24T23:31:57.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.816+0000 7f8f3047d740 1 client.16059 shutdown 2024-04-24T23:31:57.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.816+0000 7f8f3047d740 1 -- 172.21.15.80:0/3213383446 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ecc7f6dd90 msgr2=0x55ecc7fe7100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:57.837 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.816+0000 7f8f3047d740 1 --2- 172.21.15.80:0/3213383446 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ecc7f6dd90 0x55ecc7fe7100 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c005890 tx=0x7f8f1c011750 comp rx=0 tx=0).stop 2024-04-24T23:31:57.837 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.816+0000 7f8f3047d740 1 -- 172.21.15.80:0/3213383446 shutdown_connections 2024-04-24T23:31:57.837 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.816+0000 7f8f3047d740 1 --2- 172.21.15.80:0/3213383446 >> [v2:172.21.15.148:6838/2952757594,v1:172.21.15.148:6839/2952757594] conn(0x55ecc7fe9220 0x55ecc7feb640 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.837 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.816+0000 7f8f3047d740 1 --2- 172.21.15.80:0/3213383446 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ecc7f6dd90 0x55ecc7fe7100 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.837 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.816+0000 7f8f3047d740 1 --2- 172.21.15.80:0/3213383446 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ecc7f68c20 0x55ecc7fe0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.837 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.816+0000 7f8f0aff5640 0 client.16059 ms_handle_reset on v2:172.21.15.148:6838/2952757594 2024-04-24T23:31:57.837 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.816+0000 7f8f3047d740 1 --2- 172.21.15.80:0/3213383446 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ecc7f68230 0x55ecc7fdfbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.837 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.816+0000 7f8f3047d740 1 -- 172.21.15.80:0/3213383446 >> 172.21.15.80:0/3213383446 conn(0x55ecc7f638a0 msgr2=0x55ecc7f66830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:57.837 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.817+0000 7f8f3047d740 1 -- 172.21.15.80:0/3213383446 shutdown_connections 2024-04-24T23:31:57.838 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.817+0000 7f8f3047d740 1 -- 172.21.15.80:0/3213383446 wait complete. 2024-04-24T23:31:57.838 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.817+0000 7f8f3047d740 20 client.16059 trim_cache size 0 max 16384 2024-04-24T23:31:57.838 INFO:teuthology.orchestra.run.smithi080.stdout:[] 2024-04-24T23:31:57.840 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.817+0000 7f8f2d770640 1 -- 172.21.15.80:0/1526791608 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f8f100a2b90 msgr2=0x7f8f100a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:57.840 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.817+0000 7f8f2d770640 1 --2- 172.21.15.80:0/1526791608 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f8f100a2b90 0x7f8f100a5050 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f8f200980e0 tx=0x7f8f2006a690 comp rx=0 tx=0).stop 2024-04-24T23:31:57.840 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.817+0000 7f8f2d770640 1 -- 172.21.15.80:0/1526791608 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8f28102230 msgr2=0x7f8f2819ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:57.840 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.817+0000 7f8f2d770640 1 --2- 172.21.15.80:0/1526791608 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8f28102230 0x7f8f2819ee90 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f8f20009fd0 tx=0x7f8f2000b260 comp rx=0 tx=0).stop 2024-04-24T23:31:57.840 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.817+0000 7f8f2d770640 1 -- 172.21.15.80:0/1526791608 shutdown_connections 2024-04-24T23:31:57.841 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.817+0000 7f8f2d770640 1 --2- 172.21.15.80:0/1526791608 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f8f100a2b90 0x7f8f100a5050 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.841 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.817+0000 7f8f2d770640 1 --2- 172.21.15.80:0/1526791608 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8f28102230 0x7f8f2819ee90 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.841 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.817+0000 7f8f2d770640 1 --2- 172.21.15.80:0/1526791608 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8f28101870 0x7f8f2819c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.841 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.817+0000 7f8f2d770640 1 --2- 172.21.15.80:0/1526791608 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f28100680 0x7f8f280694a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:57.841 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.817+0000 7f8f2d770640 1 -- 172.21.15.80:0/1526791608 >> 172.21.15.80:0/1526791608 conn(0x7f8f280fbbf0 msgr2=0x7f8f28104c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:57.841 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.817+0000 7f8f2d770640 1 -- 172.21.15.80:0/1526791608 shutdown_connections 2024-04-24T23:31:57.841 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:57.817+0000 7f8f2d770640 1 -- 172.21.15.80:0/1526791608 wait complete. 2024-04-24T23:31:58.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:31:58 smithi134 ceph-mon[32619]: pgmap v439: 129 pgs: 129 active+clean; 69 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 296 KiB/s wr, 24 op/s 2024-04-24T23:31:58.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:31:58 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/1349285704' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:31:58.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:31:58 smithi148 ceph-mon[32528]: pgmap v439: 129 pgs: 129 active+clean; 69 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 296 KiB/s wr, 24 op/s 2024-04-24T23:31:58.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:31:58 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/1349285704' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:31:58.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:31:58 smithi080 ceph-mon[29152]: pgmap v439: 129 pgs: 129 active+clean; 69 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 296 KiB/s wr, 24 op/s 2024-04-24T23:31:58.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:31:58 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/1349285704' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:31:58.842 INFO:tasks.fwd_scrub.fs.[cephfs]:end scrubbing fs: cephfs 2024-04-24T23:31:58.843 INFO:tasks.fwd_scrub:done joining 2024-04-24T23:31:58.843 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-04-24T23:31:58.855 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:31:58.969 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:58.965+0000 7f5a5f438640 1 -- 172.21.15.80:0/601464687 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5a58100690 msgr2=0x7f5a58100ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:58.969 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:58.965+0000 7f5a5f438640 1 --2- 172.21.15.80:0/601464687 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5a58100690 0x7f5a58100ab0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f5a54009900 tx=0x7f5a5402a410 comp rx=0 tx=0).stop 2024-04-24T23:31:58.969 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:58.965+0000 7f5a5f438640 1 -- 172.21.15.80:0/601464687 shutdown_connections 2024-04-24T23:31:58.969 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:58.965+0000 7f5a5f438640 1 --2- 172.21.15.80:0/601464687 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a58102240 0x7f5a58108ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:58.970 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:58.965+0000 7f5a5f438640 1 --2- 172.21.15.80:0/601464687 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a58101880 0x7f5a58101d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:58.970 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:58.965+0000 7f5a5f438640 1 --2- 172.21.15.80:0/601464687 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5a58100690 0x7f5a58100ab0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:58.970 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:58.965+0000 7f5a5f438640 1 -- 172.21.15.80:0/601464687 >> 172.21.15.80:0/601464687 conn(0x7f5a580fbc20 msgr2=0x7f5a580fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:58.970 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:58.965+0000 7f5a5f438640 1 -- 172.21.15.80:0/601464687 shutdown_connections 2024-04-24T23:31:58.970 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:58.965+0000 7f5a5f438640 1 -- 172.21.15.80:0/601464687 wait complete. 2024-04-24T23:31:58.970 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:58.966+0000 7f5a5f438640 1 Processor -- start 2024-04-24T23:31:58.970 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:58.966+0000 7f5a5f438640 1 -- start start 2024-04-24T23:31:58.970 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:58.966+0000 7f5a5f438640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5a58100690 0x7f5a58197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:58.971 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:58.966+0000 7f5a5f438640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a58101880 0x7f5a58198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:58.971 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:58.966+0000 7f5a5f438640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a58102240 0x7f5a5819f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:58.971 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:58.966+0000 7f5a5f438640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f5a5810d490 con 0x7f5a58100690 2024-04-24T23:31:58.971 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:58.966+0000 7f5a5f438640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5a5810d310 con 0x7f5a58101880 2024-04-24T23:31:58.971 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:58.966+0000 7f5a5f438640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5a5810d610 con 0x7f5a58102240 2024-04-24T23:31:58.971 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:58.966+0000 7f5a5d1d4640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5a58100690 0x7f5a58197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:58.971 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:58.966+0000 7f5a5d1d4640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5a58100690 0x7f5a58197b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53666/0 (socket says 172.21.15.80:53666) 2024-04-24T23:31:58.972 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:58.966+0000 7f5a5d1d4640 1 -- 172.21.15.80:0/163075360 learned_addr learned my addr 172.21.15.80:0/163075360 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:31:58.972 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:58.966+0000 7f5a5d9d5640 1 --2- 172.21.15.80:0/163075360 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a58102240 0x7f5a5819f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:58.972 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:58.966+0000 7f5a5c9d3640 1 --2- 172.21.15.80:0/163075360 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a58101880 0x7f5a58198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:58.972 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:58.967+0000 7f5a5d1d4640 1 -- 172.21.15.80:0/163075360 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a58102240 msgr2=0x7f5a5819f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:58.972 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:58.967+0000 7f5a5d1d4640 1 --2- 172.21.15.80:0/163075360 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a58102240 0x7f5a5819f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:58.972 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:58.967+0000 7f5a5d1d4640 1 -- 172.21.15.80:0/163075360 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a58101880 msgr2=0x7f5a58198080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:58.972 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:58.967+0000 7f5a5d1d4640 1 --2- 172.21.15.80:0/163075360 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a58101880 0x7f5a58198080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:58.972 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:58.967+0000 7f5a5d1d4640 1 -- 172.21.15.80:0/163075360 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a5819f760 con 0x7f5a58100690 2024-04-24T23:31:58.973 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:58.967+0000 7f5a5d1d4640 1 --2- 172.21.15.80:0/163075360 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5a58100690 0x7f5a58197b40 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f5a54031610 tx=0x7f5a540043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:58.973 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:58.967+0000 7f5a4e7fc640 1 -- 172.21.15.80:0/163075360 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a5403f030 con 0x7f5a58100690 2024-04-24T23:31:58.973 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:58.967+0000 7f5a4e7fc640 1 -- 172.21.15.80:0/163075360 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a54003ce0 con 0x7f5a58100690 2024-04-24T23:31:58.973 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:58.967+0000 7f5a5f438640 1 -- 172.21.15.80:0/163075360 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a5819f9f0 con 0x7f5a58100690 2024-04-24T23:31:58.973 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:58.967+0000 7f5a4e7fc640 1 -- 172.21.15.80:0/163075360 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a54035070 con 0x7f5a58100690 2024-04-24T23:31:58.974 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:58.967+0000 7f5a5f438640 1 -- 172.21.15.80:0/163075360 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a5819ff60 con 0x7f5a58100690 2024-04-24T23:31:58.974 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:58.968+0000 7f5a5f438640 1 -- 172.21.15.80:0/163075360 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a58107a30 con 0x7f5a58100690 2024-04-24T23:31:58.975 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:58.969+0000 7f5a4e7fc640 1 -- 172.21.15.80:0/163075360 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5a54042030 con 0x7f5a58100690 2024-04-24T23:31:58.975 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:58.969+0000 7f5a4e7fc640 1 --2- 172.21.15.80:0/163075360 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f5a440a2b90 0x7f5a440a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:58.975 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:58.970+0000 7f5a5c9d3640 1 --2- 172.21.15.80:0/163075360 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f5a440a2b90 0x7f5a440a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:58.975 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:58.970+0000 7f5a4e7fc640 1 -- 172.21.15.80:0/163075360 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f5a540efbf0 con 0x7f5a58100690 2024-04-24T23:31:58.975 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:58.970+0000 7f5a5c9d3640 1 --2- 172.21.15.80:0/163075360 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f5a440a2b90 0x7f5a440a5050 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f5a48005fd0 tx=0x7f5a48005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:58.976 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:58.974+0000 7f5a4e7fc640 1 -- 172.21.15.80:0/163075360 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f5a540bc780 con 0x7f5a58100690 2024-04-24T23:31:59.153 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.152+0000 7f5a5f438640 1 -- 172.21.15.80:0/163075360 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5a5802d340 con 0x7f5a58100690 2024-04-24T23:31:59.154 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.154+0000 7f5a4e7fc640 1 -- 172.21.15.80:0/163075360 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24839 (secure 0 0 0) 0x7f5a540bc080 con 0x7f5a58100690 2024-04-24T23:31:59.155 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-24T23:31:59.155 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":94,"fsid":"ec7885aa-0290-11ef-bc93-c7b262605968","created":"2024-04-24T23:19:05.586407+0000","modified":"2024-04-24T23:29:08.209684+0000","last_up_change":"2024-04-24T23:26:25.938316+0000","last_in_change":"2024-04-24T23:26:04.786699+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:22:33.124428+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:27:17.575788+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:27:18.492762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T23:27:22.438503+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a525741d-b46c-4eef-aa64-3f057cac1497","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":2597718112},{"type":"v1","addr":"172.21.15.80:6803","nonce":2597718112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":2597718112},{"type":"v1","addr":"172.21.15.80:6805","nonce":2597718112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2597718112},{"type":"v1","addr":"172.21.15.80:6809","nonce":2597718112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":2597718112},{"type":"v1","addr":"172.21.15.80:6807","nonce":2597718112}]},"public_addr":"172.21.15.80:6803/2597718112","cluster_addr":"172.21.15.80:6805/2597718112","heartbeat_back_addr":"172.21.15.80:6809/2597718112","heartbeat_front_addr":"172.21.15.80:6807/2597718112","state":["exists","up"]},{"osd":1,"uuid":"2d0f0d67-6db4-40ac-91cc-58604ab6795a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":2395610201},{"type":"v1","addr":"172.21.15.134:6801","nonce":2395610201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":2395610201},{"type":"v1","addr":"172.21.15.134:6803","nonce":2395610201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":2395610201},{"type":"v1","addr":"172.21.15.134:6807","nonce":2395610201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":2395610201},{"type":"v1","addr":"172.21.15.134:6805","nonce":2395610201}]},"public_addr":"172.21.15.134:6801/2395610201","cluster_addr":"172.21.15.134:6803/2395610201","heartbeat_back_addr":"172.21.15.134:6807/2395610201","heartbeat_front_addr":"172.21.15.134:6805/2395610201","state":["exists","up"]},{"osd":2,"uuid":"781fb879-fedc-4c84-82fb-59e073bc4cf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1629152187},{"type":"v1","addr":"172.21.15.148:6801","nonce":1629152187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1629152187},{"type":"v1","addr":"172.21.15.148:6803","nonce":1629152187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1629152187},{"type":"v1","addr":"172.21.15.148:6807","nonce":1629152187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1629152187},{"type":"v1","addr":"172.21.15.148:6805","nonce":1629152187}]},"public_addr":"172.21.15.148:6801/1629152187","cluster_addr":"172.21.15.148:6803/1629152187","heartbeat_back_addr":"172.21.15.148:6807/1629152187","heartbeat_front_addr":"172.21.15.148:6805/1629152187","state":["exists","up"]},{"osd":3,"uuid":"cd982757-f2f6-4296-b3e7-4cd40fbd8d9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":3128298510},{"type":"v1","addr":"172.21.15.80:6811","nonce":3128298510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":3128298510},{"type":"v1","addr":"172.21.15.80:6813","nonce":3128298510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":3128298510},{"type":"v1","addr":"172.21.15.80:6817","nonce":3128298510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":3128298510},{"type":"v1","addr":"172.21.15.80:6815","nonce":3128298510}]},"public_addr":"172.21.15.80:6811/3128298510","cluster_addr":"172.21.15.80:6813/3128298510","heartbeat_back_addr":"172.21.15.80:6817/3128298510","heartbeat_front_addr":"172.21.15.80:6815/3128298510","state":["exists","up"]},{"osd":4,"uuid":"dfc55a79-a7e8-417a-b898-ec44171a54af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1917556060},{"type":"v1","addr":"172.21.15.134:6809","nonce":1917556060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1917556060},{"type":"v1","addr":"172.21.15.134:6811","nonce":1917556060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1917556060},{"type":"v1","addr":"172.21.15.134:6815","nonce":1917556060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1917556060},{"type":"v1","addr":"172.21.15.134:6813","nonce":1917556060}]},"public_addr":"172.21.15.134:6809/1917556060","cluster_addr":"172.21.15.134:6811/1917556060","heartbeat_back_addr":"172.21.15.134:6815/1917556060","heartbeat_front_addr":"172.21.15.134:6813/1917556060","state":["exists","up"]},{"osd":5,"uuid":"b5491337-0694-440c-b679-312f3d14cf65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":2715951281},{"type":"v1","addr":"172.21.15.148:6809","nonce":2715951281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":2715951281},{"type":"v1","addr":"172.21.15.148:6811","nonce":2715951281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":2715951281},{"type":"v1","addr":"172.21.15.148:6815","nonce":2715951281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":2715951281},{"type":"v1","addr":"172.21.15.148:6813","nonce":2715951281}]},"public_addr":"172.21.15.148:6809/2715951281","cluster_addr":"172.21.15.148:6811/2715951281","heartbeat_back_addr":"172.21.15.148:6815/2715951281","heartbeat_front_addr":"172.21.15.148:6813/2715951281","state":["exists","up"]},{"osd":6,"uuid":"ba4b5df7-d86e-47cd-ab67-cf763c4e4d6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":953261724},{"type":"v1","addr":"172.21.15.80:6819","nonce":953261724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":953261724},{"type":"v1","addr":"172.21.15.80:6821","nonce":953261724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":953261724},{"type":"v1","addr":"172.21.15.80:6825","nonce":953261724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":953261724},{"type":"v1","addr":"172.21.15.80:6823","nonce":953261724}]},"public_addr":"172.21.15.80:6819/953261724","cluster_addr":"172.21.15.80:6821/953261724","heartbeat_back_addr":"172.21.15.80:6825/953261724","heartbeat_front_addr":"172.21.15.80:6823/953261724","state":["exists","up"]},{"osd":7,"uuid":"464a503a-75ae-49e7-ab6e-a7f9f2a6a307","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":108354844},{"type":"v1","addr":"172.21.15.134:6817","nonce":108354844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":108354844},{"type":"v1","addr":"172.21.15.134:6819","nonce":108354844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":108354844},{"type":"v1","addr":"172.21.15.134:6823","nonce":108354844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":108354844},{"type":"v1","addr":"172.21.15.134:6821","nonce":108354844}]},"public_addr":"172.21.15.134:6817/108354844","cluster_addr":"172.21.15.134:6819/108354844","heartbeat_back_addr":"172.21.15.134:6823/108354844","heartbeat_front_addr":"172.21.15.134:6821/108354844","state":["exists","up"]},{"osd":8,"uuid":"ce544b2e-4540-47c0-9997-500c45351330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3559655329},{"type":"v1","addr":"172.21.15.148:6817","nonce":3559655329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":3559655329},{"type":"v1","addr":"172.21.15.148:6819","nonce":3559655329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":3559655329},{"type":"v1","addr":"172.21.15.148:6823","nonce":3559655329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3559655329},{"type":"v1","addr":"172.21.15.148:6821","nonce":3559655329}]},"public_addr":"172.21.15.148:6817/3559655329","cluster_addr":"172.21.15.148:6819/3559655329","heartbeat_back_addr":"172.21.15.148:6823/3559655329","heartbeat_front_addr":"172.21.15.148:6821/3559655329","state":["exists","up"]},{"osd":9,"uuid":"06de3732-19b9-496c-a464-55322efa6759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":4019913545},{"type":"v1","addr":"172.21.15.80:6827","nonce":4019913545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":4019913545},{"type":"v1","addr":"172.21.15.80:6829","nonce":4019913545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":4019913545},{"type":"v1","addr":"172.21.15.80:6833","nonce":4019913545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":4019913545},{"type":"v1","addr":"172.21.15.80:6831","nonce":4019913545}]},"public_addr":"172.21.15.80:6827/4019913545","cluster_addr":"172.21.15.80:6829/4019913545","heartbeat_back_addr":"172.21.15.80:6833/4019913545","heartbeat_front_addr":"172.21.15.80:6831/4019913545","state":["exists","up"]},{"osd":10,"uuid":"8d92f0d9-5c64-429a-b750-c33ae0e070c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":647343766},{"type":"v1","addr":"172.21.15.134:6825","nonce":647343766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":647343766},{"type":"v1","addr":"172.21.15.134:6827","nonce":647343766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":647343766},{"type":"v1","addr":"172.21.15.134:6831","nonce":647343766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":647343766},{"type":"v1","addr":"172.21.15.134:6829","nonce":647343766}]},"public_addr":"172.21.15.134:6825/647343766","cluster_addr":"172.21.15.134:6827/647343766","heartbeat_back_addr":"172.21.15.134:6831/647343766","heartbeat_front_addr":"172.21.15.134:6829/647343766","state":["exists","up"]},{"osd":11,"uuid":"4c003a41-f65b-479f-bc80-b27d680cf936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":4068024116},{"type":"v1","addr":"172.21.15.148:6825","nonce":4068024116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":4068024116},{"type":"v1","addr":"172.21.15.148:6827","nonce":4068024116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":4068024116},{"type":"v1","addr":"172.21.15.148:6831","nonce":4068024116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":4068024116},{"type":"v1","addr":"172.21.15.148:6829","nonce":4068024116}]},"public_addr":"172.21.15.148:6825/4068024116","cluster_addr":"172.21.15.148:6827/4068024116","heartbeat_back_addr":"172.21.15.148:6831/4068024116","heartbeat_front_addr":"172.21.15.148:6829/4068024116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:21:42.433631+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:22:06.145933+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:22:30.726143+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:22:55.729961+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:23:21.280913+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:23:46.609983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:24:12.585247+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:24:38.359287+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:25:04.491232+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:25:30.788089+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:25:56.891078+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:26:23.899113+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":7}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"4.a","mappings":[{"from":5,"to":8}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.80:0/2828378749":"2024-04-25T23:20:06.793609+0000","172.21.15.80:6801/233364790":"2024-04-25T23:20:06.793609+0000","172.21.15.80:6800/1614691410":"2024-04-25T23:19:40.490692+0000","172.21.15.80:0/1406187145":"2024-04-25T23:20:06.793609+0000","172.21.15.80:0/92953396":"2024-04-25T23:19:40.490692+0000","172.21.15.80:0/598129432":"2024-04-25T23:20:06.793609+0000","172.21.15.80:0/232193118":"2024-04-25T23:19:40.490692+0000","172.21.15.80:6800/233364790":"2024-04-25T23:20:06.793609+0000","172.21.15.80:6801/1614691410":"2024-04-25T23:19:40.490692+0000","172.21.15.80:0/2081562529":"2024-04-25T23:19:40.490692+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:31:59.157 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.156+0000 7f5a5f438640 1 -- 172.21.15.80:0/163075360 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f5a440a2b90 msgr2=0x7f5a440a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:59.157 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.156+0000 7f5a5f438640 1 --2- 172.21.15.80:0/163075360 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f5a440a2b90 0x7f5a440a5050 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f5a48005fd0 tx=0x7f5a48005850 comp rx=0 tx=0).stop 2024-04-24T23:31:59.157 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.156+0000 7f5a5f438640 1 -- 172.21.15.80:0/163075360 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5a58100690 msgr2=0x7f5a58197b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:59.157 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.156+0000 7f5a5f438640 1 --2- 172.21.15.80:0/163075360 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5a58100690 0x7f5a58197b40 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f5a54031610 tx=0x7f5a540043b0 comp rx=0 tx=0).stop 2024-04-24T23:31:59.157 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.156+0000 7f5a5f438640 1 -- 172.21.15.80:0/163075360 shutdown_connections 2024-04-24T23:31:59.157 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.156+0000 7f5a5f438640 1 --2- 172.21.15.80:0/163075360 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f5a440a2b90 0x7f5a440a5050 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:59.157 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.156+0000 7f5a5f438640 1 --2- 172.21.15.80:0/163075360 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a58102240 0x7f5a5819f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:59.158 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.156+0000 7f5a5f438640 1 --2- 172.21.15.80:0/163075360 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a58101880 0x7f5a58198080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:59.158 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.156+0000 7f5a5f438640 1 --2- 172.21.15.80:0/163075360 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5a58100690 0x7f5a58197b40 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:59.158 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.156+0000 7f5a5f438640 1 -- 172.21.15.80:0/163075360 >> 172.21.15.80:0/163075360 conn(0x7f5a580fbc20 msgr2=0x7f5a58104c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:59.158 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.156+0000 7f5a5f438640 1 -- 172.21.15.80:0/163075360 shutdown_connections 2024-04-24T23:31:59.158 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.156+0000 7f5a5f438640 1 -- 172.21.15.80:0/163075360 wait complete. 2024-04-24T23:31:59.168 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:22:33.124428+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T23:27:17.575788+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T23:27:18.492762+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-24T23:27:22.438503+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-24T23:31:59.169 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:31:59.272 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.269+0000 7f21e38dd640 1 -- 172.21.15.80:0/716755105 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f21dc102200 msgr2=0x7f21dc1045f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:59.273 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.269+0000 7f21e38dd640 1 --2- 172.21.15.80:0/716755105 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f21dc102200 0x7f21dc1045f0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f21d8009ca0 tx=0x7f21d802a8c0 comp rx=0 tx=0).stop 2024-04-24T23:31:59.273 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.269+0000 7f21e38dd640 1 -- 172.21.15.80:0/716755105 shutdown_connections 2024-04-24T23:31:59.273 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.269+0000 7f21e38dd640 1 --2- 172.21.15.80:0/716755105 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f21dc107460 0x7f21dc1098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:59.273 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.269+0000 7f21e38dd640 1 --2- 172.21.15.80:0/716755105 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f21dc104b30 0x7f21dc106f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:59.273 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.269+0000 7f21e38dd640 1 --2- 172.21.15.80:0/716755105 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f21dc102200 0x7f21dc1045f0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:59.273 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.269+0000 7f21e38dd640 1 -- 172.21.15.80:0/716755105 >> 172.21.15.80:0/716755105 conn(0x7f21dc0fbbf0 msgr2=0x7f21dc0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:59.273 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.270+0000 7f21e38dd640 1 -- 172.21.15.80:0/716755105 shutdown_connections 2024-04-24T23:31:59.274 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.270+0000 7f21e38dd640 1 -- 172.21.15.80:0/716755105 wait complete. 2024-04-24T23:31:59.274 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.270+0000 7f21e38dd640 1 Processor -- start 2024-04-24T23:31:59.274 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.270+0000 7f21e38dd640 1 -- start start 2024-04-24T23:31:59.274 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.270+0000 7f21e38dd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f21dc102200 0x7f21dc101b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:59.274 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.270+0000 7f21e38dd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f21dc104b30 0x7f21dc1001b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:59.274 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.270+0000 7f21e38dd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f21dc107460 0x7f21dc1006f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:59.274 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.270+0000 7f21e38dd640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f21dc10d400 con 0x7f21dc102200 2024-04-24T23:31:59.275 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.270+0000 7f21e38dd640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f21dc10d280 con 0x7f21dc104b30 2024-04-24T23:31:59.275 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.270+0000 7f21e38dd640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f21dc10d580 con 0x7f21dc107460 2024-04-24T23:31:59.275 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.270+0000 7f21e1679640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f21dc102200 0x7f21dc101b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:59.275 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.270+0000 7f21e1679640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f21dc102200 0x7f21dc101b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53684/0 (socket says 172.21.15.80:53684) 2024-04-24T23:31:59.275 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.270+0000 7f21e1679640 1 -- 172.21.15.80:0/628900366 learned_addr learned my addr 172.21.15.80:0/628900366 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:31:59.275 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.270+0000 7f21e1e7a640 1 --2- 172.21.15.80:0/628900366 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f21dc107460 0x7f21dc1006f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:59.276 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.271+0000 7f21e0e78640 1 --2- 172.21.15.80:0/628900366 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f21dc104b30 0x7f21dc1001b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:59.276 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.271+0000 7f21e1679640 1 -- 172.21.15.80:0/628900366 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f21dc107460 msgr2=0x7f21dc1006f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:59.276 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.271+0000 7f21e1679640 1 --2- 172.21.15.80:0/628900366 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f21dc107460 0x7f21dc1006f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:59.276 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.271+0000 7f21e1679640 1 -- 172.21.15.80:0/628900366 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f21dc104b30 msgr2=0x7f21dc1001b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:59.276 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.271+0000 7f21e1679640 1 --2- 172.21.15.80:0/628900366 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f21dc104b30 0x7f21dc1001b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:59.276 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.271+0000 7f21e1679640 1 -- 172.21.15.80:0/628900366 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21dc100f80 con 0x7f21dc102200 2024-04-24T23:31:59.276 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.271+0000 7f21e1679640 1 --2- 172.21.15.80:0/628900366 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f21dc102200 0x7f21dc101b70 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f21d8031a40 tx=0x7f21d802f840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:59.277 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.271+0000 7f21d27fc640 1 -- 172.21.15.80:0/628900366 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21d802fdc0 con 0x7f21dc102200 2024-04-24T23:31:59.277 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.271+0000 7f21d27fc640 1 -- 172.21.15.80:0/628900366 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21d8030aa0 con 0x7f21dc102200 2024-04-24T23:31:59.277 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.271+0000 7f21e38dd640 1 -- 172.21.15.80:0/628900366 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21dc1a1bd0 con 0x7f21dc102200 2024-04-24T23:31:59.277 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.271+0000 7f21d27fc640 1 -- 172.21.15.80:0/628900366 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21d8046a10 con 0x7f21dc102200 2024-04-24T23:31:59.277 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.271+0000 7f21e38dd640 1 -- 172.21.15.80:0/628900366 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21dc1a2120 con 0x7f21dc102200 2024-04-24T23:31:59.277 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.272+0000 7f21e38dd640 1 -- 172.21.15.80:0/628900366 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21dc0fd6e0 con 0x7f21dc102200 2024-04-24T23:31:59.279 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.273+0000 7f21d27fc640 1 -- 172.21.15.80:0/628900366 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f21d8030c50 con 0x7f21dc102200 2024-04-24T23:31:59.279 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.274+0000 7f21d27fc640 1 --2- 172.21.15.80:0/628900366 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f21c80a2b90 0x7f21c80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:59.279 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.274+0000 7f21e0e78640 1 --2- 172.21.15.80:0/628900366 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f21c80a2b90 0x7f21c80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:59.279 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.274+0000 7f21d27fc640 1 -- 172.21.15.80:0/628900366 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f21d80f0db0 con 0x7f21dc102200 2024-04-24T23:31:59.279 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.274+0000 7f21e0e78640 1 --2- 172.21.15.80:0/628900366 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f21c80a2b90 0x7f21c80a5050 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f21dc1018e0 tx=0x7f21cc00a4c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:59.280 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.278+0000 7f21d27fc640 1 -- 172.21.15.80:0/628900366 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f21d80bd940 con 0x7f21dc102200 2024-04-24T23:31:59.451 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.450+0000 7f21e38dd640 1 -- 172.21.15.80:0/628900366 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f21dc062340 con 0x7f21dc102200 2024-04-24T23:31:59.451 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.451+0000 7f21d27fc640 1 -- 172.21.15.80:0/628900366 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7f21d80bd240 con 0x7f21dc102200 2024-04-24T23:31:59.452 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 1 2024-04-24T23:31:59.453 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.453+0000 7f21e38dd640 1 -- 172.21.15.80:0/628900366 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f21c80a2b90 msgr2=0x7f21c80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:59.453 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.453+0000 7f21e38dd640 1 --2- 172.21.15.80:0/628900366 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f21c80a2b90 0x7f21c80a5050 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f21dc1018e0 tx=0x7f21cc00a4c0 comp rx=0 tx=0).stop 2024-04-24T23:31:59.454 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.453+0000 7f21e38dd640 1 -- 172.21.15.80:0/628900366 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f21dc102200 msgr2=0x7f21dc101b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:59.454 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.453+0000 7f21e38dd640 1 --2- 172.21.15.80:0/628900366 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f21dc102200 0x7f21dc101b70 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f21d8031a40 tx=0x7f21d802f840 comp rx=0 tx=0).stop 2024-04-24T23:31:59.454 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.453+0000 7f21e38dd640 1 -- 172.21.15.80:0/628900366 shutdown_connections 2024-04-24T23:31:59.454 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.453+0000 7f21e38dd640 1 --2- 172.21.15.80:0/628900366 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f21c80a2b90 0x7f21c80a5050 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:59.454 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.453+0000 7f21e38dd640 1 --2- 172.21.15.80:0/628900366 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f21dc107460 0x7f21dc1006f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:59.454 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.453+0000 7f21e38dd640 1 --2- 172.21.15.80:0/628900366 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f21dc104b30 0x7f21dc1001b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:59.454 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.453+0000 7f21e38dd640 1 --2- 172.21.15.80:0/628900366 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f21dc102200 0x7f21dc101b70 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:59.455 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.453+0000 7f21e38dd640 1 -- 172.21.15.80:0/628900366 >> 172.21.15.80:0/628900366 conn(0x7f21dc0fbbf0 msgr2=0x7f21dc102b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:59.455 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.453+0000 7f21e38dd640 1 -- 172.21.15.80:0/628900366 shutdown_connections 2024-04-24T23:31:59.455 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.453+0000 7f21e38dd640 1 -- 172.21.15.80:0/628900366 wait complete. 2024-04-24T23:31:59.464 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T23:31:59.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.565+0000 7f865d04c640 1 -- 172.21.15.80:0/776859369 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8658102250 msgr2=0x7f8658108ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:59.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.565+0000 7f865d04c640 1 --2- 172.21.15.80:0/776859369 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8658102250 0x7f8658108ae0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f8650009ca0 tx=0x7f865002a8c0 comp rx=0 tx=0).stop 2024-04-24T23:31:59.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.565+0000 7f865d04c640 1 -- 172.21.15.80:0/776859369 shutdown_connections 2024-04-24T23:31:59.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.565+0000 7f865d04c640 1 --2- 172.21.15.80:0/776859369 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8658102250 0x7f8658108ae0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:59.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.565+0000 7f865d04c640 1 --2- 172.21.15.80:0/776859369 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8658101890 0x7f8658101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:59.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.565+0000 7f865d04c640 1 --2- 172.21.15.80:0/776859369 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86581006a0 0x7f8658100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:59.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.565+0000 7f865d04c640 1 -- 172.21.15.80:0/776859369 >> 172.21.15.80:0/776859369 conn(0x7f86580fbc30 msgr2=0x7f86580fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:59.570 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.565+0000 7f865d04c640 1 -- 172.21.15.80:0/776859369 shutdown_connections 2024-04-24T23:31:59.570 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.565+0000 7f865d04c640 1 -- 172.21.15.80:0/776859369 wait complete. 2024-04-24T23:31:59.570 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.565+0000 7f865d04c640 1 Processor -- start 2024-04-24T23:31:59.570 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.566+0000 7f865d04c640 1 -- start start 2024-04-24T23:31:59.570 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.566+0000 7f865d04c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86581006a0 0x7f865810b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:59.570 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.566+0000 7f865d04c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8658101890 0x7f8658109660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:59.570 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.566+0000 7f865d04c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8658102250 0x7f8658109ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:59.570 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.566+0000 7f865d04c640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f865810d480 con 0x7f8658102250 2024-04-24T23:31:59.571 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.566+0000 7f865d04c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f865810d300 con 0x7f86581006a0 2024-04-24T23:31:59.571 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.566+0000 7f865d04c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f865810d600 con 0x7f8658101890 2024-04-24T23:31:59.571 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.566+0000 7f865759e640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8658102250 0x7f8658109ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:59.571 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.566+0000 7f865759e640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8658102250 0x7f8658109ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53698/0 (socket says 172.21.15.80:53698) 2024-04-24T23:31:59.571 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.566+0000 7f865759e640 1 -- 172.21.15.80:0/2175459580 learned_addr learned my addr 172.21.15.80:0/2175459580 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:31:59.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.566+0000 7f865659c640 1 --2- 172.21.15.80:0/2175459580 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8658101890 0x7f8658109660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:59.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.566+0000 7f8656d9d640 1 --2- 172.21.15.80:0/2175459580 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86581006a0 0x7f865810b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:59.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.566+0000 7f865759e640 1 -- 172.21.15.80:0/2175459580 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8658101890 msgr2=0x7f8658109660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:59.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.566+0000 7f865759e640 1 --2- 172.21.15.80:0/2175459580 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8658101890 0x7f8658109660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:59.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.566+0000 7f865759e640 1 -- 172.21.15.80:0/2175459580 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86581006a0 msgr2=0x7f865810b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:59.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.566+0000 7f865759e640 1 --2- 172.21.15.80:0/2175459580 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86581006a0 0x7f865810b020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:59.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.566+0000 7f865759e640 1 -- 172.21.15.80:0/2175459580 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f865810a430 con 0x7f8658102250 2024-04-24T23:31:59.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.566+0000 7f865659c640 1 --2- 172.21.15.80:0/2175459580 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8658101890 0x7f8658109660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:31:59.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.566+0000 7f865759e640 1 --2- 172.21.15.80:0/2175459580 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8658102250 0x7f8658109ba0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f8650005e30 tx=0x7f8650031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:59.573 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.567+0000 7f863ffff640 1 -- 172.21.15.80:0/2175459580 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8650003c30 con 0x7f8658102250 2024-04-24T23:31:59.573 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.567+0000 7f865d04c640 1 -- 172.21.15.80:0/2175459580 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86581a1c10 con 0x7f8658102250 2024-04-24T23:31:59.573 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.567+0000 7f863ffff640 1 -- 172.21.15.80:0/2175459580 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8650003dd0 con 0x7f8658102250 2024-04-24T23:31:59.573 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.567+0000 7f863ffff640 1 -- 172.21.15.80:0/2175459580 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f865002fde0 con 0x7f8658102250 2024-04-24T23:31:59.573 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.567+0000 7f865d04c640 1 -- 172.21.15.80:0/2175459580 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86581a21b0 con 0x7f8658102250 2024-04-24T23:31:59.575 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.568+0000 7f865d04c640 1 -- 172.21.15.80:0/2175459580 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86581079c0 con 0x7f8658102250 2024-04-24T23:31:59.575 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.569+0000 7f863ffff640 1 -- 172.21.15.80:0/2175459580 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8650004050 con 0x7f8658102250 2024-04-24T23:31:59.575 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.569+0000 7f863ffff640 1 --2- 172.21.15.80:0/2175459580 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f86400a2b90 0x7f86400a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:59.575 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.569+0000 7f863ffff640 1 -- 172.21.15.80:0/2175459580 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f86500f09d0 con 0x7f8658102250 2024-04-24T23:31:59.575 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.569+0000 7f8656d9d640 1 --2- 172.21.15.80:0/2175459580 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f86400a2b90 0x7f86400a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:59.575 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.570+0000 7f8656d9d640 1 --2- 172.21.15.80:0/2175459580 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f86400a2b90 0x7f86400a5050 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f8648005fd0 tx=0x7f8648005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:59.575 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.574+0000 7f863ffff640 1 -- 172.21.15.80:0/2175459580 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f86500bd680 con 0x7f8658102250 2024-04-24T23:31:59.749 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.748+0000 7f865d04c640 1 -- 172.21.15.80:0/2175459580 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f865802d170 con 0x7f8658102250 2024-04-24T23:31:59.749 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.748+0000 7f863ffff640 1 -- 172.21.15.80:0/2175459580 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) ==== 104+0+11 (secure 0 0 0) 0x7f86500bcf80 con 0x7f8658102250 2024-04-24T23:31:59.749 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 32 2024-04-24T23:31:59.751 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.750+0000 7f865d04c640 1 -- 172.21.15.80:0/2175459580 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f86400a2b90 msgr2=0x7f86400a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:59.751 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.750+0000 7f865d04c640 1 --2- 172.21.15.80:0/2175459580 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f86400a2b90 0x7f86400a5050 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f8648005fd0 tx=0x7f8648005850 comp rx=0 tx=0).stop 2024-04-24T23:31:59.751 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.750+0000 7f865d04c640 1 -- 172.21.15.80:0/2175459580 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8658102250 msgr2=0x7f8658109ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:59.751 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.750+0000 7f865d04c640 1 --2- 172.21.15.80:0/2175459580 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8658102250 0x7f8658109ba0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f8650005e30 tx=0x7f8650031ee0 comp rx=0 tx=0).stop 2024-04-24T23:31:59.752 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.750+0000 7f865d04c640 1 -- 172.21.15.80:0/2175459580 shutdown_connections 2024-04-24T23:31:59.752 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.750+0000 7f865d04c640 1 --2- 172.21.15.80:0/2175459580 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f86400a2b90 0x7f86400a5050 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:59.752 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.750+0000 7f865d04c640 1 --2- 172.21.15.80:0/2175459580 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8658102250 0x7f8658109ba0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:59.752 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.750+0000 7f865d04c640 1 --2- 172.21.15.80:0/2175459580 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8658101890 0x7f8658109660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:59.752 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.750+0000 7f865d04c640 1 --2- 172.21.15.80:0/2175459580 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86581006a0 0x7f865810b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:59.752 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.750+0000 7f865d04c640 1 -- 172.21.15.80:0/2175459580 >> 172.21.15.80:0/2175459580 conn(0x7f86580fbc30 msgr2=0x7f8658104b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:59.752 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.750+0000 7f865d04c640 1 -- 172.21.15.80:0/2175459580 shutdown_connections 2024-04-24T23:31:59.752 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.750+0000 7f865d04c640 1 -- 172.21.15.80:0/2175459580 wait complete. 2024-04-24T23:31:59.762 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T23:31:59.866 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.863+0000 7f0083c38640 1 -- 172.21.15.80:0/419828111 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f007c102980 msgr2=0x7f007c102e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:59.866 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.863+0000 7f0083c38640 1 --2- 172.21.15.80:0/419828111 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f007c102980 0x7f007c102e00 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f006c009900 tx=0x7f006c02a410 comp rx=0 tx=0).stop 2024-04-24T23:31:59.866 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.863+0000 7f0083c38640 1 -- 172.21.15.80:0/419828111 shutdown_connections 2024-04-24T23:31:59.866 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.863+0000 7f0083c38640 1 --2- 172.21.15.80:0/419828111 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f007c103340 0x7f007c109bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:59.866 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.863+0000 7f0083c38640 1 --2- 172.21.15.80:0/419828111 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f007c102980 0x7f007c102e00 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:59.867 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.863+0000 7f0083c38640 1 --2- 172.21.15.80:0/419828111 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f007c101790 0x7f007c101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:59.867 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.863+0000 7f0083c38640 1 -- 172.21.15.80:0/419828111 >> 172.21.15.80:0/419828111 conn(0x7f007c0fcd20 msgr2=0x7f007c0ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:31:59.867 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.863+0000 7f0083c38640 1 -- 172.21.15.80:0/419828111 shutdown_connections 2024-04-24T23:31:59.867 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.863+0000 7f0083c38640 1 -- 172.21.15.80:0/419828111 wait complete. 2024-04-24T23:31:59.867 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.864+0000 7f0083c38640 1 Processor -- start 2024-04-24T23:31:59.867 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.864+0000 7f0083c38640 1 -- start start 2024-04-24T23:31:59.867 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.864+0000 7f0083c38640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f007c101790 0x7f007c197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:59.867 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.864+0000 7f0083c38640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f007c102980 0x7f007c197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:59.867 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.864+0000 7f0083c38640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f007c103340 0x7f007c19f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:59.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.864+0000 7f0083c38640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f007c10d3f0 con 0x7f007c103340 2024-04-24T23:31:59.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.864+0000 7f0083c38640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f007c10d270 con 0x7f007c102980 2024-04-24T23:31:59.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.864+0000 7f0083c38640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f007c10d570 con 0x7f007c101790 2024-04-24T23:31:59.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.864+0000 7f00821d5640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f007c103340 0x7f007c19f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:59.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.864+0000 7f00821d5640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f007c103340 0x7f007c19f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53726/0 (socket says 172.21.15.80:53726) 2024-04-24T23:31:59.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.864+0000 7f00821d5640 1 -- 172.21.15.80:0/645663656 learned_addr learned my addr 172.21.15.80:0/645663656 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:31:59.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.864+0000 7f00819d4640 1 --2- 172.21.15.80:0/645663656 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f007c101790 0x7f007c197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:59.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.864+0000 7f00811d3640 1 --2- 172.21.15.80:0/645663656 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f007c102980 0x7f007c197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:59.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.864+0000 7f00821d5640 1 -- 172.21.15.80:0/645663656 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f007c101790 msgr2=0x7f007c197a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:59.869 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.864+0000 7f00821d5640 1 --2- 172.21.15.80:0/645663656 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f007c101790 0x7f007c197a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:59.869 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.864+0000 7f00821d5640 1 -- 172.21.15.80:0/645663656 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f007c102980 msgr2=0x7f007c197fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:31:59.869 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.864+0000 7f00821d5640 1 --2- 172.21.15.80:0/645663656 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f007c102980 0x7f007c197fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:31:59.869 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.864+0000 7f00821d5640 1 -- 172.21.15.80:0/645663656 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f007c19f680 con 0x7f007c103340 2024-04-24T23:31:59.869 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.864+0000 7f00811d3640 1 --2- 172.21.15.80:0/645663656 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f007c102980 0x7f007c197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:31:59.869 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.864+0000 7f00821d5640 1 --2- 172.21.15.80:0/645663656 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f007c103340 0x7f007c19f010 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f007400cce0 tx=0x7f007400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:59.869 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.865+0000 7f0072ffd640 1 -- 172.21.15.80:0/645663656 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f007400f450 con 0x7f007c103340 2024-04-24T23:31:59.869 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.865+0000 7f0072ffd640 1 -- 172.21.15.80:0/645663656 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f007400fad0 con 0x7f007c103340 2024-04-24T23:31:59.869 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.865+0000 7f0083c38640 1 -- 172.21.15.80:0/645663656 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f007c19f970 con 0x7f007c103340 2024-04-24T23:31:59.870 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.865+0000 7f0072ffd640 1 -- 172.21.15.80:0/645663656 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0074018c10 con 0x7f007c103340 2024-04-24T23:31:59.870 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.865+0000 7f0083c38640 1 -- 172.21.15.80:0/645663656 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f007c19fe40 con 0x7f007c103340 2024-04-24T23:31:59.870 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.866+0000 7f0083c38640 1 -- 172.21.15.80:0/645663656 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f007c108ab0 con 0x7f007c103340 2024-04-24T23:31:59.871 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.867+0000 7f0072ffd640 1 -- 172.21.15.80:0/645663656 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f007400f5f0 con 0x7f007c103340 2024-04-24T23:31:59.871 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.868+0000 7f0072ffd640 1 --2- 172.21.15.80:0/645663656 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f00680a2b90 0x7f00680a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:31:59.871 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.868+0000 7f0072ffd640 1 -- 172.21.15.80:0/645663656 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f00740cfb60 con 0x7f007c103340 2024-04-24T23:31:59.871 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.868+0000 7f00819d4640 1 --2- 172.21.15.80:0/645663656 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f00680a2b90 0x7f00680a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:31:59.871 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.868+0000 7f00819d4640 1 --2- 172.21.15.80:0/645663656 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f00680a2b90 0x7f00680a5050 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f007c1027e0 tx=0x7f007800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:31:59.872 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:31:59.871+0000 7f0072ffd640 1 -- 172.21.15.80:0/645663656 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f0074014030 con 0x7f007c103340 2024-04-24T23:32:00.053 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:00.052+0000 7f0083c38640 1 -- 172.21.15.80:0/645663656 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f007c02d170 con 0x7f007c103340 2024-04-24T23:32:00.053 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:00.052+0000 7f0072ffd640 1 -- 172.21.15.80:0/645663656 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) ==== 100+0+11 (secure 0 0 0) 0x7f007409c780 con 0x7f007c103340 2024-04-24T23:32:00.053 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 64 2024-04-24T23:32:00.055 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:00.054+0000 7f0083c38640 1 -- 172.21.15.80:0/645663656 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f00680a2b90 msgr2=0x7f00680a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:00.055 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:00.054+0000 7f0083c38640 1 --2- 172.21.15.80:0/645663656 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f00680a2b90 0x7f00680a5050 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f007c1027e0 tx=0x7f007800b390 comp rx=0 tx=0).stop 2024-04-24T23:32:00.055 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:00.054+0000 7f0083c38640 1 -- 172.21.15.80:0/645663656 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f007c103340 msgr2=0x7f007c19f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:00.055 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:00.054+0000 7f0083c38640 1 --2- 172.21.15.80:0/645663656 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f007c103340 0x7f007c19f010 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f007400cce0 tx=0x7f007400a620 comp rx=0 tx=0).stop 2024-04-24T23:32:00.055 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:00.054+0000 7f0083c38640 1 -- 172.21.15.80:0/645663656 shutdown_connections 2024-04-24T23:32:00.056 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:00.054+0000 7f0083c38640 1 --2- 172.21.15.80:0/645663656 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f00680a2b90 0x7f00680a5050 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:00.056 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:00.054+0000 7f0083c38640 1 --2- 172.21.15.80:0/645663656 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f007c103340 0x7f007c19f010 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:00.056 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:00.054+0000 7f0083c38640 1 --2- 172.21.15.80:0/645663656 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f007c102980 0x7f007c197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:00.056 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:00.054+0000 7f0083c38640 1 --2- 172.21.15.80:0/645663656 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f007c101790 0x7f007c197a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:00.056 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:00.054+0000 7f0083c38640 1 -- 172.21.15.80:0/645663656 >> 172.21.15.80:0/645663656 conn(0x7f007c0fcd20 msgr2=0x7f007c105c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:32:00.056 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:00.054+0000 7f0083c38640 1 -- 172.21.15.80:0/645663656 shutdown_connections 2024-04-24T23:32:00.056 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:00.054+0000 7f0083c38640 1 -- 172.21.15.80:0/645663656 wait complete. 2024-04-24T23:32:00.067 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-24T23:32:00.170 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:00.167+0000 7fda29a9e640 1 -- 172.21.15.80:0/1188885557 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fda24102230 msgr2=0x7fda24108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:00.170 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:00.167+0000 7fda29a9e640 1 --2- 172.21.15.80:0/1188885557 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fda24102230 0x7fda24108ac0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fda1c009ca0 tx=0x7fda1c02a8c0 comp rx=0 tx=0).stop 2024-04-24T23:32:00.171 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:00.167+0000 7fda29a9e640 1 -- 172.21.15.80:0/1188885557 shutdown_connections 2024-04-24T23:32:00.171 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:00.167+0000 7fda29a9e640 1 --2- 172.21.15.80:0/1188885557 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fda24102230 0x7fda24108ac0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:00.171 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:00.167+0000 7fda29a9e640 1 --2- 172.21.15.80:0/1188885557 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fda24101870 0x7fda24101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:00.171 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:00.167+0000 7fda29a9e640 1 --2- 172.21.15.80:0/1188885557 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda24100680 0x7fda24100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:00.171 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:00.167+0000 7fda29a9e640 1 -- 172.21.15.80:0/1188885557 >> 172.21.15.80:0/1188885557 conn(0x7fda240fbbf0 msgr2=0x7fda240fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:32:00.171 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:00.167+0000 7fda29a9e640 1 -- 172.21.15.80:0/1188885557 shutdown_connections 2024-04-24T23:32:00.171 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:00.167+0000 7fda29a9e640 1 -- 172.21.15.80:0/1188885557 wait complete. 2024-04-24T23:32:00.171 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:00.168+0000 7fda29a9e640 1 Processor -- start 2024-04-24T23:32:00.171 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:00.168+0000 7fda29a9e640 1 -- start start 2024-04-24T23:32:00.172 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:00.168+0000 7fda29a9e640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fda24100680 0x7fda24197970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:00.172 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:00.168+0000 7fda29a9e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda24101870 0x7fda24197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:00.172 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:00.168+0000 7fda29a9e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fda24102230 0x7fda2419ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:00.172 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:00.168+0000 7fda29a9e640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fda2410d420 con 0x7fda24100680 2024-04-24T23:32:00.172 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:00.168+0000 7fda29a9e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fda2410d2a0 con 0x7fda24102230 2024-04-24T23:32:00.172 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:00.168+0000 7fda29a9e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fda2410d5a0 con 0x7fda24101870 2024-04-24T23:32:00.172 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:00.168+0000 7fda237fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fda24100680 0x7fda24197970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:00.172 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:00.168+0000 7fda237fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fda24100680 0x7fda24197970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53754/0 (socket says 172.21.15.80:53754) 2024-04-24T23:32:00.173 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:00.168+0000 7fda237fe640 1 -- 172.21.15.80:0/1180168202 learned_addr learned my addr 172.21.15.80:0/1180168202 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:32:00.173 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:00.168+0000 7fda1affd640 1 --2- 172.21.15.80:0/1180168202 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda24101870 0x7fda24197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:00.173 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:00.168+0000 7fda23fff640 1 --2- 172.21.15.80:0/1180168202 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fda24102230 0x7fda2419ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:00.173 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:00.169+0000 7fda237fe640 1 -- 172.21.15.80:0/1180168202 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda24101870 msgr2=0x7fda24197eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:00.173 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:00.169+0000 7fda237fe640 1 --2- 172.21.15.80:0/1180168202 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda24101870 0x7fda24197eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:00.173 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:00.169+0000 7fda237fe640 1 -- 172.21.15.80:0/1180168202 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fda24102230 msgr2=0x7fda2419ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:00.173 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:00.169+0000 7fda237fe640 1 --2- 172.21.15.80:0/1180168202 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fda24102230 0x7fda2419ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:00.173 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:00.169+0000 7fda237fe640 1 -- 172.21.15.80:0/1180168202 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda2419f590 con 0x7fda24100680 2024-04-24T23:32:00.173 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:00.169+0000 7fda1affd640 1 --2- 172.21.15.80:0/1180168202 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda24101870 0x7fda24197eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:32:00.174 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:00.169+0000 7fda237fe640 1 --2- 172.21.15.80:0/1180168202 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fda24100680 0x7fda24197970 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fda1400c970 tx=0x7fda1400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:32:00.174 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:00.169+0000 7fda217fa640 1 -- 172.21.15.80:0/1180168202 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda14007c10 con 0x7fda24100680 2024-04-24T23:32:00.174 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:00.169+0000 7fda29a9e640 1 -- 172.21.15.80:0/1180168202 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda2419f880 con 0x7fda24100680 2024-04-24T23:32:00.174 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:00.169+0000 7fda217fa640 1 -- 172.21.15.80:0/1180168202 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda14007db0 con 0x7fda24100680 2024-04-24T23:32:00.174 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:00.169+0000 7fda217fa640 1 -- 172.21.15.80:0/1180168202 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda1400f8a0 con 0x7fda24100680 2024-04-24T23:32:00.174 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:00.169+0000 7fda29a9e640 1 -- 172.21.15.80:0/1180168202 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda2419fdd0 con 0x7fda24100680 2024-04-24T23:32:00.175 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:00.170+0000 7fda29a9e640 1 -- 172.21.15.80:0/1180168202 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda241078d0 con 0x7fda24100680 2024-04-24T23:32:00.175 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:00.171+0000 7fda217fa640 1 -- 172.21.15.80:0/1180168202 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fda1400fa80 con 0x7fda24100680 2024-04-24T23:32:00.176 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:00.171+0000 7fda217fa640 1 --2- 172.21.15.80:0/1180168202 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fda100a2b90 0x7fda100a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:00.176 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:00.172+0000 7fda1affd640 1 --2- 172.21.15.80:0/1180168202 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fda100a2b90 0x7fda100a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:00.176 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:00.172+0000 7fda217fa640 1 -- 172.21.15.80:0/1180168202 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fda140cf830 con 0x7fda24100680 2024-04-24T23:32:00.176 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:00.172+0000 7fda1affd640 1 --2- 172.21.15.80:0/1180168202 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fda100a2b90 0x7fda100a5050 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fda24198e80 tx=0x7fda0c00a5c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:32:00.176 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:00.176+0000 7fda217fa640 1 -- 172.21.15.80:0/1180168202 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fda1409c450 con 0x7fda24100680 2024-04-24T23:32:00.355 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:00.354+0000 7fda29a9e640 1 -- 172.21.15.80:0/1180168202 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fda24062340 con 0x7fda24100680 2024-04-24T23:32:00.355 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:00.354+0000 7fda217fa640 1 -- 172.21.15.80:0/1180168202 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) ==== 103+0+11 (secure 0 0 0) 0x7fda1409bd50 con 0x7fda24100680 2024-04-24T23:32:00.355 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 32 2024-04-24T23:32:00.357 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:00.356+0000 7fda29a9e640 1 -- 172.21.15.80:0/1180168202 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fda100a2b90 msgr2=0x7fda100a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:00.357 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:00.356+0000 7fda29a9e640 1 --2- 172.21.15.80:0/1180168202 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fda100a2b90 0x7fda100a5050 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fda24198e80 tx=0x7fda0c00a5c0 comp rx=0 tx=0).stop 2024-04-24T23:32:00.357 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:00.356+0000 7fda29a9e640 1 -- 172.21.15.80:0/1180168202 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fda24100680 msgr2=0x7fda24197970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:00.358 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:00.356+0000 7fda29a9e640 1 --2- 172.21.15.80:0/1180168202 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fda24100680 0x7fda24197970 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fda1400c970 tx=0x7fda1400ce30 comp rx=0 tx=0).stop 2024-04-24T23:32:00.358 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:00.356+0000 7fda29a9e640 1 -- 172.21.15.80:0/1180168202 shutdown_connections 2024-04-24T23:32:00.358 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:00.356+0000 7fda29a9e640 1 --2- 172.21.15.80:0/1180168202 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fda100a2b90 0x7fda100a5050 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:00.358 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:00.356+0000 7fda29a9e640 1 --2- 172.21.15.80:0/1180168202 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fda24102230 0x7fda2419ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:00.358 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:00.356+0000 7fda29a9e640 1 --2- 172.21.15.80:0/1180168202 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda24101870 0x7fda24197eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:00.358 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:00.356+0000 7fda29a9e640 1 --2- 172.21.15.80:0/1180168202 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fda24100680 0x7fda24197970 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:00.358 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:00.356+0000 7fda29a9e640 1 -- 172.21.15.80:0/1180168202 >> 172.21.15.80:0/1180168202 conn(0x7fda240fbbf0 msgr2=0x7fda24104aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:32:00.359 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:00.357+0000 7fda29a9e640 1 -- 172.21.15.80:0/1180168202 shutdown_connections 2024-04-24T23:32:00.359 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:00.357+0000 7fda29a9e640 1 -- 172.21.15.80:0/1180168202 wait complete. 2024-04-24T23:32:00.367 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:32:00 smithi080 ceph-mon[29152]: pgmap v440: 129 pgs: 129 active+clean; 69 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 297 KiB/s wr, 25 op/s 2024-04-24T23:32:00.367 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:32:00 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/163075360' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:32:00.368 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:32:00 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/628900366' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:32:00.368 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:32:00 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/2175459580' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T23:32:00.368 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:32:00 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/645663656' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T23:32:00.369 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-04-24T23:32:00.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:32:00 smithi134 ceph-mon[32619]: pgmap v440: 129 pgs: 129 active+clean; 69 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 297 KiB/s wr, 25 op/s 2024-04-24T23:32:00.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:32:00 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/163075360' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:32:00.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:32:00 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/628900366' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:32:00.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:32:00 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/2175459580' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T23:32:00.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:32:00 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/645663656' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T23:32:00.469 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:00.466+0000 7fedbb3a5640 1 -- 172.21.15.80:0/668512626 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fedb4101870 msgr2=0x7fedb4101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:00.469 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:00.466+0000 7fedbb3a5640 1 --2- 172.21.15.80:0/668512626 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fedb4101870 0x7fedb4101cf0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7feda4009900 tx=0x7feda402a410 comp rx=0 tx=0).stop 2024-04-24T23:32:00.470 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:00.466+0000 7fedbb3a5640 1 -- 172.21.15.80:0/668512626 shutdown_connections 2024-04-24T23:32:00.470 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:00.466+0000 7fedbb3a5640 1 --2- 172.21.15.80:0/668512626 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fedb4102230 0x7fedb4108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:00.470 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:00.466+0000 7fedbb3a5640 1 --2- 172.21.15.80:0/668512626 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fedb4101870 0x7fedb4101cf0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:00.470 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:00.466+0000 7fedbb3a5640 1 --2- 172.21.15.80:0/668512626 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fedb4100680 0x7fedb4100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:00.470 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:00.466+0000 7fedbb3a5640 1 -- 172.21.15.80:0/668512626 >> 172.21.15.80:0/668512626 conn(0x7fedb40fbbf0 msgr2=0x7fedb40fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:32:00.470 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:00.466+0000 7fedbb3a5640 1 -- 172.21.15.80:0/668512626 shutdown_connections 2024-04-24T23:32:00.470 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:00.467+0000 7fedbb3a5640 1 -- 172.21.15.80:0/668512626 wait complete. 2024-04-24T23:32:00.470 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:00.467+0000 7fedbb3a5640 1 Processor -- start 2024-04-24T23:32:00.471 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:00.467+0000 7fedbb3a5640 1 -- start start 2024-04-24T23:32:00.471 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:00.467+0000 7fedbb3a5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fedb4100680 0x7fedb4196970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:00.471 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:00.467+0000 7fedbb3a5640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fedb4101870 0x7fedb4196eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:00.471 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:00.467+0000 7fedbb3a5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fedb4102230 0x7fedb419df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:00.471 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:00.467+0000 7fedbb3a5640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fedb410c300 con 0x7fedb4101870 2024-04-24T23:32:00.471 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:00.467+0000 7fedbb3a5640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fedb410c180 con 0x7fedb4102230 2024-04-24T23:32:00.471 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:00.467+0000 7fedbb3a5640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fedb410c480 con 0x7fedb4100680 2024-04-24T23:32:00.472 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:00.467+0000 7fedb8940640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fedb4101870 0x7fedb4196eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:00.472 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:00.467+0000 7fedb8940640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fedb4101870 0x7fedb4196eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53780/0 (socket says 172.21.15.80:53780) 2024-04-24T23:32:00.472 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:00.467+0000 7fedb8940640 1 -- 172.21.15.80:0/2968146966 learned_addr learned my addr 172.21.15.80:0/2968146966 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:32:00.472 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:00.467+0000 7fedb9141640 1 --2- 172.21.15.80:0/2968146966 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fedb4100680 0x7fedb4196970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:00.472 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:00.467+0000 7fedb9942640 1 --2- 172.21.15.80:0/2968146966 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fedb4102230 0x7fedb419df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:00.472 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:00.468+0000 7fedb8940640 1 -- 172.21.15.80:0/2968146966 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fedb4100680 msgr2=0x7fedb4196970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:00.472 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:00.468+0000 7fedb8940640 1 --2- 172.21.15.80:0/2968146966 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fedb4100680 0x7fedb4196970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:00.472 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:00.468+0000 7fedb8940640 1 -- 172.21.15.80:0/2968146966 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fedb4102230 msgr2=0x7fedb419df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:00.472 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:00.468+0000 7fedb8940640 1 --2- 172.21.15.80:0/2968146966 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fedb4102230 0x7fedb419df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:00.473 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:00.468+0000 7fedb8940640 1 -- 172.21.15.80:0/2968146966 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedb419e590 con 0x7fedb4101870 2024-04-24T23:32:00.473 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:00.468+0000 7fedb8940640 1 --2- 172.21.15.80:0/2968146966 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fedb4101870 0x7fedb4196eb0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7feda4009230 tx=0x7feda4004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:32:00.473 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:00.468+0000 7fedaa7fc640 1 -- 172.21.15.80:0/2968146966 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feda403f030 con 0x7fedb4101870 2024-04-24T23:32:00.473 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:00.468+0000 7fedaa7fc640 1 -- 172.21.15.80:0/2968146966 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feda4003ce0 con 0x7fedb4101870 2024-04-24T23:32:00.473 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:00.468+0000 7fedbb3a5640 1 -- 172.21.15.80:0/2968146966 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedb419e820 con 0x7fedb4101870 2024-04-24T23:32:00.473 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:00.468+0000 7fedaa7fc640 1 -- 172.21.15.80:0/2968146966 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feda4035070 con 0x7fedb4101870 2024-04-24T23:32:00.473 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:00.468+0000 7fedbb3a5640 1 -- 172.21.15.80:0/2968146966 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedb419ed10 con 0x7fedb4101870 2024-04-24T23:32:00.473 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:00.469+0000 7fedbb3a5640 1 -- 172.21.15.80:0/2968146966 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fedb41079a0 con 0x7fedb4101870 2024-04-24T23:32:00.474 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:00.470+0000 7fedaa7fc640 1 -- 172.21.15.80:0/2968146966 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7feda4042030 con 0x7fedb4101870 2024-04-24T23:32:00.475 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:00.470+0000 7fedaa7fc640 1 --2- 172.21.15.80:0/2968146966 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7feda00a2b90 0x7feda00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:00.475 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:00.471+0000 7fedb9141640 1 --2- 172.21.15.80:0/2968146966 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7feda00a2b90 0x7feda00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:00.475 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:00.471+0000 7fedaa7fc640 1 -- 172.21.15.80:0/2968146966 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7feda40efeb0 con 0x7fedb4101870 2024-04-24T23:32:00.475 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:00.471+0000 7fedb9141640 1 --2- 172.21.15.80:0/2968146966 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7feda00a2b90 0x7feda00a5050 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fedb41016d0 tx=0x7fedb0009870 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:32:00.475 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:00.474+0000 7fedaa7fc640 1 -- 172.21.15.80:0/2968146966 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7feda40bca40 con 0x7fedb4101870 2024-04-24T23:32:00.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:32:00 smithi148 ceph-mon[32528]: pgmap v440: 129 pgs: 129 active+clean; 69 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 297 KiB/s wr, 25 op/s 2024-04-24T23:32:00.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:32:00 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/163075360' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:32:00.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:32:00 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/628900366' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:32:00.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:32:00 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/2175459580' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T23:32:00.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:32:00 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/645663656' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T23:32:00.696 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:00.695+0000 7fedbb3a5640 1 -- 172.21.15.80:0/2968146966 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7fedb402d340 con 0x7fedb4101870 2024-04-24T23:32:00.704 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:00.703+0000 7fedaa7fc640 1 -- 172.21.15.80:0/2968146966 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v18) ==== 81+0+740368 (secure 0 0 0) 0x7feda41a5c20 con 0x7fedb4101870 2024-04-24T23:32:00.714 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-24T23:32:00.715 INFO:teuthology.orchestra.run.smithi080.stdout:{ 2024-04-24T23:32:00.715 INFO:teuthology.orchestra.run.smithi080.stdout: "epoch": 18, 2024-04-24T23:32:00.715 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:00.715 INFO:teuthology.orchestra.run.smithi080.stdout: "active_gid": 14152, 2024-04-24T23:32:00.715 INFO:teuthology.orchestra.run.smithi080.stdout: "active_name": "x", 2024-04-24T23:32:00.715 INFO:teuthology.orchestra.run.smithi080.stdout: "active_addrs": { 2024-04-24T23:32:00.715 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-24T23:32:00.716 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-24T23:32:00.716 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-24T23:32:00.716 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6800", 2024-04-24T23:32:00.716 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3830596922 2024-04-24T23:32:00.716 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.716 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-24T23:32:00.716 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-24T23:32:00.716 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6801", 2024-04-24T23:32:00.716 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3830596922 2024-04-24T23:32:00.717 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:00.717 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-24T23:32:00.717 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.717 INFO:teuthology.orchestra.run.smithi080.stdout: "active_addr": "172.21.15.80:6801/3830596922", 2024-04-24T23:32:00.717 INFO:teuthology.orchestra.run.smithi080.stdout: "active_change": "2024-04-24T23:20:06.793760+0000", 2024-04-24T23:32:00.717 INFO:teuthology.orchestra.run.smithi080.stdout: "active_mgr_features": 4540701547738038271, 2024-04-24T23:32:00.717 INFO:teuthology.orchestra.run.smithi080.stdout: "available": true, 2024-04-24T23:32:00.717 INFO:teuthology.orchestra.run.smithi080.stdout: "standbys": [ 2024-04-24T23:32:00.718 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-24T23:32:00.718 INFO:teuthology.orchestra.run.smithi080.stdout: "gid": 24118, 2024-04-24T23:32:00.718 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "z", 2024-04-24T23:32:00.718 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr_features": 4540701547738038271, 2024-04-24T23:32:00.718 INFO:teuthology.orchestra.run.smithi080.stdout: "available_modules": [ 2024-04-24T23:32:00.718 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-24T23:32:00.718 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "alerts", 2024-04-24T23:32:00.718 INFO:teuthology.orchestra.run.smithi080.stdout: "can_run": true, 2024-04-24T23:32:00.719 INFO:teuthology.orchestra.run.smithi080.stdout: "error_string": "", 2024-04-24T23:32:00.719 INFO:teuthology.orchestra.run.smithi080.stdout: "module_options": { 2024-04-24T23:32:00.719 INFO:teuthology.orchestra.run.smithi080.stdout: "interval": { 2024-04-24T23:32:00.719 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "interval", 2024-04-24T23:32:00.719 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "secs", 2024-04-24T23:32:00.719 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.719 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:00.720 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "60", 2024-04-24T23:32:00.720 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.720 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.720 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.720 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "How frequently to reexamine health status", 2024-04-24T23:32:00.720 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.720 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.720 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.721 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.721 INFO:teuthology.orchestra.run.smithi080.stdout: "log_level": { 2024-04-24T23:32:00.721 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_level", 2024-04-24T23:32:00.721 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:00.721 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.721 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:00.721 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:00.721 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.722 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.722 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:00.722 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:00.722 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:00.722 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:00.722 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:00.723 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:00.723 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:00.723 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:00.723 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:00.723 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.723 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.723 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.723 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.724 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster": { 2024-04-24T23:32:00.724 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster", 2024-04-24T23:32:00.724 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:00.724 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.724 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:00.724 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:00.724 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.724 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.724 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.725 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:00.725 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.725 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.725 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.725 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.725 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster_level": { 2024-04-24T23:32:00.725 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster_level", 2024-04-24T23:32:00.725 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:00.726 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.726 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:00.726 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "info", 2024-04-24T23:32:00.726 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.726 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.726 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:00.726 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:00.727 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:00.727 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:00.727 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:00.727 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:00.727 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:00.727 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:00.727 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:00.727 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.727 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.728 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.728 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.728 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_file": { 2024-04-24T23:32:00.728 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_file", 2024-04-24T23:32:00.728 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:00.728 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.728 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:00.728 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:00.729 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.729 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.729 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.729 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:00.729 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.729 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.729 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.729 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.730 INFO:teuthology.orchestra.run.smithi080.stdout: "smtp_destination": { 2024-04-24T23:32:00.730 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "smtp_destination", 2024-04-24T23:32:00.730 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:00.730 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.730 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:00.730 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:00.730 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.730 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.730 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.731 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Email address to send alerts to", 2024-04-24T23:32:00.731 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.731 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.731 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.731 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.731 INFO:teuthology.orchestra.run.smithi080.stdout: "smtp_from_name": { 2024-04-24T23:32:00.731 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "smtp_from_name", 2024-04-24T23:32:00.731 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:00.732 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.732 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:00.732 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "Ceph", 2024-04-24T23:32:00.732 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.732 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.732 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.732 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Email From: name", 2024-04-24T23:32:00.732 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.732 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.733 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.733 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.733 INFO:teuthology.orchestra.run.smithi080.stdout: "smtp_host": { 2024-04-24T23:32:00.733 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "smtp_host", 2024-04-24T23:32:00.733 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:00.733 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.733 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:00.733 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:00.734 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.734 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.734 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.734 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "SMTP server", 2024-04-24T23:32:00.734 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.734 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.734 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.734 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.735 INFO:teuthology.orchestra.run.smithi080.stdout: "smtp_password": { 2024-04-24T23:32:00.735 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "smtp_password", 2024-04-24T23:32:00.735 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:00.735 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.735 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:00.735 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:00.735 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.735 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.736 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.736 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Password to authenticate with", 2024-04-24T23:32:00.736 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.736 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.736 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.736 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.736 INFO:teuthology.orchestra.run.smithi080.stdout: "smtp_port": { 2024-04-24T23:32:00.736 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "smtp_port", 2024-04-24T23:32:00.736 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "int", 2024-04-24T23:32:00.737 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.737 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:00.737 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "465", 2024-04-24T23:32:00.737 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.737 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.737 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.737 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "SMTP port", 2024-04-24T23:32:00.737 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.737 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.738 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.738 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.738 INFO:teuthology.orchestra.run.smithi080.stdout: "smtp_sender": { 2024-04-24T23:32:00.738 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "smtp_sender", 2024-04-24T23:32:00.738 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:00.738 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.738 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:00.738 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:00.739 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.739 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.739 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.739 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "SMTP envelope sender", 2024-04-24T23:32:00.739 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.739 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.739 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.740 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.740 INFO:teuthology.orchestra.run.smithi080.stdout: "smtp_ssl": { 2024-04-24T23:32:00.740 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "smtp_ssl", 2024-04-24T23:32:00.740 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:00.740 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.740 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:00.740 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "True", 2024-04-24T23:32:00.740 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.740 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.741 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.741 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-24T23:32:00.741 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.741 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.741 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.741 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.741 INFO:teuthology.orchestra.run.smithi080.stdout: "smtp_user": { 2024-04-24T23:32:00.741 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "smtp_user", 2024-04-24T23:32:00.742 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:00.742 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.742 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:00.742 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:00.742 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.742 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.742 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.742 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "User to authenticate as", 2024-04-24T23:32:00.743 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.743 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.743 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.743 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:00.743 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:00.743 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.743 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-24T23:32:00.743 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "balancer", 2024-04-24T23:32:00.743 INFO:teuthology.orchestra.run.smithi080.stdout: "can_run": true, 2024-04-24T23:32:00.744 INFO:teuthology.orchestra.run.smithi080.stdout: "error_string": "", 2024-04-24T23:32:00.744 INFO:teuthology.orchestra.run.smithi080.stdout: "module_options": { 2024-04-24T23:32:00.744 INFO:teuthology.orchestra.run.smithi080.stdout: "active": { 2024-04-24T23:32:00.744 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "active", 2024-04-24T23:32:00.744 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:00.744 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.744 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:00.744 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "True", 2024-04-24T23:32:00.745 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.745 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.745 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.745 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "automatically balance PGs across cluster", 2024-04-24T23:32:00.745 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.745 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.745 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.745 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.746 INFO:teuthology.orchestra.run.smithi080.stdout: "begin_time": { 2024-04-24T23:32:00.746 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "begin_time", 2024-04-24T23:32:00.746 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:00.746 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.746 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:00.746 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "0000", 2024-04-24T23:32:00.746 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.746 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.747 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.747 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "beginning time of day to automatically balance", 2024-04-24T23:32:00.747 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-24T23:32:00.747 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.747 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.747 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.747 INFO:teuthology.orchestra.run.smithi080.stdout: "begin_weekday": { 2024-04-24T23:32:00.747 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "begin_weekday", 2024-04-24T23:32:00.747 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "uint", 2024-04-24T23:32:00.748 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.748 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:00.748 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "0", 2024-04-24T23:32:00.748 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "0", 2024-04-24T23:32:00.748 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "6", 2024-04-24T23:32:00.748 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.748 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-24T23:32:00.748 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-24T23:32:00.749 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.749 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.749 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.749 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_compat_max_iterations": { 2024-04-24T23:32:00.749 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "crush_compat_max_iterations", 2024-04-24T23:32:00.749 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "uint", 2024-04-24T23:32:00.749 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.749 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:00.749 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "25", 2024-04-24T23:32:00.750 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "1", 2024-04-24T23:32:00.750 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "250", 2024-04-24T23:32:00.750 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.750 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-24T23:32:00.750 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.754 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.754 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.754 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.754 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_compat_metrics": { 2024-04-24T23:32:00.754 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "crush_compat_metrics", 2024-04-24T23:32:00.754 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:00.755 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.755 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:00.755 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "pgs,objects,bytes", 2024-04-24T23:32:00.755 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.755 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.755 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.755 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-24T23:32:00.756 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-24T23:32:00.756 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.756 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.756 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.756 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_compat_step": { 2024-04-24T23:32:00.756 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "crush_compat_step", 2024-04-24T23:32:00.756 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "float", 2024-04-24T23:32:00.756 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.757 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:00.757 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "0.5", 2024-04-24T23:32:00.757 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "0.001", 2024-04-24T23:32:00.757 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "0.999", 2024-04-24T23:32:00.757 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.757 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "aggressiveness of optimization", 2024-04-24T23:32:00.757 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-24T23:32:00.757 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.758 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.758 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.758 INFO:teuthology.orchestra.run.smithi080.stdout: "end_time": { 2024-04-24T23:32:00.758 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "end_time", 2024-04-24T23:32:00.758 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:00.758 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.758 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:00.758 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "2359", 2024-04-24T23:32:00.758 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.759 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.759 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.759 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "ending time of day to automatically balance", 2024-04-24T23:32:00.759 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-24T23:32:00.759 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.759 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.759 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.759 INFO:teuthology.orchestra.run.smithi080.stdout: "end_weekday": { 2024-04-24T23:32:00.760 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "end_weekday", 2024-04-24T23:32:00.760 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "uint", 2024-04-24T23:32:00.760 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.760 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:00.760 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "0", 2024-04-24T23:32:00.760 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "0", 2024-04-24T23:32:00.760 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "6", 2024-04-24T23:32:00.760 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.760 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-24T23:32:00.761 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-24T23:32:00.761 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.761 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.761 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.761 INFO:teuthology.orchestra.run.smithi080.stdout: "log_level": { 2024-04-24T23:32:00.761 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_level", 2024-04-24T23:32:00.761 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:00.761 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.762 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:00.762 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:00.762 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.762 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.762 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:00.762 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:00.762 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:00.763 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:00.763 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:00.763 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:00.763 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:00.763 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:00.763 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:00.763 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.763 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.764 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.764 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.764 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster": { 2024-04-24T23:32:00.764 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster", 2024-04-24T23:32:00.764 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:00.764 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.764 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:00.765 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:00.765 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.765 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.765 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.765 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:00.765 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.765 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.765 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.766 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.766 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster_level": { 2024-04-24T23:32:00.766 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster_level", 2024-04-24T23:32:00.766 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:00.766 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.766 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:00.766 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "info", 2024-04-24T23:32:00.767 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.767 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.767 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:00.767 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:00.767 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:00.767 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:00.767 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:00.767 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:00.767 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:00.768 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:00.768 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:00.768 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.768 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.768 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.768 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.768 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_file": { 2024-04-24T23:32:00.768 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_file", 2024-04-24T23:32:00.769 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:00.769 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.769 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:00.769 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:00.769 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.769 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.769 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.769 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:00.770 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.770 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.770 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.770 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.770 INFO:teuthology.orchestra.run.smithi080.stdout: "min_score": { 2024-04-24T23:32:00.770 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "min_score", 2024-04-24T23:32:00.770 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "float", 2024-04-24T23:32:00.770 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.771 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:00.771 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "0", 2024-04-24T23:32:00.771 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.771 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.771 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.771 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-24T23:32:00.771 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.771 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.772 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.772 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.772 INFO:teuthology.orchestra.run.smithi080.stdout: "mode": { 2024-04-24T23:32:00.772 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "mode", 2024-04-24T23:32:00.772 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:00.772 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.772 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:00.772 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "upmap", 2024-04-24T23:32:00.772 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.773 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.773 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:00.773 INFO:teuthology.orchestra.run.smithi080.stdout: "crush-compat", 2024-04-24T23:32:00.773 INFO:teuthology.orchestra.run.smithi080.stdout: "none", 2024-04-24T23:32:00.773 INFO:teuthology.orchestra.run.smithi080.stdout: "read", 2024-04-24T23:32:00.773 INFO:teuthology.orchestra.run.smithi080.stdout: "upmap", 2024-04-24T23:32:00.773 INFO:teuthology.orchestra.run.smithi080.stdout: "upmap-read" 2024-04-24T23:32:00.773 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:00.774 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Balancer mode", 2024-04-24T23:32:00.774 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.774 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.774 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.774 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.774 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_ids": { 2024-04-24T23:32:00.774 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "pool_ids", 2024-04-24T23:32:00.774 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:00.774 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.775 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:00.775 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:00.775 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.775 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.775 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.775 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-24T23:32:00.775 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.776 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.776 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.776 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.776 INFO:teuthology.orchestra.run.smithi080.stdout: "sleep_interval": { 2024-04-24T23:32:00.776 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "sleep_interval", 2024-04-24T23:32:00.776 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "secs", 2024-04-24T23:32:00.776 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.776 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:00.776 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "60", 2024-04-24T23:32:00.777 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.777 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.777 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.777 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-24T23:32:00.777 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.777 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.777 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.777 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.778 INFO:teuthology.orchestra.run.smithi080.stdout: "upmap_max_deviation": { 2024-04-24T23:32:00.778 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "upmap_max_deviation", 2024-04-24T23:32:00.778 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "int", 2024-04-24T23:32:00.778 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.778 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:00.778 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "5", 2024-04-24T23:32:00.778 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "1", 2024-04-24T23:32:00.778 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.778 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.779 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-24T23:32:00.779 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-24T23:32:00.779 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.779 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.779 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.779 INFO:teuthology.orchestra.run.smithi080.stdout: "upmap_max_optimizations": { 2024-04-24T23:32:00.779 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "upmap_max_optimizations", 2024-04-24T23:32:00.779 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "uint", 2024-04-24T23:32:00.780 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.780 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:00.780 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "10", 2024-04-24T23:32:00.780 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.780 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.780 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.780 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-24T23:32:00.780 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.781 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.781 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.781 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:00.781 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:00.781 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.781 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-24T23:32:00.781 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "cephadm", 2024-04-24T23:32:00.781 INFO:teuthology.orchestra.run.smithi080.stdout: "can_run": true, 2024-04-24T23:32:00.781 INFO:teuthology.orchestra.run.smithi080.stdout: "error_string": "", 2024-04-24T23:32:00.782 INFO:teuthology.orchestra.run.smithi080.stdout: "module_options": { 2024-04-24T23:32:00.782 INFO:teuthology.orchestra.run.smithi080.stdout: "agent_down_multiplier": { 2024-04-24T23:32:00.782 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "agent_down_multiplier", 2024-04-24T23:32:00.782 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "float", 2024-04-24T23:32:00.782 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.782 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:00.782 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "3.0", 2024-04-24T23:32:00.782 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.783 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.783 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.783 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-24T23:32:00.783 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.783 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.783 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.784 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.784 INFO:teuthology.orchestra.run.smithi080.stdout: "agent_refresh_rate": { 2024-04-24T23:32:00.784 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "agent_refresh_rate", 2024-04-24T23:32:00.784 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "secs", 2024-04-24T23:32:00.784 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.784 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:00.784 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "20", 2024-04-24T23:32:00.784 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.785 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.785 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.785 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-24T23:32:00.785 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.785 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.785 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.785 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.785 INFO:teuthology.orchestra.run.smithi080.stdout: "agent_starting_port": { 2024-04-24T23:32:00.785 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "agent_starting_port", 2024-04-24T23:32:00.786 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "int", 2024-04-24T23:32:00.786 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.786 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:00.786 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "4721", 2024-04-24T23:32:00.786 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.786 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.786 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.786 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-24T23:32:00.787 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.787 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.787 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.787 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.787 INFO:teuthology.orchestra.run.smithi080.stdout: "allow_ptrace": { 2024-04-24T23:32:00.787 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "allow_ptrace", 2024-04-24T23:32:00.787 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:00.788 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.788 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:00.788 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:00.788 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.788 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.788 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.788 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-24T23:32:00.788 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-24T23:32:00.788 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.789 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.789 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.789 INFO:teuthology.orchestra.run.smithi080.stdout: "autotune_interval": { 2024-04-24T23:32:00.789 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "autotune_interval", 2024-04-24T23:32:00.789 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "secs", 2024-04-24T23:32:00.789 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.789 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:00.789 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "600", 2024-04-24T23:32:00.790 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.790 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.790 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.790 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-24T23:32:00.790 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.790 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.790 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.790 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.791 INFO:teuthology.orchestra.run.smithi080.stdout: "autotune_memory_target_ratio": { 2024-04-24T23:32:00.791 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "autotune_memory_target_ratio", 2024-04-24T23:32:00.791 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "float", 2024-04-24T23:32:00.791 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.791 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:00.791 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "0.7", 2024-04-24T23:32:00.791 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.791 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.792 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.792 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-24T23:32:00.792 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.792 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.792 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.792 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.792 INFO:teuthology.orchestra.run.smithi080.stdout: "cephadm_log_destination": { 2024-04-24T23:32:00.792 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "cephadm_log_destination", 2024-04-24T23:32:00.792 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:00.793 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.793 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:00.793 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:00.793 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.793 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.793 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:00.793 INFO:teuthology.orchestra.run.smithi080.stdout: "file", 2024-04-24T23:32:00.793 INFO:teuthology.orchestra.run.smithi080.stdout: "file,syslog", 2024-04-24T23:32:00.794 INFO:teuthology.orchestra.run.smithi080.stdout: "syslog" 2024-04-24T23:32:00.794 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:00.794 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-24T23:32:00.794 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.794 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.794 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.794 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.794 INFO:teuthology.orchestra.run.smithi080.stdout: "cgroups_split": { 2024-04-24T23:32:00.795 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "cgroups_split", 2024-04-24T23:32:00.795 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:00.795 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.795 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:00.795 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "True", 2024-04-24T23:32:00.795 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.795 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.795 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.795 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-24T23:32:00.796 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.796 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.796 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.796 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.796 INFO:teuthology.orchestra.run.smithi080.stdout: "config_checks_enabled": { 2024-04-24T23:32:00.796 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "config_checks_enabled", 2024-04-24T23:32:00.796 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:00.797 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.797 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:00.797 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:00.797 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.797 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.797 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.797 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-24T23:32:00.797 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.798 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.798 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.798 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.798 INFO:teuthology.orchestra.run.smithi080.stdout: "config_dashboard": { 2024-04-24T23:32:00.798 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "config_dashboard", 2024-04-24T23:32:00.798 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:00.798 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.798 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:00.798 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "True", 2024-04-24T23:32:00.799 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.799 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.799 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.799 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-24T23:32:00.799 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.799 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.799 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.799 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.800 INFO:teuthology.orchestra.run.smithi080.stdout: "container_image_alertmanager": { 2024-04-24T23:32:00.800 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "container_image_alertmanager", 2024-04-24T23:32:00.800 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:00.800 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.800 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:00.800 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-24T23:32:00.800 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.800 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.801 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.801 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Prometheus container image", 2024-04-24T23:32:00.801 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.801 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.801 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.801 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.801 INFO:teuthology.orchestra.run.smithi080.stdout: "container_image_base": { 2024-04-24T23:32:00.801 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "container_image_base", 2024-04-24T23:32:00.801 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:00.802 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.802 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:00.802 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-24T23:32:00.802 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.802 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.802 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.802 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Container image name, without the tag", 2024-04-24T23:32:00.802 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.803 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.803 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.803 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.803 INFO:teuthology.orchestra.run.smithi080.stdout: "container_image_elasticsearch": { 2024-04-24T23:32:00.803 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "container_image_elasticsearch", 2024-04-24T23:32:00.803 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:00.803 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.803 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:00.804 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-24T23:32:00.804 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.804 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.804 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.804 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "elasticsearch container image", 2024-04-24T23:32:00.804 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.804 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.804 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.805 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.805 INFO:teuthology.orchestra.run.smithi080.stdout: "container_image_grafana": { 2024-04-24T23:32:00.805 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "container_image_grafana", 2024-04-24T23:32:00.805 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:00.805 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.805 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:00.805 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-24T23:32:00.805 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.805 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.806 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.806 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Prometheus container image", 2024-04-24T23:32:00.806 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.806 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.806 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.806 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.806 INFO:teuthology.orchestra.run.smithi080.stdout: "container_image_haproxy": { 2024-04-24T23:32:00.806 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "container_image_haproxy", 2024-04-24T23:32:00.807 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:00.807 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.807 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:00.807 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-24T23:32:00.807 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.807 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.807 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.807 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "HAproxy container image", 2024-04-24T23:32:00.808 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.808 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.808 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.808 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.808 INFO:teuthology.orchestra.run.smithi080.stdout: "container_image_jaeger_agent": { 2024-04-24T23:32:00.808 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "container_image_jaeger_agent", 2024-04-24T23:32:00.808 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:00.808 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.809 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:00.809 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-24T23:32:00.809 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.809 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.809 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.809 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Jaeger agent container image", 2024-04-24T23:32:00.809 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.809 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.810 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.810 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.810 INFO:teuthology.orchestra.run.smithi080.stdout: "container_image_jaeger_collector": { 2024-04-24T23:32:00.810 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "container_image_jaeger_collector", 2024-04-24T23:32:00.810 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:00.810 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.810 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:00.810 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-24T23:32:00.810 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.811 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.811 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.811 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Jaeger collector container image", 2024-04-24T23:32:00.811 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.811 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.811 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.811 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.811 INFO:teuthology.orchestra.run.smithi080.stdout: "container_image_jaeger_query": { 2024-04-24T23:32:00.812 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "container_image_jaeger_query", 2024-04-24T23:32:00.812 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:00.812 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.812 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:00.812 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-24T23:32:00.812 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.812 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.812 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.813 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Jaeger query container image", 2024-04-24T23:32:00.813 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.813 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.813 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.813 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.813 INFO:teuthology.orchestra.run.smithi080.stdout: "container_image_keepalived": { 2024-04-24T23:32:00.813 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "container_image_keepalived", 2024-04-24T23:32:00.813 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:00.813 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.814 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:00.814 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-24T23:32:00.814 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.814 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.814 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.814 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Keepalived container image", 2024-04-24T23:32:00.814 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.814 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.815 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.815 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.815 INFO:teuthology.orchestra.run.smithi080.stdout: "container_image_loki": { 2024-04-24T23:32:00.815 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "container_image_loki", 2024-04-24T23:32:00.815 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:00.815 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.815 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:00.815 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-24T23:32:00.816 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.816 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.816 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.816 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Loki container image", 2024-04-24T23:32:00.816 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.816 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.816 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.816 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.817 INFO:teuthology.orchestra.run.smithi080.stdout: "container_image_node_exporter": { 2024-04-24T23:32:00.817 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "container_image_node_exporter", 2024-04-24T23:32:00.817 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:00.817 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.817 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:00.817 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-24T23:32:00.817 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.817 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.817 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.818 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Prometheus container image", 2024-04-24T23:32:00.818 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.818 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.818 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.818 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.818 INFO:teuthology.orchestra.run.smithi080.stdout: "container_image_nvmeof": { 2024-04-24T23:32:00.819 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "container_image_nvmeof", 2024-04-24T23:32:00.819 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:00.819 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.819 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:00.819 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-24T23:32:00.819 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.819 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.819 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.819 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Nvme-of container image", 2024-04-24T23:32:00.820 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.820 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.820 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.820 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.820 INFO:teuthology.orchestra.run.smithi080.stdout: "container_image_prometheus": { 2024-04-24T23:32:00.820 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "container_image_prometheus", 2024-04-24T23:32:00.820 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:00.820 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.821 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:00.821 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-24T23:32:00.821 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.821 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.821 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.821 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Prometheus container image", 2024-04-24T23:32:00.821 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.821 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.821 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.822 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.822 INFO:teuthology.orchestra.run.smithi080.stdout: "container_image_promtail": { 2024-04-24T23:32:00.822 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "container_image_promtail", 2024-04-24T23:32:00.822 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:00.822 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.822 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:00.822 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-24T23:32:00.822 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.823 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.823 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.823 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Promtail container image", 2024-04-24T23:32:00.823 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.823 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.823 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.823 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.823 INFO:teuthology.orchestra.run.smithi080.stdout: "container_image_samba": { 2024-04-24T23:32:00.823 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "container_image_samba", 2024-04-24T23:32:00.824 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:00.824 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.824 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:00.824 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-24T23:32:00.824 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.824 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.824 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.824 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Samba/SMB container image", 2024-04-24T23:32:00.824 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.825 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.825 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.825 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.825 INFO:teuthology.orchestra.run.smithi080.stdout: "container_image_snmp_gateway": { 2024-04-24T23:32:00.825 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "container_image_snmp_gateway", 2024-04-24T23:32:00.825 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:00.825 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.825 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:00.825 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-24T23:32:00.826 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.826 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.826 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.826 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "SNMP Gateway container image", 2024-04-24T23:32:00.826 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.826 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.826 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.826 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.827 INFO:teuthology.orchestra.run.smithi080.stdout: "container_init": { 2024-04-24T23:32:00.827 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "container_init", 2024-04-24T23:32:00.827 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:00.827 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.827 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:00.827 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "True", 2024-04-24T23:32:00.827 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.827 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.828 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.828 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Run podman/docker with `--init`", 2024-04-24T23:32:00.828 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.828 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.828 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.828 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.828 INFO:teuthology.orchestra.run.smithi080.stdout: "daemon_cache_timeout": { 2024-04-24T23:32:00.828 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "daemon_cache_timeout", 2024-04-24T23:32:00.828 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "secs", 2024-04-24T23:32:00.829 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.829 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:00.829 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "600", 2024-04-24T23:32:00.829 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.829 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.829 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.829 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-24T23:32:00.829 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.830 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.830 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.830 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.830 INFO:teuthology.orchestra.run.smithi080.stdout: "default_cephadm_command_timeout": { 2024-04-24T23:32:00.830 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "default_cephadm_command_timeout", 2024-04-24T23:32:00.830 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "int", 2024-04-24T23:32:00.830 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.830 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:00.830 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "900", 2024-04-24T23:32:00.831 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.831 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.831 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.831 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-24T23:32:00.831 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.831 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.831 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.831 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.832 INFO:teuthology.orchestra.run.smithi080.stdout: "default_registry": { 2024-04-24T23:32:00.832 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "default_registry", 2024-04-24T23:32:00.832 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:00.832 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.832 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:00.832 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "docker.io", 2024-04-24T23:32:00.832 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.832 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.832 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.833 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-24T23:32:00.833 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.833 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.833 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.833 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.833 INFO:teuthology.orchestra.run.smithi080.stdout: "device_cache_timeout": { 2024-04-24T23:32:00.833 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "device_cache_timeout", 2024-04-24T23:32:00.833 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "secs", 2024-04-24T23:32:00.833 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.834 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:00.834 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "1800", 2024-04-24T23:32:00.834 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.834 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.834 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.834 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "seconds to cache device inventory", 2024-04-24T23:32:00.834 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.835 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.835 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.835 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.835 INFO:teuthology.orchestra.run.smithi080.stdout: "device_enhanced_scan": { 2024-04-24T23:32:00.835 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "device_enhanced_scan", 2024-04-24T23:32:00.835 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:00.836 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.836 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:00.836 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:00.836 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.836 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.836 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.836 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-24T23:32:00.836 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.836 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.837 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.837 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.837 INFO:teuthology.orchestra.run.smithi080.stdout: "facts_cache_timeout": { 2024-04-24T23:32:00.837 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "facts_cache_timeout", 2024-04-24T23:32:00.837 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "secs", 2024-04-24T23:32:00.837 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.837 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:00.837 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "60", 2024-04-24T23:32:00.837 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.838 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.838 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.838 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "seconds to cache host facts data", 2024-04-24T23:32:00.838 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.838 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.838 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.838 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.838 INFO:teuthology.orchestra.run.smithi080.stdout: "grafana_dashboards_path": { 2024-04-24T23:32:00.839 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "grafana_dashboards_path", 2024-04-24T23:32:00.839 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:00.839 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.839 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:00.839 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-24T23:32:00.839 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.839 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.839 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.839 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-24T23:32:00.840 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.840 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.840 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.840 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.840 INFO:teuthology.orchestra.run.smithi080.stdout: "host_check_interval": { 2024-04-24T23:32:00.840 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "host_check_interval", 2024-04-24T23:32:00.840 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "secs", 2024-04-24T23:32:00.840 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.840 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:00.841 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "600", 2024-04-24T23:32:00.841 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.841 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.841 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.841 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "how frequently to perform a host check", 2024-04-24T23:32:00.841 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.841 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.841 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.842 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.842 INFO:teuthology.orchestra.run.smithi080.stdout: "hw_monitoring": { 2024-04-24T23:32:00.842 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "hw_monitoring", 2024-04-24T23:32:00.842 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:00.842 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.842 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:00.842 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:00.842 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.843 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.843 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.843 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-24T23:32:00.843 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.843 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.843 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.843 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.843 INFO:teuthology.orchestra.run.smithi080.stdout: "inventory_list_all": { 2024-04-24T23:32:00.843 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "inventory_list_all", 2024-04-24T23:32:00.844 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:00.844 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.844 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:00.844 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:00.844 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.844 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.844 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.844 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-24T23:32:00.844 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.845 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.845 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.845 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.845 INFO:teuthology.orchestra.run.smithi080.stdout: "log_level": { 2024-04-24T23:32:00.845 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_level", 2024-04-24T23:32:00.845 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:00.845 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.845 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:00.845 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:00.846 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.846 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.846 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:00.846 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:00.846 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:00.846 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:00.846 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:00.846 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:00.847 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:00.847 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:00.847 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:00.847 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.847 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.847 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.847 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.847 INFO:teuthology.orchestra.run.smithi080.stdout: "log_refresh_metadata": { 2024-04-24T23:32:00.847 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_refresh_metadata", 2024-04-24T23:32:00.848 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:00.848 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.848 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:00.848 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:00.848 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.848 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.848 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.848 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-24T23:32:00.849 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.849 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.849 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.849 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.849 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster": { 2024-04-24T23:32:00.849 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster", 2024-04-24T23:32:00.849 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:00.849 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.850 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:00.850 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "True", 2024-04-24T23:32:00.850 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.850 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.850 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.850 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-24T23:32:00.850 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.850 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.851 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.851 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.851 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster_level": { 2024-04-24T23:32:00.851 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster_level", 2024-04-24T23:32:00.851 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:00.851 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.851 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:00.851 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "info", 2024-04-24T23:32:00.852 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.852 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.852 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:00.852 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:00.852 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:00.852 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:00.853 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:00.853 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:00.853 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:00.853 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:00.853 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:00.853 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.853 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.853 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.854 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.854 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_file": { 2024-04-24T23:32:00.854 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_file", 2024-04-24T23:32:00.854 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:00.854 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.854 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:00.854 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:00.854 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.854 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.855 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.855 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:00.855 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.855 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.855 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.855 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.855 INFO:teuthology.orchestra.run.smithi080.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-24T23:32:00.855 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-24T23:32:00.856 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:00.856 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.856 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:00.856 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:00.856 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.856 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.856 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.856 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-24T23:32:00.856 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.857 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.857 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.857 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.857 INFO:teuthology.orchestra.run.smithi080.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-24T23:32:00.857 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-24T23:32:00.857 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:00.857 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.857 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:00.857 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "*", 2024-04-24T23:32:00.858 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.858 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.858 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.858 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-24T23:32:00.858 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.858 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.858 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.858 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.859 INFO:teuthology.orchestra.run.smithi080.stdout: "max_count_per_host": { 2024-04-24T23:32:00.859 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "max_count_per_host", 2024-04-24T23:32:00.859 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "int", 2024-04-24T23:32:00.859 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.859 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:00.859 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "10", 2024-04-24T23:32:00.859 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.859 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.859 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.860 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "max number of daemons per service per host", 2024-04-24T23:32:00.860 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.860 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.860 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.860 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.860 INFO:teuthology.orchestra.run.smithi080.stdout: "max_osd_draining_count": { 2024-04-24T23:32:00.860 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "max_osd_draining_count", 2024-04-24T23:32:00.860 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "int", 2024-04-24T23:32:00.860 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.861 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:00.861 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "10", 2024-04-24T23:32:00.861 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.861 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.861 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.861 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-24T23:32:00.861 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.861 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.862 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.862 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.862 INFO:teuthology.orchestra.run.smithi080.stdout: "migration_current": { 2024-04-24T23:32:00.862 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "migration_current", 2024-04-24T23:32:00.862 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "int", 2024-04-24T23:32:00.862 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.862 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:00.862 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:00.862 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.863 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.863 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.863 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "internal - do not modify", 2024-04-24T23:32:00.863 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.863 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.863 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.863 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.863 INFO:teuthology.orchestra.run.smithi080.stdout: "mode": { 2024-04-24T23:32:00.864 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "mode", 2024-04-24T23:32:00.864 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:00.864 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.864 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:00.864 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "root", 2024-04-24T23:32:00.864 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.864 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.864 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:00.864 INFO:teuthology.orchestra.run.smithi080.stdout: "cephadm-package", 2024-04-24T23:32:00.865 INFO:teuthology.orchestra.run.smithi080.stdout: "root" 2024-04-24T23:32:00.865 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:00.865 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "mode for remote execution of cephadm", 2024-04-24T23:32:00.865 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.865 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.865 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.865 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.865 INFO:teuthology.orchestra.run.smithi080.stdout: "oob_default_addr": { 2024-04-24T23:32:00.865 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "oob_default_addr", 2024-04-24T23:32:00.866 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:00.866 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.866 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:00.866 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "169.254.1.1", 2024-04-24T23:32:00.866 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.866 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.866 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.866 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-24T23:32:00.866 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.867 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.867 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.867 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.867 INFO:teuthology.orchestra.run.smithi080.stdout: "prometheus_alerts_path": { 2024-04-24T23:32:00.867 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "prometheus_alerts_path", 2024-04-24T23:32:00.867 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:00.867 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.867 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:00.868 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-24T23:32:00.868 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.868 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.868 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.868 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-24T23:32:00.868 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.868 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.868 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.868 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.869 INFO:teuthology.orchestra.run.smithi080.stdout: "registry_insecure": { 2024-04-24T23:32:00.869 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "registry_insecure", 2024-04-24T23:32:00.869 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:00.869 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.870 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:00.870 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:00.870 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.870 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.870 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.870 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-24T23:32:00.870 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.871 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.871 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.871 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.871 INFO:teuthology.orchestra.run.smithi080.stdout: "registry_password": { 2024-04-24T23:32:00.871 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "registry_password", 2024-04-24T23:32:00.871 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:00.871 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.871 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:00.871 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:00.872 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.872 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.872 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.872 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-24T23:32:00.872 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.872 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.872 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.872 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.872 INFO:teuthology.orchestra.run.smithi080.stdout: "registry_url": { 2024-04-24T23:32:00.873 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "registry_url", 2024-04-24T23:32:00.873 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:00.873 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.873 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:00.873 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:00.873 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.873 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.873 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.874 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-24T23:32:00.874 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.874 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.874 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.874 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.874 INFO:teuthology.orchestra.run.smithi080.stdout: "registry_username": { 2024-04-24T23:32:00.874 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "registry_username", 2024-04-24T23:32:00.874 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:00.874 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.875 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:00.875 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:00.875 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.875 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.875 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.875 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-24T23:32:00.875 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.875 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.876 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.876 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.876 INFO:teuthology.orchestra.run.smithi080.stdout: "secure_monitoring_stack": { 2024-04-24T23:32:00.876 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "secure_monitoring_stack", 2024-04-24T23:32:00.876 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:00.876 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.876 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:00.876 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:00.876 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.877 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.877 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.877 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-24T23:32:00.877 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.877 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.877 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.877 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.877 INFO:teuthology.orchestra.run.smithi080.stdout: "service_discovery_port": { 2024-04-24T23:32:00.877 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "service_discovery_port", 2024-04-24T23:32:00.878 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "int", 2024-04-24T23:32:00.878 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.878 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:00.878 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "8765", 2024-04-24T23:32:00.878 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.878 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.878 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.878 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "cephadm service discovery port", 2024-04-24T23:32:00.879 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.879 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.879 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.879 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.879 INFO:teuthology.orchestra.run.smithi080.stdout: "ssh_config_file": { 2024-04-24T23:32:00.879 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "ssh_config_file", 2024-04-24T23:32:00.879 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:00.879 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.879 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:00.880 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:00.880 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.880 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.880 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.880 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-24T23:32:00.880 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.880 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.880 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.880 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.881 INFO:teuthology.orchestra.run.smithi080.stdout: "use_agent": { 2024-04-24T23:32:00.881 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "use_agent", 2024-04-24T23:32:00.881 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:00.881 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.881 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:00.881 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:00.881 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.881 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.881 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.882 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-24T23:32:00.882 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.882 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.882 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.882 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.882 INFO:teuthology.orchestra.run.smithi080.stdout: "use_repo_digest": { 2024-04-24T23:32:00.882 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "use_repo_digest", 2024-04-24T23:32:00.882 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:00.883 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.883 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:00.883 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "True", 2024-04-24T23:32:00.883 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.883 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.883 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.883 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-24T23:32:00.883 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.883 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.884 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.884 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.884 INFO:teuthology.orchestra.run.smithi080.stdout: "warn_on_failed_host_check": { 2024-04-24T23:32:00.884 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "warn_on_failed_host_check", 2024-04-24T23:32:00.884 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:00.884 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.884 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:00.884 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "True", 2024-04-24T23:32:00.884 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.885 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.885 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.885 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "raise a health warning if the host check fails", 2024-04-24T23:32:00.885 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.885 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.885 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.885 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.885 INFO:teuthology.orchestra.run.smithi080.stdout: "warn_on_stray_daemons": { 2024-04-24T23:32:00.885 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "warn_on_stray_daemons", 2024-04-24T23:32:00.886 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:00.886 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.886 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:00.886 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "True", 2024-04-24T23:32:00.886 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.886 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.886 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.886 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-24T23:32:00.887 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.887 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.887 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.887 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.887 INFO:teuthology.orchestra.run.smithi080.stdout: "warn_on_stray_hosts": { 2024-04-24T23:32:00.887 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "warn_on_stray_hosts", 2024-04-24T23:32:00.887 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:00.888 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.888 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:00.888 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "True", 2024-04-24T23:32:00.888 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.888 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.888 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.889 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-24T23:32:00.889 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.889 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.889 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.889 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:00.889 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:00.889 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.889 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-24T23:32:00.889 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "crash", 2024-04-24T23:32:00.890 INFO:teuthology.orchestra.run.smithi080.stdout: "can_run": true, 2024-04-24T23:32:00.890 INFO:teuthology.orchestra.run.smithi080.stdout: "error_string": "", 2024-04-24T23:32:00.890 INFO:teuthology.orchestra.run.smithi080.stdout: "module_options": { 2024-04-24T23:32:00.890 INFO:teuthology.orchestra.run.smithi080.stdout: "log_level": { 2024-04-24T23:32:00.890 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_level", 2024-04-24T23:32:00.890 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:00.890 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.890 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:00.891 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:00.891 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.891 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.891 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:00.891 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:00.891 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:00.891 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:00.891 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:00.892 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:00.892 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:00.892 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:00.892 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:00.892 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.892 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.892 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.892 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.892 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster": { 2024-04-24T23:32:00.893 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster", 2024-04-24T23:32:00.893 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:00.893 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.893 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:00.893 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:00.893 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.893 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.893 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.893 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:00.894 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.894 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.894 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.894 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.894 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster_level": { 2024-04-24T23:32:00.894 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster_level", 2024-04-24T23:32:00.894 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:00.894 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.895 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:00.895 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "info", 2024-04-24T23:32:00.895 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.895 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.895 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:00.895 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:00.895 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:00.895 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:00.895 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:00.896 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:00.896 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:00.896 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:00.896 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:00.896 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.896 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.896 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.896 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.896 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_file": { 2024-04-24T23:32:00.897 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_file", 2024-04-24T23:32:00.897 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:00.897 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.897 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:00.897 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:00.897 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.897 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.897 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.897 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:00.898 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.898 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.898 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.898 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.898 INFO:teuthology.orchestra.run.smithi080.stdout: "retain_interval": { 2024-04-24T23:32:00.898 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "retain_interval", 2024-04-24T23:32:00.898 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "secs", 2024-04-24T23:32:00.898 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.899 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:00.899 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "31536000", 2024-04-24T23:32:00.899 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.899 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.899 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.899 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-24T23:32:00.899 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.899 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.899 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.900 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.900 INFO:teuthology.orchestra.run.smithi080.stdout: "warn_recent_interval": { 2024-04-24T23:32:00.900 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "warn_recent_interval", 2024-04-24T23:32:00.900 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "secs", 2024-04-24T23:32:00.900 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.900 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:00.900 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "1209600", 2024-04-24T23:32:00.900 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.900 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.901 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.901 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-24T23:32:00.901 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.901 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.901 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.901 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:00.901 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:00.901 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.901 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-24T23:32:00.902 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "dashboard", 2024-04-24T23:32:00.902 INFO:teuthology.orchestra.run.smithi080.stdout: "can_run": true, 2024-04-24T23:32:00.902 INFO:teuthology.orchestra.run.smithi080.stdout: "error_string": "", 2024-04-24T23:32:00.902 INFO:teuthology.orchestra.run.smithi080.stdout: "module_options": { 2024-04-24T23:32:00.902 INFO:teuthology.orchestra.run.smithi080.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-24T23:32:00.902 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-24T23:32:00.902 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "int", 2024-04-24T23:32:00.902 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.903 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:00.903 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "10", 2024-04-24T23:32:00.903 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.903 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.903 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.903 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:00.903 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.903 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.903 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.904 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.904 INFO:teuthology.orchestra.run.smithi080.stdout: "ALERTMANAGER_API_HOST": { 2024-04-24T23:32:00.904 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-24T23:32:00.904 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:00.904 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.904 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:00.904 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:00.904 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.904 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.905 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.905 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:00.905 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.905 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.905 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.905 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.906 INFO:teuthology.orchestra.run.smithi080.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-24T23:32:00.906 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-24T23:32:00.906 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:00.906 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.906 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:00.906 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "True", 2024-04-24T23:32:00.906 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.907 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.907 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.907 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:00.907 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.907 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.907 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.907 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.907 INFO:teuthology.orchestra.run.smithi080.stdout: "AUDIT_API_ENABLED": { 2024-04-24T23:32:00.907 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "AUDIT_API_ENABLED", 2024-04-24T23:32:00.908 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:00.908 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.908 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:00.908 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:00.908 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.908 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.908 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.908 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:00.908 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.909 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.909 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.909 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.909 INFO:teuthology.orchestra.run.smithi080.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-24T23:32:00.909 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-24T23:32:00.909 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:00.909 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.909 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:00.910 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "True", 2024-04-24T23:32:00.910 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.910 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.910 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.910 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:00.910 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.910 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.910 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.910 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.911 INFO:teuthology.orchestra.run.smithi080.stdout: "ENABLE_BROWSABLE_API": { 2024-04-24T23:32:00.911 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-24T23:32:00.911 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:00.911 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.911 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:00.911 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "True", 2024-04-24T23:32:00.911 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.911 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.911 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.912 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:00.912 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.912 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.912 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.912 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.912 INFO:teuthology.orchestra.run.smithi080.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-24T23:32:00.912 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-24T23:32:00.913 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:00.913 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.913 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:00.913 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "True", 2024-04-24T23:32:00.913 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.913 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.913 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.913 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:00.913 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.914 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.914 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.914 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.914 INFO:teuthology.orchestra.run.smithi080.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-24T23:32:00.914 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-24T23:32:00.914 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:00.914 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.914 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:00.915 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "True", 2024-04-24T23:32:00.915 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.915 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.915 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.915 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:00.915 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.915 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.915 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.915 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.916 INFO:teuthology.orchestra.run.smithi080.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-24T23:32:00.916 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-24T23:32:00.916 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:00.916 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.916 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:00.916 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "True", 2024-04-24T23:32:00.916 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.916 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.916 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.917 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:00.917 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.917 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.917 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.917 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.917 INFO:teuthology.orchestra.run.smithi080.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-24T23:32:00.917 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-24T23:32:00.917 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:00.917 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.918 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:00.918 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "True", 2024-04-24T23:32:00.918 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.918 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.918 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.918 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:00.918 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.918 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.919 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.919 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.919 INFO:teuthology.orchestra.run.smithi080.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-24T23:32:00.919 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-24T23:32:00.919 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:00.919 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.919 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:00.919 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "True", 2024-04-24T23:32:00.919 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.920 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.920 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.920 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:00.920 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.920 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.920 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.920 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.920 INFO:teuthology.orchestra.run.smithi080.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-24T23:32:00.920 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-24T23:32:00.921 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:00.921 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.921 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:00.921 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "True", 2024-04-24T23:32:00.921 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.921 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.921 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.921 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:00.922 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.922 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.922 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.922 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.922 INFO:teuthology.orchestra.run.smithi080.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-24T23:32:00.922 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-24T23:32:00.922 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:00.922 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.923 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:00.923 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "True", 2024-04-24T23:32:00.923 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.923 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.923 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.923 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:00.924 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.924 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.924 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.924 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.924 INFO:teuthology.orchestra.run.smithi080.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-24T23:32:00.924 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-24T23:32:00.924 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:00.924 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.925 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:00.925 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:00.925 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.925 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.925 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.925 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:00.925 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.925 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.925 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.926 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.926 INFO:teuthology.orchestra.run.smithi080.stdout: "GRAFANA_API_PASSWORD": { 2024-04-24T23:32:00.926 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-24T23:32:00.926 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:00.926 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.926 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:00.926 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "admin", 2024-04-24T23:32:00.926 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.927 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.927 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.927 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:00.927 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.927 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.927 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.927 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.927 INFO:teuthology.orchestra.run.smithi080.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-24T23:32:00.927 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-24T23:32:00.928 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:00.928 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.928 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:00.928 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "True", 2024-04-24T23:32:00.928 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.928 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.928 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.928 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:00.928 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.929 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.929 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.929 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.929 INFO:teuthology.orchestra.run.smithi080.stdout: "GRAFANA_API_URL": { 2024-04-24T23:32:00.929 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "GRAFANA_API_URL", 2024-04-24T23:32:00.929 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:00.929 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.929 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:00.930 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:00.930 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.930 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.930 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.930 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:00.930 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.930 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.930 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.930 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.931 INFO:teuthology.orchestra.run.smithi080.stdout: "GRAFANA_API_USERNAME": { 2024-04-24T23:32:00.931 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-24T23:32:00.931 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:00.931 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.931 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:00.931 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "admin", 2024-04-24T23:32:00.931 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.931 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.931 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.932 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:00.932 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.932 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.932 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.932 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.932 INFO:teuthology.orchestra.run.smithi080.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-24T23:32:00.932 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-24T23:32:00.932 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:00.933 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.933 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:00.933 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:00.933 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.933 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.933 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.933 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:00.933 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.933 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.934 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.934 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.934 INFO:teuthology.orchestra.run.smithi080.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-24T23:32:00.934 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-24T23:32:00.934 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:00.934 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.934 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:00.935 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:00.935 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.935 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.935 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.935 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:00.935 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.935 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.935 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.935 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.936 INFO:teuthology.orchestra.run.smithi080.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-24T23:32:00.936 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-24T23:32:00.936 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:00.936 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.936 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:00.936 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "True", 2024-04-24T23:32:00.936 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.936 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.936 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.937 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:00.937 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.937 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.937 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.937 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.937 INFO:teuthology.orchestra.run.smithi080.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-24T23:32:00.937 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-24T23:32:00.937 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:00.937 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.938 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:00.938 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:00.938 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.938 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.938 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.938 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:00.938 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.938 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.939 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.939 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.939 INFO:teuthology.orchestra.run.smithi080.stdout: "PROMETHEUS_API_HOST": { 2024-04-24T23:32:00.939 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-24T23:32:00.939 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:00.939 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.939 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:00.939 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:00.939 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.940 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.940 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.940 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:00.940 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.940 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.940 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.940 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.940 INFO:teuthology.orchestra.run.smithi080.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-24T23:32:00.940 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-24T23:32:00.941 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:00.941 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.941 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:00.941 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "True", 2024-04-24T23:32:00.941 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.941 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.941 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.941 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:00.942 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.942 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.942 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.942 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.942 INFO:teuthology.orchestra.run.smithi080.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-24T23:32:00.942 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-24T23:32:00.942 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:00.942 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.942 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:00.943 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:00.943 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.943 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.943 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.943 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:00.943 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.943 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.943 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.943 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.944 INFO:teuthology.orchestra.run.smithi080.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-24T23:32:00.944 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-24T23:32:00.944 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:00.944 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.944 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:00.944 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:00.944 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.944 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.945 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.945 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:00.945 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.945 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.945 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.945 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.945 INFO:teuthology.orchestra.run.smithi080.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-24T23:32:00.945 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-24T23:32:00.945 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:00.946 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.946 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:00.946 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "True", 2024-04-24T23:32:00.946 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.946 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.947 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.947 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:00.947 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.947 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.947 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.947 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.947 INFO:teuthology.orchestra.run.smithi080.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-24T23:32:00.947 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-24T23:32:00.947 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:00.948 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.948 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:00.948 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "True", 2024-04-24T23:32:00.948 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.948 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.948 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.948 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:00.948 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.949 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.949 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.949 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.949 INFO:teuthology.orchestra.run.smithi080.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-24T23:32:00.949 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-24T23:32:00.949 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:00.949 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.949 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:00.950 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:00.950 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.950 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.950 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.950 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:00.950 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.950 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.951 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.951 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.951 INFO:teuthology.orchestra.run.smithi080.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-24T23:32:00.951 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-24T23:32:00.951 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:00.951 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.951 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:00.951 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:00.951 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.952 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.952 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.952 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:00.952 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.952 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.952 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.952 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.952 INFO:teuthology.orchestra.run.smithi080.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-24T23:32:00.953 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-24T23:32:00.953 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:00.953 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.953 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:00.953 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:00.953 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.953 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.953 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.953 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:00.954 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.954 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.954 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.954 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.954 INFO:teuthology.orchestra.run.smithi080.stdout: "PWD_POLICY_ENABLED": { 2024-04-24T23:32:00.955 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-24T23:32:00.955 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:00.955 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.955 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:00.955 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "True", 2024-04-24T23:32:00.957 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.957 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.958 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.958 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:00.958 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.958 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.958 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.958 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.958 INFO:teuthology.orchestra.run.smithi080.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-24T23:32:00.959 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-24T23:32:00.959 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:00.959 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.959 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:00.959 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-24T23:32:00.959 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.959 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.959 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.960 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:00.960 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.960 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.960 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.960 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.960 INFO:teuthology.orchestra.run.smithi080.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-24T23:32:00.961 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-24T23:32:00.961 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "int", 2024-04-24T23:32:00.961 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.961 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:00.961 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "10", 2024-04-24T23:32:00.961 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.962 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.962 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.962 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:00.962 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.962 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.963 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.963 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.963 INFO:teuthology.orchestra.run.smithi080.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-24T23:32:00.963 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-24T23:32:00.963 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "int", 2024-04-24T23:32:00.963 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.964 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:00.964 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "8", 2024-04-24T23:32:00.964 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.964 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.964 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.964 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:00.964 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.965 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.965 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.965 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.965 INFO:teuthology.orchestra.run.smithi080.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-24T23:32:00.965 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-24T23:32:00.965 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "int", 2024-04-24T23:32:00.965 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.966 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:00.966 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "45", 2024-04-24T23:32:00.966 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.966 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.966 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.966 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:00.966 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.966 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.966 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.967 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.967 INFO:teuthology.orchestra.run.smithi080.stdout: "RGW_API_ACCESS_KEY": { 2024-04-24T23:32:00.967 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-24T23:32:00.967 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:00.967 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.967 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:00.967 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:00.967 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.968 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.968 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.968 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:00.968 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.968 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.968 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.968 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.968 INFO:teuthology.orchestra.run.smithi080.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-24T23:32:00.968 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-24T23:32:00.969 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:00.969 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.969 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:00.969 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "admin", 2024-04-24T23:32:00.969 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.969 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.969 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.969 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:00.970 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.970 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.970 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.970 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.970 INFO:teuthology.orchestra.run.smithi080.stdout: "RGW_API_SECRET_KEY": { 2024-04-24T23:32:00.970 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-24T23:32:00.970 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:00.970 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.970 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:00.971 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:00.971 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.971 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.971 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.971 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:00.971 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.971 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.971 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.972 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.972 INFO:teuthology.orchestra.run.smithi080.stdout: "RGW_API_SSL_VERIFY": { 2024-04-24T23:32:00.972 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-24T23:32:00.972 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:00.972 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.972 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:00.972 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "True", 2024-04-24T23:32:00.972 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.973 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.973 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.973 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:00.973 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.973 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.973 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.973 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.973 INFO:teuthology.orchestra.run.smithi080.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-24T23:32:00.974 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-24T23:32:00.974 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "int", 2024-04-24T23:32:00.974 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.974 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:00.974 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "0", 2024-04-24T23:32:00.974 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.974 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.974 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.974 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:00.975 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.975 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.975 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.975 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.975 INFO:teuthology.orchestra.run.smithi080.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-24T23:32:00.975 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-24T23:32:00.975 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "int", 2024-04-24T23:32:00.975 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.976 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:00.976 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "10", 2024-04-24T23:32:00.976 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.976 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.976 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.976 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:00.976 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.976 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.976 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.977 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.977 INFO:teuthology.orchestra.run.smithi080.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-24T23:32:00.977 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-24T23:32:00.977 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "int", 2024-04-24T23:32:00.977 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.977 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:00.977 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "5", 2024-04-24T23:32:00.977 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.977 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.978 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.978 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:00.978 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.978 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.978 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.978 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.978 INFO:teuthology.orchestra.run.smithi080.stdout: "cross_origin_url": { 2024-04-24T23:32:00.979 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "cross_origin_url", 2024-04-24T23:32:00.979 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:00.979 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.979 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:00.979 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:00.979 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.980 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.980 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.980 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:00.980 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.980 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.980 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.981 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.981 INFO:teuthology.orchestra.run.smithi080.stdout: "crt_file": { 2024-04-24T23:32:00.981 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "crt_file", 2024-04-24T23:32:00.981 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:00.981 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.981 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:00.982 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:00.982 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.982 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.982 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.982 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:00.983 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.983 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.983 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.983 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.983 INFO:teuthology.orchestra.run.smithi080.stdout: "debug": { 2024-04-24T23:32:00.984 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "debug", 2024-04-24T23:32:00.984 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:00.984 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.984 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:00.984 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:00.984 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.985 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.985 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.985 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Enable/disable debug options", 2024-04-24T23:32:00.985 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.985 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.985 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.985 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.985 INFO:teuthology.orchestra.run.smithi080.stdout: "jwt_token_ttl": { 2024-04-24T23:32:00.986 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "jwt_token_ttl", 2024-04-24T23:32:00.986 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "int", 2024-04-24T23:32:00.986 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.986 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:00.986 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "28800", 2024-04-24T23:32:00.986 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.986 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.986 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.987 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:00.987 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.987 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.987 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.987 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.987 INFO:teuthology.orchestra.run.smithi080.stdout: "key_file": { 2024-04-24T23:32:00.988 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "key_file", 2024-04-24T23:32:00.988 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:00.988 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.988 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:00.988 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:00.988 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.988 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.988 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.989 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:00.989 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.989 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.989 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.989 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.989 INFO:teuthology.orchestra.run.smithi080.stdout: "log_level": { 2024-04-24T23:32:00.989 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_level", 2024-04-24T23:32:00.989 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:00.990 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.990 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:00.990 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:00.990 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.990 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.990 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:00.991 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:00.991 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:00.991 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:00.991 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:00.991 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:00.991 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:00.991 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:00.991 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:00.992 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.992 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.992 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.992 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.992 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster": { 2024-04-24T23:32:00.992 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster", 2024-04-24T23:32:00.992 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:00.992 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.993 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:00.993 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:00.993 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.993 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.993 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.993 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:00.993 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.993 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.993 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.994 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.994 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster_level": { 2024-04-24T23:32:00.994 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster_level", 2024-04-24T23:32:00.994 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:00.994 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.994 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:00.994 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "info", 2024-04-24T23:32:00.994 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.995 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.995 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:00.995 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:00.995 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:00.995 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:00.995 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:00.995 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:00.995 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:00.996 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:00.996 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:00.996 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.996 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.996 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.996 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.996 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_file": { 2024-04-24T23:32:00.996 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_file", 2024-04-24T23:32:00.996 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:00.997 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.997 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:00.997 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:00.997 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:00.997 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:00.997 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:00.997 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:00.998 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:00.998 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:00.998 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:00.998 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:00.998 INFO:teuthology.orchestra.run.smithi080.stdout: "motd": { 2024-04-24T23:32:00.999 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "motd", 2024-04-24T23:32:00.999 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:00.999 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:00.999 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:00.999 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:00.999 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.000 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.000 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.000 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "The message of the day", 2024-04-24T23:32:01.000 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.000 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.001 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.001 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.001 INFO:teuthology.orchestra.run.smithi080.stdout: "redirect_resolve_ip_addr": { 2024-04-24T23:32:01.001 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "redirect_resolve_ip_addr", 2024-04-24T23:32:01.001 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.002 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.002 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.002 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.002 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.002 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.002 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.003 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.003 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.003 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.003 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.003 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.003 INFO:teuthology.orchestra.run.smithi080.stdout: "server_addr": { 2024-04-24T23:32:01.003 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "server_addr", 2024-04-24T23:32:01.003 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.004 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.004 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.004 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "::", 2024-04-24T23:32:01.004 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.004 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.004 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.004 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.004 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.004 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.005 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.005 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.005 INFO:teuthology.orchestra.run.smithi080.stdout: "server_port": { 2024-04-24T23:32:01.005 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "server_port", 2024-04-24T23:32:01.005 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "int", 2024-04-24T23:32:01.005 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.005 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.005 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "8080", 2024-04-24T23:32:01.006 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.006 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.006 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.006 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.006 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.006 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.007 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.007 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.007 INFO:teuthology.orchestra.run.smithi080.stdout: "ssl": { 2024-04-24T23:32:01.007 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "ssl", 2024-04-24T23:32:01.007 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.007 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.007 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.007 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "True", 2024-04-24T23:32:01.008 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.008 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.008 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.008 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.008 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.008 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.008 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.008 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.008 INFO:teuthology.orchestra.run.smithi080.stdout: "ssl_server_port": { 2024-04-24T23:32:01.009 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "ssl_server_port", 2024-04-24T23:32:01.009 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "int", 2024-04-24T23:32:01.009 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.009 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.009 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "8443", 2024-04-24T23:32:01.009 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.009 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.009 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.009 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.010 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.010 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.010 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.010 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.010 INFO:teuthology.orchestra.run.smithi080.stdout: "standby_behaviour": { 2024-04-24T23:32:01.010 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "standby_behaviour", 2024-04-24T23:32:01.010 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.011 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.011 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.011 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "redirect", 2024-04-24T23:32:01.011 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.011 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.011 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:01.011 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:01.011 INFO:teuthology.orchestra.run.smithi080.stdout: "redirect" 2024-04-24T23:32:01.011 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:01.012 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.012 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.012 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.012 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.012 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.012 INFO:teuthology.orchestra.run.smithi080.stdout: "standby_error_status_code": { 2024-04-24T23:32:01.012 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "standby_error_status_code", 2024-04-24T23:32:01.012 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "int", 2024-04-24T23:32:01.012 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.013 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.013 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "500", 2024-04-24T23:32:01.013 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "400", 2024-04-24T23:32:01.013 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "599", 2024-04-24T23:32:01.013 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.013 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.013 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.013 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.014 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.014 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.014 INFO:teuthology.orchestra.run.smithi080.stdout: "url_prefix": { 2024-04-24T23:32:01.014 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "url_prefix", 2024-04-24T23:32:01.014 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.014 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.014 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.014 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.014 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.015 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.015 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.015 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.015 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.015 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.015 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.015 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:01.015 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:01.015 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.016 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-24T23:32:01.016 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "devicehealth", 2024-04-24T23:32:01.016 INFO:teuthology.orchestra.run.smithi080.stdout: "can_run": true, 2024-04-24T23:32:01.016 INFO:teuthology.orchestra.run.smithi080.stdout: "error_string": "", 2024-04-24T23:32:01.016 INFO:teuthology.orchestra.run.smithi080.stdout: "module_options": { 2024-04-24T23:32:01.016 INFO:teuthology.orchestra.run.smithi080.stdout: "enable_monitoring": { 2024-04-24T23:32:01.016 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "enable_monitoring", 2024-04-24T23:32:01.016 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.017 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.017 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.017 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "True", 2024-04-24T23:32:01.017 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.017 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.017 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.017 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "monitor device health metrics", 2024-04-24T23:32:01.017 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.017 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.018 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.018 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.018 INFO:teuthology.orchestra.run.smithi080.stdout: "log_level": { 2024-04-24T23:32:01.018 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_level", 2024-04-24T23:32:01.018 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.018 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.018 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.018 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.018 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.019 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.019 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:01.019 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:01.019 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:01.019 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:01.019 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:01.019 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:01.020 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:01.020 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:01.020 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.020 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.020 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.020 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.020 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.020 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster": { 2024-04-24T23:32:01.020 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster", 2024-04-24T23:32:01.021 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.021 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.021 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.021 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.021 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.021 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.021 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.021 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.021 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.022 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.022 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.022 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.022 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster_level": { 2024-04-24T23:32:01.022 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster_level", 2024-04-24T23:32:01.022 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.022 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.022 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.023 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "info", 2024-04-24T23:32:01.023 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.023 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.023 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:01.023 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:01.023 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:01.023 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:01.023 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:01.023 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:01.024 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:01.024 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:01.024 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.024 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.024 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.024 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.024 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.024 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_file": { 2024-04-24T23:32:01.024 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_file", 2024-04-24T23:32:01.025 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.025 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.025 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.025 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.025 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.025 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.025 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.025 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.025 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.026 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.026 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.026 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.026 INFO:teuthology.orchestra.run.smithi080.stdout: "mark_out_threshold": { 2024-04-24T23:32:01.026 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "mark_out_threshold", 2024-04-24T23:32:01.026 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "secs", 2024-04-24T23:32:01.026 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.026 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.027 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "2419200", 2024-04-24T23:32:01.027 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.027 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.027 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.027 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-24T23:32:01.027 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.027 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.027 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.027 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.028 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_name": { 2024-04-24T23:32:01.028 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "pool_name", 2024-04-24T23:32:01.028 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.028 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.028 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.028 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "device_health_metrics", 2024-04-24T23:32:01.028 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.028 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.028 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.029 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-24T23:32:01.029 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.029 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.029 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.029 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.029 INFO:teuthology.orchestra.run.smithi080.stdout: "retention_period": { 2024-04-24T23:32:01.029 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "retention_period", 2024-04-24T23:32:01.029 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "secs", 2024-04-24T23:32:01.029 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.030 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.030 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "15552000", 2024-04-24T23:32:01.030 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.030 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.030 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.030 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "how long to retain device health metrics", 2024-04-24T23:32:01.030 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.030 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.031 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.031 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.031 INFO:teuthology.orchestra.run.smithi080.stdout: "scrape_frequency": { 2024-04-24T23:32:01.031 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "scrape_frequency", 2024-04-24T23:32:01.031 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "secs", 2024-04-24T23:32:01.031 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.031 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.031 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "86400", 2024-04-24T23:32:01.031 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.032 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.032 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.032 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-24T23:32:01.032 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.032 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.032 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.032 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.033 INFO:teuthology.orchestra.run.smithi080.stdout: "self_heal": { 2024-04-24T23:32:01.033 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "self_heal", 2024-04-24T23:32:01.033 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.033 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.033 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.033 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "True", 2024-04-24T23:32:01.033 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.033 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.033 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.034 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-24T23:32:01.034 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.034 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.034 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.034 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.034 INFO:teuthology.orchestra.run.smithi080.stdout: "sleep_interval": { 2024-04-24T23:32:01.034 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "sleep_interval", 2024-04-24T23:32:01.034 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "secs", 2024-04-24T23:32:01.034 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.035 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.035 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "600", 2024-04-24T23:32:01.035 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.035 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.035 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.035 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "how frequently to wake up and check device health", 2024-04-24T23:32:01.035 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.035 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.036 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.036 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.036 INFO:teuthology.orchestra.run.smithi080.stdout: "warn_threshold": { 2024-04-24T23:32:01.036 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "warn_threshold", 2024-04-24T23:32:01.036 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "secs", 2024-04-24T23:32:01.036 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.036 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.036 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "7257600", 2024-04-24T23:32:01.036 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.037 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.037 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.037 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-24T23:32:01.037 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.037 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.037 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.037 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:01.037 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:01.037 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.038 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-24T23:32:01.038 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "diskprediction_local", 2024-04-24T23:32:01.038 INFO:teuthology.orchestra.run.smithi080.stdout: "can_run": true, 2024-04-24T23:32:01.038 INFO:teuthology.orchestra.run.smithi080.stdout: "error_string": "", 2024-04-24T23:32:01.038 INFO:teuthology.orchestra.run.smithi080.stdout: "module_options": { 2024-04-24T23:32:01.038 INFO:teuthology.orchestra.run.smithi080.stdout: "log_level": { 2024-04-24T23:32:01.038 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_level", 2024-04-24T23:32:01.038 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.039 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.039 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.039 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.039 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.039 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.039 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:01.039 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:01.039 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:01.039 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:01.040 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:01.040 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:01.040 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:01.040 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:01.040 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.040 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.040 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.040 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.040 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.041 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster": { 2024-04-24T23:32:01.041 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster", 2024-04-24T23:32:01.041 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.041 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.041 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.041 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.041 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.041 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.041 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.042 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.042 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.042 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.042 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.042 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.042 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster_level": { 2024-04-24T23:32:01.042 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster_level", 2024-04-24T23:32:01.042 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.043 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.043 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.043 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "info", 2024-04-24T23:32:01.043 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.043 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.043 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:01.043 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:01.043 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:01.043 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:01.044 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:01.044 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:01.044 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:01.044 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:01.044 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.044 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.044 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.044 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.044 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.045 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_file": { 2024-04-24T23:32:01.045 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_file", 2024-04-24T23:32:01.045 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.045 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.045 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.045 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.045 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.045 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.046 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.046 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.046 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.046 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.046 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.046 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.046 INFO:teuthology.orchestra.run.smithi080.stdout: "predict_interval": { 2024-04-24T23:32:01.046 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "predict_interval", 2024-04-24T23:32:01.046 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.047 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.047 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.047 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "86400", 2024-04-24T23:32:01.047 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.047 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.047 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.047 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.047 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.048 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.048 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.048 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.048 INFO:teuthology.orchestra.run.smithi080.stdout: "predictor_model": { 2024-04-24T23:32:01.048 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "predictor_model", 2024-04-24T23:32:01.048 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.048 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.048 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.048 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "prophetstor", 2024-04-24T23:32:01.049 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.049 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.049 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.049 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.049 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.049 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.049 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.049 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.049 INFO:teuthology.orchestra.run.smithi080.stdout: "sleep_interval": { 2024-04-24T23:32:01.050 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "sleep_interval", 2024-04-24T23:32:01.050 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.050 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.050 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.050 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "600", 2024-04-24T23:32:01.050 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.050 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.050 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.050 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.051 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.051 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.051 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.051 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:01.051 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:01.051 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.051 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-24T23:32:01.051 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "influx", 2024-04-24T23:32:01.052 INFO:teuthology.orchestra.run.smithi080.stdout: "can_run": false, 2024-04-24T23:32:01.052 INFO:teuthology.orchestra.run.smithi080.stdout: "error_string": "influxdb python module not found", 2024-04-24T23:32:01.052 INFO:teuthology.orchestra.run.smithi080.stdout: "module_options": { 2024-04-24T23:32:01.052 INFO:teuthology.orchestra.run.smithi080.stdout: "batch_size": { 2024-04-24T23:32:01.052 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "batch_size", 2024-04-24T23:32:01.052 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "int", 2024-04-24T23:32:01.052 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.052 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.052 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "5000", 2024-04-24T23:32:01.053 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.053 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.053 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.053 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-24T23:32:01.053 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.053 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.053 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.053 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.054 INFO:teuthology.orchestra.run.smithi080.stdout: "database": { 2024-04-24T23:32:01.054 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "database", 2024-04-24T23:32:01.054 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.054 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.054 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.054 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "ceph", 2024-04-24T23:32:01.054 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.054 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.055 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.055 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-24T23:32:01.055 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.055 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.055 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.055 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.055 INFO:teuthology.orchestra.run.smithi080.stdout: "hostname": { 2024-04-24T23:32:01.055 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "hostname", 2024-04-24T23:32:01.055 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.056 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.056 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.056 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.056 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.056 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.056 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.056 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "InfluxDB server hostname", 2024-04-24T23:32:01.056 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.056 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.057 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.057 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.057 INFO:teuthology.orchestra.run.smithi080.stdout: "interval": { 2024-04-24T23:32:01.057 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "interval", 2024-04-24T23:32:01.057 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "secs", 2024-04-24T23:32:01.057 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.057 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.057 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "30", 2024-04-24T23:32:01.057 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "5", 2024-04-24T23:32:01.058 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.058 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.058 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-24T23:32:01.058 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.058 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.058 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.058 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.058 INFO:teuthology.orchestra.run.smithi080.stdout: "log_level": { 2024-04-24T23:32:01.059 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_level", 2024-04-24T23:32:01.059 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.059 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.059 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.059 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.059 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.059 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.059 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:01.059 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:01.060 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:01.060 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:01.060 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:01.060 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:01.060 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:01.060 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:01.060 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.060 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.060 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.061 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.061 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.061 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster": { 2024-04-24T23:32:01.061 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster", 2024-04-24T23:32:01.061 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.061 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.061 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.061 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.062 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.062 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.062 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.062 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.062 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.062 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.062 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.062 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.062 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster_level": { 2024-04-24T23:32:01.063 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster_level", 2024-04-24T23:32:01.063 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.063 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.063 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.063 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "info", 2024-04-24T23:32:01.063 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.063 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.063 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:01.063 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:01.064 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:01.064 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:01.064 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:01.064 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:01.064 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:01.064 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:01.064 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.064 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.065 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.065 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.065 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.065 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_file": { 2024-04-24T23:32:01.065 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_file", 2024-04-24T23:32:01.065 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.065 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.065 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.065 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.066 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.066 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.066 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.066 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.066 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.066 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.066 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.066 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.066 INFO:teuthology.orchestra.run.smithi080.stdout: "password": { 2024-04-24T23:32:01.067 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "password", 2024-04-24T23:32:01.067 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.067 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.067 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.067 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.067 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.067 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.067 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.067 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "password of InfluxDB server user", 2024-04-24T23:32:01.068 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.068 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.068 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.068 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.068 INFO:teuthology.orchestra.run.smithi080.stdout: "port": { 2024-04-24T23:32:01.068 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "port", 2024-04-24T23:32:01.068 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "int", 2024-04-24T23:32:01.068 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.069 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.069 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "8086", 2024-04-24T23:32:01.069 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.069 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.069 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.069 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "InfluxDB server port", 2024-04-24T23:32:01.069 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.069 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.069 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.070 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.070 INFO:teuthology.orchestra.run.smithi080.stdout: "ssl": { 2024-04-24T23:32:01.070 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "ssl", 2024-04-24T23:32:01.070 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.070 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.070 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.070 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "false", 2024-04-24T23:32:01.070 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.070 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.071 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.071 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-24T23:32:01.071 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.071 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.071 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.071 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.071 INFO:teuthology.orchestra.run.smithi080.stdout: "threads": { 2024-04-24T23:32:01.071 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "threads", 2024-04-24T23:32:01.071 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "int", 2024-04-24T23:32:01.072 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.072 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.072 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "5", 2024-04-24T23:32:01.072 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "1", 2024-04-24T23:32:01.072 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "32", 2024-04-24T23:32:01.072 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.072 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-24T23:32:01.072 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.073 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.073 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.073 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.073 INFO:teuthology.orchestra.run.smithi080.stdout: "username": { 2024-04-24T23:32:01.073 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "username", 2024-04-24T23:32:01.073 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.073 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.073 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.073 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.074 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.074 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.074 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.074 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "username of InfluxDB server user", 2024-04-24T23:32:01.074 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.074 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.074 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.074 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.075 INFO:teuthology.orchestra.run.smithi080.stdout: "verify_ssl": { 2024-04-24T23:32:01.075 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "verify_ssl", 2024-04-24T23:32:01.075 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.075 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.075 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.075 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "true", 2024-04-24T23:32:01.075 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.075 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.076 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.076 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-24T23:32:01.076 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.076 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.076 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.076 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:01.076 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:01.076 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.076 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-24T23:32:01.077 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "insights", 2024-04-24T23:32:01.077 INFO:teuthology.orchestra.run.smithi080.stdout: "can_run": true, 2024-04-24T23:32:01.077 INFO:teuthology.orchestra.run.smithi080.stdout: "error_string": "", 2024-04-24T23:32:01.077 INFO:teuthology.orchestra.run.smithi080.stdout: "module_options": { 2024-04-24T23:32:01.077 INFO:teuthology.orchestra.run.smithi080.stdout: "log_level": { 2024-04-24T23:32:01.077 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_level", 2024-04-24T23:32:01.077 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.077 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.078 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.078 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.078 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.078 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.078 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:01.078 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:01.078 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:01.078 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:01.078 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:01.079 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:01.079 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:01.079 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:01.079 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.079 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.079 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.079 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.079 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.080 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster": { 2024-04-24T23:32:01.080 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster", 2024-04-24T23:32:01.080 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.080 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.080 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.080 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.080 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.080 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.080 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.081 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.081 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.081 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.081 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.081 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.081 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster_level": { 2024-04-24T23:32:01.081 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster_level", 2024-04-24T23:32:01.081 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.081 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.082 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.082 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "info", 2024-04-24T23:32:01.082 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.082 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.082 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:01.082 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:01.082 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:01.082 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:01.083 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:01.083 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:01.083 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:01.083 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:01.083 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.083 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.083 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.083 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.083 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.084 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_file": { 2024-04-24T23:32:01.084 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_file", 2024-04-24T23:32:01.084 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.084 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.084 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.084 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.084 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.084 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.084 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.085 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.085 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.085 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.085 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.085 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:01.085 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:01.085 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.085 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-24T23:32:01.085 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "iostat", 2024-04-24T23:32:01.086 INFO:teuthology.orchestra.run.smithi080.stdout: "can_run": true, 2024-04-24T23:32:01.086 INFO:teuthology.orchestra.run.smithi080.stdout: "error_string": "", 2024-04-24T23:32:01.086 INFO:teuthology.orchestra.run.smithi080.stdout: "module_options": { 2024-04-24T23:32:01.086 INFO:teuthology.orchestra.run.smithi080.stdout: "log_level": { 2024-04-24T23:32:01.086 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_level", 2024-04-24T23:32:01.086 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.086 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.086 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.087 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.087 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.087 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.087 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:01.087 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:01.087 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:01.087 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:01.087 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:01.087 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:01.088 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:01.088 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:01.088 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.088 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.088 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.088 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.088 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.088 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster": { 2024-04-24T23:32:01.088 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster", 2024-04-24T23:32:01.089 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.089 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.089 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.089 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.089 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.089 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.089 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.089 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.090 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.090 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.090 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.090 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.090 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster_level": { 2024-04-24T23:32:01.090 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster_level", 2024-04-24T23:32:01.090 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.090 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.090 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.091 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "info", 2024-04-24T23:32:01.091 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.091 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.091 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:01.091 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:01.091 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:01.091 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:01.091 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:01.092 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:01.092 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:01.092 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:01.092 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.092 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.092 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.092 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.092 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.092 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_file": { 2024-04-24T23:32:01.093 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_file", 2024-04-24T23:32:01.093 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.093 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.093 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.093 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.093 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.093 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.093 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.093 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.094 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.094 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.094 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.094 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:01.094 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:01.094 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.094 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-24T23:32:01.094 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "k8sevents", 2024-04-24T23:32:01.094 INFO:teuthology.orchestra.run.smithi080.stdout: "can_run": true, 2024-04-24T23:32:01.095 INFO:teuthology.orchestra.run.smithi080.stdout: "error_string": "", 2024-04-24T23:32:01.095 INFO:teuthology.orchestra.run.smithi080.stdout: "module_options": { 2024-04-24T23:32:01.095 INFO:teuthology.orchestra.run.smithi080.stdout: "ceph_event_retention_days": { 2024-04-24T23:32:01.095 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "ceph_event_retention_days", 2024-04-24T23:32:01.095 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "int", 2024-04-24T23:32:01.095 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.095 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.095 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "7", 2024-04-24T23:32:01.096 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.096 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.096 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.096 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-24T23:32:01.096 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.096 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.096 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.096 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.096 INFO:teuthology.orchestra.run.smithi080.stdout: "config_check_secs": { 2024-04-24T23:32:01.097 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "config_check_secs", 2024-04-24T23:32:01.097 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "int", 2024-04-24T23:32:01.097 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.097 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.097 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "10", 2024-04-24T23:32:01.097 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "10", 2024-04-24T23:32:01.097 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.097 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.098 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-24T23:32:01.098 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.098 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.098 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.098 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.098 INFO:teuthology.orchestra.run.smithi080.stdout: "log_level": { 2024-04-24T23:32:01.098 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_level", 2024-04-24T23:32:01.098 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.099 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.099 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.099 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.099 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.099 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.099 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:01.099 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:01.099 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:01.099 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:01.100 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:01.100 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:01.100 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:01.100 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:01.100 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.100 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.100 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.100 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.100 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.101 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster": { 2024-04-24T23:32:01.101 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster", 2024-04-24T23:32:01.101 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.101 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.101 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.101 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.101 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.101 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.101 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.102 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.102 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.102 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.102 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.102 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.102 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster_level": { 2024-04-24T23:32:01.102 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster_level", 2024-04-24T23:32:01.102 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.103 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.103 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.103 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "info", 2024-04-24T23:32:01.103 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.103 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.103 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:01.103 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:01.103 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:01.103 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:01.104 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:01.104 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:01.104 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:01.104 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:01.104 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.104 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.104 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.104 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.104 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.105 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_file": { 2024-04-24T23:32:01.105 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_file", 2024-04-24T23:32:01.105 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.105 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.105 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.105 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.105 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.105 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.105 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.106 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.106 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.106 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.106 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.106 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:01.106 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:01.106 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.106 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-24T23:32:01.107 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "localpool", 2024-04-24T23:32:01.107 INFO:teuthology.orchestra.run.smithi080.stdout: "can_run": true, 2024-04-24T23:32:01.107 INFO:teuthology.orchestra.run.smithi080.stdout: "error_string": "", 2024-04-24T23:32:01.107 INFO:teuthology.orchestra.run.smithi080.stdout: "module_options": { 2024-04-24T23:32:01.107 INFO:teuthology.orchestra.run.smithi080.stdout: "failure_domain": { 2024-04-24T23:32:01.107 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "failure_domain", 2024-04-24T23:32:01.107 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.107 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.108 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.108 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "host", 2024-04-24T23:32:01.108 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.108 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.108 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.108 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "failure domain for any created local pool", 2024-04-24T23:32:01.109 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-24T23:32:01.109 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.109 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.109 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.109 INFO:teuthology.orchestra.run.smithi080.stdout: "log_level": { 2024-04-24T23:32:01.109 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_level", 2024-04-24T23:32:01.109 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.109 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.110 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.110 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.110 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.110 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.110 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:01.110 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:01.110 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:01.110 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:01.110 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:01.111 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:01.111 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:01.111 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:01.111 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.111 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.111 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.111 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.111 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.111 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster": { 2024-04-24T23:32:01.112 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster", 2024-04-24T23:32:01.112 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.112 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.112 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.112 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.112 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.112 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.112 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.113 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.113 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.113 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.113 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.113 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.113 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster_level": { 2024-04-24T23:32:01.113 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster_level", 2024-04-24T23:32:01.113 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.113 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.114 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.114 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "info", 2024-04-24T23:32:01.114 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.114 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.114 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:01.114 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:01.114 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:01.114 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:01.114 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:01.115 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:01.115 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:01.115 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:01.115 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.115 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.115 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.115 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.115 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.116 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_file": { 2024-04-24T23:32:01.116 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_file", 2024-04-24T23:32:01.116 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.116 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.116 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.116 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.116 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.116 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.116 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.117 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.117 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.117 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.117 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.117 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.117 INFO:teuthology.orchestra.run.smithi080.stdout: "min_size": { 2024-04-24T23:32:01.117 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "min_size", 2024-04-24T23:32:01.117 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "int", 2024-04-24T23:32:01.117 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.118 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.118 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.118 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.118 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.118 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.118 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "default min_size for any created local pool", 2024-04-24T23:32:01.118 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-24T23:32:01.118 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.118 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.119 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.119 INFO:teuthology.orchestra.run.smithi080.stdout: "num_rep": { 2024-04-24T23:32:01.119 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "num_rep", 2024-04-24T23:32:01.119 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "int", 2024-04-24T23:32:01.119 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.119 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.119 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "3", 2024-04-24T23:32:01.119 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.120 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.120 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.120 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "default replica count for any created local pool", 2024-04-24T23:32:01.120 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.120 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.120 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.120 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.120 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num": { 2024-04-24T23:32:01.121 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "pg_num", 2024-04-24T23:32:01.121 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "int", 2024-04-24T23:32:01.121 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.121 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.121 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "128", 2024-04-24T23:32:01.121 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.121 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.121 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.121 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "default pg_num for any created local pool", 2024-04-24T23:32:01.122 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.122 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.122 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.122 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.122 INFO:teuthology.orchestra.run.smithi080.stdout: "prefix": { 2024-04-24T23:32:01.122 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "prefix", 2024-04-24T23:32:01.122 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.122 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.122 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.123 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.123 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.123 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.123 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.123 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "name prefix for any created local pool", 2024-04-24T23:32:01.123 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.123 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.123 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.124 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.124 INFO:teuthology.orchestra.run.smithi080.stdout: "subtree": { 2024-04-24T23:32:01.124 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "subtree", 2024-04-24T23:32:01.124 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.124 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.124 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.124 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "rack", 2024-04-24T23:32:01.124 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.124 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.125 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.125 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-24T23:32:01.125 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-24T23:32:01.125 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.125 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.125 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:01.125 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:01.125 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.125 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-24T23:32:01.126 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "mds_autoscaler", 2024-04-24T23:32:01.126 INFO:teuthology.orchestra.run.smithi080.stdout: "can_run": true, 2024-04-24T23:32:01.126 INFO:teuthology.orchestra.run.smithi080.stdout: "error_string": "", 2024-04-24T23:32:01.126 INFO:teuthology.orchestra.run.smithi080.stdout: "module_options": { 2024-04-24T23:32:01.126 INFO:teuthology.orchestra.run.smithi080.stdout: "log_level": { 2024-04-24T23:32:01.126 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_level", 2024-04-24T23:32:01.126 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.126 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.126 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.127 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.127 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.127 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.127 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:01.127 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:01.127 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:01.127 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:01.127 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:01.127 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:01.128 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:01.128 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:01.128 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.128 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.128 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.128 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.128 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.128 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster": { 2024-04-24T23:32:01.128 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster", 2024-04-24T23:32:01.129 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.129 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.129 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.129 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.129 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.129 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.129 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.129 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.130 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.130 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.130 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.130 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.130 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster_level": { 2024-04-24T23:32:01.130 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster_level", 2024-04-24T23:32:01.130 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.130 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.130 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.131 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "info", 2024-04-24T23:32:01.131 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.131 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.131 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:01.131 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:01.131 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:01.131 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:01.131 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:01.131 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:01.132 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:01.132 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:01.132 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.132 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.132 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.132 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.132 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.132 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_file": { 2024-04-24T23:32:01.132 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_file", 2024-04-24T23:32:01.133 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.133 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.133 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.133 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.133 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.133 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.133 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.133 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.134 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.134 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.134 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.134 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:01.134 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:01.134 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.134 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-24T23:32:01.134 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "mirroring", 2024-04-24T23:32:01.134 INFO:teuthology.orchestra.run.smithi080.stdout: "can_run": true, 2024-04-24T23:32:01.135 INFO:teuthology.orchestra.run.smithi080.stdout: "error_string": "", 2024-04-24T23:32:01.135 INFO:teuthology.orchestra.run.smithi080.stdout: "module_options": { 2024-04-24T23:32:01.135 INFO:teuthology.orchestra.run.smithi080.stdout: "log_level": { 2024-04-24T23:32:01.135 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_level", 2024-04-24T23:32:01.135 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.135 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.135 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.135 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.135 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.136 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.136 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:01.136 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:01.136 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:01.136 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:01.136 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:01.136 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:01.136 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:01.137 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:01.137 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.137 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.137 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.137 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.137 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.137 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster": { 2024-04-24T23:32:01.137 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster", 2024-04-24T23:32:01.137 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.138 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.138 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.138 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.138 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.138 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.138 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.138 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.138 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.138 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.139 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.139 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.139 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster_level": { 2024-04-24T23:32:01.139 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster_level", 2024-04-24T23:32:01.139 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.139 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.139 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.139 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "info", 2024-04-24T23:32:01.139 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.140 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.140 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:01.140 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:01.140 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:01.140 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:01.140 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:01.140 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:01.140 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:01.141 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:01.141 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.141 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.141 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.141 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.141 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.141 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_file": { 2024-04-24T23:32:01.141 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_file", 2024-04-24T23:32:01.141 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.142 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.142 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.142 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.142 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.142 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.142 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.142 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.142 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.143 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.143 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.143 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:01.143 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:01.143 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.143 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-24T23:32:01.143 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "nfs", 2024-04-24T23:32:01.143 INFO:teuthology.orchestra.run.smithi080.stdout: "can_run": true, 2024-04-24T23:32:01.144 INFO:teuthology.orchestra.run.smithi080.stdout: "error_string": "", 2024-04-24T23:32:01.144 INFO:teuthology.orchestra.run.smithi080.stdout: "module_options": { 2024-04-24T23:32:01.144 INFO:teuthology.orchestra.run.smithi080.stdout: "log_level": { 2024-04-24T23:32:01.144 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_level", 2024-04-24T23:32:01.144 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.144 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.144 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.144 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.144 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.145 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.145 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:01.145 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:01.145 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:01.145 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:01.145 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:01.145 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:01.145 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:01.145 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:01.146 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.146 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.146 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.146 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.146 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.146 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster": { 2024-04-24T23:32:01.146 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster", 2024-04-24T23:32:01.146 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.147 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.147 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.147 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.147 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.147 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.147 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.147 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.147 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.147 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.148 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.148 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.148 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster_level": { 2024-04-24T23:32:01.148 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster_level", 2024-04-24T23:32:01.148 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.148 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.148 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.148 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "info", 2024-04-24T23:32:01.148 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.149 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.149 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:01.149 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:01.149 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:01.149 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:01.149 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:01.149 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:01.149 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:01.149 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:01.150 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.150 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.150 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.150 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.150 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.150 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_file": { 2024-04-24T23:32:01.150 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_file", 2024-04-24T23:32:01.150 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.151 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.151 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.151 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.151 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.151 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.151 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.151 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.151 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.151 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.152 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.152 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:01.152 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:01.152 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.152 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-24T23:32:01.152 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "orchestrator", 2024-04-24T23:32:01.152 INFO:teuthology.orchestra.run.smithi080.stdout: "can_run": true, 2024-04-24T23:32:01.152 INFO:teuthology.orchestra.run.smithi080.stdout: "error_string": "", 2024-04-24T23:32:01.152 INFO:teuthology.orchestra.run.smithi080.stdout: "module_options": { 2024-04-24T23:32:01.153 INFO:teuthology.orchestra.run.smithi080.stdout: "fail_fs": { 2024-04-24T23:32:01.153 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "fail_fs", 2024-04-24T23:32:01.153 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.153 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.153 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.153 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.153 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.153 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.153 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.154 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-24T23:32:01.154 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.154 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.154 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.154 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.154 INFO:teuthology.orchestra.run.smithi080.stdout: "log_level": { 2024-04-24T23:32:01.154 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_level", 2024-04-24T23:32:01.154 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.155 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.155 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.155 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.155 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.155 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.155 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:01.155 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:01.155 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:01.155 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:01.156 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:01.156 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:01.156 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:01.156 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:01.156 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.156 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.156 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.156 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.156 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.157 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster": { 2024-04-24T23:32:01.157 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster", 2024-04-24T23:32:01.157 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.157 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.157 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.157 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.157 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.157 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.157 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.158 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.158 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.158 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.158 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.158 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.158 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster_level": { 2024-04-24T23:32:01.158 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster_level", 2024-04-24T23:32:01.158 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.159 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.159 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.159 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "info", 2024-04-24T23:32:01.159 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.159 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.159 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:01.159 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:01.159 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:01.159 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:01.160 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:01.160 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:01.160 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:01.160 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:01.160 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.160 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.160 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.160 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.160 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.161 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_file": { 2024-04-24T23:32:01.161 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_file", 2024-04-24T23:32:01.161 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.161 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.161 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.161 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.161 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.161 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.162 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.162 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.162 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.162 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.162 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.162 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.162 INFO:teuthology.orchestra.run.smithi080.stdout: "orchestrator": { 2024-04-24T23:32:01.162 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "orchestrator", 2024-04-24T23:32:01.162 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.163 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.163 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.163 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.163 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.163 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.163 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:01.163 INFO:teuthology.orchestra.run.smithi080.stdout: "cephadm", 2024-04-24T23:32:01.163 INFO:teuthology.orchestra.run.smithi080.stdout: "rook", 2024-04-24T23:32:01.163 INFO:teuthology.orchestra.run.smithi080.stdout: "test_orchestrator" 2024-04-24T23:32:01.164 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:01.164 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Orchestrator backend", 2024-04-24T23:32:01.164 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.164 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.164 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.164 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:01.164 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:01.165 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.165 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-24T23:32:01.165 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "osd_perf_query", 2024-04-24T23:32:01.165 INFO:teuthology.orchestra.run.smithi080.stdout: "can_run": true, 2024-04-24T23:32:01.165 INFO:teuthology.orchestra.run.smithi080.stdout: "error_string": "", 2024-04-24T23:32:01.165 INFO:teuthology.orchestra.run.smithi080.stdout: "module_options": { 2024-04-24T23:32:01.165 INFO:teuthology.orchestra.run.smithi080.stdout: "log_level": { 2024-04-24T23:32:01.165 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_level", 2024-04-24T23:32:01.165 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.166 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.166 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.166 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.166 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.166 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.166 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:01.166 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:01.166 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:01.167 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:01.167 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:01.167 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:01.167 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:01.167 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:01.167 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.167 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.167 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.167 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.168 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.168 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster": { 2024-04-24T23:32:01.168 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster", 2024-04-24T23:32:01.168 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.168 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.168 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.168 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.168 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.168 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.169 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.169 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.169 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.169 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.169 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.169 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.169 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster_level": { 2024-04-24T23:32:01.169 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster_level", 2024-04-24T23:32:01.169 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.170 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.170 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.170 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "info", 2024-04-24T23:32:01.170 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.170 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.170 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:01.170 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:01.170 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:01.171 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:01.171 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:01.171 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:01.171 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:01.171 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:01.171 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.171 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.171 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.171 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.172 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.172 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_file": { 2024-04-24T23:32:01.172 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_file", 2024-04-24T23:32:01.172 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.172 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.172 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.172 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.172 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.172 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.173 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.173 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.173 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.173 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.173 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.173 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:01.173 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:01.173 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.174 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-24T23:32:01.174 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "osd_support", 2024-04-24T23:32:01.174 INFO:teuthology.orchestra.run.smithi080.stdout: "can_run": true, 2024-04-24T23:32:01.174 INFO:teuthology.orchestra.run.smithi080.stdout: "error_string": "", 2024-04-24T23:32:01.174 INFO:teuthology.orchestra.run.smithi080.stdout: "module_options": { 2024-04-24T23:32:01.174 INFO:teuthology.orchestra.run.smithi080.stdout: "log_level": { 2024-04-24T23:32:01.174 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_level", 2024-04-24T23:32:01.174 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.174 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.175 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.175 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.175 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.175 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.175 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:01.175 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:01.175 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:01.175 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:01.175 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:01.176 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:01.176 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:01.176 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:01.176 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.176 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.176 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.176 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.176 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.177 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster": { 2024-04-24T23:32:01.177 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster", 2024-04-24T23:32:01.177 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.177 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.177 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.177 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.177 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.177 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.177 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.178 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.178 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.178 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.178 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.178 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.178 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster_level": { 2024-04-24T23:32:01.178 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster_level", 2024-04-24T23:32:01.178 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.178 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.179 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.179 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "info", 2024-04-24T23:32:01.179 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.179 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.179 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:01.179 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:01.179 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:01.179 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:01.180 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:01.180 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:01.180 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:01.180 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:01.180 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.180 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.180 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.180 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.180 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.181 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_file": { 2024-04-24T23:32:01.181 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_file", 2024-04-24T23:32:01.181 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.181 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.181 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.181 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.181 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.181 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.181 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.182 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.182 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.182 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.182 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.182 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:01.182 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:01.182 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.182 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-24T23:32:01.183 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "pg_autoscaler", 2024-04-24T23:32:01.183 INFO:teuthology.orchestra.run.smithi080.stdout: "can_run": true, 2024-04-24T23:32:01.183 INFO:teuthology.orchestra.run.smithi080.stdout: "error_string": "", 2024-04-24T23:32:01.183 INFO:teuthology.orchestra.run.smithi080.stdout: "module_options": { 2024-04-24T23:32:01.183 INFO:teuthology.orchestra.run.smithi080.stdout: "log_level": { 2024-04-24T23:32:01.183 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_level", 2024-04-24T23:32:01.183 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.183 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.183 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.184 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.184 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.184 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.184 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:01.184 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:01.184 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:01.184 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:01.184 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:01.184 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:01.185 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:01.185 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:01.185 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.185 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.185 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.185 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.185 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.185 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster": { 2024-04-24T23:32:01.185 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster", 2024-04-24T23:32:01.186 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.186 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.186 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.186 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.186 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.186 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.186 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.186 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.187 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.187 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.187 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.187 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.187 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster_level": { 2024-04-24T23:32:01.187 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster_level", 2024-04-24T23:32:01.187 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.187 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.188 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.188 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "info", 2024-04-24T23:32:01.188 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.188 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.188 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:01.188 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:01.188 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:01.188 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:01.188 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:01.189 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:01.189 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:01.189 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:01.189 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.189 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.189 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.189 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.189 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.189 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_file": { 2024-04-24T23:32:01.190 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_file", 2024-04-24T23:32:01.190 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.190 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.190 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.190 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.190 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.190 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.190 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.191 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.191 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.191 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.191 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.191 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.191 INFO:teuthology.orchestra.run.smithi080.stdout: "sleep_interval": { 2024-04-24T23:32:01.191 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "sleep_interval", 2024-04-24T23:32:01.191 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "secs", 2024-04-24T23:32:01.191 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.192 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.192 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "60", 2024-04-24T23:32:01.192 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.192 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.192 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.192 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.192 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.192 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.193 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.193 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.193 INFO:teuthology.orchestra.run.smithi080.stdout: "threshold": { 2024-04-24T23:32:01.193 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "threshold", 2024-04-24T23:32:01.193 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "float", 2024-04-24T23:32:01.193 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.193 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.193 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "3.0", 2024-04-24T23:32:01.193 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "1.0", 2024-04-24T23:32:01.194 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.194 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.194 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "scaling threshold", 2024-04-24T23:32:01.194 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-24T23:32:01.194 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.194 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.194 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:01.194 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:01.195 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.195 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-24T23:32:01.195 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "progress", 2024-04-24T23:32:01.195 INFO:teuthology.orchestra.run.smithi080.stdout: "can_run": true, 2024-04-24T23:32:01.195 INFO:teuthology.orchestra.run.smithi080.stdout: "error_string": "", 2024-04-24T23:32:01.195 INFO:teuthology.orchestra.run.smithi080.stdout: "module_options": { 2024-04-24T23:32:01.195 INFO:teuthology.orchestra.run.smithi080.stdout: "allow_pg_recovery_event": { 2024-04-24T23:32:01.195 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "allow_pg_recovery_event", 2024-04-24T23:32:01.195 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.196 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.196 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.196 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.196 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.196 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.196 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.196 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-24T23:32:01.196 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.196 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.197 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.197 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.197 INFO:teuthology.orchestra.run.smithi080.stdout: "enabled": { 2024-04-24T23:32:01.197 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "enabled", 2024-04-24T23:32:01.197 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.197 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.197 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.197 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "True", 2024-04-24T23:32:01.197 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.198 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.198 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.198 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.198 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.198 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.198 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.198 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.198 INFO:teuthology.orchestra.run.smithi080.stdout: "log_level": { 2024-04-24T23:32:01.199 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_level", 2024-04-24T23:32:01.199 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.199 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.199 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.199 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.199 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.199 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.199 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:01.199 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:01.200 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:01.200 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:01.200 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:01.200 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:01.200 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:01.200 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:01.200 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.200 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.200 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.201 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.201 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.201 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster": { 2024-04-24T23:32:01.201 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster", 2024-04-24T23:32:01.201 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.201 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.201 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.201 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.201 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.202 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.202 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.202 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.202 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.202 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.202 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.202 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.202 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster_level": { 2024-04-24T23:32:01.203 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster_level", 2024-04-24T23:32:01.203 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.203 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.203 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.203 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "info", 2024-04-24T23:32:01.203 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.203 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.203 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:01.203 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:01.204 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:01.204 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:01.204 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:01.204 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:01.204 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:01.204 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:01.204 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.204 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.204 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.205 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.205 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.205 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_file": { 2024-04-24T23:32:01.205 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_file", 2024-04-24T23:32:01.205 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.205 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.205 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.205 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.205 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.206 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.206 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.206 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.206 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.206 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.206 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.206 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.206 INFO:teuthology.orchestra.run.smithi080.stdout: "max_completed_events": { 2024-04-24T23:32:01.207 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "max_completed_events", 2024-04-24T23:32:01.207 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "int", 2024-04-24T23:32:01.207 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.207 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.207 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "50", 2024-04-24T23:32:01.207 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.207 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.207 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.207 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "number of past completed events to remember", 2024-04-24T23:32:01.208 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.208 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.208 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.208 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.208 INFO:teuthology.orchestra.run.smithi080.stdout: "sleep_interval": { 2024-04-24T23:32:01.208 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "sleep_interval", 2024-04-24T23:32:01.208 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "secs", 2024-04-24T23:32:01.208 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.208 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.209 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "5", 2024-04-24T23:32:01.209 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.209 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.209 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.209 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "how long the module is going to sleep", 2024-04-24T23:32:01.209 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.209 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.209 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.210 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:01.210 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:01.210 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.210 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-24T23:32:01.210 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "prometheus", 2024-04-24T23:32:01.210 INFO:teuthology.orchestra.run.smithi080.stdout: "can_run": true, 2024-04-24T23:32:01.210 INFO:teuthology.orchestra.run.smithi080.stdout: "error_string": "", 2024-04-24T23:32:01.210 INFO:teuthology.orchestra.run.smithi080.stdout: "module_options": { 2024-04-24T23:32:01.211 INFO:teuthology.orchestra.run.smithi080.stdout: "cache": { 2024-04-24T23:32:01.211 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "cache", 2024-04-24T23:32:01.211 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.211 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.211 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.211 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "True", 2024-04-24T23:32:01.211 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.211 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.211 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.212 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.212 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.212 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.212 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.212 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.212 INFO:teuthology.orchestra.run.smithi080.stdout: "exclude_perf_counters": { 2024-04-24T23:32:01.212 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "exclude_perf_counters", 2024-04-24T23:32:01.212 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.213 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.213 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.213 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "True", 2024-04-24T23:32:01.213 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.213 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.213 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.213 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-24T23:32:01.213 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-24T23:32:01.213 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.214 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.214 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.214 INFO:teuthology.orchestra.run.smithi080.stdout: "log_level": { 2024-04-24T23:32:01.214 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_level", 2024-04-24T23:32:01.214 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.214 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.214 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.214 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.214 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.215 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.215 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:01.215 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:01.215 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:01.215 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:01.215 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:01.215 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:01.215 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:01.216 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:01.216 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.216 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.216 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.216 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.216 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.216 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster": { 2024-04-24T23:32:01.216 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster", 2024-04-24T23:32:01.216 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.217 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.217 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.217 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.217 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.217 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.217 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.217 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.217 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.217 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.218 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.218 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.218 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster_level": { 2024-04-24T23:32:01.218 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster_level", 2024-04-24T23:32:01.218 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.218 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.218 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.218 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "info", 2024-04-24T23:32:01.219 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.219 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.219 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:01.219 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:01.219 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:01.219 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:01.219 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:01.219 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:01.219 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:01.220 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:01.220 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.220 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.220 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.220 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.220 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.220 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_file": { 2024-04-24T23:32:01.220 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_file", 2024-04-24T23:32:01.220 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.221 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.221 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.221 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.221 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.221 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.221 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.221 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.221 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.221 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.222 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.222 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.222 INFO:teuthology.orchestra.run.smithi080.stdout: "rbd_stats_pools": { 2024-04-24T23:32:01.222 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "rbd_stats_pools", 2024-04-24T23:32:01.222 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.222 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.222 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.222 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.222 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.223 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.223 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.223 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.223 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.223 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.223 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.223 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.223 INFO:teuthology.orchestra.run.smithi080.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-24T23:32:01.224 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-24T23:32:01.224 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "int", 2024-04-24T23:32:01.224 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.224 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.224 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "300", 2024-04-24T23:32:01.224 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.224 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.224 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.224 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.225 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.225 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.225 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.225 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.225 INFO:teuthology.orchestra.run.smithi080.stdout: "scrape_interval": { 2024-04-24T23:32:01.225 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "scrape_interval", 2024-04-24T23:32:01.225 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "float", 2024-04-24T23:32:01.225 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.225 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.226 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "15.0", 2024-04-24T23:32:01.226 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.226 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.226 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.226 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.226 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.226 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.226 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.226 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.227 INFO:teuthology.orchestra.run.smithi080.stdout: "server_addr": { 2024-04-24T23:32:01.227 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "server_addr", 2024-04-24T23:32:01.227 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.227 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.227 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.227 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "::", 2024-04-24T23:32:01.227 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.227 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.227 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.228 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-24T23:32:01.228 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.228 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.228 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.228 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.228 INFO:teuthology.orchestra.run.smithi080.stdout: "server_port": { 2024-04-24T23:32:01.228 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "server_port", 2024-04-24T23:32:01.228 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "int", 2024-04-24T23:32:01.229 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.229 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.229 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "9283", 2024-04-24T23:32:01.229 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.229 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.229 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.229 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-24T23:32:01.229 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.229 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.230 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.230 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.230 INFO:teuthology.orchestra.run.smithi080.stdout: "stale_cache_strategy": { 2024-04-24T23:32:01.230 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "stale_cache_strategy", 2024-04-24T23:32:01.230 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.230 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.230 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.230 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "log", 2024-04-24T23:32:01.231 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.231 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.231 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.231 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.231 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.231 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.231 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.231 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.232 INFO:teuthology.orchestra.run.smithi080.stdout: "standby_behaviour": { 2024-04-24T23:32:01.232 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "standby_behaviour", 2024-04-24T23:32:01.232 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.232 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.232 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.232 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "default", 2024-04-24T23:32:01.232 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.232 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.232 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:01.233 INFO:teuthology.orchestra.run.smithi080.stdout: "default", 2024-04-24T23:32:01.233 INFO:teuthology.orchestra.run.smithi080.stdout: "error" 2024-04-24T23:32:01.233 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:01.233 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.233 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.233 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.233 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.233 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.233 INFO:teuthology.orchestra.run.smithi080.stdout: "standby_error_status_code": { 2024-04-24T23:32:01.234 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "standby_error_status_code", 2024-04-24T23:32:01.234 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "int", 2024-04-24T23:32:01.234 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.234 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.234 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "500", 2024-04-24T23:32:01.234 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "400", 2024-04-24T23:32:01.234 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "599", 2024-04-24T23:32:01.234 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.235 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.235 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.235 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.235 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.235 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:01.235 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:01.235 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.235 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-24T23:32:01.235 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "rbd_support", 2024-04-24T23:32:01.236 INFO:teuthology.orchestra.run.smithi080.stdout: "can_run": true, 2024-04-24T23:32:01.236 INFO:teuthology.orchestra.run.smithi080.stdout: "error_string": "", 2024-04-24T23:32:01.236 INFO:teuthology.orchestra.run.smithi080.stdout: "module_options": { 2024-04-24T23:32:01.236 INFO:teuthology.orchestra.run.smithi080.stdout: "log_level": { 2024-04-24T23:32:01.236 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_level", 2024-04-24T23:32:01.236 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.236 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.236 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.236 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.237 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.237 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.237 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:01.237 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:01.237 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:01.237 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:01.237 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:01.237 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:01.237 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:01.238 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:01.238 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.238 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.238 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.238 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.238 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.238 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster": { 2024-04-24T23:32:01.238 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster", 2024-04-24T23:32:01.239 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.239 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.239 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.239 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.239 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.239 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.239 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.239 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.239 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.240 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.240 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.240 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.240 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster_level": { 2024-04-24T23:32:01.240 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster_level", 2024-04-24T23:32:01.240 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.240 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.240 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.240 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "info", 2024-04-24T23:32:01.241 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.241 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.241 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:01.241 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:01.241 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:01.241 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:01.241 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:01.241 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:01.242 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:01.242 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:01.242 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.242 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.242 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.242 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.242 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.242 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_file": { 2024-04-24T23:32:01.242 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_file", 2024-04-24T23:32:01.243 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.243 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.243 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.243 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.243 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.243 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.243 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.243 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.243 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.244 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.244 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.244 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.244 INFO:teuthology.orchestra.run.smithi080.stdout: "max_concurrent_snap_create": { 2024-04-24T23:32:01.244 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "max_concurrent_snap_create", 2024-04-24T23:32:01.244 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "int", 2024-04-24T23:32:01.244 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.244 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.244 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "10", 2024-04-24T23:32:01.245 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.245 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.245 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.245 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.245 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.245 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.245 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.245 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.246 INFO:teuthology.orchestra.run.smithi080.stdout: "mirror_snapshot_schedule": { 2024-04-24T23:32:01.246 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "mirror_snapshot_schedule", 2024-04-24T23:32:01.246 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.246 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.246 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.246 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.246 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.246 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.247 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.247 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.247 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.247 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.247 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.247 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.247 INFO:teuthology.orchestra.run.smithi080.stdout: "trash_purge_schedule": { 2024-04-24T23:32:01.247 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "trash_purge_schedule", 2024-04-24T23:32:01.247 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.248 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.248 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.248 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.248 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.248 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.248 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.248 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.248 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.248 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.249 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.249 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:01.249 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:01.249 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.249 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-24T23:32:01.249 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "restful", 2024-04-24T23:32:01.249 INFO:teuthology.orchestra.run.smithi080.stdout: "can_run": true, 2024-04-24T23:32:01.249 INFO:teuthology.orchestra.run.smithi080.stdout: "error_string": "", 2024-04-24T23:32:01.250 INFO:teuthology.orchestra.run.smithi080.stdout: "module_options": { 2024-04-24T23:32:01.250 INFO:teuthology.orchestra.run.smithi080.stdout: "enable_auth": { 2024-04-24T23:32:01.250 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "enable_auth", 2024-04-24T23:32:01.250 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.250 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.250 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.250 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "True", 2024-04-24T23:32:01.250 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.250 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.251 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.251 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.251 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.251 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.251 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.251 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.251 INFO:teuthology.orchestra.run.smithi080.stdout: "key_file": { 2024-04-24T23:32:01.251 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "key_file", 2024-04-24T23:32:01.251 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.252 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.252 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.252 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.252 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.252 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.252 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.252 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.252 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.253 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.253 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.253 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.253 INFO:teuthology.orchestra.run.smithi080.stdout: "log_level": { 2024-04-24T23:32:01.253 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_level", 2024-04-24T23:32:01.253 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.254 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.254 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.254 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.254 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.254 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.254 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:01.254 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:01.254 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:01.254 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:01.255 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:01.255 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:01.255 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:01.255 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:01.255 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.255 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.255 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.255 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.256 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.256 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster": { 2024-04-24T23:32:01.256 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster", 2024-04-24T23:32:01.256 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.256 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.256 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.256 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.256 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.256 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.257 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.257 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.257 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.257 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.257 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.257 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.257 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster_level": { 2024-04-24T23:32:01.257 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster_level", 2024-04-24T23:32:01.257 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.258 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.258 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.258 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "info", 2024-04-24T23:32:01.258 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.258 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.258 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:01.258 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:01.258 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:01.259 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:01.259 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:01.259 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:01.259 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:01.259 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:01.259 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.259 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.259 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.259 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.260 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.260 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_file": { 2024-04-24T23:32:01.260 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_file", 2024-04-24T23:32:01.260 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.260 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.260 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.260 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.260 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.260 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.261 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.261 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.261 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.261 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.261 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.261 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.261 INFO:teuthology.orchestra.run.smithi080.stdout: "server_addr": { 2024-04-24T23:32:01.261 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "server_addr", 2024-04-24T23:32:01.261 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.262 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.262 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.262 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.262 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.262 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.262 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.262 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.262 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.263 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.263 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.263 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.263 INFO:teuthology.orchestra.run.smithi080.stdout: "server_port": { 2024-04-24T23:32:01.263 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "server_port", 2024-04-24T23:32:01.263 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.263 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.263 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.263 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.264 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.264 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.264 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.264 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.264 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.264 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.264 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.264 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:01.265 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:01.265 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.265 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-24T23:32:01.265 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "rgw", 2024-04-24T23:32:01.265 INFO:teuthology.orchestra.run.smithi080.stdout: "can_run": true, 2024-04-24T23:32:01.265 INFO:teuthology.orchestra.run.smithi080.stdout: "error_string": "", 2024-04-24T23:32:01.265 INFO:teuthology.orchestra.run.smithi080.stdout: "module_options": { 2024-04-24T23:32:01.265 INFO:teuthology.orchestra.run.smithi080.stdout: "log_level": { 2024-04-24T23:32:01.265 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_level", 2024-04-24T23:32:01.266 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.266 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.266 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.266 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.266 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.266 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.266 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:01.266 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:01.267 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:01.267 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:01.267 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:01.267 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:01.267 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:01.267 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:01.267 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.267 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.267 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.268 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.268 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.268 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster": { 2024-04-24T23:32:01.268 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster", 2024-04-24T23:32:01.268 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.268 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.268 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.268 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.268 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.269 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.269 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.269 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.269 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.269 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.269 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.269 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.269 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster_level": { 2024-04-24T23:32:01.269 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster_level", 2024-04-24T23:32:01.270 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.270 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.270 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.270 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "info", 2024-04-24T23:32:01.270 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.270 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.270 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:01.270 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:01.271 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:01.271 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:01.271 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:01.271 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:01.271 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:01.271 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:01.271 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.271 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.271 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.272 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.272 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.272 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_file": { 2024-04-24T23:32:01.272 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_file", 2024-04-24T23:32:01.272 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.272 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.272 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.272 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.272 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.273 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.273 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.273 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.273 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.273 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.273 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.273 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:01.273 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:01.273 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.274 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-24T23:32:01.274 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "rook", 2024-04-24T23:32:01.274 INFO:teuthology.orchestra.run.smithi080.stdout: "can_run": true, 2024-04-24T23:32:01.274 INFO:teuthology.orchestra.run.smithi080.stdout: "error_string": "", 2024-04-24T23:32:01.274 INFO:teuthology.orchestra.run.smithi080.stdout: "module_options": { 2024-04-24T23:32:01.274 INFO:teuthology.orchestra.run.smithi080.stdout: "log_level": { 2024-04-24T23:32:01.274 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_level", 2024-04-24T23:32:01.274 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.275 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.275 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.275 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.275 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.275 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.275 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:01.275 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:01.275 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:01.275 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:01.276 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:01.276 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:01.276 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:01.276 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:01.276 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.276 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.276 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.276 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.277 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.277 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster": { 2024-04-24T23:32:01.277 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster", 2024-04-24T23:32:01.277 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.277 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.277 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.277 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.277 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.278 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.278 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.278 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.278 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.278 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.278 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.278 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.278 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster_level": { 2024-04-24T23:32:01.278 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster_level", 2024-04-24T23:32:01.279 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.279 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.279 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.279 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "info", 2024-04-24T23:32:01.279 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.279 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.279 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:01.279 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:01.279 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:01.280 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:01.280 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:01.280 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:01.280 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:01.280 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:01.280 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.280 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.280 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.281 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.281 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.281 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_file": { 2024-04-24T23:32:01.281 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_file", 2024-04-24T23:32:01.281 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.281 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.281 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.281 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.281 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.282 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.282 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.282 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.282 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.282 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.282 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.282 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.282 INFO:teuthology.orchestra.run.smithi080.stdout: "storage_class": { 2024-04-24T23:32:01.283 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "storage_class", 2024-04-24T23:32:01.283 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.283 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.283 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.283 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "local", 2024-04-24T23:32:01.283 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.283 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.283 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.283 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-24T23:32:01.284 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.284 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.284 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.284 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:01.284 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:01.284 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.284 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-24T23:32:01.284 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "selftest", 2024-04-24T23:32:01.284 INFO:teuthology.orchestra.run.smithi080.stdout: "can_run": true, 2024-04-24T23:32:01.285 INFO:teuthology.orchestra.run.smithi080.stdout: "error_string": "", 2024-04-24T23:32:01.285 INFO:teuthology.orchestra.run.smithi080.stdout: "module_options": { 2024-04-24T23:32:01.285 INFO:teuthology.orchestra.run.smithi080.stdout: "log_level": { 2024-04-24T23:32:01.285 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_level", 2024-04-24T23:32:01.285 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.285 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.285 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.285 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.285 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.286 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.286 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:01.286 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:01.286 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:01.286 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:01.286 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:01.286 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:01.286 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:01.287 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:01.287 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.287 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.287 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.287 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.287 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.287 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster": { 2024-04-24T23:32:01.287 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster", 2024-04-24T23:32:01.287 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.288 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.288 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.288 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.288 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.288 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.288 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.288 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.288 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.288 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.289 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.289 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.289 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster_level": { 2024-04-24T23:32:01.289 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster_level", 2024-04-24T23:32:01.289 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.289 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.289 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.289 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "info", 2024-04-24T23:32:01.289 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.290 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.290 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:01.290 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:01.290 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:01.290 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:01.290 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:01.290 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:01.290 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:01.291 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:01.291 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.291 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.291 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.291 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.291 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.291 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_file": { 2024-04-24T23:32:01.291 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_file", 2024-04-24T23:32:01.291 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.292 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.292 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.292 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.292 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.292 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.292 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.292 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.292 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.292 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.293 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.293 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.293 INFO:teuthology.orchestra.run.smithi080.stdout: "roption1": { 2024-04-24T23:32:01.293 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "roption1", 2024-04-24T23:32:01.293 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.293 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.293 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.293 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.294 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.294 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.294 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.294 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.294 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.294 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.294 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.294 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.294 INFO:teuthology.orchestra.run.smithi080.stdout: "roption2": { 2024-04-24T23:32:01.295 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "roption2", 2024-04-24T23:32:01.295 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.295 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.295 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.295 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "xyz", 2024-04-24T23:32:01.295 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.295 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.295 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.295 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.296 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.296 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.296 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.296 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.296 INFO:teuthology.orchestra.run.smithi080.stdout: "rwoption1": { 2024-04-24T23:32:01.296 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "rwoption1", 2024-04-24T23:32:01.296 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.296 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.296 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.297 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.297 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.297 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.297 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.297 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.297 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.297 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.297 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.298 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.298 INFO:teuthology.orchestra.run.smithi080.stdout: "rwoption2": { 2024-04-24T23:32:01.298 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "rwoption2", 2024-04-24T23:32:01.298 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "int", 2024-04-24T23:32:01.298 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.298 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.298 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.298 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.299 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.299 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.299 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.299 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.299 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.299 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.299 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.299 INFO:teuthology.orchestra.run.smithi080.stdout: "rwoption3": { 2024-04-24T23:32:01.300 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "rwoption3", 2024-04-24T23:32:01.300 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "float", 2024-04-24T23:32:01.300 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.300 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.300 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.300 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.300 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.300 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.300 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.301 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.301 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.301 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.301 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.301 INFO:teuthology.orchestra.run.smithi080.stdout: "rwoption4": { 2024-04-24T23:32:01.301 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "rwoption4", 2024-04-24T23:32:01.301 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.301 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.301 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.302 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.302 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.302 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.302 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.302 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.302 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.302 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.302 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.303 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.303 INFO:teuthology.orchestra.run.smithi080.stdout: "rwoption5": { 2024-04-24T23:32:01.303 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "rwoption5", 2024-04-24T23:32:01.303 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.303 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.303 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.303 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.303 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.303 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.304 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.304 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.304 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.304 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.304 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.304 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.304 INFO:teuthology.orchestra.run.smithi080.stdout: "rwoption6": { 2024-04-24T23:32:01.304 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "rwoption6", 2024-04-24T23:32:01.305 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.305 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.305 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.305 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "True", 2024-04-24T23:32:01.305 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.305 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.305 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.305 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.305 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.306 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.306 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.306 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.306 INFO:teuthology.orchestra.run.smithi080.stdout: "rwoption7": { 2024-04-24T23:32:01.306 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "rwoption7", 2024-04-24T23:32:01.306 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "int", 2024-04-24T23:32:01.306 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.306 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.307 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.307 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "1", 2024-04-24T23:32:01.307 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "42", 2024-04-24T23:32:01.307 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.307 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.307 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.307 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.307 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.307 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.308 INFO:teuthology.orchestra.run.smithi080.stdout: "testkey": { 2024-04-24T23:32:01.308 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "testkey", 2024-04-24T23:32:01.308 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.308 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.308 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.308 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.308 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.308 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.308 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.309 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.309 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.309 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.309 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.309 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.309 INFO:teuthology.orchestra.run.smithi080.stdout: "testlkey": { 2024-04-24T23:32:01.309 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "testlkey", 2024-04-24T23:32:01.309 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.309 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.310 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.310 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.310 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.310 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.310 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.310 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.310 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.310 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.311 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.311 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.311 INFO:teuthology.orchestra.run.smithi080.stdout: "testnewline": { 2024-04-24T23:32:01.311 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "testnewline", 2024-04-24T23:32:01.311 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.311 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.311 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.311 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.311 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.312 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.312 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.312 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.312 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.312 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.312 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.312 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:01.312 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:01.312 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.313 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-24T23:32:01.313 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "snap_schedule", 2024-04-24T23:32:01.313 INFO:teuthology.orchestra.run.smithi080.stdout: "can_run": true, 2024-04-24T23:32:01.313 INFO:teuthology.orchestra.run.smithi080.stdout: "error_string": "", 2024-04-24T23:32:01.313 INFO:teuthology.orchestra.run.smithi080.stdout: "module_options": { 2024-04-24T23:32:01.313 INFO:teuthology.orchestra.run.smithi080.stdout: "allow_m_granularity": { 2024-04-24T23:32:01.313 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "allow_m_granularity", 2024-04-24T23:32:01.313 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.314 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.314 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.314 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.314 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.314 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.314 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.314 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "allow minute scheduled snapshots", 2024-04-24T23:32:01.314 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.314 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.315 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.315 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.315 INFO:teuthology.orchestra.run.smithi080.stdout: "dump_on_update": { 2024-04-24T23:32:01.315 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "dump_on_update", 2024-04-24T23:32:01.315 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.315 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.315 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.315 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.316 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.316 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.316 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.316 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "dump database to debug log on update", 2024-04-24T23:32:01.316 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.316 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.316 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.316 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.316 INFO:teuthology.orchestra.run.smithi080.stdout: "log_level": { 2024-04-24T23:32:01.317 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_level", 2024-04-24T23:32:01.317 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.317 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.317 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.317 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.317 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.317 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.317 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:01.317 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:01.318 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:01.318 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:01.318 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:01.318 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:01.318 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:01.318 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:01.318 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.318 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.319 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.319 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.319 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.319 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster": { 2024-04-24T23:32:01.319 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster", 2024-04-24T23:32:01.319 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.319 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.319 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.319 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.320 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.320 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.320 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.320 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.320 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.320 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.320 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.320 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.320 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster_level": { 2024-04-24T23:32:01.321 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster_level", 2024-04-24T23:32:01.321 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.321 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.321 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.321 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "info", 2024-04-24T23:32:01.321 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.321 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.321 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:01.322 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:01.322 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:01.322 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:01.322 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:01.322 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:01.322 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:01.322 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:01.322 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.323 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.323 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.323 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.323 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.323 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_file": { 2024-04-24T23:32:01.323 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_file", 2024-04-24T23:32:01.323 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.323 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.323 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.324 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.324 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.324 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.324 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.324 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.324 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.324 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.324 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.324 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:01.325 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:01.325 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.325 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-24T23:32:01.325 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "stats", 2024-04-24T23:32:01.325 INFO:teuthology.orchestra.run.smithi080.stdout: "can_run": true, 2024-04-24T23:32:01.325 INFO:teuthology.orchestra.run.smithi080.stdout: "error_string": "", 2024-04-24T23:32:01.325 INFO:teuthology.orchestra.run.smithi080.stdout: "module_options": { 2024-04-24T23:32:01.325 INFO:teuthology.orchestra.run.smithi080.stdout: "log_level": { 2024-04-24T23:32:01.326 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_level", 2024-04-24T23:32:01.326 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.326 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.326 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.326 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.326 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.326 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.326 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:01.326 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:01.327 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:01.327 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:01.327 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:01.327 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:01.327 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:01.327 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:01.327 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.327 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.327 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.328 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.328 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.328 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster": { 2024-04-24T23:32:01.328 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster", 2024-04-24T23:32:01.328 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.328 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.328 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.328 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.329 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.329 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.329 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.329 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.329 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.329 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.329 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.329 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.329 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster_level": { 2024-04-24T23:32:01.330 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster_level", 2024-04-24T23:32:01.330 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.330 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.330 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.330 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "info", 2024-04-24T23:32:01.330 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.330 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.330 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:01.330 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:01.331 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:01.331 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:01.331 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:01.331 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:01.331 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:01.331 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:01.331 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.331 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.331 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.332 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.332 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.332 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_file": { 2024-04-24T23:32:01.332 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_file", 2024-04-24T23:32:01.332 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.332 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.332 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.332 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.332 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.333 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.333 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.333 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.333 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.333 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.333 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.333 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:01.333 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:01.334 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.334 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-24T23:32:01.334 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "status", 2024-04-24T23:32:01.334 INFO:teuthology.orchestra.run.smithi080.stdout: "can_run": true, 2024-04-24T23:32:01.334 INFO:teuthology.orchestra.run.smithi080.stdout: "error_string": "", 2024-04-24T23:32:01.334 INFO:teuthology.orchestra.run.smithi080.stdout: "module_options": { 2024-04-24T23:32:01.334 INFO:teuthology.orchestra.run.smithi080.stdout: "log_level": { 2024-04-24T23:32:01.334 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_level", 2024-04-24T23:32:01.334 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.335 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.335 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.335 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.335 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.335 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.335 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:01.335 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:01.335 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:01.335 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:01.336 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:01.336 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:01.336 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:01.336 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:01.336 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.336 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.336 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.336 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.337 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.337 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster": { 2024-04-24T23:32:01.337 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster", 2024-04-24T23:32:01.337 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.337 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.337 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.337 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.337 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.337 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.338 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.338 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.338 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.338 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.338 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.338 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.338 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster_level": { 2024-04-24T23:32:01.338 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster_level", 2024-04-24T23:32:01.338 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.339 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.339 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.339 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "info", 2024-04-24T23:32:01.339 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.339 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.339 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:01.339 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:01.339 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:01.339 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:01.340 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:01.340 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:01.340 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:01.340 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:01.340 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.340 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.340 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.340 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.341 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.341 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_file": { 2024-04-24T23:32:01.341 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_file", 2024-04-24T23:32:01.341 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.341 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.341 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.341 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.341 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.341 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.342 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.342 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.342 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.342 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.342 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.342 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:01.342 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:01.342 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.342 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-24T23:32:01.343 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "telegraf", 2024-04-24T23:32:01.343 INFO:teuthology.orchestra.run.smithi080.stdout: "can_run": true, 2024-04-24T23:32:01.343 INFO:teuthology.orchestra.run.smithi080.stdout: "error_string": "", 2024-04-24T23:32:01.343 INFO:teuthology.orchestra.run.smithi080.stdout: "module_options": { 2024-04-24T23:32:01.343 INFO:teuthology.orchestra.run.smithi080.stdout: "address": { 2024-04-24T23:32:01.343 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "address", 2024-04-24T23:32:01.343 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.343 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.344 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.344 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-24T23:32:01.344 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.344 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.344 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.344 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.344 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.344 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.345 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.345 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.345 INFO:teuthology.orchestra.run.smithi080.stdout: "interval": { 2024-04-24T23:32:01.345 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "interval", 2024-04-24T23:32:01.345 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "secs", 2024-04-24T23:32:01.345 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.345 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.345 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "15", 2024-04-24T23:32:01.345 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.346 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.346 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.346 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.346 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.346 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.346 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.346 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.346 INFO:teuthology.orchestra.run.smithi080.stdout: "log_level": { 2024-04-24T23:32:01.347 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_level", 2024-04-24T23:32:01.347 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.347 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.347 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.347 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.347 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.347 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.347 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:01.347 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:01.348 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:01.348 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:01.348 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:01.348 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:01.348 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:01.348 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:01.348 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.348 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.348 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.349 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.349 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.349 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster": { 2024-04-24T23:32:01.349 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster", 2024-04-24T23:32:01.349 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.349 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.349 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.349 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.349 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.350 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.350 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.350 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.350 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.350 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.350 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.350 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.350 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster_level": { 2024-04-24T23:32:01.351 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster_level", 2024-04-24T23:32:01.351 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.351 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.351 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.351 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "info", 2024-04-24T23:32:01.351 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.351 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.351 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:01.351 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:01.352 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:01.352 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:01.352 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:01.352 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:01.352 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:01.352 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:01.352 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.352 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.352 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.353 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.353 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.353 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_file": { 2024-04-24T23:32:01.353 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_file", 2024-04-24T23:32:01.353 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.353 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.353 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.353 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.353 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.354 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.354 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.354 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.354 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.354 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.354 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.354 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:01.354 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:01.355 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.355 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-24T23:32:01.355 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "telemetry", 2024-04-24T23:32:01.355 INFO:teuthology.orchestra.run.smithi080.stdout: "can_run": true, 2024-04-24T23:32:01.355 INFO:teuthology.orchestra.run.smithi080.stdout: "error_string": "", 2024-04-24T23:32:01.355 INFO:teuthology.orchestra.run.smithi080.stdout: "module_options": { 2024-04-24T23:32:01.355 INFO:teuthology.orchestra.run.smithi080.stdout: "channel_basic": { 2024-04-24T23:32:01.355 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "channel_basic", 2024-04-24T23:32:01.355 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.356 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.356 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.356 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "True", 2024-04-24T23:32:01.356 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.356 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.356 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.356 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-24T23:32:01.356 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.356 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.357 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.357 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.357 INFO:teuthology.orchestra.run.smithi080.stdout: "channel_crash": { 2024-04-24T23:32:01.357 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "channel_crash", 2024-04-24T23:32:01.357 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.357 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.357 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.357 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "True", 2024-04-24T23:32:01.357 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.358 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.358 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.358 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-24T23:32:01.358 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.358 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.358 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.358 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.358 INFO:teuthology.orchestra.run.smithi080.stdout: "channel_device": { 2024-04-24T23:32:01.359 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "channel_device", 2024-04-24T23:32:01.359 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.359 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.359 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.359 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "True", 2024-04-24T23:32:01.359 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.359 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.359 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.360 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-24T23:32:01.360 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.360 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.360 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.360 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.360 INFO:teuthology.orchestra.run.smithi080.stdout: "channel_ident": { 2024-04-24T23:32:01.360 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "channel_ident", 2024-04-24T23:32:01.360 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.360 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.361 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.361 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.361 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.361 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.361 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.361 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-24T23:32:01.361 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.361 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.361 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.362 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.362 INFO:teuthology.orchestra.run.smithi080.stdout: "channel_perf": { 2024-04-24T23:32:01.362 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "channel_perf", 2024-04-24T23:32:01.362 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.362 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.362 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.362 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.362 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.363 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.363 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.363 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-24T23:32:01.363 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.363 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.363 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.363 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.363 INFO:teuthology.orchestra.run.smithi080.stdout: "contact": { 2024-04-24T23:32:01.363 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "contact", 2024-04-24T23:32:01.364 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.364 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.365 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.365 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.365 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.365 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.366 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.366 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.366 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.366 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.366 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.366 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.366 INFO:teuthology.orchestra.run.smithi080.stdout: "description": { 2024-04-24T23:32:01.366 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "description", 2024-04-24T23:32:01.366 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.367 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.367 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.367 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.367 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.367 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.367 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.367 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.367 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.368 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.368 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.368 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.368 INFO:teuthology.orchestra.run.smithi080.stdout: "device_url": { 2024-04-24T23:32:01.368 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "device_url", 2024-04-24T23:32:01.368 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.368 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.368 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.368 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-24T23:32:01.369 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.369 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.369 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.369 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.369 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.369 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.369 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.369 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.370 INFO:teuthology.orchestra.run.smithi080.stdout: "enabled": { 2024-04-24T23:32:01.370 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "enabled", 2024-04-24T23:32:01.370 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.370 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.370 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.370 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.370 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.370 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.370 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.371 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.371 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.371 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.371 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.371 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.371 INFO:teuthology.orchestra.run.smithi080.stdout: "interval": { 2024-04-24T23:32:01.371 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "interval", 2024-04-24T23:32:01.371 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "int", 2024-04-24T23:32:01.372 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.372 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.372 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "24", 2024-04-24T23:32:01.372 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "8", 2024-04-24T23:32:01.372 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.372 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.372 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.372 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.372 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.373 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.373 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.373 INFO:teuthology.orchestra.run.smithi080.stdout: "last_opt_revision": { 2024-04-24T23:32:01.373 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "last_opt_revision", 2024-04-24T23:32:01.373 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "int", 2024-04-24T23:32:01.373 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.373 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.373 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "1", 2024-04-24T23:32:01.373 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.374 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.374 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.374 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.374 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.374 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.374 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.374 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.374 INFO:teuthology.orchestra.run.smithi080.stdout: "leaderboard": { 2024-04-24T23:32:01.375 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "leaderboard", 2024-04-24T23:32:01.375 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.375 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.375 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.375 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.375 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.375 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.375 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.375 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.376 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.376 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.376 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.376 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.376 INFO:teuthology.orchestra.run.smithi080.stdout: "leaderboard_description": { 2024-04-24T23:32:01.376 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "leaderboard_description", 2024-04-24T23:32:01.376 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.376 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.376 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.377 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.377 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.377 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.377 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.377 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.377 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.377 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.377 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.377 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.378 INFO:teuthology.orchestra.run.smithi080.stdout: "log_level": { 2024-04-24T23:32:01.378 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_level", 2024-04-24T23:32:01.378 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.378 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.378 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.378 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.378 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.378 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.379 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:01.379 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:01.379 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:01.379 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:01.379 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:01.379 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:01.379 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:01.379 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:01.379 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.380 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.380 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.380 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.380 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.380 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster": { 2024-04-24T23:32:01.380 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster", 2024-04-24T23:32:01.380 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.380 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.380 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.381 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.381 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.381 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.381 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.381 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.381 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.381 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.381 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.381 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.382 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster_level": { 2024-04-24T23:32:01.382 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster_level", 2024-04-24T23:32:01.382 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.382 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.382 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.382 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "info", 2024-04-24T23:32:01.382 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.382 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.383 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:01.383 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:01.383 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:01.383 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:01.383 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:01.383 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:01.383 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:01.383 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:01.383 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.384 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.384 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.384 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.384 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.384 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_file": { 2024-04-24T23:32:01.384 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_file", 2024-04-24T23:32:01.384 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.384 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.384 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.385 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.385 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.385 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.385 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.385 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.385 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.385 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.385 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.386 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.386 INFO:teuthology.orchestra.run.smithi080.stdout: "organization": { 2024-04-24T23:32:01.386 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "organization", 2024-04-24T23:32:01.386 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.386 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.386 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.386 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.386 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.386 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.387 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.387 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.387 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.387 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.387 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.387 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.387 INFO:teuthology.orchestra.run.smithi080.stdout: "proxy": { 2024-04-24T23:32:01.387 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "proxy", 2024-04-24T23:32:01.388 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.388 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.388 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.388 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.388 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.388 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.388 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.388 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.388 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.389 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.389 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.389 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.389 INFO:teuthology.orchestra.run.smithi080.stdout: "url": { 2024-04-24T23:32:01.389 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "url", 2024-04-24T23:32:01.389 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.389 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.389 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.390 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-24T23:32:01.390 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.390 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.390 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.390 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.390 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.390 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.390 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.391 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:01.391 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:01.391 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.391 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-24T23:32:01.391 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "test_orchestrator", 2024-04-24T23:32:01.391 INFO:teuthology.orchestra.run.smithi080.stdout: "can_run": true, 2024-04-24T23:32:01.391 INFO:teuthology.orchestra.run.smithi080.stdout: "error_string": "", 2024-04-24T23:32:01.391 INFO:teuthology.orchestra.run.smithi080.stdout: "module_options": { 2024-04-24T23:32:01.391 INFO:teuthology.orchestra.run.smithi080.stdout: "log_level": { 2024-04-24T23:32:01.392 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_level", 2024-04-24T23:32:01.392 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.392 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.392 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.392 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.392 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.392 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.392 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:01.392 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:01.393 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:01.393 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:01.393 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:01.393 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:01.393 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:01.393 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:01.393 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.393 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.393 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.394 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.394 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.394 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster": { 2024-04-24T23:32:01.394 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster", 2024-04-24T23:32:01.394 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.394 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.394 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.394 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.395 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.395 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.395 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.395 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.395 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.395 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.395 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.395 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.395 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster_level": { 2024-04-24T23:32:01.396 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster_level", 2024-04-24T23:32:01.396 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.396 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.396 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.396 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "info", 2024-04-24T23:32:01.396 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.396 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.396 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:01.396 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:01.397 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:01.397 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:01.397 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:01.397 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:01.397 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:01.397 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:01.397 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.397 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.398 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.398 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.398 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.398 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_file": { 2024-04-24T23:32:01.398 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_file", 2024-04-24T23:32:01.398 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.398 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.398 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.398 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.399 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.399 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.399 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.399 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.399 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.399 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.399 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.399 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:01.399 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:01.400 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.400 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-24T23:32:01.400 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "volumes", 2024-04-24T23:32:01.400 INFO:teuthology.orchestra.run.smithi080.stdout: "can_run": true, 2024-04-24T23:32:01.400 INFO:teuthology.orchestra.run.smithi080.stdout: "error_string": "", 2024-04-24T23:32:01.400 INFO:teuthology.orchestra.run.smithi080.stdout: "module_options": { 2024-04-24T23:32:01.400 INFO:teuthology.orchestra.run.smithi080.stdout: "log_level": { 2024-04-24T23:32:01.400 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_level", 2024-04-24T23:32:01.400 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.401 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.401 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.401 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.401 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.401 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.401 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:01.401 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:01.401 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:01.402 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:01.402 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:01.402 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:01.402 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:01.402 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:01.402 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.402 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.402 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.402 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.403 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.403 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster": { 2024-04-24T23:32:01.403 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster", 2024-04-24T23:32:01.403 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.403 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.403 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.403 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.403 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.403 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.404 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.404 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.404 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.404 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.404 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.404 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.404 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster_level": { 2024-04-24T23:32:01.404 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster_level", 2024-04-24T23:32:01.405 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.405 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.405 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.405 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "info", 2024-04-24T23:32:01.405 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.405 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.405 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:01.405 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:01.405 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:01.406 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:01.406 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:01.406 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:01.406 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:01.406 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:01.406 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.406 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.406 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.406 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.407 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.407 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_file": { 2024-04-24T23:32:01.407 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_file", 2024-04-24T23:32:01.407 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.407 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.407 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.407 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.407 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.407 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.408 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.408 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.408 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.408 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.408 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.408 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.408 INFO:teuthology.orchestra.run.smithi080.stdout: "max_concurrent_clones": { 2024-04-24T23:32:01.408 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "max_concurrent_clones", 2024-04-24T23:32:01.408 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "int", 2024-04-24T23:32:01.409 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.409 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.409 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "4", 2024-04-24T23:32:01.409 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.409 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.409 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.409 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-24T23:32:01.409 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.410 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.410 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.410 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.410 INFO:teuthology.orchestra.run.smithi080.stdout: "periodic_async_work": { 2024-04-24T23:32:01.410 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "periodic_async_work", 2024-04-24T23:32:01.410 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.410 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.410 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.410 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.411 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.411 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.411 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.411 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Periodically check for async work", 2024-04-24T23:32:01.411 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.411 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.411 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.411 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.412 INFO:teuthology.orchestra.run.smithi080.stdout: "snapshot_clone_delay": { 2024-04-24T23:32:01.412 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "snapshot_clone_delay", 2024-04-24T23:32:01.412 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "int", 2024-04-24T23:32:01.412 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.412 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.412 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "0", 2024-04-24T23:32:01.412 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.412 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.413 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.413 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-24T23:32:01.413 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.413 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.413 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.413 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.413 INFO:teuthology.orchestra.run.smithi080.stdout: "snapshot_clone_no_wait": { 2024-04-24T23:32:01.413 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "snapshot_clone_no_wait", 2024-04-24T23:32:01.413 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.414 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.414 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.414 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "True", 2024-04-24T23:32:01.414 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.414 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.414 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.414 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-24T23:32:01.414 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.414 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.415 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.415 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:01.415 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:01.415 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.415 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-24T23:32:01.415 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "zabbix", 2024-04-24T23:32:01.415 INFO:teuthology.orchestra.run.smithi080.stdout: "can_run": true, 2024-04-24T23:32:01.415 INFO:teuthology.orchestra.run.smithi080.stdout: "error_string": "", 2024-04-24T23:32:01.416 INFO:teuthology.orchestra.run.smithi080.stdout: "module_options": { 2024-04-24T23:32:01.416 INFO:teuthology.orchestra.run.smithi080.stdout: "discovery_interval": { 2024-04-24T23:32:01.416 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "discovery_interval", 2024-04-24T23:32:01.416 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "uint", 2024-04-24T23:32:01.416 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.416 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.416 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "100", 2024-04-24T23:32:01.416 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.416 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.417 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.417 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.417 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.417 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.417 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.417 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.417 INFO:teuthology.orchestra.run.smithi080.stdout: "identifier": { 2024-04-24T23:32:01.417 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "identifier", 2024-04-24T23:32:01.417 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.418 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.418 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.418 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.418 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.418 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.418 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.418 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.418 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.418 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.419 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.419 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.419 INFO:teuthology.orchestra.run.smithi080.stdout: "interval": { 2024-04-24T23:32:01.419 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "interval", 2024-04-24T23:32:01.419 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "secs", 2024-04-24T23:32:01.419 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.419 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.419 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "60", 2024-04-24T23:32:01.420 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.420 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.420 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.420 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.420 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.420 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.420 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.420 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.420 INFO:teuthology.orchestra.run.smithi080.stdout: "log_level": { 2024-04-24T23:32:01.421 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_level", 2024-04-24T23:32:01.421 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.421 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.421 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.421 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.421 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.421 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.421 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:01.421 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:01.422 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:01.422 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:01.422 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:01.422 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:01.422 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:01.422 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:01.422 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.422 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.423 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.423 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.423 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.423 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster": { 2024-04-24T23:32:01.423 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster", 2024-04-24T23:32:01.423 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.423 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.423 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.423 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.424 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.424 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.424 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.424 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.424 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.424 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.424 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.424 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.424 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster_level": { 2024-04-24T23:32:01.425 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster_level", 2024-04-24T23:32:01.425 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.425 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.425 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.425 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "info", 2024-04-24T23:32:01.425 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.425 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.425 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:01.425 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:01.426 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:01.426 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:01.426 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:01.426 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:01.426 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:01.426 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:01.426 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.426 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.427 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.427 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.427 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.427 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_file": { 2024-04-24T23:32:01.427 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_file", 2024-04-24T23:32:01.427 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.427 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.427 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.427 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.428 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.428 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.428 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.428 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.428 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.428 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.428 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.428 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.428 INFO:teuthology.orchestra.run.smithi080.stdout: "zabbix_host": { 2024-04-24T23:32:01.429 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "zabbix_host", 2024-04-24T23:32:01.429 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.429 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.429 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.429 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.429 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.429 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.429 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.429 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.430 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.430 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.430 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.430 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.430 INFO:teuthology.orchestra.run.smithi080.stdout: "zabbix_port": { 2024-04-24T23:32:01.430 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "zabbix_port", 2024-04-24T23:32:01.430 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "int", 2024-04-24T23:32:01.430 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.431 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.431 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "10051", 2024-04-24T23:32:01.431 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.431 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.431 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.431 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.431 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.431 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.431 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.432 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.432 INFO:teuthology.orchestra.run.smithi080.stdout: "zabbix_sender": { 2024-04-24T23:32:01.432 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "zabbix_sender", 2024-04-24T23:32:01.432 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.432 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.432 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.432 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-24T23:32:01.432 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.432 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.433 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.433 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.433 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.433 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.433 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.433 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:01.433 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:01.433 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:01.434 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-24T23:32:01.434 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.434 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-24T23:32:01.434 INFO:teuthology.orchestra.run.smithi080.stdout: "gid": 24125, 2024-04-24T23:32:01.434 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "y", 2024-04-24T23:32:01.434 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr_features": 4540701547738038271, 2024-04-24T23:32:01.434 INFO:teuthology.orchestra.run.smithi080.stdout: "available_modules": [ 2024-04-24T23:32:01.434 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-24T23:32:01.435 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "alerts", 2024-04-24T23:32:01.435 INFO:teuthology.orchestra.run.smithi080.stdout: "can_run": true, 2024-04-24T23:32:01.435 INFO:teuthology.orchestra.run.smithi080.stdout: "error_string": "", 2024-04-24T23:32:01.435 INFO:teuthology.orchestra.run.smithi080.stdout: "module_options": { 2024-04-24T23:32:01.435 INFO:teuthology.orchestra.run.smithi080.stdout: "interval": { 2024-04-24T23:32:01.435 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "interval", 2024-04-24T23:32:01.435 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "secs", 2024-04-24T23:32:01.435 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.435 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.436 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "60", 2024-04-24T23:32:01.436 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.436 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.436 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.436 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "How frequently to reexamine health status", 2024-04-24T23:32:01.436 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.436 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.436 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.436 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.437 INFO:teuthology.orchestra.run.smithi080.stdout: "log_level": { 2024-04-24T23:32:01.437 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_level", 2024-04-24T23:32:01.437 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.437 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.437 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.437 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.437 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.437 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.438 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:01.438 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:01.438 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:01.438 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:01.438 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:01.438 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:01.438 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:01.438 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:01.438 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.439 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.439 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.439 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.439 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.439 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster": { 2024-04-24T23:32:01.439 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster", 2024-04-24T23:32:01.439 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.439 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.439 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.440 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.440 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.440 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.440 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.440 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.440 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.440 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.440 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.440 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.441 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster_level": { 2024-04-24T23:32:01.441 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster_level", 2024-04-24T23:32:01.441 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.441 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.441 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.441 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "info", 2024-04-24T23:32:01.441 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.441 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.441 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:01.442 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:01.442 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:01.442 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:01.442 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:01.442 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:01.442 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:01.442 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:01.442 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.443 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.443 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.443 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.443 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.443 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_file": { 2024-04-24T23:32:01.443 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_file", 2024-04-24T23:32:01.443 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.443 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.443 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.444 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.444 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.444 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.444 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.444 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.444 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.444 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.444 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.444 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.445 INFO:teuthology.orchestra.run.smithi080.stdout: "smtp_destination": { 2024-04-24T23:32:01.445 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "smtp_destination", 2024-04-24T23:32:01.445 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.445 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.445 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.445 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.445 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.445 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.446 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.446 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Email address to send alerts to", 2024-04-24T23:32:01.446 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.446 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.446 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.446 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.446 INFO:teuthology.orchestra.run.smithi080.stdout: "smtp_from_name": { 2024-04-24T23:32:01.446 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "smtp_from_name", 2024-04-24T23:32:01.446 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.447 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.447 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.447 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "Ceph", 2024-04-24T23:32:01.447 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.447 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.447 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.447 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Email From: name", 2024-04-24T23:32:01.447 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.447 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.448 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.448 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.448 INFO:teuthology.orchestra.run.smithi080.stdout: "smtp_host": { 2024-04-24T23:32:01.448 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "smtp_host", 2024-04-24T23:32:01.448 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.448 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.448 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.448 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.449 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.449 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.449 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.449 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "SMTP server", 2024-04-24T23:32:01.449 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.449 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.449 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.449 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.449 INFO:teuthology.orchestra.run.smithi080.stdout: "smtp_password": { 2024-04-24T23:32:01.450 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "smtp_password", 2024-04-24T23:32:01.450 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.450 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.450 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.450 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.450 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.450 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.450 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.450 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Password to authenticate with", 2024-04-24T23:32:01.451 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.451 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.451 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.451 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.451 INFO:teuthology.orchestra.run.smithi080.stdout: "smtp_port": { 2024-04-24T23:32:01.451 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "smtp_port", 2024-04-24T23:32:01.451 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "int", 2024-04-24T23:32:01.451 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.452 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.452 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "465", 2024-04-24T23:32:01.452 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.452 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.452 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.452 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "SMTP port", 2024-04-24T23:32:01.452 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.452 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.452 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.453 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.453 INFO:teuthology.orchestra.run.smithi080.stdout: "smtp_sender": { 2024-04-24T23:32:01.453 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "smtp_sender", 2024-04-24T23:32:01.453 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.453 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.453 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.453 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.453 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.453 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.454 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.454 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "SMTP envelope sender", 2024-04-24T23:32:01.454 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.454 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.454 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.454 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.454 INFO:teuthology.orchestra.run.smithi080.stdout: "smtp_ssl": { 2024-04-24T23:32:01.454 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "smtp_ssl", 2024-04-24T23:32:01.454 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.455 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.455 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.455 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "True", 2024-04-24T23:32:01.455 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.455 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.455 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.455 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-24T23:32:01.455 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.456 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.456 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.456 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.456 INFO:teuthology.orchestra.run.smithi080.stdout: "smtp_user": { 2024-04-24T23:32:01.456 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "smtp_user", 2024-04-24T23:32:01.456 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.456 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.456 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.457 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.457 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.457 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.457 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.457 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "User to authenticate as", 2024-04-24T23:32:01.457 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.457 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.457 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.457 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:01.458 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:01.458 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.458 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-24T23:32:01.458 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "balancer", 2024-04-24T23:32:01.458 INFO:teuthology.orchestra.run.smithi080.stdout: "can_run": true, 2024-04-24T23:32:01.458 INFO:teuthology.orchestra.run.smithi080.stdout: "error_string": "", 2024-04-24T23:32:01.458 INFO:teuthology.orchestra.run.smithi080.stdout: "module_options": { 2024-04-24T23:32:01.458 INFO:teuthology.orchestra.run.smithi080.stdout: "active": { 2024-04-24T23:32:01.459 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "active", 2024-04-24T23:32:01.459 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.459 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.459 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.459 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "True", 2024-04-24T23:32:01.459 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.459 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.459 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.459 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "automatically balance PGs across cluster", 2024-04-24T23:32:01.460 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.460 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.460 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.460 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.460 INFO:teuthology.orchestra.run.smithi080.stdout: "begin_time": { 2024-04-24T23:32:01.460 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "begin_time", 2024-04-24T23:32:01.460 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.460 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.460 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.461 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "0000", 2024-04-24T23:32:01.461 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.461 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.461 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.461 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "beginning time of day to automatically balance", 2024-04-24T23:32:01.461 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-24T23:32:01.461 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.461 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.461 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.462 INFO:teuthology.orchestra.run.smithi080.stdout: "begin_weekday": { 2024-04-24T23:32:01.462 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "begin_weekday", 2024-04-24T23:32:01.462 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "uint", 2024-04-24T23:32:01.462 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.462 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.462 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "0", 2024-04-24T23:32:01.462 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "0", 2024-04-24T23:32:01.462 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "6", 2024-04-24T23:32:01.463 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.463 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-24T23:32:01.463 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-24T23:32:01.463 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.463 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.463 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.463 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_compat_max_iterations": { 2024-04-24T23:32:01.463 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "crush_compat_max_iterations", 2024-04-24T23:32:01.463 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "uint", 2024-04-24T23:32:01.464 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.464 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.464 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "25", 2024-04-24T23:32:01.464 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "1", 2024-04-24T23:32:01.464 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "250", 2024-04-24T23:32:01.464 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.464 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-24T23:32:01.464 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.464 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.465 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.465 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.465 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_compat_metrics": { 2024-04-24T23:32:01.465 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "crush_compat_metrics", 2024-04-24T23:32:01.465 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.465 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.465 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.465 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "pgs,objects,bytes", 2024-04-24T23:32:01.465 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.466 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.466 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.466 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-24T23:32:01.466 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-24T23:32:01.466 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.466 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.466 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.466 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_compat_step": { 2024-04-24T23:32:01.467 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "crush_compat_step", 2024-04-24T23:32:01.467 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "float", 2024-04-24T23:32:01.467 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.467 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.467 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "0.5", 2024-04-24T23:32:01.467 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "0.001", 2024-04-24T23:32:01.467 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "0.999", 2024-04-24T23:32:01.467 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.467 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "aggressiveness of optimization", 2024-04-24T23:32:01.468 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-24T23:32:01.468 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.468 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.468 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.468 INFO:teuthology.orchestra.run.smithi080.stdout: "end_time": { 2024-04-24T23:32:01.468 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "end_time", 2024-04-24T23:32:01.468 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.468 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.468 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.469 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "2359", 2024-04-24T23:32:01.469 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.469 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.469 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.469 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "ending time of day to automatically balance", 2024-04-24T23:32:01.469 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-24T23:32:01.469 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.469 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.469 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.470 INFO:teuthology.orchestra.run.smithi080.stdout: "end_weekday": { 2024-04-24T23:32:01.470 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "end_weekday", 2024-04-24T23:32:01.470 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "uint", 2024-04-24T23:32:01.470 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.470 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.470 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "0", 2024-04-24T23:32:01.470 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "0", 2024-04-24T23:32:01.470 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "6", 2024-04-24T23:32:01.471 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.471 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-24T23:32:01.471 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-24T23:32:01.471 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.471 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.471 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.471 INFO:teuthology.orchestra.run.smithi080.stdout: "log_level": { 2024-04-24T23:32:01.471 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_level", 2024-04-24T23:32:01.471 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.472 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.472 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.472 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.472 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.472 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.472 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:01.472 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:01.472 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:01.472 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:01.473 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:01.473 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:01.473 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:01.473 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:01.473 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.473 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.473 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.473 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.473 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.474 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster": { 2024-04-24T23:32:01.474 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster", 2024-04-24T23:32:01.474 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.474 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.474 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.474 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.474 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.474 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.475 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.475 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.475 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.475 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.475 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.475 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.475 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster_level": { 2024-04-24T23:32:01.475 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster_level", 2024-04-24T23:32:01.475 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.476 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.476 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.476 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "info", 2024-04-24T23:32:01.476 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.476 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.476 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:01.476 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:01.476 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:01.477 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:01.477 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:01.477 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:01.477 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:01.477 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:01.477 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.477 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.477 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.477 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.478 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.478 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_file": { 2024-04-24T23:32:01.478 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_file", 2024-04-24T23:32:01.478 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.478 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.478 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.478 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.478 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.479 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.479 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.479 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.479 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.479 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.479 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.479 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.479 INFO:teuthology.orchestra.run.smithi080.stdout: "min_score": { 2024-04-24T23:32:01.479 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "min_score", 2024-04-24T23:32:01.480 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "float", 2024-04-24T23:32:01.480 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.480 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.480 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "0", 2024-04-24T23:32:01.480 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.480 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.480 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.480 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-24T23:32:01.480 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.481 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.481 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.481 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.481 INFO:teuthology.orchestra.run.smithi080.stdout: "mode": { 2024-04-24T23:32:01.481 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "mode", 2024-04-24T23:32:01.481 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.481 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.481 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.481 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "upmap", 2024-04-24T23:32:01.482 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.482 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.482 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:01.482 INFO:teuthology.orchestra.run.smithi080.stdout: "crush-compat", 2024-04-24T23:32:01.482 INFO:teuthology.orchestra.run.smithi080.stdout: "none", 2024-04-24T23:32:01.482 INFO:teuthology.orchestra.run.smithi080.stdout: "read", 2024-04-24T23:32:01.482 INFO:teuthology.orchestra.run.smithi080.stdout: "upmap", 2024-04-24T23:32:01.482 INFO:teuthology.orchestra.run.smithi080.stdout: "upmap-read" 2024-04-24T23:32:01.483 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:01.483 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Balancer mode", 2024-04-24T23:32:01.483 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.483 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.483 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.483 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.483 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_ids": { 2024-04-24T23:32:01.483 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "pool_ids", 2024-04-24T23:32:01.483 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.484 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.484 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.484 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.484 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.484 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.484 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.484 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-24T23:32:01.484 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.484 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.485 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.485 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.485 INFO:teuthology.orchestra.run.smithi080.stdout: "sleep_interval": { 2024-04-24T23:32:01.485 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "sleep_interval", 2024-04-24T23:32:01.485 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "secs", 2024-04-24T23:32:01.485 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.485 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.485 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "60", 2024-04-24T23:32:01.485 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.486 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.486 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.486 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-24T23:32:01.486 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.486 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.486 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.486 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.486 INFO:teuthology.orchestra.run.smithi080.stdout: "upmap_max_deviation": { 2024-04-24T23:32:01.486 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "upmap_max_deviation", 2024-04-24T23:32:01.487 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "int", 2024-04-24T23:32:01.487 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.487 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.487 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "5", 2024-04-24T23:32:01.487 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "1", 2024-04-24T23:32:01.487 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.487 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.487 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-24T23:32:01.487 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-24T23:32:01.488 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.488 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.488 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.488 INFO:teuthology.orchestra.run.smithi080.stdout: "upmap_max_optimizations": { 2024-04-24T23:32:01.488 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "upmap_max_optimizations", 2024-04-24T23:32:01.488 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "uint", 2024-04-24T23:32:01.488 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.488 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.489 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "10", 2024-04-24T23:32:01.489 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.489 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.489 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.489 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-24T23:32:01.489 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.489 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.489 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.489 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:01.490 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:01.490 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.490 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-24T23:32:01.490 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "cephadm", 2024-04-24T23:32:01.490 INFO:teuthology.orchestra.run.smithi080.stdout: "can_run": true, 2024-04-24T23:32:01.490 INFO:teuthology.orchestra.run.smithi080.stdout: "error_string": "", 2024-04-24T23:32:01.490 INFO:teuthology.orchestra.run.smithi080.stdout: "module_options": { 2024-04-24T23:32:01.490 INFO:teuthology.orchestra.run.smithi080.stdout: "agent_down_multiplier": { 2024-04-24T23:32:01.490 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "agent_down_multiplier", 2024-04-24T23:32:01.491 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "float", 2024-04-24T23:32:01.491 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.491 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.491 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "3.0", 2024-04-24T23:32:01.491 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.491 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.491 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.491 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-24T23:32:01.492 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.492 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.492 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.492 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.492 INFO:teuthology.orchestra.run.smithi080.stdout: "agent_refresh_rate": { 2024-04-24T23:32:01.492 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "agent_refresh_rate", 2024-04-24T23:32:01.492 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "secs", 2024-04-24T23:32:01.492 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.492 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.493 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "20", 2024-04-24T23:32:01.493 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.493 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.493 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.493 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-24T23:32:01.493 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.493 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.493 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.493 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.494 INFO:teuthology.orchestra.run.smithi080.stdout: "agent_starting_port": { 2024-04-24T23:32:01.494 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "agent_starting_port", 2024-04-24T23:32:01.494 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "int", 2024-04-24T23:32:01.494 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.494 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.494 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "4721", 2024-04-24T23:32:01.494 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.494 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.494 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.495 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-24T23:32:01.495 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.495 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.495 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.495 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.495 INFO:teuthology.orchestra.run.smithi080.stdout: "allow_ptrace": { 2024-04-24T23:32:01.495 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "allow_ptrace", 2024-04-24T23:32:01.495 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.496 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.496 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.496 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.496 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.496 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.496 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.496 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-24T23:32:01.496 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-24T23:32:01.497 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.497 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.497 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.497 INFO:teuthology.orchestra.run.smithi080.stdout: "autotune_interval": { 2024-04-24T23:32:01.497 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "autotune_interval", 2024-04-24T23:32:01.497 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "secs", 2024-04-24T23:32:01.497 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.497 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.497 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "600", 2024-04-24T23:32:01.498 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.498 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.498 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.498 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-24T23:32:01.498 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.498 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.498 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.498 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.499 INFO:teuthology.orchestra.run.smithi080.stdout: "autotune_memory_target_ratio": { 2024-04-24T23:32:01.499 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "autotune_memory_target_ratio", 2024-04-24T23:32:01.499 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "float", 2024-04-24T23:32:01.499 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.499 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.499 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "0.7", 2024-04-24T23:32:01.499 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.499 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.499 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.500 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-24T23:32:01.500 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.500 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.500 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.500 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.500 INFO:teuthology.orchestra.run.smithi080.stdout: "cephadm_log_destination": { 2024-04-24T23:32:01.500 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "cephadm_log_destination", 2024-04-24T23:32:01.500 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.500 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.501 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.501 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.501 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.501 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.501 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:01.501 INFO:teuthology.orchestra.run.smithi080.stdout: "file", 2024-04-24T23:32:01.501 INFO:teuthology.orchestra.run.smithi080.stdout: "file,syslog", 2024-04-24T23:32:01.501 INFO:teuthology.orchestra.run.smithi080.stdout: "syslog" 2024-04-24T23:32:01.501 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:01.502 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-24T23:32:01.502 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.502 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.502 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.502 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.502 INFO:teuthology.orchestra.run.smithi080.stdout: "cgroups_split": { 2024-04-24T23:32:01.502 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "cgroups_split", 2024-04-24T23:32:01.502 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.503 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.503 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.503 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "True", 2024-04-24T23:32:01.503 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.503 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.503 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.503 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-24T23:32:01.503 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.503 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.504 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.504 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.504 INFO:teuthology.orchestra.run.smithi080.stdout: "config_checks_enabled": { 2024-04-24T23:32:01.504 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "config_checks_enabled", 2024-04-24T23:32:01.504 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.504 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.504 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.504 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.504 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.505 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.505 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.505 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-24T23:32:01.505 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.505 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.505 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.505 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.505 INFO:teuthology.orchestra.run.smithi080.stdout: "config_dashboard": { 2024-04-24T23:32:01.506 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "config_dashboard", 2024-04-24T23:32:01.506 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.506 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.506 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.506 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "True", 2024-04-24T23:32:01.506 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.506 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.506 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.506 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-24T23:32:01.507 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.507 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.507 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.507 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.507 INFO:teuthology.orchestra.run.smithi080.stdout: "container_image_alertmanager": { 2024-04-24T23:32:01.507 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "container_image_alertmanager", 2024-04-24T23:32:01.507 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.507 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.507 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.508 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-24T23:32:01.508 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.508 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.508 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.508 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Prometheus container image", 2024-04-24T23:32:01.508 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.508 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.508 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.509 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.509 INFO:teuthology.orchestra.run.smithi080.stdout: "container_image_base": { 2024-04-24T23:32:01.509 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "container_image_base", 2024-04-24T23:32:01.509 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.509 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.509 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.509 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-24T23:32:01.509 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.509 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.510 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.510 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Container image name, without the tag", 2024-04-24T23:32:01.510 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.510 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.510 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.510 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.510 INFO:teuthology.orchestra.run.smithi080.stdout: "container_image_elasticsearch": { 2024-04-24T23:32:01.510 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "container_image_elasticsearch", 2024-04-24T23:32:01.510 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.511 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.511 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.511 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-24T23:32:01.511 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.511 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.511 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.511 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "elasticsearch container image", 2024-04-24T23:32:01.511 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.512 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.512 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.512 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.512 INFO:teuthology.orchestra.run.smithi080.stdout: "container_image_grafana": { 2024-04-24T23:32:01.512 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "container_image_grafana", 2024-04-24T23:32:01.512 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.512 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.512 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.512 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-24T23:32:01.513 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.513 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.513 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.513 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Prometheus container image", 2024-04-24T23:32:01.513 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.513 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.513 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.513 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.513 INFO:teuthology.orchestra.run.smithi080.stdout: "container_image_haproxy": { 2024-04-24T23:32:01.514 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "container_image_haproxy", 2024-04-24T23:32:01.514 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.514 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.514 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.514 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-24T23:32:01.514 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.514 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.514 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.514 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "HAproxy container image", 2024-04-24T23:32:01.515 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.515 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.515 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.515 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.515 INFO:teuthology.orchestra.run.smithi080.stdout: "container_image_jaeger_agent": { 2024-04-24T23:32:01.515 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "container_image_jaeger_agent", 2024-04-24T23:32:01.515 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.515 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.516 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.516 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-24T23:32:01.516 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.516 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.516 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.516 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Jaeger agent container image", 2024-04-24T23:32:01.516 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.516 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.517 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.517 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.517 INFO:teuthology.orchestra.run.smithi080.stdout: "container_image_jaeger_collector": { 2024-04-24T23:32:01.517 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "container_image_jaeger_collector", 2024-04-24T23:32:01.517 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.517 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.517 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.517 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-24T23:32:01.517 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.518 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.518 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.518 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Jaeger collector container image", 2024-04-24T23:32:01.518 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.518 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.518 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.518 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.518 INFO:teuthology.orchestra.run.smithi080.stdout: "container_image_jaeger_query": { 2024-04-24T23:32:01.519 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "container_image_jaeger_query", 2024-04-24T23:32:01.519 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.519 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.519 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.519 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-24T23:32:01.519 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.519 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.519 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.519 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Jaeger query container image", 2024-04-24T23:32:01.520 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.520 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.520 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.520 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.520 INFO:teuthology.orchestra.run.smithi080.stdout: "container_image_keepalived": { 2024-04-24T23:32:01.520 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "container_image_keepalived", 2024-04-24T23:32:01.520 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.520 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.520 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.521 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-24T23:32:01.521 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.521 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.521 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.521 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Keepalived container image", 2024-04-24T23:32:01.521 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.521 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.521 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.521 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.522 INFO:teuthology.orchestra.run.smithi080.stdout: "container_image_loki": { 2024-04-24T23:32:01.522 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "container_image_loki", 2024-04-24T23:32:01.522 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.522 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.522 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.522 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-24T23:32:01.522 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.522 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.523 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.523 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Loki container image", 2024-04-24T23:32:01.523 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.523 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.523 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.523 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.523 INFO:teuthology.orchestra.run.smithi080.stdout: "container_image_node_exporter": { 2024-04-24T23:32:01.523 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "container_image_node_exporter", 2024-04-24T23:32:01.523 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.524 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.524 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.524 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-24T23:32:01.524 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.524 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.524 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.524 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Prometheus container image", 2024-04-24T23:32:01.524 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.524 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.525 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.525 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.525 INFO:teuthology.orchestra.run.smithi080.stdout: "container_image_nvmeof": { 2024-04-24T23:32:01.525 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "container_image_nvmeof", 2024-04-24T23:32:01.525 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.525 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.525 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.525 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-24T23:32:01.525 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.526 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.526 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.526 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Nvme-of container image", 2024-04-24T23:32:01.526 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.526 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.526 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.526 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.526 INFO:teuthology.orchestra.run.smithi080.stdout: "container_image_prometheus": { 2024-04-24T23:32:01.527 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "container_image_prometheus", 2024-04-24T23:32:01.527 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.527 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.527 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.527 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-24T23:32:01.527 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.527 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.527 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.527 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Prometheus container image", 2024-04-24T23:32:01.528 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.528 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.528 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.528 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.528 INFO:teuthology.orchestra.run.smithi080.stdout: "container_image_promtail": { 2024-04-24T23:32:01.528 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "container_image_promtail", 2024-04-24T23:32:01.528 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.528 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.528 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.529 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-24T23:32:01.529 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.529 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.529 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.529 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Promtail container image", 2024-04-24T23:32:01.529 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.529 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.529 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.530 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.530 INFO:teuthology.orchestra.run.smithi080.stdout: "container_image_samba": { 2024-04-24T23:32:01.530 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "container_image_samba", 2024-04-24T23:32:01.530 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.530 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.530 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.530 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-24T23:32:01.530 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.530 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.531 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.531 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Samba/SMB container image", 2024-04-24T23:32:01.531 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.531 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.531 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.531 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.531 INFO:teuthology.orchestra.run.smithi080.stdout: "container_image_snmp_gateway": { 2024-04-24T23:32:01.531 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "container_image_snmp_gateway", 2024-04-24T23:32:01.531 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.532 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.532 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.532 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-24T23:32:01.532 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.532 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.532 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.532 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "SNMP Gateway container image", 2024-04-24T23:32:01.532 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.532 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.533 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.533 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.533 INFO:teuthology.orchestra.run.smithi080.stdout: "container_init": { 2024-04-24T23:32:01.533 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "container_init", 2024-04-24T23:32:01.533 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.533 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.533 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.533 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "True", 2024-04-24T23:32:01.534 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.534 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.534 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.534 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Run podman/docker with `--init`", 2024-04-24T23:32:01.534 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.534 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.534 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.534 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.534 INFO:teuthology.orchestra.run.smithi080.stdout: "daemon_cache_timeout": { 2024-04-24T23:32:01.535 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "daemon_cache_timeout", 2024-04-24T23:32:01.535 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "secs", 2024-04-24T23:32:01.535 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.535 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.535 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "600", 2024-04-24T23:32:01.535 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.535 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.535 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.536 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-24T23:32:01.536 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.536 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.536 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.536 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.536 INFO:teuthology.orchestra.run.smithi080.stdout: "default_cephadm_command_timeout": { 2024-04-24T23:32:01.536 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "default_cephadm_command_timeout", 2024-04-24T23:32:01.536 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "int", 2024-04-24T23:32:01.537 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.537 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.537 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "900", 2024-04-24T23:32:01.537 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.537 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.537 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.537 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-24T23:32:01.537 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.537 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.538 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.538 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.538 INFO:teuthology.orchestra.run.smithi080.stdout: "default_registry": { 2024-04-24T23:32:01.538 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "default_registry", 2024-04-24T23:32:01.538 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.538 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.538 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.538 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "docker.io", 2024-04-24T23:32:01.538 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.539 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.539 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.539 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-24T23:32:01.539 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.539 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.539 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.539 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.539 INFO:teuthology.orchestra.run.smithi080.stdout: "device_cache_timeout": { 2024-04-24T23:32:01.540 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "device_cache_timeout", 2024-04-24T23:32:01.540 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "secs", 2024-04-24T23:32:01.540 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.540 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.540 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "1800", 2024-04-24T23:32:01.540 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.540 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.540 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.540 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "seconds to cache device inventory", 2024-04-24T23:32:01.541 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.541 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.541 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.541 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.541 INFO:teuthology.orchestra.run.smithi080.stdout: "device_enhanced_scan": { 2024-04-24T23:32:01.541 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "device_enhanced_scan", 2024-04-24T23:32:01.541 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.541 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.541 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.542 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.542 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.542 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.542 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.542 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-24T23:32:01.542 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.542 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.542 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.543 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.543 INFO:teuthology.orchestra.run.smithi080.stdout: "facts_cache_timeout": { 2024-04-24T23:32:01.543 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "facts_cache_timeout", 2024-04-24T23:32:01.543 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "secs", 2024-04-24T23:32:01.543 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.543 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.543 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "60", 2024-04-24T23:32:01.543 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.543 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.544 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.544 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "seconds to cache host facts data", 2024-04-24T23:32:01.544 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.544 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.544 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.544 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.544 INFO:teuthology.orchestra.run.smithi080.stdout: "grafana_dashboards_path": { 2024-04-24T23:32:01.544 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "grafana_dashboards_path", 2024-04-24T23:32:01.544 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.545 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.545 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.545 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-24T23:32:01.545 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.545 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.545 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.545 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-24T23:32:01.545 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.545 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.546 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.546 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.546 INFO:teuthology.orchestra.run.smithi080.stdout: "host_check_interval": { 2024-04-24T23:32:01.546 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "host_check_interval", 2024-04-24T23:32:01.546 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "secs", 2024-04-24T23:32:01.546 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.546 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.546 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "600", 2024-04-24T23:32:01.547 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.547 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.547 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.547 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "how frequently to perform a host check", 2024-04-24T23:32:01.547 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.547 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.547 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.547 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.547 INFO:teuthology.orchestra.run.smithi080.stdout: "hw_monitoring": { 2024-04-24T23:32:01.548 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "hw_monitoring", 2024-04-24T23:32:01.548 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.548 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.548 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.548 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.548 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.548 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.548 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.549 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-24T23:32:01.549 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.549 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.549 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.549 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.549 INFO:teuthology.orchestra.run.smithi080.stdout: "inventory_list_all": { 2024-04-24T23:32:01.549 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "inventory_list_all", 2024-04-24T23:32:01.549 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.549 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.550 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.550 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.550 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.550 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.550 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.550 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-24T23:32:01.550 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.550 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.551 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.551 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.551 INFO:teuthology.orchestra.run.smithi080.stdout: "log_level": { 2024-04-24T23:32:01.551 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_level", 2024-04-24T23:32:01.551 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.551 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.551 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.551 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.551 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.552 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.552 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:01.552 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:01.552 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:01.552 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:01.552 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:01.552 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:01.552 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:01.552 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:01.553 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.553 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.553 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.553 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.553 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.553 INFO:teuthology.orchestra.run.smithi080.stdout: "log_refresh_metadata": { 2024-04-24T23:32:01.553 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_refresh_metadata", 2024-04-24T23:32:01.553 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.553 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.554 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.554 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.554 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.554 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.554 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.554 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-24T23:32:01.554 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.554 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.555 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.555 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.555 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster": { 2024-04-24T23:32:01.555 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster", 2024-04-24T23:32:01.555 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.555 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.555 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.555 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "True", 2024-04-24T23:32:01.556 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.556 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.556 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.556 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-24T23:32:01.556 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.556 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.556 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.556 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.556 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster_level": { 2024-04-24T23:32:01.557 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster_level", 2024-04-24T23:32:01.557 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.557 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.557 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.557 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "info", 2024-04-24T23:32:01.557 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.557 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.557 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:01.558 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:01.558 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:01.558 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:01.558 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:01.558 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:01.558 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:01.558 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:01.558 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.558 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.559 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.559 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.559 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.559 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_file": { 2024-04-24T23:32:01.559 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_file", 2024-04-24T23:32:01.559 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.559 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.559 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.560 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.560 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.560 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.560 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.560 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.560 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.560 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.560 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.560 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.561 INFO:teuthology.orchestra.run.smithi080.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-24T23:32:01.561 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-24T23:32:01.561 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.561 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.561 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.561 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.561 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.561 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.561 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.562 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-24T23:32:01.562 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.562 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.562 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.562 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.562 INFO:teuthology.orchestra.run.smithi080.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-24T23:32:01.562 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-24T23:32:01.562 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.562 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.563 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.563 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "*", 2024-04-24T23:32:01.563 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.563 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.563 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.563 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-24T23:32:01.563 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.563 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.564 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.564 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.564 INFO:teuthology.orchestra.run.smithi080.stdout: "max_count_per_host": { 2024-04-24T23:32:01.564 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "max_count_per_host", 2024-04-24T23:32:01.564 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "int", 2024-04-24T23:32:01.564 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.564 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.564 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "10", 2024-04-24T23:32:01.564 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.565 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.565 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.565 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "max number of daemons per service per host", 2024-04-24T23:32:01.565 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.565 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.565 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.565 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.565 INFO:teuthology.orchestra.run.smithi080.stdout: "max_osd_draining_count": { 2024-04-24T23:32:01.565 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "max_osd_draining_count", 2024-04-24T23:32:01.566 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "int", 2024-04-24T23:32:01.566 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.566 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.566 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "10", 2024-04-24T23:32:01.566 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.566 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.566 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.566 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-24T23:32:01.567 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.567 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.567 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.567 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.567 INFO:teuthology.orchestra.run.smithi080.stdout: "migration_current": { 2024-04-24T23:32:01.569 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "migration_current", 2024-04-24T23:32:01.569 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "int", 2024-04-24T23:32:01.569 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.569 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.569 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.570 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.570 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.570 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.570 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "internal - do not modify", 2024-04-24T23:32:01.570 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.570 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.570 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.570 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.571 INFO:teuthology.orchestra.run.smithi080.stdout: "mode": { 2024-04-24T23:32:01.571 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "mode", 2024-04-24T23:32:01.571 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.571 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.571 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.571 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "root", 2024-04-24T23:32:01.571 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.571 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.571 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:01.572 INFO:teuthology.orchestra.run.smithi080.stdout: "cephadm-package", 2024-04-24T23:32:01.572 INFO:teuthology.orchestra.run.smithi080.stdout: "root" 2024-04-24T23:32:01.572 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:01.572 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "mode for remote execution of cephadm", 2024-04-24T23:32:01.572 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.572 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.572 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.572 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.572 INFO:teuthology.orchestra.run.smithi080.stdout: "oob_default_addr": { 2024-04-24T23:32:01.573 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "oob_default_addr", 2024-04-24T23:32:01.573 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.573 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.573 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.573 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "169.254.1.1", 2024-04-24T23:32:01.573 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.573 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.573 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.574 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-24T23:32:01.574 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.574 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.574 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.574 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.574 INFO:teuthology.orchestra.run.smithi080.stdout: "prometheus_alerts_path": { 2024-04-24T23:32:01.574 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "prometheus_alerts_path", 2024-04-24T23:32:01.574 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.574 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.575 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.575 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-24T23:32:01.575 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.575 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.575 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.575 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-24T23:32:01.575 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.575 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.575 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.576 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.576 INFO:teuthology.orchestra.run.smithi080.stdout: "registry_insecure": { 2024-04-24T23:32:01.576 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "registry_insecure", 2024-04-24T23:32:01.576 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.576 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.576 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.576 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.576 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.577 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.577 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.577 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-24T23:32:01.577 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.577 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.577 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.577 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.577 INFO:teuthology.orchestra.run.smithi080.stdout: "registry_password": { 2024-04-24T23:32:01.578 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "registry_password", 2024-04-24T23:32:01.578 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.578 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.578 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.578 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.578 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.578 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.578 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.578 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-24T23:32:01.579 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.579 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.579 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.579 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.579 INFO:teuthology.orchestra.run.smithi080.stdout: "registry_url": { 2024-04-24T23:32:01.579 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "registry_url", 2024-04-24T23:32:01.579 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.579 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.579 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.580 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.580 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.580 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.580 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.580 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-24T23:32:01.580 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.580 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.580 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.581 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.581 INFO:teuthology.orchestra.run.smithi080.stdout: "registry_username": { 2024-04-24T23:32:01.581 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "registry_username", 2024-04-24T23:32:01.581 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.581 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.581 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.581 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.581 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.581 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.582 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.582 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-24T23:32:01.582 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.582 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.582 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.582 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.582 INFO:teuthology.orchestra.run.smithi080.stdout: "secure_monitoring_stack": { 2024-04-24T23:32:01.582 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "secure_monitoring_stack", 2024-04-24T23:32:01.583 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.583 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.583 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.583 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.583 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.583 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.583 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.583 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-24T23:32:01.583 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.584 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.584 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.584 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.584 INFO:teuthology.orchestra.run.smithi080.stdout: "service_discovery_port": { 2024-04-24T23:32:01.584 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "service_discovery_port", 2024-04-24T23:32:01.584 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "int", 2024-04-24T23:32:01.584 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.584 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.584 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "8765", 2024-04-24T23:32:01.585 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.585 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.585 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.585 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "cephadm service discovery port", 2024-04-24T23:32:01.585 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.585 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.585 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.585 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.585 INFO:teuthology.orchestra.run.smithi080.stdout: "ssh_config_file": { 2024-04-24T23:32:01.586 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "ssh_config_file", 2024-04-24T23:32:01.586 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.586 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.586 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.586 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.586 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.586 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.586 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.587 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-24T23:32:01.587 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.587 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.587 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.587 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.587 INFO:teuthology.orchestra.run.smithi080.stdout: "use_agent": { 2024-04-24T23:32:01.587 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "use_agent", 2024-04-24T23:32:01.587 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.587 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.588 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.588 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.588 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.588 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.588 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.588 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-24T23:32:01.588 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.588 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.588 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.589 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.589 INFO:teuthology.orchestra.run.smithi080.stdout: "use_repo_digest": { 2024-04-24T23:32:01.589 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "use_repo_digest", 2024-04-24T23:32:01.589 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.589 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.589 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.589 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "True", 2024-04-24T23:32:01.589 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.590 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.590 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.590 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-24T23:32:01.590 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.590 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.590 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.590 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.590 INFO:teuthology.orchestra.run.smithi080.stdout: "warn_on_failed_host_check": { 2024-04-24T23:32:01.591 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "warn_on_failed_host_check", 2024-04-24T23:32:01.591 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.591 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.591 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.591 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "True", 2024-04-24T23:32:01.591 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.591 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.591 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.591 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "raise a health warning if the host check fails", 2024-04-24T23:32:01.592 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.592 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.592 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.592 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.592 INFO:teuthology.orchestra.run.smithi080.stdout: "warn_on_stray_daemons": { 2024-04-24T23:32:01.592 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "warn_on_stray_daemons", 2024-04-24T23:32:01.592 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.592 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.592 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.593 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "True", 2024-04-24T23:32:01.593 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.593 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.593 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.593 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-24T23:32:01.593 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.593 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.593 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.594 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.594 INFO:teuthology.orchestra.run.smithi080.stdout: "warn_on_stray_hosts": { 2024-04-24T23:32:01.594 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "warn_on_stray_hosts", 2024-04-24T23:32:01.594 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.594 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.594 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.594 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "True", 2024-04-24T23:32:01.594 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.594 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.595 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.595 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-24T23:32:01.595 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.595 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.595 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.595 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:01.595 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:01.595 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.595 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-24T23:32:01.596 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "crash", 2024-04-24T23:32:01.596 INFO:teuthology.orchestra.run.smithi080.stdout: "can_run": true, 2024-04-24T23:32:01.596 INFO:teuthology.orchestra.run.smithi080.stdout: "error_string": "", 2024-04-24T23:32:01.596 INFO:teuthology.orchestra.run.smithi080.stdout: "module_options": { 2024-04-24T23:32:01.596 INFO:teuthology.orchestra.run.smithi080.stdout: "log_level": { 2024-04-24T23:32:01.596 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_level", 2024-04-24T23:32:01.596 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.596 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.597 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.597 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.597 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.597 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.597 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:01.597 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:01.597 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:01.597 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:01.598 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:01.598 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:01.598 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:01.598 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:01.598 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.598 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.598 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.598 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.598 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.599 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster": { 2024-04-24T23:32:01.599 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster", 2024-04-24T23:32:01.599 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.599 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.599 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.599 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.599 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.599 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.599 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.600 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.600 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.600 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.600 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.600 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.600 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster_level": { 2024-04-24T23:32:01.600 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster_level", 2024-04-24T23:32:01.600 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.601 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.601 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.601 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "info", 2024-04-24T23:32:01.601 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.601 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.601 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:01.601 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:01.601 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:01.601 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:01.602 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:01.602 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:01.602 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:01.602 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:01.602 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.602 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.602 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.602 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.603 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.603 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_file": { 2024-04-24T23:32:01.603 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_file", 2024-04-24T23:32:01.603 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.603 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.603 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.603 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.603 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.603 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.604 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.604 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.604 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.604 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.604 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.604 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.604 INFO:teuthology.orchestra.run.smithi080.stdout: "retain_interval": { 2024-04-24T23:32:01.604 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "retain_interval", 2024-04-24T23:32:01.604 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "secs", 2024-04-24T23:32:01.605 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.605 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.605 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "31536000", 2024-04-24T23:32:01.605 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.605 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.605 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.605 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-24T23:32:01.605 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.605 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.606 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.606 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.606 INFO:teuthology.orchestra.run.smithi080.stdout: "warn_recent_interval": { 2024-04-24T23:32:01.606 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "warn_recent_interval", 2024-04-24T23:32:01.606 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "secs", 2024-04-24T23:32:01.606 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.606 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.606 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "1209600", 2024-04-24T23:32:01.607 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.607 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.607 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.607 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-24T23:32:01.607 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.607 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.607 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.607 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:01.607 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:01.608 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.608 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-24T23:32:01.608 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "dashboard", 2024-04-24T23:32:01.608 INFO:teuthology.orchestra.run.smithi080.stdout: "can_run": true, 2024-04-24T23:32:01.608 INFO:teuthology.orchestra.run.smithi080.stdout: "error_string": "", 2024-04-24T23:32:01.608 INFO:teuthology.orchestra.run.smithi080.stdout: "module_options": { 2024-04-24T23:32:01.608 INFO:teuthology.orchestra.run.smithi080.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-24T23:32:01.608 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-24T23:32:01.609 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "int", 2024-04-24T23:32:01.609 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.609 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.609 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "10", 2024-04-24T23:32:01.609 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.609 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.609 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.609 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.609 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.610 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.610 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.610 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.610 INFO:teuthology.orchestra.run.smithi080.stdout: "ALERTMANAGER_API_HOST": { 2024-04-24T23:32:01.610 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-24T23:32:01.610 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.610 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.610 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.611 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.611 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.611 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.611 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.611 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.611 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.611 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.611 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.611 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.612 INFO:teuthology.orchestra.run.smithi080.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-24T23:32:01.612 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-24T23:32:01.612 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.612 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.612 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.612 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "True", 2024-04-24T23:32:01.612 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.612 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.612 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.613 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.613 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.613 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.613 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.613 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.613 INFO:teuthology.orchestra.run.smithi080.stdout: "AUDIT_API_ENABLED": { 2024-04-24T23:32:01.613 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "AUDIT_API_ENABLED", 2024-04-24T23:32:01.613 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.613 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.614 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.614 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.614 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.614 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.614 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.614 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.614 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.614 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.615 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.615 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.615 INFO:teuthology.orchestra.run.smithi080.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-24T23:32:01.615 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-24T23:32:01.615 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.615 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.615 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.615 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "True", 2024-04-24T23:32:01.615 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.616 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.616 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.616 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.616 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.616 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.616 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.616 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.616 INFO:teuthology.orchestra.run.smithi080.stdout: "ENABLE_BROWSABLE_API": { 2024-04-24T23:32:01.616 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-24T23:32:01.617 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.617 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.617 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.617 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "True", 2024-04-24T23:32:01.617 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.617 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.617 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.617 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.617 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.618 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.618 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.618 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.618 INFO:teuthology.orchestra.run.smithi080.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-24T23:32:01.618 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-24T23:32:01.618 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.618 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.618 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.619 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "True", 2024-04-24T23:32:01.619 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.619 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.619 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.619 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.619 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.619 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.619 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.620 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.620 INFO:teuthology.orchestra.run.smithi080.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-24T23:32:01.620 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-24T23:32:01.620 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.620 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.620 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.620 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "True", 2024-04-24T23:32:01.620 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.620 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.621 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.621 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.621 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.621 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.621 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.621 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.621 INFO:teuthology.orchestra.run.smithi080.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-24T23:32:01.621 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-24T23:32:01.622 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.622 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.622 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.622 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "True", 2024-04-24T23:32:01.622 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.622 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.622 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.622 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.622 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.623 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.623 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.623 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.623 INFO:teuthology.orchestra.run.smithi080.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-24T23:32:01.623 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-24T23:32:01.623 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.623 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.623 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.624 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "True", 2024-04-24T23:32:01.624 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.624 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.624 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.624 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.624 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.624 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.624 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.624 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.625 INFO:teuthology.orchestra.run.smithi080.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-24T23:32:01.625 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-24T23:32:01.625 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.625 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.625 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.625 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "True", 2024-04-24T23:32:01.625 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.625 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.625 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.626 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.626 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.626 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.626 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.626 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.626 INFO:teuthology.orchestra.run.smithi080.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-24T23:32:01.626 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-24T23:32:01.626 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.627 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.627 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.627 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "True", 2024-04-24T23:32:01.627 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.627 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.627 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.627 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.627 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.627 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.628 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.628 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.628 INFO:teuthology.orchestra.run.smithi080.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-24T23:32:01.628 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-24T23:32:01.628 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.628 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.628 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.628 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "True", 2024-04-24T23:32:01.628 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.629 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.629 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.629 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.629 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.629 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.629 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.629 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.629 INFO:teuthology.orchestra.run.smithi080.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-24T23:32:01.629 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-24T23:32:01.630 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.630 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.630 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.630 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.630 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.630 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.630 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.630 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.631 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.631 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.631 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.631 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.631 INFO:teuthology.orchestra.run.smithi080.stdout: "GRAFANA_API_PASSWORD": { 2024-04-24T23:32:01.631 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-24T23:32:01.631 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.631 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.631 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.632 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "admin", 2024-04-24T23:32:01.632 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.632 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.632 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.632 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.632 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.632 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.632 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.632 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.633 INFO:teuthology.orchestra.run.smithi080.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-24T23:32:01.633 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-24T23:32:01.633 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.633 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.633 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.633 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "True", 2024-04-24T23:32:01.633 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.633 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.633 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.634 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.634 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.634 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.634 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.634 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.634 INFO:teuthology.orchestra.run.smithi080.stdout: "GRAFANA_API_URL": { 2024-04-24T23:32:01.634 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "GRAFANA_API_URL", 2024-04-24T23:32:01.634 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.635 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.635 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.635 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.635 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.635 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.635 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.635 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.635 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.635 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.636 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.636 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.636 INFO:teuthology.orchestra.run.smithi080.stdout: "GRAFANA_API_USERNAME": { 2024-04-24T23:32:01.636 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-24T23:32:01.636 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.636 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.636 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.636 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "admin", 2024-04-24T23:32:01.636 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.637 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.637 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.637 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.637 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.637 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.637 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.637 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.637 INFO:teuthology.orchestra.run.smithi080.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-24T23:32:01.637 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-24T23:32:01.638 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.638 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.638 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.638 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.638 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.638 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.638 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.638 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.639 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.639 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.639 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.639 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.639 INFO:teuthology.orchestra.run.smithi080.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-24T23:32:01.639 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-24T23:32:01.639 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.639 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.639 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.640 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.640 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.640 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.640 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.640 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.640 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.640 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.640 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.641 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.641 INFO:teuthology.orchestra.run.smithi080.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-24T23:32:01.641 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-24T23:32:01.641 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.641 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.641 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.641 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "True", 2024-04-24T23:32:01.641 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.641 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.642 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.642 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.642 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.642 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.642 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.642 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.642 INFO:teuthology.orchestra.run.smithi080.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-24T23:32:01.642 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-24T23:32:01.643 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.643 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.643 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.643 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.643 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.643 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.643 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.643 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.643 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.644 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.644 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.644 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.644 INFO:teuthology.orchestra.run.smithi080.stdout: "PROMETHEUS_API_HOST": { 2024-04-24T23:32:01.644 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-24T23:32:01.644 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.644 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.644 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.644 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.645 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.645 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.645 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.645 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.645 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.645 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.645 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.645 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.646 INFO:teuthology.orchestra.run.smithi080.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-24T23:32:01.646 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-24T23:32:01.646 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.646 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.646 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.646 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "True", 2024-04-24T23:32:01.646 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.646 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.646 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.647 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.647 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.647 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.647 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.647 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.647 INFO:teuthology.orchestra.run.smithi080.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-24T23:32:01.647 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-24T23:32:01.647 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.647 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.648 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.648 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.648 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.648 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.648 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.648 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.648 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.648 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.649 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.649 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.649 INFO:teuthology.orchestra.run.smithi080.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-24T23:32:01.649 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-24T23:32:01.649 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.649 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.649 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.649 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.649 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.650 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.650 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.650 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.650 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.650 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.650 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.650 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.650 INFO:teuthology.orchestra.run.smithi080.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-24T23:32:01.650 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-24T23:32:01.651 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.651 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.651 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.651 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "True", 2024-04-24T23:32:01.651 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.651 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.651 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.651 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.652 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.652 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.652 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.652 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.652 INFO:teuthology.orchestra.run.smithi080.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-24T23:32:01.652 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-24T23:32:01.652 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.652 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.652 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.653 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "True", 2024-04-24T23:32:01.653 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.653 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.653 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.653 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.653 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.653 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.653 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.653 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.654 INFO:teuthology.orchestra.run.smithi080.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-24T23:32:01.654 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-24T23:32:01.654 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.654 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.654 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.654 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.654 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.654 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.654 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.655 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.655 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.655 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.655 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.655 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.655 INFO:teuthology.orchestra.run.smithi080.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-24T23:32:01.655 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-24T23:32:01.655 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.656 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.656 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.656 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.656 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.656 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.656 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.656 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.656 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.656 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.657 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.657 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.657 INFO:teuthology.orchestra.run.smithi080.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-24T23:32:01.657 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-24T23:32:01.657 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.657 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.657 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.657 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.657 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.658 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.658 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.658 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.658 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.658 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.658 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.658 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.658 INFO:teuthology.orchestra.run.smithi080.stdout: "PWD_POLICY_ENABLED": { 2024-04-24T23:32:01.659 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-24T23:32:01.659 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.659 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.659 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.659 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "True", 2024-04-24T23:32:01.659 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.659 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.659 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.659 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.660 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.660 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.660 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.660 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.660 INFO:teuthology.orchestra.run.smithi080.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-24T23:32:01.660 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-24T23:32:01.660 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.660 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.660 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.661 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-24T23:32:01.661 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.661 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.661 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.661 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.661 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.661 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.661 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.662 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.662 INFO:teuthology.orchestra.run.smithi080.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-24T23:32:01.662 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-24T23:32:01.662 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "int", 2024-04-24T23:32:01.662 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.662 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.662 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "10", 2024-04-24T23:32:01.662 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.663 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.663 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.663 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.663 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.663 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.663 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.663 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.663 INFO:teuthology.orchestra.run.smithi080.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-24T23:32:01.663 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-24T23:32:01.664 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "int", 2024-04-24T23:32:01.664 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.664 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.664 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "8", 2024-04-24T23:32:01.664 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.664 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.664 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.664 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.664 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.665 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.665 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.665 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.665 INFO:teuthology.orchestra.run.smithi080.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-24T23:32:01.665 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-24T23:32:01.665 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "int", 2024-04-24T23:32:01.665 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.665 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.665 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "45", 2024-04-24T23:32:01.666 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.666 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.666 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.666 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.666 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.666 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.666 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.666 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.667 INFO:teuthology.orchestra.run.smithi080.stdout: "RGW_API_ACCESS_KEY": { 2024-04-24T23:32:01.667 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-24T23:32:01.667 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.667 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.667 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.667 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.667 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.667 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.667 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.668 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.668 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.668 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.668 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.668 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.668 INFO:teuthology.orchestra.run.smithi080.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-24T23:32:01.668 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-24T23:32:01.668 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.669 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.669 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.669 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "admin", 2024-04-24T23:32:01.669 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.669 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.669 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.669 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.669 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.669 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.670 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.670 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.670 INFO:teuthology.orchestra.run.smithi080.stdout: "RGW_API_SECRET_KEY": { 2024-04-24T23:32:01.670 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-24T23:32:01.670 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.670 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.670 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.670 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.670 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.671 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.671 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.671 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.671 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.671 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.671 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.671 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.671 INFO:teuthology.orchestra.run.smithi080.stdout: "RGW_API_SSL_VERIFY": { 2024-04-24T23:32:01.672 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-24T23:32:01.672 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.672 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.672 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.672 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "True", 2024-04-24T23:32:01.672 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.672 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.672 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.672 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.673 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.673 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.673 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.673 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.673 INFO:teuthology.orchestra.run.smithi080.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-24T23:32:01.673 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-24T23:32:01.673 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "int", 2024-04-24T23:32:01.673 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.673 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.674 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "0", 2024-04-24T23:32:01.674 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.674 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.674 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.674 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.674 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.674 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.674 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.675 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.675 INFO:teuthology.orchestra.run.smithi080.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-24T23:32:01.675 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-24T23:32:01.675 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "int", 2024-04-24T23:32:01.675 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.675 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.675 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "10", 2024-04-24T23:32:01.675 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.675 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.676 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.676 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.676 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.676 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.676 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.676 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.676 INFO:teuthology.orchestra.run.smithi080.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-24T23:32:01.676 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-24T23:32:01.676 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "int", 2024-04-24T23:32:01.677 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.677 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.677 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "5", 2024-04-24T23:32:01.677 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.677 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.677 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.677 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.677 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.678 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.678 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.678 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.678 INFO:teuthology.orchestra.run.smithi080.stdout: "cross_origin_url": { 2024-04-24T23:32:01.678 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "cross_origin_url", 2024-04-24T23:32:01.678 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.678 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.678 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.678 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.679 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.679 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.679 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.679 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.679 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.679 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.679 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.679 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.679 INFO:teuthology.orchestra.run.smithi080.stdout: "crt_file": { 2024-04-24T23:32:01.680 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "crt_file", 2024-04-24T23:32:01.680 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.680 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.680 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.680 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.680 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.680 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.680 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.680 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.681 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.681 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.681 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.681 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.681 INFO:teuthology.orchestra.run.smithi080.stdout: "debug": { 2024-04-24T23:32:01.681 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "debug", 2024-04-24T23:32:01.681 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.681 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.682 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.682 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.682 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.682 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.682 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.682 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Enable/disable debug options", 2024-04-24T23:32:01.682 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.682 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.682 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.683 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.683 INFO:teuthology.orchestra.run.smithi080.stdout: "jwt_token_ttl": { 2024-04-24T23:32:01.683 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "jwt_token_ttl", 2024-04-24T23:32:01.683 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "int", 2024-04-24T23:32:01.683 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.683 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.683 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "28800", 2024-04-24T23:32:01.683 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.684 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.684 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.684 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.684 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.684 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.684 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.684 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.684 INFO:teuthology.orchestra.run.smithi080.stdout: "key_file": { 2024-04-24T23:32:01.685 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "key_file", 2024-04-24T23:32:01.685 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.685 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.685 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.685 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.685 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.685 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.685 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.685 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.686 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.686 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.686 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.686 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.686 INFO:teuthology.orchestra.run.smithi080.stdout: "log_level": { 2024-04-24T23:32:01.686 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_level", 2024-04-24T23:32:01.686 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.686 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.686 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.687 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.687 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.687 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.687 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:01.687 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:01.687 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:01.687 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:01.687 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:01.688 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:01.688 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:01.688 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:01.688 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.688 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.688 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.688 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.688 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.688 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster": { 2024-04-24T23:32:01.689 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster", 2024-04-24T23:32:01.689 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.689 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.689 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.689 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.689 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.689 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.689 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.689 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.690 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.690 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.690 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.690 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.690 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster_level": { 2024-04-24T23:32:01.690 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster_level", 2024-04-24T23:32:01.690 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.690 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.691 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.691 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "info", 2024-04-24T23:32:01.691 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.691 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.691 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:01.691 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:01.691 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:01.691 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:01.691 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:01.692 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:01.692 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:01.692 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:01.692 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.692 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.692 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.692 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.692 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.692 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_file": { 2024-04-24T23:32:01.693 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_file", 2024-04-24T23:32:01.693 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.693 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.693 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.693 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.693 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.693 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.693 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.693 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.694 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.694 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.694 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.694 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.694 INFO:teuthology.orchestra.run.smithi080.stdout: "motd": { 2024-04-24T23:32:01.694 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "motd", 2024-04-24T23:32:01.694 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.694 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.695 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.695 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.695 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.695 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.695 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.695 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "The message of the day", 2024-04-24T23:32:01.695 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.695 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.695 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.696 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.696 INFO:teuthology.orchestra.run.smithi080.stdout: "redirect_resolve_ip_addr": { 2024-04-24T23:32:01.696 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "redirect_resolve_ip_addr", 2024-04-24T23:32:01.696 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.696 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.696 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.696 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.696 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.696 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.697 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.697 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.697 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.697 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.697 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.697 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.697 INFO:teuthology.orchestra.run.smithi080.stdout: "server_addr": { 2024-04-24T23:32:01.697 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "server_addr", 2024-04-24T23:32:01.698 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.698 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.698 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.698 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "::", 2024-04-24T23:32:01.698 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.698 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.698 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.698 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.698 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.699 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.699 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.699 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.699 INFO:teuthology.orchestra.run.smithi080.stdout: "server_port": { 2024-04-24T23:32:01.699 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "server_port", 2024-04-24T23:32:01.699 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "int", 2024-04-24T23:32:01.699 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.699 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.700 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "8080", 2024-04-24T23:32:01.700 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.700 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.700 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.700 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.700 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.700 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.700 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.700 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.701 INFO:teuthology.orchestra.run.smithi080.stdout: "ssl": { 2024-04-24T23:32:01.701 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "ssl", 2024-04-24T23:32:01.701 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.701 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.701 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.701 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "True", 2024-04-24T23:32:01.701 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.701 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.701 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.702 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.702 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.702 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.702 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.702 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.702 INFO:teuthology.orchestra.run.smithi080.stdout: "ssl_server_port": { 2024-04-24T23:32:01.702 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "ssl_server_port", 2024-04-24T23:32:01.702 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "int", 2024-04-24T23:32:01.702 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.703 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.703 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "8443", 2024-04-24T23:32:01.703 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.703 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.703 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.703 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.703 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.703 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.704 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.704 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.704 INFO:teuthology.orchestra.run.smithi080.stdout: "standby_behaviour": { 2024-04-24T23:32:01.704 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "standby_behaviour", 2024-04-24T23:32:01.704 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.704 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.704 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.704 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "redirect", 2024-04-24T23:32:01.704 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.705 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.705 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:01.705 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:01.705 INFO:teuthology.orchestra.run.smithi080.stdout: "redirect" 2024-04-24T23:32:01.705 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:01.705 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.705 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.705 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.706 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.706 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.706 INFO:teuthology.orchestra.run.smithi080.stdout: "standby_error_status_code": { 2024-04-24T23:32:01.706 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "standby_error_status_code", 2024-04-24T23:32:01.706 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "int", 2024-04-24T23:32:01.706 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.706 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.706 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "500", 2024-04-24T23:32:01.706 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "400", 2024-04-24T23:32:01.707 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "599", 2024-04-24T23:32:01.707 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.707 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.707 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.707 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.707 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.707 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.707 INFO:teuthology.orchestra.run.smithi080.stdout: "url_prefix": { 2024-04-24T23:32:01.708 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "url_prefix", 2024-04-24T23:32:01.708 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.708 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.708 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.708 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.708 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.708 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.708 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.708 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.709 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.709 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.709 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.709 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:01.709 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:01.709 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.709 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-24T23:32:01.709 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "devicehealth", 2024-04-24T23:32:01.709 INFO:teuthology.orchestra.run.smithi080.stdout: "can_run": true, 2024-04-24T23:32:01.710 INFO:teuthology.orchestra.run.smithi080.stdout: "error_string": "", 2024-04-24T23:32:01.710 INFO:teuthology.orchestra.run.smithi080.stdout: "module_options": { 2024-04-24T23:32:01.710 INFO:teuthology.orchestra.run.smithi080.stdout: "enable_monitoring": { 2024-04-24T23:32:01.710 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "enable_monitoring", 2024-04-24T23:32:01.710 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.710 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.710 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.710 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "True", 2024-04-24T23:32:01.711 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.711 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.711 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.711 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "monitor device health metrics", 2024-04-24T23:32:01.711 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.711 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.711 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.711 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.711 INFO:teuthology.orchestra.run.smithi080.stdout: "log_level": { 2024-04-24T23:32:01.712 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_level", 2024-04-24T23:32:01.712 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.712 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.712 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.712 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.712 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.712 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.712 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:01.712 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:01.713 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:01.713 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:01.713 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:01.713 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:01.713 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:01.713 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:01.713 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.713 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.714 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.714 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.714 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.714 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster": { 2024-04-24T23:32:01.714 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster", 2024-04-24T23:32:01.714 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.714 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.714 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.715 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.715 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.715 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.715 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.715 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.715 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.715 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.715 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.715 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.716 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster_level": { 2024-04-24T23:32:01.716 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster_level", 2024-04-24T23:32:01.716 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.716 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.716 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.716 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "info", 2024-04-24T23:32:01.716 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.716 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.716 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:01.717 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:01.717 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:01.717 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:01.717 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:01.717 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:01.717 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:01.717 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:01.717 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.718 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.718 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.718 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.718 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.718 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_file": { 2024-04-24T23:32:01.718 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_file", 2024-04-24T23:32:01.718 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.718 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.718 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.719 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.719 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.719 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.719 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.719 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.719 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.719 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.719 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.719 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.720 INFO:teuthology.orchestra.run.smithi080.stdout: "mark_out_threshold": { 2024-04-24T23:32:01.720 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "mark_out_threshold", 2024-04-24T23:32:01.720 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "secs", 2024-04-24T23:32:01.720 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.720 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.720 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "2419200", 2024-04-24T23:32:01.720 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.720 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.721 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.721 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-24T23:32:01.721 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.721 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.721 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.721 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.721 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_name": { 2024-04-24T23:32:01.721 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "pool_name", 2024-04-24T23:32:01.721 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.722 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.722 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.722 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "device_health_metrics", 2024-04-24T23:32:01.722 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.722 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.722 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.722 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-24T23:32:01.722 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.722 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.723 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.723 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.723 INFO:teuthology.orchestra.run.smithi080.stdout: "retention_period": { 2024-04-24T23:32:01.723 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "retention_period", 2024-04-24T23:32:01.723 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "secs", 2024-04-24T23:32:01.723 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.723 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.723 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "15552000", 2024-04-24T23:32:01.724 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.724 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.724 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.724 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "how long to retain device health metrics", 2024-04-24T23:32:01.724 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.724 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.724 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.724 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.724 INFO:teuthology.orchestra.run.smithi080.stdout: "scrape_frequency": { 2024-04-24T23:32:01.725 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "scrape_frequency", 2024-04-24T23:32:01.725 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "secs", 2024-04-24T23:32:01.725 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.725 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.725 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "86400", 2024-04-24T23:32:01.725 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.725 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.725 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.725 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-24T23:32:01.726 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.726 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.726 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.726 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.726 INFO:teuthology.orchestra.run.smithi080.stdout: "self_heal": { 2024-04-24T23:32:01.726 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "self_heal", 2024-04-24T23:32:01.726 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.726 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.727 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.727 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "True", 2024-04-24T23:32:01.727 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.727 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.727 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.727 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-24T23:32:01.727 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.727 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.728 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.728 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.728 INFO:teuthology.orchestra.run.smithi080.stdout: "sleep_interval": { 2024-04-24T23:32:01.728 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "sleep_interval", 2024-04-24T23:32:01.728 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "secs", 2024-04-24T23:32:01.728 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.728 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.728 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "600", 2024-04-24T23:32:01.728 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.729 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.729 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.729 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "how frequently to wake up and check device health", 2024-04-24T23:32:01.729 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.729 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.729 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.729 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.729 INFO:teuthology.orchestra.run.smithi080.stdout: "warn_threshold": { 2024-04-24T23:32:01.730 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "warn_threshold", 2024-04-24T23:32:01.730 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "secs", 2024-04-24T23:32:01.730 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.730 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.730 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "7257600", 2024-04-24T23:32:01.730 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.730 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.730 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.730 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-24T23:32:01.731 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.731 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.731 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.731 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:01.731 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:01.731 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.731 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-24T23:32:01.731 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "diskprediction_local", 2024-04-24T23:32:01.732 INFO:teuthology.orchestra.run.smithi080.stdout: "can_run": true, 2024-04-24T23:32:01.732 INFO:teuthology.orchestra.run.smithi080.stdout: "error_string": "", 2024-04-24T23:32:01.732 INFO:teuthology.orchestra.run.smithi080.stdout: "module_options": { 2024-04-24T23:32:01.732 INFO:teuthology.orchestra.run.smithi080.stdout: "log_level": { 2024-04-24T23:32:01.732 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_level", 2024-04-24T23:32:01.732 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.732 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.732 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.732 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.733 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.733 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.733 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:01.733 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:01.733 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:01.733 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:01.733 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:01.733 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:01.733 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:01.734 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:01.734 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.734 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.734 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.734 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.734 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.734 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster": { 2024-04-24T23:32:01.734 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster", 2024-04-24T23:32:01.734 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.735 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.735 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.735 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.735 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.735 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.735 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.735 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.735 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.736 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.736 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.736 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.736 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster_level": { 2024-04-24T23:32:01.736 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster_level", 2024-04-24T23:32:01.736 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.736 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.736 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.736 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "info", 2024-04-24T23:32:01.737 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.737 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.737 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:01.737 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:01.737 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:01.737 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:01.737 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:01.737 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:01.737 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:01.738 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:01.738 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.738 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.738 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.738 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.738 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.738 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_file": { 2024-04-24T23:32:01.738 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_file", 2024-04-24T23:32:01.739 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.739 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.739 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.739 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.739 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.739 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.739 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.739 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.739 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.740 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.740 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.740 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.740 INFO:teuthology.orchestra.run.smithi080.stdout: "predict_interval": { 2024-04-24T23:32:01.740 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "predict_interval", 2024-04-24T23:32:01.740 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.740 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.740 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.740 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "86400", 2024-04-24T23:32:01.741 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.741 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.741 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.741 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.741 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.741 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.741 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.741 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.741 INFO:teuthology.orchestra.run.smithi080.stdout: "predictor_model": { 2024-04-24T23:32:01.742 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "predictor_model", 2024-04-24T23:32:01.742 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.742 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.742 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.742 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "prophetstor", 2024-04-24T23:32:01.742 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.742 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.742 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.743 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.743 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.743 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.743 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.743 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.743 INFO:teuthology.orchestra.run.smithi080.stdout: "sleep_interval": { 2024-04-24T23:32:01.743 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "sleep_interval", 2024-04-24T23:32:01.743 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.743 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.744 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.744 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "600", 2024-04-24T23:32:01.744 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.744 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.744 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.744 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.744 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.744 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.744 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.745 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:01.745 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:01.745 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.745 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-24T23:32:01.745 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "influx", 2024-04-24T23:32:01.745 INFO:teuthology.orchestra.run.smithi080.stdout: "can_run": false, 2024-04-24T23:32:01.745 INFO:teuthology.orchestra.run.smithi080.stdout: "error_string": "influxdb python module not found", 2024-04-24T23:32:01.745 INFO:teuthology.orchestra.run.smithi080.stdout: "module_options": { 2024-04-24T23:32:01.746 INFO:teuthology.orchestra.run.smithi080.stdout: "batch_size": { 2024-04-24T23:32:01.746 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "batch_size", 2024-04-24T23:32:01.746 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "int", 2024-04-24T23:32:01.746 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.746 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.746 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "5000", 2024-04-24T23:32:01.746 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.746 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.746 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.747 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-24T23:32:01.747 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.747 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.747 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.747 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.747 INFO:teuthology.orchestra.run.smithi080.stdout: "database": { 2024-04-24T23:32:01.747 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "database", 2024-04-24T23:32:01.747 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.748 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.748 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.748 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "ceph", 2024-04-24T23:32:01.748 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.748 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.748 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.748 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-24T23:32:01.748 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.749 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.749 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.749 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.749 INFO:teuthology.orchestra.run.smithi080.stdout: "hostname": { 2024-04-24T23:32:01.749 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "hostname", 2024-04-24T23:32:01.749 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.749 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.749 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.749 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.750 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.750 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.750 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.750 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "InfluxDB server hostname", 2024-04-24T23:32:01.750 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.750 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.750 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.750 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.751 INFO:teuthology.orchestra.run.smithi080.stdout: "interval": { 2024-04-24T23:32:01.751 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "interval", 2024-04-24T23:32:01.751 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "secs", 2024-04-24T23:32:01.751 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.751 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.751 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "30", 2024-04-24T23:32:01.751 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "5", 2024-04-24T23:32:01.751 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.751 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.752 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-24T23:32:01.752 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.752 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.752 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.752 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.752 INFO:teuthology.orchestra.run.smithi080.stdout: "log_level": { 2024-04-24T23:32:01.752 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_level", 2024-04-24T23:32:01.752 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.752 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.753 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.753 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.753 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.753 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.753 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:01.753 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:01.753 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:01.753 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:01.753 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:01.754 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:01.754 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:01.754 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:01.754 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.754 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.754 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.754 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.754 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.755 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster": { 2024-04-24T23:32:01.755 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster", 2024-04-24T23:32:01.755 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.755 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.755 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.755 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.755 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.755 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.755 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.756 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.756 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.756 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.756 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.756 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.756 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster_level": { 2024-04-24T23:32:01.756 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster_level", 2024-04-24T23:32:01.756 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.756 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.757 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.757 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "info", 2024-04-24T23:32:01.757 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.757 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.757 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:01.757 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:01.757 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:01.757 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:01.758 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:01.758 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:01.758 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:01.758 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:01.758 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.758 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.758 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.758 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.758 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.759 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_file": { 2024-04-24T23:32:01.759 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_file", 2024-04-24T23:32:01.759 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.759 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.759 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.759 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.759 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.759 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.759 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.760 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.760 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.760 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.760 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.760 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.760 INFO:teuthology.orchestra.run.smithi080.stdout: "password": { 2024-04-24T23:32:01.760 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "password", 2024-04-24T23:32:01.760 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.761 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.761 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.761 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.761 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.761 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.761 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.761 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "password of InfluxDB server user", 2024-04-24T23:32:01.761 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.761 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.762 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.762 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.762 INFO:teuthology.orchestra.run.smithi080.stdout: "port": { 2024-04-24T23:32:01.762 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "port", 2024-04-24T23:32:01.762 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "int", 2024-04-24T23:32:01.762 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.762 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.762 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "8086", 2024-04-24T23:32:01.763 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.763 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.763 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.763 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "InfluxDB server port", 2024-04-24T23:32:01.763 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.763 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.763 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.763 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.763 INFO:teuthology.orchestra.run.smithi080.stdout: "ssl": { 2024-04-24T23:32:01.764 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "ssl", 2024-04-24T23:32:01.764 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.764 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.764 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.764 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "false", 2024-04-24T23:32:01.764 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.764 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.764 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.764 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-24T23:32:01.765 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.765 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.765 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.765 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.765 INFO:teuthology.orchestra.run.smithi080.stdout: "threads": { 2024-04-24T23:32:01.765 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "threads", 2024-04-24T23:32:01.765 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "int", 2024-04-24T23:32:01.765 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.765 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.766 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "5", 2024-04-24T23:32:01.766 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "1", 2024-04-24T23:32:01.766 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "32", 2024-04-24T23:32:01.766 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.766 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-24T23:32:01.766 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.766 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.766 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.767 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.767 INFO:teuthology.orchestra.run.smithi080.stdout: "username": { 2024-04-24T23:32:01.767 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "username", 2024-04-24T23:32:01.767 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.767 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.767 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.767 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.767 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.767 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.768 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.768 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "username of InfluxDB server user", 2024-04-24T23:32:01.768 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.768 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.768 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.768 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.768 INFO:teuthology.orchestra.run.smithi080.stdout: "verify_ssl": { 2024-04-24T23:32:01.768 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "verify_ssl", 2024-04-24T23:32:01.768 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.769 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.769 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.769 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "true", 2024-04-24T23:32:01.769 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.769 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.769 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.769 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-24T23:32:01.769 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.770 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.770 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.770 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:01.770 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:01.770 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.770 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-24T23:32:01.770 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "insights", 2024-04-24T23:32:01.770 INFO:teuthology.orchestra.run.smithi080.stdout: "can_run": true, 2024-04-24T23:32:01.771 INFO:teuthology.orchestra.run.smithi080.stdout: "error_string": "", 2024-04-24T23:32:01.771 INFO:teuthology.orchestra.run.smithi080.stdout: "module_options": { 2024-04-24T23:32:01.771 INFO:teuthology.orchestra.run.smithi080.stdout: "log_level": { 2024-04-24T23:32:01.771 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_level", 2024-04-24T23:32:01.771 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.771 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.771 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.771 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.771 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.773 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.773 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:01.773 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:01.773 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:01.773 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:01.774 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:01.774 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:01.774 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:01.774 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:01.774 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.774 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.774 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.774 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.774 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.775 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster": { 2024-04-24T23:32:01.775 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster", 2024-04-24T23:32:01.775 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.775 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.775 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.775 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.775 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.775 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.776 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.776 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.776 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.776 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.776 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.776 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.776 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster_level": { 2024-04-24T23:32:01.776 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster_level", 2024-04-24T23:32:01.776 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.777 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.777 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.777 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "info", 2024-04-24T23:32:01.777 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.777 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.777 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:01.777 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:01.777 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:01.777 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:01.778 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:01.778 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:01.778 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:01.778 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:01.778 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.778 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.778 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.778 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.779 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.779 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_file": { 2024-04-24T23:32:01.779 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_file", 2024-04-24T23:32:01.779 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.779 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.779 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.779 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.779 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.779 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.780 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.780 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.780 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.780 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.780 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.780 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:01.780 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:01.780 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.780 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-24T23:32:01.781 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "iostat", 2024-04-24T23:32:01.781 INFO:teuthology.orchestra.run.smithi080.stdout: "can_run": true, 2024-04-24T23:32:01.781 INFO:teuthology.orchestra.run.smithi080.stdout: "error_string": "", 2024-04-24T23:32:01.781 INFO:teuthology.orchestra.run.smithi080.stdout: "module_options": { 2024-04-24T23:32:01.781 INFO:teuthology.orchestra.run.smithi080.stdout: "log_level": { 2024-04-24T23:32:01.781 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_level", 2024-04-24T23:32:01.781 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.781 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.781 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.782 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.782 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.782 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.782 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:01.782 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:01.782 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:01.782 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:01.782 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:01.783 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:01.783 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:01.783 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:01.783 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.783 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.783 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.783 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.783 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.783 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster": { 2024-04-24T23:32:01.784 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster", 2024-04-24T23:32:01.784 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.784 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.784 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.784 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.784 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.784 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.784 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.784 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.785 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.785 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.785 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.785 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.785 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster_level": { 2024-04-24T23:32:01.785 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster_level", 2024-04-24T23:32:01.785 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.785 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.786 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.786 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "info", 2024-04-24T23:32:01.786 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.786 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.786 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:01.786 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:01.786 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:01.786 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:01.786 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:01.787 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:01.787 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:01.787 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:01.787 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.787 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.787 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.787 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.787 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.788 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_file": { 2024-04-24T23:32:01.788 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_file", 2024-04-24T23:32:01.788 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.788 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.788 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.788 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.788 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.788 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.788 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.789 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.789 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.789 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.789 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.789 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:01.789 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:01.789 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.789 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-24T23:32:01.789 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "k8sevents", 2024-04-24T23:32:01.790 INFO:teuthology.orchestra.run.smithi080.stdout: "can_run": true, 2024-04-24T23:32:01.790 INFO:teuthology.orchestra.run.smithi080.stdout: "error_string": "", 2024-04-24T23:32:01.790 INFO:teuthology.orchestra.run.smithi080.stdout: "module_options": { 2024-04-24T23:32:01.790 INFO:teuthology.orchestra.run.smithi080.stdout: "ceph_event_retention_days": { 2024-04-24T23:32:01.790 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "ceph_event_retention_days", 2024-04-24T23:32:01.790 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "int", 2024-04-24T23:32:01.790 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.790 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.790 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "7", 2024-04-24T23:32:01.791 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.791 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.791 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.791 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-24T23:32:01.791 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.791 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.791 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.791 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.792 INFO:teuthology.orchestra.run.smithi080.stdout: "config_check_secs": { 2024-04-24T23:32:01.792 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "config_check_secs", 2024-04-24T23:32:01.792 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "int", 2024-04-24T23:32:01.792 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.792 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.792 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "10", 2024-04-24T23:32:01.792 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "10", 2024-04-24T23:32:01.792 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.792 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.793 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-24T23:32:01.793 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.793 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.793 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.793 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.793 INFO:teuthology.orchestra.run.smithi080.stdout: "log_level": { 2024-04-24T23:32:01.793 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_level", 2024-04-24T23:32:01.793 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.793 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.794 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.794 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.794 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.794 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.794 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:01.794 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:01.794 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:01.795 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:01.795 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:01.795 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:01.795 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:01.795 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:01.795 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.795 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.795 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.795 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.796 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.796 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster": { 2024-04-24T23:32:01.796 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster", 2024-04-24T23:32:01.796 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.796 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.796 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.796 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.796 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.797 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.797 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.797 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.797 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.797 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.797 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.797 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.797 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster_level": { 2024-04-24T23:32:01.797 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster_level", 2024-04-24T23:32:01.798 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.798 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.798 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.798 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "info", 2024-04-24T23:32:01.798 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.798 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.798 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:01.798 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:01.798 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:01.799 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:01.799 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:01.799 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:01.799 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:01.799 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:01.799 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.799 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.799 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.800 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.800 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.800 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_file": { 2024-04-24T23:32:01.800 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_file", 2024-04-24T23:32:01.800 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.800 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.800 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.800 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.800 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.801 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.801 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.801 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.801 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.801 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.801 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.801 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:01.801 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:01.801 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.802 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-24T23:32:01.802 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "localpool", 2024-04-24T23:32:01.802 INFO:teuthology.orchestra.run.smithi080.stdout: "can_run": true, 2024-04-24T23:32:01.802 INFO:teuthology.orchestra.run.smithi080.stdout: "error_string": "", 2024-04-24T23:32:01.802 INFO:teuthology.orchestra.run.smithi080.stdout: "module_options": { 2024-04-24T23:32:01.802 INFO:teuthology.orchestra.run.smithi080.stdout: "failure_domain": { 2024-04-24T23:32:01.802 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "failure_domain", 2024-04-24T23:32:01.802 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.803 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.803 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.803 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "host", 2024-04-24T23:32:01.803 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.803 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.803 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.803 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "failure domain for any created local pool", 2024-04-24T23:32:01.803 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-24T23:32:01.803 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.804 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.804 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.804 INFO:teuthology.orchestra.run.smithi080.stdout: "log_level": { 2024-04-24T23:32:01.804 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_level", 2024-04-24T23:32:01.804 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.804 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.804 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.804 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.804 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.805 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.805 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:01.805 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:01.805 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:01.805 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:01.805 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:01.805 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:01.805 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:01.806 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:01.806 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.806 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.806 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.806 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.806 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.806 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster": { 2024-04-24T23:32:01.806 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster", 2024-04-24T23:32:01.806 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.807 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.807 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.807 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.807 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.807 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.807 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.807 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.807 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.807 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.808 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.808 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.808 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster_level": { 2024-04-24T23:32:01.808 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster_level", 2024-04-24T23:32:01.808 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.808 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.808 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.808 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "info", 2024-04-24T23:32:01.809 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.809 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.809 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:01.809 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:01.809 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:01.809 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:01.809 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:01.809 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:01.809 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:01.810 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:01.810 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.810 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.810 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.810 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.810 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.810 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_file": { 2024-04-24T23:32:01.810 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_file", 2024-04-24T23:32:01.810 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.811 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.811 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.811 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.811 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.811 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.811 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.811 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.811 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.812 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.812 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.812 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.812 INFO:teuthology.orchestra.run.smithi080.stdout: "min_size": { 2024-04-24T23:32:01.812 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "min_size", 2024-04-24T23:32:01.812 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "int", 2024-04-24T23:32:01.812 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.812 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.812 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.813 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.813 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.813 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.813 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "default min_size for any created local pool", 2024-04-24T23:32:01.813 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-24T23:32:01.813 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.813 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.813 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.813 INFO:teuthology.orchestra.run.smithi080.stdout: "num_rep": { 2024-04-24T23:32:01.814 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "num_rep", 2024-04-24T23:32:01.814 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "int", 2024-04-24T23:32:01.814 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.814 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.814 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "3", 2024-04-24T23:32:01.814 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.814 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.814 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.815 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "default replica count for any created local pool", 2024-04-24T23:32:01.815 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.815 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.815 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.815 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.815 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num": { 2024-04-24T23:32:01.815 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "pg_num", 2024-04-24T23:32:01.815 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "int", 2024-04-24T23:32:01.815 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.816 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.816 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "128", 2024-04-24T23:32:01.816 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.816 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.816 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.816 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "default pg_num for any created local pool", 2024-04-24T23:32:01.817 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.817 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.817 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.817 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.817 INFO:teuthology.orchestra.run.smithi080.stdout: "prefix": { 2024-04-24T23:32:01.817 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "prefix", 2024-04-24T23:32:01.817 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.817 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.817 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.818 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.818 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.818 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.818 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.818 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "name prefix for any created local pool", 2024-04-24T23:32:01.818 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.818 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.818 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.819 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.819 INFO:teuthology.orchestra.run.smithi080.stdout: "subtree": { 2024-04-24T23:32:01.819 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "subtree", 2024-04-24T23:32:01.819 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.819 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.819 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.819 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "rack", 2024-04-24T23:32:01.819 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.819 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.820 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.820 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-24T23:32:01.820 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-24T23:32:01.820 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.820 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.820 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:01.820 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:01.820 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.820 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-24T23:32:01.821 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "mds_autoscaler", 2024-04-24T23:32:01.821 INFO:teuthology.orchestra.run.smithi080.stdout: "can_run": true, 2024-04-24T23:32:01.821 INFO:teuthology.orchestra.run.smithi080.stdout: "error_string": "", 2024-04-24T23:32:01.821 INFO:teuthology.orchestra.run.smithi080.stdout: "module_options": { 2024-04-24T23:32:01.821 INFO:teuthology.orchestra.run.smithi080.stdout: "log_level": { 2024-04-24T23:32:01.821 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_level", 2024-04-24T23:32:01.821 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.821 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.822 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.822 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.822 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.822 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.822 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:01.822 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:01.822 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:01.822 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:01.822 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:01.823 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:01.823 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:01.823 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:01.823 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.823 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.823 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.823 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.823 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.824 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster": { 2024-04-24T23:32:01.824 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster", 2024-04-24T23:32:01.824 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.824 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.824 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.824 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.824 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.824 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.824 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.825 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.825 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.825 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.825 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.825 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.825 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster_level": { 2024-04-24T23:32:01.825 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster_level", 2024-04-24T23:32:01.825 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.825 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.826 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.826 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "info", 2024-04-24T23:32:01.826 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.826 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.826 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:01.826 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:01.826 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:01.826 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:01.827 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:01.827 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:01.827 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:01.827 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:01.827 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.827 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.827 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.827 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.827 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.828 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_file": { 2024-04-24T23:32:01.828 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_file", 2024-04-24T23:32:01.828 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.828 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.828 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.828 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.828 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.828 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.829 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.829 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.829 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.829 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.829 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.829 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:01.829 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:01.829 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.829 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-24T23:32:01.830 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "mirroring", 2024-04-24T23:32:01.830 INFO:teuthology.orchestra.run.smithi080.stdout: "can_run": true, 2024-04-24T23:32:01.830 INFO:teuthology.orchestra.run.smithi080.stdout: "error_string": "", 2024-04-24T23:32:01.830 INFO:teuthology.orchestra.run.smithi080.stdout: "module_options": { 2024-04-24T23:32:01.830 INFO:teuthology.orchestra.run.smithi080.stdout: "log_level": { 2024-04-24T23:32:01.830 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_level", 2024-04-24T23:32:01.830 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.830 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.830 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.831 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.831 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.831 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.831 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:01.831 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:01.831 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:01.831 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:01.831 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:01.832 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:01.832 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:01.832 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:01.832 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.832 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.832 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.832 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.832 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.832 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster": { 2024-04-24T23:32:01.833 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster", 2024-04-24T23:32:01.833 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.833 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.833 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.833 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.833 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.833 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.833 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.833 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.834 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.834 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.834 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.834 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.834 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster_level": { 2024-04-24T23:32:01.834 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster_level", 2024-04-24T23:32:01.834 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.834 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.835 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.835 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "info", 2024-04-24T23:32:01.835 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.835 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.835 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:01.835 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:01.835 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:01.835 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:01.835 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:01.836 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:01.836 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:01.836 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:01.836 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.836 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.836 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.836 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.836 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.836 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_file": { 2024-04-24T23:32:01.837 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_file", 2024-04-24T23:32:01.837 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.837 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.837 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.837 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.837 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.837 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.837 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.837 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.838 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.838 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.838 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.838 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:01.838 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:01.838 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.838 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-24T23:32:01.838 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "nfs", 2024-04-24T23:32:01.839 INFO:teuthology.orchestra.run.smithi080.stdout: "can_run": true, 2024-04-24T23:32:01.839 INFO:teuthology.orchestra.run.smithi080.stdout: "error_string": "", 2024-04-24T23:32:01.839 INFO:teuthology.orchestra.run.smithi080.stdout: "module_options": { 2024-04-24T23:32:01.839 INFO:teuthology.orchestra.run.smithi080.stdout: "log_level": { 2024-04-24T23:32:01.839 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_level", 2024-04-24T23:32:01.839 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.839 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.839 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.840 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.840 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.840 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.840 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:01.840 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:01.840 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:01.840 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:01.840 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:01.840 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:01.841 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:01.841 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:01.841 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.841 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.841 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.841 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.841 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.841 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster": { 2024-04-24T23:32:01.841 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster", 2024-04-24T23:32:01.842 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.842 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.842 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.842 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.842 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.842 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.842 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.842 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.843 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.843 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.843 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.843 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.843 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster_level": { 2024-04-24T23:32:01.843 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster_level", 2024-04-24T23:32:01.843 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.843 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.843 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.844 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "info", 2024-04-24T23:32:01.844 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.844 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.844 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:01.844 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:01.844 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:01.844 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:01.844 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:01.845 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:01.845 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:01.845 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:01.845 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.845 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.845 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.845 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.845 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.845 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_file": { 2024-04-24T23:32:01.846 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_file", 2024-04-24T23:32:01.846 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.846 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.846 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.846 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.846 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.846 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.846 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.846 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.847 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.847 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.847 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.847 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:01.847 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:01.847 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.847 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-24T23:32:01.847 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "orchestrator", 2024-04-24T23:32:01.848 INFO:teuthology.orchestra.run.smithi080.stdout: "can_run": true, 2024-04-24T23:32:01.848 INFO:teuthology.orchestra.run.smithi080.stdout: "error_string": "", 2024-04-24T23:32:01.848 INFO:teuthology.orchestra.run.smithi080.stdout: "module_options": { 2024-04-24T23:32:01.848 INFO:teuthology.orchestra.run.smithi080.stdout: "fail_fs": { 2024-04-24T23:32:01.848 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "fail_fs", 2024-04-24T23:32:01.848 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.848 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.848 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.848 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.849 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.849 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.849 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.849 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-24T23:32:01.849 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.849 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.849 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.849 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.849 INFO:teuthology.orchestra.run.smithi080.stdout: "log_level": { 2024-04-24T23:32:01.850 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_level", 2024-04-24T23:32:01.850 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.850 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.850 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.850 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.850 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.850 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.850 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:01.851 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:01.851 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:01.851 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:01.851 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:01.851 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:01.851 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:01.851 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:01.851 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.851 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.852 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.852 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.852 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.852 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster": { 2024-04-24T23:32:01.852 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster", 2024-04-24T23:32:01.852 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.852 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.852 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.852 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.853 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.853 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.853 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.853 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.853 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.853 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.853 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.853 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.853 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster_level": { 2024-04-24T23:32:01.854 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster_level", 2024-04-24T23:32:01.854 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.854 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.854 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.854 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "info", 2024-04-24T23:32:01.854 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.854 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.854 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:01.854 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:01.855 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:01.855 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:01.855 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:01.855 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:01.855 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:01.855 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:01.855 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.855 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.856 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.856 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.856 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.856 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_file": { 2024-04-24T23:32:01.856 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_file", 2024-04-24T23:32:01.856 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.856 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.856 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.856 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.857 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.857 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.857 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.857 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.857 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.857 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.857 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.857 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.857 INFO:teuthology.orchestra.run.smithi080.stdout: "orchestrator": { 2024-04-24T23:32:01.858 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "orchestrator", 2024-04-24T23:32:01.858 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.858 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.858 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.858 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.858 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.858 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.858 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:01.858 INFO:teuthology.orchestra.run.smithi080.stdout: "cephadm", 2024-04-24T23:32:01.859 INFO:teuthology.orchestra.run.smithi080.stdout: "rook", 2024-04-24T23:32:01.859 INFO:teuthology.orchestra.run.smithi080.stdout: "test_orchestrator" 2024-04-24T23:32:01.859 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:01.859 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Orchestrator backend", 2024-04-24T23:32:01.859 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.859 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.859 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.859 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:01.860 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:01.860 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.860 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-24T23:32:01.860 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "osd_perf_query", 2024-04-24T23:32:01.860 INFO:teuthology.orchestra.run.smithi080.stdout: "can_run": true, 2024-04-24T23:32:01.860 INFO:teuthology.orchestra.run.smithi080.stdout: "error_string": "", 2024-04-24T23:32:01.860 INFO:teuthology.orchestra.run.smithi080.stdout: "module_options": { 2024-04-24T23:32:01.860 INFO:teuthology.orchestra.run.smithi080.stdout: "log_level": { 2024-04-24T23:32:01.860 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_level", 2024-04-24T23:32:01.861 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.861 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.861 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.861 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.861 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.861 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.861 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:01.861 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:01.862 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:01.862 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:01.862 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:01.862 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:01.862 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:01.862 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:01.862 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.862 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.862 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.863 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.863 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.863 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster": { 2024-04-24T23:32:01.863 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster", 2024-04-24T23:32:01.863 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.863 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.863 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.863 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.864 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.864 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.864 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.864 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.864 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.864 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.864 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.864 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.864 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster_level": { 2024-04-24T23:32:01.865 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster_level", 2024-04-24T23:32:01.865 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.865 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.865 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.865 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "info", 2024-04-24T23:32:01.865 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.865 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.865 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:01.866 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:01.866 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:01.866 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:01.866 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:01.866 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:01.866 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:01.866 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:01.866 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.866 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.867 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.867 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.867 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.867 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_file": { 2024-04-24T23:32:01.867 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_file", 2024-04-24T23:32:01.867 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.867 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.867 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.867 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.868 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.868 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.868 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.868 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.868 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.868 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.868 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.868 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:01.869 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:01.869 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.869 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-24T23:32:01.869 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "osd_support", 2024-04-24T23:32:01.869 INFO:teuthology.orchestra.run.smithi080.stdout: "can_run": true, 2024-04-24T23:32:01.869 INFO:teuthology.orchestra.run.smithi080.stdout: "error_string": "", 2024-04-24T23:32:01.869 INFO:teuthology.orchestra.run.smithi080.stdout: "module_options": { 2024-04-24T23:32:01.869 INFO:teuthology.orchestra.run.smithi080.stdout: "log_level": { 2024-04-24T23:32:01.869 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_level", 2024-04-24T23:32:01.870 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.870 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.870 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.870 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.870 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.870 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.870 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:01.870 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:01.871 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:01.871 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:01.871 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:01.871 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:01.871 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:01.871 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:01.871 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.871 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.871 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.872 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.872 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.872 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster": { 2024-04-24T23:32:01.872 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster", 2024-04-24T23:32:01.872 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.872 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.872 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.872 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.873 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.873 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.873 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.873 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.873 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.873 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.873 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.873 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.873 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster_level": { 2024-04-24T23:32:01.874 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster_level", 2024-04-24T23:32:01.874 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.874 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.874 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.874 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "info", 2024-04-24T23:32:01.874 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.874 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.874 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:01.874 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:01.875 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:01.875 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:01.875 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:01.875 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:01.875 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:01.875 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:01.875 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.875 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.876 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.876 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.876 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.876 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_file": { 2024-04-24T23:32:01.876 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_file", 2024-04-24T23:32:01.876 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.876 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.876 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.876 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.877 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.877 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.877 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.877 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.877 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.877 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.877 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.877 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:01.878 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:01.878 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.878 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-24T23:32:01.878 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "pg_autoscaler", 2024-04-24T23:32:01.878 INFO:teuthology.orchestra.run.smithi080.stdout: "can_run": true, 2024-04-24T23:32:01.878 INFO:teuthology.orchestra.run.smithi080.stdout: "error_string": "", 2024-04-24T23:32:01.878 INFO:teuthology.orchestra.run.smithi080.stdout: "module_options": { 2024-04-24T23:32:01.878 INFO:teuthology.orchestra.run.smithi080.stdout: "log_level": { 2024-04-24T23:32:01.878 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_level", 2024-04-24T23:32:01.879 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.879 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.879 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.879 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.879 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.879 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.879 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:01.879 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:01.879 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:01.880 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:01.880 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:01.880 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:01.880 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:01.880 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:01.880 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.880 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.880 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.881 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.881 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.881 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster": { 2024-04-24T23:32:01.881 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster", 2024-04-24T23:32:01.881 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.881 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.881 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.881 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.881 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.882 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.882 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.882 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.882 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.882 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.882 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.882 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.882 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster_level": { 2024-04-24T23:32:01.882 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster_level", 2024-04-24T23:32:01.883 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.883 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.883 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.883 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "info", 2024-04-24T23:32:01.883 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.883 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.883 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:01.883 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:01.884 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:01.884 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:01.884 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:01.884 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:01.884 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:01.884 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:01.884 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.884 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.885 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.885 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.885 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.885 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_file": { 2024-04-24T23:32:01.885 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_file", 2024-04-24T23:32:01.885 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.885 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.885 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.885 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.886 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.886 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.886 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.886 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.886 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.886 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.886 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.886 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.886 INFO:teuthology.orchestra.run.smithi080.stdout: "sleep_interval": { 2024-04-24T23:32:01.887 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "sleep_interval", 2024-04-24T23:32:01.887 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "secs", 2024-04-24T23:32:01.887 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.887 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.887 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "60", 2024-04-24T23:32:01.887 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.887 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.887 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.888 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.888 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.888 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.888 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.888 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.888 INFO:teuthology.orchestra.run.smithi080.stdout: "threshold": { 2024-04-24T23:32:01.888 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "threshold", 2024-04-24T23:32:01.888 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "float", 2024-04-24T23:32:01.888 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.889 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.889 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "3.0", 2024-04-24T23:32:01.889 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "1.0", 2024-04-24T23:32:01.889 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.889 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.889 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "scaling threshold", 2024-04-24T23:32:01.889 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-24T23:32:01.889 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.889 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.890 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:01.890 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:01.890 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.890 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-24T23:32:01.890 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "progress", 2024-04-24T23:32:01.890 INFO:teuthology.orchestra.run.smithi080.stdout: "can_run": true, 2024-04-24T23:32:01.890 INFO:teuthology.orchestra.run.smithi080.stdout: "error_string": "", 2024-04-24T23:32:01.890 INFO:teuthology.orchestra.run.smithi080.stdout: "module_options": { 2024-04-24T23:32:01.891 INFO:teuthology.orchestra.run.smithi080.stdout: "allow_pg_recovery_event": { 2024-04-24T23:32:01.891 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "allow_pg_recovery_event", 2024-04-24T23:32:01.891 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.891 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.891 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.891 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.891 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.891 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.891 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.892 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-24T23:32:01.892 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.892 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.892 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.892 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.892 INFO:teuthology.orchestra.run.smithi080.stdout: "enabled": { 2024-04-24T23:32:01.892 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "enabled", 2024-04-24T23:32:01.892 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.892 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.893 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.893 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "True", 2024-04-24T23:32:01.893 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.893 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.893 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.893 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.893 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.893 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.894 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.894 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.894 INFO:teuthology.orchestra.run.smithi080.stdout: "log_level": { 2024-04-24T23:32:01.894 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_level", 2024-04-24T23:32:01.894 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.894 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.894 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.894 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.894 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.895 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.895 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:01.895 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:01.895 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:01.895 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:01.895 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:01.895 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:01.895 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:01.896 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:01.896 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.896 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.896 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.896 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.896 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.896 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster": { 2024-04-24T23:32:01.896 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster", 2024-04-24T23:32:01.896 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.897 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.897 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.897 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.897 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.897 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.897 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.897 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.897 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.898 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.898 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.898 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.898 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster_level": { 2024-04-24T23:32:01.898 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster_level", 2024-04-24T23:32:01.898 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.898 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.898 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.898 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "info", 2024-04-24T23:32:01.899 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.899 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.899 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:01.899 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:01.899 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:01.899 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:01.899 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:01.899 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:01.899 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:01.900 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:01.900 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.900 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.900 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.900 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.900 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.900 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_file": { 2024-04-24T23:32:01.900 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_file", 2024-04-24T23:32:01.901 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.901 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.901 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.901 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.901 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.901 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.901 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.901 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.901 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.902 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.902 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.902 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.902 INFO:teuthology.orchestra.run.smithi080.stdout: "max_completed_events": { 2024-04-24T23:32:01.902 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "max_completed_events", 2024-04-24T23:32:01.902 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "int", 2024-04-24T23:32:01.902 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.902 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.903 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "50", 2024-04-24T23:32:01.903 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.903 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.903 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.903 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "number of past completed events to remember", 2024-04-24T23:32:01.903 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.903 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.903 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.903 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.904 INFO:teuthology.orchestra.run.smithi080.stdout: "sleep_interval": { 2024-04-24T23:32:01.904 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "sleep_interval", 2024-04-24T23:32:01.904 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "secs", 2024-04-24T23:32:01.904 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.904 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.904 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "5", 2024-04-24T23:32:01.904 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.904 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.905 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.905 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "how long the module is going to sleep", 2024-04-24T23:32:01.905 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.905 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.905 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.905 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:01.905 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:01.905 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.905 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-24T23:32:01.906 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "prometheus", 2024-04-24T23:32:01.906 INFO:teuthology.orchestra.run.smithi080.stdout: "can_run": true, 2024-04-24T23:32:01.906 INFO:teuthology.orchestra.run.smithi080.stdout: "error_string": "", 2024-04-24T23:32:01.906 INFO:teuthology.orchestra.run.smithi080.stdout: "module_options": { 2024-04-24T23:32:01.906 INFO:teuthology.orchestra.run.smithi080.stdout: "cache": { 2024-04-24T23:32:01.906 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "cache", 2024-04-24T23:32:01.906 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.906 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.907 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.907 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "True", 2024-04-24T23:32:01.907 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.907 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.907 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.907 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.907 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.907 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.908 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.908 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.908 INFO:teuthology.orchestra.run.smithi080.stdout: "exclude_perf_counters": { 2024-04-24T23:32:01.908 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "exclude_perf_counters", 2024-04-24T23:32:01.908 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.908 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.908 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.908 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "True", 2024-04-24T23:32:01.908 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.909 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.909 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.909 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-24T23:32:01.909 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-24T23:32:01.909 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.909 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.909 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.909 INFO:teuthology.orchestra.run.smithi080.stdout: "log_level": { 2024-04-24T23:32:01.910 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_level", 2024-04-24T23:32:01.910 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.910 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.910 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.910 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.910 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.910 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.910 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:01.910 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:01.911 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:01.911 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:01.911 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:01.911 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:01.911 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:01.911 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:01.911 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.911 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.911 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.912 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.912 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.912 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster": { 2024-04-24T23:32:01.912 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster", 2024-04-24T23:32:01.912 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.912 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.912 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.912 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.912 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.913 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.913 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.914 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.914 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.914 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.914 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.914 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.914 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster_level": { 2024-04-24T23:32:01.914 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster_level", 2024-04-24T23:32:01.915 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.915 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.915 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.915 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "info", 2024-04-24T23:32:01.915 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.915 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.915 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:01.915 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:01.915 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:01.916 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:01.916 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:01.916 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:01.916 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:01.916 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:01.916 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.916 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.916 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.917 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.917 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.917 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_file": { 2024-04-24T23:32:01.917 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_file", 2024-04-24T23:32:01.917 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.917 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.917 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.917 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.917 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.918 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.918 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.918 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.918 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.918 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.918 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.918 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.918 INFO:teuthology.orchestra.run.smithi080.stdout: "rbd_stats_pools": { 2024-04-24T23:32:01.919 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "rbd_stats_pools", 2024-04-24T23:32:01.919 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.919 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.919 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.919 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.919 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.919 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.919 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.919 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.920 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.920 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.920 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.920 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.920 INFO:teuthology.orchestra.run.smithi080.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-24T23:32:01.920 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-24T23:32:01.920 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "int", 2024-04-24T23:32:01.920 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.920 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.921 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "300", 2024-04-24T23:32:01.921 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.921 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.921 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.921 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.921 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.921 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.921 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.921 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.922 INFO:teuthology.orchestra.run.smithi080.stdout: "scrape_interval": { 2024-04-24T23:32:01.922 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "scrape_interval", 2024-04-24T23:32:01.922 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "float", 2024-04-24T23:32:01.922 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.922 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.922 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "15.0", 2024-04-24T23:32:01.922 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.922 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.923 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.923 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.923 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.923 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.923 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.923 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.923 INFO:teuthology.orchestra.run.smithi080.stdout: "server_addr": { 2024-04-24T23:32:01.923 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "server_addr", 2024-04-24T23:32:01.924 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.924 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.924 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.924 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "::", 2024-04-24T23:32:01.924 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.924 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.924 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.924 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-24T23:32:01.924 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.925 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.925 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.925 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.925 INFO:teuthology.orchestra.run.smithi080.stdout: "server_port": { 2024-04-24T23:32:01.925 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "server_port", 2024-04-24T23:32:01.925 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "int", 2024-04-24T23:32:01.925 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.925 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.925 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "9283", 2024-04-24T23:32:01.926 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.926 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.926 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.926 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-24T23:32:01.926 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.926 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.926 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.926 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.927 INFO:teuthology.orchestra.run.smithi080.stdout: "stale_cache_strategy": { 2024-04-24T23:32:01.927 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "stale_cache_strategy", 2024-04-24T23:32:01.927 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.927 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.927 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.927 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "log", 2024-04-24T23:32:01.927 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.927 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.927 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.928 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.928 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.928 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.928 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.928 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.928 INFO:teuthology.orchestra.run.smithi080.stdout: "standby_behaviour": { 2024-04-24T23:32:01.928 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "standby_behaviour", 2024-04-24T23:32:01.928 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.929 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.929 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.929 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "default", 2024-04-24T23:32:01.929 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.929 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.929 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:01.929 INFO:teuthology.orchestra.run.smithi080.stdout: "default", 2024-04-24T23:32:01.929 INFO:teuthology.orchestra.run.smithi080.stdout: "error" 2024-04-24T23:32:01.929 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:01.930 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.930 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.930 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.930 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.930 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.930 INFO:teuthology.orchestra.run.smithi080.stdout: "standby_error_status_code": { 2024-04-24T23:32:01.930 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "standby_error_status_code", 2024-04-24T23:32:01.930 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "int", 2024-04-24T23:32:01.931 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.931 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.931 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "500", 2024-04-24T23:32:01.931 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "400", 2024-04-24T23:32:01.931 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "599", 2024-04-24T23:32:01.931 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.931 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.931 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.931 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.932 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.932 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:01.932 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:01.932 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.932 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-24T23:32:01.932 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "rbd_support", 2024-04-24T23:32:01.932 INFO:teuthology.orchestra.run.smithi080.stdout: "can_run": true, 2024-04-24T23:32:01.932 INFO:teuthology.orchestra.run.smithi080.stdout: "error_string": "", 2024-04-24T23:32:01.932 INFO:teuthology.orchestra.run.smithi080.stdout: "module_options": { 2024-04-24T23:32:01.933 INFO:teuthology.orchestra.run.smithi080.stdout: "log_level": { 2024-04-24T23:32:01.933 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_level", 2024-04-24T23:32:01.933 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.933 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.933 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.933 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.933 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.933 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.933 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:01.934 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:01.934 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:01.934 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:01.934 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:01.934 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:01.934 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:01.934 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:01.934 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.935 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.935 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.935 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.935 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.935 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster": { 2024-04-24T23:32:01.935 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster", 2024-04-24T23:32:01.935 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.935 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.935 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.936 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.936 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.936 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.936 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.936 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.936 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.936 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.936 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.936 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.937 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster_level": { 2024-04-24T23:32:01.937 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster_level", 2024-04-24T23:32:01.937 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.937 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.937 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.937 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "info", 2024-04-24T23:32:01.937 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.937 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.938 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:01.938 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:01.938 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:01.938 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:01.938 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:01.938 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:01.938 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:01.938 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:01.938 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.939 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.939 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.939 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.939 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.939 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_file": { 2024-04-24T23:32:01.939 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_file", 2024-04-24T23:32:01.939 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.939 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.940 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.940 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.940 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.940 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.940 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.940 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.940 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.940 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.940 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.941 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.941 INFO:teuthology.orchestra.run.smithi080.stdout: "max_concurrent_snap_create": { 2024-04-24T23:32:01.941 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "max_concurrent_snap_create", 2024-04-24T23:32:01.941 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "int", 2024-04-24T23:32:01.941 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.941 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.941 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "10", 2024-04-24T23:32:01.941 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.941 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.942 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.942 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.942 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.942 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.942 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.942 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.942 INFO:teuthology.orchestra.run.smithi080.stdout: "mirror_snapshot_schedule": { 2024-04-24T23:32:01.942 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "mirror_snapshot_schedule", 2024-04-24T23:32:01.943 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.943 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.943 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.943 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.943 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.943 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.943 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.943 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.943 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.944 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.944 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.944 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.944 INFO:teuthology.orchestra.run.smithi080.stdout: "trash_purge_schedule": { 2024-04-24T23:32:01.944 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "trash_purge_schedule", 2024-04-24T23:32:01.944 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.944 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.944 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.944 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.945 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.945 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.945 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.945 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.945 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.945 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.945 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.945 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:01.945 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:01.946 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.946 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-24T23:32:01.946 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "restful", 2024-04-24T23:32:01.946 INFO:teuthology.orchestra.run.smithi080.stdout: "can_run": true, 2024-04-24T23:32:01.946 INFO:teuthology.orchestra.run.smithi080.stdout: "error_string": "", 2024-04-24T23:32:01.946 INFO:teuthology.orchestra.run.smithi080.stdout: "module_options": { 2024-04-24T23:32:01.946 INFO:teuthology.orchestra.run.smithi080.stdout: "enable_auth": { 2024-04-24T23:32:01.946 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "enable_auth", 2024-04-24T23:32:01.947 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.947 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.947 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.947 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "True", 2024-04-24T23:32:01.947 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.947 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.947 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.947 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.947 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.948 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.948 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.948 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.948 INFO:teuthology.orchestra.run.smithi080.stdout: "key_file": { 2024-04-24T23:32:01.948 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "key_file", 2024-04-24T23:32:01.948 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.948 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.948 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.949 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.949 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.949 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.949 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.949 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.949 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.949 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.949 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.949 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.950 INFO:teuthology.orchestra.run.smithi080.stdout: "log_level": { 2024-04-24T23:32:01.950 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_level", 2024-04-24T23:32:01.950 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.950 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.950 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.950 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.950 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.950 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.951 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:01.951 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:01.951 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:01.951 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:01.951 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:01.951 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:01.951 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:01.951 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:01.952 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.952 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.952 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.952 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.952 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.952 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster": { 2024-04-24T23:32:01.952 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster", 2024-04-24T23:32:01.952 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.952 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.953 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.953 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.953 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.953 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.953 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.953 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.953 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.953 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.953 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.954 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.954 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster_level": { 2024-04-24T23:32:01.954 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster_level", 2024-04-24T23:32:01.954 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.954 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.954 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.954 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "info", 2024-04-24T23:32:01.954 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.955 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.955 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:01.955 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:01.955 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:01.955 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:01.955 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:01.955 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:01.955 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:01.955 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:01.956 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.956 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.956 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.956 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.956 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.956 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_file": { 2024-04-24T23:32:01.956 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_file", 2024-04-24T23:32:01.956 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.957 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.957 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.957 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.957 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.957 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.957 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.957 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.957 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.957 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.958 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.958 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.958 INFO:teuthology.orchestra.run.smithi080.stdout: "server_addr": { 2024-04-24T23:32:01.958 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "server_addr", 2024-04-24T23:32:01.958 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.958 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.958 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.958 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.958 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.959 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.959 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.959 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.959 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.959 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.959 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.959 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.959 INFO:teuthology.orchestra.run.smithi080.stdout: "server_port": { 2024-04-24T23:32:01.960 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "server_port", 2024-04-24T23:32:01.960 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.960 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.960 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.960 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.960 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.960 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.960 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.960 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.961 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.961 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.961 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.961 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:01.961 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:01.961 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.961 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-24T23:32:01.961 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "rgw", 2024-04-24T23:32:01.961 INFO:teuthology.orchestra.run.smithi080.stdout: "can_run": true, 2024-04-24T23:32:01.962 INFO:teuthology.orchestra.run.smithi080.stdout: "error_string": "", 2024-04-24T23:32:01.962 INFO:teuthology.orchestra.run.smithi080.stdout: "module_options": { 2024-04-24T23:32:01.962 INFO:teuthology.orchestra.run.smithi080.stdout: "log_level": { 2024-04-24T23:32:01.962 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_level", 2024-04-24T23:32:01.962 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.962 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.962 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.962 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.963 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.963 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.963 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:01.963 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:01.963 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:01.963 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:01.963 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:01.963 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:01.963 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:01.964 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:01.964 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.964 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.964 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.964 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.964 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.964 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster": { 2024-04-24T23:32:01.964 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster", 2024-04-24T23:32:01.964 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.965 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.965 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.965 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.965 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.965 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.965 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.965 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.965 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.965 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.966 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.966 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.966 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster_level": { 2024-04-24T23:32:01.966 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster_level", 2024-04-24T23:32:01.966 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.966 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.966 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.966 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "info", 2024-04-24T23:32:01.967 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.967 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.967 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:01.967 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:01.967 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:01.967 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:01.967 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:01.967 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:01.967 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:01.968 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:01.968 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.968 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.968 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.968 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.968 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.968 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_file": { 2024-04-24T23:32:01.968 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_file", 2024-04-24T23:32:01.969 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.969 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.969 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.969 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.969 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.969 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.969 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.969 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.969 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.970 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.970 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.970 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:01.970 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:01.970 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.970 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-24T23:32:01.970 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "rook", 2024-04-24T23:32:01.970 INFO:teuthology.orchestra.run.smithi080.stdout: "can_run": true, 2024-04-24T23:32:01.970 INFO:teuthology.orchestra.run.smithi080.stdout: "error_string": "", 2024-04-24T23:32:01.971 INFO:teuthology.orchestra.run.smithi080.stdout: "module_options": { 2024-04-24T23:32:01.971 INFO:teuthology.orchestra.run.smithi080.stdout: "log_level": { 2024-04-24T23:32:01.971 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_level", 2024-04-24T23:32:01.971 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.971 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.971 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.971 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.971 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.971 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.972 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:01.972 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:01.972 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:01.972 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:01.972 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:01.972 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:01.972 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:01.972 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:01.973 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.973 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.973 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.973 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.973 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.973 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster": { 2024-04-24T23:32:01.973 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster", 2024-04-24T23:32:01.973 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.973 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.974 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.974 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.974 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.974 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.974 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.974 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.974 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.974 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.975 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.975 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.975 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster_level": { 2024-04-24T23:32:01.975 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster_level", 2024-04-24T23:32:01.975 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.975 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.975 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.975 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "info", 2024-04-24T23:32:01.975 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.976 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.976 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:01.978 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:01.979 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:01.979 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:01.979 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:01.979 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:01.979 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:01.979 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:01.980 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.980 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.980 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.980 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.980 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.980 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_file": { 2024-04-24T23:32:01.980 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_file", 2024-04-24T23:32:01.980 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.981 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.981 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.981 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.981 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.981 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.981 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.981 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.981 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.981 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.982 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.982 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.982 INFO:teuthology.orchestra.run.smithi080.stdout: "storage_class": { 2024-04-24T23:32:01.982 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "storage_class", 2024-04-24T23:32:01.982 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.982 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.982 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.982 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "local", 2024-04-24T23:32:01.983 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.983 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.983 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.983 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-24T23:32:01.983 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.983 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.983 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.983 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:01.983 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:01.984 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.984 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-24T23:32:01.984 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "selftest", 2024-04-24T23:32:01.984 INFO:teuthology.orchestra.run.smithi080.stdout: "can_run": true, 2024-04-24T23:32:01.984 INFO:teuthology.orchestra.run.smithi080.stdout: "error_string": "", 2024-04-24T23:32:01.984 INFO:teuthology.orchestra.run.smithi080.stdout: "module_options": { 2024-04-24T23:32:01.984 INFO:teuthology.orchestra.run.smithi080.stdout: "log_level": { 2024-04-24T23:32:01.984 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_level", 2024-04-24T23:32:01.985 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.985 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.985 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.985 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.985 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.985 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.985 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:01.985 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:01.985 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:01.986 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:01.986 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:01.986 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:01.986 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:01.986 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:01.986 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.986 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.986 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.986 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.987 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.987 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster": { 2024-04-24T23:32:01.987 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster", 2024-04-24T23:32:01.987 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.987 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.987 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.987 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.987 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.988 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.988 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.988 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.988 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.988 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.988 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.988 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.988 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster_level": { 2024-04-24T23:32:01.988 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster_level", 2024-04-24T23:32:01.989 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.989 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.989 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.989 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "info", 2024-04-24T23:32:01.989 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.989 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.989 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:01.989 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:01.989 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:01.990 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:01.990 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:01.990 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:01.990 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:01.990 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:01.990 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.990 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.990 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.990 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.991 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.991 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_file": { 2024-04-24T23:32:01.991 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_file", 2024-04-24T23:32:01.991 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:01.991 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.991 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:01.991 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:01.991 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.992 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.992 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.992 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.992 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.992 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.992 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.992 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.992 INFO:teuthology.orchestra.run.smithi080.stdout: "roption1": { 2024-04-24T23:32:01.992 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "roption1", 2024-04-24T23:32:01.993 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.993 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.993 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.993 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.993 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.993 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.993 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.993 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.993 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.994 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.994 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.994 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.994 INFO:teuthology.orchestra.run.smithi080.stdout: "roption2": { 2024-04-24T23:32:01.994 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "roption2", 2024-04-24T23:32:01.994 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.994 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.994 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.994 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "xyz", 2024-04-24T23:32:01.995 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.995 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.995 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.995 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.995 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.995 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.995 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.995 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.995 INFO:teuthology.orchestra.run.smithi080.stdout: "rwoption1": { 2024-04-24T23:32:01.996 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "rwoption1", 2024-04-24T23:32:01.996 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:01.996 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.996 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.996 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.996 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.996 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.996 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.997 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.997 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.997 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.997 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.997 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.997 INFO:teuthology.orchestra.run.smithi080.stdout: "rwoption2": { 2024-04-24T23:32:01.997 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "rwoption2", 2024-04-24T23:32:01.997 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "int", 2024-04-24T23:32:01.997 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.998 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.998 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.998 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:01.998 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:01.998 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:01.998 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:01.998 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:01.998 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:01.998 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:01.999 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:01.999 INFO:teuthology.orchestra.run.smithi080.stdout: "rwoption3": { 2024-04-24T23:32:01.999 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "rwoption3", 2024-04-24T23:32:01.999 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "float", 2024-04-24T23:32:01.999 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:01.999 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:01.999 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:01.999 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.000 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.000 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.000 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.000 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.000 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.000 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.000 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.000 INFO:teuthology.orchestra.run.smithi080.stdout: "rwoption4": { 2024-04-24T23:32:02.000 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "rwoption4", 2024-04-24T23:32:02.001 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.001 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.001 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.001 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:02.001 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.001 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.001 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.001 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.002 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.002 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.002 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.002 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.002 INFO:teuthology.orchestra.run.smithi080.stdout: "rwoption5": { 2024-04-24T23:32:02.002 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "rwoption5", 2024-04-24T23:32:02.002 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.002 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.002 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.003 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:02.003 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.003 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.003 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.003 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.003 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.003 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.003 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.003 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.004 INFO:teuthology.orchestra.run.smithi080.stdout: "rwoption6": { 2024-04-24T23:32:02.004 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "rwoption6", 2024-04-24T23:32:02.004 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.004 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.004 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.004 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "True", 2024-04-24T23:32:02.004 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.004 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.005 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.005 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.005 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.005 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.005 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.005 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.005 INFO:teuthology.orchestra.run.smithi080.stdout: "rwoption7": { 2024-04-24T23:32:02.005 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "rwoption7", 2024-04-24T23:32:02.005 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "int", 2024-04-24T23:32:02.006 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.006 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.006 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:02.006 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "1", 2024-04-24T23:32:02.006 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "42", 2024-04-24T23:32:02.006 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.006 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.006 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.007 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.007 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.007 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.007 INFO:teuthology.orchestra.run.smithi080.stdout: "testkey": { 2024-04-24T23:32:02.007 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "testkey", 2024-04-24T23:32:02.007 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.007 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.007 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.007 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:02.008 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.008 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.008 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.008 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.008 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.008 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.008 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.008 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.008 INFO:teuthology.orchestra.run.smithi080.stdout: "testlkey": { 2024-04-24T23:32:02.009 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "testlkey", 2024-04-24T23:32:02.009 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.009 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.009 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.009 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:02.009 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.009 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.009 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.009 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.010 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.010 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.010 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.010 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.010 INFO:teuthology.orchestra.run.smithi080.stdout: "testnewline": { 2024-04-24T23:32:02.010 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "testnewline", 2024-04-24T23:32:02.010 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.010 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.011 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.011 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:02.011 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.011 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.011 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.011 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.011 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.011 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.011 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.012 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:02.012 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:02.012 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.012 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-24T23:32:02.012 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "snap_schedule", 2024-04-24T23:32:02.012 INFO:teuthology.orchestra.run.smithi080.stdout: "can_run": true, 2024-04-24T23:32:02.012 INFO:teuthology.orchestra.run.smithi080.stdout: "error_string": "", 2024-04-24T23:32:02.012 INFO:teuthology.orchestra.run.smithi080.stdout: "module_options": { 2024-04-24T23:32:02.012 INFO:teuthology.orchestra.run.smithi080.stdout: "allow_m_granularity": { 2024-04-24T23:32:02.013 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "allow_m_granularity", 2024-04-24T23:32:02.013 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.013 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.013 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.013 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:02.013 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.013 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.013 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.014 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "allow minute scheduled snapshots", 2024-04-24T23:32:02.014 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.014 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.014 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.014 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.014 INFO:teuthology.orchestra.run.smithi080.stdout: "dump_on_update": { 2024-04-24T23:32:02.014 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "dump_on_update", 2024-04-24T23:32:02.014 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.014 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.015 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.015 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:02.015 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.015 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.015 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.015 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "dump database to debug log on update", 2024-04-24T23:32:02.015 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.015 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.015 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.016 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.016 INFO:teuthology.orchestra.run.smithi080.stdout: "log_level": { 2024-04-24T23:32:02.016 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_level", 2024-04-24T23:32:02.016 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.016 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.016 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.016 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:02.016 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.016 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.017 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:02.017 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:02.017 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:02.017 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:02.017 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:02.017 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:02.017 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:02.017 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:02.018 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.018 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.018 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.018 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.018 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.018 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster": { 2024-04-24T23:32:02.018 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster", 2024-04-24T23:32:02.018 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.019 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.019 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.019 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:02.019 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.019 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.019 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.019 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.019 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.019 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.020 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.020 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.020 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster_level": { 2024-04-24T23:32:02.020 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster_level", 2024-04-24T23:32:02.020 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.020 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.020 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.020 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "info", 2024-04-24T23:32:02.020 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.021 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.021 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:02.021 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:02.021 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:02.021 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:02.021 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:02.021 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:02.021 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:02.022 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:02.022 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.022 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.022 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.022 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.022 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.022 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_file": { 2024-04-24T23:32:02.022 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_file", 2024-04-24T23:32:02.022 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.023 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.023 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.023 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:02.023 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.023 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.023 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.023 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.023 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.024 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.024 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.024 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:02.024 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:02.024 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.024 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-24T23:32:02.024 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "stats", 2024-04-24T23:32:02.024 INFO:teuthology.orchestra.run.smithi080.stdout: "can_run": true, 2024-04-24T23:32:02.024 INFO:teuthology.orchestra.run.smithi080.stdout: "error_string": "", 2024-04-24T23:32:02.025 INFO:teuthology.orchestra.run.smithi080.stdout: "module_options": { 2024-04-24T23:32:02.025 INFO:teuthology.orchestra.run.smithi080.stdout: "log_level": { 2024-04-24T23:32:02.025 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_level", 2024-04-24T23:32:02.025 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.025 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.025 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.025 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:02.025 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.025 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.026 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:02.026 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:02.026 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:02.026 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:02.026 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:02.026 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:02.026 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:02.026 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:02.027 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.027 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.027 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.027 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.027 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.027 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster": { 2024-04-24T23:32:02.027 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster", 2024-04-24T23:32:02.027 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.027 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.028 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.028 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:02.028 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.028 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.028 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.028 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.028 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.028 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.028 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.029 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.029 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster_level": { 2024-04-24T23:32:02.029 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster_level", 2024-04-24T23:32:02.029 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.029 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.029 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.029 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "info", 2024-04-24T23:32:02.029 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.030 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.030 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:02.030 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:02.030 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:02.030 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:02.030 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:02.030 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:02.030 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:02.030 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:02.031 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.031 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.031 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.031 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.031 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.031 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_file": { 2024-04-24T23:32:02.031 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_file", 2024-04-24T23:32:02.031 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.031 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.032 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.032 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:02.032 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.032 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.032 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.032 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.032 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.032 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.033 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.033 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:02.033 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:02.033 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.033 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-24T23:32:02.033 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "status", 2024-04-24T23:32:02.033 INFO:teuthology.orchestra.run.smithi080.stdout: "can_run": true, 2024-04-24T23:32:02.033 INFO:teuthology.orchestra.run.smithi080.stdout: "error_string": "", 2024-04-24T23:32:02.033 INFO:teuthology.orchestra.run.smithi080.stdout: "module_options": { 2024-04-24T23:32:02.034 INFO:teuthology.orchestra.run.smithi080.stdout: "log_level": { 2024-04-24T23:32:02.034 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_level", 2024-04-24T23:32:02.034 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.034 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.034 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.034 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:02.034 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.034 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.034 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:02.035 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:02.035 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:02.035 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:02.035 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:02.035 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:02.035 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:02.035 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:02.035 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.036 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.036 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.036 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.036 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.036 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster": { 2024-04-24T23:32:02.036 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster", 2024-04-24T23:32:02.036 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.036 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.036 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.037 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:02.037 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.037 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.037 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.037 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.037 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.037 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.037 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.037 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.038 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster_level": { 2024-04-24T23:32:02.038 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster_level", 2024-04-24T23:32:02.038 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.038 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.038 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.038 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "info", 2024-04-24T23:32:02.038 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.038 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.038 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:02.039 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:02.039 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:02.039 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:02.039 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:02.039 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:02.039 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:02.039 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:02.039 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.040 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.040 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.040 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.040 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.040 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_file": { 2024-04-24T23:32:02.040 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_file", 2024-04-24T23:32:02.040 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.040 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.040 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.041 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:02.041 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.041 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.041 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.041 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.041 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.041 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.041 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.041 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:02.042 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:02.042 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.042 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-24T23:32:02.042 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "telegraf", 2024-04-24T23:32:02.042 INFO:teuthology.orchestra.run.smithi080.stdout: "can_run": true, 2024-04-24T23:32:02.042 INFO:teuthology.orchestra.run.smithi080.stdout: "error_string": "", 2024-04-24T23:32:02.042 INFO:teuthology.orchestra.run.smithi080.stdout: "module_options": { 2024-04-24T23:32:02.042 INFO:teuthology.orchestra.run.smithi080.stdout: "address": { 2024-04-24T23:32:02.042 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "address", 2024-04-24T23:32:02.043 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.043 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.043 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.043 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-24T23:32:02.043 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.043 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.043 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.043 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.044 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.044 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.044 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.044 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.044 INFO:teuthology.orchestra.run.smithi080.stdout: "interval": { 2024-04-24T23:32:02.044 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "interval", 2024-04-24T23:32:02.044 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "secs", 2024-04-24T23:32:02.044 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.044 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.045 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "15", 2024-04-24T23:32:02.045 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.045 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.045 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.045 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.045 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.045 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.045 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.046 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.046 INFO:teuthology.orchestra.run.smithi080.stdout: "log_level": { 2024-04-24T23:32:02.046 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_level", 2024-04-24T23:32:02.046 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.046 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.046 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.046 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:02.046 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.046 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.047 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:02.047 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:02.047 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:02.047 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:02.047 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:02.047 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:02.047 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:02.047 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:02.048 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.048 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.048 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.048 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.048 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.048 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster": { 2024-04-24T23:32:02.048 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster", 2024-04-24T23:32:02.048 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.048 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.049 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.049 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:02.049 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.049 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.049 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.049 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.049 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.049 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.049 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.050 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.050 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster_level": { 2024-04-24T23:32:02.050 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster_level", 2024-04-24T23:32:02.050 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.050 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.050 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.050 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "info", 2024-04-24T23:32:02.050 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.050 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.051 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:02.051 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:02.051 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:02.051 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:02.051 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:02.051 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:02.051 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:02.051 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:02.052 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.052 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.052 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.052 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.052 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.052 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_file": { 2024-04-24T23:32:02.052 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_file", 2024-04-24T23:32:02.052 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.052 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.053 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.053 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:02.053 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.053 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.053 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.053 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.053 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.053 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.053 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.054 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:02.054 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:02.054 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.054 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-24T23:32:02.054 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "telemetry", 2024-04-24T23:32:02.054 INFO:teuthology.orchestra.run.smithi080.stdout: "can_run": true, 2024-04-24T23:32:02.054 INFO:teuthology.orchestra.run.smithi080.stdout: "error_string": "", 2024-04-24T23:32:02.054 INFO:teuthology.orchestra.run.smithi080.stdout: "module_options": { 2024-04-24T23:32:02.055 INFO:teuthology.orchestra.run.smithi080.stdout: "channel_basic": { 2024-04-24T23:32:02.055 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "channel_basic", 2024-04-24T23:32:02.055 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.055 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.055 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.055 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "True", 2024-04-24T23:32:02.055 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.055 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.055 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.056 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-24T23:32:02.056 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.056 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.056 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.056 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.056 INFO:teuthology.orchestra.run.smithi080.stdout: "channel_crash": { 2024-04-24T23:32:02.056 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "channel_crash", 2024-04-24T23:32:02.056 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.057 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.057 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.057 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "True", 2024-04-24T23:32:02.057 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.057 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.057 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.057 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-24T23:32:02.057 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.057 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.058 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.058 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.058 INFO:teuthology.orchestra.run.smithi080.stdout: "channel_device": { 2024-04-24T23:32:02.058 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "channel_device", 2024-04-24T23:32:02.058 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.058 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.058 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.058 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "True", 2024-04-24T23:32:02.058 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.059 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.059 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.059 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-24T23:32:02.059 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.059 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.059 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.059 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.059 INFO:teuthology.orchestra.run.smithi080.stdout: "channel_ident": { 2024-04-24T23:32:02.060 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "channel_ident", 2024-04-24T23:32:02.060 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.060 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.060 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.060 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:02.060 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.060 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.060 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.060 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-24T23:32:02.061 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.061 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.061 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.061 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.061 INFO:teuthology.orchestra.run.smithi080.stdout: "channel_perf": { 2024-04-24T23:32:02.061 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "channel_perf", 2024-04-24T23:32:02.061 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.061 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.061 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.062 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:02.062 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.062 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.062 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.062 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-24T23:32:02.062 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.062 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.062 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.063 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.063 INFO:teuthology.orchestra.run.smithi080.stdout: "contact": { 2024-04-24T23:32:02.063 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "contact", 2024-04-24T23:32:02.063 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.063 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.063 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.063 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:02.063 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.063 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.064 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.064 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.064 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.064 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.064 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.064 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.064 INFO:teuthology.orchestra.run.smithi080.stdout: "description": { 2024-04-24T23:32:02.064 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "description", 2024-04-24T23:32:02.064 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.065 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.065 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.065 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:02.065 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.065 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.065 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.065 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.065 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.065 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.066 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.066 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.066 INFO:teuthology.orchestra.run.smithi080.stdout: "device_url": { 2024-04-24T23:32:02.066 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "device_url", 2024-04-24T23:32:02.066 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.066 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.066 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.066 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-24T23:32:02.067 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.067 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.067 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.067 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.067 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.067 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.067 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.067 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.067 INFO:teuthology.orchestra.run.smithi080.stdout: "enabled": { 2024-04-24T23:32:02.068 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "enabled", 2024-04-24T23:32:02.068 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.068 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.068 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.068 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:02.068 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.068 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.068 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.069 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.069 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.069 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.069 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.069 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.069 INFO:teuthology.orchestra.run.smithi080.stdout: "interval": { 2024-04-24T23:32:02.069 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "interval", 2024-04-24T23:32:02.069 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "int", 2024-04-24T23:32:02.069 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.070 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.070 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "24", 2024-04-24T23:32:02.070 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "8", 2024-04-24T23:32:02.070 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.070 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.070 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.070 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.070 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.070 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.071 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.071 INFO:teuthology.orchestra.run.smithi080.stdout: "last_opt_revision": { 2024-04-24T23:32:02.071 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "last_opt_revision", 2024-04-24T23:32:02.071 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "int", 2024-04-24T23:32:02.071 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.071 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.071 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "1", 2024-04-24T23:32:02.071 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.072 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.072 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.072 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.072 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.072 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.072 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.072 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.072 INFO:teuthology.orchestra.run.smithi080.stdout: "leaderboard": { 2024-04-24T23:32:02.072 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "leaderboard", 2024-04-24T23:32:02.073 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.073 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.073 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.073 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:02.073 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.073 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.073 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.073 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.073 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.074 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.074 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.074 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.074 INFO:teuthology.orchestra.run.smithi080.stdout: "leaderboard_description": { 2024-04-24T23:32:02.074 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "leaderboard_description", 2024-04-24T23:32:02.074 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.074 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.074 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.075 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:02.075 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.075 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.075 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.075 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.075 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.075 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.075 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.075 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.076 INFO:teuthology.orchestra.run.smithi080.stdout: "log_level": { 2024-04-24T23:32:02.076 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_level", 2024-04-24T23:32:02.076 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.076 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.076 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.076 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:02.076 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.076 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.076 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:02.077 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:02.077 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:02.077 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:02.077 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:02.077 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:02.077 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:02.077 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:02.077 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.077 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.078 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.078 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.078 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.078 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster": { 2024-04-24T23:32:02.078 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster", 2024-04-24T23:32:02.078 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.078 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.078 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.079 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:02.079 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.079 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.079 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.079 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.079 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.079 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.079 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.079 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.080 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster_level": { 2024-04-24T23:32:02.080 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster_level", 2024-04-24T23:32:02.080 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.080 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.080 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.080 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "info", 2024-04-24T23:32:02.080 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.080 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.080 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:02.081 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:02.081 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:02.081 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:02.081 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:02.081 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:02.081 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:02.081 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:02.081 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.081 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.082 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.082 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.082 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.082 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_file": { 2024-04-24T23:32:02.082 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_file", 2024-04-24T23:32:02.082 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.082 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.082 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.083 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:02.083 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.083 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.083 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.083 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.083 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.083 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.083 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.083 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.084 INFO:teuthology.orchestra.run.smithi080.stdout: "organization": { 2024-04-24T23:32:02.084 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "organization", 2024-04-24T23:32:02.084 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.084 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.084 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.084 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:02.084 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.084 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.084 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.085 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.085 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.085 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.085 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.085 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.085 INFO:teuthology.orchestra.run.smithi080.stdout: "proxy": { 2024-04-24T23:32:02.085 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "proxy", 2024-04-24T23:32:02.085 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.085 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.086 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.086 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:02.086 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.086 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.086 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.086 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.086 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.086 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.087 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.087 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.087 INFO:teuthology.orchestra.run.smithi080.stdout: "url": { 2024-04-24T23:32:02.087 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "url", 2024-04-24T23:32:02.087 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.087 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.087 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.087 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-24T23:32:02.087 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.088 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.088 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.088 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.088 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.088 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.088 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.088 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:02.088 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:02.089 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.089 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-24T23:32:02.089 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "test_orchestrator", 2024-04-24T23:32:02.089 INFO:teuthology.orchestra.run.smithi080.stdout: "can_run": true, 2024-04-24T23:32:02.089 INFO:teuthology.orchestra.run.smithi080.stdout: "error_string": "", 2024-04-24T23:32:02.089 INFO:teuthology.orchestra.run.smithi080.stdout: "module_options": { 2024-04-24T23:32:02.089 INFO:teuthology.orchestra.run.smithi080.stdout: "log_level": { 2024-04-24T23:32:02.089 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_level", 2024-04-24T23:32:02.089 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.090 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.090 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.090 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:02.090 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.090 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.090 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:02.090 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:02.090 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:02.091 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:02.091 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:02.091 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:02.091 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:02.091 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:02.091 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.091 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.091 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.091 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.092 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.092 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster": { 2024-04-24T23:32:02.092 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster", 2024-04-24T23:32:02.092 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.092 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.092 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.092 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:02.092 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.092 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.093 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.093 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.093 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.093 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.093 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.093 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.093 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster_level": { 2024-04-24T23:32:02.093 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster_level", 2024-04-24T23:32:02.094 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.094 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.094 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.094 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "info", 2024-04-24T23:32:02.094 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.094 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.094 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:02.094 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:02.094 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:02.095 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:02.095 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:02.095 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:02.095 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:02.095 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:02.095 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.095 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.095 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.095 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.096 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.096 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_file": { 2024-04-24T23:32:02.096 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_file", 2024-04-24T23:32:02.096 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.096 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.096 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.096 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:02.096 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.096 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.097 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.097 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.097 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.097 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.097 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.097 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:02.097 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:02.097 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.098 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-24T23:32:02.098 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "volumes", 2024-04-24T23:32:02.098 INFO:teuthology.orchestra.run.smithi080.stdout: "can_run": true, 2024-04-24T23:32:02.098 INFO:teuthology.orchestra.run.smithi080.stdout: "error_string": "", 2024-04-24T23:32:02.098 INFO:teuthology.orchestra.run.smithi080.stdout: "module_options": { 2024-04-24T23:32:02.098 INFO:teuthology.orchestra.run.smithi080.stdout: "log_level": { 2024-04-24T23:32:02.098 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_level", 2024-04-24T23:32:02.098 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.098 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.099 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.099 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:02.099 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.099 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.099 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:02.099 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:02.099 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:02.099 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:02.099 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:02.100 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:02.100 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:02.100 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:02.100 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.100 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.100 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.100 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.100 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.100 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster": { 2024-04-24T23:32:02.101 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster", 2024-04-24T23:32:02.101 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.101 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.101 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.101 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:02.101 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.101 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.101 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.102 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.102 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.102 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.102 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.102 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.102 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster_level": { 2024-04-24T23:32:02.102 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster_level", 2024-04-24T23:32:02.102 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.102 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.103 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.103 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "info", 2024-04-24T23:32:02.103 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.103 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.103 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:02.103 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:02.103 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:02.103 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:02.103 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:02.104 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:02.104 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:02.104 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:02.104 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.104 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.104 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.104 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.104 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.104 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_file": { 2024-04-24T23:32:02.105 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_file", 2024-04-24T23:32:02.105 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.105 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.105 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.105 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:02.105 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.105 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.105 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.106 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.106 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.106 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.106 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.106 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.106 INFO:teuthology.orchestra.run.smithi080.stdout: "max_concurrent_clones": { 2024-04-24T23:32:02.106 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "max_concurrent_clones", 2024-04-24T23:32:02.106 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "int", 2024-04-24T23:32:02.106 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.107 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.107 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "4", 2024-04-24T23:32:02.107 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.107 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.107 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.107 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-24T23:32:02.107 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.107 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.107 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.108 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.108 INFO:teuthology.orchestra.run.smithi080.stdout: "periodic_async_work": { 2024-04-24T23:32:02.108 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "periodic_async_work", 2024-04-24T23:32:02.108 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.108 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.108 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.108 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:02.108 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.108 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.109 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.109 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Periodically check for async work", 2024-04-24T23:32:02.109 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.109 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.109 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.109 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.109 INFO:teuthology.orchestra.run.smithi080.stdout: "snapshot_clone_delay": { 2024-04-24T23:32:02.109 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "snapshot_clone_delay", 2024-04-24T23:32:02.110 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "int", 2024-04-24T23:32:02.110 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.110 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.110 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "0", 2024-04-24T23:32:02.110 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.110 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.110 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.110 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-24T23:32:02.111 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.111 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.111 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.111 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.111 INFO:teuthology.orchestra.run.smithi080.stdout: "snapshot_clone_no_wait": { 2024-04-24T23:32:02.111 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "snapshot_clone_no_wait", 2024-04-24T23:32:02.111 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.111 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.111 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.112 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "True", 2024-04-24T23:32:02.112 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.112 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.112 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.112 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-24T23:32:02.112 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.112 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.112 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.112 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:02.113 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:02.113 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.113 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-24T23:32:02.113 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "zabbix", 2024-04-24T23:32:02.113 INFO:teuthology.orchestra.run.smithi080.stdout: "can_run": true, 2024-04-24T23:32:02.113 INFO:teuthology.orchestra.run.smithi080.stdout: "error_string": "", 2024-04-24T23:32:02.113 INFO:teuthology.orchestra.run.smithi080.stdout: "module_options": { 2024-04-24T23:32:02.113 INFO:teuthology.orchestra.run.smithi080.stdout: "discovery_interval": { 2024-04-24T23:32:02.113 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "discovery_interval", 2024-04-24T23:32:02.114 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "uint", 2024-04-24T23:32:02.114 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.114 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.114 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "100", 2024-04-24T23:32:02.114 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.114 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.114 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.114 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.115 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.115 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.115 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.115 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.115 INFO:teuthology.orchestra.run.smithi080.stdout: "identifier": { 2024-04-24T23:32:02.115 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "identifier", 2024-04-24T23:32:02.115 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.115 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.115 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.116 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:02.116 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.116 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.116 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.116 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.116 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.116 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.116 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.117 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.117 INFO:teuthology.orchestra.run.smithi080.stdout: "interval": { 2024-04-24T23:32:02.117 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "interval", 2024-04-24T23:32:02.117 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "secs", 2024-04-24T23:32:02.117 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.117 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.117 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "60", 2024-04-24T23:32:02.117 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.117 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.118 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.118 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.118 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.118 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.118 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.118 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.118 INFO:teuthology.orchestra.run.smithi080.stdout: "log_level": { 2024-04-24T23:32:02.118 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_level", 2024-04-24T23:32:02.118 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.119 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.119 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.119 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:02.119 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.119 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.119 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:02.119 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:02.119 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:02.120 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:02.120 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:02.120 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:02.120 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:02.120 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:02.120 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.120 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.120 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.120 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.121 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.121 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster": { 2024-04-24T23:32:02.121 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster", 2024-04-24T23:32:02.121 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.121 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.121 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.121 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:02.121 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.121 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.122 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.122 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.122 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.122 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.122 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.122 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.122 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster_level": { 2024-04-24T23:32:02.122 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster_level", 2024-04-24T23:32:02.123 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.123 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.123 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.123 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "info", 2024-04-24T23:32:02.123 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.123 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.123 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:02.123 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:02.123 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:02.124 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:02.124 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:02.124 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:02.124 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:02.124 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:02.124 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.124 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.124 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.124 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.125 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.125 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_file": { 2024-04-24T23:32:02.125 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_file", 2024-04-24T23:32:02.125 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.125 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.125 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.125 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:02.125 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.126 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.126 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.126 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.126 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.126 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.126 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.126 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.126 INFO:teuthology.orchestra.run.smithi080.stdout: "zabbix_host": { 2024-04-24T23:32:02.126 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "zabbix_host", 2024-04-24T23:32:02.127 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.127 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.127 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.127 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:02.127 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.127 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.127 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.127 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.127 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.128 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.128 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.128 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.128 INFO:teuthology.orchestra.run.smithi080.stdout: "zabbix_port": { 2024-04-24T23:32:02.128 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "zabbix_port", 2024-04-24T23:32:02.128 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "int", 2024-04-24T23:32:02.128 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.128 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.128 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "10051", 2024-04-24T23:32:02.129 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.129 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.129 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.129 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.129 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.129 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.129 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.129 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.130 INFO:teuthology.orchestra.run.smithi080.stdout: "zabbix_sender": { 2024-04-24T23:32:02.130 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "zabbix_sender", 2024-04-24T23:32:02.130 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.130 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.130 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.130 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-24T23:32:02.130 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.130 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.130 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.131 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.131 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.131 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.131 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.131 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:02.131 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:02.131 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:02.131 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-24T23:32:02.132 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:02.132 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:02.132 INFO:teuthology.orchestra.run.smithi080.stdout: "modules": [ 2024-04-24T23:32:02.132 INFO:teuthology.orchestra.run.smithi080.stdout: "cephadm", 2024-04-24T23:32:02.132 INFO:teuthology.orchestra.run.smithi080.stdout: "dashboard", 2024-04-24T23:32:02.132 INFO:teuthology.orchestra.run.smithi080.stdout: "iostat", 2024-04-24T23:32:02.132 INFO:teuthology.orchestra.run.smithi080.stdout: "nfs", 2024-04-24T23:32:02.132 INFO:teuthology.orchestra.run.smithi080.stdout: "restful" 2024-04-24T23:32:02.132 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:02.133 INFO:teuthology.orchestra.run.smithi080.stdout: "available_modules": [ 2024-04-24T23:32:02.133 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-24T23:32:02.133 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "alerts", 2024-04-24T23:32:02.133 INFO:teuthology.orchestra.run.smithi080.stdout: "can_run": true, 2024-04-24T23:32:02.133 INFO:teuthology.orchestra.run.smithi080.stdout: "error_string": "", 2024-04-24T23:32:02.133 INFO:teuthology.orchestra.run.smithi080.stdout: "module_options": { 2024-04-24T23:32:02.133 INFO:teuthology.orchestra.run.smithi080.stdout: "interval": { 2024-04-24T23:32:02.133 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "interval", 2024-04-24T23:32:02.133 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "secs", 2024-04-24T23:32:02.134 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.134 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.134 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "60", 2024-04-24T23:32:02.134 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.134 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.134 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.134 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "How frequently to reexamine health status", 2024-04-24T23:32:02.134 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.135 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.135 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.135 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.135 INFO:teuthology.orchestra.run.smithi080.stdout: "log_level": { 2024-04-24T23:32:02.135 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_level", 2024-04-24T23:32:02.135 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.135 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.135 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.135 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:02.136 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.136 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.136 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:02.136 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:02.136 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:02.136 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:02.136 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:02.136 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:02.136 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:02.137 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:02.137 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.137 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.137 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.137 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.137 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.137 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster": { 2024-04-24T23:32:02.137 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster", 2024-04-24T23:32:02.137 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.138 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.138 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.138 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:02.138 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.138 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.138 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.138 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.138 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.139 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.139 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.139 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.139 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster_level": { 2024-04-24T23:32:02.139 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster_level", 2024-04-24T23:32:02.139 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.139 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.139 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.139 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "info", 2024-04-24T23:32:02.140 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.140 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.140 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:02.140 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:02.140 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:02.140 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:02.140 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:02.140 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:02.140 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:02.141 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:02.141 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.141 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.141 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.141 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.141 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.141 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_file": { 2024-04-24T23:32:02.141 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_file", 2024-04-24T23:32:02.141 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.142 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.142 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.142 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:02.142 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.142 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.142 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.142 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.142 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.143 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.143 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.143 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.143 INFO:teuthology.orchestra.run.smithi080.stdout: "smtp_destination": { 2024-04-24T23:32:02.143 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "smtp_destination", 2024-04-24T23:32:02.143 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.143 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.143 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.143 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:02.144 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.144 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.144 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.144 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Email address to send alerts to", 2024-04-24T23:32:02.144 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.144 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.144 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.144 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.144 INFO:teuthology.orchestra.run.smithi080.stdout: "smtp_from_name": { 2024-04-24T23:32:02.145 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "smtp_from_name", 2024-04-24T23:32:02.145 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.145 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.145 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.145 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "Ceph", 2024-04-24T23:32:02.145 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.145 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.145 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.145 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Email From: name", 2024-04-24T23:32:02.146 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.146 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.146 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.146 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.146 INFO:teuthology.orchestra.run.smithi080.stdout: "smtp_host": { 2024-04-24T23:32:02.146 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "smtp_host", 2024-04-24T23:32:02.146 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.146 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.146 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.147 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:02.147 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.147 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.147 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.147 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "SMTP server", 2024-04-24T23:32:02.147 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.147 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.147 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.148 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.148 INFO:teuthology.orchestra.run.smithi080.stdout: "smtp_password": { 2024-04-24T23:32:02.148 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "smtp_password", 2024-04-24T23:32:02.148 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.148 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.148 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.148 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:02.148 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.148 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.149 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.149 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Password to authenticate with", 2024-04-24T23:32:02.149 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.149 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.149 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.149 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.149 INFO:teuthology.orchestra.run.smithi080.stdout: "smtp_port": { 2024-04-24T23:32:02.149 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "smtp_port", 2024-04-24T23:32:02.149 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "int", 2024-04-24T23:32:02.150 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.150 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.150 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "465", 2024-04-24T23:32:02.150 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.150 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.150 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.150 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "SMTP port", 2024-04-24T23:32:02.150 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.150 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.151 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.151 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.151 INFO:teuthology.orchestra.run.smithi080.stdout: "smtp_sender": { 2024-04-24T23:32:02.151 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "smtp_sender", 2024-04-24T23:32:02.151 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.151 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.151 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.151 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:02.152 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.152 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.152 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.152 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "SMTP envelope sender", 2024-04-24T23:32:02.152 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.152 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.152 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.152 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.152 INFO:teuthology.orchestra.run.smithi080.stdout: "smtp_ssl": { 2024-04-24T23:32:02.153 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "smtp_ssl", 2024-04-24T23:32:02.153 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.153 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.153 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.153 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "True", 2024-04-24T23:32:02.153 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.153 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.153 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.153 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-24T23:32:02.154 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.154 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.154 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.154 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.154 INFO:teuthology.orchestra.run.smithi080.stdout: "smtp_user": { 2024-04-24T23:32:02.154 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "smtp_user", 2024-04-24T23:32:02.154 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.154 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.155 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.155 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:02.155 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.155 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.155 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.155 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "User to authenticate as", 2024-04-24T23:32:02.155 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.155 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.155 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.156 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:02.156 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:02.156 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.156 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-24T23:32:02.156 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "balancer", 2024-04-24T23:32:02.156 INFO:teuthology.orchestra.run.smithi080.stdout: "can_run": true, 2024-04-24T23:32:02.156 INFO:teuthology.orchestra.run.smithi080.stdout: "error_string": "", 2024-04-24T23:32:02.156 INFO:teuthology.orchestra.run.smithi080.stdout: "module_options": { 2024-04-24T23:32:02.156 INFO:teuthology.orchestra.run.smithi080.stdout: "active": { 2024-04-24T23:32:02.157 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "active", 2024-04-24T23:32:02.157 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.157 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.157 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.157 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "True", 2024-04-24T23:32:02.157 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.157 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.157 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.157 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "automatically balance PGs across cluster", 2024-04-24T23:32:02.158 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.158 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.158 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.158 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.158 INFO:teuthology.orchestra.run.smithi080.stdout: "begin_time": { 2024-04-24T23:32:02.158 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "begin_time", 2024-04-24T23:32:02.158 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.158 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.159 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.159 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "0000", 2024-04-24T23:32:02.159 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.159 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.159 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.159 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "beginning time of day to automatically balance", 2024-04-24T23:32:02.159 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-24T23:32:02.159 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.159 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.160 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.160 INFO:teuthology.orchestra.run.smithi080.stdout: "begin_weekday": { 2024-04-24T23:32:02.160 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "begin_weekday", 2024-04-24T23:32:02.160 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "uint", 2024-04-24T23:32:02.160 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.160 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.160 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "0", 2024-04-24T23:32:02.160 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "0", 2024-04-24T23:32:02.161 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "6", 2024-04-24T23:32:02.161 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.161 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-24T23:32:02.161 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-24T23:32:02.161 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.161 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.161 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.161 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_compat_max_iterations": { 2024-04-24T23:32:02.161 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "crush_compat_max_iterations", 2024-04-24T23:32:02.162 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "uint", 2024-04-24T23:32:02.162 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.162 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.162 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "25", 2024-04-24T23:32:02.162 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "1", 2024-04-24T23:32:02.162 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "250", 2024-04-24T23:32:02.162 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.162 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-24T23:32:02.162 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.163 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.163 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.163 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.163 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_compat_metrics": { 2024-04-24T23:32:02.163 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "crush_compat_metrics", 2024-04-24T23:32:02.163 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.163 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.163 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.164 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "pgs,objects,bytes", 2024-04-24T23:32:02.164 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.164 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.164 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.164 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-24T23:32:02.164 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-24T23:32:02.164 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.164 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.164 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.165 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_compat_step": { 2024-04-24T23:32:02.165 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "crush_compat_step", 2024-04-24T23:32:02.165 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "float", 2024-04-24T23:32:02.165 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.165 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.165 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "0.5", 2024-04-24T23:32:02.165 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "0.001", 2024-04-24T23:32:02.165 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "0.999", 2024-04-24T23:32:02.165 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.166 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "aggressiveness of optimization", 2024-04-24T23:32:02.166 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-24T23:32:02.166 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.166 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.166 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.166 INFO:teuthology.orchestra.run.smithi080.stdout: "end_time": { 2024-04-24T23:32:02.166 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "end_time", 2024-04-24T23:32:02.166 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.166 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.167 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.167 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "2359", 2024-04-24T23:32:02.167 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.167 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.167 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.167 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "ending time of day to automatically balance", 2024-04-24T23:32:02.167 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-24T23:32:02.167 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.168 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.168 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.168 INFO:teuthology.orchestra.run.smithi080.stdout: "end_weekday": { 2024-04-24T23:32:02.168 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "end_weekday", 2024-04-24T23:32:02.168 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "uint", 2024-04-24T23:32:02.168 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.168 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.168 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "0", 2024-04-24T23:32:02.168 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "0", 2024-04-24T23:32:02.169 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "6", 2024-04-24T23:32:02.169 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.169 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-24T23:32:02.169 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-24T23:32:02.169 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.169 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.169 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.169 INFO:teuthology.orchestra.run.smithi080.stdout: "log_level": { 2024-04-24T23:32:02.169 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_level", 2024-04-24T23:32:02.170 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.170 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.170 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.170 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:02.170 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.170 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.170 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:02.170 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:02.171 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:02.171 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:02.171 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:02.171 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:02.171 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:02.171 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:02.171 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.171 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.171 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.172 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.172 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.172 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster": { 2024-04-24T23:32:02.172 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster", 2024-04-24T23:32:02.172 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.172 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.172 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.172 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:02.172 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.173 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.173 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.173 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.173 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.173 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.173 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.173 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.173 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster_level": { 2024-04-24T23:32:02.173 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster_level", 2024-04-24T23:32:02.174 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.174 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.174 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.174 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "info", 2024-04-24T23:32:02.174 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.174 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.174 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:02.174 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:02.175 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:02.175 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:02.175 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:02.175 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:02.175 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:02.175 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:02.175 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.175 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.175 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.176 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.176 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.176 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_file": { 2024-04-24T23:32:02.176 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_file", 2024-04-24T23:32:02.176 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.176 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.176 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.176 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:02.176 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.177 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.177 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.177 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.177 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.177 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.177 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.177 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.177 INFO:teuthology.orchestra.run.smithi080.stdout: "min_score": { 2024-04-24T23:32:02.177 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "min_score", 2024-04-24T23:32:02.178 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "float", 2024-04-24T23:32:02.178 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.178 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.178 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "0", 2024-04-24T23:32:02.178 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.178 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.178 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.178 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-24T23:32:02.178 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.179 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.179 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.179 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.179 INFO:teuthology.orchestra.run.smithi080.stdout: "mode": { 2024-04-24T23:32:02.179 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "mode", 2024-04-24T23:32:02.179 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.179 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.179 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.179 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "upmap", 2024-04-24T23:32:02.180 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.180 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.180 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:02.182 INFO:teuthology.orchestra.run.smithi080.stdout: "crush-compat", 2024-04-24T23:32:02.182 INFO:teuthology.orchestra.run.smithi080.stdout: "none", 2024-04-24T23:32:02.182 INFO:teuthology.orchestra.run.smithi080.stdout: "read", 2024-04-24T23:32:02.182 INFO:teuthology.orchestra.run.smithi080.stdout: "upmap", 2024-04-24T23:32:02.182 INFO:teuthology.orchestra.run.smithi080.stdout: "upmap-read" 2024-04-24T23:32:02.182 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:02.182 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Balancer mode", 2024-04-24T23:32:02.183 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.183 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.183 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.183 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.183 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_ids": { 2024-04-24T23:32:02.183 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "pool_ids", 2024-04-24T23:32:02.183 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.183 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.183 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.184 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:02.184 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.184 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.184 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.184 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-24T23:32:02.184 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.184 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.184 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.184 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.185 INFO:teuthology.orchestra.run.smithi080.stdout: "sleep_interval": { 2024-04-24T23:32:02.185 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "sleep_interval", 2024-04-24T23:32:02.185 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "secs", 2024-04-24T23:32:02.185 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.185 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.185 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "60", 2024-04-24T23:32:02.185 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.185 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.185 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.186 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-24T23:32:02.186 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.186 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.186 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.186 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.186 INFO:teuthology.orchestra.run.smithi080.stdout: "upmap_max_deviation": { 2024-04-24T23:32:02.186 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "upmap_max_deviation", 2024-04-24T23:32:02.186 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "int", 2024-04-24T23:32:02.187 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.187 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.187 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "5", 2024-04-24T23:32:02.187 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "1", 2024-04-24T23:32:02.187 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.187 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.187 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-24T23:32:02.187 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-24T23:32:02.188 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.188 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.188 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.188 INFO:teuthology.orchestra.run.smithi080.stdout: "upmap_max_optimizations": { 2024-04-24T23:32:02.188 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "upmap_max_optimizations", 2024-04-24T23:32:02.188 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "uint", 2024-04-24T23:32:02.188 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.188 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.188 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "10", 2024-04-24T23:32:02.189 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.189 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.189 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.189 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-24T23:32:02.189 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.189 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.189 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.189 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:02.189 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:02.190 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.190 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-24T23:32:02.190 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "cephadm", 2024-04-24T23:32:02.190 INFO:teuthology.orchestra.run.smithi080.stdout: "can_run": true, 2024-04-24T23:32:02.190 INFO:teuthology.orchestra.run.smithi080.stdout: "error_string": "", 2024-04-24T23:32:02.190 INFO:teuthology.orchestra.run.smithi080.stdout: "module_options": { 2024-04-24T23:32:02.190 INFO:teuthology.orchestra.run.smithi080.stdout: "agent_down_multiplier": { 2024-04-24T23:32:02.190 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "agent_down_multiplier", 2024-04-24T23:32:02.190 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "float", 2024-04-24T23:32:02.191 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.191 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.191 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "3.0", 2024-04-24T23:32:02.191 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.191 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.191 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.191 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-24T23:32:02.191 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.191 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.192 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.192 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.192 INFO:teuthology.orchestra.run.smithi080.stdout: "agent_refresh_rate": { 2024-04-24T23:32:02.192 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "agent_refresh_rate", 2024-04-24T23:32:02.192 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "secs", 2024-04-24T23:32:02.192 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.192 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.192 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "20", 2024-04-24T23:32:02.193 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.193 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.193 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.193 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-24T23:32:02.193 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.193 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.193 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.193 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.193 INFO:teuthology.orchestra.run.smithi080.stdout: "agent_starting_port": { 2024-04-24T23:32:02.194 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "agent_starting_port", 2024-04-24T23:32:02.194 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "int", 2024-04-24T23:32:02.194 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.194 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.194 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "4721", 2024-04-24T23:32:02.194 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.194 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.194 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.194 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-24T23:32:02.195 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.195 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.195 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.195 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.195 INFO:teuthology.orchestra.run.smithi080.stdout: "allow_ptrace": { 2024-04-24T23:32:02.195 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "allow_ptrace", 2024-04-24T23:32:02.195 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.195 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.195 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.196 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:02.196 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.196 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.196 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.196 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-24T23:32:02.196 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-24T23:32:02.196 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.196 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.197 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.197 INFO:teuthology.orchestra.run.smithi080.stdout: "autotune_interval": { 2024-04-24T23:32:02.197 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "autotune_interval", 2024-04-24T23:32:02.197 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "secs", 2024-04-24T23:32:02.197 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.197 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.197 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "600", 2024-04-24T23:32:02.197 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.197 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.198 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.198 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-24T23:32:02.198 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.198 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.198 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.198 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.198 INFO:teuthology.orchestra.run.smithi080.stdout: "autotune_memory_target_ratio": { 2024-04-24T23:32:02.198 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "autotune_memory_target_ratio", 2024-04-24T23:32:02.198 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "float", 2024-04-24T23:32:02.199 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.199 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.199 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "0.7", 2024-04-24T23:32:02.199 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.199 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.199 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.199 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-24T23:32:02.199 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.200 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.200 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.200 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.200 INFO:teuthology.orchestra.run.smithi080.stdout: "cephadm_log_destination": { 2024-04-24T23:32:02.200 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "cephadm_log_destination", 2024-04-24T23:32:02.200 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.200 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.200 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.200 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:02.201 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.201 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.201 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:02.201 INFO:teuthology.orchestra.run.smithi080.stdout: "file", 2024-04-24T23:32:02.201 INFO:teuthology.orchestra.run.smithi080.stdout: "file,syslog", 2024-04-24T23:32:02.201 INFO:teuthology.orchestra.run.smithi080.stdout: "syslog" 2024-04-24T23:32:02.201 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:02.201 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-24T23:32:02.201 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.202 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.202 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.202 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.202 INFO:teuthology.orchestra.run.smithi080.stdout: "cgroups_split": { 2024-04-24T23:32:02.202 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "cgroups_split", 2024-04-24T23:32:02.202 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.202 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.202 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.202 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "True", 2024-04-24T23:32:02.203 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.203 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.203 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.203 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-24T23:32:02.203 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.203 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.203 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.203 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.204 INFO:teuthology.orchestra.run.smithi080.stdout: "config_checks_enabled": { 2024-04-24T23:32:02.204 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "config_checks_enabled", 2024-04-24T23:32:02.204 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.204 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.204 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.204 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:02.204 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.204 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.204 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.205 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-24T23:32:02.205 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.205 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.205 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.205 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.205 INFO:teuthology.orchestra.run.smithi080.stdout: "config_dashboard": { 2024-04-24T23:32:02.205 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "config_dashboard", 2024-04-24T23:32:02.205 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.205 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.206 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.206 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "True", 2024-04-24T23:32:02.206 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.206 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.206 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.206 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-24T23:32:02.206 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.206 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.206 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.207 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.207 INFO:teuthology.orchestra.run.smithi080.stdout: "container_image_alertmanager": { 2024-04-24T23:32:02.207 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "container_image_alertmanager", 2024-04-24T23:32:02.207 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.207 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.207 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.207 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-24T23:32:02.207 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.208 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.208 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.208 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Prometheus container image", 2024-04-24T23:32:02.208 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.208 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.208 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.208 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.208 INFO:teuthology.orchestra.run.smithi080.stdout: "container_image_base": { 2024-04-24T23:32:02.208 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "container_image_base", 2024-04-24T23:32:02.209 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.209 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.209 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.209 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-24T23:32:02.209 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.209 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.209 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.209 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Container image name, without the tag", 2024-04-24T23:32:02.210 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.210 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.210 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.210 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.210 INFO:teuthology.orchestra.run.smithi080.stdout: "container_image_elasticsearch": { 2024-04-24T23:32:02.210 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "container_image_elasticsearch", 2024-04-24T23:32:02.210 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.210 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.210 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.211 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-24T23:32:02.211 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.211 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.211 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.211 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "elasticsearch container image", 2024-04-24T23:32:02.211 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.211 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.211 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.211 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.212 INFO:teuthology.orchestra.run.smithi080.stdout: "container_image_grafana": { 2024-04-24T23:32:02.212 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "container_image_grafana", 2024-04-24T23:32:02.212 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.212 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.212 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.212 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-24T23:32:02.212 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.212 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.212 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.213 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Prometheus container image", 2024-04-24T23:32:02.213 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.213 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.213 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.213 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.213 INFO:teuthology.orchestra.run.smithi080.stdout: "container_image_haproxy": { 2024-04-24T23:32:02.213 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "container_image_haproxy", 2024-04-24T23:32:02.213 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.214 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.214 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.214 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-24T23:32:02.214 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.214 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.214 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.214 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "HAproxy container image", 2024-04-24T23:32:02.214 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.214 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.215 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.215 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.215 INFO:teuthology.orchestra.run.smithi080.stdout: "container_image_jaeger_agent": { 2024-04-24T23:32:02.215 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "container_image_jaeger_agent", 2024-04-24T23:32:02.215 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.215 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.215 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.215 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-24T23:32:02.215 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.216 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.216 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.216 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Jaeger agent container image", 2024-04-24T23:32:02.216 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.216 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.216 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.216 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.216 INFO:teuthology.orchestra.run.smithi080.stdout: "container_image_jaeger_collector": { 2024-04-24T23:32:02.217 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "container_image_jaeger_collector", 2024-04-24T23:32:02.217 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.217 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.217 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.217 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-24T23:32:02.217 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.217 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.217 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.217 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Jaeger collector container image", 2024-04-24T23:32:02.218 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.218 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.218 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.218 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.218 INFO:teuthology.orchestra.run.smithi080.stdout: "container_image_jaeger_query": { 2024-04-24T23:32:02.218 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "container_image_jaeger_query", 2024-04-24T23:32:02.218 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.218 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.218 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.219 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-24T23:32:02.219 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.219 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.219 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.219 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Jaeger query container image", 2024-04-24T23:32:02.219 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.219 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.219 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.219 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.220 INFO:teuthology.orchestra.run.smithi080.stdout: "container_image_keepalived": { 2024-04-24T23:32:02.220 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "container_image_keepalived", 2024-04-24T23:32:02.220 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.220 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.220 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.220 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-24T23:32:02.220 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.220 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.220 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.221 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Keepalived container image", 2024-04-24T23:32:02.221 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.221 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.221 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.221 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.221 INFO:teuthology.orchestra.run.smithi080.stdout: "container_image_loki": { 2024-04-24T23:32:02.221 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "container_image_loki", 2024-04-24T23:32:02.221 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.221 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.222 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.222 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-24T23:32:02.222 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.222 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.222 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.222 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Loki container image", 2024-04-24T23:32:02.222 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.222 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.223 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.223 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.223 INFO:teuthology.orchestra.run.smithi080.stdout: "container_image_node_exporter": { 2024-04-24T23:32:02.223 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "container_image_node_exporter", 2024-04-24T23:32:02.223 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.223 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.223 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.223 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-24T23:32:02.223 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.224 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.224 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.224 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Prometheus container image", 2024-04-24T23:32:02.224 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.224 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.224 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.224 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.224 INFO:teuthology.orchestra.run.smithi080.stdout: "container_image_nvmeof": { 2024-04-24T23:32:02.224 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "container_image_nvmeof", 2024-04-24T23:32:02.225 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.225 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.225 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.225 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-24T23:32:02.225 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.225 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.225 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.225 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Nvme-of container image", 2024-04-24T23:32:02.225 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.226 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.226 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.226 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.226 INFO:teuthology.orchestra.run.smithi080.stdout: "container_image_prometheus": { 2024-04-24T23:32:02.226 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "container_image_prometheus", 2024-04-24T23:32:02.226 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.226 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.226 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.227 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-24T23:32:02.227 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.227 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.227 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.227 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Prometheus container image", 2024-04-24T23:32:02.227 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.227 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.227 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.227 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.228 INFO:teuthology.orchestra.run.smithi080.stdout: "container_image_promtail": { 2024-04-24T23:32:02.228 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "container_image_promtail", 2024-04-24T23:32:02.228 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.228 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.228 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.228 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-24T23:32:02.228 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.228 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.228 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.229 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Promtail container image", 2024-04-24T23:32:02.229 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.229 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.229 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.229 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.229 INFO:teuthology.orchestra.run.smithi080.stdout: "container_image_samba": { 2024-04-24T23:32:02.229 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "container_image_samba", 2024-04-24T23:32:02.229 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.229 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.230 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.230 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-24T23:32:02.230 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.230 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.230 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.230 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Samba/SMB container image", 2024-04-24T23:32:02.230 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.230 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.231 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.231 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.231 INFO:teuthology.orchestra.run.smithi080.stdout: "container_image_snmp_gateway": { 2024-04-24T23:32:02.231 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "container_image_snmp_gateway", 2024-04-24T23:32:02.231 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.231 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.231 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.231 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-24T23:32:02.231 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.232 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.232 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.232 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "SNMP Gateway container image", 2024-04-24T23:32:02.232 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.232 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.232 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.232 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.232 INFO:teuthology.orchestra.run.smithi080.stdout: "container_init": { 2024-04-24T23:32:02.233 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "container_init", 2024-04-24T23:32:02.233 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.233 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.233 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.233 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "True", 2024-04-24T23:32:02.233 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.233 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.233 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.233 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Run podman/docker with `--init`", 2024-04-24T23:32:02.234 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.234 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.234 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.234 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.234 INFO:teuthology.orchestra.run.smithi080.stdout: "daemon_cache_timeout": { 2024-04-24T23:32:02.234 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "daemon_cache_timeout", 2024-04-24T23:32:02.234 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "secs", 2024-04-24T23:32:02.234 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.235 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.235 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "600", 2024-04-24T23:32:02.235 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.235 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.235 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.235 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-24T23:32:02.235 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.235 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.235 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.236 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.236 INFO:teuthology.orchestra.run.smithi080.stdout: "default_cephadm_command_timeout": { 2024-04-24T23:32:02.236 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "default_cephadm_command_timeout", 2024-04-24T23:32:02.236 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "int", 2024-04-24T23:32:02.236 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.236 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.236 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "900", 2024-04-24T23:32:02.236 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.236 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.237 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.237 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-24T23:32:02.237 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.237 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.237 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.237 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.237 INFO:teuthology.orchestra.run.smithi080.stdout: "default_registry": { 2024-04-24T23:32:02.237 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "default_registry", 2024-04-24T23:32:02.237 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.238 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.238 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.238 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "docker.io", 2024-04-24T23:32:02.238 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.238 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.238 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.238 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-24T23:32:02.238 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.239 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.239 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.239 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.239 INFO:teuthology.orchestra.run.smithi080.stdout: "device_cache_timeout": { 2024-04-24T23:32:02.239 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "device_cache_timeout", 2024-04-24T23:32:02.239 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "secs", 2024-04-24T23:32:02.239 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.239 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.239 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "1800", 2024-04-24T23:32:02.240 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.240 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.240 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.240 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "seconds to cache device inventory", 2024-04-24T23:32:02.240 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.240 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.240 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.240 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.241 INFO:teuthology.orchestra.run.smithi080.stdout: "device_enhanced_scan": { 2024-04-24T23:32:02.241 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "device_enhanced_scan", 2024-04-24T23:32:02.241 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.241 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.241 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.241 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:02.241 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.241 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.241 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.242 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-24T23:32:02.242 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.242 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.242 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.242 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.242 INFO:teuthology.orchestra.run.smithi080.stdout: "facts_cache_timeout": { 2024-04-24T23:32:02.242 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "facts_cache_timeout", 2024-04-24T23:32:02.242 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "secs", 2024-04-24T23:32:02.242 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.243 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.243 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "60", 2024-04-24T23:32:02.243 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.243 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.243 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.243 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "seconds to cache host facts data", 2024-04-24T23:32:02.243 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.243 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.244 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.244 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.244 INFO:teuthology.orchestra.run.smithi080.stdout: "grafana_dashboards_path": { 2024-04-24T23:32:02.244 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "grafana_dashboards_path", 2024-04-24T23:32:02.244 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.244 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.244 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.244 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-24T23:32:02.244 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.245 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.245 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.245 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-24T23:32:02.245 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.245 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.245 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.245 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.245 INFO:teuthology.orchestra.run.smithi080.stdout: "host_check_interval": { 2024-04-24T23:32:02.245 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "host_check_interval", 2024-04-24T23:32:02.246 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "secs", 2024-04-24T23:32:02.246 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.246 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.246 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "600", 2024-04-24T23:32:02.246 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.246 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.246 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.246 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "how frequently to perform a host check", 2024-04-24T23:32:02.247 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.247 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.247 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.247 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.247 INFO:teuthology.orchestra.run.smithi080.stdout: "hw_monitoring": { 2024-04-24T23:32:02.247 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "hw_monitoring", 2024-04-24T23:32:02.247 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.247 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.247 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.248 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:02.248 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.248 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.248 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.248 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-24T23:32:02.248 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.248 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.248 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.248 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.249 INFO:teuthology.orchestra.run.smithi080.stdout: "inventory_list_all": { 2024-04-24T23:32:02.249 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "inventory_list_all", 2024-04-24T23:32:02.249 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.249 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.249 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.249 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:02.249 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.249 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.250 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.250 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-24T23:32:02.250 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.250 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.250 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.250 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.250 INFO:teuthology.orchestra.run.smithi080.stdout: "log_level": { 2024-04-24T23:32:02.250 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_level", 2024-04-24T23:32:02.250 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.251 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.251 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.251 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:02.251 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.251 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.251 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:02.251 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:02.251 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:02.251 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:02.252 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:02.252 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:02.252 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:02.252 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:02.252 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.252 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.252 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.252 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.253 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.253 INFO:teuthology.orchestra.run.smithi080.stdout: "log_refresh_metadata": { 2024-04-24T23:32:02.253 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_refresh_metadata", 2024-04-24T23:32:02.253 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.253 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.253 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.253 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:02.253 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.253 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.254 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.254 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-24T23:32:02.254 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.254 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.254 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.254 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.254 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster": { 2024-04-24T23:32:02.254 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster", 2024-04-24T23:32:02.254 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.255 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.255 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.255 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "True", 2024-04-24T23:32:02.255 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.255 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.255 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.255 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-24T23:32:02.255 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.256 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.256 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.256 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.256 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster_level": { 2024-04-24T23:32:02.256 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster_level", 2024-04-24T23:32:02.256 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.256 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.256 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.256 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "info", 2024-04-24T23:32:02.257 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.257 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.257 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:02.257 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:02.257 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:02.257 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:02.257 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:02.257 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:02.258 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:02.258 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:02.258 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.258 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.258 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.258 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.258 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.258 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_file": { 2024-04-24T23:32:02.258 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_file", 2024-04-24T23:32:02.259 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.259 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.259 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.259 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:02.259 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.259 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.259 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.259 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.259 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.260 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.260 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.260 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.260 INFO:teuthology.orchestra.run.smithi080.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-24T23:32:02.260 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-24T23:32:02.260 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.260 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.260 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.261 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:02.261 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.261 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.261 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.261 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-24T23:32:02.261 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.261 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.261 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.261 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.262 INFO:teuthology.orchestra.run.smithi080.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-24T23:32:02.262 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-24T23:32:02.262 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.262 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.262 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.262 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "*", 2024-04-24T23:32:02.262 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.262 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.263 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.263 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-24T23:32:02.263 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.263 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.263 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.263 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.263 INFO:teuthology.orchestra.run.smithi080.stdout: "max_count_per_host": { 2024-04-24T23:32:02.263 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "max_count_per_host", 2024-04-24T23:32:02.263 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "int", 2024-04-24T23:32:02.264 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.264 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.264 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "10", 2024-04-24T23:32:02.264 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.264 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.264 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.264 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "max number of daemons per service per host", 2024-04-24T23:32:02.264 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.264 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.265 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.265 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.265 INFO:teuthology.orchestra.run.smithi080.stdout: "max_osd_draining_count": { 2024-04-24T23:32:02.265 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "max_osd_draining_count", 2024-04-24T23:32:02.265 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "int", 2024-04-24T23:32:02.265 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.265 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.265 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "10", 2024-04-24T23:32:02.265 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.266 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.266 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.266 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-24T23:32:02.266 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.266 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.266 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.266 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.266 INFO:teuthology.orchestra.run.smithi080.stdout: "migration_current": { 2024-04-24T23:32:02.267 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "migration_current", 2024-04-24T23:32:02.267 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "int", 2024-04-24T23:32:02.267 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.267 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.267 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:02.267 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.267 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.267 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.267 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "internal - do not modify", 2024-04-24T23:32:02.268 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.268 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.268 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.268 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.268 INFO:teuthology.orchestra.run.smithi080.stdout: "mode": { 2024-04-24T23:32:02.268 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "mode", 2024-04-24T23:32:02.268 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.268 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.268 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.269 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "root", 2024-04-24T23:32:02.269 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.269 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.269 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:02.269 INFO:teuthology.orchestra.run.smithi080.stdout: "cephadm-package", 2024-04-24T23:32:02.269 INFO:teuthology.orchestra.run.smithi080.stdout: "root" 2024-04-24T23:32:02.269 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:02.269 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "mode for remote execution of cephadm", 2024-04-24T23:32:02.269 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.270 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.270 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.270 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.270 INFO:teuthology.orchestra.run.smithi080.stdout: "oob_default_addr": { 2024-04-24T23:32:02.270 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "oob_default_addr", 2024-04-24T23:32:02.270 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.270 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.270 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.271 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "169.254.1.1", 2024-04-24T23:32:02.271 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.271 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.271 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.271 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-24T23:32:02.271 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.271 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.271 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.271 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.272 INFO:teuthology.orchestra.run.smithi080.stdout: "prometheus_alerts_path": { 2024-04-24T23:32:02.272 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "prometheus_alerts_path", 2024-04-24T23:32:02.272 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.272 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.272 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.272 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-24T23:32:02.272 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.272 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.272 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.273 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-24T23:32:02.273 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.273 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.273 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.273 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.273 INFO:teuthology.orchestra.run.smithi080.stdout: "registry_insecure": { 2024-04-24T23:32:02.273 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "registry_insecure", 2024-04-24T23:32:02.273 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.274 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.274 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.274 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:02.274 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.274 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.274 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.274 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-24T23:32:02.274 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.274 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.275 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.275 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.275 INFO:teuthology.orchestra.run.smithi080.stdout: "registry_password": { 2024-04-24T23:32:02.275 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "registry_password", 2024-04-24T23:32:02.275 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.275 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.275 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.275 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:02.275 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.276 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.276 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.276 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-24T23:32:02.276 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.276 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.276 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.276 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.276 INFO:teuthology.orchestra.run.smithi080.stdout: "registry_url": { 2024-04-24T23:32:02.277 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "registry_url", 2024-04-24T23:32:02.277 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.277 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.277 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.277 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:02.277 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.277 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.277 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.277 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-24T23:32:02.278 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.278 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.278 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.278 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.278 INFO:teuthology.orchestra.run.smithi080.stdout: "registry_username": { 2024-04-24T23:32:02.278 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "registry_username", 2024-04-24T23:32:02.278 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.278 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.278 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.279 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:02.279 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.279 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.279 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.279 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-24T23:32:02.279 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.279 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.279 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.280 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.280 INFO:teuthology.orchestra.run.smithi080.stdout: "secure_monitoring_stack": { 2024-04-24T23:32:02.280 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "secure_monitoring_stack", 2024-04-24T23:32:02.280 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.280 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.280 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.280 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:02.280 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.280 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.281 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.281 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-24T23:32:02.281 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.281 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.281 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.281 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.281 INFO:teuthology.orchestra.run.smithi080.stdout: "service_discovery_port": { 2024-04-24T23:32:02.281 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "service_discovery_port", 2024-04-24T23:32:02.282 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "int", 2024-04-24T23:32:02.282 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.282 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.282 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "8765", 2024-04-24T23:32:02.282 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.282 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.282 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.282 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "cephadm service discovery port", 2024-04-24T23:32:02.282 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.283 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.283 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.283 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.283 INFO:teuthology.orchestra.run.smithi080.stdout: "ssh_config_file": { 2024-04-24T23:32:02.283 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "ssh_config_file", 2024-04-24T23:32:02.283 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.283 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.283 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.284 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:02.284 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.284 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.284 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.284 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-24T23:32:02.284 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.284 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.284 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.284 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.285 INFO:teuthology.orchestra.run.smithi080.stdout: "use_agent": { 2024-04-24T23:32:02.285 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "use_agent", 2024-04-24T23:32:02.285 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.285 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.285 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.285 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:02.285 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.285 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.285 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.286 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-24T23:32:02.286 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.286 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.286 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.286 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.286 INFO:teuthology.orchestra.run.smithi080.stdout: "use_repo_digest": { 2024-04-24T23:32:02.286 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "use_repo_digest", 2024-04-24T23:32:02.286 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.287 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.287 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.287 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "True", 2024-04-24T23:32:02.287 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.287 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.287 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.287 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-24T23:32:02.287 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.287 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.288 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.288 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.288 INFO:teuthology.orchestra.run.smithi080.stdout: "warn_on_failed_host_check": { 2024-04-24T23:32:02.288 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "warn_on_failed_host_check", 2024-04-24T23:32:02.288 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.288 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.288 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.288 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "True", 2024-04-24T23:32:02.288 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.289 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.289 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.289 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "raise a health warning if the host check fails", 2024-04-24T23:32:02.289 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.289 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.289 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.289 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.289 INFO:teuthology.orchestra.run.smithi080.stdout: "warn_on_stray_daemons": { 2024-04-24T23:32:02.289 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "warn_on_stray_daemons", 2024-04-24T23:32:02.290 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.290 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.290 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.290 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "True", 2024-04-24T23:32:02.290 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.290 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.290 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.290 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-24T23:32:02.291 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.291 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.291 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.291 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.291 INFO:teuthology.orchestra.run.smithi080.stdout: "warn_on_stray_hosts": { 2024-04-24T23:32:02.291 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "warn_on_stray_hosts", 2024-04-24T23:32:02.291 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.291 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.291 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.292 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "True", 2024-04-24T23:32:02.292 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.292 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.292 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.292 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-24T23:32:02.292 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.292 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.292 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.292 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:02.293 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:02.293 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.293 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-24T23:32:02.293 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "crash", 2024-04-24T23:32:02.293 INFO:teuthology.orchestra.run.smithi080.stdout: "can_run": true, 2024-04-24T23:32:02.293 INFO:teuthology.orchestra.run.smithi080.stdout: "error_string": "", 2024-04-24T23:32:02.293 INFO:teuthology.orchestra.run.smithi080.stdout: "module_options": { 2024-04-24T23:32:02.293 INFO:teuthology.orchestra.run.smithi080.stdout: "log_level": { 2024-04-24T23:32:02.293 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_level", 2024-04-24T23:32:02.294 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.294 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.294 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.294 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:02.294 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.294 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.294 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:02.294 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:02.295 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:02.295 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:02.295 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:02.295 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:02.295 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:02.295 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:02.295 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.295 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.295 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.296 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.296 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.296 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster": { 2024-04-24T23:32:02.296 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster", 2024-04-24T23:32:02.296 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.296 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.296 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.296 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:02.296 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.297 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.297 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.297 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.297 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.297 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.297 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.297 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.297 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster_level": { 2024-04-24T23:32:02.298 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster_level", 2024-04-24T23:32:02.298 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.298 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.298 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.298 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "info", 2024-04-24T23:32:02.298 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.298 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.298 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:02.298 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:02.299 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:02.299 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:02.299 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:02.299 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:02.299 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:02.299 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:02.299 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.299 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.300 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.300 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.300 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.300 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_file": { 2024-04-24T23:32:02.300 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_file", 2024-04-24T23:32:02.300 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.300 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.300 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.300 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:02.301 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.301 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.301 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.301 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.301 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.301 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.301 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.301 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.301 INFO:teuthology.orchestra.run.smithi080.stdout: "retain_interval": { 2024-04-24T23:32:02.302 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "retain_interval", 2024-04-24T23:32:02.302 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "secs", 2024-04-24T23:32:02.302 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.302 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.302 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "31536000", 2024-04-24T23:32:02.302 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.302 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.302 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.302 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-24T23:32:02.303 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.303 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.303 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.303 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.303 INFO:teuthology.orchestra.run.smithi080.stdout: "warn_recent_interval": { 2024-04-24T23:32:02.303 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "warn_recent_interval", 2024-04-24T23:32:02.303 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "secs", 2024-04-24T23:32:02.303 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.304 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.304 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "1209600", 2024-04-24T23:32:02.304 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.304 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.304 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.304 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-24T23:32:02.304 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.304 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.304 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.305 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:02.305 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:02.305 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.305 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-24T23:32:02.305 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "dashboard", 2024-04-24T23:32:02.305 INFO:teuthology.orchestra.run.smithi080.stdout: "can_run": true, 2024-04-24T23:32:02.305 INFO:teuthology.orchestra.run.smithi080.stdout: "error_string": "", 2024-04-24T23:32:02.305 INFO:teuthology.orchestra.run.smithi080.stdout: "module_options": { 2024-04-24T23:32:02.305 INFO:teuthology.orchestra.run.smithi080.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-24T23:32:02.306 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-24T23:32:02.306 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "int", 2024-04-24T23:32:02.306 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.306 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.306 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "10", 2024-04-24T23:32:02.306 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.306 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.306 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.307 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.307 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.307 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.307 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.307 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.307 INFO:teuthology.orchestra.run.smithi080.stdout: "ALERTMANAGER_API_HOST": { 2024-04-24T23:32:02.307 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-24T23:32:02.307 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.308 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.308 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.308 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:02.308 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.308 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.308 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.308 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.308 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.308 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.309 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.309 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.309 INFO:teuthology.orchestra.run.smithi080.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-24T23:32:02.309 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-24T23:32:02.309 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.309 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.309 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.309 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "True", 2024-04-24T23:32:02.309 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.310 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.310 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.310 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.310 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.310 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.310 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.310 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.310 INFO:teuthology.orchestra.run.smithi080.stdout: "AUDIT_API_ENABLED": { 2024-04-24T23:32:02.311 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "AUDIT_API_ENABLED", 2024-04-24T23:32:02.311 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.311 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.311 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.311 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:02.311 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.311 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.311 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.311 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.312 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.312 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.312 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.312 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.312 INFO:teuthology.orchestra.run.smithi080.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-24T23:32:02.312 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-24T23:32:02.312 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.312 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.312 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.313 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "True", 2024-04-24T23:32:02.313 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.313 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.313 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.313 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.313 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.313 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.313 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.313 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.314 INFO:teuthology.orchestra.run.smithi080.stdout: "ENABLE_BROWSABLE_API": { 2024-04-24T23:32:02.314 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-24T23:32:02.314 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.314 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.314 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.314 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "True", 2024-04-24T23:32:02.314 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.314 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.315 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.315 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.315 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.315 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.315 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.315 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.315 INFO:teuthology.orchestra.run.smithi080.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-24T23:32:02.315 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-24T23:32:02.315 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.316 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.316 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.316 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "True", 2024-04-24T23:32:02.316 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.316 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.316 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.316 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.316 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.316 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.317 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.317 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.317 INFO:teuthology.orchestra.run.smithi080.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-24T23:32:02.317 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-24T23:32:02.317 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.317 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.317 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.317 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "True", 2024-04-24T23:32:02.318 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.318 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.318 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.318 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.318 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.318 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.318 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.318 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.318 INFO:teuthology.orchestra.run.smithi080.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-24T23:32:02.319 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-24T23:32:02.319 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.319 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.319 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.319 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "True", 2024-04-24T23:32:02.319 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.319 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.319 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.319 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.320 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.320 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.320 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.320 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.320 INFO:teuthology.orchestra.run.smithi080.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-24T23:32:02.320 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-24T23:32:02.320 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.320 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.321 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.321 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "True", 2024-04-24T23:32:02.321 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.321 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.321 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.321 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.321 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.321 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.321 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.322 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.322 INFO:teuthology.orchestra.run.smithi080.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-24T23:32:02.322 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-24T23:32:02.322 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.322 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.322 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.322 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "True", 2024-04-24T23:32:02.322 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.323 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.323 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.323 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.323 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.323 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.323 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.323 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.323 INFO:teuthology.orchestra.run.smithi080.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-24T23:32:02.324 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-24T23:32:02.324 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.324 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.324 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.324 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "True", 2024-04-24T23:32:02.324 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.324 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.324 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.324 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.325 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.325 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.325 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.325 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.325 INFO:teuthology.orchestra.run.smithi080.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-24T23:32:02.325 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-24T23:32:02.325 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.325 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.325 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.326 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "True", 2024-04-24T23:32:02.326 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.326 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.326 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.326 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.326 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.326 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.326 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.327 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.327 INFO:teuthology.orchestra.run.smithi080.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-24T23:32:02.327 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-24T23:32:02.327 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.327 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.327 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.327 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:02.327 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.327 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.328 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.328 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.328 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.328 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.328 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.328 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.328 INFO:teuthology.orchestra.run.smithi080.stdout: "GRAFANA_API_PASSWORD": { 2024-04-24T23:32:02.328 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-24T23:32:02.328 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.329 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.329 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.329 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "admin", 2024-04-24T23:32:02.329 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.329 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.329 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.329 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.329 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.329 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.330 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.330 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.330 INFO:teuthology.orchestra.run.smithi080.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-24T23:32:02.330 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-24T23:32:02.330 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.330 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.330 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.330 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "True", 2024-04-24T23:32:02.331 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.331 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.331 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.331 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.331 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.331 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.331 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.331 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.331 INFO:teuthology.orchestra.run.smithi080.stdout: "GRAFANA_API_URL": { 2024-04-24T23:32:02.332 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "GRAFANA_API_URL", 2024-04-24T23:32:02.332 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.332 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.332 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.332 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:02.332 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.332 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.332 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.333 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.333 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.333 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.333 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.333 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.333 INFO:teuthology.orchestra.run.smithi080.stdout: "GRAFANA_API_USERNAME": { 2024-04-24T23:32:02.333 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-24T23:32:02.333 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.333 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.334 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.334 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "admin", 2024-04-24T23:32:02.334 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.334 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.334 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.334 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.334 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.334 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.335 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.335 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.335 INFO:teuthology.orchestra.run.smithi080.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-24T23:32:02.335 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-24T23:32:02.335 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.335 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.335 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.335 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:02.335 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.336 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.336 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.336 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.336 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.336 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.336 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.336 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.336 INFO:teuthology.orchestra.run.smithi080.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-24T23:32:02.336 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-24T23:32:02.337 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.337 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.337 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.337 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:02.337 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.337 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.337 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.337 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.338 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.338 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.338 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.338 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.338 INFO:teuthology.orchestra.run.smithi080.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-24T23:32:02.338 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-24T23:32:02.338 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.338 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.338 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.339 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "True", 2024-04-24T23:32:02.339 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.339 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.339 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.339 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.339 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.339 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.339 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.339 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.340 INFO:teuthology.orchestra.run.smithi080.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-24T23:32:02.340 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-24T23:32:02.340 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.340 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.340 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.340 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:02.340 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.340 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.340 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.341 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.341 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.341 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.341 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.341 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.341 INFO:teuthology.orchestra.run.smithi080.stdout: "PROMETHEUS_API_HOST": { 2024-04-24T23:32:02.341 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-24T23:32:02.341 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.342 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.342 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.342 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:02.342 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.342 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.342 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.342 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.342 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.342 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.343 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.343 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.343 INFO:teuthology.orchestra.run.smithi080.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-24T23:32:02.343 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-24T23:32:02.343 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.343 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.343 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.343 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "True", 2024-04-24T23:32:02.343 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.344 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.344 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.344 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.344 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.344 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.344 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.344 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.344 INFO:teuthology.orchestra.run.smithi080.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-24T23:32:02.345 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-24T23:32:02.345 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.345 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.345 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.345 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:02.345 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.345 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.345 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.345 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.346 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.346 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.346 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.346 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.346 INFO:teuthology.orchestra.run.smithi080.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-24T23:32:02.346 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-24T23:32:02.346 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.346 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.346 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.347 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:02.347 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.347 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.347 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.347 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.347 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.347 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.347 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.347 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.348 INFO:teuthology.orchestra.run.smithi080.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-24T23:32:02.348 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-24T23:32:02.348 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.348 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.348 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.348 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "True", 2024-04-24T23:32:02.348 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.348 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.349 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.349 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.349 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.349 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.349 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.349 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.349 INFO:teuthology.orchestra.run.smithi080.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-24T23:32:02.349 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-24T23:32:02.349 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.350 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.350 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.350 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "True", 2024-04-24T23:32:02.350 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.350 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.350 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.350 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.350 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.350 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.351 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.351 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.351 INFO:teuthology.orchestra.run.smithi080.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-24T23:32:02.351 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-24T23:32:02.351 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.351 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.351 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.351 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:02.352 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.352 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.352 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.352 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.352 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.352 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.352 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.352 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.352 INFO:teuthology.orchestra.run.smithi080.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-24T23:32:02.353 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-24T23:32:02.353 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.353 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.353 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.353 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:02.353 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.353 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.353 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.353 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.354 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.354 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.354 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.354 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.354 INFO:teuthology.orchestra.run.smithi080.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-24T23:32:02.354 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-24T23:32:02.354 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.354 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.354 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.355 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:02.355 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.355 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.355 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.355 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.355 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.355 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.355 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.355 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.356 INFO:teuthology.orchestra.run.smithi080.stdout: "PWD_POLICY_ENABLED": { 2024-04-24T23:32:02.356 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-24T23:32:02.356 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.356 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.356 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.356 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "True", 2024-04-24T23:32:02.356 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.356 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.357 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.357 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.357 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.357 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.357 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.357 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.357 INFO:teuthology.orchestra.run.smithi080.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-24T23:32:02.357 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-24T23:32:02.357 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.358 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.358 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.358 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-24T23:32:02.358 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.358 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.358 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.358 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.358 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.359 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.359 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.359 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.359 INFO:teuthology.orchestra.run.smithi080.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-24T23:32:02.359 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-24T23:32:02.359 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "int", 2024-04-24T23:32:02.359 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.359 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.359 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "10", 2024-04-24T23:32:02.360 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.360 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.360 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.360 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.360 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.360 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.360 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.360 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.360 INFO:teuthology.orchestra.run.smithi080.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-24T23:32:02.361 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-24T23:32:02.361 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "int", 2024-04-24T23:32:02.361 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.361 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.361 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "8", 2024-04-24T23:32:02.361 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.361 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.361 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.362 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.362 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.362 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.362 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.362 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.362 INFO:teuthology.orchestra.run.smithi080.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-24T23:32:02.362 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-24T23:32:02.362 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "int", 2024-04-24T23:32:02.363 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.363 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.363 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "45", 2024-04-24T23:32:02.363 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.363 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.363 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.363 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.364 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.364 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.364 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.364 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.364 INFO:teuthology.orchestra.run.smithi080.stdout: "RGW_API_ACCESS_KEY": { 2024-04-24T23:32:02.364 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-24T23:32:02.364 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.364 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.364 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.365 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:02.365 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.365 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.365 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.365 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.365 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.365 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.365 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.365 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.366 INFO:teuthology.orchestra.run.smithi080.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-24T23:32:02.366 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-24T23:32:02.366 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.366 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.366 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.366 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "admin", 2024-04-24T23:32:02.366 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.366 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.367 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.367 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.367 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.367 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.367 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.367 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.367 INFO:teuthology.orchestra.run.smithi080.stdout: "RGW_API_SECRET_KEY": { 2024-04-24T23:32:02.367 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-24T23:32:02.367 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.368 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.368 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.368 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:02.368 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.368 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.368 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.368 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.368 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.368 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.369 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.369 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.369 INFO:teuthology.orchestra.run.smithi080.stdout: "RGW_API_SSL_VERIFY": { 2024-04-24T23:32:02.369 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-24T23:32:02.369 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.369 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.369 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.369 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "True", 2024-04-24T23:32:02.370 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.370 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.370 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.370 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.370 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.370 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.370 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.370 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.370 INFO:teuthology.orchestra.run.smithi080.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-24T23:32:02.371 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-24T23:32:02.371 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "int", 2024-04-24T23:32:02.371 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.371 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.371 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "0", 2024-04-24T23:32:02.371 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.371 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.371 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.372 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.372 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.372 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.372 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.372 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.372 INFO:teuthology.orchestra.run.smithi080.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-24T23:32:02.372 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-24T23:32:02.372 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "int", 2024-04-24T23:32:02.372 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.373 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.373 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "10", 2024-04-24T23:32:02.373 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.373 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.373 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.373 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.373 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.373 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.373 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.374 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.374 INFO:teuthology.orchestra.run.smithi080.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-24T23:32:02.374 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-24T23:32:02.374 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "int", 2024-04-24T23:32:02.374 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.374 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.374 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "5", 2024-04-24T23:32:02.374 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.375 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.375 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.375 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.375 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.375 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.375 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.375 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.375 INFO:teuthology.orchestra.run.smithi080.stdout: "cross_origin_url": { 2024-04-24T23:32:02.375 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "cross_origin_url", 2024-04-24T23:32:02.376 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.376 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.376 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.376 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:02.376 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.376 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.376 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.376 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.376 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.377 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.377 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.377 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.377 INFO:teuthology.orchestra.run.smithi080.stdout: "crt_file": { 2024-04-24T23:32:02.377 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "crt_file", 2024-04-24T23:32:02.377 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.377 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.377 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.378 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:02.378 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.378 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.378 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.378 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.378 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.378 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.378 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.378 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.379 INFO:teuthology.orchestra.run.smithi080.stdout: "debug": { 2024-04-24T23:32:02.379 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "debug", 2024-04-24T23:32:02.379 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.379 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.379 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.379 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:02.379 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.379 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.379 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.380 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Enable/disable debug options", 2024-04-24T23:32:02.380 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.380 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.380 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.380 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.380 INFO:teuthology.orchestra.run.smithi080.stdout: "jwt_token_ttl": { 2024-04-24T23:32:02.380 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "jwt_token_ttl", 2024-04-24T23:32:02.380 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "int", 2024-04-24T23:32:02.380 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.381 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.381 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "28800", 2024-04-24T23:32:02.381 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.381 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.381 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.381 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.381 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.381 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.382 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.382 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.382 INFO:teuthology.orchestra.run.smithi080.stdout: "key_file": { 2024-04-24T23:32:02.382 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "key_file", 2024-04-24T23:32:02.382 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.382 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.382 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.382 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:02.382 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.383 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.383 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.383 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.383 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.383 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.383 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.383 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.383 INFO:teuthology.orchestra.run.smithi080.stdout: "log_level": { 2024-04-24T23:32:02.384 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_level", 2024-04-24T23:32:02.384 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.384 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.384 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.384 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:02.384 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.384 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.384 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:02.384 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:02.385 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:02.385 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:02.385 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:02.385 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:02.386 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:02.386 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:02.386 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.386 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.386 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.386 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.386 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.386 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster": { 2024-04-24T23:32:02.387 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster", 2024-04-24T23:32:02.387 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.387 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.387 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.387 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:02.387 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.387 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.387 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.387 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.388 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.388 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.388 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.388 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.388 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster_level": { 2024-04-24T23:32:02.388 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster_level", 2024-04-24T23:32:02.388 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.388 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.388 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.389 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "info", 2024-04-24T23:32:02.389 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.389 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.389 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:02.389 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:02.389 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:02.389 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:02.389 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:02.389 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:02.390 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:02.390 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:02.390 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.390 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.390 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.390 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.390 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.390 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_file": { 2024-04-24T23:32:02.391 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_file", 2024-04-24T23:32:02.391 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.391 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.391 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.391 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:02.391 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.391 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.391 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.391 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.392 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.392 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.392 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.392 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.392 INFO:teuthology.orchestra.run.smithi080.stdout: "motd": { 2024-04-24T23:32:02.392 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "motd", 2024-04-24T23:32:02.392 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.392 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.392 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.393 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:02.393 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.393 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.393 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.393 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "The message of the day", 2024-04-24T23:32:02.393 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.393 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.393 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.394 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.394 INFO:teuthology.orchestra.run.smithi080.stdout: "redirect_resolve_ip_addr": { 2024-04-24T23:32:02.394 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "redirect_resolve_ip_addr", 2024-04-24T23:32:02.394 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.394 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.394 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.394 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:02.394 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.394 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.395 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.395 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.395 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.395 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.395 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.395 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.395 INFO:teuthology.orchestra.run.smithi080.stdout: "server_addr": { 2024-04-24T23:32:02.395 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "server_addr", 2024-04-24T23:32:02.396 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.396 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.396 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.396 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "::", 2024-04-24T23:32:02.396 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.396 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.396 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.396 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.396 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.397 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.397 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.397 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.397 INFO:teuthology.orchestra.run.smithi080.stdout: "server_port": { 2024-04-24T23:32:02.397 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "server_port", 2024-04-24T23:32:02.397 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "int", 2024-04-24T23:32:02.397 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.397 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.397 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "8080", 2024-04-24T23:32:02.398 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.398 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.398 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.398 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.398 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.398 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.398 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.398 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.399 INFO:teuthology.orchestra.run.smithi080.stdout: "ssl": { 2024-04-24T23:32:02.399 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "ssl", 2024-04-24T23:32:02.399 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.399 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.399 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.399 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "True", 2024-04-24T23:32:02.399 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.399 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.399 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.400 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.400 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.400 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.400 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.400 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.400 INFO:teuthology.orchestra.run.smithi080.stdout: "ssl_server_port": { 2024-04-24T23:32:02.400 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "ssl_server_port", 2024-04-24T23:32:02.400 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "int", 2024-04-24T23:32:02.401 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.401 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.401 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "8443", 2024-04-24T23:32:02.401 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.401 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.401 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.401 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.401 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.401 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.402 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.402 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.402 INFO:teuthology.orchestra.run.smithi080.stdout: "standby_behaviour": { 2024-04-24T23:32:02.402 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "standby_behaviour", 2024-04-24T23:32:02.402 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.402 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.402 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.402 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "redirect", 2024-04-24T23:32:02.403 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.403 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.403 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:02.403 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:02.403 INFO:teuthology.orchestra.run.smithi080.stdout: "redirect" 2024-04-24T23:32:02.403 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:02.403 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.403 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.403 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.404 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.404 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.404 INFO:teuthology.orchestra.run.smithi080.stdout: "standby_error_status_code": { 2024-04-24T23:32:02.404 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "standby_error_status_code", 2024-04-24T23:32:02.404 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "int", 2024-04-24T23:32:02.404 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.404 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.404 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "500", 2024-04-24T23:32:02.404 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "400", 2024-04-24T23:32:02.405 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "599", 2024-04-24T23:32:02.405 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.405 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.405 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.405 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.405 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.405 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.405 INFO:teuthology.orchestra.run.smithi080.stdout: "url_prefix": { 2024-04-24T23:32:02.405 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "url_prefix", 2024-04-24T23:32:02.406 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.406 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.406 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.406 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:02.406 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.406 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.406 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.406 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.407 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.407 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.407 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.407 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:02.407 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:02.407 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.407 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-24T23:32:02.407 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "devicehealth", 2024-04-24T23:32:02.407 INFO:teuthology.orchestra.run.smithi080.stdout: "can_run": true, 2024-04-24T23:32:02.408 INFO:teuthology.orchestra.run.smithi080.stdout: "error_string": "", 2024-04-24T23:32:02.408 INFO:teuthology.orchestra.run.smithi080.stdout: "module_options": { 2024-04-24T23:32:02.408 INFO:teuthology.orchestra.run.smithi080.stdout: "enable_monitoring": { 2024-04-24T23:32:02.408 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "enable_monitoring", 2024-04-24T23:32:02.408 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.408 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.408 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.408 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "True", 2024-04-24T23:32:02.408 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.409 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.409 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.409 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "monitor device health metrics", 2024-04-24T23:32:02.409 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.409 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.409 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.409 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.409 INFO:teuthology.orchestra.run.smithi080.stdout: "log_level": { 2024-04-24T23:32:02.409 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_level", 2024-04-24T23:32:02.410 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.410 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.410 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.410 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:02.410 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.410 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.410 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:02.410 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:02.411 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:02.411 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:02.411 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:02.411 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:02.411 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:02.411 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:02.411 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.411 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.411 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.412 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.412 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.412 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster": { 2024-04-24T23:32:02.412 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster", 2024-04-24T23:32:02.412 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.412 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.412 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.412 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:02.413 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.413 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.413 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.413 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.413 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.413 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.413 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.413 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.413 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster_level": { 2024-04-24T23:32:02.414 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster_level", 2024-04-24T23:32:02.414 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.414 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.414 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.414 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "info", 2024-04-24T23:32:02.414 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.414 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.414 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:02.415 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:02.415 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:02.415 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:02.415 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:02.415 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:02.415 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:02.415 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:02.415 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.415 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.416 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.416 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.416 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.416 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_file": { 2024-04-24T23:32:02.416 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_file", 2024-04-24T23:32:02.416 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.416 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.416 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.417 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:02.417 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.417 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.417 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.417 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.417 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.417 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.417 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.417 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.418 INFO:teuthology.orchestra.run.smithi080.stdout: "mark_out_threshold": { 2024-04-24T23:32:02.418 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "mark_out_threshold", 2024-04-24T23:32:02.418 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "secs", 2024-04-24T23:32:02.418 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.418 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.418 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "2419200", 2024-04-24T23:32:02.418 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.418 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.418 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.419 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-24T23:32:02.419 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.419 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.419 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.419 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.419 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_name": { 2024-04-24T23:32:02.419 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "pool_name", 2024-04-24T23:32:02.419 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.420 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.420 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.420 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "device_health_metrics", 2024-04-24T23:32:02.420 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.420 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.420 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.420 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-24T23:32:02.420 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.420 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.421 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.421 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.421 INFO:teuthology.orchestra.run.smithi080.stdout: "retention_period": { 2024-04-24T23:32:02.421 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "retention_period", 2024-04-24T23:32:02.421 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "secs", 2024-04-24T23:32:02.421 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.421 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.421 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "15552000", 2024-04-24T23:32:02.421 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.422 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.422 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.422 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "how long to retain device health metrics", 2024-04-24T23:32:02.422 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.422 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.422 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.422 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.422 INFO:teuthology.orchestra.run.smithi080.stdout: "scrape_frequency": { 2024-04-24T23:32:02.422 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "scrape_frequency", 2024-04-24T23:32:02.423 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "secs", 2024-04-24T23:32:02.423 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.423 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.423 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "86400", 2024-04-24T23:32:02.423 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.423 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.423 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.423 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-24T23:32:02.424 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.424 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.424 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.424 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.424 INFO:teuthology.orchestra.run.smithi080.stdout: "self_heal": { 2024-04-24T23:32:02.424 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "self_heal", 2024-04-24T23:32:02.424 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.424 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.424 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.425 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "True", 2024-04-24T23:32:02.425 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.425 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.425 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.425 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-24T23:32:02.425 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.425 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.425 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.425 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.426 INFO:teuthology.orchestra.run.smithi080.stdout: "sleep_interval": { 2024-04-24T23:32:02.426 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "sleep_interval", 2024-04-24T23:32:02.426 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "secs", 2024-04-24T23:32:02.426 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.426 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.426 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "600", 2024-04-24T23:32:02.426 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.426 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.427 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.427 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "how frequently to wake up and check device health", 2024-04-24T23:32:02.427 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.427 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.427 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.427 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.427 INFO:teuthology.orchestra.run.smithi080.stdout: "warn_threshold": { 2024-04-24T23:32:02.427 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "warn_threshold", 2024-04-24T23:32:02.427 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "secs", 2024-04-24T23:32:02.428 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.428 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.428 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "7257600", 2024-04-24T23:32:02.428 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.428 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.428 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.428 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-24T23:32:02.428 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.428 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.429 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.429 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:02.429 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:02.429 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.429 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-24T23:32:02.429 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "diskprediction_local", 2024-04-24T23:32:02.429 INFO:teuthology.orchestra.run.smithi080.stdout: "can_run": true, 2024-04-24T23:32:02.429 INFO:teuthology.orchestra.run.smithi080.stdout: "error_string": "", 2024-04-24T23:32:02.430 INFO:teuthology.orchestra.run.smithi080.stdout: "module_options": { 2024-04-24T23:32:02.430 INFO:teuthology.orchestra.run.smithi080.stdout: "log_level": { 2024-04-24T23:32:02.430 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_level", 2024-04-24T23:32:02.430 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.430 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.430 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.430 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:02.430 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.430 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.431 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:02.431 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:02.431 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:02.431 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:02.431 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:02.431 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:02.431 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:02.431 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:02.431 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.432 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.432 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.432 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.432 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.432 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster": { 2024-04-24T23:32:02.432 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster", 2024-04-24T23:32:02.432 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.432 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.433 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.433 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:02.433 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.433 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.433 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.433 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.433 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.433 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.433 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.434 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.434 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster_level": { 2024-04-24T23:32:02.434 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster_level", 2024-04-24T23:32:02.434 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.434 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.434 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.434 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "info", 2024-04-24T23:32:02.434 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.434 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.435 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:02.435 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:02.435 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:02.435 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:02.435 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:02.435 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:02.435 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:02.435 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:02.435 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.436 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.436 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.436 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.436 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.436 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_file": { 2024-04-24T23:32:02.436 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_file", 2024-04-24T23:32:02.436 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.436 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.437 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.437 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:02.437 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.437 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.437 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.437 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.437 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.437 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.437 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.438 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.438 INFO:teuthology.orchestra.run.smithi080.stdout: "predict_interval": { 2024-04-24T23:32:02.438 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "predict_interval", 2024-04-24T23:32:02.438 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.438 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.438 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.438 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "86400", 2024-04-24T23:32:02.439 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.439 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.439 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.439 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.439 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.439 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.439 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.439 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.440 INFO:teuthology.orchestra.run.smithi080.stdout: "predictor_model": { 2024-04-24T23:32:02.440 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "predictor_model", 2024-04-24T23:32:02.440 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.440 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.440 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.440 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "prophetstor", 2024-04-24T23:32:02.440 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.440 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.440 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.441 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.441 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.441 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.441 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.441 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.441 INFO:teuthology.orchestra.run.smithi080.stdout: "sleep_interval": { 2024-04-24T23:32:02.441 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "sleep_interval", 2024-04-24T23:32:02.441 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.441 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.442 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.442 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "600", 2024-04-24T23:32:02.442 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.442 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.442 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.442 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.442 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.442 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.443 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.443 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:02.443 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:02.443 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.443 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-24T23:32:02.443 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "influx", 2024-04-24T23:32:02.443 INFO:teuthology.orchestra.run.smithi080.stdout: "can_run": false, 2024-04-24T23:32:02.443 INFO:teuthology.orchestra.run.smithi080.stdout: "error_string": "influxdb python module not found", 2024-04-24T23:32:02.443 INFO:teuthology.orchestra.run.smithi080.stdout: "module_options": { 2024-04-24T23:32:02.444 INFO:teuthology.orchestra.run.smithi080.stdout: "batch_size": { 2024-04-24T23:32:02.444 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "batch_size", 2024-04-24T23:32:02.444 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "int", 2024-04-24T23:32:02.444 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.444 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.444 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "5000", 2024-04-24T23:32:02.444 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.444 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.445 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.445 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-24T23:32:02.445 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.445 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.445 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.445 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.445 INFO:teuthology.orchestra.run.smithi080.stdout: "database": { 2024-04-24T23:32:02.445 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "database", 2024-04-24T23:32:02.445 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.446 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.446 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.446 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "ceph", 2024-04-24T23:32:02.446 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.446 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.446 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.446 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-24T23:32:02.446 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.447 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.447 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.447 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.447 INFO:teuthology.orchestra.run.smithi080.stdout: "hostname": { 2024-04-24T23:32:02.447 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "hostname", 2024-04-24T23:32:02.447 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.447 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.447 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.447 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:02.448 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.448 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.448 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.448 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "InfluxDB server hostname", 2024-04-24T23:32:02.448 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.448 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.448 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.448 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.448 INFO:teuthology.orchestra.run.smithi080.stdout: "interval": { 2024-04-24T23:32:02.449 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "interval", 2024-04-24T23:32:02.449 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "secs", 2024-04-24T23:32:02.449 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.449 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.449 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "30", 2024-04-24T23:32:02.449 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "5", 2024-04-24T23:32:02.449 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.449 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.450 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-24T23:32:02.450 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.450 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.450 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.450 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.450 INFO:teuthology.orchestra.run.smithi080.stdout: "log_level": { 2024-04-24T23:32:02.450 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_level", 2024-04-24T23:32:02.450 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.450 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.451 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.451 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:02.451 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.451 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.451 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:02.451 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:02.451 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:02.451 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:02.451 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:02.452 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:02.452 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:02.452 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:02.452 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.452 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.452 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.452 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.452 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.453 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster": { 2024-04-24T23:32:02.453 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster", 2024-04-24T23:32:02.453 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.453 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.453 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.453 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:02.453 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.453 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.453 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.454 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.454 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.454 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.454 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.454 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.454 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster_level": { 2024-04-24T23:32:02.454 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster_level", 2024-04-24T23:32:02.454 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.455 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.455 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.455 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "info", 2024-04-24T23:32:02.455 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.455 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.455 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:02.455 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:02.455 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:02.456 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:02.456 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:02.456 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:02.456 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:02.456 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:02.456 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.456 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.456 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.456 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.457 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.457 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_file": { 2024-04-24T23:32:02.457 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_file", 2024-04-24T23:32:02.457 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.457 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.457 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.457 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:02.457 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.457 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.458 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.458 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.458 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.458 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.458 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.458 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.458 INFO:teuthology.orchestra.run.smithi080.stdout: "password": { 2024-04-24T23:32:02.458 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "password", 2024-04-24T23:32:02.458 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.459 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.459 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.459 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:02.459 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.459 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.459 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.459 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "password of InfluxDB server user", 2024-04-24T23:32:02.459 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.460 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.460 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.460 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.460 INFO:teuthology.orchestra.run.smithi080.stdout: "port": { 2024-04-24T23:32:02.460 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "port", 2024-04-24T23:32:02.460 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "int", 2024-04-24T23:32:02.460 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.460 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.460 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "8086", 2024-04-24T23:32:02.461 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.461 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.461 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.461 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "InfluxDB server port", 2024-04-24T23:32:02.461 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.461 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.461 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.461 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.461 INFO:teuthology.orchestra.run.smithi080.stdout: "ssl": { 2024-04-24T23:32:02.462 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "ssl", 2024-04-24T23:32:02.462 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.462 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.462 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.462 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "false", 2024-04-24T23:32:02.462 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.462 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.462 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.463 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-24T23:32:02.463 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.463 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.463 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.463 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.463 INFO:teuthology.orchestra.run.smithi080.stdout: "threads": { 2024-04-24T23:32:02.463 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "threads", 2024-04-24T23:32:02.463 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "int", 2024-04-24T23:32:02.463 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.464 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.464 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "5", 2024-04-24T23:32:02.464 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "1", 2024-04-24T23:32:02.464 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "32", 2024-04-24T23:32:02.464 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.464 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-24T23:32:02.464 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.464 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.464 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.465 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.465 INFO:teuthology.orchestra.run.smithi080.stdout: "username": { 2024-04-24T23:32:02.465 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "username", 2024-04-24T23:32:02.465 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.465 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.465 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.465 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:02.465 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.466 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.466 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.466 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "username of InfluxDB server user", 2024-04-24T23:32:02.466 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.466 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.466 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.466 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.466 INFO:teuthology.orchestra.run.smithi080.stdout: "verify_ssl": { 2024-04-24T23:32:02.467 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "verify_ssl", 2024-04-24T23:32:02.467 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.467 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.467 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.467 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "true", 2024-04-24T23:32:02.467 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.467 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.467 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.467 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-24T23:32:02.468 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.468 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.468 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.468 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:02.468 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:02.468 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.468 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-24T23:32:02.468 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "insights", 2024-04-24T23:32:02.468 INFO:teuthology.orchestra.run.smithi080.stdout: "can_run": true, 2024-04-24T23:32:02.469 INFO:teuthology.orchestra.run.smithi080.stdout: "error_string": "", 2024-04-24T23:32:02.469 INFO:teuthology.orchestra.run.smithi080.stdout: "module_options": { 2024-04-24T23:32:02.469 INFO:teuthology.orchestra.run.smithi080.stdout: "log_level": { 2024-04-24T23:32:02.469 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_level", 2024-04-24T23:32:02.469 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.469 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.469 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.469 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:02.469 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.470 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.470 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:02.470 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:02.470 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:02.470 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:02.470 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:02.470 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:02.470 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:02.471 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:02.471 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.471 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.471 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.471 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.471 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.471 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster": { 2024-04-24T23:32:02.471 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster", 2024-04-24T23:32:02.471 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.472 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.472 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.472 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:02.472 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.472 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.472 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.472 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.472 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.472 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.473 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.473 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.473 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster_level": { 2024-04-24T23:32:02.473 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster_level", 2024-04-24T23:32:02.473 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.473 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.473 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.473 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "info", 2024-04-24T23:32:02.473 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.474 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.474 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:02.474 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:02.474 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:02.474 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:02.474 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:02.474 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:02.474 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:02.475 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:02.475 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.475 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.475 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.475 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.475 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.475 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_file": { 2024-04-24T23:32:02.475 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_file", 2024-04-24T23:32:02.475 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.476 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.476 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.476 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:02.476 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.476 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.476 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.476 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.476 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.476 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.477 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.477 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:02.477 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:02.477 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.477 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-24T23:32:02.477 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "iostat", 2024-04-24T23:32:02.477 INFO:teuthology.orchestra.run.smithi080.stdout: "can_run": true, 2024-04-24T23:32:02.477 INFO:teuthology.orchestra.run.smithi080.stdout: "error_string": "", 2024-04-24T23:32:02.477 INFO:teuthology.orchestra.run.smithi080.stdout: "module_options": { 2024-04-24T23:32:02.478 INFO:teuthology.orchestra.run.smithi080.stdout: "log_level": { 2024-04-24T23:32:02.478 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_level", 2024-04-24T23:32:02.478 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.478 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.478 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.478 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:02.478 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.478 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.479 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:02.479 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:02.479 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:02.479 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:02.479 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:02.479 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:02.479 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:02.479 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:02.479 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.480 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.480 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.480 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.480 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.480 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster": { 2024-04-24T23:32:02.480 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster", 2024-04-24T23:32:02.480 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.480 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.481 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.481 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:02.481 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.481 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.481 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.481 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.481 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.481 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.481 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.482 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.482 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster_level": { 2024-04-24T23:32:02.482 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster_level", 2024-04-24T23:32:02.482 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.482 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.482 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.482 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "info", 2024-04-24T23:32:02.482 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.482 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.483 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:02.483 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:02.483 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:02.483 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:02.483 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:02.483 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:02.483 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:02.483 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:02.484 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.484 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.484 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.484 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.484 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.484 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_file": { 2024-04-24T23:32:02.484 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_file", 2024-04-24T23:32:02.484 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.484 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.485 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.485 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:02.485 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.485 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.485 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.485 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.485 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.485 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.485 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.486 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:02.486 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:02.486 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.486 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-24T23:32:02.486 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "k8sevents", 2024-04-24T23:32:02.486 INFO:teuthology.orchestra.run.smithi080.stdout: "can_run": true, 2024-04-24T23:32:02.486 INFO:teuthology.orchestra.run.smithi080.stdout: "error_string": "", 2024-04-24T23:32:02.486 INFO:teuthology.orchestra.run.smithi080.stdout: "module_options": { 2024-04-24T23:32:02.486 INFO:teuthology.orchestra.run.smithi080.stdout: "ceph_event_retention_days": { 2024-04-24T23:32:02.487 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "ceph_event_retention_days", 2024-04-24T23:32:02.487 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "int", 2024-04-24T23:32:02.487 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.487 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.487 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "7", 2024-04-24T23:32:02.487 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.487 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.487 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.488 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-24T23:32:02.488 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.488 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.488 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.488 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.488 INFO:teuthology.orchestra.run.smithi080.stdout: "config_check_secs": { 2024-04-24T23:32:02.488 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "config_check_secs", 2024-04-24T23:32:02.488 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "int", 2024-04-24T23:32:02.488 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.489 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.489 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "10", 2024-04-24T23:32:02.489 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "10", 2024-04-24T23:32:02.489 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.489 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.489 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-24T23:32:02.489 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.489 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.489 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.490 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.490 INFO:teuthology.orchestra.run.smithi080.stdout: "log_level": { 2024-04-24T23:32:02.490 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_level", 2024-04-24T23:32:02.490 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.490 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.490 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.490 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:02.490 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.491 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.491 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:02.491 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:02.491 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:02.491 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:02.491 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:02.491 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:02.491 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:02.491 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:02.492 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.492 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.492 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.492 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.492 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.492 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster": { 2024-04-24T23:32:02.492 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster", 2024-04-24T23:32:02.492 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.493 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.493 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.493 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:02.493 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.493 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.493 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.493 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.493 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.493 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.494 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.494 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.494 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster_level": { 2024-04-24T23:32:02.494 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster_level", 2024-04-24T23:32:02.494 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.494 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.494 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.494 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "info", 2024-04-24T23:32:02.495 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.495 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.495 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:02.495 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:02.495 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:02.495 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:02.495 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:02.495 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:02.495 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:02.496 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:02.496 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.496 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.496 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.496 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.496 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.496 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_file": { 2024-04-24T23:32:02.496 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_file", 2024-04-24T23:32:02.496 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.497 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.497 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.497 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:02.497 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.497 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.497 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.497 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.497 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.497 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.498 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.498 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:02.498 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:02.498 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.498 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-24T23:32:02.498 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "localpool", 2024-04-24T23:32:02.498 INFO:teuthology.orchestra.run.smithi080.stdout: "can_run": true, 2024-04-24T23:32:02.498 INFO:teuthology.orchestra.run.smithi080.stdout: "error_string": "", 2024-04-24T23:32:02.499 INFO:teuthology.orchestra.run.smithi080.stdout: "module_options": { 2024-04-24T23:32:02.499 INFO:teuthology.orchestra.run.smithi080.stdout: "failure_domain": { 2024-04-24T23:32:02.499 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "failure_domain", 2024-04-24T23:32:02.499 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.499 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.499 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.499 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "host", 2024-04-24T23:32:02.499 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.499 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.500 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.500 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "failure domain for any created local pool", 2024-04-24T23:32:02.500 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-24T23:32:02.500 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.500 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.500 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.500 INFO:teuthology.orchestra.run.smithi080.stdout: "log_level": { 2024-04-24T23:32:02.500 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_level", 2024-04-24T23:32:02.501 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.501 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.501 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.501 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:02.501 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.501 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.501 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:02.501 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:02.501 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:02.502 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:02.502 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:02.502 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:02.502 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:02.502 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:02.502 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.502 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.502 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.502 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.503 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.503 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster": { 2024-04-24T23:32:02.503 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster", 2024-04-24T23:32:02.503 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.503 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.503 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.503 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:02.503 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.503 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.504 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.504 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.504 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.504 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.504 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.504 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.504 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster_level": { 2024-04-24T23:32:02.504 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster_level", 2024-04-24T23:32:02.505 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.505 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.505 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.505 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "info", 2024-04-24T23:32:02.505 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.505 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.505 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:02.505 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:02.505 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:02.506 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:02.506 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:02.506 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:02.506 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:02.506 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:02.506 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.506 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.506 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.507 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.507 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.507 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_file": { 2024-04-24T23:32:02.507 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_file", 2024-04-24T23:32:02.507 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.507 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.507 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.507 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:02.507 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.508 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.508 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.508 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.508 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.508 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.508 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.508 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.508 INFO:teuthology.orchestra.run.smithi080.stdout: "min_size": { 2024-04-24T23:32:02.508 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "min_size", 2024-04-24T23:32:02.509 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "int", 2024-04-24T23:32:02.509 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.509 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.509 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:02.509 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.509 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.509 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.509 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "default min_size for any created local pool", 2024-04-24T23:32:02.510 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-24T23:32:02.510 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.510 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.510 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.510 INFO:teuthology.orchestra.run.smithi080.stdout: "num_rep": { 2024-04-24T23:32:02.510 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "num_rep", 2024-04-24T23:32:02.510 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "int", 2024-04-24T23:32:02.510 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.510 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.511 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "3", 2024-04-24T23:32:02.511 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.511 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.511 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.511 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "default replica count for any created local pool", 2024-04-24T23:32:02.511 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.511 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.511 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.512 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.512 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num": { 2024-04-24T23:32:02.512 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "pg_num", 2024-04-24T23:32:02.512 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "int", 2024-04-24T23:32:02.512 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.512 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.512 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "128", 2024-04-24T23:32:02.512 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.512 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.513 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.513 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "default pg_num for any created local pool", 2024-04-24T23:32:02.513 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.513 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.513 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.513 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.513 INFO:teuthology.orchestra.run.smithi080.stdout: "prefix": { 2024-04-24T23:32:02.513 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "prefix", 2024-04-24T23:32:02.513 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.514 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.514 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.514 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:02.514 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.514 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.514 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.514 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "name prefix for any created local pool", 2024-04-24T23:32:02.514 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.515 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.515 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.515 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.515 INFO:teuthology.orchestra.run.smithi080.stdout: "subtree": { 2024-04-24T23:32:02.515 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "subtree", 2024-04-24T23:32:02.515 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.515 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.515 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.515 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "rack", 2024-04-24T23:32:02.516 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.516 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.516 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.516 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-24T23:32:02.516 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-24T23:32:02.516 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.516 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.516 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:02.516 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:02.517 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.517 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-24T23:32:02.517 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "mds_autoscaler", 2024-04-24T23:32:02.517 INFO:teuthology.orchestra.run.smithi080.stdout: "can_run": true, 2024-04-24T23:32:02.517 INFO:teuthology.orchestra.run.smithi080.stdout: "error_string": "", 2024-04-24T23:32:02.517 INFO:teuthology.orchestra.run.smithi080.stdout: "module_options": { 2024-04-24T23:32:02.517 INFO:teuthology.orchestra.run.smithi080.stdout: "log_level": { 2024-04-24T23:32:02.517 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_level", 2024-04-24T23:32:02.518 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.518 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.518 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.518 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:02.518 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.518 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.518 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:02.518 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:02.518 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:02.519 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:02.519 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:02.519 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:02.519 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:02.519 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:02.519 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.519 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.519 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.520 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.520 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.520 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster": { 2024-04-24T23:32:02.520 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster", 2024-04-24T23:32:02.520 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.520 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.520 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.520 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:02.520 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.521 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.521 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.521 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.521 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.521 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.521 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.521 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.521 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster_level": { 2024-04-24T23:32:02.522 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster_level", 2024-04-24T23:32:02.522 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.522 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.522 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.522 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "info", 2024-04-24T23:32:02.522 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.522 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.522 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:02.523 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:02.523 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:02.523 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:02.523 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:02.523 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:02.523 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:02.523 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:02.523 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.523 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.524 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.524 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.524 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.524 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_file": { 2024-04-24T23:32:02.524 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_file", 2024-04-24T23:32:02.524 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.524 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.524 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.524 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:02.525 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.525 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.525 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.525 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.525 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.525 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.525 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.525 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:02.526 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:02.526 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.526 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-24T23:32:02.526 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "mirroring", 2024-04-24T23:32:02.526 INFO:teuthology.orchestra.run.smithi080.stdout: "can_run": true, 2024-04-24T23:32:02.526 INFO:teuthology.orchestra.run.smithi080.stdout: "error_string": "", 2024-04-24T23:32:02.526 INFO:teuthology.orchestra.run.smithi080.stdout: "module_options": { 2024-04-24T23:32:02.526 INFO:teuthology.orchestra.run.smithi080.stdout: "log_level": { 2024-04-24T23:32:02.526 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_level", 2024-04-24T23:32:02.527 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.527 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.527 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.527 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:02.527 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.527 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.527 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:02.527 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:02.528 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:02.528 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:02.528 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:02.528 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:02.528 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:02.528 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:02.528 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.528 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.528 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.529 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.529 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.529 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster": { 2024-04-24T23:32:02.529 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster", 2024-04-24T23:32:02.529 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.529 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.529 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.529 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:02.529 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.530 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.530 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.530 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.530 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.530 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.530 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.530 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.530 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster_level": { 2024-04-24T23:32:02.530 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster_level", 2024-04-24T23:32:02.531 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.531 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.531 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.531 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "info", 2024-04-24T23:32:02.531 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.531 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.531 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:02.531 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:02.532 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:02.532 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:02.532 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:02.532 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:02.532 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:02.532 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:02.532 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.532 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.532 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.533 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.533 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.533 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_file": { 2024-04-24T23:32:02.533 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_file", 2024-04-24T23:32:02.533 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.533 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.533 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.533 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:02.533 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.534 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.534 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.534 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.534 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.534 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.534 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.534 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:02.534 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:02.535 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.535 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-24T23:32:02.535 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "nfs", 2024-04-24T23:32:02.535 INFO:teuthology.orchestra.run.smithi080.stdout: "can_run": true, 2024-04-24T23:32:02.535 INFO:teuthology.orchestra.run.smithi080.stdout: "error_string": "", 2024-04-24T23:32:02.535 INFO:teuthology.orchestra.run.smithi080.stdout: "module_options": { 2024-04-24T23:32:02.535 INFO:teuthology.orchestra.run.smithi080.stdout: "log_level": { 2024-04-24T23:32:02.535 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_level", 2024-04-24T23:32:02.535 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.536 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.536 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.536 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:02.536 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.536 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.536 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:02.536 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:02.536 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:02.536 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:02.537 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:02.537 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:02.537 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:02.537 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:02.537 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.537 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.537 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.537 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.537 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.538 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster": { 2024-04-24T23:32:02.538 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster", 2024-04-24T23:32:02.538 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.538 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.538 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.538 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:02.538 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.538 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.539 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.539 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.539 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.539 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.539 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.539 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.539 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster_level": { 2024-04-24T23:32:02.539 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster_level", 2024-04-24T23:32:02.539 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.540 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.540 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.540 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "info", 2024-04-24T23:32:02.540 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.540 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.540 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:02.540 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:02.540 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:02.541 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:02.541 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:02.541 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:02.541 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:02.541 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:02.541 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.541 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.541 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.541 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.542 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.542 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_file": { 2024-04-24T23:32:02.542 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_file", 2024-04-24T23:32:02.542 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.542 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.542 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.542 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:02.542 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.543 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.543 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.543 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.543 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.543 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.543 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.543 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:02.543 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:02.543 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.544 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-24T23:32:02.544 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "orchestrator", 2024-04-24T23:32:02.544 INFO:teuthology.orchestra.run.smithi080.stdout: "can_run": true, 2024-04-24T23:32:02.544 INFO:teuthology.orchestra.run.smithi080.stdout: "error_string": "", 2024-04-24T23:32:02.544 INFO:teuthology.orchestra.run.smithi080.stdout: "module_options": { 2024-04-24T23:32:02.544 INFO:teuthology.orchestra.run.smithi080.stdout: "fail_fs": { 2024-04-24T23:32:02.544 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "fail_fs", 2024-04-24T23:32:02.544 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.544 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.545 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.545 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:02.545 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.545 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.545 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.545 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-24T23:32:02.545 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.545 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.546 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.546 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.546 INFO:teuthology.orchestra.run.smithi080.stdout: "log_level": { 2024-04-24T23:32:02.546 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_level", 2024-04-24T23:32:02.546 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.546 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.546 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.546 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:02.546 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.547 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.547 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:02.547 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:02.547 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:02.547 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:02.547 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:02.547 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:02.547 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:02.548 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:02.548 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.548 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.548 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.548 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.548 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.548 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster": { 2024-04-24T23:32:02.548 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster", 2024-04-24T23:32:02.548 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.549 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.549 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.549 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:02.549 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.549 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.549 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.549 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.549 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.549 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.550 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.550 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.550 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster_level": { 2024-04-24T23:32:02.550 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster_level", 2024-04-24T23:32:02.550 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.550 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.550 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.550 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "info", 2024-04-24T23:32:02.551 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.551 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.551 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:02.551 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:02.551 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:02.551 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:02.551 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:02.551 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:02.551 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:02.552 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:02.552 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.552 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.552 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.552 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.552 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.552 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_file": { 2024-04-24T23:32:02.552 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_file", 2024-04-24T23:32:02.552 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.553 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.553 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.553 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:02.553 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.553 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.553 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.553 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.553 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.554 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.554 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.554 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.554 INFO:teuthology.orchestra.run.smithi080.stdout: "orchestrator": { 2024-04-24T23:32:02.554 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "orchestrator", 2024-04-24T23:32:02.554 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.554 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.554 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.554 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:02.555 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.555 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.555 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:02.555 INFO:teuthology.orchestra.run.smithi080.stdout: "cephadm", 2024-04-24T23:32:02.555 INFO:teuthology.orchestra.run.smithi080.stdout: "rook", 2024-04-24T23:32:02.555 INFO:teuthology.orchestra.run.smithi080.stdout: "test_orchestrator" 2024-04-24T23:32:02.555 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:02.555 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Orchestrator backend", 2024-04-24T23:32:02.555 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.556 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.556 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.556 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:02.556 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:02.556 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.556 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-24T23:32:02.556 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "osd_perf_query", 2024-04-24T23:32:02.556 INFO:teuthology.orchestra.run.smithi080.stdout: "can_run": true, 2024-04-24T23:32:02.557 INFO:teuthology.orchestra.run.smithi080.stdout: "error_string": "", 2024-04-24T23:32:02.557 INFO:teuthology.orchestra.run.smithi080.stdout: "module_options": { 2024-04-24T23:32:02.557 INFO:teuthology.orchestra.run.smithi080.stdout: "log_level": { 2024-04-24T23:32:02.557 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_level", 2024-04-24T23:32:02.557 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.557 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.557 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.557 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:02.557 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.558 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.558 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:02.558 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:02.558 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:02.558 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:02.558 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:02.558 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:02.558 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:02.559 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:02.559 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.559 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.559 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.559 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.559 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.559 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster": { 2024-04-24T23:32:02.559 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster", 2024-04-24T23:32:02.559 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.560 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.560 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.560 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:02.560 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.560 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.560 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.560 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.560 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.560 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.561 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.561 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.561 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster_level": { 2024-04-24T23:32:02.561 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster_level", 2024-04-24T23:32:02.561 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.561 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.561 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.561 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "info", 2024-04-24T23:32:02.561 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.562 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.562 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:02.562 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:02.562 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:02.562 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:02.562 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:02.562 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:02.562 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:02.563 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:02.563 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.563 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.563 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.563 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.563 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.563 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_file": { 2024-04-24T23:32:02.563 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_file", 2024-04-24T23:32:02.563 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.564 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.564 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.564 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:02.564 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.564 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.564 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.564 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.564 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.564 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.565 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.565 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:02.565 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:02.565 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.565 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-24T23:32:02.565 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "osd_support", 2024-04-24T23:32:02.565 INFO:teuthology.orchestra.run.smithi080.stdout: "can_run": true, 2024-04-24T23:32:02.565 INFO:teuthology.orchestra.run.smithi080.stdout: "error_string": "", 2024-04-24T23:32:02.565 INFO:teuthology.orchestra.run.smithi080.stdout: "module_options": { 2024-04-24T23:32:02.566 INFO:teuthology.orchestra.run.smithi080.stdout: "log_level": { 2024-04-24T23:32:02.566 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_level", 2024-04-24T23:32:02.566 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.566 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.566 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.566 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:02.566 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.566 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.567 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:02.567 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:02.567 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:02.567 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:02.567 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:02.567 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:02.567 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:02.567 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:02.567 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.568 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.568 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.568 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.568 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.568 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster": { 2024-04-24T23:32:02.568 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster", 2024-04-24T23:32:02.568 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.568 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.568 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.569 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:02.569 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.569 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.569 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.569 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.569 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.569 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.569 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.569 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.570 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster_level": { 2024-04-24T23:32:02.570 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster_level", 2024-04-24T23:32:02.570 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.570 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.570 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.570 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "info", 2024-04-24T23:32:02.570 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.570 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.571 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:02.571 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:02.571 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:02.571 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:02.571 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:02.571 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:02.571 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:02.571 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:02.571 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.572 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.572 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.572 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.572 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.572 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_file": { 2024-04-24T23:32:02.572 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_file", 2024-04-24T23:32:02.572 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.572 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.572 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.573 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:02.573 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.573 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.573 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.573 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.573 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.573 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.573 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.573 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:02.574 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:02.574 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.574 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-24T23:32:02.574 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "pg_autoscaler", 2024-04-24T23:32:02.574 INFO:teuthology.orchestra.run.smithi080.stdout: "can_run": true, 2024-04-24T23:32:02.574 INFO:teuthology.orchestra.run.smithi080.stdout: "error_string": "", 2024-04-24T23:32:02.574 INFO:teuthology.orchestra.run.smithi080.stdout: "module_options": { 2024-04-24T23:32:02.574 INFO:teuthology.orchestra.run.smithi080.stdout: "log_level": { 2024-04-24T23:32:02.575 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_level", 2024-04-24T23:32:02.575 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.575 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.575 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.575 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:02.575 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.575 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.575 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:02.575 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:02.576 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:02.576 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:02.576 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:02.576 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:02.576 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:02.576 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:02.576 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.576 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.577 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.577 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.577 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.577 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster": { 2024-04-24T23:32:02.577 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster", 2024-04-24T23:32:02.577 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.577 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.577 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.577 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:02.578 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.578 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.578 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.578 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.578 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.578 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.578 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.578 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.578 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster_level": { 2024-04-24T23:32:02.579 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster_level", 2024-04-24T23:32:02.579 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.579 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.579 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.579 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "info", 2024-04-24T23:32:02.579 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.579 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.579 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:02.579 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:02.580 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:02.580 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:02.580 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:02.580 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:02.580 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:02.580 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:02.580 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.580 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.581 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.581 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.581 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.581 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_file": { 2024-04-24T23:32:02.581 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_file", 2024-04-24T23:32:02.581 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.581 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.581 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.581 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:02.582 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.582 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.582 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.582 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.582 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.582 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.582 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.582 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.582 INFO:teuthology.orchestra.run.smithi080.stdout: "sleep_interval": { 2024-04-24T23:32:02.583 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "sleep_interval", 2024-04-24T23:32:02.583 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "secs", 2024-04-24T23:32:02.583 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.583 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.583 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "60", 2024-04-24T23:32:02.583 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.583 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.583 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.584 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.584 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.584 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.584 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.584 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.584 INFO:teuthology.orchestra.run.smithi080.stdout: "threshold": { 2024-04-24T23:32:02.584 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "threshold", 2024-04-24T23:32:02.584 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "float", 2024-04-24T23:32:02.584 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.585 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.585 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "3.0", 2024-04-24T23:32:02.585 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "1.0", 2024-04-24T23:32:02.585 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.585 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.585 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "scaling threshold", 2024-04-24T23:32:02.585 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-24T23:32:02.585 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.585 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.586 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:02.586 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:02.586 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.586 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-24T23:32:02.586 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "progress", 2024-04-24T23:32:02.586 INFO:teuthology.orchestra.run.smithi080.stdout: "can_run": true, 2024-04-24T23:32:02.586 INFO:teuthology.orchestra.run.smithi080.stdout: "error_string": "", 2024-04-24T23:32:02.586 INFO:teuthology.orchestra.run.smithi080.stdout: "module_options": { 2024-04-24T23:32:02.587 INFO:teuthology.orchestra.run.smithi080.stdout: "allow_pg_recovery_event": { 2024-04-24T23:32:02.587 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "allow_pg_recovery_event", 2024-04-24T23:32:02.587 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.587 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.590 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.590 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:02.590 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.591 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.591 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.591 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-24T23:32:02.591 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.591 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.591 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.591 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.591 INFO:teuthology.orchestra.run.smithi080.stdout: "enabled": { 2024-04-24T23:32:02.592 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "enabled", 2024-04-24T23:32:02.592 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.592 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.592 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.592 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "True", 2024-04-24T23:32:02.592 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.592 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.592 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.592 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.593 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.593 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.593 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.593 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.593 INFO:teuthology.orchestra.run.smithi080.stdout: "log_level": { 2024-04-24T23:32:02.593 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_level", 2024-04-24T23:32:02.593 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.593 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.593 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.594 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:02.594 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.594 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.594 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:02.594 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:02.594 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:02.594 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:02.594 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:02.595 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:02.595 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:02.595 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:02.595 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.595 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.595 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.595 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.595 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.595 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster": { 2024-04-24T23:32:02.596 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster", 2024-04-24T23:32:02.596 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.596 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.596 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.596 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:02.596 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.596 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.596 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.596 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.597 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.597 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.597 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.597 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.597 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster_level": { 2024-04-24T23:32:02.597 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster_level", 2024-04-24T23:32:02.597 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.597 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.598 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.598 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "info", 2024-04-24T23:32:02.598 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.598 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.598 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:02.598 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:02.598 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:02.598 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:02.598 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:02.599 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:02.599 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:02.599 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:02.599 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.599 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.599 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.599 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.599 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.599 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_file": { 2024-04-24T23:32:02.600 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_file", 2024-04-24T23:32:02.600 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.600 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.600 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.600 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:02.600 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.600 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.600 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.601 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.601 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.601 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.601 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.601 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.601 INFO:teuthology.orchestra.run.smithi080.stdout: "max_completed_events": { 2024-04-24T23:32:02.601 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "max_completed_events", 2024-04-24T23:32:02.601 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "int", 2024-04-24T23:32:02.601 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.602 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.602 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "50", 2024-04-24T23:32:02.602 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.602 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.602 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.602 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "number of past completed events to remember", 2024-04-24T23:32:02.602 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.602 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.602 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.603 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.603 INFO:teuthology.orchestra.run.smithi080.stdout: "sleep_interval": { 2024-04-24T23:32:02.603 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "sleep_interval", 2024-04-24T23:32:02.603 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "secs", 2024-04-24T23:32:02.603 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.603 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.603 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "5", 2024-04-24T23:32:02.603 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.604 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.604 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.604 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "how long the module is going to sleep", 2024-04-24T23:32:02.604 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.604 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.604 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.604 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:02.604 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:02.604 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.605 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-24T23:32:02.605 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "prometheus", 2024-04-24T23:32:02.605 INFO:teuthology.orchestra.run.smithi080.stdout: "can_run": true, 2024-04-24T23:32:02.605 INFO:teuthology.orchestra.run.smithi080.stdout: "error_string": "", 2024-04-24T23:32:02.605 INFO:teuthology.orchestra.run.smithi080.stdout: "module_options": { 2024-04-24T23:32:02.605 INFO:teuthology.orchestra.run.smithi080.stdout: "cache": { 2024-04-24T23:32:02.605 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "cache", 2024-04-24T23:32:02.605 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.605 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.606 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.606 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "True", 2024-04-24T23:32:02.606 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.606 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.606 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.606 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.606 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.606 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.607 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.607 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.607 INFO:teuthology.orchestra.run.smithi080.stdout: "exclude_perf_counters": { 2024-04-24T23:32:02.607 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "exclude_perf_counters", 2024-04-24T23:32:02.607 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.607 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.607 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.607 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "True", 2024-04-24T23:32:02.607 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.608 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.608 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.608 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-24T23:32:02.608 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-24T23:32:02.608 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.608 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.608 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.608 INFO:teuthology.orchestra.run.smithi080.stdout: "log_level": { 2024-04-24T23:32:02.609 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_level", 2024-04-24T23:32:02.609 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.609 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.609 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.609 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:02.609 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.609 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.609 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:02.609 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:02.610 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:02.610 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:02.610 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:02.610 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:02.610 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:02.610 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:02.610 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.610 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.610 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.611 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.611 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.611 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster": { 2024-04-24T23:32:02.611 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster", 2024-04-24T23:32:02.611 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.611 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.611 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.611 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:02.612 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.612 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.612 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.612 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.612 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.612 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.612 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.612 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.612 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster_level": { 2024-04-24T23:32:02.613 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster_level", 2024-04-24T23:32:02.613 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.613 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.613 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.613 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "info", 2024-04-24T23:32:02.613 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.613 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.613 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:02.613 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:02.614 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:02.614 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:02.614 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:02.614 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:02.614 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:02.614 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:02.614 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.614 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.615 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.615 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.615 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.615 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_file": { 2024-04-24T23:32:02.615 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_file", 2024-04-24T23:32:02.615 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.615 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.615 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.615 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:02.616 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.616 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.616 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.616 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.616 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.616 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.616 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.616 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.616 INFO:teuthology.orchestra.run.smithi080.stdout: "rbd_stats_pools": { 2024-04-24T23:32:02.617 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "rbd_stats_pools", 2024-04-24T23:32:02.617 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.617 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.617 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.617 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:02.617 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.617 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.617 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.617 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.618 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.618 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.618 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.618 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.618 INFO:teuthology.orchestra.run.smithi080.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-24T23:32:02.618 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-24T23:32:02.618 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "int", 2024-04-24T23:32:02.618 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.619 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.619 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "300", 2024-04-24T23:32:02.619 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.619 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.619 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.619 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.619 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.619 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.619 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.620 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.620 INFO:teuthology.orchestra.run.smithi080.stdout: "scrape_interval": { 2024-04-24T23:32:02.620 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "scrape_interval", 2024-04-24T23:32:02.620 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "float", 2024-04-24T23:32:02.620 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.620 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.620 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "15.0", 2024-04-24T23:32:02.620 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.620 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.621 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.621 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.621 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.621 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.621 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.621 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.621 INFO:teuthology.orchestra.run.smithi080.stdout: "server_addr": { 2024-04-24T23:32:02.621 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "server_addr", 2024-04-24T23:32:02.621 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.622 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.622 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.622 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "::", 2024-04-24T23:32:02.622 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.622 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.622 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.622 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-24T23:32:02.622 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.623 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.623 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.623 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.623 INFO:teuthology.orchestra.run.smithi080.stdout: "server_port": { 2024-04-24T23:32:02.623 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "server_port", 2024-04-24T23:32:02.623 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "int", 2024-04-24T23:32:02.623 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.623 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.623 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "9283", 2024-04-24T23:32:02.624 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.624 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.624 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.624 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-24T23:32:02.624 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.624 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.624 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.624 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.624 INFO:teuthology.orchestra.run.smithi080.stdout: "stale_cache_strategy": { 2024-04-24T23:32:02.625 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "stale_cache_strategy", 2024-04-24T23:32:02.625 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.625 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.625 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.625 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "log", 2024-04-24T23:32:02.625 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.625 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.625 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.625 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.626 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.626 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.626 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.626 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.626 INFO:teuthology.orchestra.run.smithi080.stdout: "standby_behaviour": { 2024-04-24T23:32:02.626 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "standby_behaviour", 2024-04-24T23:32:02.626 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.626 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.627 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.627 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "default", 2024-04-24T23:32:02.627 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.627 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.627 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:02.627 INFO:teuthology.orchestra.run.smithi080.stdout: "default", 2024-04-24T23:32:02.627 INFO:teuthology.orchestra.run.smithi080.stdout: "error" 2024-04-24T23:32:02.627 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:02.627 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.628 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.628 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.628 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.628 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.628 INFO:teuthology.orchestra.run.smithi080.stdout: "standby_error_status_code": { 2024-04-24T23:32:02.628 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "standby_error_status_code", 2024-04-24T23:32:02.628 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "int", 2024-04-24T23:32:02.628 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.628 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.629 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "500", 2024-04-24T23:32:02.629 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "400", 2024-04-24T23:32:02.629 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "599", 2024-04-24T23:32:02.629 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.629 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.629 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.629 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.629 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.629 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:02.630 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:02.630 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.630 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-24T23:32:02.630 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "rbd_support", 2024-04-24T23:32:02.630 INFO:teuthology.orchestra.run.smithi080.stdout: "can_run": true, 2024-04-24T23:32:02.630 INFO:teuthology.orchestra.run.smithi080.stdout: "error_string": "", 2024-04-24T23:32:02.630 INFO:teuthology.orchestra.run.smithi080.stdout: "module_options": { 2024-04-24T23:32:02.630 INFO:teuthology.orchestra.run.smithi080.stdout: "log_level": { 2024-04-24T23:32:02.631 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_level", 2024-04-24T23:32:02.631 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.631 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.631 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.631 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:02.631 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.631 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.631 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:02.631 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:02.632 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:02.632 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:02.632 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:02.632 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:02.632 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:02.632 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:02.632 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.632 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.632 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.633 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.633 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.633 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster": { 2024-04-24T23:32:02.633 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster", 2024-04-24T23:32:02.633 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.633 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.633 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.633 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:02.634 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.634 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.634 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.634 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.634 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.634 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.634 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.634 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.634 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster_level": { 2024-04-24T23:32:02.635 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster_level", 2024-04-24T23:32:02.635 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.635 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.635 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.635 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "info", 2024-04-24T23:32:02.635 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.635 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.635 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:02.635 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:02.636 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:02.636 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:02.636 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:02.636 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:02.636 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:02.636 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:02.636 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.636 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.637 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.637 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.637 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.637 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_file": { 2024-04-24T23:32:02.637 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_file", 2024-04-24T23:32:02.637 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.637 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.637 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.637 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:02.638 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.638 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.638 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.638 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.638 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.638 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.638 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.638 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.638 INFO:teuthology.orchestra.run.smithi080.stdout: "max_concurrent_snap_create": { 2024-04-24T23:32:02.639 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "max_concurrent_snap_create", 2024-04-24T23:32:02.639 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "int", 2024-04-24T23:32:02.639 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.639 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.639 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "10", 2024-04-24T23:32:02.639 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.639 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.639 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.639 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.640 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.640 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.640 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.640 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.640 INFO:teuthology.orchestra.run.smithi080.stdout: "mirror_snapshot_schedule": { 2024-04-24T23:32:02.640 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "mirror_snapshot_schedule", 2024-04-24T23:32:02.640 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.640 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.641 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.641 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:02.641 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.641 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.641 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.641 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.641 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.641 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.641 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.642 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.642 INFO:teuthology.orchestra.run.smithi080.stdout: "trash_purge_schedule": { 2024-04-24T23:32:02.642 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "trash_purge_schedule", 2024-04-24T23:32:02.642 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.642 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.642 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.642 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:02.642 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.642 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.643 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.643 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.643 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.643 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.643 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.643 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:02.643 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:02.643 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.643 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-24T23:32:02.644 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "restful", 2024-04-24T23:32:02.644 INFO:teuthology.orchestra.run.smithi080.stdout: "can_run": true, 2024-04-24T23:32:02.644 INFO:teuthology.orchestra.run.smithi080.stdout: "error_string": "", 2024-04-24T23:32:02.644 INFO:teuthology.orchestra.run.smithi080.stdout: "module_options": { 2024-04-24T23:32:02.644 INFO:teuthology.orchestra.run.smithi080.stdout: "enable_auth": { 2024-04-24T23:32:02.644 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "enable_auth", 2024-04-24T23:32:02.644 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.644 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.645 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.645 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "True", 2024-04-24T23:32:02.645 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.645 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.645 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.645 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.645 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.645 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.645 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.646 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.646 INFO:teuthology.orchestra.run.smithi080.stdout: "key_file": { 2024-04-24T23:32:02.646 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "key_file", 2024-04-24T23:32:02.646 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.646 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.646 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.646 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:02.646 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.646 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.647 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.647 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.647 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.647 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.647 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.647 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.647 INFO:teuthology.orchestra.run.smithi080.stdout: "log_level": { 2024-04-24T23:32:02.647 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_level", 2024-04-24T23:32:02.647 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.648 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.648 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.648 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:02.648 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.648 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.648 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:02.648 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:02.648 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:02.648 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:02.649 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:02.649 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:02.649 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:02.649 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:02.649 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.649 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.649 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.649 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.650 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.650 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster": { 2024-04-24T23:32:02.650 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster", 2024-04-24T23:32:02.650 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.650 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.650 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.650 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:02.650 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.650 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.651 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.651 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.651 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.651 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.651 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.651 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.651 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster_level": { 2024-04-24T23:32:02.651 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster_level", 2024-04-24T23:32:02.651 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.652 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.652 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.652 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "info", 2024-04-24T23:32:02.652 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.652 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.652 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:02.652 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:02.652 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:02.653 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:02.653 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:02.653 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:02.653 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:02.653 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:02.653 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.653 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.653 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.653 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.654 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.654 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_file": { 2024-04-24T23:32:02.654 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_file", 2024-04-24T23:32:02.654 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.654 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.654 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.654 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:02.654 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.654 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.655 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.655 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.655 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.655 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.655 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.655 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.655 INFO:teuthology.orchestra.run.smithi080.stdout: "server_addr": { 2024-04-24T23:32:02.655 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "server_addr", 2024-04-24T23:32:02.655 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.656 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.656 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.656 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:02.656 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.656 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.656 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.656 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.656 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.657 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.657 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.657 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.657 INFO:teuthology.orchestra.run.smithi080.stdout: "server_port": { 2024-04-24T23:32:02.657 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "server_port", 2024-04-24T23:32:02.657 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.657 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.657 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.657 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:02.658 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.658 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.658 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.658 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.658 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.658 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.658 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.658 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:02.658 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:02.659 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.659 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-24T23:32:02.659 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "rgw", 2024-04-24T23:32:02.659 INFO:teuthology.orchestra.run.smithi080.stdout: "can_run": true, 2024-04-24T23:32:02.659 INFO:teuthology.orchestra.run.smithi080.stdout: "error_string": "", 2024-04-24T23:32:02.659 INFO:teuthology.orchestra.run.smithi080.stdout: "module_options": { 2024-04-24T23:32:02.659 INFO:teuthology.orchestra.run.smithi080.stdout: "log_level": { 2024-04-24T23:32:02.659 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_level", 2024-04-24T23:32:02.660 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.660 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.660 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.660 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:02.660 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.660 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.660 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:02.660 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:02.660 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:02.661 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:02.661 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:02.661 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:02.661 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:02.661 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:02.661 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.661 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.661 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.662 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.662 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.662 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster": { 2024-04-24T23:32:02.662 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster", 2024-04-24T23:32:02.662 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.662 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.662 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.662 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:02.662 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.663 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.663 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.663 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.663 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.663 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.663 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.663 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.663 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster_level": { 2024-04-24T23:32:02.664 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster_level", 2024-04-24T23:32:02.664 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.664 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.664 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.664 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "info", 2024-04-24T23:32:02.664 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.664 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.664 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:02.664 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:02.665 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:02.665 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:02.665 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:02.665 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:02.665 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:02.665 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:02.665 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.665 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.665 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.666 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.666 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.666 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_file": { 2024-04-24T23:32:02.666 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_file", 2024-04-24T23:32:02.666 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.666 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.666 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.666 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:02.667 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.667 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.667 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.667 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.667 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.667 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.667 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.667 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:02.667 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:02.668 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.668 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-24T23:32:02.668 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "rook", 2024-04-24T23:32:02.668 INFO:teuthology.orchestra.run.smithi080.stdout: "can_run": true, 2024-04-24T23:32:02.668 INFO:teuthology.orchestra.run.smithi080.stdout: "error_string": "", 2024-04-24T23:32:02.668 INFO:teuthology.orchestra.run.smithi080.stdout: "module_options": { 2024-04-24T23:32:02.668 INFO:teuthology.orchestra.run.smithi080.stdout: "log_level": { 2024-04-24T23:32:02.668 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_level", 2024-04-24T23:32:02.668 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.669 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.669 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.669 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:02.669 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.669 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.669 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:02.669 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:02.669 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:02.670 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:02.670 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:02.670 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:02.670 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:02.670 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:02.670 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.670 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.670 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.670 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.671 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.671 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster": { 2024-04-24T23:32:02.671 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster", 2024-04-24T23:32:02.671 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.671 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.671 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.671 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:02.671 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.672 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.672 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.672 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.672 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.672 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.672 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.672 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.672 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster_level": { 2024-04-24T23:32:02.672 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster_level", 2024-04-24T23:32:02.673 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.673 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.673 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.673 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "info", 2024-04-24T23:32:02.673 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.673 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.673 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:02.673 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:02.673 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:02.674 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:02.674 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:02.674 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:02.674 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:02.674 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:02.674 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.674 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.674 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.675 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.675 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.675 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_file": { 2024-04-24T23:32:02.675 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_file", 2024-04-24T23:32:02.675 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.675 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.675 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.675 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:02.675 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.676 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.676 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.676 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.676 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.676 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.676 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.676 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.676 INFO:teuthology.orchestra.run.smithi080.stdout: "storage_class": { 2024-04-24T23:32:02.676 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "storage_class", 2024-04-24T23:32:02.677 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.677 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.677 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.677 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "local", 2024-04-24T23:32:02.677 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.677 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.677 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.677 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-24T23:32:02.677 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.678 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.678 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.678 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:02.678 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:02.678 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.678 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-24T23:32:02.678 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "selftest", 2024-04-24T23:32:02.678 INFO:teuthology.orchestra.run.smithi080.stdout: "can_run": true, 2024-04-24T23:32:02.679 INFO:teuthology.orchestra.run.smithi080.stdout: "error_string": "", 2024-04-24T23:32:02.679 INFO:teuthology.orchestra.run.smithi080.stdout: "module_options": { 2024-04-24T23:32:02.679 INFO:teuthology.orchestra.run.smithi080.stdout: "log_level": { 2024-04-24T23:32:02.679 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_level", 2024-04-24T23:32:02.679 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.679 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.679 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.679 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:02.679 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.680 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.680 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:02.680 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:02.680 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:02.680 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:02.680 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:02.680 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:02.680 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:02.680 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:02.681 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.681 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.681 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.681 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.681 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.681 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster": { 2024-04-24T23:32:02.681 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster", 2024-04-24T23:32:02.681 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.681 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.682 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.682 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:02.682 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.682 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.682 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.682 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.682 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.682 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.683 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.683 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.683 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster_level": { 2024-04-24T23:32:02.683 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster_level", 2024-04-24T23:32:02.683 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.683 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.683 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.683 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "info", 2024-04-24T23:32:02.683 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.684 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.684 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:02.684 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:02.684 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:02.684 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:02.684 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:02.684 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:02.684 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:02.684 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:02.685 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.685 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.685 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.685 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.685 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.685 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_file": { 2024-04-24T23:32:02.685 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_file", 2024-04-24T23:32:02.685 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.686 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.686 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.686 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:02.686 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.686 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.686 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.686 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.686 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.686 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.687 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.687 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.687 INFO:teuthology.orchestra.run.smithi080.stdout: "roption1": { 2024-04-24T23:32:02.687 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "roption1", 2024-04-24T23:32:02.687 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.687 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.687 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.687 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:02.688 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.688 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.688 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.688 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.688 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.688 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.688 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.688 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.688 INFO:teuthology.orchestra.run.smithi080.stdout: "roption2": { 2024-04-24T23:32:02.689 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "roption2", 2024-04-24T23:32:02.689 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.689 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.689 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.689 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "xyz", 2024-04-24T23:32:02.689 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.689 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.689 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.689 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.690 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.690 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.690 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.690 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.690 INFO:teuthology.orchestra.run.smithi080.stdout: "rwoption1": { 2024-04-24T23:32:02.690 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "rwoption1", 2024-04-24T23:32:02.690 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.690 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.691 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.691 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:02.691 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.691 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.691 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.691 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.691 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.691 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.691 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.692 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.692 INFO:teuthology.orchestra.run.smithi080.stdout: "rwoption2": { 2024-04-24T23:32:02.692 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "rwoption2", 2024-04-24T23:32:02.692 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "int", 2024-04-24T23:32:02.692 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.692 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.692 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:02.692 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.692 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.693 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.693 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.693 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.693 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.693 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.693 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.693 INFO:teuthology.orchestra.run.smithi080.stdout: "rwoption3": { 2024-04-24T23:32:02.693 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "rwoption3", 2024-04-24T23:32:02.694 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "float", 2024-04-24T23:32:02.694 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.694 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.694 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:02.694 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.694 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.694 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.694 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.694 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.695 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.695 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.695 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.695 INFO:teuthology.orchestra.run.smithi080.stdout: "rwoption4": { 2024-04-24T23:32:02.695 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "rwoption4", 2024-04-24T23:32:02.695 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.695 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.695 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.695 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:02.696 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.696 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.696 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.696 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.696 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.696 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.696 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.696 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.697 INFO:teuthology.orchestra.run.smithi080.stdout: "rwoption5": { 2024-04-24T23:32:02.697 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "rwoption5", 2024-04-24T23:32:02.697 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.697 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.697 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.697 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:02.697 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.697 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.697 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.698 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.698 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.698 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.698 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.698 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.698 INFO:teuthology.orchestra.run.smithi080.stdout: "rwoption6": { 2024-04-24T23:32:02.698 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "rwoption6", 2024-04-24T23:32:02.698 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.698 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.699 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.699 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "True", 2024-04-24T23:32:02.699 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.699 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.699 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.699 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.699 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.699 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.700 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.700 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.700 INFO:teuthology.orchestra.run.smithi080.stdout: "rwoption7": { 2024-04-24T23:32:02.700 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "rwoption7", 2024-04-24T23:32:02.700 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "int", 2024-04-24T23:32:02.700 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.700 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.700 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:02.700 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "1", 2024-04-24T23:32:02.701 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "42", 2024-04-24T23:32:02.701 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.701 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.701 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.701 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.701 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.701 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.701 INFO:teuthology.orchestra.run.smithi080.stdout: "testkey": { 2024-04-24T23:32:02.701 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "testkey", 2024-04-24T23:32:02.702 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.702 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.702 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.702 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:02.702 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.702 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.702 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.702 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.702 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.703 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.703 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.703 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.703 INFO:teuthology.orchestra.run.smithi080.stdout: "testlkey": { 2024-04-24T23:32:02.703 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "testlkey", 2024-04-24T23:32:02.703 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.703 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.703 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.704 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:02.704 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.704 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.704 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.704 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.704 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.704 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.704 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.704 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.705 INFO:teuthology.orchestra.run.smithi080.stdout: "testnewline": { 2024-04-24T23:32:02.705 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "testnewline", 2024-04-24T23:32:02.705 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.705 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.705 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.705 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:02.705 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.705 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.705 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.706 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.706 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.706 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.706 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.706 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:02.706 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:02.706 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.706 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-24T23:32:02.707 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "snap_schedule", 2024-04-24T23:32:02.707 INFO:teuthology.orchestra.run.smithi080.stdout: "can_run": true, 2024-04-24T23:32:02.707 INFO:teuthology.orchestra.run.smithi080.stdout: "error_string": "", 2024-04-24T23:32:02.707 INFO:teuthology.orchestra.run.smithi080.stdout: "module_options": { 2024-04-24T23:32:02.707 INFO:teuthology.orchestra.run.smithi080.stdout: "allow_m_granularity": { 2024-04-24T23:32:02.707 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "allow_m_granularity", 2024-04-24T23:32:02.707 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.707 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.707 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.708 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:02.708 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.708 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.708 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.708 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "allow minute scheduled snapshots", 2024-04-24T23:32:02.708 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.708 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.708 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.708 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.709 INFO:teuthology.orchestra.run.smithi080.stdout: "dump_on_update": { 2024-04-24T23:32:02.709 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "dump_on_update", 2024-04-24T23:32:02.709 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.709 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.709 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.709 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:02.709 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.709 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.709 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.710 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "dump database to debug log on update", 2024-04-24T23:32:02.710 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.710 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.710 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.710 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.710 INFO:teuthology.orchestra.run.smithi080.stdout: "log_level": { 2024-04-24T23:32:02.710 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_level", 2024-04-24T23:32:02.710 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.711 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.711 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.711 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:02.711 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.711 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.711 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:02.711 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:02.711 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:02.711 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:02.712 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:02.712 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:02.712 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:02.712 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:02.712 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.712 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.712 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.712 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.712 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.713 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster": { 2024-04-24T23:32:02.713 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster", 2024-04-24T23:32:02.713 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.713 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.713 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.713 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:02.713 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.713 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.713 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.714 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.714 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.714 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.714 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.714 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.714 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster_level": { 2024-04-24T23:32:02.714 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster_level", 2024-04-24T23:32:02.714 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.715 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.715 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.715 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "info", 2024-04-24T23:32:02.715 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.715 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.715 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:02.715 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:02.715 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:02.715 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:02.716 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:02.716 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:02.716 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:02.716 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:02.716 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.716 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.716 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.716 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.717 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.717 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_file": { 2024-04-24T23:32:02.717 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_file", 2024-04-24T23:32:02.717 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.717 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.717 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.717 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:02.717 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.717 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.718 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.718 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.718 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.718 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.718 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.718 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:02.718 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:02.718 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.719 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-24T23:32:02.719 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "stats", 2024-04-24T23:32:02.719 INFO:teuthology.orchestra.run.smithi080.stdout: "can_run": true, 2024-04-24T23:32:02.719 INFO:teuthology.orchestra.run.smithi080.stdout: "error_string": "", 2024-04-24T23:32:02.719 INFO:teuthology.orchestra.run.smithi080.stdout: "module_options": { 2024-04-24T23:32:02.719 INFO:teuthology.orchestra.run.smithi080.stdout: "log_level": { 2024-04-24T23:32:02.719 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_level", 2024-04-24T23:32:02.719 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.719 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.720 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.720 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:02.720 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.720 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.720 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:02.720 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:02.720 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:02.720 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:02.720 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:02.721 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:02.721 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:02.721 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:02.721 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.721 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.721 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.721 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.721 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.722 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster": { 2024-04-24T23:32:02.722 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster", 2024-04-24T23:32:02.722 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.722 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.722 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.722 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:02.722 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.722 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.722 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.723 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.723 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.723 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.723 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.723 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.723 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster_level": { 2024-04-24T23:32:02.723 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster_level", 2024-04-24T23:32:02.723 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.724 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.724 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.724 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "info", 2024-04-24T23:32:02.724 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.724 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.724 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:02.724 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:02.724 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:02.724 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:02.725 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:02.725 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:02.725 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:02.725 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:02.725 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.725 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.725 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.725 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.725 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.726 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_file": { 2024-04-24T23:32:02.726 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_file", 2024-04-24T23:32:02.726 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.726 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.726 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.726 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:02.726 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.726 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.726 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.727 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.727 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.727 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.727 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.727 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:02.727 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:02.727 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.727 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-24T23:32:02.728 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "status", 2024-04-24T23:32:02.728 INFO:teuthology.orchestra.run.smithi080.stdout: "can_run": true, 2024-04-24T23:32:02.728 INFO:teuthology.orchestra.run.smithi080.stdout: "error_string": "", 2024-04-24T23:32:02.728 INFO:teuthology.orchestra.run.smithi080.stdout: "module_options": { 2024-04-24T23:32:02.728 INFO:teuthology.orchestra.run.smithi080.stdout: "log_level": { 2024-04-24T23:32:02.728 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_level", 2024-04-24T23:32:02.728 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.728 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.728 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.729 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:02.729 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.729 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.729 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:02.729 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:02.729 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:02.729 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:02.729 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:02.729 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:02.730 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:02.730 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:02.730 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.730 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.730 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.730 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.730 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.730 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster": { 2024-04-24T23:32:02.731 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster", 2024-04-24T23:32:02.731 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.731 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.731 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.731 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:02.731 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.731 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.731 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.731 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.732 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.732 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.732 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.732 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.732 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster_level": { 2024-04-24T23:32:02.732 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster_level", 2024-04-24T23:32:02.732 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.732 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.732 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.733 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "info", 2024-04-24T23:32:02.733 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.733 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.733 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:02.733 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:02.733 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:02.733 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:02.733 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:02.733 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:02.734 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:02.734 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:02.734 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.734 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.734 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.734 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.734 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.734 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_file": { 2024-04-24T23:32:02.735 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_file", 2024-04-24T23:32:02.735 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.735 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.735 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.735 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:02.735 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.735 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.735 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.735 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.736 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.736 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.736 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.736 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:02.736 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:02.736 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.736 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-24T23:32:02.736 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "telegraf", 2024-04-24T23:32:02.736 INFO:teuthology.orchestra.run.smithi080.stdout: "can_run": true, 2024-04-24T23:32:02.737 INFO:teuthology.orchestra.run.smithi080.stdout: "error_string": "", 2024-04-24T23:32:02.737 INFO:teuthology.orchestra.run.smithi080.stdout: "module_options": { 2024-04-24T23:32:02.737 INFO:teuthology.orchestra.run.smithi080.stdout: "address": { 2024-04-24T23:32:02.737 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "address", 2024-04-24T23:32:02.737 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.737 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.737 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.737 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-24T23:32:02.738 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.738 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.738 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.738 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.738 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.738 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.738 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.738 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.738 INFO:teuthology.orchestra.run.smithi080.stdout: "interval": { 2024-04-24T23:32:02.739 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "interval", 2024-04-24T23:32:02.739 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "secs", 2024-04-24T23:32:02.739 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.739 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.739 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "15", 2024-04-24T23:32:02.739 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.739 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.739 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.740 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.740 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.740 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.740 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.740 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.740 INFO:teuthology.orchestra.run.smithi080.stdout: "log_level": { 2024-04-24T23:32:02.740 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_level", 2024-04-24T23:32:02.740 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.740 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.741 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.741 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:02.741 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.741 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.741 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:02.741 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:02.741 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:02.741 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:02.741 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:02.742 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:02.742 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:02.742 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:02.742 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.742 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.742 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.742 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.742 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.743 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster": { 2024-04-24T23:32:02.743 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster", 2024-04-24T23:32:02.743 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.743 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.743 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.743 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:02.743 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.743 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.744 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.744 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.744 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.744 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.744 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.744 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.744 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster_level": { 2024-04-24T23:32:02.744 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster_level", 2024-04-24T23:32:02.744 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.745 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.745 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.745 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "info", 2024-04-24T23:32:02.745 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.745 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.745 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:02.745 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:02.745 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:02.746 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:02.746 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:02.746 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:02.746 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:02.746 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:02.746 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.746 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.746 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.746 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.747 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.747 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_file": { 2024-04-24T23:32:02.747 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_file", 2024-04-24T23:32:02.747 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.747 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.747 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.747 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:02.747 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.747 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.748 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.748 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.748 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.748 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.748 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.748 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:02.748 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:02.748 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.749 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-24T23:32:02.749 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "telemetry", 2024-04-24T23:32:02.749 INFO:teuthology.orchestra.run.smithi080.stdout: "can_run": true, 2024-04-24T23:32:02.749 INFO:teuthology.orchestra.run.smithi080.stdout: "error_string": "", 2024-04-24T23:32:02.749 INFO:teuthology.orchestra.run.smithi080.stdout: "module_options": { 2024-04-24T23:32:02.749 INFO:teuthology.orchestra.run.smithi080.stdout: "channel_basic": { 2024-04-24T23:32:02.749 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "channel_basic", 2024-04-24T23:32:02.749 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.749 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.750 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.750 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "True", 2024-04-24T23:32:02.750 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.750 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.750 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.750 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-24T23:32:02.750 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.750 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.750 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.751 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.751 INFO:teuthology.orchestra.run.smithi080.stdout: "channel_crash": { 2024-04-24T23:32:02.751 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "channel_crash", 2024-04-24T23:32:02.751 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.751 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.751 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.751 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "True", 2024-04-24T23:32:02.751 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.752 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.752 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.752 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-24T23:32:02.752 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.752 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.752 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.752 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.752 INFO:teuthology.orchestra.run.smithi080.stdout: "channel_device": { 2024-04-24T23:32:02.752 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "channel_device", 2024-04-24T23:32:02.753 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.753 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.753 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.753 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "True", 2024-04-24T23:32:02.753 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.753 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.753 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.753 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-24T23:32:02.753 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.754 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.754 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.754 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.754 INFO:teuthology.orchestra.run.smithi080.stdout: "channel_ident": { 2024-04-24T23:32:02.754 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "channel_ident", 2024-04-24T23:32:02.754 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.754 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.754 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.755 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:02.755 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.755 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.755 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.755 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-24T23:32:02.755 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.755 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.755 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.755 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.756 INFO:teuthology.orchestra.run.smithi080.stdout: "channel_perf": { 2024-04-24T23:32:02.756 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "channel_perf", 2024-04-24T23:32:02.756 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.756 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.756 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.756 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:02.756 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.756 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.757 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.757 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-24T23:32:02.757 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.757 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.757 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.757 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.757 INFO:teuthology.orchestra.run.smithi080.stdout: "contact": { 2024-04-24T23:32:02.757 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "contact", 2024-04-24T23:32:02.757 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.758 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.758 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.758 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:02.758 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.758 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.758 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.758 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.758 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.759 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.759 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.759 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.759 INFO:teuthology.orchestra.run.smithi080.stdout: "description": { 2024-04-24T23:32:02.759 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "description", 2024-04-24T23:32:02.759 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.759 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.759 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.759 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:02.760 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.760 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.760 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.760 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.760 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.760 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.760 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.760 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.760 INFO:teuthology.orchestra.run.smithi080.stdout: "device_url": { 2024-04-24T23:32:02.761 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "device_url", 2024-04-24T23:32:02.761 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.761 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.761 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.761 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-24T23:32:02.761 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.761 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.761 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.761 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.762 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.762 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.762 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.762 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.762 INFO:teuthology.orchestra.run.smithi080.stdout: "enabled": { 2024-04-24T23:32:02.762 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "enabled", 2024-04-24T23:32:02.762 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.762 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.763 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.763 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:02.763 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.763 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.763 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.763 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.763 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.763 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.763 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.764 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.764 INFO:teuthology.orchestra.run.smithi080.stdout: "interval": { 2024-04-24T23:32:02.764 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "interval", 2024-04-24T23:32:02.764 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "int", 2024-04-24T23:32:02.764 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.764 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.764 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "24", 2024-04-24T23:32:02.764 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "8", 2024-04-24T23:32:02.764 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.765 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.765 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.765 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.765 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.765 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.765 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.765 INFO:teuthology.orchestra.run.smithi080.stdout: "last_opt_revision": { 2024-04-24T23:32:02.765 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "last_opt_revision", 2024-04-24T23:32:02.766 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "int", 2024-04-24T23:32:02.766 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.766 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.766 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "1", 2024-04-24T23:32:02.766 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.766 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.766 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.766 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.766 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.767 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.767 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.767 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.767 INFO:teuthology.orchestra.run.smithi080.stdout: "leaderboard": { 2024-04-24T23:32:02.767 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "leaderboard", 2024-04-24T23:32:02.767 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.767 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.767 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.767 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:02.768 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.768 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.768 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.768 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.768 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.768 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.768 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.768 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.769 INFO:teuthology.orchestra.run.smithi080.stdout: "leaderboard_description": { 2024-04-24T23:32:02.769 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "leaderboard_description", 2024-04-24T23:32:02.769 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.769 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.769 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.769 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:02.769 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.769 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.769 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.770 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.770 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.770 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.770 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.770 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.770 INFO:teuthology.orchestra.run.smithi080.stdout: "log_level": { 2024-04-24T23:32:02.770 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_level", 2024-04-24T23:32:02.770 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.771 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.771 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.771 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:02.771 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.771 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.771 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:02.771 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:02.771 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:02.771 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:02.772 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:02.772 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:02.772 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:02.772 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:02.772 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.772 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.772 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.772 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.773 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.773 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster": { 2024-04-24T23:32:02.773 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster", 2024-04-24T23:32:02.773 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.773 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.773 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.773 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:02.773 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.773 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.774 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.774 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.774 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.774 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.774 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.774 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.774 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster_level": { 2024-04-24T23:32:02.774 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster_level", 2024-04-24T23:32:02.774 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.775 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.775 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.775 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "info", 2024-04-24T23:32:02.775 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.775 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.775 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:02.775 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:02.775 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:02.776 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:02.776 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:02.776 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:02.776 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:02.776 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:02.776 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.776 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.776 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.776 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.777 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.777 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_file": { 2024-04-24T23:32:02.777 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_file", 2024-04-24T23:32:02.777 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.777 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.777 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.777 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:02.777 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.778 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.778 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.778 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.778 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.778 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.778 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.778 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.778 INFO:teuthology.orchestra.run.smithi080.stdout: "organization": { 2024-04-24T23:32:02.778 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "organization", 2024-04-24T23:32:02.779 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.779 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.779 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.779 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:02.779 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.779 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.779 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.779 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.780 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.780 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.780 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.780 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.780 INFO:teuthology.orchestra.run.smithi080.stdout: "proxy": { 2024-04-24T23:32:02.780 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "proxy", 2024-04-24T23:32:02.780 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.780 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.780 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.781 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:02.781 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.781 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.781 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.781 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.781 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.781 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.781 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.781 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.782 INFO:teuthology.orchestra.run.smithi080.stdout: "url": { 2024-04-24T23:32:02.782 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "url", 2024-04-24T23:32:02.782 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.782 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.782 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.782 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-24T23:32:02.782 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.782 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.783 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.783 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.783 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.783 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.783 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.783 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:02.783 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:02.783 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.783 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-24T23:32:02.784 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "test_orchestrator", 2024-04-24T23:32:02.784 INFO:teuthology.orchestra.run.smithi080.stdout: "can_run": true, 2024-04-24T23:32:02.784 INFO:teuthology.orchestra.run.smithi080.stdout: "error_string": "", 2024-04-24T23:32:02.784 INFO:teuthology.orchestra.run.smithi080.stdout: "module_options": { 2024-04-24T23:32:02.784 INFO:teuthology.orchestra.run.smithi080.stdout: "log_level": { 2024-04-24T23:32:02.784 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_level", 2024-04-24T23:32:02.784 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.784 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.784 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.785 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:02.785 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.785 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.785 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:02.785 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:02.785 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:02.785 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:02.785 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:02.786 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:02.786 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:02.786 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:02.786 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.786 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.786 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.786 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.786 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.786 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster": { 2024-04-24T23:32:02.787 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster", 2024-04-24T23:32:02.787 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.787 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.787 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.787 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:02.787 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.787 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.787 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.787 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.788 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.788 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.788 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.788 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.788 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster_level": { 2024-04-24T23:32:02.788 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster_level", 2024-04-24T23:32:02.788 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.788 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.789 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.789 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "info", 2024-04-24T23:32:02.789 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.789 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.789 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:02.789 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:02.789 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:02.789 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:02.789 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:02.790 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:02.790 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:02.790 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:02.790 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.790 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.790 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.790 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.790 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.790 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_file": { 2024-04-24T23:32:02.791 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_file", 2024-04-24T23:32:02.791 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.791 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.791 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.791 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:02.791 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.791 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.791 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.792 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.792 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.792 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.792 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.792 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:02.792 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:02.792 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.792 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-24T23:32:02.792 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "volumes", 2024-04-24T23:32:02.793 INFO:teuthology.orchestra.run.smithi080.stdout: "can_run": true, 2024-04-24T23:32:02.793 INFO:teuthology.orchestra.run.smithi080.stdout: "error_string": "", 2024-04-24T23:32:02.793 INFO:teuthology.orchestra.run.smithi080.stdout: "module_options": { 2024-04-24T23:32:02.793 INFO:teuthology.orchestra.run.smithi080.stdout: "log_level": { 2024-04-24T23:32:02.793 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_level", 2024-04-24T23:32:02.793 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.793 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.793 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.793 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:02.794 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.794 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.794 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:02.794 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:02.794 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:02.794 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:02.794 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:02.794 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:02.795 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:02.795 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:02.795 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.795 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.795 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.795 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.797 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.797 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster": { 2024-04-24T23:32:02.798 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster", 2024-04-24T23:32:02.798 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.798 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.798 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.798 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:02.798 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.798 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.798 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.799 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.799 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.799 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.799 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.799 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.799 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster_level": { 2024-04-24T23:32:02.799 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster_level", 2024-04-24T23:32:02.799 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.799 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.800 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.800 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "info", 2024-04-24T23:32:02.800 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.800 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.800 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:02.800 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:02.800 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:02.800 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:02.801 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:02.801 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:02.801 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:02.801 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:02.801 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.801 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.801 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.801 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.801 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.802 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_file": { 2024-04-24T23:32:02.802 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_file", 2024-04-24T23:32:02.802 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.802 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.802 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.802 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:02.802 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.802 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.803 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.803 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.803 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.803 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.803 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.803 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.803 INFO:teuthology.orchestra.run.smithi080.stdout: "max_concurrent_clones": { 2024-04-24T23:32:02.803 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "max_concurrent_clones", 2024-04-24T23:32:02.803 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "int", 2024-04-24T23:32:02.804 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.804 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.804 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "4", 2024-04-24T23:32:02.804 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.804 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.804 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.804 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-24T23:32:02.804 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.805 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.805 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.805 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.805 INFO:teuthology.orchestra.run.smithi080.stdout: "periodic_async_work": { 2024-04-24T23:32:02.805 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "periodic_async_work", 2024-04-24T23:32:02.805 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.805 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.805 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.805 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:02.806 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.806 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.806 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.806 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Periodically check for async work", 2024-04-24T23:32:02.806 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.806 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.806 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.806 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.807 INFO:teuthology.orchestra.run.smithi080.stdout: "snapshot_clone_delay": { 2024-04-24T23:32:02.807 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "snapshot_clone_delay", 2024-04-24T23:32:02.807 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "int", 2024-04-24T23:32:02.807 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.807 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.807 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "0", 2024-04-24T23:32:02.807 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.807 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.807 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.808 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-24T23:32:02.808 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.808 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.808 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.808 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.808 INFO:teuthology.orchestra.run.smithi080.stdout: "snapshot_clone_no_wait": { 2024-04-24T23:32:02.808 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "snapshot_clone_no_wait", 2024-04-24T23:32:02.808 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.808 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.809 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.809 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "True", 2024-04-24T23:32:02.809 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.809 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.809 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.809 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-24T23:32:02.809 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.809 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.809 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.810 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:02.810 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:02.810 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.810 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-24T23:32:02.810 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "zabbix", 2024-04-24T23:32:02.810 INFO:teuthology.orchestra.run.smithi080.stdout: "can_run": true, 2024-04-24T23:32:02.810 INFO:teuthology.orchestra.run.smithi080.stdout: "error_string": "", 2024-04-24T23:32:02.810 INFO:teuthology.orchestra.run.smithi080.stdout: "module_options": { 2024-04-24T23:32:02.811 INFO:teuthology.orchestra.run.smithi080.stdout: "discovery_interval": { 2024-04-24T23:32:02.811 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "discovery_interval", 2024-04-24T23:32:02.811 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "uint", 2024-04-24T23:32:02.811 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.811 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.811 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "100", 2024-04-24T23:32:02.811 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.811 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.811 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.812 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.812 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.812 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.812 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.812 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.812 INFO:teuthology.orchestra.run.smithi080.stdout: "identifier": { 2024-04-24T23:32:02.812 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "identifier", 2024-04-24T23:32:02.812 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.812 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.813 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.813 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:02.813 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.813 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.813 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.813 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.813 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.813 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.814 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.814 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.814 INFO:teuthology.orchestra.run.smithi080.stdout: "interval": { 2024-04-24T23:32:02.814 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "interval", 2024-04-24T23:32:02.814 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "secs", 2024-04-24T23:32:02.814 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.814 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.814 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "60", 2024-04-24T23:32:02.814 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.815 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.815 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.815 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.815 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.815 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.815 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.815 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.815 INFO:teuthology.orchestra.run.smithi080.stdout: "log_level": { 2024-04-24T23:32:02.815 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_level", 2024-04-24T23:32:02.816 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.816 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.816 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.816 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:02.816 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.816 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.816 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:02.816 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:02.817 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:02.817 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:02.817 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:02.817 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:02.817 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:02.817 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:02.817 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.817 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.817 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.818 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.818 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.818 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster": { 2024-04-24T23:32:02.818 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster", 2024-04-24T23:32:02.818 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.818 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.818 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.818 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:02.819 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.819 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.819 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.819 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.819 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.819 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.819 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.819 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.819 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_cluster_level": { 2024-04-24T23:32:02.820 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_cluster_level", 2024-04-24T23:32:02.820 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.820 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.820 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.820 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "info", 2024-04-24T23:32:02.820 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.820 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.820 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [ 2024-04-24T23:32:02.820 INFO:teuthology.orchestra.run.smithi080.stdout: "", 2024-04-24T23:32:02.821 INFO:teuthology.orchestra.run.smithi080.stdout: "critical", 2024-04-24T23:32:02.821 INFO:teuthology.orchestra.run.smithi080.stdout: "debug", 2024-04-24T23:32:02.821 INFO:teuthology.orchestra.run.smithi080.stdout: "error", 2024-04-24T23:32:02.821 INFO:teuthology.orchestra.run.smithi080.stdout: "info", 2024-04-24T23:32:02.821 INFO:teuthology.orchestra.run.smithi080.stdout: "warning" 2024-04-24T23:32:02.821 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:02.821 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.821 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.821 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.822 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.822 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.822 INFO:teuthology.orchestra.run.smithi080.stdout: "log_to_file": { 2024-04-24T23:32:02.822 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "log_to_file", 2024-04-24T23:32:02.822 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "bool", 2024-04-24T23:32:02.822 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.822 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-24T23:32:02.822 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "False", 2024-04-24T23:32:02.823 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.823 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.823 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.823 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.823 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.823 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.823 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.823 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.823 INFO:teuthology.orchestra.run.smithi080.stdout: "zabbix_host": { 2024-04-24T23:32:02.824 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "zabbix_host", 2024-04-24T23:32:02.824 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.824 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.824 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.824 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "", 2024-04-24T23:32:02.824 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.824 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.824 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.824 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.825 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.825 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.825 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.825 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.825 INFO:teuthology.orchestra.run.smithi080.stdout: "zabbix_port": { 2024-04-24T23:32:02.825 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "zabbix_port", 2024-04-24T23:32:02.825 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "int", 2024-04-24T23:32:02.825 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.826 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.826 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "10051", 2024-04-24T23:32:02.826 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.826 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.826 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.826 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.826 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.826 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.826 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.827 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.827 INFO:teuthology.orchestra.run.smithi080.stdout: "zabbix_sender": { 2024-04-24T23:32:02.827 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "zabbix_sender", 2024-04-24T23:32:02.827 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "str", 2024-04-24T23:32:02.827 INFO:teuthology.orchestra.run.smithi080.stdout: "level": "advanced", 2024-04-24T23:32:02.827 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 0, 2024-04-24T23:32:02.827 INFO:teuthology.orchestra.run.smithi080.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-24T23:32:02.827 INFO:teuthology.orchestra.run.smithi080.stdout: "min": "", 2024-04-24T23:32:02.828 INFO:teuthology.orchestra.run.smithi080.stdout: "max": "", 2024-04-24T23:32:02.828 INFO:teuthology.orchestra.run.smithi080.stdout: "enum_allowed": [], 2024-04-24T23:32:02.828 INFO:teuthology.orchestra.run.smithi080.stdout: "desc": "", 2024-04-24T23:32:02.828 INFO:teuthology.orchestra.run.smithi080.stdout: "long_desc": "", 2024-04-24T23:32:02.828 INFO:teuthology.orchestra.run.smithi080.stdout: "tags": [], 2024-04-24T23:32:02.828 INFO:teuthology.orchestra.run.smithi080.stdout: "see_also": [] 2024-04-24T23:32:02.828 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:02.828 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:02.828 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:02.829 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:02.829 INFO:teuthology.orchestra.run.smithi080.stdout: "services": { 2024-04-24T23:32:02.829 INFO:teuthology.orchestra.run.smithi080.stdout: "dashboard": "https://172.21.15.80:8443/" 2024-04-24T23:32:02.829 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.829 INFO:teuthology.orchestra.run.smithi080.stdout: "always_on_modules": { 2024-04-24T23:32:02.829 INFO:teuthology.orchestra.run.smithi080.stdout: "octopus": [ 2024-04-24T23:32:02.829 INFO:teuthology.orchestra.run.smithi080.stdout: "balancer", 2024-04-24T23:32:02.829 INFO:teuthology.orchestra.run.smithi080.stdout: "crash", 2024-04-24T23:32:02.830 INFO:teuthology.orchestra.run.smithi080.stdout: "devicehealth", 2024-04-24T23:32:02.830 INFO:teuthology.orchestra.run.smithi080.stdout: "orchestrator", 2024-04-24T23:32:02.830 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_autoscaler", 2024-04-24T23:32:02.830 INFO:teuthology.orchestra.run.smithi080.stdout: "progress", 2024-04-24T23:32:02.830 INFO:teuthology.orchestra.run.smithi080.stdout: "rbd_support", 2024-04-24T23:32:02.830 INFO:teuthology.orchestra.run.smithi080.stdout: "status", 2024-04-24T23:32:02.830 INFO:teuthology.orchestra.run.smithi080.stdout: "telemetry", 2024-04-24T23:32:02.830 INFO:teuthology.orchestra.run.smithi080.stdout: "volumes" 2024-04-24T23:32:02.830 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:02.831 INFO:teuthology.orchestra.run.smithi080.stdout: "pacific": [ 2024-04-24T23:32:02.831 INFO:teuthology.orchestra.run.smithi080.stdout: "balancer", 2024-04-24T23:32:02.831 INFO:teuthology.orchestra.run.smithi080.stdout: "crash", 2024-04-24T23:32:02.831 INFO:teuthology.orchestra.run.smithi080.stdout: "devicehealth", 2024-04-24T23:32:02.831 INFO:teuthology.orchestra.run.smithi080.stdout: "orchestrator", 2024-04-24T23:32:02.831 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_autoscaler", 2024-04-24T23:32:02.831 INFO:teuthology.orchestra.run.smithi080.stdout: "progress", 2024-04-24T23:32:02.831 INFO:teuthology.orchestra.run.smithi080.stdout: "rbd_support", 2024-04-24T23:32:02.831 INFO:teuthology.orchestra.run.smithi080.stdout: "status", 2024-04-24T23:32:02.832 INFO:teuthology.orchestra.run.smithi080.stdout: "telemetry", 2024-04-24T23:32:02.832 INFO:teuthology.orchestra.run.smithi080.stdout: "volumes" 2024-04-24T23:32:02.832 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:02.832 INFO:teuthology.orchestra.run.smithi080.stdout: "quincy": [ 2024-04-24T23:32:02.832 INFO:teuthology.orchestra.run.smithi080.stdout: "balancer", 2024-04-24T23:32:02.832 INFO:teuthology.orchestra.run.smithi080.stdout: "crash", 2024-04-24T23:32:02.832 INFO:teuthology.orchestra.run.smithi080.stdout: "devicehealth", 2024-04-24T23:32:02.832 INFO:teuthology.orchestra.run.smithi080.stdout: "orchestrator", 2024-04-24T23:32:02.833 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_autoscaler", 2024-04-24T23:32:02.833 INFO:teuthology.orchestra.run.smithi080.stdout: "progress", 2024-04-24T23:32:02.833 INFO:teuthology.orchestra.run.smithi080.stdout: "rbd_support", 2024-04-24T23:32:02.833 INFO:teuthology.orchestra.run.smithi080.stdout: "status", 2024-04-24T23:32:02.833 INFO:teuthology.orchestra.run.smithi080.stdout: "telemetry", 2024-04-24T23:32:02.833 INFO:teuthology.orchestra.run.smithi080.stdout: "volumes" 2024-04-24T23:32:02.833 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:02.833 INFO:teuthology.orchestra.run.smithi080.stdout: "reef": [ 2024-04-24T23:32:02.833 INFO:teuthology.orchestra.run.smithi080.stdout: "balancer", 2024-04-24T23:32:02.834 INFO:teuthology.orchestra.run.smithi080.stdout: "crash", 2024-04-24T23:32:02.834 INFO:teuthology.orchestra.run.smithi080.stdout: "devicehealth", 2024-04-24T23:32:02.834 INFO:teuthology.orchestra.run.smithi080.stdout: "orchestrator", 2024-04-24T23:32:02.834 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_autoscaler", 2024-04-24T23:32:02.834 INFO:teuthology.orchestra.run.smithi080.stdout: "progress", 2024-04-24T23:32:02.834 INFO:teuthology.orchestra.run.smithi080.stdout: "rbd_support", 2024-04-24T23:32:02.834 INFO:teuthology.orchestra.run.smithi080.stdout: "status", 2024-04-24T23:32:02.834 INFO:teuthology.orchestra.run.smithi080.stdout: "telemetry", 2024-04-24T23:32:02.834 INFO:teuthology.orchestra.run.smithi080.stdout: "volumes" 2024-04-24T23:32:02.835 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-24T23:32:02.835 INFO:teuthology.orchestra.run.smithi080.stdout: "squid": [ 2024-04-24T23:32:02.835 INFO:teuthology.orchestra.run.smithi080.stdout: "balancer", 2024-04-24T23:32:02.835 INFO:teuthology.orchestra.run.smithi080.stdout: "crash", 2024-04-24T23:32:02.835 INFO:teuthology.orchestra.run.smithi080.stdout: "devicehealth", 2024-04-24T23:32:02.835 INFO:teuthology.orchestra.run.smithi080.stdout: "orchestrator", 2024-04-24T23:32:02.835 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_autoscaler", 2024-04-24T23:32:02.835 INFO:teuthology.orchestra.run.smithi080.stdout: "progress", 2024-04-24T23:32:02.835 INFO:teuthology.orchestra.run.smithi080.stdout: "rbd_support", 2024-04-24T23:32:02.836 INFO:teuthology.orchestra.run.smithi080.stdout: "status", 2024-04-24T23:32:02.836 INFO:teuthology.orchestra.run.smithi080.stdout: "telemetry", 2024-04-24T23:32:02.836 INFO:teuthology.orchestra.run.smithi080.stdout: "volumes" 2024-04-24T23:32:02.836 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-24T23:32:02.836 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.836 INFO:teuthology.orchestra.run.smithi080.stdout: "last_failure_osd_epoch": 3, 2024-04-24T23:32:02.836 INFO:teuthology.orchestra.run.smithi080.stdout: "active_clients": [ 2024-04-24T23:32:02.836 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-24T23:32:02.837 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "devicehealth", 2024-04-24T23:32:02.837 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-24T23:32:02.837 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-24T23:32:02.837 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-24T23:32:02.837 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:0", 2024-04-24T23:32:02.837 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 990423954 2024-04-24T23:32:02.837 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:02.837 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-24T23:32:02.837 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.838 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-24T23:32:02.838 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "libcephsqlite", 2024-04-24T23:32:02.838 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-24T23:32:02.838 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-24T23:32:02.838 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-24T23:32:02.838 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:0", 2024-04-24T23:32:02.838 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1833070360 2024-04-24T23:32:02.838 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:02.839 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-24T23:32:02.839 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.839 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-24T23:32:02.839 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "rbd_support", 2024-04-24T23:32:02.839 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-24T23:32:02.839 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-24T23:32:02.839 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-24T23:32:02.839 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:0", 2024-04-24T23:32:02.839 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3400278491 2024-04-24T23:32:02.840 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:02.840 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-24T23:32:02.840 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:02.840 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-24T23:32:02.840 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "volumes", 2024-04-24T23:32:02.840 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-24T23:32:02.840 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-24T23:32:02.840 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-24T23:32:02.840 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:0", 2024-04-24T23:32:02.841 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 63470302 2024-04-24T23:32:02.841 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:02.841 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-24T23:32:02.841 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:02.841 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-24T23:32:02.841 INFO:teuthology.orchestra.run.smithi080.stdout:} 2024-04-24T23:32:02.843 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:32:01 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/1180168202' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T23:32:02.843 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:32:01 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/2968146966' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-24T23:32:02.843 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:32:01 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:32:02.843 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:32:02 smithi148 ceph-mon[32528]: pgmap v441: 129 pgs: 129 active+clean; 69 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 295 KiB/s wr, 24 op/s 2024-04-24T23:32:02.843 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:32:02 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:32:02.843 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:32:02 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:32:02.844 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:32:02 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:32:02.844 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:32:01 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/1180168202' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T23:32:02.844 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:32:01 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/2968146966' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-24T23:32:02.844 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:32:01 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:32:02.844 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:32:02 smithi134 ceph-mon[32619]: pgmap v441: 129 pgs: 129 active+clean; 69 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 295 KiB/s wr, 24 op/s 2024-04-24T23:32:02.844 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:32:02 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:32:02.844 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:32:02 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:32:02.844 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:32:02 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:32:02.845 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:00.708+0000 7fedbb3a5640 1 -- 172.21.15.80:0/2968146966 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7feda00a2b90 msgr2=0x7feda00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:02.845 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:00.708+0000 7fedbb3a5640 1 --2- 172.21.15.80:0/2968146966 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7feda00a2b90 0x7feda00a5050 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fedb41016d0 tx=0x7fedb0009870 comp rx=0 tx=0).stop 2024-04-24T23:32:02.846 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:00.708+0000 7fedbb3a5640 1 -- 172.21.15.80:0/2968146966 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fedb4101870 msgr2=0x7fedb4196eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:02.846 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:00.708+0000 7fedbb3a5640 1 --2- 172.21.15.80:0/2968146966 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fedb4101870 0x7fedb4196eb0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7feda4009230 tx=0x7feda4004360 comp rx=0 tx=0).stop 2024-04-24T23:32:02.846 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:00.708+0000 7fedbb3a5640 1 -- 172.21.15.80:0/2968146966 shutdown_connections 2024-04-24T23:32:02.846 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:00.708+0000 7fedbb3a5640 1 --2- 172.21.15.80:0/2968146966 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7feda00a2b90 0x7feda00a5050 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:02.846 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:00.708+0000 7fedbb3a5640 1 --2- 172.21.15.80:0/2968146966 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fedb4102230 0x7fedb419df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:02.846 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:00.708+0000 7fedbb3a5640 1 --2- 172.21.15.80:0/2968146966 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fedb4101870 0x7fedb4196eb0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:02.846 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:00.708+0000 7fedbb3a5640 1 --2- 172.21.15.80:0/2968146966 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fedb4100680 0x7fedb4196970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:02.846 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:00.708+0000 7fedbb3a5640 1 -- 172.21.15.80:0/2968146966 >> 172.21.15.80:0/2968146966 conn(0x7fedb40fbbf0 msgr2=0x7fedb4104b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:32:02.847 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:00.708+0000 7fedbb3a5640 1 -- 172.21.15.80:0/2968146966 shutdown_connections 2024-04-24T23:32:02.847 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:00.708+0000 7fedbb3a5640 1 -- 172.21.15.80:0/2968146966 wait complete. 2024-04-24T23:32:02.852 DEBUG:tasks.check_counter:Getting stats from a 2024-04-24T23:32:02.853 DEBUG:teuthology.orchestra.run.smithi080:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid ec7885aa-0290-11ef-bc93-c7b262605968 -- ceph daemon mds.a perf dump 2024-04-24T23:32:02.853 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:32:01 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/1180168202' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T23:32:02.854 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:32:01 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/2968146966' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-24T23:32:02.854 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:32:01 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:32:02.854 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:32:02 smithi080 ceph-mon[29152]: pgmap v441: 129 pgs: 129 active+clean; 69 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 295 KiB/s wr, 24 op/s 2024-04-24T23:32:02.854 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:32:02 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:32:02.854 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:32:02 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:32:02.854 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:32:02 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' 2024-04-24T23:32:03.091 INFO:teuthology.orchestra.run.smithi080.stderr:Inferring config /var/lib/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:32:04.393 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:32:04 smithi080 ceph-mon[29152]: pgmap v442: 129 pgs: 129 active+clean; 69 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 100 KiB/s wr, 15 op/s 2024-04-24T23:32:04.446 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:32:04 smithi134 ceph-mon[32619]: pgmap v442: 129 pgs: 129 active+clean; 69 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 100 KiB/s wr, 15 op/s 2024-04-24T23:32:04.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:32:04 smithi148 ceph-mon[32528]: pgmap v442: 129 pgs: 129 active+clean; 69 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 100 KiB/s wr, 15 op/s 2024-04-24T23:32:04.976 INFO:teuthology.orchestra.run.smithi080.stdout:{ 2024-04-24T23:32:04.976 INFO:teuthology.orchestra.run.smithi080.stdout: "AsyncMessenger::Worker-0": { 2024-04-24T23:32:04.977 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_recv_messages": 4, 2024-04-24T23:32:04.977 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_messages": 4, 2024-04-24T23:32:04.977 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_recv_bytes": 2992, 2024-04-24T23:32:04.977 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_bytes": 448, 2024-04-24T23:32:04.978 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_created_connections": 4, 2024-04-24T23:32:04.978 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_active_connections": 0, 2024-04-24T23:32:04.978 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_total_time": 0.001383368, 2024-04-24T23:32:04.978 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_send_time": 0.000034641, 2024-04-24T23:32:04.978 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_recv_time": 0.000465811, 2024-04-24T23:32:04.978 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-24T23:32:04.978 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:32:04.978 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 3, 2024-04-24T23:32:04.979 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000720266, 2024-04-24T23:32:04.979 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000240088 2024-04-24T23:32:04.979 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:04.979 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:32:04.979 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-04-24T23:32:04.979 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-04-24T23:32:04.979 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-04-24T23:32:04.979 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:04.980 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-04-24T23:32:04.980 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_encrypted_bytes": 448 2024-04-24T23:32:04.980 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:04.980 INFO:teuthology.orchestra.run.smithi080.stdout: "AsyncMessenger::Worker-1": { 2024-04-24T23:32:04.981 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_recv_messages": 1, 2024-04-24T23:32:04.981 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_messages": 62, 2024-04-24T23:32:04.981 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_recv_bytes": 160, 2024-04-24T23:32:04.981 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_bytes": 88464, 2024-04-24T23:32:04.981 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_created_connections": 3, 2024-04-24T23:32:04.981 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_active_connections": 2, 2024-04-24T23:32:04.981 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_total_time": 0.005301435, 2024-04-24T23:32:04.982 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_send_time": 0.003628821, 2024-04-24T23:32:04.982 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_recv_time": 0.001048676, 2024-04-24T23:32:04.982 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-24T23:32:04.982 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:32:04.982 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 60, 2024-04-24T23:32:04.982 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.002610602, 2024-04-24T23:32:04.982 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000043510 2024-04-24T23:32:04.982 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:04.983 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:32:04.983 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-04-24T23:32:04.983 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-04-24T23:32:04.983 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-04-24T23:32:04.983 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:04.983 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-24T23:32:04.984 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_encrypted_bytes": 88464 2024-04-24T23:32:04.984 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:04.984 INFO:teuthology.orchestra.run.smithi080.stdout: "AsyncMessenger::Worker-2": { 2024-04-24T23:32:04.984 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_recv_messages": 85, 2024-04-24T23:32:04.984 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_messages": 82, 2024-04-24T23:32:04.984 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_recv_bytes": 439632, 2024-04-24T23:32:04.984 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_bytes": 49472, 2024-04-24T23:32:04.985 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_created_connections": 2, 2024-04-24T23:32:04.985 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_active_connections": 1, 2024-04-24T23:32:04.985 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_total_time": 0.014429376, 2024-04-24T23:32:04.985 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_send_time": 0.005043813, 2024-04-24T23:32:04.985 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_recv_time": 0.010019624, 2024-04-24T23:32:04.985 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_fast_dispatch_time": 0.001483703, 2024-04-24T23:32:04.985 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:32:04.985 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 81, 2024-04-24T23:32:04.985 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.004038051, 2024-04-24T23:32:04.986 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000049852 2024-04-24T23:32:04.986 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:04.986 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:32:04.986 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-04-24T23:32:04.987 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-04-24T23:32:04.987 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-04-24T23:32:04.987 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:04.987 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_recv_encrypted_bytes": 439632, 2024-04-24T23:32:04.987 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_encrypted_bytes": 49472 2024-04-24T23:32:04.987 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:04.987 INFO:teuthology.orchestra.run.smithi080.stdout: "cct": { 2024-04-24T23:32:04.987 INFO:teuthology.orchestra.run.smithi080.stdout: "total_workers": 0, 2024-04-24T23:32:04.987 INFO:teuthology.orchestra.run.smithi080.stdout: "unhealthy_workers": 0 2024-04-24T23:32:04.988 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:04.988 INFO:teuthology.orchestra.run.smithi080.stdout: "mempool": { 2024-04-24T23:32:04.988 INFO:teuthology.orchestra.run.smithi080.stdout: "bloom_filter_bytes": 0, 2024-04-24T23:32:04.988 INFO:teuthology.orchestra.run.smithi080.stdout: "bloom_filter_items": 0, 2024-04-24T23:32:04.988 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_alloc_bytes": 0, 2024-04-24T23:32:04.988 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_alloc_items": 0, 2024-04-24T23:32:04.988 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_cache_data_bytes": 0, 2024-04-24T23:32:04.988 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_cache_data_items": 0, 2024-04-24T23:32:04.988 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-24T23:32:04.989 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_cache_onode_items": 0, 2024-04-24T23:32:04.989 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-24T23:32:04.989 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_cache_meta_items": 0, 2024-04-24T23:32:04.989 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_cache_other_bytes": 0, 2024-04-24T23:32:04.990 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_cache_other_items": 0, 2024-04-24T23:32:04.990 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-24T23:32:04.990 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_cache_buffer_items": 0, 2024-04-24T23:32:04.990 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_extent_bytes": 0, 2024-04-24T23:32:04.990 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_extent_items": 0, 2024-04-24T23:32:04.990 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_blob_bytes": 0, 2024-04-24T23:32:04.990 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_blob_items": 0, 2024-04-24T23:32:04.990 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-24T23:32:04.990 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_shared_blob_items": 0, 2024-04-24T23:32:04.991 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-24T23:32:04.991 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_inline_bl_items": 0, 2024-04-24T23:32:04.991 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_fsck_bytes": 0, 2024-04-24T23:32:04.991 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_fsck_items": 0, 2024-04-24T23:32:04.991 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_txc_bytes": 0, 2024-04-24T23:32:04.991 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_txc_items": 0, 2024-04-24T23:32:04.991 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-24T23:32:04.991 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_writing_deferred_items": 0, 2024-04-24T23:32:04.992 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_writing_bytes": 0, 2024-04-24T23:32:04.992 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_writing_items": 0, 2024-04-24T23:32:04.992 INFO:teuthology.orchestra.run.smithi080.stdout: "bluefs_bytes": 0, 2024-04-24T23:32:04.992 INFO:teuthology.orchestra.run.smithi080.stdout: "bluefs_items": 0, 2024-04-24T23:32:04.992 INFO:teuthology.orchestra.run.smithi080.stdout: "bluefs_file_reader_bytes": 0, 2024-04-24T23:32:04.992 INFO:teuthology.orchestra.run.smithi080.stdout: "bluefs_file_reader_items": 0, 2024-04-24T23:32:04.992 INFO:teuthology.orchestra.run.smithi080.stdout: "bluefs_file_writer_bytes": 0, 2024-04-24T23:32:04.992 INFO:teuthology.orchestra.run.smithi080.stdout: "bluefs_file_writer_items": 0, 2024-04-24T23:32:04.992 INFO:teuthology.orchestra.run.smithi080.stdout: "buffer_anon_bytes": 148779, 2024-04-24T23:32:04.993 INFO:teuthology.orchestra.run.smithi080.stdout: "buffer_anon_items": 27, 2024-04-24T23:32:04.993 INFO:teuthology.orchestra.run.smithi080.stdout: "buffer_meta_bytes": 0, 2024-04-24T23:32:04.993 INFO:teuthology.orchestra.run.smithi080.stdout: "buffer_meta_items": 0, 2024-04-24T23:32:04.993 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_bytes": 0, 2024-04-24T23:32:04.993 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_items": 0, 2024-04-24T23:32:04.993 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_mapbl_bytes": 0, 2024-04-24T23:32:04.993 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_mapbl_items": 0, 2024-04-24T23:32:04.993 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_pglog_bytes": 0, 2024-04-24T23:32:04.993 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_pglog_items": 0, 2024-04-24T23:32:04.994 INFO:teuthology.orchestra.run.smithi080.stdout: "osdmap_bytes": 0, 2024-04-24T23:32:04.994 INFO:teuthology.orchestra.run.smithi080.stdout: "osdmap_items": 0, 2024-04-24T23:32:04.994 INFO:teuthology.orchestra.run.smithi080.stdout: "osdmap_mapping_bytes": 0, 2024-04-24T23:32:04.994 INFO:teuthology.orchestra.run.smithi080.stdout: "osdmap_mapping_items": 0, 2024-04-24T23:32:04.994 INFO:teuthology.orchestra.run.smithi080.stdout: "pgmap_bytes": 0, 2024-04-24T23:32:04.994 INFO:teuthology.orchestra.run.smithi080.stdout: "pgmap_items": 0, 2024-04-24T23:32:04.995 INFO:teuthology.orchestra.run.smithi080.stdout: "mds_co_bytes": 928, 2024-04-24T23:32:04.995 INFO:teuthology.orchestra.run.smithi080.stdout: "mds_co_items": 2, 2024-04-24T23:32:04.995 INFO:teuthology.orchestra.run.smithi080.stdout: "unittest_1_bytes": 0, 2024-04-24T23:32:04.995 INFO:teuthology.orchestra.run.smithi080.stdout: "unittest_1_items": 0, 2024-04-24T23:32:04.995 INFO:teuthology.orchestra.run.smithi080.stdout: "unittest_2_bytes": 0, 2024-04-24T23:32:04.995 INFO:teuthology.orchestra.run.smithi080.stdout: "unittest_2_items": 0 2024-04-24T23:32:04.995 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:04.995 INFO:teuthology.orchestra.run.smithi080.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-24T23:32:04.996 INFO:teuthology.orchestra.run.smithi080.stdout: "val": 0, 2024-04-24T23:32:04.996 INFO:teuthology.orchestra.run.smithi080.stdout: "max": 104857600, 2024-04-24T23:32:04.996 INFO:teuthology.orchestra.run.smithi080.stdout: "get_started": 0, 2024-04-24T23:32:04.996 INFO:teuthology.orchestra.run.smithi080.stdout: "get": 86, 2024-04-24T23:32:04.996 INFO:teuthology.orchestra.run.smithi080.stdout: "get_sum": 427662, 2024-04-24T23:32:04.996 INFO:teuthology.orchestra.run.smithi080.stdout: "get_or_fail_fail": 0, 2024-04-24T23:32:04.996 INFO:teuthology.orchestra.run.smithi080.stdout: "get_or_fail_success": 86, 2024-04-24T23:32:04.996 INFO:teuthology.orchestra.run.smithi080.stdout: "take": 0, 2024-04-24T23:32:04.996 INFO:teuthology.orchestra.run.smithi080.stdout: "take_sum": 0, 2024-04-24T23:32:04.997 INFO:teuthology.orchestra.run.smithi080.stdout: "put": 86, 2024-04-24T23:32:04.997 INFO:teuthology.orchestra.run.smithi080.stdout: "put_sum": 427662, 2024-04-24T23:32:04.997 INFO:teuthology.orchestra.run.smithi080.stdout: "wait": { 2024-04-24T23:32:04.997 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-04-24T23:32:04.997 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-04-24T23:32:04.997 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-04-24T23:32:04.997 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:04.997 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:04.998 INFO:teuthology.orchestra.run.smithi080.stdout:} 2024-04-24T23:32:05.778 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-04-24T23:32:05.778 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-04-24T23:32:05.778 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.a 2024-04-24T23:32:05.778 DEBUG:tasks.check_counter:Getting stats from d 2024-04-24T23:32:05.779 DEBUG:teuthology.orchestra.run.smithi080:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid ec7885aa-0290-11ef-bc93-c7b262605968 -- ceph daemon mds.d perf dump 2024-04-24T23:32:06.029 INFO:teuthology.orchestra.run.smithi080.stderr:Inferring config /var/lib/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:32:06.371 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:32:06 smithi080 ceph-mon[29152]: pgmap v443: 129 pgs: 129 active+clean; 74 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 558 KiB/s wr, 69 op/s 2024-04-24T23:32:06.446 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:32:06 smithi134 ceph-mon[32619]: pgmap v443: 129 pgs: 129 active+clean; 74 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 558 KiB/s wr, 69 op/s 2024-04-24T23:32:06.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:32:06 smithi148 ceph-mon[32528]: pgmap v443: 129 pgs: 129 active+clean; 74 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 558 KiB/s wr, 69 op/s 2024-04-24T23:32:08.278 INFO:teuthology.orchestra.run.smithi080.stdout:{ 2024-04-24T23:32:08.278 INFO:teuthology.orchestra.run.smithi080.stdout: "AsyncMessenger::Worker-0": { 2024-04-24T23:32:08.279 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_recv_messages": 4, 2024-04-24T23:32:08.279 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_messages": 4, 2024-04-24T23:32:08.279 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_recv_bytes": 2992, 2024-04-24T23:32:08.279 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_bytes": 448, 2024-04-24T23:32:08.279 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_created_connections": 4, 2024-04-24T23:32:08.279 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_active_connections": 0, 2024-04-24T23:32:08.279 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_total_time": 0.001516385, 2024-04-24T23:32:08.279 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_send_time": 0.000036819, 2024-04-24T23:32:08.279 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_recv_time": 0.000643391, 2024-04-24T23:32:08.280 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-24T23:32:08.280 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:32:08.280 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 3, 2024-04-24T23:32:08.280 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000688570, 2024-04-24T23:32:08.280 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000229523 2024-04-24T23:32:08.280 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:08.280 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:32:08.280 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-04-24T23:32:08.281 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-04-24T23:32:08.281 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-04-24T23:32:08.281 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:08.281 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-04-24T23:32:08.281 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_encrypted_bytes": 448 2024-04-24T23:32:08.281 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:08.281 INFO:teuthology.orchestra.run.smithi080.stdout: "AsyncMessenger::Worker-1": { 2024-04-24T23:32:08.281 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_recv_messages": 92, 2024-04-24T23:32:08.281 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_messages": 89, 2024-04-24T23:32:08.282 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_recv_bytes": 441600, 2024-04-24T23:32:08.282 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_bytes": 53472, 2024-04-24T23:32:08.282 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_created_connections": 2, 2024-04-24T23:32:08.282 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_active_connections": 1, 2024-04-24T23:32:08.282 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_total_time": 0.014313685, 2024-04-24T23:32:08.282 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_send_time": 0.004951992, 2024-04-24T23:32:08.282 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_recv_time": 0.010148316, 2024-04-24T23:32:08.282 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_fast_dispatch_time": 0.001569283, 2024-04-24T23:32:08.283 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:32:08.283 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 88, 2024-04-24T23:32:08.283 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.004290313, 2024-04-24T23:32:08.283 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000048753 2024-04-24T23:32:08.283 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:08.283 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:32:08.283 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-04-24T23:32:08.283 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-04-24T23:32:08.283 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-04-24T23:32:08.284 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:08.284 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_recv_encrypted_bytes": 441600, 2024-04-24T23:32:08.284 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_encrypted_bytes": 53472 2024-04-24T23:32:08.284 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:08.284 INFO:teuthology.orchestra.run.smithi080.stdout: "AsyncMessenger::Worker-2": { 2024-04-24T23:32:08.284 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_recv_messages": 1, 2024-04-24T23:32:08.284 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_messages": 67, 2024-04-24T23:32:08.284 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_recv_bytes": 160, 2024-04-24T23:32:08.284 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_bytes": 89344, 2024-04-24T23:32:08.285 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_created_connections": 3, 2024-04-24T23:32:08.285 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_active_connections": 2, 2024-04-24T23:32:08.285 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_total_time": 0.005823514, 2024-04-24T23:32:08.285 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_send_time": 0.003780887, 2024-04-24T23:32:08.285 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_recv_time": 0.001309735, 2024-04-24T23:32:08.285 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-24T23:32:08.285 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:32:08.285 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 65, 2024-04-24T23:32:08.285 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.003433839, 2024-04-24T23:32:08.286 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000052828 2024-04-24T23:32:08.286 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:08.286 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:32:08.286 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-04-24T23:32:08.286 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-04-24T23:32:08.286 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-04-24T23:32:08.286 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:08.286 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-24T23:32:08.287 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_encrypted_bytes": 89344 2024-04-24T23:32:08.287 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:08.287 INFO:teuthology.orchestra.run.smithi080.stdout: "cct": { 2024-04-24T23:32:08.287 INFO:teuthology.orchestra.run.smithi080.stdout: "total_workers": 0, 2024-04-24T23:32:08.287 INFO:teuthology.orchestra.run.smithi080.stdout: "unhealthy_workers": 0 2024-04-24T23:32:08.287 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:08.287 INFO:teuthology.orchestra.run.smithi080.stdout: "mempool": { 2024-04-24T23:32:08.287 INFO:teuthology.orchestra.run.smithi080.stdout: "bloom_filter_bytes": 0, 2024-04-24T23:32:08.287 INFO:teuthology.orchestra.run.smithi080.stdout: "bloom_filter_items": 0, 2024-04-24T23:32:08.288 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_alloc_bytes": 0, 2024-04-24T23:32:08.288 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_alloc_items": 0, 2024-04-24T23:32:08.288 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_cache_data_bytes": 0, 2024-04-24T23:32:08.288 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_cache_data_items": 0, 2024-04-24T23:32:08.288 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-24T23:32:08.288 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_cache_onode_items": 0, 2024-04-24T23:32:08.288 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-24T23:32:08.288 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_cache_meta_items": 0, 2024-04-24T23:32:08.289 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_cache_other_bytes": 0, 2024-04-24T23:32:08.289 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_cache_other_items": 0, 2024-04-24T23:32:08.289 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-24T23:32:08.289 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_cache_buffer_items": 0, 2024-04-24T23:32:08.289 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_extent_bytes": 0, 2024-04-24T23:32:08.289 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_extent_items": 0, 2024-04-24T23:32:08.289 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_blob_bytes": 0, 2024-04-24T23:32:08.289 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_blob_items": 0, 2024-04-24T23:32:08.289 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-24T23:32:08.290 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_shared_blob_items": 0, 2024-04-24T23:32:08.290 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-24T23:32:08.290 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_inline_bl_items": 0, 2024-04-24T23:32:08.290 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_fsck_bytes": 0, 2024-04-24T23:32:08.290 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_fsck_items": 0, 2024-04-24T23:32:08.290 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_txc_bytes": 0, 2024-04-24T23:32:08.290 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_txc_items": 0, 2024-04-24T23:32:08.290 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-24T23:32:08.291 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_writing_deferred_items": 0, 2024-04-24T23:32:08.291 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_writing_bytes": 0, 2024-04-24T23:32:08.291 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_writing_items": 0, 2024-04-24T23:32:08.291 INFO:teuthology.orchestra.run.smithi080.stdout: "bluefs_bytes": 0, 2024-04-24T23:32:08.291 INFO:teuthology.orchestra.run.smithi080.stdout: "bluefs_items": 0, 2024-04-24T23:32:08.291 INFO:teuthology.orchestra.run.smithi080.stdout: "bluefs_file_reader_bytes": 0, 2024-04-24T23:32:08.291 INFO:teuthology.orchestra.run.smithi080.stdout: "bluefs_file_reader_items": 0, 2024-04-24T23:32:08.291 INFO:teuthology.orchestra.run.smithi080.stdout: "bluefs_file_writer_bytes": 0, 2024-04-24T23:32:08.291 INFO:teuthology.orchestra.run.smithi080.stdout: "bluefs_file_writer_items": 0, 2024-04-24T23:32:08.292 INFO:teuthology.orchestra.run.smithi080.stdout: "buffer_anon_bytes": 148823, 2024-04-24T23:32:08.292 INFO:teuthology.orchestra.run.smithi080.stdout: "buffer_anon_items": 27, 2024-04-24T23:32:08.292 INFO:teuthology.orchestra.run.smithi080.stdout: "buffer_meta_bytes": 0, 2024-04-24T23:32:08.292 INFO:teuthology.orchestra.run.smithi080.stdout: "buffer_meta_items": 0, 2024-04-24T23:32:08.292 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_bytes": 0, 2024-04-24T23:32:08.292 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_items": 0, 2024-04-24T23:32:08.292 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_mapbl_bytes": 0, 2024-04-24T23:32:08.292 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_mapbl_items": 0, 2024-04-24T23:32:08.292 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_pglog_bytes": 0, 2024-04-24T23:32:08.293 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_pglog_items": 0, 2024-04-24T23:32:08.293 INFO:teuthology.orchestra.run.smithi080.stdout: "osdmap_bytes": 0, 2024-04-24T23:32:08.293 INFO:teuthology.orchestra.run.smithi080.stdout: "osdmap_items": 0, 2024-04-24T23:32:08.293 INFO:teuthology.orchestra.run.smithi080.stdout: "osdmap_mapping_bytes": 0, 2024-04-24T23:32:08.293 INFO:teuthology.orchestra.run.smithi080.stdout: "osdmap_mapping_items": 0, 2024-04-24T23:32:08.293 INFO:teuthology.orchestra.run.smithi080.stdout: "pgmap_bytes": 0, 2024-04-24T23:32:08.293 INFO:teuthology.orchestra.run.smithi080.stdout: "pgmap_items": 0, 2024-04-24T23:32:08.293 INFO:teuthology.orchestra.run.smithi080.stdout: "mds_co_bytes": 928, 2024-04-24T23:32:08.294 INFO:teuthology.orchestra.run.smithi080.stdout: "mds_co_items": 2, 2024-04-24T23:32:08.294 INFO:teuthology.orchestra.run.smithi080.stdout: "unittest_1_bytes": 0, 2024-04-24T23:32:08.294 INFO:teuthology.orchestra.run.smithi080.stdout: "unittest_1_items": 0, 2024-04-24T23:32:08.294 INFO:teuthology.orchestra.run.smithi080.stdout: "unittest_2_bytes": 0, 2024-04-24T23:32:08.294 INFO:teuthology.orchestra.run.smithi080.stdout: "unittest_2_items": 0 2024-04-24T23:32:08.294 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:08.294 INFO:teuthology.orchestra.run.smithi080.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-24T23:32:08.294 INFO:teuthology.orchestra.run.smithi080.stdout: "val": 0, 2024-04-24T23:32:08.294 INFO:teuthology.orchestra.run.smithi080.stdout: "max": 104857600, 2024-04-24T23:32:08.295 INFO:teuthology.orchestra.run.smithi080.stdout: "get_started": 0, 2024-04-24T23:32:08.295 INFO:teuthology.orchestra.run.smithi080.stdout: "get": 93, 2024-04-24T23:32:08.295 INFO:teuthology.orchestra.run.smithi080.stdout: "get_sum": 428636, 2024-04-24T23:32:08.295 INFO:teuthology.orchestra.run.smithi080.stdout: "get_or_fail_fail": 0, 2024-04-24T23:32:08.295 INFO:teuthology.orchestra.run.smithi080.stdout: "get_or_fail_success": 93, 2024-04-24T23:32:08.295 INFO:teuthology.orchestra.run.smithi080.stdout: "take": 0, 2024-04-24T23:32:08.295 INFO:teuthology.orchestra.run.smithi080.stdout: "take_sum": 0, 2024-04-24T23:32:08.295 INFO:teuthology.orchestra.run.smithi080.stdout: "put": 93, 2024-04-24T23:32:08.296 INFO:teuthology.orchestra.run.smithi080.stdout: "put_sum": 428636, 2024-04-24T23:32:08.296 INFO:teuthology.orchestra.run.smithi080.stdout: "wait": { 2024-04-24T23:32:08.296 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-04-24T23:32:08.296 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-04-24T23:32:08.296 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-04-24T23:32:08.296 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:08.296 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:08.296 INFO:teuthology.orchestra.run.smithi080.stdout:} 2024-04-24T23:32:08.446 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:32:08 smithi134 ceph-mon[32619]: pgmap v444: 129 pgs: 129 active+clean; 74 MiB data, 538 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 534 KiB/s wr, 67 op/s 2024-04-24T23:32:08.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:32:08 smithi148 ceph-mon[32528]: pgmap v444: 129 pgs: 129 active+clean; 74 MiB data, 538 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 534 KiB/s wr, 67 op/s 2024-04-24T23:32:08.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:32:08 smithi080 ceph-mon[29152]: pgmap v444: 129 pgs: 129 active+clean; 74 MiB data, 538 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 534 KiB/s wr, 67 op/s 2024-04-24T23:32:09.099 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-04-24T23:32:09.100 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-04-24T23:32:09.100 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.d 2024-04-24T23:32:09.100 DEBUG:tasks.check_counter:Getting stats from g 2024-04-24T23:32:09.100 DEBUG:teuthology.orchestra.run.smithi080:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid ec7885aa-0290-11ef-bc93-c7b262605968 -- ceph daemon mds.g perf dump 2024-04-24T23:32:09.343 INFO:teuthology.orchestra.run.smithi080.stderr:Inferring config /var/lib/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:32:10.446 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:32:10 smithi134 ceph-mon[32619]: pgmap v445: 129 pgs: 129 active+clean; 74 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 552 KiB/s wr, 76 op/s 2024-04-24T23:32:10.505 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:32:10 smithi080 ceph-mon[29152]: pgmap v445: 129 pgs: 129 active+clean; 74 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 552 KiB/s wr, 76 op/s 2024-04-24T23:32:10.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:32:10 smithi148 ceph-mon[32528]: pgmap v445: 129 pgs: 129 active+clean; 74 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 552 KiB/s wr, 76 op/s 2024-04-24T23:32:11.085 INFO:teuthology.orchestra.run.smithi080.stdout:{ 2024-04-24T23:32:11.085 INFO:teuthology.orchestra.run.smithi080.stdout: "AsyncMessenger::Worker-0": { 2024-04-24T23:32:11.085 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_recv_messages": 88, 2024-04-24T23:32:11.086 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_messages": 85, 2024-04-24T23:32:11.086 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_recv_bytes": 440512, 2024-04-24T23:32:11.086 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_bytes": 50912, 2024-04-24T23:32:11.086 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_created_connections": 4, 2024-04-24T23:32:11.086 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_active_connections": 0, 2024-04-24T23:32:11.086 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_total_time": 0.015836430, 2024-04-24T23:32:11.086 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_send_time": 0.005307678, 2024-04-24T23:32:11.086 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_recv_time": 0.011022590, 2024-04-24T23:32:11.087 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_fast_dispatch_time": 0.001493517, 2024-04-24T23:32:11.087 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:32:11.087 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 84, 2024-04-24T23:32:11.087 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.004173442, 2024-04-24T23:32:11.087 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000049683 2024-04-24T23:32:11.087 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:11.087 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:32:11.087 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-04-24T23:32:11.088 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-04-24T23:32:11.088 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-04-24T23:32:11.088 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:11.088 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_recv_encrypted_bytes": 440512, 2024-04-24T23:32:11.088 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_encrypted_bytes": 50912 2024-04-24T23:32:11.088 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:11.088 INFO:teuthology.orchestra.run.smithi080.stdout: "AsyncMessenger::Worker-1": { 2024-04-24T23:32:11.088 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_recv_messages": 1, 2024-04-24T23:32:11.089 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_messages": 64, 2024-04-24T23:32:11.089 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_recv_bytes": 160, 2024-04-24T23:32:11.089 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_bytes": 88816, 2024-04-24T23:32:11.089 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_created_connections": 3, 2024-04-24T23:32:11.089 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_active_connections": 2, 2024-04-24T23:32:11.089 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_total_time": 0.005441113, 2024-04-24T23:32:11.089 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_send_time": 0.003697791, 2024-04-24T23:32:11.089 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_recv_time": 0.001074722, 2024-04-24T23:32:11.089 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-24T23:32:11.090 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:32:11.090 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 62, 2024-04-24T23:32:11.090 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.003061798, 2024-04-24T23:32:11.090 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000049383 2024-04-24T23:32:11.090 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:11.090 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:32:11.090 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-04-24T23:32:11.090 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-04-24T23:32:11.091 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-04-24T23:32:11.091 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:11.091 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-24T23:32:11.091 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_encrypted_bytes": 88816 2024-04-24T23:32:11.091 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:11.091 INFO:teuthology.orchestra.run.smithi080.stdout: "AsyncMessenger::Worker-2": { 2024-04-24T23:32:11.091 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_recv_messages": 4, 2024-04-24T23:32:11.091 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_messages": 4, 2024-04-24T23:32:11.092 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_recv_bytes": 2992, 2024-04-24T23:32:11.092 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_bytes": 448, 2024-04-24T23:32:11.092 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_created_connections": 2, 2024-04-24T23:32:11.092 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_active_connections": 1, 2024-04-24T23:32:11.092 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_total_time": 0.000653293, 2024-04-24T23:32:11.092 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_send_time": 0.000034185, 2024-04-24T23:32:11.092 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_recv_time": 0.000614444, 2024-04-24T23:32:11.092 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-24T23:32:11.093 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:32:11.093 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 3, 2024-04-24T23:32:11.093 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000788744, 2024-04-24T23:32:11.093 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000262914 2024-04-24T23:32:11.093 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:11.093 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:32:11.093 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-04-24T23:32:11.093 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-04-24T23:32:11.094 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-04-24T23:32:11.094 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:11.094 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-04-24T23:32:11.094 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_encrypted_bytes": 448 2024-04-24T23:32:11.094 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:11.094 INFO:teuthology.orchestra.run.smithi080.stdout: "cct": { 2024-04-24T23:32:11.094 INFO:teuthology.orchestra.run.smithi080.stdout: "total_workers": 0, 2024-04-24T23:32:11.094 INFO:teuthology.orchestra.run.smithi080.stdout: "unhealthy_workers": 0 2024-04-24T23:32:11.094 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:11.095 INFO:teuthology.orchestra.run.smithi080.stdout: "mempool": { 2024-04-24T23:32:11.095 INFO:teuthology.orchestra.run.smithi080.stdout: "bloom_filter_bytes": 0, 2024-04-24T23:32:11.095 INFO:teuthology.orchestra.run.smithi080.stdout: "bloom_filter_items": 0, 2024-04-24T23:32:11.095 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_alloc_bytes": 0, 2024-04-24T23:32:11.095 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_alloc_items": 0, 2024-04-24T23:32:11.095 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_cache_data_bytes": 0, 2024-04-24T23:32:11.095 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_cache_data_items": 0, 2024-04-24T23:32:11.095 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-24T23:32:11.096 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_cache_onode_items": 0, 2024-04-24T23:32:11.096 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-24T23:32:11.096 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_cache_meta_items": 0, 2024-04-24T23:32:11.096 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_cache_other_bytes": 0, 2024-04-24T23:32:11.096 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_cache_other_items": 0, 2024-04-24T23:32:11.096 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-24T23:32:11.096 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_cache_buffer_items": 0, 2024-04-24T23:32:11.096 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_extent_bytes": 0, 2024-04-24T23:32:11.096 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_extent_items": 0, 2024-04-24T23:32:11.097 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_blob_bytes": 0, 2024-04-24T23:32:11.097 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_blob_items": 0, 2024-04-24T23:32:11.097 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-24T23:32:11.097 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_shared_blob_items": 0, 2024-04-24T23:32:11.097 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-24T23:32:11.097 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_inline_bl_items": 0, 2024-04-24T23:32:11.097 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_fsck_bytes": 0, 2024-04-24T23:32:11.098 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_fsck_items": 0, 2024-04-24T23:32:11.098 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_txc_bytes": 0, 2024-04-24T23:32:11.098 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_txc_items": 0, 2024-04-24T23:32:11.098 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-24T23:32:11.098 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_writing_deferred_items": 0, 2024-04-24T23:32:11.098 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_writing_bytes": 0, 2024-04-24T23:32:11.098 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_writing_items": 0, 2024-04-24T23:32:11.099 INFO:teuthology.orchestra.run.smithi080.stdout: "bluefs_bytes": 0, 2024-04-24T23:32:11.099 INFO:teuthology.orchestra.run.smithi080.stdout: "bluefs_items": 0, 2024-04-24T23:32:11.099 INFO:teuthology.orchestra.run.smithi080.stdout: "bluefs_file_reader_bytes": 0, 2024-04-24T23:32:11.099 INFO:teuthology.orchestra.run.smithi080.stdout: "bluefs_file_reader_items": 0, 2024-04-24T23:32:11.099 INFO:teuthology.orchestra.run.smithi080.stdout: "bluefs_file_writer_bytes": 0, 2024-04-24T23:32:11.099 INFO:teuthology.orchestra.run.smithi080.stdout: "bluefs_file_writer_items": 0, 2024-04-24T23:32:11.099 INFO:teuthology.orchestra.run.smithi080.stdout: "buffer_anon_bytes": 148823, 2024-04-24T23:32:11.099 INFO:teuthology.orchestra.run.smithi080.stdout: "buffer_anon_items": 27, 2024-04-24T23:32:11.099 INFO:teuthology.orchestra.run.smithi080.stdout: "buffer_meta_bytes": 0, 2024-04-24T23:32:11.100 INFO:teuthology.orchestra.run.smithi080.stdout: "buffer_meta_items": 0, 2024-04-24T23:32:11.100 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_bytes": 0, 2024-04-24T23:32:11.100 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_items": 0, 2024-04-24T23:32:11.100 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_mapbl_bytes": 0, 2024-04-24T23:32:11.100 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_mapbl_items": 0, 2024-04-24T23:32:11.100 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_pglog_bytes": 0, 2024-04-24T23:32:11.100 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_pglog_items": 0, 2024-04-24T23:32:11.100 INFO:teuthology.orchestra.run.smithi080.stdout: "osdmap_bytes": 0, 2024-04-24T23:32:11.100 INFO:teuthology.orchestra.run.smithi080.stdout: "osdmap_items": 0, 2024-04-24T23:32:11.101 INFO:teuthology.orchestra.run.smithi080.stdout: "osdmap_mapping_bytes": 0, 2024-04-24T23:32:11.101 INFO:teuthology.orchestra.run.smithi080.stdout: "osdmap_mapping_items": 0, 2024-04-24T23:32:11.101 INFO:teuthology.orchestra.run.smithi080.stdout: "pgmap_bytes": 0, 2024-04-24T23:32:11.101 INFO:teuthology.orchestra.run.smithi080.stdout: "pgmap_items": 0, 2024-04-24T23:32:11.101 INFO:teuthology.orchestra.run.smithi080.stdout: "mds_co_bytes": 928, 2024-04-24T23:32:11.101 INFO:teuthology.orchestra.run.smithi080.stdout: "mds_co_items": 2, 2024-04-24T23:32:11.101 INFO:teuthology.orchestra.run.smithi080.stdout: "unittest_1_bytes": 0, 2024-04-24T23:32:11.101 INFO:teuthology.orchestra.run.smithi080.stdout: "unittest_1_items": 0, 2024-04-24T23:32:11.102 INFO:teuthology.orchestra.run.smithi080.stdout: "unittest_2_bytes": 0, 2024-04-24T23:32:11.102 INFO:teuthology.orchestra.run.smithi080.stdout: "unittest_2_items": 0 2024-04-24T23:32:11.102 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:11.102 INFO:teuthology.orchestra.run.smithi080.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-24T23:32:11.102 INFO:teuthology.orchestra.run.smithi080.stdout: "val": 0, 2024-04-24T23:32:11.102 INFO:teuthology.orchestra.run.smithi080.stdout: "max": 104857600, 2024-04-24T23:32:11.102 INFO:teuthology.orchestra.run.smithi080.stdout: "get_started": 0, 2024-04-24T23:32:11.102 INFO:teuthology.orchestra.run.smithi080.stdout: "get": 89, 2024-04-24T23:32:11.103 INFO:teuthology.orchestra.run.smithi080.stdout: "get_sum": 428116, 2024-04-24T23:32:11.103 INFO:teuthology.orchestra.run.smithi080.stdout: "get_or_fail_fail": 0, 2024-04-24T23:32:11.103 INFO:teuthology.orchestra.run.smithi080.stdout: "get_or_fail_success": 89, 2024-04-24T23:32:11.103 INFO:teuthology.orchestra.run.smithi080.stdout: "take": 0, 2024-04-24T23:32:11.103 INFO:teuthology.orchestra.run.smithi080.stdout: "take_sum": 0, 2024-04-24T23:32:11.103 INFO:teuthology.orchestra.run.smithi080.stdout: "put": 89, 2024-04-24T23:32:11.103 INFO:teuthology.orchestra.run.smithi080.stdout: "put_sum": 428116, 2024-04-24T23:32:11.103 INFO:teuthology.orchestra.run.smithi080.stdout: "wait": { 2024-04-24T23:32:11.104 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-04-24T23:32:11.104 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-04-24T23:32:11.104 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-04-24T23:32:11.104 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:11.104 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:11.105 INFO:teuthology.orchestra.run.smithi080.stdout:} 2024-04-24T23:32:11.803 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-04-24T23:32:11.803 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-04-24T23:32:11.803 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.g 2024-04-24T23:32:11.803 DEBUG:tasks.check_counter:Getting stats from j 2024-04-24T23:32:11.803 DEBUG:teuthology.orchestra.run.smithi080:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid ec7885aa-0290-11ef-bc93-c7b262605968 -- ceph daemon mds.j perf dump 2024-04-24T23:32:12.047 INFO:teuthology.orchestra.run.smithi080.stderr:Inferring config /var/lib/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:32:12.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:32:12 smithi134 ceph-mon[32619]: pgmap v446: 129 pgs: 129 active+clean; 74 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 551 KiB/s wr, 76 op/s 2024-04-24T23:32:12.466 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:32:12 smithi080 ceph-mon[29152]: pgmap v446: 129 pgs: 129 active+clean; 74 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 551 KiB/s wr, 76 op/s 2024-04-24T23:32:12.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:32:12 smithi148 ceph-mon[32528]: pgmap v446: 129 pgs: 129 active+clean; 74 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 551 KiB/s wr, 76 op/s 2024-04-24T23:32:13.962 INFO:teuthology.orchestra.run.smithi080.stdout:{ 2024-04-24T23:32:13.962 INFO:teuthology.orchestra.run.smithi080.stdout: "AsyncMessenger::Worker-0": { 2024-04-24T23:32:13.962 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_recv_messages": 4, 2024-04-24T23:32:13.962 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_messages": 4, 2024-04-24T23:32:13.963 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_recv_bytes": 2992, 2024-04-24T23:32:13.963 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_bytes": 448, 2024-04-24T23:32:13.963 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_created_connections": 4, 2024-04-24T23:32:13.963 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_active_connections": 0, 2024-04-24T23:32:13.963 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_total_time": 0.001373174, 2024-04-24T23:32:13.963 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_send_time": 0.000033908, 2024-04-24T23:32:13.963 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_recv_time": 0.000537040, 2024-04-24T23:32:13.964 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-24T23:32:13.964 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:32:13.964 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 3, 2024-04-24T23:32:13.964 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000664176, 2024-04-24T23:32:13.964 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000221392 2024-04-24T23:32:13.964 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:13.964 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:32:13.965 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-04-24T23:32:13.965 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-04-24T23:32:13.965 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-04-24T23:32:13.965 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:13.965 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-04-24T23:32:13.965 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_encrypted_bytes": 448 2024-04-24T23:32:13.966 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:13.966 INFO:teuthology.orchestra.run.smithi080.stdout: "AsyncMessenger::Worker-1": { 2024-04-24T23:32:13.966 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_recv_messages": 94, 2024-04-24T23:32:13.966 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_messages": 91, 2024-04-24T23:32:13.966 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_recv_bytes": 442144, 2024-04-24T23:32:13.966 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_bytes": 54752, 2024-04-24T23:32:13.966 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_created_connections": 2, 2024-04-24T23:32:13.967 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_active_connections": 1, 2024-04-24T23:32:13.967 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_total_time": 0.014741808, 2024-04-24T23:32:13.967 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_send_time": 0.005490692, 2024-04-24T23:32:13.967 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_recv_time": 0.009795899, 2024-04-24T23:32:13.967 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_fast_dispatch_time": 0.001617101, 2024-04-24T23:32:13.967 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:32:13.967 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 90, 2024-04-24T23:32:13.968 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.004659562, 2024-04-24T23:32:13.968 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000051772 2024-04-24T23:32:13.968 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:13.968 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:32:13.968 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-04-24T23:32:13.968 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-04-24T23:32:13.969 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-04-24T23:32:13.969 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:13.969 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_recv_encrypted_bytes": 442144, 2024-04-24T23:32:13.969 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_encrypted_bytes": 54752 2024-04-24T23:32:13.969 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:13.969 INFO:teuthology.orchestra.run.smithi080.stdout: "AsyncMessenger::Worker-2": { 2024-04-24T23:32:13.969 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_recv_messages": 1, 2024-04-24T23:32:13.969 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_messages": 69, 2024-04-24T23:32:13.969 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_recv_bytes": 160, 2024-04-24T23:32:13.970 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_bytes": 89696, 2024-04-24T23:32:13.970 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_created_connections": 3, 2024-04-24T23:32:13.970 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_active_connections": 2, 2024-04-24T23:32:13.970 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_total_time": 0.005514695, 2024-04-24T23:32:13.970 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_send_time": 0.003513527, 2024-04-24T23:32:13.970 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_recv_time": 0.001341456, 2024-04-24T23:32:13.970 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-24T23:32:13.970 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:32:13.971 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 67, 2024-04-24T23:32:13.971 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.003304394, 2024-04-24T23:32:13.971 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000049319 2024-04-24T23:32:13.971 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:13.971 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:32:13.972 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-04-24T23:32:13.972 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-04-24T23:32:13.972 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-04-24T23:32:13.972 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:13.972 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-24T23:32:13.972 INFO:teuthology.orchestra.run.smithi080.stdout: "msgr_send_encrypted_bytes": 89696 2024-04-24T23:32:13.972 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:13.973 INFO:teuthology.orchestra.run.smithi080.stdout: "cct": { 2024-04-24T23:32:13.973 INFO:teuthology.orchestra.run.smithi080.stdout: "total_workers": 0, 2024-04-24T23:32:13.973 INFO:teuthology.orchestra.run.smithi080.stdout: "unhealthy_workers": 0 2024-04-24T23:32:13.973 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:13.973 INFO:teuthology.orchestra.run.smithi080.stdout: "mempool": { 2024-04-24T23:32:13.973 INFO:teuthology.orchestra.run.smithi080.stdout: "bloom_filter_bytes": 0, 2024-04-24T23:32:13.973 INFO:teuthology.orchestra.run.smithi080.stdout: "bloom_filter_items": 0, 2024-04-24T23:32:13.973 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_alloc_bytes": 0, 2024-04-24T23:32:13.973 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_alloc_items": 0, 2024-04-24T23:32:13.974 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_cache_data_bytes": 0, 2024-04-24T23:32:13.974 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_cache_data_items": 0, 2024-04-24T23:32:13.974 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-24T23:32:13.974 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_cache_onode_items": 0, 2024-04-24T23:32:13.974 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-24T23:32:13.975 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_cache_meta_items": 0, 2024-04-24T23:32:13.975 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_cache_other_bytes": 0, 2024-04-24T23:32:13.975 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_cache_other_items": 0, 2024-04-24T23:32:13.975 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-24T23:32:13.975 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_cache_buffer_items": 0, 2024-04-24T23:32:13.975 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_extent_bytes": 0, 2024-04-24T23:32:13.975 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_extent_items": 0, 2024-04-24T23:32:13.975 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_blob_bytes": 0, 2024-04-24T23:32:13.976 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_blob_items": 0, 2024-04-24T23:32:13.976 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-24T23:32:13.976 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_shared_blob_items": 0, 2024-04-24T23:32:13.976 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-24T23:32:13.976 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_inline_bl_items": 0, 2024-04-24T23:32:13.976 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_fsck_bytes": 0, 2024-04-24T23:32:13.976 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_fsck_items": 0, 2024-04-24T23:32:13.976 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_txc_bytes": 0, 2024-04-24T23:32:13.977 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_txc_items": 0, 2024-04-24T23:32:13.977 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-24T23:32:13.977 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_writing_deferred_items": 0, 2024-04-24T23:32:13.977 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_writing_bytes": 0, 2024-04-24T23:32:13.977 INFO:teuthology.orchestra.run.smithi080.stdout: "bluestore_writing_items": 0, 2024-04-24T23:32:13.977 INFO:teuthology.orchestra.run.smithi080.stdout: "bluefs_bytes": 0, 2024-04-24T23:32:13.977 INFO:teuthology.orchestra.run.smithi080.stdout: "bluefs_items": 0, 2024-04-24T23:32:13.978 INFO:teuthology.orchestra.run.smithi080.stdout: "bluefs_file_reader_bytes": 0, 2024-04-24T23:32:13.978 INFO:teuthology.orchestra.run.smithi080.stdout: "bluefs_file_reader_items": 0, 2024-04-24T23:32:13.978 INFO:teuthology.orchestra.run.smithi080.stdout: "bluefs_file_writer_bytes": 0, 2024-04-24T23:32:13.978 INFO:teuthology.orchestra.run.smithi080.stdout: "bluefs_file_writer_items": 0, 2024-04-24T23:32:13.978 INFO:teuthology.orchestra.run.smithi080.stdout: "buffer_anon_bytes": 148823, 2024-04-24T23:32:13.978 INFO:teuthology.orchestra.run.smithi080.stdout: "buffer_anon_items": 27, 2024-04-24T23:32:13.979 INFO:teuthology.orchestra.run.smithi080.stdout: "buffer_meta_bytes": 0, 2024-04-24T23:32:13.979 INFO:teuthology.orchestra.run.smithi080.stdout: "buffer_meta_items": 0, 2024-04-24T23:32:13.979 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_bytes": 0, 2024-04-24T23:32:13.979 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_items": 0, 2024-04-24T23:32:13.979 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_mapbl_bytes": 0, 2024-04-24T23:32:13.979 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_mapbl_items": 0, 2024-04-24T23:32:13.979 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_pglog_bytes": 0, 2024-04-24T23:32:13.979 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_pglog_items": 0, 2024-04-24T23:32:13.979 INFO:teuthology.orchestra.run.smithi080.stdout: "osdmap_bytes": 0, 2024-04-24T23:32:13.980 INFO:teuthology.orchestra.run.smithi080.stdout: "osdmap_items": 0, 2024-04-24T23:32:13.980 INFO:teuthology.orchestra.run.smithi080.stdout: "osdmap_mapping_bytes": 0, 2024-04-24T23:32:13.980 INFO:teuthology.orchestra.run.smithi080.stdout: "osdmap_mapping_items": 0, 2024-04-24T23:32:13.980 INFO:teuthology.orchestra.run.smithi080.stdout: "pgmap_bytes": 0, 2024-04-24T23:32:13.980 INFO:teuthology.orchestra.run.smithi080.stdout: "pgmap_items": 0, 2024-04-24T23:32:13.980 INFO:teuthology.orchestra.run.smithi080.stdout: "mds_co_bytes": 928, 2024-04-24T23:32:13.980 INFO:teuthology.orchestra.run.smithi080.stdout: "mds_co_items": 2, 2024-04-24T23:32:13.981 INFO:teuthology.orchestra.run.smithi080.stdout: "unittest_1_bytes": 0, 2024-04-24T23:32:13.981 INFO:teuthology.orchestra.run.smithi080.stdout: "unittest_1_items": 0, 2024-04-24T23:32:13.981 INFO:teuthology.orchestra.run.smithi080.stdout: "unittest_2_bytes": 0, 2024-04-24T23:32:13.981 INFO:teuthology.orchestra.run.smithi080.stdout: "unittest_2_items": 0 2024-04-24T23:32:13.981 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-24T23:32:13.981 INFO:teuthology.orchestra.run.smithi080.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-24T23:32:13.982 INFO:teuthology.orchestra.run.smithi080.stdout: "val": 0, 2024-04-24T23:32:13.982 INFO:teuthology.orchestra.run.smithi080.stdout: "max": 104857600, 2024-04-24T23:32:13.982 INFO:teuthology.orchestra.run.smithi080.stdout: "get_started": 0, 2024-04-24T23:32:13.982 INFO:teuthology.orchestra.run.smithi080.stdout: "get": 95, 2024-04-24T23:32:13.982 INFO:teuthology.orchestra.run.smithi080.stdout: "get_sum": 428896, 2024-04-24T23:32:13.982 INFO:teuthology.orchestra.run.smithi080.stdout: "get_or_fail_fail": 0, 2024-04-24T23:32:13.982 INFO:teuthology.orchestra.run.smithi080.stdout: "get_or_fail_success": 95, 2024-04-24T23:32:13.983 INFO:teuthology.orchestra.run.smithi080.stdout: "take": 0, 2024-04-24T23:32:13.983 INFO:teuthology.orchestra.run.smithi080.stdout: "take_sum": 0, 2024-04-24T23:32:13.983 INFO:teuthology.orchestra.run.smithi080.stdout: "put": 95, 2024-04-24T23:32:13.983 INFO:teuthology.orchestra.run.smithi080.stdout: "put_sum": 428896, 2024-04-24T23:32:13.983 INFO:teuthology.orchestra.run.smithi080.stdout: "wait": { 2024-04-24T23:32:13.983 INFO:teuthology.orchestra.run.smithi080.stdout: "avgcount": 0, 2024-04-24T23:32:13.983 INFO:teuthology.orchestra.run.smithi080.stdout: "sum": 0.000000000, 2024-04-24T23:32:13.983 INFO:teuthology.orchestra.run.smithi080.stdout: "avgtime": 0.000000000 2024-04-24T23:32:13.984 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:13.984 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-24T23:32:13.984 INFO:teuthology.orchestra.run.smithi080.stdout:} 2024-04-24T23:32:14.246 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:32:14 smithi080 ceph-mon[29152]: pgmap v447: 129 pgs: 129 active+clean; 74 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 548 KiB/s wr, 74 op/s 2024-04-24T23:32:14.446 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:32:14 smithi134 ceph-mon[32619]: pgmap v447: 129 pgs: 129 active+clean; 74 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 548 KiB/s wr, 74 op/s 2024-04-24T23:32:14.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:32:14 smithi148 ceph-mon[32528]: pgmap v447: 129 pgs: 129 active+clean; 74 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 548 KiB/s wr, 74 op/s 2024-04-24T23:32:14.594 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-04-24T23:32:14.594 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-04-24T23:32:14.594 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.j 2024-04-24T23:32:14.594 DEBUG:tasks.check_counter:Getting stats from b 2024-04-24T23:32:14.595 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid ec7885aa-0290-11ef-bc93-c7b262605968 -- ceph daemon mds.b perf dump 2024-04-24T23:32:14.850 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/mon.b/config 2024-04-24T23:32:16.307 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:32:16 smithi134 ceph-mon[32619]: pgmap v448: 129 pgs: 129 active+clean; 74 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 550 KiB/s wr, 78 op/s 2024-04-24T23:32:16.503 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:32:16 smithi080 ceph-mon[29152]: pgmap v448: 129 pgs: 129 active+clean; 74 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 550 KiB/s wr, 78 op/s 2024-04-24T23:32:16.505 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:32:16 smithi148 ceph-mon[32528]: pgmap v448: 129 pgs: 129 active+clean; 74 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 550 KiB/s wr, 78 op/s 2024-04-24T23:32:16.721 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-04-24T23:32:16.722 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-0": { 2024-04-24T23:32:16.722 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 2171, 2024-04-24T23:32:16.722 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 4062, 2024-04-24T23:32:16.722 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 3093861, 2024-04-24T23:32:16.722 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 5893965, 2024-04-24T23:32:16.722 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 13, 2024-04-24T23:32:16.723 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 5, 2024-04-24T23:32:16.723 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.177232423, 2024-04-24T23:32:16.723 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.075411854, 2024-04-24T23:32:16.723 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.091813262, 2024-04-24T23:32:16.723 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.014998267, 2024-04-24T23:32:16.723 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:32:16.723 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 4060, 2024-04-24T23:32:16.723 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.114014673, 2024-04-24T23:32:16.724 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000028082 2024-04-24T23:32:16.724 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:16.724 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:32:16.724 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 4976, 2024-04-24T23:32:16.724 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000936100, 2024-04-24T23:32:16.724 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000188 2024-04-24T23:32:16.724 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:16.724 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-24T23:32:16.725 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 0 2024-04-24T23:32:16.725 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:16.725 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-1": { 2024-04-24T23:32:16.725 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 3363, 2024-04-24T23:32:16.725 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 6648, 2024-04-24T23:32:16.725 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 1627202, 2024-04-24T23:32:16.725 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 3663682, 2024-04-24T23:32:16.725 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 17, 2024-04-24T23:32:16.726 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 7, 2024-04-24T23:32:16.726 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.254539295, 2024-04-24T23:32:16.726 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.115906419, 2024-04-24T23:32:16.726 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.170578259, 2024-04-24T23:32:16.726 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.009360984, 2024-04-24T23:32:16.726 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:32:16.726 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 6645, 2024-04-24T23:32:16.726 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.231565300, 2024-04-24T23:32:16.727 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000034848 2024-04-24T23:32:16.727 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:16.727 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:32:16.727 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 7821, 2024-04-24T23:32:16.727 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.002209924, 2024-04-24T23:32:16.727 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000282 2024-04-24T23:32:16.727 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:16.727 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 1536912, 2024-04-24T23:32:16.727 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 3452448 2024-04-24T23:32:16.728 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:16.728 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-2": { 2024-04-24T23:32:16.728 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 740, 2024-04-24T23:32:16.728 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 799, 2024-04-24T23:32:16.728 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 249853, 2024-04-24T23:32:16.729 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 605443, 2024-04-24T23:32:16.729 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 8, 2024-04-24T23:32:16.729 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 5, 2024-04-24T23:32:16.729 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.050163821, 2024-04-24T23:32:16.729 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.016635892, 2024-04-24T23:32:16.729 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.017358439, 2024-04-24T23:32:16.729 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.012672788, 2024-04-24T23:32:16.730 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:32:16.730 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 800, 2024-04-24T23:32:16.730 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.038723647, 2024-04-24T23:32:16.730 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000048404 2024-04-24T23:32:16.730 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:16.730 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:32:16.730 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 145, 2024-04-24T23:32:16.730 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000020986, 2024-04-24T23:32:16.731 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000144 2024-04-24T23:32:16.731 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:16.731 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 3152, 2024-04-24T23:32:16.731 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 185872 2024-04-24T23:32:16.731 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:16.731 INFO:teuthology.orchestra.run.smithi134.stdout: "cct": { 2024-04-24T23:32:16.731 INFO:teuthology.orchestra.run.smithi134.stdout: "total_workers": 1, 2024-04-24T23:32:16.731 INFO:teuthology.orchestra.run.smithi134.stdout: "unhealthy_workers": 0 2024-04-24T23:32:16.732 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:16.732 INFO:teuthology.orchestra.run.smithi134.stdout: "finisher-MDSRank": { 2024-04-24T23:32:16.732 INFO:teuthology.orchestra.run.smithi134.stdout: "queue_len": 0, 2024-04-24T23:32:16.732 INFO:teuthology.orchestra.run.smithi134.stdout: "complete_latency": { 2024-04-24T23:32:16.732 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 446, 2024-04-24T23:32:16.732 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 1.088000988, 2024-04-24T23:32:16.732 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.002439464 2024-04-24T23:32:16.732 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-24T23:32:16.732 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:16.733 INFO:teuthology.orchestra.run.smithi134.stdout: "finisher-PurgeQueue": { 2024-04-24T23:32:16.733 INFO:teuthology.orchestra.run.smithi134.stdout: "queue_len": 0, 2024-04-24T23:32:16.733 INFO:teuthology.orchestra.run.smithi134.stdout: "complete_latency": { 2024-04-24T23:32:16.733 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 603, 2024-04-24T23:32:16.733 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.055069099, 2024-04-24T23:32:16.733 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000091325 2024-04-24T23:32:16.733 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-24T23:32:16.733 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:16.734 INFO:teuthology.orchestra.run.smithi134.stdout: "mds": { 2024-04-24T23:32:16.734 INFO:teuthology.orchestra.run.smithi134.stdout: "request": 1212, 2024-04-24T23:32:16.734 INFO:teuthology.orchestra.run.smithi134.stdout: "reply": 1209, 2024-04-24T23:32:16.734 INFO:teuthology.orchestra.run.smithi134.stdout: "reply_latency": { 2024-04-24T23:32:16.734 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 1209, 2024-04-24T23:32:16.734 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 1.252116930, 2024-04-24T23:32:16.734 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.001035663 2024-04-24T23:32:16.734 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:16.735 INFO:teuthology.orchestra.run.smithi134.stdout: "slow_reply": 0, 2024-04-24T23:32:16.735 INFO:teuthology.orchestra.run.smithi134.stdout: "forward": 3, 2024-04-24T23:32:16.735 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_fetch_complete": 0, 2024-04-24T23:32:16.735 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_fetch_keys": 0, 2024-04-24T23:32:16.735 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_commit": 11, 2024-04-24T23:32:16.735 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_split": 0, 2024-04-24T23:32:16.735 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_merge": 0, 2024-04-24T23:32:16.735 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes": 3260, 2024-04-24T23:32:16.736 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_top": 193, 2024-04-24T23:32:16.736 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_bottom": 3067, 2024-04-24T23:32:16.736 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_pin_tail": 0, 2024-04-24T23:32:16.736 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_pinned": 2970, 2024-04-24T23:32:16.736 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_expired": 0, 2024-04-24T23:32:16.736 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_with_caps": 2012, 2024-04-24T23:32:16.736 INFO:teuthology.orchestra.run.smithi134.stdout: "caps": 2012, 2024-04-24T23:32:16.736 INFO:teuthology.orchestra.run.smithi134.stdout: "subtrees": 75, 2024-04-24T23:32:16.736 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse": 2452, 2024-04-24T23:32:16.737 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_hit": 1490, 2024-04-24T23:32:16.737 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_forward": 3, 2024-04-24T23:32:16.737 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_discover": 15, 2024-04-24T23:32:16.737 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_dir_fetch": 0, 2024-04-24T23:32:16.737 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_remote_ino": 0, 2024-04-24T23:32:16.737 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_lock": 0, 2024-04-24T23:32:16.737 INFO:teuthology.orchestra.run.smithi134.stdout: "load_cent": 7744, 2024-04-24T23:32:16.737 INFO:teuthology.orchestra.run.smithi134.stdout: "q": 0, 2024-04-24T23:32:16.738 INFO:teuthology.orchestra.run.smithi134.stdout: "exported": 0, 2024-04-24T23:32:16.738 INFO:teuthology.orchestra.run.smithi134.stdout: "exported_inodes": 0, 2024-04-24T23:32:16.738 INFO:teuthology.orchestra.run.smithi134.stdout: "imported": 49, 2024-04-24T23:32:16.738 INFO:teuthology.orchestra.run.smithi134.stdout: "imported_inodes": 3162, 2024-04-24T23:32:16.738 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_dir_fetch": 0, 2024-04-24T23:32:16.738 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_backtrace_fetch": 659, 2024-04-24T23:32:16.738 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_peer_discover": 0, 2024-04-24T23:32:16.738 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rfiles": 3429, 2024-04-24T23:32:16.739 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rbytes": 23106296, 2024-04-24T23:32:16.739 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rsnaps": 0, 2024-04-24T23:32:16.739 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_backtrace_fetch": 659, 2024-04-24T23:32:16.739 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_set_tag": 0, 2024-04-24T23:32:16.739 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_backtrace_repaired": 0, 2024-04-24T23:32:16.739 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_inotable_repaired": 0, 2024-04-24T23:32:16.739 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dir_inodes": 16, 2024-04-24T23:32:16.739 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dir_base_inodes": 0, 2024-04-24T23:32:16.739 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dirfrag_rstats": 16, 2024-04-24T23:32:16.740 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_file_inodes": 643, 2024-04-24T23:32:16.740 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_inode_file_caps": 0, 2024-04-24T23:32:16.740 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_revoke": 119, 2024-04-24T23:32:16.740 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_grant": 766, 2024-04-24T23:32:16.740 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_trunc": 0, 2024-04-24T23:32:16.740 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-24T23:32:16.740 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-24T23:32:16.740 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_caps": 1616, 2024-04-24T23:32:16.741 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_caps_dirty": 1146, 2024-04-24T23:32:16.741 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_cap_release": 33, 2024-04-24T23:32:16.741 INFO:teuthology.orchestra.run.smithi134.stdout: "process_request_cap_release": 735 2024-04-24T23:32:16.741 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:16.741 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_cache": { 2024-04-24T23:32:16.741 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_update": 0, 2024-04-24T23:32:16.741 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_update_receipt": 0, 2024-04-24T23:32:16.741 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_try_discover": 0, 2024-04-24T23:32:16.742 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_send_discover": 140, 2024-04-24T23:32:16.742 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_handle_discover": 0, 2024-04-24T23:32:16.742 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays": 0, 2024-04-24T23:32:16.742 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays_delayed": 0, 2024-04-24T23:32:16.742 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays_enqueuing": 0, 2024-04-24T23:32:16.742 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_created": 352, 2024-04-24T23:32:16.742 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_enqueued": 352, 2024-04-24T23:32:16.742 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_reintegrated": 0, 2024-04-24T23:32:16.742 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_migrated": 0, 2024-04-24T23:32:16.743 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_processing": 0, 2024-04-24T23:32:16.743 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_enqueued": 0, 2024-04-24T23:32:16.743 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_prioritized": 0, 2024-04-24T23:32:16.743 INFO:teuthology.orchestra.run.smithi134.stdout: "recovery_started": 0, 2024-04-24T23:32:16.743 INFO:teuthology.orchestra.run.smithi134.stdout: "recovery_completed": 0, 2024-04-24T23:32:16.743 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_quiesce_path": 0, 2024-04-24T23:32:16.743 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_quiesce_inode": 0, 2024-04-24T23:32:16.743 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_enqueue_scrub": 0, 2024-04-24T23:32:16.744 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_exportdir": 0, 2024-04-24T23:32:16.744 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_flush": 0, 2024-04-24T23:32:16.744 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_fragmentdir": 0, 2024-04-24T23:32:16.744 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_fragstats": 0, 2024-04-24T23:32:16.744 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_inodestats": 0 2024-04-24T23:32:16.744 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:16.744 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_log": { 2024-04-24T23:32:16.744 INFO:teuthology.orchestra.run.smithi134.stdout: "evlrg": 1, 2024-04-24T23:32:16.744 INFO:teuthology.orchestra.run.smithi134.stdout: "evadd": 2951, 2024-04-24T23:32:16.745 INFO:teuthology.orchestra.run.smithi134.stdout: "evex": 0, 2024-04-24T23:32:16.745 INFO:teuthology.orchestra.run.smithi134.stdout: "evtrm": 0, 2024-04-24T23:32:16.745 INFO:teuthology.orchestra.run.smithi134.stdout: "ev": 2951, 2024-04-24T23:32:16.745 INFO:teuthology.orchestra.run.smithi134.stdout: "evexg": 0, 2024-04-24T23:32:16.745 INFO:teuthology.orchestra.run.smithi134.stdout: "evexd": 0, 2024-04-24T23:32:16.745 INFO:teuthology.orchestra.run.smithi134.stdout: "segadd": 5, 2024-04-24T23:32:16.745 INFO:teuthology.orchestra.run.smithi134.stdout: "segex": 0, 2024-04-24T23:32:16.745 INFO:teuthology.orchestra.run.smithi134.stdout: "segtrm": 0, 2024-04-24T23:32:16.746 INFO:teuthology.orchestra.run.smithi134.stdout: "seg": 5, 2024-04-24T23:32:16.746 INFO:teuthology.orchestra.run.smithi134.stdout: "segmjr": 2, 2024-04-24T23:32:16.746 INFO:teuthology.orchestra.run.smithi134.stdout: "segexg": 0, 2024-04-24T23:32:16.746 INFO:teuthology.orchestra.run.smithi134.stdout: "segexd": 0, 2024-04-24T23:32:16.746 INFO:teuthology.orchestra.run.smithi134.stdout: "expos": 4194304, 2024-04-24T23:32:16.746 INFO:teuthology.orchestra.run.smithi134.stdout: "wrpos": 9927977, 2024-04-24T23:32:16.746 INFO:teuthology.orchestra.run.smithi134.stdout: "rdpos": 0, 2024-04-24T23:32:16.746 INFO:teuthology.orchestra.run.smithi134.stdout: "jlat": { 2024-04-24T23:32:16.746 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 86, 2024-04-24T23:32:16.747 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 1.383409423, 2024-04-24T23:32:16.747 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.016086156 2024-04-24T23:32:16.747 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:16.747 INFO:teuthology.orchestra.run.smithi134.stdout: "replayed": 0 2024-04-24T23:32:16.747 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:16.747 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_mem": { 2024-04-24T23:32:16.747 INFO:teuthology.orchestra.run.smithi134.stdout: "ino": 2234, 2024-04-24T23:32:16.747 INFO:teuthology.orchestra.run.smithi134.stdout: "ino+": 2586, 2024-04-24T23:32:16.748 INFO:teuthology.orchestra.run.smithi134.stdout: "ino-": 352, 2024-04-24T23:32:16.748 INFO:teuthology.orchestra.run.smithi134.stdout: "dir": 221, 2024-04-24T23:32:16.748 INFO:teuthology.orchestra.run.smithi134.stdout: "dir+": 222, 2024-04-24T23:32:16.748 INFO:teuthology.orchestra.run.smithi134.stdout: "dir-": 1, 2024-04-24T23:32:16.748 INFO:teuthology.orchestra.run.smithi134.stdout: "dn": 3260, 2024-04-24T23:32:16.748 INFO:teuthology.orchestra.run.smithi134.stdout: "dn+": 3638, 2024-04-24T23:32:16.748 INFO:teuthology.orchestra.run.smithi134.stdout: "dn-": 378, 2024-04-24T23:32:16.748 INFO:teuthology.orchestra.run.smithi134.stdout: "cap": 1828, 2024-04-24T23:32:16.748 INFO:teuthology.orchestra.run.smithi134.stdout: "cap+": 2564, 2024-04-24T23:32:16.749 INFO:teuthology.orchestra.run.smithi134.stdout: "cap-": 736, 2024-04-24T23:32:16.749 INFO:teuthology.orchestra.run.smithi134.stdout: "rss": 76368, 2024-04-24T23:32:16.749 INFO:teuthology.orchestra.run.smithi134.stdout: "heap": 223612 2024-04-24T23:32:16.749 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:16.749 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_server": { 2024-04-24T23:32:16.749 INFO:teuthology.orchestra.run.smithi134.stdout: "dispatch_client_request": 1946, 2024-04-24T23:32:16.749 INFO:teuthology.orchestra.run.smithi134.stdout: "dispatch_server_request": 0, 2024-04-24T23:32:16.749 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_request": 1212, 2024-04-24T23:32:16.749 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_session": 6, 2024-04-24T23:32:16.750 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_peer_request": 707, 2024-04-24T23:32:16.750 INFO:teuthology.orchestra.run.smithi134.stdout: "req_create_latency": { 2024-04-24T23:32:16.750 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 355, 2024-04-24T23:32:16.750 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.428074123, 2024-04-24T23:32:16.750 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.001205842 2024-04-24T23:32:16.750 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:16.750 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getattr_latency": { 2024-04-24T23:32:16.750 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-24T23:32:16.751 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-24T23:32:16.751 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-24T23:32:16.751 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:16.751 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getfilelock_latency": { 2024-04-24T23:32:16.751 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-24T23:32:16.751 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-24T23:32:16.751 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-24T23:32:16.751 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:16.751 INFO:teuthology.orchestra.run.smithi134.stdout: "req_link_latency": { 2024-04-24T23:32:16.752 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-24T23:32:16.752 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-24T23:32:16.752 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-24T23:32:16.752 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:16.752 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookup_latency": { 2024-04-24T23:32:16.752 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 484, 2024-04-24T23:32:16.752 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.135861206, 2024-04-24T23:32:16.752 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000280704 2024-04-24T23:32:16.753 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:16.753 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookuphash_latency": { 2024-04-24T23:32:16.753 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-24T23:32:16.753 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-24T23:32:16.753 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-24T23:32:16.753 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:16.753 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupino_latency": { 2024-04-24T23:32:16.753 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-24T23:32:16.754 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-24T23:32:16.754 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-24T23:32:16.754 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:16.754 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupname_latency": { 2024-04-24T23:32:16.754 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-24T23:32:16.754 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-24T23:32:16.754 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-24T23:32:16.754 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:16.755 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupparent_latency": { 2024-04-24T23:32:16.755 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-24T23:32:16.755 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-24T23:32:16.755 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-24T23:32:16.755 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:16.755 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupsnap_latency": { 2024-04-24T23:32:16.755 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-24T23:32:16.755 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-24T23:32:16.755 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-24T23:32:16.756 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:16.756 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lssnap_latency": { 2024-04-24T23:32:16.756 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-24T23:32:16.756 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-24T23:32:16.757 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-24T23:32:16.757 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:16.757 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mkdir_latency": { 2024-04-24T23:32:16.757 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-24T23:32:16.757 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-24T23:32:16.757 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-24T23:32:16.757 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:16.757 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mknod_latency": { 2024-04-24T23:32:16.758 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-24T23:32:16.758 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-24T23:32:16.758 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-24T23:32:16.758 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:16.758 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mksnap_latency": { 2024-04-24T23:32:16.758 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-24T23:32:16.758 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-24T23:32:16.759 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-24T23:32:16.759 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:16.759 INFO:teuthology.orchestra.run.smithi134.stdout: "req_open_latency": { 2024-04-24T23:32:16.759 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 6, 2024-04-24T23:32:16.759 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.002116332, 2024-04-24T23:32:16.759 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000352722 2024-04-24T23:32:16.759 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:16.759 INFO:teuthology.orchestra.run.smithi134.stdout: "req_readdir_latency": { 2024-04-24T23:32:16.759 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 11, 2024-04-24T23:32:16.760 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.011549309, 2024-04-24T23:32:16.760 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.001049937 2024-04-24T23:32:16.760 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:16.760 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rename_latency": { 2024-04-24T23:32:16.760 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 3, 2024-04-24T23:32:16.760 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.008142834, 2024-04-24T23:32:16.760 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.002714278 2024-04-24T23:32:16.760 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:16.761 INFO:teuthology.orchestra.run.smithi134.stdout: "req_renamesnap_latency": { 2024-04-24T23:32:16.761 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-24T23:32:16.761 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-24T23:32:16.761 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-24T23:32:16.761 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:16.761 INFO:teuthology.orchestra.run.smithi134.stdout: "req_snapdiff_latency": { 2024-04-24T23:32:16.761 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-24T23:32:16.761 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-24T23:32:16.762 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-24T23:32:16.762 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:16.762 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmdir_latency": { 2024-04-24T23:32:16.762 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-24T23:32:16.762 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-24T23:32:16.762 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-24T23:32:16.762 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:16.762 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmsnap_latency": { 2024-04-24T23:32:16.762 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-24T23:32:16.763 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-24T23:32:16.763 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-24T23:32:16.763 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:16.763 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmxattr_latency": { 2024-04-24T23:32:16.763 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-24T23:32:16.763 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-24T23:32:16.763 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-24T23:32:16.763 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:16.764 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setattr_latency": { 2024-04-24T23:32:16.764 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-24T23:32:16.764 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-24T23:32:16.764 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-24T23:32:16.764 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:16.764 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setdirlayout_latency": { 2024-04-24T23:32:16.764 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-24T23:32:16.764 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-24T23:32:16.765 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-24T23:32:16.765 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:16.765 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setfilelock_latency": { 2024-04-24T23:32:16.765 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-24T23:32:16.765 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-24T23:32:16.765 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-24T23:32:16.765 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:16.765 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setlayout_latency": { 2024-04-24T23:32:16.766 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-24T23:32:16.766 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-24T23:32:16.766 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-24T23:32:16.766 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:16.766 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setxattr_latency": { 2024-04-24T23:32:16.766 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-24T23:32:16.766 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-24T23:32:16.766 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-24T23:32:16.766 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:16.767 INFO:teuthology.orchestra.run.smithi134.stdout: "req_symlink_latency": { 2024-04-24T23:32:16.767 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-24T23:32:16.767 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-24T23:32:16.767 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-24T23:32:16.767 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:16.767 INFO:teuthology.orchestra.run.smithi134.stdout: "req_unlink_latency": { 2024-04-24T23:32:16.767 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 350, 2024-04-24T23:32:16.767 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.666373126, 2024-04-24T23:32:16.768 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.001903923 2024-04-24T23:32:16.768 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:16.768 INFO:teuthology.orchestra.run.smithi134.stdout: "cap_revoke_eviction": 0, 2024-04-24T23:32:16.768 INFO:teuthology.orchestra.run.smithi134.stdout: "cap_acquisition_throttle": 0, 2024-04-24T23:32:16.768 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getvxattr_latency": { 2024-04-24T23:32:16.768 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-24T23:32:16.768 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-24T23:32:16.769 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-24T23:32:16.769 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-24T23:32:16.769 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:16.769 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_sessions": { 2024-04-24T23:32:16.769 INFO:teuthology.orchestra.run.smithi134.stdout: "session_count": 1, 2024-04-24T23:32:16.769 INFO:teuthology.orchestra.run.smithi134.stdout: "session_add": 2, 2024-04-24T23:32:16.769 INFO:teuthology.orchestra.run.smithi134.stdout: "session_remove": 1, 2024-04-24T23:32:16.769 INFO:teuthology.orchestra.run.smithi134.stdout: "sessions_open": 1, 2024-04-24T23:32:16.770 INFO:teuthology.orchestra.run.smithi134.stdout: "sessions_stale": 0, 2024-04-24T23:32:16.770 INFO:teuthology.orchestra.run.smithi134.stdout: "total_load": 930, 2024-04-24T23:32:16.770 INFO:teuthology.orchestra.run.smithi134.stdout: "average_load": 930, 2024-04-24T23:32:16.770 INFO:teuthology.orchestra.run.smithi134.stdout: "avg_session_uptime": 1271, 2024-04-24T23:32:16.770 INFO:teuthology.orchestra.run.smithi134.stdout: "mdthresh_evicted": 0 2024-04-24T23:32:16.770 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:16.770 INFO:teuthology.orchestra.run.smithi134.stdout: "mempool": { 2024-04-24T23:32:16.770 INFO:teuthology.orchestra.run.smithi134.stdout: "bloom_filter_bytes": 0, 2024-04-24T23:32:16.771 INFO:teuthology.orchestra.run.smithi134.stdout: "bloom_filter_items": 0, 2024-04-24T23:32:16.771 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_alloc_bytes": 0, 2024-04-24T23:32:16.771 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_alloc_items": 0, 2024-04-24T23:32:16.771 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_data_bytes": 0, 2024-04-24T23:32:16.771 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_data_items": 0, 2024-04-24T23:32:16.771 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-24T23:32:16.771 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_onode_items": 0, 2024-04-24T23:32:16.771 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-24T23:32:16.771 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_meta_items": 0, 2024-04-24T23:32:16.772 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_other_bytes": 0, 2024-04-24T23:32:16.772 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_other_items": 0, 2024-04-24T23:32:16.772 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-24T23:32:16.772 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_buffer_items": 0, 2024-04-24T23:32:16.772 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_extent_bytes": 0, 2024-04-24T23:32:16.772 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_extent_items": 0, 2024-04-24T23:32:16.772 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_blob_bytes": 0, 2024-04-24T23:32:16.772 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_blob_items": 0, 2024-04-24T23:32:16.773 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-24T23:32:16.773 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_shared_blob_items": 0, 2024-04-24T23:32:16.773 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-24T23:32:16.773 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_inline_bl_items": 0, 2024-04-24T23:32:16.773 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_fsck_bytes": 0, 2024-04-24T23:32:16.773 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_fsck_items": 0, 2024-04-24T23:32:16.773 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_txc_bytes": 0, 2024-04-24T23:32:16.773 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_txc_items": 0, 2024-04-24T23:32:16.773 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-24T23:32:16.774 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_deferred_items": 0, 2024-04-24T23:32:16.774 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_bytes": 0, 2024-04-24T23:32:16.774 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_items": 0, 2024-04-24T23:32:16.774 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_bytes": 0, 2024-04-24T23:32:16.774 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_items": 0, 2024-04-24T23:32:16.774 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_reader_bytes": 0, 2024-04-24T23:32:16.774 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_reader_items": 0, 2024-04-24T23:32:16.774 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_writer_bytes": 0, 2024-04-24T23:32:16.775 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_writer_items": 0, 2024-04-24T23:32:16.775 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_anon_bytes": 417772, 2024-04-24T23:32:16.775 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_anon_items": 2351, 2024-04-24T23:32:16.775 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_meta_bytes": 880, 2024-04-24T23:32:16.775 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_meta_items": 10, 2024-04-24T23:32:16.775 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_bytes": 0, 2024-04-24T23:32:16.775 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_items": 0, 2024-04-24T23:32:16.775 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_mapbl_bytes": 0, 2024-04-24T23:32:16.776 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_mapbl_items": 0, 2024-04-24T23:32:16.776 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_pglog_bytes": 0, 2024-04-24T23:32:16.776 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_pglog_items": 0, 2024-04-24T23:32:16.776 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_bytes": 9080, 2024-04-24T23:32:16.776 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_items": 174, 2024-04-24T23:32:16.776 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_mapping_bytes": 0, 2024-04-24T23:32:16.776 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_mapping_items": 0, 2024-04-24T23:32:16.776 INFO:teuthology.orchestra.run.smithi134.stdout: "pgmap_bytes": 0, 2024-04-24T23:32:16.777 INFO:teuthology.orchestra.run.smithi134.stdout: "pgmap_items": 0, 2024-04-24T23:32:16.777 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_co_bytes": 8691329, 2024-04-24T23:32:16.777 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_co_items": 90187, 2024-04-24T23:32:16.777 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_1_bytes": 0, 2024-04-24T23:32:16.777 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_1_items": 0, 2024-04-24T23:32:16.777 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_2_bytes": 0, 2024-04-24T23:32:16.777 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_2_items": 0 2024-04-24T23:32:16.777 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:16.777 INFO:teuthology.orchestra.run.smithi134.stdout: "objecter": { 2024-04-24T23:32:16.778 INFO:teuthology.orchestra.run.smithi134.stdout: "op_active": 0, 2024-04-24T23:32:16.778 INFO:teuthology.orchestra.run.smithi134.stdout: "op_laggy": 0, 2024-04-24T23:32:16.778 INFO:teuthology.orchestra.run.smithi134.stdout: "op_send": 1528, 2024-04-24T23:32:16.778 INFO:teuthology.orchestra.run.smithi134.stdout: "op_send_bytes": 5924456, 2024-04-24T23:32:16.778 INFO:teuthology.orchestra.run.smithi134.stdout: "op_resend": 33, 2024-04-24T23:32:16.778 INFO:teuthology.orchestra.run.smithi134.stdout: "op_reply": 1528, 2024-04-24T23:32:16.778 INFO:teuthology.orchestra.run.smithi134.stdout: "op_latency": { 2024-04-24T23:32:16.778 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 1528, 2024-04-24T23:32:16.779 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 9.239784786, 2024-04-24T23:32:16.779 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.006046979 2024-04-24T23:32:16.779 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:16.779 INFO:teuthology.orchestra.run.smithi134.stdout: "op_inflight": 0, 2024-04-24T23:32:16.779 INFO:teuthology.orchestra.run.smithi134.stdout: "oplen_avg": { 2024-04-24T23:32:16.779 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 1528, 2024-04-24T23:32:16.779 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 1573 2024-04-24T23:32:16.779 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:16.780 INFO:teuthology.orchestra.run.smithi134.stdout: "op": 1528, 2024-04-24T23:32:16.780 INFO:teuthology.orchestra.run.smithi134.stdout: "op_r": 664, 2024-04-24T23:32:16.780 INFO:teuthology.orchestra.run.smithi134.stdout: "op_w": 864, 2024-04-24T23:32:16.780 INFO:teuthology.orchestra.run.smithi134.stdout: "op_rmw": 0, 2024-04-24T23:32:16.780 INFO:teuthology.orchestra.run.smithi134.stdout: "op_pg": 0, 2024-04-24T23:32:16.780 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_stat": 0, 2024-04-24T23:32:16.780 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_create": 10, 2024-04-24T23:32:16.780 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_read": 5, 2024-04-24T23:32:16.780 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_write": 91, 2024-04-24T23:32:16.781 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_writefull": 13, 2024-04-24T23:32:16.781 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_writesame": 0, 2024-04-24T23:32:16.781 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_append": 0, 2024-04-24T23:32:16.781 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_zero": 0, 2024-04-24T23:32:16.781 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_truncate": 0, 2024-04-24T23:32:16.781 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_delete": 717, 2024-04-24T23:32:16.781 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_mapext": 0, 2024-04-24T23:32:16.781 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_sparse_read": 0, 2024-04-24T23:32:16.782 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_clonerange": 0, 2024-04-24T23:32:16.782 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_getxattr": 659, 2024-04-24T23:32:16.782 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_setxattr": 20, 2024-04-24T23:32:16.782 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_cmpxattr": 0, 2024-04-24T23:32:16.782 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_rmxattr": 0, 2024-04-24T23:32:16.782 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_resetxattrs": 0, 2024-04-24T23:32:16.782 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_call": 0, 2024-04-24T23:32:16.782 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_watch": 0, 2024-04-24T23:32:16.782 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_notify": 0, 2024-04-24T23:32:16.783 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_src_cmpxattr": 0, 2024-04-24T23:32:16.783 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_pgls": 0, 2024-04-24T23:32:16.783 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_pgls_filter": 0, 2024-04-24T23:32:16.783 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_other": 0, 2024-04-24T23:32:16.783 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_active": 0, 2024-04-24T23:32:16.783 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_send": 0, 2024-04-24T23:32:16.783 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_resend": 0, 2024-04-24T23:32:16.783 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_ping": 0, 2024-04-24T23:32:16.784 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_active": 0, 2024-04-24T23:32:16.784 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_send": 0, 2024-04-24T23:32:16.784 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_resend": 0, 2024-04-24T23:32:16.784 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_active": 0, 2024-04-24T23:32:16.784 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_send": 0, 2024-04-24T23:32:16.784 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_resend": 0, 2024-04-24T23:32:16.784 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_active": 0, 2024-04-24T23:32:16.784 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_send": 0, 2024-04-24T23:32:16.784 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_resend": 0, 2024-04-24T23:32:16.785 INFO:teuthology.orchestra.run.smithi134.stdout: "command_active": 0, 2024-04-24T23:32:16.785 INFO:teuthology.orchestra.run.smithi134.stdout: "command_send": 0, 2024-04-24T23:32:16.785 INFO:teuthology.orchestra.run.smithi134.stdout: "command_resend": 0, 2024-04-24T23:32:16.785 INFO:teuthology.orchestra.run.smithi134.stdout: "map_epoch": 94, 2024-04-24T23:32:16.785 INFO:teuthology.orchestra.run.smithi134.stdout: "map_full": 0, 2024-04-24T23:32:16.785 INFO:teuthology.orchestra.run.smithi134.stdout: "map_inc": 4, 2024-04-24T23:32:16.785 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_sessions": 12, 2024-04-24T23:32:16.785 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_session_open": 12, 2024-04-24T23:32:16.786 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_session_close": 0, 2024-04-24T23:32:16.786 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_laggy": 0, 2024-04-24T23:32:16.786 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_wr": 51, 2024-04-24T23:32:16.786 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_rd": 0, 2024-04-24T23:32:16.786 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_del": 7 2024-04-24T23:32:16.786 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:16.786 INFO:teuthology.orchestra.run.smithi134.stdout: "oft": { 2024-04-24T23:32:16.786 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_objs": 1, 2024-04-24T23:32:16.786 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_kv_pairs": 0, 2024-04-24T23:32:16.787 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_updates": 0, 2024-04-24T23:32:16.787 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_removes": 0 2024-04-24T23:32:16.787 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:16.787 INFO:teuthology.orchestra.run.smithi134.stdout: "osd-slow-ops": { 2024-04-24T23:32:16.787 INFO:teuthology.orchestra.run.smithi134.stdout: "slow_ops_count": 0 2024-04-24T23:32:16.787 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:16.787 INFO:teuthology.orchestra.run.smithi134.stdout: "purge_queue": { 2024-04-24T23:32:16.787 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_ops": 0, 2024-04-24T23:32:16.788 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_ops_high_water": 16, 2024-04-24T23:32:16.788 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing": 0, 2024-04-24T23:32:16.788 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_high_water": 8, 2024-04-24T23:32:16.788 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executed_ops": 704, 2024-04-24T23:32:16.788 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executed": 352, 2024-04-24T23:32:16.788 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_item_in_journal": 0 2024-04-24T23:32:16.788 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:16.788 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-24T23:32:16.789 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-04-24T23:32:16.789 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 104857600, 2024-04-24T23:32:16.789 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-04-24T23:32:16.789 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 6270, 2024-04-24T23:32:16.789 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 4268453, 2024-04-24T23:32:16.789 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-04-24T23:32:16.789 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 6270, 2024-04-24T23:32:16.789 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2024-04-24T23:32:16.790 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2024-04-24T23:32:16.790 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 6270, 2024-04-24T23:32:16.790 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 4268453, 2024-04-24T23:32:16.790 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-04-24T23:32:16.790 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-24T23:32:16.790 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-24T23:32:16.790 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-24T23:32:16.790 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-24T23:32:16.791 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:16.791 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-objecter_bytes": { 2024-04-24T23:32:16.791 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-04-24T23:32:16.791 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 104857600, 2024-04-24T23:32:16.791 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-04-24T23:32:16.791 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2024-04-24T23:32:16.791 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2024-04-24T23:32:16.791 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-04-24T23:32:16.792 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2024-04-24T23:32:16.792 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 1528, 2024-04-24T23:32:16.792 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 5962824, 2024-04-24T23:32:16.792 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 1528, 2024-04-24T23:32:16.792 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 5962824, 2024-04-24T23:32:16.792 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-04-24T23:32:16.792 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-24T23:32:16.792 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-24T23:32:16.792 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-24T23:32:16.793 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-24T23:32:16.793 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:16.793 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-objecter_ops": { 2024-04-24T23:32:16.793 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-04-24T23:32:16.793 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 1024, 2024-04-24T23:32:16.793 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-04-24T23:32:16.794 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2024-04-24T23:32:16.794 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2024-04-24T23:32:16.794 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-04-24T23:32:16.794 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2024-04-24T23:32:16.794 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 1528, 2024-04-24T23:32:16.794 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 1528, 2024-04-24T23:32:16.794 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 1528, 2024-04-24T23:32:16.794 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 1528, 2024-04-24T23:32:16.795 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-04-24T23:32:16.795 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-24T23:32:16.795 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-24T23:32:16.795 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-24T23:32:16.795 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-24T23:32:16.795 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:16.795 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-write_buf_throttle": { 2024-04-24T23:32:16.795 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-04-24T23:32:16.796 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 3758096384, 2024-04-24T23:32:16.796 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-04-24T23:32:16.796 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 352, 2024-04-24T23:32:16.796 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 38368, 2024-04-24T23:32:16.796 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-04-24T23:32:16.796 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 352, 2024-04-24T23:32:16.796 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2024-04-24T23:32:16.796 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2024-04-24T23:32:16.796 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 5, 2024-04-24T23:32:16.797 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 38368, 2024-04-24T23:32:16.797 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-04-24T23:32:16.797 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-24T23:32:16.797 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-24T23:32:16.797 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-24T23:32:16.797 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-24T23:32:16.797 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:16.797 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-write_buf_throttle-0x55d451166500": { 2024-04-24T23:32:16.798 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-04-24T23:32:16.798 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 3758096384, 2024-04-24T23:32:16.798 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-04-24T23:32:16.798 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 2951, 2024-04-24T23:32:16.798 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 5733673, 2024-04-24T23:32:16.798 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-04-24T23:32:16.798 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 2951, 2024-04-24T23:32:16.798 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2024-04-24T23:32:16.799 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2024-04-24T23:32:16.799 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 86, 2024-04-24T23:32:16.799 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 5733673, 2024-04-24T23:32:16.799 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-04-24T23:32:16.799 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-24T23:32:16.799 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-24T23:32:16.799 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-24T23:32:16.799 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-24T23:32:16.800 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-24T23:32:16.800 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-04-24T23:32:17.601 INFO:tasks.check_counter:Daemon mds.b mds.exported=0 2024-04-24T23:32:17.601 INFO:tasks.check_counter:Daemon mds.b mds.imported=49 2024-04-24T23:32:17.601 INFO:tasks.check_counter:Daemon mds.b mds.dir_split=0 2024-04-24T23:32:17.601 DEBUG:tasks.check_counter:Getting stats from e 2024-04-24T23:32:17.601 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid ec7885aa-0290-11ef-bc93-c7b262605968 -- ceph daemon mds.e perf dump 2024-04-24T23:32:17.841 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/mon.b/config 2024-04-24T23:32:18.217 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:32:18 smithi134 ceph-mon[32619]: pgmap v449: 129 pgs: 129 active+clean; 74 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 91 KiB/s wr, 24 op/s 2024-04-24T23:32:18.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:32:18 smithi148 ceph-mon[32528]: pgmap v449: 129 pgs: 129 active+clean; 74 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 91 KiB/s wr, 24 op/s 2024-04-24T23:32:18.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:32:18 smithi080 ceph-mon[29152]: pgmap v449: 129 pgs: 129 active+clean; 74 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 91 KiB/s wr, 24 op/s 2024-04-24T23:32:19.560 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-04-24T23:32:19.560 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-0": { 2024-04-24T23:32:19.560 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 107, 2024-04-24T23:32:19.561 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 101, 2024-04-24T23:32:19.561 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 472018, 2024-04-24T23:32:19.561 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 56612, 2024-04-24T23:32:19.561 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 5, 2024-04-24T23:32:19.561 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 1, 2024-04-24T23:32:19.561 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.018807214, 2024-04-24T23:32:19.561 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.006058580, 2024-04-24T23:32:19.562 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.012032656, 2024-04-24T23:32:19.562 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.002153103, 2024-04-24T23:32:19.562 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:32:19.562 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 100, 2024-04-24T23:32:19.562 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.006423626, 2024-04-24T23:32:19.562 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000064236 2024-04-24T23:32:19.562 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:19.562 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:32:19.563 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-24T23:32:19.563 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-24T23:32:19.563 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-24T23:32:19.563 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:19.563 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 471184, 2024-04-24T23:32:19.563 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 56288 2024-04-24T23:32:19.563 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:19.563 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-1": { 2024-04-24T23:32:19.563 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 211, 2024-04-24T23:32:19.564 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 277, 2024-04-24T23:32:19.564 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 1595505, 2024-04-24T23:32:19.564 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 253817, 2024-04-24T23:32:19.564 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 5, 2024-04-24T23:32:19.564 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 4, 2024-04-24T23:32:19.564 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.024408609, 2024-04-24T23:32:19.564 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.008451677, 2024-04-24T23:32:19.565 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.006719603, 2024-04-24T23:32:19.565 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.006789893, 2024-04-24T23:32:19.565 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:32:19.565 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 276, 2024-04-24T23:32:19.565 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.013346849, 2024-04-24T23:32:19.565 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000048358 2024-04-24T23:32:19.565 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:19.565 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:32:19.565 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-24T23:32:19.566 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-24T23:32:19.566 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-24T23:32:19.566 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:19.566 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 3152, 2024-04-24T23:32:19.566 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 188304 2024-04-24T23:32:19.566 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:19.566 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-2": { 2024-04-24T23:32:19.566 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 562, 2024-04-24T23:32:19.567 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 564, 2024-04-24T23:32:19.567 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 4361226, 2024-04-24T23:32:19.567 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 178774, 2024-04-24T23:32:19.567 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 5, 2024-04-24T23:32:19.567 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 4, 2024-04-24T23:32:19.567 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.059940990, 2024-04-24T23:32:19.567 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.019795720, 2024-04-24T23:32:19.567 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.017136341, 2024-04-24T23:32:19.568 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.017875516, 2024-04-24T23:32:19.568 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:32:19.568 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 562, 2024-04-24T23:32:19.568 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.024966688, 2024-04-24T23:32:19.568 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000044424 2024-04-24T23:32:19.568 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:19.568 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:32:19.569 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-24T23:32:19.569 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-24T23:32:19.569 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-24T23:32:19.569 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:19.569 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-24T23:32:19.569 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 0 2024-04-24T23:32:19.569 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:19.569 INFO:teuthology.orchestra.run.smithi134.stdout: "cct": { 2024-04-24T23:32:19.570 INFO:teuthology.orchestra.run.smithi134.stdout: "total_workers": 1, 2024-04-24T23:32:19.570 INFO:teuthology.orchestra.run.smithi134.stdout: "unhealthy_workers": 0 2024-04-24T23:32:19.570 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:19.570 INFO:teuthology.orchestra.run.smithi134.stdout: "finisher-MDSRank": { 2024-04-24T23:32:19.570 INFO:teuthology.orchestra.run.smithi134.stdout: "queue_len": 0, 2024-04-24T23:32:19.570 INFO:teuthology.orchestra.run.smithi134.stdout: "complete_latency": { 2024-04-24T23:32:19.570 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 1312, 2024-04-24T23:32:19.571 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.038640301, 2024-04-24T23:32:19.571 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000029451 2024-04-24T23:32:19.571 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-24T23:32:19.571 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:19.571 INFO:teuthology.orchestra.run.smithi134.stdout: "finisher-PurgeQueue": { 2024-04-24T23:32:19.571 INFO:teuthology.orchestra.run.smithi134.stdout: "queue_len": 0, 2024-04-24T23:32:19.571 INFO:teuthology.orchestra.run.smithi134.stdout: "complete_latency": { 2024-04-24T23:32:19.571 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-24T23:32:19.572 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-24T23:32:19.572 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-24T23:32:19.572 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-24T23:32:19.572 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:19.572 INFO:teuthology.orchestra.run.smithi134.stdout: "mds": { 2024-04-24T23:32:19.572 INFO:teuthology.orchestra.run.smithi134.stdout: "request": 0, 2024-04-24T23:32:19.572 INFO:teuthology.orchestra.run.smithi134.stdout: "reply": 0, 2024-04-24T23:32:19.572 INFO:teuthology.orchestra.run.smithi134.stdout: "reply_latency": { 2024-04-24T23:32:19.572 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-24T23:32:19.573 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-24T23:32:19.573 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-24T23:32:19.573 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:19.573 INFO:teuthology.orchestra.run.smithi134.stdout: "slow_reply": 0, 2024-04-24T23:32:19.573 INFO:teuthology.orchestra.run.smithi134.stdout: "forward": 0, 2024-04-24T23:32:19.573 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_fetch_complete": 0, 2024-04-24T23:32:19.573 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_fetch_keys": 0, 2024-04-24T23:32:19.573 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_commit": 0, 2024-04-24T23:32:19.574 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_split": 0, 2024-04-24T23:32:19.574 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_merge": 0, 2024-04-24T23:32:19.574 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes": 2125, 2024-04-24T23:32:19.574 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_top": 0, 2024-04-24T23:32:19.574 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_bottom": 2125, 2024-04-24T23:32:19.574 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_pin_tail": 0, 2024-04-24T23:32:19.574 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_pinned": 2125, 2024-04-24T23:32:19.575 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_expired": 0, 2024-04-24T23:32:19.575 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_with_caps": 0, 2024-04-24T23:32:19.575 INFO:teuthology.orchestra.run.smithi134.stdout: "caps": 0, 2024-04-24T23:32:19.575 INFO:teuthology.orchestra.run.smithi134.stdout: "subtrees": 75, 2024-04-24T23:32:19.575 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse": 0, 2024-04-24T23:32:19.575 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_hit": 0, 2024-04-24T23:32:19.575 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_forward": 0, 2024-04-24T23:32:19.575 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_discover": 0, 2024-04-24T23:32:19.575 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_dir_fetch": 0, 2024-04-24T23:32:19.576 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_remote_ino": 0, 2024-04-24T23:32:19.576 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_lock": 0, 2024-04-24T23:32:19.576 INFO:teuthology.orchestra.run.smithi134.stdout: "load_cent": 0, 2024-04-24T23:32:19.576 INFO:teuthology.orchestra.run.smithi134.stdout: "q": 0, 2024-04-24T23:32:19.576 INFO:teuthology.orchestra.run.smithi134.stdout: "exported": 0, 2024-04-24T23:32:19.576 INFO:teuthology.orchestra.run.smithi134.stdout: "exported_inodes": 0, 2024-04-24T23:32:19.576 INFO:teuthology.orchestra.run.smithi134.stdout: "imported": 0, 2024-04-24T23:32:19.576 INFO:teuthology.orchestra.run.smithi134.stdout: "imported_inodes": 0, 2024-04-24T23:32:19.577 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_dir_fetch": 0, 2024-04-24T23:32:19.577 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_backtrace_fetch": 0, 2024-04-24T23:32:19.577 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_peer_discover": 0, 2024-04-24T23:32:19.577 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rfiles": 0, 2024-04-24T23:32:19.577 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rbytes": 0, 2024-04-24T23:32:19.577 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rsnaps": 0, 2024-04-24T23:32:19.577 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_backtrace_fetch": 0, 2024-04-24T23:32:19.577 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_set_tag": 0, 2024-04-24T23:32:19.578 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_backtrace_repaired": 0, 2024-04-24T23:32:19.578 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_inotable_repaired": 0, 2024-04-24T23:32:19.578 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dir_inodes": 0, 2024-04-24T23:32:19.578 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dir_base_inodes": 0, 2024-04-24T23:32:19.578 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dirfrag_rstats": 0, 2024-04-24T23:32:19.578 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_file_inodes": 0, 2024-04-24T23:32:19.578 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_inode_file_caps": 0, 2024-04-24T23:32:19.578 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_revoke": 0, 2024-04-24T23:32:19.579 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_grant": 0, 2024-04-24T23:32:19.579 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_trunc": 0, 2024-04-24T23:32:19.579 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-24T23:32:19.579 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-24T23:32:19.579 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_caps": 0, 2024-04-24T23:32:19.579 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_caps_dirty": 0, 2024-04-24T23:32:19.579 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_cap_release": 0, 2024-04-24T23:32:19.580 INFO:teuthology.orchestra.run.smithi134.stdout: "process_request_cap_release": 0 2024-04-24T23:32:19.580 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:19.580 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_cache": { 2024-04-24T23:32:19.580 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_update": 0, 2024-04-24T23:32:19.580 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_update_receipt": 0, 2024-04-24T23:32:19.580 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_try_discover": 0, 2024-04-24T23:32:19.580 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_send_discover": 0, 2024-04-24T23:32:19.580 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_handle_discover": 0, 2024-04-24T23:32:19.580 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays": 0, 2024-04-24T23:32:19.581 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays_delayed": 0, 2024-04-24T23:32:19.581 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays_enqueuing": 0, 2024-04-24T23:32:19.581 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_created": 0, 2024-04-24T23:32:19.581 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_enqueued": 0, 2024-04-24T23:32:19.581 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_reintegrated": 0, 2024-04-24T23:32:19.581 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_migrated": 0, 2024-04-24T23:32:19.581 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_processing": 0, 2024-04-24T23:32:19.581 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_enqueued": 0, 2024-04-24T23:32:19.582 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_prioritized": 0, 2024-04-24T23:32:19.582 INFO:teuthology.orchestra.run.smithi134.stdout: "recovery_started": 0, 2024-04-24T23:32:19.582 INFO:teuthology.orchestra.run.smithi134.stdout: "recovery_completed": 0, 2024-04-24T23:32:19.582 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_quiesce_path": 0, 2024-04-24T23:32:19.582 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_quiesce_inode": 0, 2024-04-24T23:32:19.582 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_enqueue_scrub": 0, 2024-04-24T23:32:19.582 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_exportdir": 0, 2024-04-24T23:32:19.582 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_flush": 0, 2024-04-24T23:32:19.586 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_fragmentdir": 0, 2024-04-24T23:32:19.586 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_fragstats": 0, 2024-04-24T23:32:19.586 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_inodestats": 0 2024-04-24T23:32:19.586 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:19.586 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_log": { 2024-04-24T23:32:19.586 INFO:teuthology.orchestra.run.smithi134.stdout: "evlrg": 0, 2024-04-24T23:32:19.587 INFO:teuthology.orchestra.run.smithi134.stdout: "evadd": 0, 2024-04-24T23:32:19.587 INFO:teuthology.orchestra.run.smithi134.stdout: "evex": 0, 2024-04-24T23:32:19.587 INFO:teuthology.orchestra.run.smithi134.stdout: "evtrm": 0, 2024-04-24T23:32:19.587 INFO:teuthology.orchestra.run.smithi134.stdout: "ev": 3279, 2024-04-24T23:32:19.587 INFO:teuthology.orchestra.run.smithi134.stdout: "evexg": 0, 2024-04-24T23:32:19.587 INFO:teuthology.orchestra.run.smithi134.stdout: "evexd": 0, 2024-04-24T23:32:19.587 INFO:teuthology.orchestra.run.smithi134.stdout: "segadd": 0, 2024-04-24T23:32:19.588 INFO:teuthology.orchestra.run.smithi134.stdout: "segex": 0, 2024-04-24T23:32:19.588 INFO:teuthology.orchestra.run.smithi134.stdout: "segtrm": 0, 2024-04-24T23:32:19.588 INFO:teuthology.orchestra.run.smithi134.stdout: "seg": 6, 2024-04-24T23:32:19.589 INFO:teuthology.orchestra.run.smithi134.stdout: "segmjr": 2, 2024-04-24T23:32:19.589 INFO:teuthology.orchestra.run.smithi134.stdout: "segexg": 0, 2024-04-24T23:32:19.589 INFO:teuthology.orchestra.run.smithi134.stdout: "segexd": 0, 2024-04-24T23:32:19.589 INFO:teuthology.orchestra.run.smithi134.stdout: "expos": 4194304, 2024-04-24T23:32:19.589 INFO:teuthology.orchestra.run.smithi134.stdout: "wrpos": 9929559, 2024-04-24T23:32:19.589 INFO:teuthology.orchestra.run.smithi134.stdout: "rdpos": 9927825, 2024-04-24T23:32:19.590 INFO:teuthology.orchestra.run.smithi134.stdout: "jlat": { 2024-04-24T23:32:19.590 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-24T23:32:19.590 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-24T23:32:19.590 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-24T23:32:19.590 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:19.590 INFO:teuthology.orchestra.run.smithi134.stdout: "replayed": 3279 2024-04-24T23:32:19.590 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:19.591 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_mem": { 2024-04-24T23:32:19.591 INFO:teuthology.orchestra.run.smithi134.stdout: "ino": 1283, 2024-04-24T23:32:19.591 INFO:teuthology.orchestra.run.smithi134.stdout: "ino+": 1706, 2024-04-24T23:32:19.591 INFO:teuthology.orchestra.run.smithi134.stdout: "ino-": 423, 2024-04-24T23:32:19.591 INFO:teuthology.orchestra.run.smithi134.stdout: "dir": 90, 2024-04-24T23:32:19.591 INFO:teuthology.orchestra.run.smithi134.stdout: "dir+": 91, 2024-04-24T23:32:19.591 INFO:teuthology.orchestra.run.smithi134.stdout: "dir-": 1, 2024-04-24T23:32:19.592 INFO:teuthology.orchestra.run.smithi134.stdout: "dn": 2125, 2024-04-24T23:32:19.592 INFO:teuthology.orchestra.run.smithi134.stdout: "dn+": 2138, 2024-04-24T23:32:19.592 INFO:teuthology.orchestra.run.smithi134.stdout: "dn-": 13, 2024-04-24T23:32:19.592 INFO:teuthology.orchestra.run.smithi134.stdout: "cap": 0, 2024-04-24T23:32:19.592 INFO:teuthology.orchestra.run.smithi134.stdout: "cap+": 0, 2024-04-24T23:32:19.592 INFO:teuthology.orchestra.run.smithi134.stdout: "cap-": 0, 2024-04-24T23:32:19.593 INFO:teuthology.orchestra.run.smithi134.stdout: "rss": 57304, 2024-04-24T23:32:19.593 INFO:teuthology.orchestra.run.smithi134.stdout: "heap": 215420 2024-04-24T23:32:19.593 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:19.593 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_server": { 2024-04-24T23:32:19.593 INFO:teuthology.orchestra.run.smithi134.stdout: "dispatch_client_request": 0, 2024-04-24T23:32:19.593 INFO:teuthology.orchestra.run.smithi134.stdout: "dispatch_server_request": 0, 2024-04-24T23:32:19.593 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_request": 0, 2024-04-24T23:32:19.593 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_session": 0, 2024-04-24T23:32:19.594 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_peer_request": 0, 2024-04-24T23:32:19.594 INFO:teuthology.orchestra.run.smithi134.stdout: "req_create_latency": { 2024-04-24T23:32:19.594 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-24T23:32:19.594 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-24T23:32:19.594 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-24T23:32:19.594 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:19.594 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getattr_latency": { 2024-04-24T23:32:19.594 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-24T23:32:19.595 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-24T23:32:19.595 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-24T23:32:19.595 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:19.595 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getfilelock_latency": { 2024-04-24T23:32:19.595 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-24T23:32:19.595 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-24T23:32:19.596 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-24T23:32:19.596 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:19.596 INFO:teuthology.orchestra.run.smithi134.stdout: "req_link_latency": { 2024-04-24T23:32:19.596 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-24T23:32:19.596 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-24T23:32:19.596 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-24T23:32:19.596 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:19.596 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookup_latency": { 2024-04-24T23:32:19.597 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-24T23:32:19.597 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-24T23:32:19.597 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-24T23:32:19.597 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:19.597 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookuphash_latency": { 2024-04-24T23:32:19.597 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-24T23:32:19.597 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-24T23:32:19.597 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-24T23:32:19.598 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:19.598 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupino_latency": { 2024-04-24T23:32:19.598 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-24T23:32:19.598 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-24T23:32:19.598 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-24T23:32:19.598 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:19.599 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupname_latency": { 2024-04-24T23:32:19.599 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-24T23:32:19.599 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-24T23:32:19.599 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-24T23:32:19.599 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:19.599 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupparent_latency": { 2024-04-24T23:32:19.599 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-24T23:32:19.600 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-24T23:32:19.600 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-24T23:32:19.600 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:19.600 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupsnap_latency": { 2024-04-24T23:32:19.600 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-24T23:32:19.600 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-24T23:32:19.600 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-24T23:32:19.600 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:19.601 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lssnap_latency": { 2024-04-24T23:32:19.601 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-24T23:32:19.601 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-24T23:32:19.601 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-24T23:32:19.601 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:19.601 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mkdir_latency": { 2024-04-24T23:32:19.602 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-24T23:32:19.602 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-24T23:32:19.602 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-24T23:32:19.603 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:19.603 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mknod_latency": { 2024-04-24T23:32:19.603 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-24T23:32:19.603 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-24T23:32:19.603 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-24T23:32:19.603 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:19.603 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mksnap_latency": { 2024-04-24T23:32:19.604 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-24T23:32:19.604 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-24T23:32:19.604 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-24T23:32:19.604 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:19.604 INFO:teuthology.orchestra.run.smithi134.stdout: "req_open_latency": { 2024-04-24T23:32:19.604 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-24T23:32:19.604 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-24T23:32:19.605 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-24T23:32:19.605 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:19.605 INFO:teuthology.orchestra.run.smithi134.stdout: "req_readdir_latency": { 2024-04-24T23:32:19.605 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-24T23:32:19.605 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-24T23:32:19.605 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-24T23:32:19.606 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:19.606 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rename_latency": { 2024-04-24T23:32:19.606 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-24T23:32:19.606 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-24T23:32:19.606 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-24T23:32:19.606 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:19.606 INFO:teuthology.orchestra.run.smithi134.stdout: "req_renamesnap_latency": { 2024-04-24T23:32:19.606 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-24T23:32:19.607 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-24T23:32:19.607 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-24T23:32:19.607 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:19.607 INFO:teuthology.orchestra.run.smithi134.stdout: "req_snapdiff_latency": { 2024-04-24T23:32:19.607 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-24T23:32:19.607 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-24T23:32:19.607 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-24T23:32:19.608 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:19.608 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmdir_latency": { 2024-04-24T23:32:19.608 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-24T23:32:19.608 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-24T23:32:19.608 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-24T23:32:19.608 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:19.608 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmsnap_latency": { 2024-04-24T23:32:19.609 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-24T23:32:19.609 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-24T23:32:19.609 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-24T23:32:19.609 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:19.609 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmxattr_latency": { 2024-04-24T23:32:19.609 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-24T23:32:19.609 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-24T23:32:19.609 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-24T23:32:19.610 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:19.610 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setattr_latency": { 2024-04-24T23:32:19.610 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-24T23:32:19.610 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-24T23:32:19.610 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-24T23:32:19.610 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:19.610 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setdirlayout_latency": { 2024-04-24T23:32:19.611 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-24T23:32:19.611 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-24T23:32:19.611 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-24T23:32:19.611 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:19.611 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setfilelock_latency": { 2024-04-24T23:32:19.611 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-24T23:32:19.611 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-24T23:32:19.611 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-24T23:32:19.612 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:19.612 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setlayout_latency": { 2024-04-24T23:32:19.612 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-24T23:32:19.612 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-24T23:32:19.612 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-24T23:32:19.612 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:19.612 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setxattr_latency": { 2024-04-24T23:32:19.612 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-24T23:32:19.613 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-24T23:32:19.613 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-24T23:32:19.613 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:19.613 INFO:teuthology.orchestra.run.smithi134.stdout: "req_symlink_latency": { 2024-04-24T23:32:19.613 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-24T23:32:19.613 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-24T23:32:19.613 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-24T23:32:19.614 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:19.614 INFO:teuthology.orchestra.run.smithi134.stdout: "req_unlink_latency": { 2024-04-24T23:32:19.614 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-24T23:32:19.614 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-24T23:32:19.614 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-24T23:32:19.614 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:19.614 INFO:teuthology.orchestra.run.smithi134.stdout: "cap_revoke_eviction": 0, 2024-04-24T23:32:19.615 INFO:teuthology.orchestra.run.smithi134.stdout: "cap_acquisition_throttle": 0, 2024-04-24T23:32:19.615 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getvxattr_latency": { 2024-04-24T23:32:19.615 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-24T23:32:19.615 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-24T23:32:19.615 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-24T23:32:19.615 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-24T23:32:19.615 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:19.616 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_sessions": { 2024-04-24T23:32:19.616 INFO:teuthology.orchestra.run.smithi134.stdout: "session_count": 1, 2024-04-24T23:32:19.616 INFO:teuthology.orchestra.run.smithi134.stdout: "session_add": 2, 2024-04-24T23:32:19.616 INFO:teuthology.orchestra.run.smithi134.stdout: "session_remove": 1, 2024-04-24T23:32:19.616 INFO:teuthology.orchestra.run.smithi134.stdout: "sessions_open": 1, 2024-04-24T23:32:19.616 INFO:teuthology.orchestra.run.smithi134.stdout: "sessions_stale": 0, 2024-04-24T23:32:19.616 INFO:teuthology.orchestra.run.smithi134.stdout: "total_load": 0, 2024-04-24T23:32:19.616 INFO:teuthology.orchestra.run.smithi134.stdout: "average_load": 0, 2024-04-24T23:32:19.617 INFO:teuthology.orchestra.run.smithi134.stdout: "avg_session_uptime": 1276, 2024-04-24T23:32:19.617 INFO:teuthology.orchestra.run.smithi134.stdout: "mdthresh_evicted": 0 2024-04-24T23:32:19.617 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:19.617 INFO:teuthology.orchestra.run.smithi134.stdout: "mempool": { 2024-04-24T23:32:19.617 INFO:teuthology.orchestra.run.smithi134.stdout: "bloom_filter_bytes": 0, 2024-04-24T23:32:19.617 INFO:teuthology.orchestra.run.smithi134.stdout: "bloom_filter_items": 0, 2024-04-24T23:32:19.617 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_alloc_bytes": 0, 2024-04-24T23:32:19.618 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_alloc_items": 0, 2024-04-24T23:32:19.618 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_data_bytes": 0, 2024-04-24T23:32:19.618 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_data_items": 0, 2024-04-24T23:32:19.618 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-24T23:32:19.618 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_onode_items": 0, 2024-04-24T23:32:19.618 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-24T23:32:19.618 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_meta_items": 0, 2024-04-24T23:32:19.619 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_other_bytes": 0, 2024-04-24T23:32:19.619 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_other_items": 0, 2024-04-24T23:32:19.619 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-24T23:32:19.619 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_buffer_items": 0, 2024-04-24T23:32:19.619 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_extent_bytes": 0, 2024-04-24T23:32:19.619 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_extent_items": 0, 2024-04-24T23:32:19.619 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_blob_bytes": 0, 2024-04-24T23:32:19.620 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_blob_items": 0, 2024-04-24T23:32:19.620 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-24T23:32:19.620 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_shared_blob_items": 0, 2024-04-24T23:32:19.620 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-24T23:32:19.620 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_inline_bl_items": 0, 2024-04-24T23:32:19.620 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_fsck_bytes": 0, 2024-04-24T23:32:19.620 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_fsck_items": 0, 2024-04-24T23:32:19.620 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_txc_bytes": 0, 2024-04-24T23:32:19.621 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_txc_items": 0, 2024-04-24T23:32:19.621 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-24T23:32:19.621 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_deferred_items": 0, 2024-04-24T23:32:19.621 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_bytes": 0, 2024-04-24T23:32:19.621 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_items": 0, 2024-04-24T23:32:19.621 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_bytes": 0, 2024-04-24T23:32:19.621 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_items": 0, 2024-04-24T23:32:19.622 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_reader_bytes": 0, 2024-04-24T23:32:19.622 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_reader_items": 0, 2024-04-24T23:32:19.622 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_writer_bytes": 0, 2024-04-24T23:32:19.622 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_writer_items": 0, 2024-04-24T23:32:19.622 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_anon_bytes": 244457, 2024-04-24T23:32:19.622 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_anon_items": 1326, 2024-04-24T23:32:19.622 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_meta_bytes": 0, 2024-04-24T23:32:19.623 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_meta_items": 0, 2024-04-24T23:32:19.623 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_bytes": 0, 2024-04-24T23:32:19.623 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_items": 0, 2024-04-24T23:32:19.623 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_mapbl_bytes": 0, 2024-04-24T23:32:19.623 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_mapbl_items": 0, 2024-04-24T23:32:19.623 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_pglog_bytes": 0, 2024-04-24T23:32:19.623 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_pglog_items": 0, 2024-04-24T23:32:19.623 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_bytes": 9080, 2024-04-24T23:32:19.624 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_items": 174, 2024-04-24T23:32:19.624 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_mapping_bytes": 0, 2024-04-24T23:32:19.624 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_mapping_items": 0, 2024-04-24T23:32:19.624 INFO:teuthology.orchestra.run.smithi134.stdout: "pgmap_bytes": 0, 2024-04-24T23:32:19.624 INFO:teuthology.orchestra.run.smithi134.stdout: "pgmap_items": 0, 2024-04-24T23:32:19.624 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_co_bytes": 4604138, 2024-04-24T23:32:19.625 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_co_items": 41271, 2024-04-24T23:32:19.625 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_1_bytes": 0, 2024-04-24T23:32:19.625 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_1_items": 0, 2024-04-24T23:32:19.625 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_2_bytes": 0, 2024-04-24T23:32:19.625 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_2_items": 0 2024-04-24T23:32:19.625 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:19.625 INFO:teuthology.orchestra.run.smithi134.stdout: "objecter": { 2024-04-24T23:32:19.626 INFO:teuthology.orchestra.run.smithi134.stdout: "op_active": 0, 2024-04-24T23:32:19.626 INFO:teuthology.orchestra.run.smithi134.stdout: "op_laggy": 0, 2024-04-24T23:32:19.626 INFO:teuthology.orchestra.run.smithi134.stdout: "op_send": 769, 2024-04-24T23:32:19.626 INFO:teuthology.orchestra.run.smithi134.stdout: "op_send_bytes": 21, 2024-04-24T23:32:19.626 INFO:teuthology.orchestra.run.smithi134.stdout: "op_resend": 3, 2024-04-24T23:32:19.626 INFO:teuthology.orchestra.run.smithi134.stdout: "op_reply": 769, 2024-04-24T23:32:19.626 INFO:teuthology.orchestra.run.smithi134.stdout: "op_latency": { 2024-04-24T23:32:19.626 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 769, 2024-04-24T23:32:19.627 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.431989908, 2024-04-24T23:32:19.627 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000561755 2024-04-24T23:32:19.627 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:19.627 INFO:teuthology.orchestra.run.smithi134.stdout: "op_inflight": 0, 2024-04-24T23:32:19.627 INFO:teuthology.orchestra.run.smithi134.stdout: "oplen_avg": { 2024-04-24T23:32:19.627 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 769, 2024-04-24T23:32:19.627 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 770 2024-04-24T23:32:19.628 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:19.628 INFO:teuthology.orchestra.run.smithi134.stdout: "op": 769, 2024-04-24T23:32:19.628 INFO:teuthology.orchestra.run.smithi134.stdout: "op_r": 769, 2024-04-24T23:32:19.628 INFO:teuthology.orchestra.run.smithi134.stdout: "op_w": 0, 2024-04-24T23:32:19.628 INFO:teuthology.orchestra.run.smithi134.stdout: "op_rmw": 0, 2024-04-24T23:32:19.628 INFO:teuthology.orchestra.run.smithi134.stdout: "op_pg": 0, 2024-04-24T23:32:19.628 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_stat": 465, 2024-04-24T23:32:19.629 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_create": 0, 2024-04-24T23:32:19.629 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_read": 302, 2024-04-24T23:32:19.629 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_write": 0, 2024-04-24T23:32:19.629 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_writefull": 0, 2024-04-24T23:32:19.629 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_writesame": 0, 2024-04-24T23:32:19.629 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_append": 0, 2024-04-24T23:32:19.629 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_zero": 0, 2024-04-24T23:32:19.629 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_truncate": 0, 2024-04-24T23:32:19.630 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_delete": 0, 2024-04-24T23:32:19.630 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_mapext": 0, 2024-04-24T23:32:19.630 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_sparse_read": 0, 2024-04-24T23:32:19.630 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_clonerange": 0, 2024-04-24T23:32:19.630 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_getxattr": 1, 2024-04-24T23:32:19.630 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_setxattr": 0, 2024-04-24T23:32:19.630 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_cmpxattr": 0, 2024-04-24T23:32:19.631 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_rmxattr": 0, 2024-04-24T23:32:19.631 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_resetxattrs": 0, 2024-04-24T23:32:19.631 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_call": 0, 2024-04-24T23:32:19.631 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_watch": 0, 2024-04-24T23:32:19.631 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_notify": 0, 2024-04-24T23:32:19.631 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_src_cmpxattr": 0, 2024-04-24T23:32:19.631 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_pgls": 0, 2024-04-24T23:32:19.631 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_pgls_filter": 0, 2024-04-24T23:32:19.632 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_other": 0, 2024-04-24T23:32:19.632 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_active": 0, 2024-04-24T23:32:19.632 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_send": 0, 2024-04-24T23:32:19.632 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_resend": 0, 2024-04-24T23:32:19.632 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_ping": 0, 2024-04-24T23:32:19.632 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_active": 0, 2024-04-24T23:32:19.632 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_send": 0, 2024-04-24T23:32:19.632 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_resend": 0, 2024-04-24T23:32:19.633 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_active": 0, 2024-04-24T23:32:19.633 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_send": 0, 2024-04-24T23:32:19.633 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_resend": 0, 2024-04-24T23:32:19.633 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_active": 0, 2024-04-24T23:32:19.633 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_send": 0, 2024-04-24T23:32:19.633 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_resend": 0, 2024-04-24T23:32:19.633 INFO:teuthology.orchestra.run.smithi134.stdout: "command_active": 0, 2024-04-24T23:32:19.634 INFO:teuthology.orchestra.run.smithi134.stdout: "command_send": 0, 2024-04-24T23:32:19.634 INFO:teuthology.orchestra.run.smithi134.stdout: "command_resend": 0, 2024-04-24T23:32:19.634 INFO:teuthology.orchestra.run.smithi134.stdout: "map_epoch": 94, 2024-04-24T23:32:19.634 INFO:teuthology.orchestra.run.smithi134.stdout: "map_full": 0, 2024-04-24T23:32:19.634 INFO:teuthology.orchestra.run.smithi134.stdout: "map_inc": 4, 2024-04-24T23:32:19.634 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_sessions": 6, 2024-04-24T23:32:19.634 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_session_open": 6, 2024-04-24T23:32:19.635 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_session_close": 0, 2024-04-24T23:32:19.635 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_laggy": 0, 2024-04-24T23:32:19.635 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_wr": 0, 2024-04-24T23:32:19.635 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_rd": 2, 2024-04-24T23:32:19.635 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_del": 0 2024-04-24T23:32:19.635 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:19.635 INFO:teuthology.orchestra.run.smithi134.stdout: "oft": { 2024-04-24T23:32:19.635 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_objs": 0, 2024-04-24T23:32:19.636 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_kv_pairs": 0, 2024-04-24T23:32:19.636 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_updates": 0, 2024-04-24T23:32:19.636 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_removes": 0 2024-04-24T23:32:19.636 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:19.636 INFO:teuthology.orchestra.run.smithi134.stdout: "osd-slow-ops": { 2024-04-24T23:32:19.636 INFO:teuthology.orchestra.run.smithi134.stdout: "slow_ops_count": 0 2024-04-24T23:32:19.636 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:19.636 INFO:teuthology.orchestra.run.smithi134.stdout: "purge_queue": { 2024-04-24T23:32:19.637 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_ops": 0, 2024-04-24T23:32:19.637 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_ops_high_water": 0, 2024-04-24T23:32:19.637 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing": 0, 2024-04-24T23:32:19.637 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_high_water": 0, 2024-04-24T23:32:19.637 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executed_ops": 0, 2024-04-24T23:32:19.637 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executed": 0, 2024-04-24T23:32:19.638 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_item_in_journal": 0 2024-04-24T23:32:19.638 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:19.638 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-24T23:32:19.638 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-04-24T23:32:19.638 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 104857600, 2024-04-24T23:32:19.638 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-04-24T23:32:19.638 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 876, 2024-04-24T23:32:19.639 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 6341499, 2024-04-24T23:32:19.639 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-04-24T23:32:19.639 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 876, 2024-04-24T23:32:19.639 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2024-04-24T23:32:19.639 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2024-04-24T23:32:19.639 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 876, 2024-04-24T23:32:19.639 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 6341499, 2024-04-24T23:32:19.639 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-04-24T23:32:19.639 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-24T23:32:19.640 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-24T23:32:19.640 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-24T23:32:19.640 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-24T23:32:19.640 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:19.640 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-objecter_bytes": { 2024-04-24T23:32:19.640 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-04-24T23:32:19.640 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 104857600, 2024-04-24T23:32:19.641 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-04-24T23:32:19.642 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2024-04-24T23:32:19.642 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2024-04-24T23:32:19.642 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-04-24T23:32:19.642 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2024-04-24T23:32:19.642 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 769, 2024-04-24T23:32:19.642 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 5735565, 2024-04-24T23:32:19.642 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 769, 2024-04-24T23:32:19.643 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 5735565, 2024-04-24T23:32:19.643 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-04-24T23:32:19.643 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-24T23:32:19.643 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-24T23:32:19.643 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-24T23:32:19.644 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-24T23:32:19.644 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:19.644 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-objecter_ops": { 2024-04-24T23:32:19.644 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-04-24T23:32:19.644 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 1024, 2024-04-24T23:32:19.644 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-04-24T23:32:19.644 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2024-04-24T23:32:19.644 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2024-04-24T23:32:19.644 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-04-24T23:32:19.645 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2024-04-24T23:32:19.645 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 769, 2024-04-24T23:32:19.645 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 769, 2024-04-24T23:32:19.645 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 769, 2024-04-24T23:32:19.645 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 769, 2024-04-24T23:32:19.645 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-04-24T23:32:19.645 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-24T23:32:19.645 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-24T23:32:19.646 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-24T23:32:19.646 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-24T23:32:19.646 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:19.646 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-write_buf_throttle": { 2024-04-24T23:32:19.646 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-04-24T23:32:19.646 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 3758096384, 2024-04-24T23:32:19.646 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-04-24T23:32:19.646 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2024-04-24T23:32:19.647 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2024-04-24T23:32:19.647 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-04-24T23:32:19.647 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2024-04-24T23:32:19.647 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2024-04-24T23:32:19.647 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2024-04-24T23:32:19.647 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 0, 2024-04-24T23:32:19.647 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 0, 2024-04-24T23:32:19.647 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-04-24T23:32:19.647 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-24T23:32:19.648 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-24T23:32:19.648 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-24T23:32:19.648 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-24T23:32:19.648 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:19.648 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-write_buf_throttle-0x555a409dc6e0": { 2024-04-24T23:32:19.648 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-04-24T23:32:19.648 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 3758096384, 2024-04-24T23:32:19.648 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-04-24T23:32:19.649 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2024-04-24T23:32:19.649 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2024-04-24T23:32:19.649 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-04-24T23:32:19.649 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2024-04-24T23:32:19.649 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2024-04-24T23:32:19.649 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2024-04-24T23:32:19.649 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 0, 2024-04-24T23:32:19.649 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 0, 2024-04-24T23:32:19.649 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-04-24T23:32:19.650 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-24T23:32:19.650 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-24T23:32:19.650 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-24T23:32:19.650 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-24T23:32:19.650 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-24T23:32:19.650 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-04-24T23:32:20.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:32:20 smithi134 ceph-mon[32619]: pgmap v450: 129 pgs: 129 active+clean; 74 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-24T23:32:20.495 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-04-24T23:32:20.495 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-04-24T23:32:20.496 INFO:tasks.check_counter:Daemon mds.e mds.dir_split=0 2024-04-24T23:32:20.496 DEBUG:tasks.check_counter:Getting stats from h 2024-04-24T23:32:20.496 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid ec7885aa-0290-11ef-bc93-c7b262605968 -- ceph daemon mds.h perf dump 2024-04-24T23:32:20.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:32:20 smithi148 ceph-mon[32528]: pgmap v450: 129 pgs: 129 active+clean; 74 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-24T23:32:20.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:32:20 smithi080 ceph-mon[29152]: pgmap v450: 129 pgs: 129 active+clean; 74 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-24T23:32:20.735 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/mon.b/config 2024-04-24T23:32:22.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:32:22 smithi134 ceph-mon[32619]: pgmap v451: 129 pgs: 129 active+clean; 74 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 3.2 KiB/s wr, 8 op/s 2024-04-24T23:32:22.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:32:22 smithi148 ceph-mon[32528]: pgmap v451: 129 pgs: 129 active+clean; 74 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 3.2 KiB/s wr, 8 op/s 2024-04-24T23:32:22.515 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-04-24T23:32:22.515 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-0": { 2024-04-24T23:32:22.515 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 776, 2024-04-24T23:32:22.515 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 778, 2024-04-24T23:32:22.515 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 5954624, 2024-04-24T23:32:22.516 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 246775, 2024-04-24T23:32:22.516 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 6, 2024-04-24T23:32:22.516 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 2, 2024-04-24T23:32:22.516 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.063445548, 2024-04-24T23:32:22.516 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.019301386, 2024-04-24T23:32:22.516 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.019412522, 2024-04-24T23:32:22.516 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.019263891, 2024-04-24T23:32:22.516 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:32:22.517 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 776, 2024-04-24T23:32:22.517 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.023164319, 2024-04-24T23:32:22.517 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000029850 2024-04-24T23:32:22.517 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:22.517 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:32:22.517 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-24T23:32:22.517 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-24T23:32:22.517 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-24T23:32:22.517 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:22.518 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-24T23:32:22.518 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 0 2024-04-24T23:32:22.518 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:22.518 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-1": { 2024-04-24T23:32:22.518 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 107, 2024-04-24T23:32:22.518 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 101, 2024-04-24T23:32:22.518 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 471468, 2024-04-24T23:32:22.518 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 56346, 2024-04-24T23:32:22.519 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 3, 2024-04-24T23:32:22.519 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 2, 2024-04-24T23:32:22.519 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.018685954, 2024-04-24T23:32:22.519 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.006199294, 2024-04-24T23:32:22.519 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.012106848, 2024-04-24T23:32:22.519 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.002389008, 2024-04-24T23:32:22.519 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:32:22.519 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 100, 2024-04-24T23:32:22.520 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.006566691, 2024-04-24T23:32:22.520 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000065666 2024-04-24T23:32:22.520 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:22.520 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:32:22.520 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-24T23:32:22.520 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-24T23:32:22.520 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-24T23:32:22.520 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:22.521 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 470912, 2024-04-24T23:32:22.521 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 55648 2024-04-24T23:32:22.521 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:22.521 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-2": { 2024-04-24T23:32:22.521 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 6, 2024-04-24T23:32:22.521 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 71, 2024-04-24T23:32:22.521 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 3420, 2024-04-24T23:32:22.521 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 190062, 2024-04-24T23:32:22.521 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 4, 2024-04-24T23:32:22.522 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 3, 2024-04-24T23:32:22.522 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.005206512, 2024-04-24T23:32:22.522 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.003026789, 2024-04-24T23:32:22.522 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.001435477, 2024-04-24T23:32:22.522 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.000018427, 2024-04-24T23:32:22.522 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:32:22.522 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 70, 2024-04-24T23:32:22.522 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.005382798, 2024-04-24T23:32:22.523 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000076897 2024-04-24T23:32:22.523 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:22.523 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:32:22.523 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-24T23:32:22.523 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-24T23:32:22.523 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-24T23:32:22.523 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:22.523 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 3152, 2024-04-24T23:32:22.524 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 189744 2024-04-24T23:32:22.524 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:22.524 INFO:teuthology.orchestra.run.smithi134.stdout: "cct": { 2024-04-24T23:32:22.524 INFO:teuthology.orchestra.run.smithi134.stdout: "total_workers": 1, 2024-04-24T23:32:22.524 INFO:teuthology.orchestra.run.smithi134.stdout: "unhealthy_workers": 0 2024-04-24T23:32:22.524 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:22.524 INFO:teuthology.orchestra.run.smithi134.stdout: "finisher-MDSRank": { 2024-04-24T23:32:22.524 INFO:teuthology.orchestra.run.smithi134.stdout: "queue_len": 0, 2024-04-24T23:32:22.524 INFO:teuthology.orchestra.run.smithi134.stdout: "complete_latency": { 2024-04-24T23:32:22.525 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 1351, 2024-04-24T23:32:22.525 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.030673787, 2024-04-24T23:32:22.525 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000022704 2024-04-24T23:32:22.525 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-24T23:32:22.525 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:22.525 INFO:teuthology.orchestra.run.smithi134.stdout: "finisher-PurgeQueue": { 2024-04-24T23:32:22.525 INFO:teuthology.orchestra.run.smithi134.stdout: "queue_len": 0, 2024-04-24T23:32:22.525 INFO:teuthology.orchestra.run.smithi134.stdout: "complete_latency": { 2024-04-24T23:32:22.526 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-24T23:32:22.526 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-24T23:32:22.526 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-24T23:32:22.526 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-24T23:32:22.526 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:22.526 INFO:teuthology.orchestra.run.smithi134.stdout: "mds": { 2024-04-24T23:32:22.526 INFO:teuthology.orchestra.run.smithi134.stdout: "request": 0, 2024-04-24T23:32:22.526 INFO:teuthology.orchestra.run.smithi134.stdout: "reply": 0, 2024-04-24T23:32:22.527 INFO:teuthology.orchestra.run.smithi134.stdout: "reply_latency": { 2024-04-24T23:32:22.527 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-24T23:32:22.527 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-24T23:32:22.527 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-24T23:32:22.527 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:22.527 INFO:teuthology.orchestra.run.smithi134.stdout: "slow_reply": 0, 2024-04-24T23:32:22.527 INFO:teuthology.orchestra.run.smithi134.stdout: "forward": 0, 2024-04-24T23:32:22.527 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_fetch_complete": 0, 2024-04-24T23:32:22.528 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_fetch_keys": 0, 2024-04-24T23:32:22.528 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_commit": 0, 2024-04-24T23:32:22.528 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_split": 0, 2024-04-24T23:32:22.528 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_merge": 0, 2024-04-24T23:32:22.528 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes": 3546, 2024-04-24T23:32:22.528 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_top": 193, 2024-04-24T23:32:22.528 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_bottom": 3353, 2024-04-24T23:32:22.528 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_pin_tail": 0, 2024-04-24T23:32:22.528 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_pinned": 3270, 2024-04-24T23:32:22.529 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_expired": 0, 2024-04-24T23:32:22.529 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_with_caps": 0, 2024-04-24T23:32:22.529 INFO:teuthology.orchestra.run.smithi134.stdout: "caps": 0, 2024-04-24T23:32:22.529 INFO:teuthology.orchestra.run.smithi134.stdout: "subtrees": 75, 2024-04-24T23:32:22.529 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse": 0, 2024-04-24T23:32:22.529 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_hit": 0, 2024-04-24T23:32:22.529 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_forward": 0, 2024-04-24T23:32:22.529 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_discover": 0, 2024-04-24T23:32:22.530 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_dir_fetch": 0, 2024-04-24T23:32:22.530 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_remote_ino": 0, 2024-04-24T23:32:22.530 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_lock": 0, 2024-04-24T23:32:22.530 INFO:teuthology.orchestra.run.smithi134.stdout: "load_cent": 0, 2024-04-24T23:32:22.530 INFO:teuthology.orchestra.run.smithi134.stdout: "q": 0, 2024-04-24T23:32:22.530 INFO:teuthology.orchestra.run.smithi134.stdout: "exported": 0, 2024-04-24T23:32:22.530 INFO:teuthology.orchestra.run.smithi134.stdout: "exported_inodes": 0, 2024-04-24T23:32:22.530 INFO:teuthology.orchestra.run.smithi134.stdout: "imported": 0, 2024-04-24T23:32:22.530 INFO:teuthology.orchestra.run.smithi134.stdout: "imported_inodes": 0, 2024-04-24T23:32:22.531 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_dir_fetch": 0, 2024-04-24T23:32:22.531 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_backtrace_fetch": 0, 2024-04-24T23:32:22.531 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_peer_discover": 0, 2024-04-24T23:32:22.531 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rfiles": 0, 2024-04-24T23:32:22.531 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rbytes": 0, 2024-04-24T23:32:22.531 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rsnaps": 0, 2024-04-24T23:32:22.531 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_backtrace_fetch": 0, 2024-04-24T23:32:22.532 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_set_tag": 0, 2024-04-24T23:32:22.532 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_backtrace_repaired": 0, 2024-04-24T23:32:22.532 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_inotable_repaired": 0, 2024-04-24T23:32:22.532 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dir_inodes": 0, 2024-04-24T23:32:22.532 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dir_base_inodes": 0, 2024-04-24T23:32:22.532 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dirfrag_rstats": 0, 2024-04-24T23:32:22.532 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_file_inodes": 0, 2024-04-24T23:32:22.532 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_inode_file_caps": 0, 2024-04-24T23:32:22.532 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_revoke": 0, 2024-04-24T23:32:22.533 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_grant": 0, 2024-04-24T23:32:22.533 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_trunc": 0, 2024-04-24T23:32:22.533 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-24T23:32:22.533 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-24T23:32:22.533 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_caps": 0, 2024-04-24T23:32:22.533 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_caps_dirty": 0, 2024-04-24T23:32:22.533 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_cap_release": 0, 2024-04-24T23:32:22.533 INFO:teuthology.orchestra.run.smithi134.stdout: "process_request_cap_release": 0 2024-04-24T23:32:22.534 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:22.534 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_cache": { 2024-04-24T23:32:22.534 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_update": 0, 2024-04-24T23:32:22.534 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_update_receipt": 0, 2024-04-24T23:32:22.534 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_try_discover": 0, 2024-04-24T23:32:22.534 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_send_discover": 0, 2024-04-24T23:32:22.534 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_handle_discover": 0, 2024-04-24T23:32:22.534 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays": 0, 2024-04-24T23:32:22.535 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays_delayed": 0, 2024-04-24T23:32:22.535 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays_enqueuing": 0, 2024-04-24T23:32:22.535 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_created": 0, 2024-04-24T23:32:22.535 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_enqueued": 0, 2024-04-24T23:32:22.535 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_reintegrated": 0, 2024-04-24T23:32:22.535 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_migrated": 0, 2024-04-24T23:32:22.535 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_processing": 0, 2024-04-24T23:32:22.535 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_enqueued": 0, 2024-04-24T23:32:22.536 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_prioritized": 0, 2024-04-24T23:32:22.536 INFO:teuthology.orchestra.run.smithi134.stdout: "recovery_started": 0, 2024-04-24T23:32:22.536 INFO:teuthology.orchestra.run.smithi134.stdout: "recovery_completed": 0, 2024-04-24T23:32:22.536 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_quiesce_path": 0, 2024-04-24T23:32:22.536 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_quiesce_inode": 0, 2024-04-24T23:32:22.536 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_enqueue_scrub": 0, 2024-04-24T23:32:22.536 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_exportdir": 0, 2024-04-24T23:32:22.536 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_flush": 0, 2024-04-24T23:32:22.536 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_fragmentdir": 0, 2024-04-24T23:32:22.537 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_fragstats": 0, 2024-04-24T23:32:22.537 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_inodestats": 0 2024-04-24T23:32:22.537 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:22.537 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_log": { 2024-04-24T23:32:22.537 INFO:teuthology.orchestra.run.smithi134.stdout: "evlrg": 0, 2024-04-24T23:32:22.537 INFO:teuthology.orchestra.run.smithi134.stdout: "evadd": 0, 2024-04-24T23:32:22.537 INFO:teuthology.orchestra.run.smithi134.stdout: "evex": 0, 2024-04-24T23:32:22.538 INFO:teuthology.orchestra.run.smithi134.stdout: "evtrm": 0, 2024-04-24T23:32:22.538 INFO:teuthology.orchestra.run.smithi134.stdout: "ev": 2951, 2024-04-24T23:32:22.538 INFO:teuthology.orchestra.run.smithi134.stdout: "evexg": 0, 2024-04-24T23:32:22.538 INFO:teuthology.orchestra.run.smithi134.stdout: "evexd": 0, 2024-04-24T23:32:22.538 INFO:teuthology.orchestra.run.smithi134.stdout: "segadd": 0, 2024-04-24T23:32:22.538 INFO:teuthology.orchestra.run.smithi134.stdout: "segex": 0, 2024-04-24T23:32:22.538 INFO:teuthology.orchestra.run.smithi134.stdout: "segtrm": 0, 2024-04-24T23:32:22.538 INFO:teuthology.orchestra.run.smithi134.stdout: "seg": 5, 2024-04-24T23:32:22.539 INFO:teuthology.orchestra.run.smithi134.stdout: "segmjr": 2, 2024-04-24T23:32:22.539 INFO:teuthology.orchestra.run.smithi134.stdout: "segexg": 0, 2024-04-24T23:32:22.539 INFO:teuthology.orchestra.run.smithi134.stdout: "segexd": 0, 2024-04-24T23:32:22.539 INFO:teuthology.orchestra.run.smithi134.stdout: "expos": 4194304, 2024-04-24T23:32:22.539 INFO:teuthology.orchestra.run.smithi134.stdout: "wrpos": 9927977, 2024-04-24T23:32:22.539 INFO:teuthology.orchestra.run.smithi134.stdout: "rdpos": 9927403, 2024-04-24T23:32:22.539 INFO:teuthology.orchestra.run.smithi134.stdout: "jlat": { 2024-04-24T23:32:22.539 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-24T23:32:22.540 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-24T23:32:22.540 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-24T23:32:22.540 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:22.540 INFO:teuthology.orchestra.run.smithi134.stdout: "replayed": 2951 2024-04-24T23:32:22.540 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:22.540 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_mem": { 2024-04-24T23:32:22.540 INFO:teuthology.orchestra.run.smithi134.stdout: "ino": 2226, 2024-04-24T23:32:22.540 INFO:teuthology.orchestra.run.smithi134.stdout: "ino+": 2578, 2024-04-24T23:32:22.541 INFO:teuthology.orchestra.run.smithi134.stdout: "ino-": 352, 2024-04-24T23:32:22.541 INFO:teuthology.orchestra.run.smithi134.stdout: "dir": 213, 2024-04-24T23:32:22.541 INFO:teuthology.orchestra.run.smithi134.stdout: "dir+": 213, 2024-04-24T23:32:22.541 INFO:teuthology.orchestra.run.smithi134.stdout: "dir-": 0, 2024-04-24T23:32:22.541 INFO:teuthology.orchestra.run.smithi134.stdout: "dn": 3546, 2024-04-24T23:32:22.541 INFO:teuthology.orchestra.run.smithi134.stdout: "dn+": 3546, 2024-04-24T23:32:22.541 INFO:teuthology.orchestra.run.smithi134.stdout: "dn-": 0, 2024-04-24T23:32:22.541 INFO:teuthology.orchestra.run.smithi134.stdout: "cap": 0, 2024-04-24T23:32:22.542 INFO:teuthology.orchestra.run.smithi134.stdout: "cap+": 0, 2024-04-24T23:32:22.542 INFO:teuthology.orchestra.run.smithi134.stdout: "cap-": 0, 2024-04-24T23:32:22.542 INFO:teuthology.orchestra.run.smithi134.stdout: "rss": 61304, 2024-04-24T23:32:22.542 INFO:teuthology.orchestra.run.smithi134.stdout: "heap": 215420 2024-04-24T23:32:22.542 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:22.542 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_server": { 2024-04-24T23:32:22.542 INFO:teuthology.orchestra.run.smithi134.stdout: "dispatch_client_request": 0, 2024-04-24T23:32:22.543 INFO:teuthology.orchestra.run.smithi134.stdout: "dispatch_server_request": 0, 2024-04-24T23:32:22.543 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_request": 0, 2024-04-24T23:32:22.543 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_session": 0, 2024-04-24T23:32:22.543 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_peer_request": 0, 2024-04-24T23:32:22.543 INFO:teuthology.orchestra.run.smithi134.stdout: "req_create_latency": { 2024-04-24T23:32:22.543 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-24T23:32:22.543 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-24T23:32:22.543 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-24T23:32:22.544 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:22.544 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getattr_latency": { 2024-04-24T23:32:22.544 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-24T23:32:22.544 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-24T23:32:22.544 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-24T23:32:22.544 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:22.544 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getfilelock_latency": { 2024-04-24T23:32:22.544 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-24T23:32:22.545 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-24T23:32:22.545 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-24T23:32:22.545 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:22.545 INFO:teuthology.orchestra.run.smithi134.stdout: "req_link_latency": { 2024-04-24T23:32:22.545 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-24T23:32:22.545 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-24T23:32:22.545 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-24T23:32:22.546 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:22.546 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookup_latency": { 2024-04-24T23:32:22.546 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-24T23:32:22.546 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-24T23:32:22.546 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-24T23:32:22.546 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:22.546 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookuphash_latency": { 2024-04-24T23:32:22.546 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-24T23:32:22.547 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-24T23:32:22.547 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-24T23:32:22.547 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:22.547 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupino_latency": { 2024-04-24T23:32:22.547 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-24T23:32:22.547 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-24T23:32:22.547 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-24T23:32:22.547 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:22.548 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupname_latency": { 2024-04-24T23:32:22.548 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-24T23:32:22.548 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-24T23:32:22.548 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-24T23:32:22.548 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:22.548 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupparent_latency": { 2024-04-24T23:32:22.548 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-24T23:32:22.548 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-24T23:32:22.549 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-24T23:32:22.549 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:22.549 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupsnap_latency": { 2024-04-24T23:32:22.549 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-24T23:32:22.549 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-24T23:32:22.549 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-24T23:32:22.549 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:22.549 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lssnap_latency": { 2024-04-24T23:32:22.549 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-24T23:32:22.550 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-24T23:32:22.550 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-24T23:32:22.550 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:22.550 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mkdir_latency": { 2024-04-24T23:32:22.551 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:32:22 smithi080 ceph-mon[29152]: pgmap v451: 129 pgs: 129 active+clean; 74 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 3.2 KiB/s wr, 8 op/s 2024-04-24T23:32:22.551 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-24T23:32:22.551 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-24T23:32:22.552 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-24T23:32:22.552 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:22.552 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mknod_latency": { 2024-04-24T23:32:22.552 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-24T23:32:22.552 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-24T23:32:22.552 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-24T23:32:22.552 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:22.552 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mksnap_latency": { 2024-04-24T23:32:22.553 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-24T23:32:22.553 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-24T23:32:22.553 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-24T23:32:22.553 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:22.553 INFO:teuthology.orchestra.run.smithi134.stdout: "req_open_latency": { 2024-04-24T23:32:22.553 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-24T23:32:22.553 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-24T23:32:22.553 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-24T23:32:22.554 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:22.554 INFO:teuthology.orchestra.run.smithi134.stdout: "req_readdir_latency": { 2024-04-24T23:32:22.554 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-24T23:32:22.554 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-24T23:32:22.554 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-24T23:32:22.554 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:22.554 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rename_latency": { 2024-04-24T23:32:22.555 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-24T23:32:22.555 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-24T23:32:22.555 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-24T23:32:22.555 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:22.555 INFO:teuthology.orchestra.run.smithi134.stdout: "req_renamesnap_latency": { 2024-04-24T23:32:22.555 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-24T23:32:22.555 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-24T23:32:22.556 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-24T23:32:22.556 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:22.556 INFO:teuthology.orchestra.run.smithi134.stdout: "req_snapdiff_latency": { 2024-04-24T23:32:22.556 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-24T23:32:22.556 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-24T23:32:22.556 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-24T23:32:22.557 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:22.557 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmdir_latency": { 2024-04-24T23:32:22.557 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-24T23:32:22.557 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-24T23:32:22.557 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-24T23:32:22.557 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:22.557 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmsnap_latency": { 2024-04-24T23:32:22.557 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-24T23:32:22.558 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-24T23:32:22.558 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-24T23:32:22.558 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:22.558 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmxattr_latency": { 2024-04-24T23:32:22.558 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-24T23:32:22.558 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-24T23:32:22.558 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-24T23:32:22.559 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:22.559 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setattr_latency": { 2024-04-24T23:32:22.559 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-24T23:32:22.559 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-24T23:32:22.559 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-24T23:32:22.559 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:22.559 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setdirlayout_latency": { 2024-04-24T23:32:22.559 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-24T23:32:22.560 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-24T23:32:22.560 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-24T23:32:22.560 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:22.560 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setfilelock_latency": { 2024-04-24T23:32:22.560 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-24T23:32:22.560 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-24T23:32:22.560 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-24T23:32:22.560 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:22.561 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setlayout_latency": { 2024-04-24T23:32:22.561 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-24T23:32:22.561 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-24T23:32:22.561 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-24T23:32:22.561 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:22.561 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setxattr_latency": { 2024-04-24T23:32:22.561 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-24T23:32:22.561 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-24T23:32:22.562 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-24T23:32:22.562 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:22.566 INFO:teuthology.orchestra.run.smithi134.stdout: "req_symlink_latency": { 2024-04-24T23:32:22.566 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-24T23:32:22.566 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-24T23:32:22.567 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-24T23:32:22.567 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:22.567 INFO:teuthology.orchestra.run.smithi134.stdout: "req_unlink_latency": { 2024-04-24T23:32:22.567 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-24T23:32:22.567 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-24T23:32:22.567 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-24T23:32:22.567 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:22.567 INFO:teuthology.orchestra.run.smithi134.stdout: "cap_revoke_eviction": 0, 2024-04-24T23:32:22.568 INFO:teuthology.orchestra.run.smithi134.stdout: "cap_acquisition_throttle": 0, 2024-04-24T23:32:22.568 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getvxattr_latency": { 2024-04-24T23:32:22.568 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-24T23:32:22.568 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-24T23:32:22.568 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-24T23:32:22.568 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-24T23:32:22.568 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:22.568 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_sessions": { 2024-04-24T23:32:22.568 INFO:teuthology.orchestra.run.smithi134.stdout: "session_count": 1, 2024-04-24T23:32:22.569 INFO:teuthology.orchestra.run.smithi134.stdout: "session_add": 2, 2024-04-24T23:32:22.569 INFO:teuthology.orchestra.run.smithi134.stdout: "session_remove": 1, 2024-04-24T23:32:22.569 INFO:teuthology.orchestra.run.smithi134.stdout: "sessions_open": 1, 2024-04-24T23:32:22.569 INFO:teuthology.orchestra.run.smithi134.stdout: "sessions_stale": 0, 2024-04-24T23:32:22.569 INFO:teuthology.orchestra.run.smithi134.stdout: "total_load": 0, 2024-04-24T23:32:22.569 INFO:teuthology.orchestra.run.smithi134.stdout: "average_load": 0, 2024-04-24T23:32:22.569 INFO:teuthology.orchestra.run.smithi134.stdout: "avg_session_uptime": 1280, 2024-04-24T23:32:22.569 INFO:teuthology.orchestra.run.smithi134.stdout: "mdthresh_evicted": 0 2024-04-24T23:32:22.569 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:22.570 INFO:teuthology.orchestra.run.smithi134.stdout: "mempool": { 2024-04-24T23:32:22.570 INFO:teuthology.orchestra.run.smithi134.stdout: "bloom_filter_bytes": 0, 2024-04-24T23:32:22.570 INFO:teuthology.orchestra.run.smithi134.stdout: "bloom_filter_items": 0, 2024-04-24T23:32:22.570 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_alloc_bytes": 0, 2024-04-24T23:32:22.570 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_alloc_items": 0, 2024-04-24T23:32:22.570 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_data_bytes": 0, 2024-04-24T23:32:22.570 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_data_items": 0, 2024-04-24T23:32:22.571 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-24T23:32:22.571 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_onode_items": 0, 2024-04-24T23:32:22.571 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-24T23:32:22.571 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_meta_items": 0, 2024-04-24T23:32:22.571 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_other_bytes": 0, 2024-04-24T23:32:22.571 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_other_items": 0, 2024-04-24T23:32:22.571 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-24T23:32:22.571 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_buffer_items": 0, 2024-04-24T23:32:22.571 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_extent_bytes": 0, 2024-04-24T23:32:22.572 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_extent_items": 0, 2024-04-24T23:32:22.572 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_blob_bytes": 0, 2024-04-24T23:32:22.572 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_blob_items": 0, 2024-04-24T23:32:22.572 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-24T23:32:22.572 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_shared_blob_items": 0, 2024-04-24T23:32:22.572 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-24T23:32:22.572 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_inline_bl_items": 0, 2024-04-24T23:32:22.572 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_fsck_bytes": 0, 2024-04-24T23:32:22.573 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_fsck_items": 0, 2024-04-24T23:32:22.573 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_txc_bytes": 0, 2024-04-24T23:32:22.573 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_txc_items": 0, 2024-04-24T23:32:22.573 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-24T23:32:22.573 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_deferred_items": 0, 2024-04-24T23:32:22.573 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_bytes": 0, 2024-04-24T23:32:22.573 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_items": 0, 2024-04-24T23:32:22.573 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_bytes": 0, 2024-04-24T23:32:22.573 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_items": 0, 2024-04-24T23:32:22.574 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_reader_bytes": 0, 2024-04-24T23:32:22.574 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_reader_items": 0, 2024-04-24T23:32:22.574 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_writer_bytes": 0, 2024-04-24T23:32:22.574 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_writer_items": 0, 2024-04-24T23:32:22.574 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_anon_bytes": 263188, 2024-04-24T23:32:22.574 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_anon_items": 2261, 2024-04-24T23:32:22.574 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_meta_bytes": 0, 2024-04-24T23:32:22.574 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_meta_items": 0, 2024-04-24T23:32:22.574 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_bytes": 0, 2024-04-24T23:32:22.575 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_items": 0, 2024-04-24T23:32:22.575 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_mapbl_bytes": 0, 2024-04-24T23:32:22.575 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_mapbl_items": 0, 2024-04-24T23:32:22.575 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_pglog_bytes": 0, 2024-04-24T23:32:22.575 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_pglog_items": 0, 2024-04-24T23:32:22.575 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_bytes": 9080, 2024-04-24T23:32:22.575 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_items": 174, 2024-04-24T23:32:22.575 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_mapping_bytes": 0, 2024-04-24T23:32:22.575 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_mapping_items": 0, 2024-04-24T23:32:22.576 INFO:teuthology.orchestra.run.smithi134.stdout: "pgmap_bytes": 0, 2024-04-24T23:32:22.576 INFO:teuthology.orchestra.run.smithi134.stdout: "pgmap_items": 0, 2024-04-24T23:32:22.576 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_co_bytes": 7937949, 2024-04-24T23:32:22.576 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_co_items": 69739, 2024-04-24T23:32:22.576 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_1_bytes": 0, 2024-04-24T23:32:22.576 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_1_items": 0, 2024-04-24T23:32:22.576 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_2_bytes": 0, 2024-04-24T23:32:22.576 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_2_items": 0 2024-04-24T23:32:22.577 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:22.577 INFO:teuthology.orchestra.run.smithi134.stdout: "objecter": { 2024-04-24T23:32:22.577 INFO:teuthology.orchestra.run.smithi134.stdout: "op_active": 0, 2024-04-24T23:32:22.577 INFO:teuthology.orchestra.run.smithi134.stdout: "op_laggy": 0, 2024-04-24T23:32:22.577 INFO:teuthology.orchestra.run.smithi134.stdout: "op_send": 779, 2024-04-24T23:32:22.577 INFO:teuthology.orchestra.run.smithi134.stdout: "op_send_bytes": 21, 2024-04-24T23:32:22.577 INFO:teuthology.orchestra.run.smithi134.stdout: "op_resend": 3, 2024-04-24T23:32:22.577 INFO:teuthology.orchestra.run.smithi134.stdout: "op_reply": 779, 2024-04-24T23:32:22.577 INFO:teuthology.orchestra.run.smithi134.stdout: "op_latency": { 2024-04-24T23:32:22.578 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 779, 2024-04-24T23:32:22.578 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.497988386, 2024-04-24T23:32:22.578 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000639266 2024-04-24T23:32:22.578 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:22.578 INFO:teuthology.orchestra.run.smithi134.stdout: "op_inflight": 0, 2024-04-24T23:32:22.578 INFO:teuthology.orchestra.run.smithi134.stdout: "oplen_avg": { 2024-04-24T23:32:22.578 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 779, 2024-04-24T23:32:22.578 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 780 2024-04-24T23:32:22.579 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:22.579 INFO:teuthology.orchestra.run.smithi134.stdout: "op": 779, 2024-04-24T23:32:22.579 INFO:teuthology.orchestra.run.smithi134.stdout: "op_r": 779, 2024-04-24T23:32:22.579 INFO:teuthology.orchestra.run.smithi134.stdout: "op_w": 0, 2024-04-24T23:32:22.579 INFO:teuthology.orchestra.run.smithi134.stdout: "op_rmw": 0, 2024-04-24T23:32:22.579 INFO:teuthology.orchestra.run.smithi134.stdout: "op_pg": 0, 2024-04-24T23:32:22.579 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_stat": 471, 2024-04-24T23:32:22.579 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_create": 0, 2024-04-24T23:32:22.579 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_read": 306, 2024-04-24T23:32:22.580 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_write": 0, 2024-04-24T23:32:22.580 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_writefull": 0, 2024-04-24T23:32:22.580 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_writesame": 0, 2024-04-24T23:32:22.580 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_append": 0, 2024-04-24T23:32:22.580 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_zero": 0, 2024-04-24T23:32:22.580 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_truncate": 0, 2024-04-24T23:32:22.580 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_delete": 0, 2024-04-24T23:32:22.580 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_mapext": 0, 2024-04-24T23:32:22.580 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_sparse_read": 0, 2024-04-24T23:32:22.581 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_clonerange": 0, 2024-04-24T23:32:22.581 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_getxattr": 1, 2024-04-24T23:32:22.581 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_setxattr": 0, 2024-04-24T23:32:22.581 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_cmpxattr": 0, 2024-04-24T23:32:22.581 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_rmxattr": 0, 2024-04-24T23:32:22.581 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_resetxattrs": 0, 2024-04-24T23:32:22.581 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_call": 0, 2024-04-24T23:32:22.581 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_watch": 0, 2024-04-24T23:32:22.582 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_notify": 0, 2024-04-24T23:32:22.582 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_src_cmpxattr": 0, 2024-04-24T23:32:22.582 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_pgls": 0, 2024-04-24T23:32:22.582 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_pgls_filter": 0, 2024-04-24T23:32:22.582 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_other": 0, 2024-04-24T23:32:22.582 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_active": 0, 2024-04-24T23:32:22.582 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_send": 0, 2024-04-24T23:32:22.582 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_resend": 0, 2024-04-24T23:32:22.583 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_ping": 0, 2024-04-24T23:32:22.583 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_active": 0, 2024-04-24T23:32:22.583 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_send": 0, 2024-04-24T23:32:22.583 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_resend": 0, 2024-04-24T23:32:22.583 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_active": 0, 2024-04-24T23:32:22.583 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_send": 0, 2024-04-24T23:32:22.583 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_resend": 0, 2024-04-24T23:32:22.583 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_active": 0, 2024-04-24T23:32:22.583 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_send": 0, 2024-04-24T23:32:22.584 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_resend": 0, 2024-04-24T23:32:22.584 INFO:teuthology.orchestra.run.smithi134.stdout: "command_active": 0, 2024-04-24T23:32:22.584 INFO:teuthology.orchestra.run.smithi134.stdout: "command_send": 0, 2024-04-24T23:32:22.584 INFO:teuthology.orchestra.run.smithi134.stdout: "command_resend": 0, 2024-04-24T23:32:22.584 INFO:teuthology.orchestra.run.smithi134.stdout: "map_epoch": 94, 2024-04-24T23:32:22.584 INFO:teuthology.orchestra.run.smithi134.stdout: "map_full": 0, 2024-04-24T23:32:22.584 INFO:teuthology.orchestra.run.smithi134.stdout: "map_inc": 4, 2024-04-24T23:32:22.584 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_sessions": 4, 2024-04-24T23:32:22.584 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_session_open": 4, 2024-04-24T23:32:22.585 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_session_close": 0, 2024-04-24T23:32:22.585 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_laggy": 0, 2024-04-24T23:32:22.585 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_wr": 0, 2024-04-24T23:32:22.585 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_rd": 2, 2024-04-24T23:32:22.585 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_del": 0 2024-04-24T23:32:22.585 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:22.585 INFO:teuthology.orchestra.run.smithi134.stdout: "oft": { 2024-04-24T23:32:22.585 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_objs": 0, 2024-04-24T23:32:22.585 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_kv_pairs": 0, 2024-04-24T23:32:22.586 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_updates": 0, 2024-04-24T23:32:22.586 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_removes": 0 2024-04-24T23:32:22.586 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:22.586 INFO:teuthology.orchestra.run.smithi134.stdout: "osd-slow-ops": { 2024-04-24T23:32:22.586 INFO:teuthology.orchestra.run.smithi134.stdout: "slow_ops_count": 0 2024-04-24T23:32:22.586 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:22.586 INFO:teuthology.orchestra.run.smithi134.stdout: "purge_queue": { 2024-04-24T23:32:22.586 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_ops": 0, 2024-04-24T23:32:22.587 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_ops_high_water": 0, 2024-04-24T23:32:22.587 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing": 0, 2024-04-24T23:32:22.587 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_high_water": 0, 2024-04-24T23:32:22.587 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executed_ops": 0, 2024-04-24T23:32:22.587 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executed": 0, 2024-04-24T23:32:22.587 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_item_in_journal": 0 2024-04-24T23:32:22.587 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:22.588 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-24T23:32:22.588 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-04-24T23:32:22.588 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 104857600, 2024-04-24T23:32:22.588 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-04-24T23:32:22.588 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 885, 2024-04-24T23:32:22.588 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 6341504, 2024-04-24T23:32:22.588 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-04-24T23:32:22.588 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 885, 2024-04-24T23:32:22.589 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2024-04-24T23:32:22.589 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2024-04-24T23:32:22.589 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 885, 2024-04-24T23:32:22.589 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 6341504, 2024-04-24T23:32:22.589 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-04-24T23:32:22.589 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-24T23:32:22.590 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-24T23:32:22.590 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-24T23:32:22.590 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-24T23:32:22.590 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:22.590 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-objecter_bytes": { 2024-04-24T23:32:22.590 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-04-24T23:32:22.591 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 104857600, 2024-04-24T23:32:22.591 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-04-24T23:32:22.591 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2024-04-24T23:32:22.591 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2024-04-24T23:32:22.591 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-04-24T23:32:22.591 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2024-04-24T23:32:22.591 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 779, 2024-04-24T23:32:22.591 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 5733678, 2024-04-24T23:32:22.591 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 779, 2024-04-24T23:32:22.592 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 5733678, 2024-04-24T23:32:22.592 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-04-24T23:32:22.592 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-24T23:32:22.592 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-24T23:32:22.592 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-24T23:32:22.592 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-24T23:32:22.592 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:22.592 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-objecter_ops": { 2024-04-24T23:32:22.593 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-04-24T23:32:22.593 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 1024, 2024-04-24T23:32:22.593 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-04-24T23:32:22.593 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2024-04-24T23:32:22.593 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2024-04-24T23:32:22.593 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-04-24T23:32:22.593 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2024-04-24T23:32:22.593 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 779, 2024-04-24T23:32:22.594 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 779, 2024-04-24T23:32:22.594 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 779, 2024-04-24T23:32:22.594 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 779, 2024-04-24T23:32:22.594 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-04-24T23:32:22.594 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-24T23:32:22.594 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-24T23:32:22.594 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-24T23:32:22.594 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-24T23:32:22.594 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:22.595 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-write_buf_throttle": { 2024-04-24T23:32:22.595 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-04-24T23:32:22.595 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 3758096384, 2024-04-24T23:32:22.595 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-04-24T23:32:22.595 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2024-04-24T23:32:22.595 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2024-04-24T23:32:22.595 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-04-24T23:32:22.595 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2024-04-24T23:32:22.596 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2024-04-24T23:32:22.596 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2024-04-24T23:32:22.596 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 0, 2024-04-24T23:32:22.596 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 0, 2024-04-24T23:32:22.596 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-04-24T23:32:22.596 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-24T23:32:22.596 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-24T23:32:22.596 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-24T23:32:22.596 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-24T23:32:22.597 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:22.597 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-write_buf_throttle-0x557b32ff26e0": { 2024-04-24T23:32:22.597 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-04-24T23:32:22.597 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 3758096384, 2024-04-24T23:32:22.597 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-04-24T23:32:22.597 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2024-04-24T23:32:22.597 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2024-04-24T23:32:22.597 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-04-24T23:32:22.598 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2024-04-24T23:32:22.598 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2024-04-24T23:32:22.598 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2024-04-24T23:32:22.598 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 0, 2024-04-24T23:32:22.598 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 0, 2024-04-24T23:32:22.598 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-04-24T23:32:22.598 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-24T23:32:22.598 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-24T23:32:22.598 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-24T23:32:22.599 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-24T23:32:22.599 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-24T23:32:22.599 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-04-24T23:32:23.366 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-04-24T23:32:23.367 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-04-24T23:32:23.367 INFO:tasks.check_counter:Daemon mds.h mds.dir_split=0 2024-04-24T23:32:23.367 DEBUG:tasks.check_counter:Getting stats from k 2024-04-24T23:32:23.367 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid ec7885aa-0290-11ef-bc93-c7b262605968 -- ceph daemon mds.k perf dump 2024-04-24T23:32:23.615 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/mon.b/config 2024-04-24T23:32:24.196 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:32:24 smithi134 ceph-mon[32619]: pgmap v452: 129 pgs: 129 active+clean; 74 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 3.0 KiB/s wr, 7 op/s 2024-04-24T23:32:24.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:32:24 smithi148 ceph-mon[32528]: pgmap v452: 129 pgs: 129 active+clean; 74 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 3.0 KiB/s wr, 7 op/s 2024-04-24T23:32:24.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:32:24 smithi080 ceph-mon[29152]: pgmap v452: 129 pgs: 129 active+clean; 74 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 3.0 KiB/s wr, 7 op/s 2024-04-24T23:32:25.387 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-04-24T23:32:25.388 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-0": { 2024-04-24T23:32:25.388 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 98, 2024-04-24T23:32:25.388 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 95, 2024-04-24T23:32:25.388 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 443232, 2024-04-24T23:32:25.389 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 57312, 2024-04-24T23:32:25.389 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 4, 2024-04-24T23:32:25.389 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 0, 2024-04-24T23:32:25.389 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.019521479, 2024-04-24T23:32:25.389 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.006031446, 2024-04-24T23:32:25.389 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.011949631, 2024-04-24T23:32:25.389 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.002452273, 2024-04-24T23:32:25.390 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:32:25.390 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 94, 2024-04-24T23:32:25.390 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.005251028, 2024-04-24T23:32:25.390 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000055862 2024-04-24T23:32:25.390 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:25.390 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:32:25.390 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-24T23:32:25.390 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-24T23:32:25.391 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-24T23:32:25.391 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:25.391 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 443232, 2024-04-24T23:32:25.391 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 57312 2024-04-24T23:32:25.391 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:25.391 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-1": { 2024-04-24T23:32:25.391 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 5, 2024-04-24T23:32:25.391 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 74, 2024-04-24T23:32:25.392 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 3152, 2024-04-24T23:32:25.392 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 90672, 2024-04-24T23:32:25.392 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 3, 2024-04-24T23:32:25.392 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 2, 2024-04-24T23:32:25.392 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.005593795, 2024-04-24T23:32:25.392 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.003379385, 2024-04-24T23:32:25.392 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.001322960, 2024-04-24T23:32:25.392 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-24T23:32:25.393 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:32:25.393 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 73, 2024-04-24T23:32:25.393 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.005459301, 2024-04-24T23:32:25.393 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000074784 2024-04-24T23:32:25.393 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:25.393 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:32:25.393 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-24T23:32:25.393 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-24T23:32:25.393 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-24T23:32:25.394 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:25.394 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 3152, 2024-04-24T23:32:25.394 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 90672 2024-04-24T23:32:25.394 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:25.394 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-2": { 2024-04-24T23:32:25.394 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 0, 2024-04-24T23:32:25.394 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 2, 2024-04-24T23:32:25.395 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 0, 2024-04-24T23:32:25.395 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 0, 2024-04-24T23:32:25.395 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 2, 2024-04-24T23:32:25.395 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 1, 2024-04-24T23:32:25.395 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.000197782, 2024-04-24T23:32:25.395 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.000000000, 2024-04-24T23:32:25.395 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.000141081, 2024-04-24T23:32:25.395 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-24T23:32:25.396 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:32:25.396 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-24T23:32:25.396 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-24T23:32:25.396 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-24T23:32:25.396 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:25.396 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:32:25.396 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-24T23:32:25.396 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-24T23:32:25.397 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-24T23:32:25.397 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:25.397 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-24T23:32:25.397 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 0 2024-04-24T23:32:25.397 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:25.397 INFO:teuthology.orchestra.run.smithi134.stdout: "cct": { 2024-04-24T23:32:25.397 INFO:teuthology.orchestra.run.smithi134.stdout: "total_workers": 0, 2024-04-24T23:32:25.397 INFO:teuthology.orchestra.run.smithi134.stdout: "unhealthy_workers": 0 2024-04-24T23:32:25.398 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:25.398 INFO:teuthology.orchestra.run.smithi134.stdout: "mempool": { 2024-04-24T23:32:25.398 INFO:teuthology.orchestra.run.smithi134.stdout: "bloom_filter_bytes": 0, 2024-04-24T23:32:25.398 INFO:teuthology.orchestra.run.smithi134.stdout: "bloom_filter_items": 0, 2024-04-24T23:32:25.398 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_alloc_bytes": 0, 2024-04-24T23:32:25.398 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_alloc_items": 0, 2024-04-24T23:32:25.398 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_data_bytes": 0, 2024-04-24T23:32:25.398 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_data_items": 0, 2024-04-24T23:32:25.399 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-24T23:32:25.399 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_onode_items": 0, 2024-04-24T23:32:25.399 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-24T23:32:25.399 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_meta_items": 0, 2024-04-24T23:32:25.399 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_other_bytes": 0, 2024-04-24T23:32:25.399 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_other_items": 0, 2024-04-24T23:32:25.399 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-24T23:32:25.399 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_buffer_items": 0, 2024-04-24T23:32:25.399 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_extent_bytes": 0, 2024-04-24T23:32:25.400 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_extent_items": 0, 2024-04-24T23:32:25.400 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_blob_bytes": 0, 2024-04-24T23:32:25.400 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_blob_items": 0, 2024-04-24T23:32:25.400 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-24T23:32:25.400 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_shared_blob_items": 0, 2024-04-24T23:32:25.400 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-24T23:32:25.400 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_inline_bl_items": 0, 2024-04-24T23:32:25.400 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_fsck_bytes": 0, 2024-04-24T23:32:25.400 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_fsck_items": 0, 2024-04-24T23:32:25.401 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_txc_bytes": 0, 2024-04-24T23:32:25.401 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_txc_items": 0, 2024-04-24T23:32:25.401 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-24T23:32:25.401 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_deferred_items": 0, 2024-04-24T23:32:25.401 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_bytes": 0, 2024-04-24T23:32:25.401 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_items": 0, 2024-04-24T23:32:25.401 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_bytes": 0, 2024-04-24T23:32:25.401 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_items": 0, 2024-04-24T23:32:25.402 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_reader_bytes": 0, 2024-04-24T23:32:25.402 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_reader_items": 0, 2024-04-24T23:32:25.402 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_writer_bytes": 0, 2024-04-24T23:32:25.402 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_writer_items": 0, 2024-04-24T23:32:25.402 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_anon_bytes": 148818, 2024-04-24T23:32:25.402 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_anon_items": 28, 2024-04-24T23:32:25.402 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_meta_bytes": 0, 2024-04-24T23:32:25.402 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_meta_items": 0, 2024-04-24T23:32:25.403 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_bytes": 0, 2024-04-24T23:32:25.403 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_items": 0, 2024-04-24T23:32:25.403 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_mapbl_bytes": 0, 2024-04-24T23:32:25.403 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_mapbl_items": 0, 2024-04-24T23:32:25.403 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_pglog_bytes": 0, 2024-04-24T23:32:25.403 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_pglog_items": 0, 2024-04-24T23:32:25.403 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_bytes": 0, 2024-04-24T23:32:25.403 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_items": 0, 2024-04-24T23:32:25.404 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_mapping_bytes": 0, 2024-04-24T23:32:25.404 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_mapping_items": 0, 2024-04-24T23:32:25.404 INFO:teuthology.orchestra.run.smithi134.stdout: "pgmap_bytes": 0, 2024-04-24T23:32:25.404 INFO:teuthology.orchestra.run.smithi134.stdout: "pgmap_items": 0, 2024-04-24T23:32:25.404 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_co_bytes": 928, 2024-04-24T23:32:25.404 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_co_items": 2, 2024-04-24T23:32:25.404 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_1_bytes": 0, 2024-04-24T23:32:25.404 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_1_items": 0, 2024-04-24T23:32:25.405 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_2_bytes": 0, 2024-04-24T23:32:25.405 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_2_items": 0 2024-04-24T23:32:25.405 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-24T23:32:25.405 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-24T23:32:25.405 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-04-24T23:32:25.405 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 104857600, 2024-04-24T23:32:25.405 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-04-24T23:32:25.405 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 99, 2024-04-24T23:32:25.406 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 429416, 2024-04-24T23:32:25.406 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-04-24T23:32:25.406 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 99, 2024-04-24T23:32:25.406 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2024-04-24T23:32:25.406 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2024-04-24T23:32:25.406 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 99, 2024-04-24T23:32:25.406 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 429416, 2024-04-24T23:32:25.406 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-04-24T23:32:25.407 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-24T23:32:25.407 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-24T23:32:25.407 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-24T23:32:25.407 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-24T23:32:25.407 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-24T23:32:25.407 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-04-24T23:32:26.123 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-04-24T23:32:26.123 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-04-24T23:32:26.123 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.k 2024-04-24T23:32:26.123 DEBUG:tasks.check_counter:Getting stats from c 2024-04-24T23:32:26.123 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid ec7885aa-0290-11ef-bc93-c7b262605968 -- ceph daemon mds.c perf dump 2024-04-24T23:32:26.374 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T23:32:26.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:32:26 smithi134 ceph-mon[32619]: pgmap v453: 129 pgs: 129 active+clean; 75 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 67 KiB/s wr, 10 op/s 2024-04-24T23:32:26.496 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:32:26 smithi148 ceph-mon[32528]: pgmap v453: 129 pgs: 129 active+clean; 75 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 67 KiB/s wr, 10 op/s 2024-04-24T23:32:26.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:32:26 smithi080 ceph-mon[29152]: pgmap v453: 129 pgs: 129 active+clean; 75 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 67 KiB/s wr, 10 op/s 2024-04-24T23:32:28.216 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-24T23:32:28.216 INFO:teuthology.orchestra.run.smithi148.stdout: "AsyncMessenger::Worker-0": { 2024-04-24T23:32:28.216 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_messages": 3885, 2024-04-24T23:32:28.216 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages": 6409, 2024-04-24T23:32:28.216 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_bytes": 1819965, 2024-04-24T23:32:28.217 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_bytes": 6102215, 2024-04-24T23:32:28.217 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_created_connections": 16, 2024-04-24T23:32:28.217 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_active_connections": 5, 2024-04-24T23:32:28.217 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_total_time": 0.299390174, 2024-04-24T23:32:28.217 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_send_time": 0.127901560, 2024-04-24T23:32:28.217 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_recv_time": 0.187870922, 2024-04-24T23:32:28.217 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_fast_dispatch_time": 0.030179794, 2024-04-24T23:32:28.217 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:32:28.218 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 6409, 2024-04-24T23:32:28.218 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 1.391757251, 2024-04-24T23:32:28.218 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000217156 2024-04-24T23:32:28.218 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:28.218 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:32:28.218 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 7490, 2024-04-24T23:32:28.218 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.002713952, 2024-04-24T23:32:28.218 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000362 2024-04-24T23:32:28.219 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:28.219 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_encrypted_bytes": 1676160, 2024-04-24T23:32:28.219 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_encrypted_bytes": 3184800 2024-04-24T23:32:28.219 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:28.219 INFO:teuthology.orchestra.run.smithi148.stdout: "AsyncMessenger::Worker-1": { 2024-04-24T23:32:28.219 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_messages": 702, 2024-04-24T23:32:28.219 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages": 770, 2024-04-24T23:32:28.219 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_bytes": 179599, 2024-04-24T23:32:28.219 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_bytes": 2717947, 2024-04-24T23:32:28.220 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_created_connections": 8, 2024-04-24T23:32:28.220 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_active_connections": 6, 2024-04-24T23:32:28.220 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_total_time": 0.056924111, 2024-04-24T23:32:28.220 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_send_time": 0.021716819, 2024-04-24T23:32:28.220 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_recv_time": 0.014060929, 2024-04-24T23:32:28.220 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_fast_dispatch_time": 0.015630062, 2024-04-24T23:32:28.221 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:32:28.221 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 768, 2024-04-24T23:32:28.221 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.067489816, 2024-04-24T23:32:28.221 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000087877 2024-04-24T23:32:28.221 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:28.221 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:32:28.221 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 150, 2024-04-24T23:32:28.221 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000013049, 2024-04-24T23:32:28.222 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000086 2024-04-24T23:32:28.222 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:28.222 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-24T23:32:28.222 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_encrypted_bytes": 189536 2024-04-24T23:32:28.222 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:28.222 INFO:teuthology.orchestra.run.smithi148.stdout: "AsyncMessenger::Worker-2": { 2024-04-24T23:32:28.222 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_messages": 2398, 2024-04-24T23:32:28.222 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages": 3496, 2024-04-24T23:32:28.223 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_bytes": 2242815, 2024-04-24T23:32:28.223 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_bytes": 1142002, 2024-04-24T23:32:28.223 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_created_connections": 15, 2024-04-24T23:32:28.223 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_active_connections": 5, 2024-04-24T23:32:28.223 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_total_time": 0.219252592, 2024-04-24T23:32:28.223 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_send_time": 0.092715791, 2024-04-24T23:32:28.223 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_recv_time": 0.094573755, 2024-04-24T23:32:28.223 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_fast_dispatch_time": 0.014465221, 2024-04-24T23:32:28.223 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:32:28.224 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 3494, 2024-04-24T23:32:28.224 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.365259682, 2024-04-24T23:32:28.224 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000104539 2024-04-24T23:32:28.224 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:28.224 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:32:28.224 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 4263, 2024-04-24T23:32:28.224 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000934657, 2024-04-24T23:32:28.224 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000219 2024-04-24T23:32:28.225 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:28.225 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-24T23:32:28.225 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_encrypted_bytes": 0 2024-04-24T23:32:28.225 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:28.225 INFO:teuthology.orchestra.run.smithi148.stdout: "cct": { 2024-04-24T23:32:28.225 INFO:teuthology.orchestra.run.smithi148.stdout: "total_workers": 1, 2024-04-24T23:32:28.225 INFO:teuthology.orchestra.run.smithi148.stdout: "unhealthy_workers": 0 2024-04-24T23:32:28.225 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:28.226 INFO:teuthology.orchestra.run.smithi148.stdout: "finisher-MDSRank": { 2024-04-24T23:32:28.226 INFO:teuthology.orchestra.run.smithi148.stdout: "queue_len": 0, 2024-04-24T23:32:28.226 INFO:teuthology.orchestra.run.smithi148.stdout: "complete_latency": { 2024-04-24T23:32:28.226 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 573, 2024-04-24T23:32:28.226 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 2.038213113, 2024-04-24T23:32:28.226 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.003557090 2024-04-24T23:32:28.226 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-24T23:32:28.226 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:28.227 INFO:teuthology.orchestra.run.smithi148.stdout: "finisher-PurgeQueue": { 2024-04-24T23:32:28.227 INFO:teuthology.orchestra.run.smithi148.stdout: "queue_len": 0, 2024-04-24T23:32:28.227 INFO:teuthology.orchestra.run.smithi148.stdout: "complete_latency": { 2024-04-24T23:32:28.227 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 734, 2024-04-24T23:32:28.227 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.088231462, 2024-04-24T23:32:28.227 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000120206 2024-04-24T23:32:28.227 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-24T23:32:28.227 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:28.228 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": { 2024-04-24T23:32:28.228 INFO:teuthology.orchestra.run.smithi148.stdout: "request": 1434, 2024-04-24T23:32:28.228 INFO:teuthology.orchestra.run.smithi148.stdout: "reply": 1430, 2024-04-24T23:32:28.228 INFO:teuthology.orchestra.run.smithi148.stdout: "reply_latency": { 2024-04-24T23:32:28.228 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 1430, 2024-04-24T23:32:28.228 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 1.373695867, 2024-04-24T23:32:28.228 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000960626 2024-04-24T23:32:28.229 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:28.229 INFO:teuthology.orchestra.run.smithi148.stdout: "slow_reply": 0, 2024-04-24T23:32:28.229 INFO:teuthology.orchestra.run.smithi148.stdout: "forward": 4, 2024-04-24T23:32:28.229 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_fetch_complete": 0, 2024-04-24T23:32:28.229 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_fetch_keys": 0, 2024-04-24T23:32:28.229 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_commit": 11, 2024-04-24T23:32:28.229 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_split": 0, 2024-04-24T23:32:28.229 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_merge": 0, 2024-04-24T23:32:28.229 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes": 1761, 2024-04-24T23:32:28.230 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_top": 11, 2024-04-24T23:32:28.230 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_bottom": 1750, 2024-04-24T23:32:28.230 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_pin_tail": 0, 2024-04-24T23:32:28.230 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_pinned": 1710, 2024-04-24T23:32:28.230 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_expired": 0, 2024-04-24T23:32:28.230 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_with_caps": 79, 2024-04-24T23:32:28.230 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": 79, 2024-04-24T23:32:28.230 INFO:teuthology.orchestra.run.smithi148.stdout: "subtrees": 76, 2024-04-24T23:32:28.231 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse": 2914, 2024-04-24T23:32:28.231 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_hit": 1783, 2024-04-24T23:32:28.231 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_forward": 1, 2024-04-24T23:32:28.231 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_discover": 86, 2024-04-24T23:32:28.231 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_dir_fetch": 0, 2024-04-24T23:32:28.231 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_remote_ino": 0, 2024-04-24T23:32:28.231 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_lock": 0, 2024-04-24T23:32:28.231 INFO:teuthology.orchestra.run.smithi148.stdout: "load_cent": 5773, 2024-04-24T23:32:28.231 INFO:teuthology.orchestra.run.smithi148.stdout: "q": 0, 2024-04-24T23:32:28.232 INFO:teuthology.orchestra.run.smithi148.stdout: "exported": 0, 2024-04-24T23:32:28.232 INFO:teuthology.orchestra.run.smithi148.stdout: "exported_inodes": 0, 2024-04-24T23:32:28.232 INFO:teuthology.orchestra.run.smithi148.stdout: "imported": 73, 2024-04-24T23:32:28.232 INFO:teuthology.orchestra.run.smithi148.stdout: "imported_inodes": 1671, 2024-04-24T23:32:28.232 INFO:teuthology.orchestra.run.smithi148.stdout: "openino_dir_fetch": 0, 2024-04-24T23:32:28.232 INFO:teuthology.orchestra.run.smithi148.stdout: "openino_backtrace_fetch": 761, 2024-04-24T23:32:28.232 INFO:teuthology.orchestra.run.smithi148.stdout: "openino_peer_discover": 0, 2024-04-24T23:32:28.232 INFO:teuthology.orchestra.run.smithi148.stdout: "root_rfiles": 3429, 2024-04-24T23:32:28.233 INFO:teuthology.orchestra.run.smithi148.stdout: "root_rbytes": 23106296, 2024-04-24T23:32:28.233 INFO:teuthology.orchestra.run.smithi148.stdout: "root_rsnaps": 0, 2024-04-24T23:32:28.233 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_backtrace_fetch": 761, 2024-04-24T23:32:28.233 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_set_tag": 0, 2024-04-24T23:32:28.233 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_backtrace_repaired": 0, 2024-04-24T23:32:28.233 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_inotable_repaired": 0, 2024-04-24T23:32:28.233 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_dir_inodes": 2, 2024-04-24T23:32:28.233 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_dir_base_inodes": 0, 2024-04-24T23:32:28.234 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_dirfrag_rstats": 2, 2024-04-24T23:32:28.234 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_file_inodes": 759, 2024-04-24T23:32:28.234 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_inode_file_caps": 0, 2024-04-24T23:32:28.234 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_revoke": 17, 2024-04-24T23:32:28.234 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_grant": 865, 2024-04-24T23:32:28.234 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_trunc": 0, 2024-04-24T23:32:28.234 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-24T23:32:28.234 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-24T23:32:28.235 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_caps": 1649, 2024-04-24T23:32:28.235 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_caps_dirty": 1210, 2024-04-24T23:32:28.235 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_cap_release": 15, 2024-04-24T23:32:28.235 INFO:teuthology.orchestra.run.smithi148.stdout: "process_request_cap_release": 874 2024-04-24T23:32:28.235 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:28.235 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_cache": { 2024-04-24T23:32:28.235 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_update": 0, 2024-04-24T23:32:28.235 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_update_receipt": 0, 2024-04-24T23:32:28.236 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_try_discover": 0, 2024-04-24T23:32:28.236 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_send_discover": 46, 2024-04-24T23:32:28.236 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_handle_discover": 63, 2024-04-24T23:32:28.236 INFO:teuthology.orchestra.run.smithi148.stdout: "num_strays": 0, 2024-04-24T23:32:28.236 INFO:teuthology.orchestra.run.smithi148.stdout: "num_strays_delayed": 0, 2024-04-24T23:32:28.236 INFO:teuthology.orchestra.run.smithi148.stdout: "num_strays_enqueuing": 0, 2024-04-24T23:32:28.236 INFO:teuthology.orchestra.run.smithi148.stdout: "strays_created": 423, 2024-04-24T23:32:28.236 INFO:teuthology.orchestra.run.smithi148.stdout: "strays_enqueued": 423, 2024-04-24T23:32:28.236 INFO:teuthology.orchestra.run.smithi148.stdout: "strays_reintegrated": 0, 2024-04-24T23:32:28.237 INFO:teuthology.orchestra.run.smithi148.stdout: "strays_migrated": 0, 2024-04-24T23:32:28.237 INFO:teuthology.orchestra.run.smithi148.stdout: "num_recovering_processing": 0, 2024-04-24T23:32:28.237 INFO:teuthology.orchestra.run.smithi148.stdout: "num_recovering_enqueued": 0, 2024-04-24T23:32:28.237 INFO:teuthology.orchestra.run.smithi148.stdout: "num_recovering_prioritized": 0, 2024-04-24T23:32:28.237 INFO:teuthology.orchestra.run.smithi148.stdout: "recovery_started": 0, 2024-04-24T23:32:28.237 INFO:teuthology.orchestra.run.smithi148.stdout: "recovery_completed": 0, 2024-04-24T23:32:28.237 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_quiesce_path": 0, 2024-04-24T23:32:28.237 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_quiesce_inode": 0, 2024-04-24T23:32:28.238 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_enqueue_scrub": 0, 2024-04-24T23:32:28.238 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_exportdir": 0, 2024-04-24T23:32:28.238 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_flush": 0, 2024-04-24T23:32:28.238 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_fragmentdir": 0, 2024-04-24T23:32:28.238 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_fragstats": 0, 2024-04-24T23:32:28.238 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_inodestats": 0 2024-04-24T23:32:28.238 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:28.238 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_log": { 2024-04-24T23:32:28.239 INFO:teuthology.orchestra.run.smithi148.stdout: "evlrg": 0, 2024-04-24T23:32:28.239 INFO:teuthology.orchestra.run.smithi148.stdout: "evadd": 3679, 2024-04-24T23:32:28.239 INFO:teuthology.orchestra.run.smithi148.stdout: "evex": 0, 2024-04-24T23:32:28.239 INFO:teuthology.orchestra.run.smithi148.stdout: "evtrm": 0, 2024-04-24T23:32:28.239 INFO:teuthology.orchestra.run.smithi148.stdout: "ev": 3679, 2024-04-24T23:32:28.239 INFO:teuthology.orchestra.run.smithi148.stdout: "evexg": 0, 2024-04-24T23:32:28.239 INFO:teuthology.orchestra.run.smithi148.stdout: "evexd": 0, 2024-04-24T23:32:28.239 INFO:teuthology.orchestra.run.smithi148.stdout: "segadd": 6, 2024-04-24T23:32:28.239 INFO:teuthology.orchestra.run.smithi148.stdout: "segex": 0, 2024-04-24T23:32:28.240 INFO:teuthology.orchestra.run.smithi148.stdout: "segtrm": 0, 2024-04-24T23:32:28.240 INFO:teuthology.orchestra.run.smithi148.stdout: "seg": 6, 2024-04-24T23:32:28.240 INFO:teuthology.orchestra.run.smithi148.stdout: "segmjr": 2, 2024-04-24T23:32:28.240 INFO:teuthology.orchestra.run.smithi148.stdout: "segexg": 0, 2024-04-24T23:32:28.240 INFO:teuthology.orchestra.run.smithi148.stdout: "segexd": 0, 2024-04-24T23:32:28.240 INFO:teuthology.orchestra.run.smithi148.stdout: "expos": 4194304, 2024-04-24T23:32:28.240 INFO:teuthology.orchestra.run.smithi148.stdout: "wrpos": 10610429, 2024-04-24T23:32:28.240 INFO:teuthology.orchestra.run.smithi148.stdout: "rdpos": 0, 2024-04-24T23:32:28.241 INFO:teuthology.orchestra.run.smithi148.stdout: "jlat": { 2024-04-24T23:32:28.241 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 112, 2024-04-24T23:32:28.241 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 2.849898844, 2024-04-24T23:32:28.241 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.025445525 2024-04-24T23:32:28.241 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:28.241 INFO:teuthology.orchestra.run.smithi148.stdout: "replayed": 0 2024-04-24T23:32:28.241 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:28.241 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_mem": { 2024-04-24T23:32:28.241 INFO:teuthology.orchestra.run.smithi148.stdout: "ino": 1291, 2024-04-24T23:32:28.242 INFO:teuthology.orchestra.run.smithi148.stdout: "ino+": 1714, 2024-04-24T23:32:28.242 INFO:teuthology.orchestra.run.smithi148.stdout: "ino-": 423, 2024-04-24T23:32:28.242 INFO:teuthology.orchestra.run.smithi148.stdout: "dir": 98, 2024-04-24T23:32:28.242 INFO:teuthology.orchestra.run.smithi148.stdout: "dir+": 100, 2024-04-24T23:32:28.242 INFO:teuthology.orchestra.run.smithi148.stdout: "dir-": 2, 2024-04-24T23:32:28.242 INFO:teuthology.orchestra.run.smithi148.stdout: "dn": 1761, 2024-04-24T23:32:28.242 INFO:teuthology.orchestra.run.smithi148.stdout: "dn+": 2208, 2024-04-24T23:32:28.242 INFO:teuthology.orchestra.run.smithi148.stdout: "dn-": 447, 2024-04-24T23:32:28.243 INFO:teuthology.orchestra.run.smithi148.stdout: "cap": 79, 2024-04-24T23:32:28.243 INFO:teuthology.orchestra.run.smithi148.stdout: "cap+": 1691, 2024-04-24T23:32:28.243 INFO:teuthology.orchestra.run.smithi148.stdout: "cap-": 1612, 2024-04-24T23:32:28.243 INFO:teuthology.orchestra.run.smithi148.stdout: "rss": 75608, 2024-04-24T23:32:28.243 INFO:teuthology.orchestra.run.smithi148.stdout: "heap": 223612 2024-04-24T23:32:28.243 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:28.243 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_server": { 2024-04-24T23:32:28.243 INFO:teuthology.orchestra.run.smithi148.stdout: "dispatch_client_request": 2286, 2024-04-24T23:32:28.243 INFO:teuthology.orchestra.run.smithi148.stdout: "dispatch_server_request": 0, 2024-04-24T23:32:28.244 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_request": 1434, 2024-04-24T23:32:28.244 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_session": 7, 2024-04-24T23:32:28.244 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_peer_request": 810, 2024-04-24T23:32:28.244 INFO:teuthology.orchestra.run.smithi148.stdout: "req_create_latency": { 2024-04-24T23:32:28.244 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 409, 2024-04-24T23:32:28.244 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.482041438, 2024-04-24T23:32:28.244 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.001178585 2024-04-24T23:32:28.244 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:28.245 INFO:teuthology.orchestra.run.smithi148.stdout: "req_getattr_latency": { 2024-04-24T23:32:28.245 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 2, 2024-04-24T23:32:28.245 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000991533, 2024-04-24T23:32:28.245 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000495766 2024-04-24T23:32:28.245 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:28.245 INFO:teuthology.orchestra.run.smithi148.stdout: "req_getfilelock_latency": { 2024-04-24T23:32:28.245 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-24T23:32:28.245 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-24T23:32:28.245 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-24T23:32:28.246 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:28.246 INFO:teuthology.orchestra.run.smithi148.stdout: "req_link_latency": { 2024-04-24T23:32:28.246 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-24T23:32:28.246 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-24T23:32:28.246 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-24T23:32:28.246 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:28.246 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookup_latency": { 2024-04-24T23:32:28.246 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 548, 2024-04-24T23:32:28.247 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.136436199, 2024-04-24T23:32:28.247 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000248971 2024-04-24T23:32:28.247 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:28.247 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookuphash_latency": { 2024-04-24T23:32:28.247 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-24T23:32:28.247 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-24T23:32:28.247 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-24T23:32:28.247 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:28.248 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookupino_latency": { 2024-04-24T23:32:28.248 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 1, 2024-04-24T23:32:28.248 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000131559, 2024-04-24T23:32:28.248 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000131559 2024-04-24T23:32:28.248 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:28.248 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookupname_latency": { 2024-04-24T23:32:28.248 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-24T23:32:28.248 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-24T23:32:28.249 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-24T23:32:28.249 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:28.249 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookupparent_latency": { 2024-04-24T23:32:28.249 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-24T23:32:28.249 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-24T23:32:28.249 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-24T23:32:28.249 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:28.249 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookupsnap_latency": { 2024-04-24T23:32:28.250 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-24T23:32:28.250 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-24T23:32:28.250 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-24T23:32:28.250 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:28.250 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lssnap_latency": { 2024-04-24T23:32:28.250 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-24T23:32:28.251 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-24T23:32:28.251 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-24T23:32:28.251 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:28.252 INFO:teuthology.orchestra.run.smithi148.stdout: "req_mkdir_latency": { 2024-04-24T23:32:28.252 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 1, 2024-04-24T23:32:28.252 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.003706704, 2024-04-24T23:32:28.252 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.003706704 2024-04-24T23:32:28.252 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:28.252 INFO:teuthology.orchestra.run.smithi148.stdout: "req_mknod_latency": { 2024-04-24T23:32:28.252 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-24T23:32:28.252 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-24T23:32:28.253 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-24T23:32:28.253 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:28.253 INFO:teuthology.orchestra.run.smithi148.stdout: "req_mksnap_latency": { 2024-04-24T23:32:28.253 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-24T23:32:28.253 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-24T23:32:28.253 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-24T23:32:28.253 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:28.253 INFO:teuthology.orchestra.run.smithi148.stdout: "req_open_latency": { 2024-04-24T23:32:28.254 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 27, 2024-04-24T23:32:28.254 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.005827328, 2024-04-24T23:32:28.254 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000215826 2024-04-24T23:32:28.254 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:28.254 INFO:teuthology.orchestra.run.smithi148.stdout: "req_readdir_latency": { 2024-04-24T23:32:28.254 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 19, 2024-04-24T23:32:28.254 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.013441066, 2024-04-24T23:32:28.254 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000707424 2024-04-24T23:32:28.255 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:28.255 INFO:teuthology.orchestra.run.smithi148.stdout: "req_rename_latency": { 2024-04-24T23:32:28.255 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-24T23:32:28.255 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-24T23:32:28.255 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-24T23:32:28.255 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:28.255 INFO:teuthology.orchestra.run.smithi148.stdout: "req_renamesnap_latency": { 2024-04-24T23:32:28.255 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-24T23:32:28.256 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-24T23:32:28.256 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-24T23:32:28.256 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:28.256 INFO:teuthology.orchestra.run.smithi148.stdout: "req_snapdiff_latency": { 2024-04-24T23:32:28.256 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-24T23:32:28.256 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-24T23:32:28.256 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-24T23:32:28.256 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:28.257 INFO:teuthology.orchestra.run.smithi148.stdout: "req_rmdir_latency": { 2024-04-24T23:32:28.257 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 1, 2024-04-24T23:32:28.257 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.030013729, 2024-04-24T23:32:28.257 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.030013729 2024-04-24T23:32:28.257 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:28.257 INFO:teuthology.orchestra.run.smithi148.stdout: "req_rmsnap_latency": { 2024-04-24T23:32:28.257 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-24T23:32:28.257 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-24T23:32:28.258 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-24T23:32:28.258 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:28.258 INFO:teuthology.orchestra.run.smithi148.stdout: "req_rmxattr_latency": { 2024-04-24T23:32:28.258 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-24T23:32:28.258 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-24T23:32:28.258 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-24T23:32:28.258 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:28.259 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setattr_latency": { 2024-04-24T23:32:28.259 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-24T23:32:28.259 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-24T23:32:28.259 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-24T23:32:28.259 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:28.259 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setdirlayout_latency": { 2024-04-24T23:32:28.259 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-24T23:32:28.259 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-24T23:32:28.260 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-24T23:32:28.260 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:28.260 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setfilelock_latency": { 2024-04-24T23:32:28.260 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-24T23:32:28.260 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-24T23:32:28.260 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-24T23:32:28.260 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:28.260 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setlayout_latency": { 2024-04-24T23:32:28.261 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-24T23:32:28.261 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-24T23:32:28.261 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-24T23:32:28.261 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:28.261 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setxattr_latency": { 2024-04-24T23:32:28.261 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-24T23:32:28.261 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-24T23:32:28.261 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-24T23:32:28.262 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:28.262 INFO:teuthology.orchestra.run.smithi148.stdout: "req_symlink_latency": { 2024-04-24T23:32:28.262 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-24T23:32:28.262 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-24T23:32:28.262 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-24T23:32:28.262 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:28.262 INFO:teuthology.orchestra.run.smithi148.stdout: "req_unlink_latency": { 2024-04-24T23:32:28.262 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 422, 2024-04-24T23:32:28.263 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.701106311, 2024-04-24T23:32:28.263 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.001661389 2024-04-24T23:32:28.263 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:28.263 INFO:teuthology.orchestra.run.smithi148.stdout: "cap_revoke_eviction": 0, 2024-04-24T23:32:28.263 INFO:teuthology.orchestra.run.smithi148.stdout: "cap_acquisition_throttle": 0, 2024-04-24T23:32:28.263 INFO:teuthology.orchestra.run.smithi148.stdout: "req_getvxattr_latency": { 2024-04-24T23:32:28.263 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-24T23:32:28.263 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-24T23:32:28.264 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-24T23:32:28.264 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-24T23:32:28.264 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:28.264 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_sessions": { 2024-04-24T23:32:28.264 INFO:teuthology.orchestra.run.smithi148.stdout: "session_count": 1, 2024-04-24T23:32:28.264 INFO:teuthology.orchestra.run.smithi148.stdout: "session_add": 2, 2024-04-24T23:32:28.264 INFO:teuthology.orchestra.run.smithi148.stdout: "session_remove": 1, 2024-04-24T23:32:28.264 INFO:teuthology.orchestra.run.smithi148.stdout: "sessions_open": 1, 2024-04-24T23:32:28.264 INFO:teuthology.orchestra.run.smithi148.stdout: "sessions_stale": 0, 2024-04-24T23:32:28.265 INFO:teuthology.orchestra.run.smithi148.stdout: "total_load": 1128, 2024-04-24T23:32:28.265 INFO:teuthology.orchestra.run.smithi148.stdout: "average_load": 1128, 2024-04-24T23:32:28.265 INFO:teuthology.orchestra.run.smithi148.stdout: "avg_session_uptime": 162, 2024-04-24T23:32:28.265 INFO:teuthology.orchestra.run.smithi148.stdout: "mdthresh_evicted": 0 2024-04-24T23:32:28.265 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:28.265 INFO:teuthology.orchestra.run.smithi148.stdout: "mempool": { 2024-04-24T23:32:28.265 INFO:teuthology.orchestra.run.smithi148.stdout: "bloom_filter_bytes": 0, 2024-04-24T23:32:28.265 INFO:teuthology.orchestra.run.smithi148.stdout: "bloom_filter_items": 0, 2024-04-24T23:32:28.266 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_alloc_bytes": 0, 2024-04-24T23:32:28.266 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_alloc_items": 0, 2024-04-24T23:32:28.266 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_data_bytes": 0, 2024-04-24T23:32:28.266 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_data_items": 0, 2024-04-24T23:32:28.266 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-24T23:32:28.266 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_onode_items": 0, 2024-04-24T23:32:28.266 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-24T23:32:28.266 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_meta_items": 0, 2024-04-24T23:32:28.267 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_other_bytes": 0, 2024-04-24T23:32:28.267 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_other_items": 0, 2024-04-24T23:32:28.267 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-24T23:32:28.267 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_buffer_items": 0, 2024-04-24T23:32:28.267 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_extent_bytes": 0, 2024-04-24T23:32:28.267 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_extent_items": 0, 2024-04-24T23:32:28.267 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_blob_bytes": 0, 2024-04-24T23:32:28.267 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_blob_items": 0, 2024-04-24T23:32:28.267 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-24T23:32:28.268 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_shared_blob_items": 0, 2024-04-24T23:32:28.268 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-24T23:32:28.268 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_inline_bl_items": 0, 2024-04-24T23:32:28.268 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_fsck_bytes": 0, 2024-04-24T23:32:28.268 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_fsck_items": 0, 2024-04-24T23:32:28.268 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_txc_bytes": 0, 2024-04-24T23:32:28.268 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_txc_items": 0, 2024-04-24T23:32:28.268 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-24T23:32:28.269 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_deferred_items": 0, 2024-04-24T23:32:28.269 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_bytes": 0, 2024-04-24T23:32:28.269 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_items": 0, 2024-04-24T23:32:28.269 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_bytes": 0, 2024-04-24T23:32:28.269 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_items": 0, 2024-04-24T23:32:28.269 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_reader_bytes": 0, 2024-04-24T23:32:28.269 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_reader_items": 0, 2024-04-24T23:32:28.269 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_writer_bytes": 0, 2024-04-24T23:32:28.269 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_writer_items": 0, 2024-04-24T23:32:28.270 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_anon_bytes": 3565417, 2024-04-24T23:32:28.270 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_anon_items": 2204, 2024-04-24T23:32:28.270 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_meta_bytes": 880, 2024-04-24T23:32:28.270 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_meta_items": 10, 2024-04-24T23:32:28.270 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_bytes": 0, 2024-04-24T23:32:28.270 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_items": 0, 2024-04-24T23:32:28.270 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_mapbl_bytes": 0, 2024-04-24T23:32:28.270 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_mapbl_items": 0, 2024-04-24T23:32:28.271 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_pglog_bytes": 0, 2024-04-24T23:32:28.271 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_pglog_items": 0, 2024-04-24T23:32:28.271 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_bytes": 9080, 2024-04-24T23:32:28.271 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_items": 174, 2024-04-24T23:32:28.271 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_mapping_bytes": 0, 2024-04-24T23:32:28.271 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_mapping_items": 0, 2024-04-24T23:32:28.271 INFO:teuthology.orchestra.run.smithi148.stdout: "pgmap_bytes": 0, 2024-04-24T23:32:28.271 INFO:teuthology.orchestra.run.smithi148.stdout: "pgmap_items": 0, 2024-04-24T23:32:28.272 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_co_bytes": 4957314, 2024-04-24T23:32:28.272 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_co_items": 56002, 2024-04-24T23:32:28.272 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_1_bytes": 0, 2024-04-24T23:32:28.272 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_1_items": 0, 2024-04-24T23:32:28.272 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_2_bytes": 0, 2024-04-24T23:32:28.272 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_2_items": 0 2024-04-24T23:32:28.272 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:28.272 INFO:teuthology.orchestra.run.smithi148.stdout: "objecter": { 2024-04-24T23:32:28.273 INFO:teuthology.orchestra.run.smithi148.stdout: "op_active": 0, 2024-04-24T23:32:28.273 INFO:teuthology.orchestra.run.smithi148.stdout: "op_laggy": 0, 2024-04-24T23:32:28.273 INFO:teuthology.orchestra.run.smithi148.stdout: "op_send": 1801, 2024-04-24T23:32:28.273 INFO:teuthology.orchestra.run.smithi148.stdout: "op_send_bytes": 6611222, 2024-04-24T23:32:28.273 INFO:teuthology.orchestra.run.smithi148.stdout: "op_resend": 33, 2024-04-24T23:32:28.273 INFO:teuthology.orchestra.run.smithi148.stdout: "op_reply": 1801, 2024-04-24T23:32:28.273 INFO:teuthology.orchestra.run.smithi148.stdout: "op_latency": { 2024-04-24T23:32:28.273 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 1801, 2024-04-24T23:32:28.273 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 4.650889151, 2024-04-24T23:32:28.274 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.002582392 2024-04-24T23:32:28.274 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:28.274 INFO:teuthology.orchestra.run.smithi148.stdout: "op_inflight": 0, 2024-04-24T23:32:28.274 INFO:teuthology.orchestra.run.smithi148.stdout: "oplen_avg": { 2024-04-24T23:32:28.274 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 1801, 2024-04-24T23:32:28.274 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 1853 2024-04-24T23:32:28.274 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:28.274 INFO:teuthology.orchestra.run.smithi148.stdout: "op": 1801, 2024-04-24T23:32:28.275 INFO:teuthology.orchestra.run.smithi148.stdout: "op_r": 765, 2024-04-24T23:32:28.275 INFO:teuthology.orchestra.run.smithi148.stdout: "op_w": 1036, 2024-04-24T23:32:28.275 INFO:teuthology.orchestra.run.smithi148.stdout: "op_rmw": 0, 2024-04-24T23:32:28.275 INFO:teuthology.orchestra.run.smithi148.stdout: "op_pg": 0, 2024-04-24T23:32:28.275 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_stat": 0, 2024-04-24T23:32:28.275 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_create": 10, 2024-04-24T23:32:28.275 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_read": 4, 2024-04-24T23:32:28.275 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_write": 116, 2024-04-24T23:32:28.276 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_writefull": 15, 2024-04-24T23:32:28.276 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_writesame": 0, 2024-04-24T23:32:28.276 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_append": 0, 2024-04-24T23:32:28.276 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_zero": 0, 2024-04-24T23:32:28.276 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_truncate": 0, 2024-04-24T23:32:28.276 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_delete": 858, 2024-04-24T23:32:28.276 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_mapext": 0, 2024-04-24T23:32:28.276 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_sparse_read": 0, 2024-04-24T23:32:28.277 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_clonerange": 0, 2024-04-24T23:32:28.277 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_getxattr": 761, 2024-04-24T23:32:28.277 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_setxattr": 20, 2024-04-24T23:32:28.277 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_cmpxattr": 0, 2024-04-24T23:32:28.277 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_rmxattr": 0, 2024-04-24T23:32:28.277 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_resetxattrs": 0, 2024-04-24T23:32:28.277 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_call": 0, 2024-04-24T23:32:28.277 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_watch": 0, 2024-04-24T23:32:28.278 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_notify": 0, 2024-04-24T23:32:28.278 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_src_cmpxattr": 0, 2024-04-24T23:32:28.278 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_pgls": 0, 2024-04-24T23:32:28.278 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_pgls_filter": 0, 2024-04-24T23:32:28.278 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_other": 0, 2024-04-24T23:32:28.278 INFO:teuthology.orchestra.run.smithi148.stdout: "linger_active": 0, 2024-04-24T23:32:28.278 INFO:teuthology.orchestra.run.smithi148.stdout: "linger_send": 0, 2024-04-24T23:32:28.278 INFO:teuthology.orchestra.run.smithi148.stdout: "linger_resend": 0, 2024-04-24T23:32:28.279 INFO:teuthology.orchestra.run.smithi148.stdout: "linger_ping": 0, 2024-04-24T23:32:28.279 INFO:teuthology.orchestra.run.smithi148.stdout: "poolop_active": 0, 2024-04-24T23:32:28.279 INFO:teuthology.orchestra.run.smithi148.stdout: "poolop_send": 0, 2024-04-24T23:32:28.279 INFO:teuthology.orchestra.run.smithi148.stdout: "poolop_resend": 0, 2024-04-24T23:32:28.279 INFO:teuthology.orchestra.run.smithi148.stdout: "poolstat_active": 0, 2024-04-24T23:32:28.279 INFO:teuthology.orchestra.run.smithi148.stdout: "poolstat_send": 0, 2024-04-24T23:32:28.279 INFO:teuthology.orchestra.run.smithi148.stdout: "poolstat_resend": 0, 2024-04-24T23:32:28.279 INFO:teuthology.orchestra.run.smithi148.stdout: "statfs_active": 0, 2024-04-24T23:32:28.280 INFO:teuthology.orchestra.run.smithi148.stdout: "statfs_send": 0, 2024-04-24T23:32:28.280 INFO:teuthology.orchestra.run.smithi148.stdout: "statfs_resend": 0, 2024-04-24T23:32:28.280 INFO:teuthology.orchestra.run.smithi148.stdout: "command_active": 0, 2024-04-24T23:32:28.280 INFO:teuthology.orchestra.run.smithi148.stdout: "command_send": 0, 2024-04-24T23:32:28.280 INFO:teuthology.orchestra.run.smithi148.stdout: "command_resend": 0, 2024-04-24T23:32:28.280 INFO:teuthology.orchestra.run.smithi148.stdout: "map_epoch": 94, 2024-04-24T23:32:28.280 INFO:teuthology.orchestra.run.smithi148.stdout: "map_full": 0, 2024-04-24T23:32:28.280 INFO:teuthology.orchestra.run.smithi148.stdout: "map_inc": 4, 2024-04-24T23:32:28.280 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_sessions": 12, 2024-04-24T23:32:28.281 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_session_open": 12, 2024-04-24T23:32:28.281 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_session_close": 0, 2024-04-24T23:32:28.281 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_laggy": 0, 2024-04-24T23:32:28.281 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_wr": 60, 2024-04-24T23:32:28.281 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_rd": 0, 2024-04-24T23:32:28.281 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_del": 9 2024-04-24T23:32:28.281 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:28.281 INFO:teuthology.orchestra.run.smithi148.stdout: "oft": { 2024-04-24T23:32:28.282 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_total_objs": 1, 2024-04-24T23:32:28.282 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_total_kv_pairs": 0, 2024-04-24T23:32:28.282 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_total_updates": 0, 2024-04-24T23:32:28.282 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_total_removes": 0 2024-04-24T23:32:28.282 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:28.282 INFO:teuthology.orchestra.run.smithi148.stdout: "osd-slow-ops": { 2024-04-24T23:32:28.282 INFO:teuthology.orchestra.run.smithi148.stdout: "slow_ops_count": 0 2024-04-24T23:32:28.282 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:28.283 INFO:teuthology.orchestra.run.smithi148.stdout: "purge_queue": { 2024-04-24T23:32:28.283 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executing_ops": 0, 2024-04-24T23:32:28.283 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executing_ops_high_water": 16, 2024-04-24T23:32:28.283 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executing": 0, 2024-04-24T23:32:28.283 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executing_high_water": 8, 2024-04-24T23:32:28.283 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executed_ops": 845, 2024-04-24T23:32:28.283 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executed": 423, 2024-04-24T23:32:28.283 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_item_in_journal": 0 2024-04-24T23:32:28.284 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:28.284 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-24T23:32:28.284 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-04-24T23:32:28.284 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 104857600, 2024-04-24T23:32:28.284 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-04-24T23:32:28.284 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 6981, 2024-04-24T23:32:28.284 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 3460948, 2024-04-24T23:32:28.284 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-04-24T23:32:28.284 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 6981, 2024-04-24T23:32:28.285 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 0, 2024-04-24T23:32:28.285 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 0, 2024-04-24T23:32:28.285 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 6981, 2024-04-24T23:32:28.285 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 3460948, 2024-04-24T23:32:28.285 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-04-24T23:32:28.285 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-24T23:32:28.285 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-24T23:32:28.285 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-24T23:32:28.286 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-24T23:32:28.286 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:28.286 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-objecter_bytes": { 2024-04-24T23:32:28.286 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-04-24T23:32:28.286 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 104857600, 2024-04-24T23:32:28.286 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-04-24T23:32:28.286 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 0, 2024-04-24T23:32:28.286 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 0, 2024-04-24T23:32:28.286 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-04-24T23:32:28.287 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 0, 2024-04-24T23:32:28.287 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 1801, 2024-04-24T23:32:28.287 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 6657321, 2024-04-24T23:32:28.287 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 1801, 2024-04-24T23:32:28.287 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 6657321, 2024-04-24T23:32:28.287 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-04-24T23:32:28.287 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-24T23:32:28.287 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-24T23:32:28.288 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-24T23:32:28.288 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-24T23:32:28.288 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:28.288 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-objecter_ops": { 2024-04-24T23:32:28.288 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-04-24T23:32:28.288 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 1024, 2024-04-24T23:32:28.288 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-04-24T23:32:28.288 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 0, 2024-04-24T23:32:28.289 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 0, 2024-04-24T23:32:28.289 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-04-24T23:32:28.289 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 0, 2024-04-24T23:32:28.289 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 1801, 2024-04-24T23:32:28.289 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 1801, 2024-04-24T23:32:28.289 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 1801, 2024-04-24T23:32:28.289 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 1801, 2024-04-24T23:32:28.289 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-04-24T23:32:28.290 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-24T23:32:28.290 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-24T23:32:28.290 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-24T23:32:28.290 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-24T23:32:28.290 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:28.290 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-write_buf_throttle": { 2024-04-24T23:32:28.290 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-04-24T23:32:28.290 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 3758096384, 2024-04-24T23:32:28.291 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-04-24T23:32:28.291 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 423, 2024-04-24T23:32:28.291 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 46099, 2024-04-24T23:32:28.291 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-04-24T23:32:28.291 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 423, 2024-04-24T23:32:28.291 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 0, 2024-04-24T23:32:28.291 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 0, 2024-04-24T23:32:28.291 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 4, 2024-04-24T23:32:28.292 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 46099, 2024-04-24T23:32:28.292 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-04-24T23:32:28.292 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-24T23:32:28.292 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-24T23:32:28.292 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-24T23:32:28.292 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-24T23:32:28.292 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:28.292 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-write_buf_throttle-0x558a4e508500": { 2024-04-24T23:32:28.293 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-04-24T23:32:28.293 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 3758096384, 2024-04-24T23:32:28.293 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-04-24T23:32:28.293 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 3679, 2024-04-24T23:32:28.293 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 6416125, 2024-04-24T23:32:28.293 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-04-24T23:32:28.293 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 3679, 2024-04-24T23:32:28.293 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 0, 2024-04-24T23:32:28.294 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 0, 2024-04-24T23:32:28.294 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 112, 2024-04-24T23:32:28.294 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 6416125, 2024-04-24T23:32:28.294 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-04-24T23:32:28.294 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-24T23:32:28.294 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-24T23:32:28.294 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-24T23:32:28.295 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-24T23:32:28.295 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-24T23:32:28.295 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-24T23:32:28.446 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:32:28 smithi134 ceph-mon[32619]: pgmap v454: 129 pgs: 129 active+clean; 75 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 65 KiB/s wr, 6 op/s 2024-04-24T23:32:28.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:32:28 smithi148 ceph-mon[32528]: pgmap v454: 129 pgs: 129 active+clean; 75 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 65 KiB/s wr, 6 op/s 2024-04-24T23:32:28.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:32:28 smithi080 ceph-mon[29152]: pgmap v454: 129 pgs: 129 active+clean; 75 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 65 KiB/s wr, 6 op/s 2024-04-24T23:32:29.030 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-04-24T23:32:29.030 INFO:tasks.check_counter:Daemon mds.c mds.imported=73 2024-04-24T23:32:29.031 INFO:tasks.check_counter:Daemon mds.c mds.dir_split=0 2024-04-24T23:32:29.031 DEBUG:tasks.check_counter:Getting stats from f 2024-04-24T23:32:29.031 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid ec7885aa-0290-11ef-bc93-c7b262605968 -- ceph daemon mds.f perf dump 2024-04-24T23:32:29.280 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T23:32:30.446 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:32:30 smithi134 ceph-mon[32619]: pgmap v455: 129 pgs: 129 active+clean; 75 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 121 KiB/s wr, 8 op/s 2024-04-24T23:32:30.487 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:32:30 smithi148 ceph-mon[32528]: pgmap v455: 129 pgs: 129 active+clean; 75 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 121 KiB/s wr, 8 op/s 2024-04-24T23:32:30.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:32:30 smithi080 ceph-mon[29152]: pgmap v455: 129 pgs: 129 active+clean; 75 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 121 KiB/s wr, 8 op/s 2024-04-24T23:32:31.071 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-24T23:32:31.071 INFO:teuthology.orchestra.run.smithi148.stdout: "AsyncMessenger::Worker-0": { 2024-04-24T23:32:31.071 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_messages": 613, 2024-04-24T23:32:31.071 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages": 613, 2024-04-24T23:32:31.071 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_bytes": 17575225, 2024-04-24T23:32:31.071 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_bytes": 194120, 2024-04-24T23:32:31.071 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_created_connections": 7, 2024-04-24T23:32:31.072 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_active_connections": 3, 2024-04-24T23:32:31.072 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_total_time": 0.068976961, 2024-04-24T23:32:31.072 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_send_time": 0.018853752, 2024-04-24T23:32:31.072 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_recv_time": 0.025629752, 2024-04-24T23:32:31.072 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_fast_dispatch_time": 0.018560539, 2024-04-24T23:32:31.072 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:32:31.072 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 612, 2024-04-24T23:32:31.072 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.027252343, 2024-04-24T23:32:31.073 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000044529 2024-04-24T23:32:31.073 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:31.073 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:32:31.073 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-24T23:32:31.073 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-24T23:32:31.073 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-24T23:32:31.073 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:31.073 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-04-24T23:32:31.074 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_encrypted_bytes": 448 2024-04-24T23:32:31.074 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:31.074 INFO:teuthology.orchestra.run.smithi148.stdout: "AsyncMessenger::Worker-1": { 2024-04-24T23:32:31.074 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_messages": 103, 2024-04-24T23:32:31.074 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages": 172, 2024-04-24T23:32:31.074 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_bytes": 12612643, 2024-04-24T23:32:31.074 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_bytes": 223766, 2024-04-24T23:32:31.074 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_created_connections": 6, 2024-04-24T23:32:31.074 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_active_connections": 5, 2024-04-24T23:32:31.075 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_total_time": 0.019206517, 2024-04-24T23:32:31.075 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_send_time": 0.005181483, 2024-04-24T23:32:31.075 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_recv_time": 0.010161322, 2024-04-24T23:32:31.075 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_fast_dispatch_time": 0.002437022, 2024-04-24T23:32:31.075 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:32:31.075 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 170, 2024-04-24T23:32:31.075 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.009379343, 2024-04-24T23:32:31.075 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000055172 2024-04-24T23:32:31.076 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:31.076 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:32:31.076 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-24T23:32:31.076 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-24T23:32:31.076 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-24T23:32:31.076 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:31.076 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-24T23:32:31.076 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_encrypted_bytes": 191264 2024-04-24T23:32:31.076 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:31.077 INFO:teuthology.orchestra.run.smithi148.stdout: "AsyncMessenger::Worker-2": { 2024-04-24T23:32:31.077 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_messages": 362, 2024-04-24T23:32:31.077 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages": 356, 2024-04-24T23:32:31.077 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_bytes": 8923301, 2024-04-24T23:32:31.077 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_bytes": 138183, 2024-04-24T23:32:31.077 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_created_connections": 4, 2024-04-24T23:32:31.077 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_active_connections": 3, 2024-04-24T23:32:31.078 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_total_time": 0.044190093, 2024-04-24T23:32:31.078 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_send_time": 0.013744939, 2024-04-24T23:32:31.078 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_recv_time": 0.020548434, 2024-04-24T23:32:31.078 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_fast_dispatch_time": 0.009800476, 2024-04-24T23:32:31.078 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:32:31.078 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 355, 2024-04-24T23:32:31.078 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.017381021, 2024-04-24T23:32:31.078 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000048960 2024-04-24T23:32:31.079 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:31.079 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:32:31.079 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-24T23:32:31.079 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-24T23:32:31.079 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-24T23:32:31.079 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:31.079 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_encrypted_bytes": 471888, 2024-04-24T23:32:31.079 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_encrypted_bytes": 57728 2024-04-24T23:32:31.079 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:31.080 INFO:teuthology.orchestra.run.smithi148.stdout: "cct": { 2024-04-24T23:32:31.080 INFO:teuthology.orchestra.run.smithi148.stdout: "total_workers": 1, 2024-04-24T23:32:31.080 INFO:teuthology.orchestra.run.smithi148.stdout: "unhealthy_workers": 0 2024-04-24T23:32:31.080 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:31.080 INFO:teuthology.orchestra.run.smithi148.stdout: "finisher-MDSRank": { 2024-04-24T23:32:31.080 INFO:teuthology.orchestra.run.smithi148.stdout: "queue_len": 0, 2024-04-24T23:32:31.080 INFO:teuthology.orchestra.run.smithi148.stdout: "complete_latency": { 2024-04-24T23:32:31.080 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 1553, 2024-04-24T23:32:31.081 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.043022335, 2024-04-24T23:32:31.081 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000027702 2024-04-24T23:32:31.081 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-24T23:32:31.081 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:31.081 INFO:teuthology.orchestra.run.smithi148.stdout: "finisher-PurgeQueue": { 2024-04-24T23:32:31.081 INFO:teuthology.orchestra.run.smithi148.stdout: "queue_len": 0, 2024-04-24T23:32:31.081 INFO:teuthology.orchestra.run.smithi148.stdout: "complete_latency": { 2024-04-24T23:32:31.081 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-24T23:32:31.081 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-24T23:32:31.082 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-24T23:32:31.082 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-24T23:32:31.082 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:31.082 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": { 2024-04-24T23:32:31.082 INFO:teuthology.orchestra.run.smithi148.stdout: "request": 0, 2024-04-24T23:32:31.082 INFO:teuthology.orchestra.run.smithi148.stdout: "reply": 0, 2024-04-24T23:32:31.082 INFO:teuthology.orchestra.run.smithi148.stdout: "reply_latency": { 2024-04-24T23:32:31.082 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-24T23:32:31.082 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-24T23:32:31.083 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-24T23:32:31.083 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:31.083 INFO:teuthology.orchestra.run.smithi148.stdout: "slow_reply": 0, 2024-04-24T23:32:31.083 INFO:teuthology.orchestra.run.smithi148.stdout: "forward": 0, 2024-04-24T23:32:31.083 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_fetch_complete": 0, 2024-04-24T23:32:31.083 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_fetch_keys": 0, 2024-04-24T23:32:31.083 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_commit": 0, 2024-04-24T23:32:31.083 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_split": 0, 2024-04-24T23:32:31.084 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_merge": 0, 2024-04-24T23:32:31.084 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes": 2578, 2024-04-24T23:32:31.084 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_top": 0, 2024-04-24T23:32:31.084 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_bottom": 2578, 2024-04-24T23:32:31.084 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_pin_tail": 0, 2024-04-24T23:32:31.084 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_pinned": 2578, 2024-04-24T23:32:31.084 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_expired": 0, 2024-04-24T23:32:31.084 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_with_caps": 0, 2024-04-24T23:32:31.084 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": 0, 2024-04-24T23:32:31.085 INFO:teuthology.orchestra.run.smithi148.stdout: "subtrees": 22, 2024-04-24T23:32:31.085 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse": 0, 2024-04-24T23:32:31.085 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_hit": 0, 2024-04-24T23:32:31.085 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_forward": 0, 2024-04-24T23:32:31.085 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_discover": 0, 2024-04-24T23:32:31.085 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_dir_fetch": 0, 2024-04-24T23:32:31.086 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_remote_ino": 0, 2024-04-24T23:32:31.086 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_lock": 0, 2024-04-24T23:32:31.086 INFO:teuthology.orchestra.run.smithi148.stdout: "load_cent": 0, 2024-04-24T23:32:31.086 INFO:teuthology.orchestra.run.smithi148.stdout: "q": 0, 2024-04-24T23:32:31.086 INFO:teuthology.orchestra.run.smithi148.stdout: "exported": 0, 2024-04-24T23:32:31.086 INFO:teuthology.orchestra.run.smithi148.stdout: "exported_inodes": 0, 2024-04-24T23:32:31.086 INFO:teuthology.orchestra.run.smithi148.stdout: "imported": 0, 2024-04-24T23:32:31.086 INFO:teuthology.orchestra.run.smithi148.stdout: "imported_inodes": 0, 2024-04-24T23:32:31.087 INFO:teuthology.orchestra.run.smithi148.stdout: "openino_dir_fetch": 0, 2024-04-24T23:32:31.087 INFO:teuthology.orchestra.run.smithi148.stdout: "openino_backtrace_fetch": 0, 2024-04-24T23:32:31.087 INFO:teuthology.orchestra.run.smithi148.stdout: "openino_peer_discover": 0, 2024-04-24T23:32:31.087 INFO:teuthology.orchestra.run.smithi148.stdout: "root_rfiles": 3454, 2024-04-24T23:32:31.087 INFO:teuthology.orchestra.run.smithi148.stdout: "root_rbytes": 26715083, 2024-04-24T23:32:31.087 INFO:teuthology.orchestra.run.smithi148.stdout: "root_rsnaps": 0, 2024-04-24T23:32:31.087 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_backtrace_fetch": 0, 2024-04-24T23:32:31.087 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_set_tag": 0, 2024-04-24T23:32:31.088 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_backtrace_repaired": 0, 2024-04-24T23:32:31.088 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_inotable_repaired": 0, 2024-04-24T23:32:31.088 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_dir_inodes": 0, 2024-04-24T23:32:31.088 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_dir_base_inodes": 0, 2024-04-24T23:32:31.088 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_dirfrag_rstats": 0, 2024-04-24T23:32:31.088 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_file_inodes": 0, 2024-04-24T23:32:31.088 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_inode_file_caps": 0, 2024-04-24T23:32:31.088 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_revoke": 0, 2024-04-24T23:32:31.088 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_grant": 0, 2024-04-24T23:32:31.089 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_trunc": 0, 2024-04-24T23:32:31.089 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-24T23:32:31.089 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-24T23:32:31.089 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_caps": 0, 2024-04-24T23:32:31.089 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_caps_dirty": 0, 2024-04-24T23:32:31.089 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_cap_release": 0, 2024-04-24T23:32:31.089 INFO:teuthology.orchestra.run.smithi148.stdout: "process_request_cap_release": 0 2024-04-24T23:32:31.089 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:31.089 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_cache": { 2024-04-24T23:32:31.090 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_update": 0, 2024-04-24T23:32:31.090 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_update_receipt": 0, 2024-04-24T23:32:31.090 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_try_discover": 0, 2024-04-24T23:32:31.090 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_send_discover": 0, 2024-04-24T23:32:31.090 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_handle_discover": 0, 2024-04-24T23:32:31.090 INFO:teuthology.orchestra.run.smithi148.stdout: "num_strays": 0, 2024-04-24T23:32:31.090 INFO:teuthology.orchestra.run.smithi148.stdout: "num_strays_delayed": 0, 2024-04-24T23:32:31.090 INFO:teuthology.orchestra.run.smithi148.stdout: "num_strays_enqueuing": 0, 2024-04-24T23:32:31.091 INFO:teuthology.orchestra.run.smithi148.stdout: "strays_created": 0, 2024-04-24T23:32:31.091 INFO:teuthology.orchestra.run.smithi148.stdout: "strays_enqueued": 0, 2024-04-24T23:32:31.091 INFO:teuthology.orchestra.run.smithi148.stdout: "strays_reintegrated": 0, 2024-04-24T23:32:31.091 INFO:teuthology.orchestra.run.smithi148.stdout: "strays_migrated": 0, 2024-04-24T23:32:31.091 INFO:teuthology.orchestra.run.smithi148.stdout: "num_recovering_processing": 0, 2024-04-24T23:32:31.091 INFO:teuthology.orchestra.run.smithi148.stdout: "num_recovering_enqueued": 0, 2024-04-24T23:32:31.091 INFO:teuthology.orchestra.run.smithi148.stdout: "num_recovering_prioritized": 0, 2024-04-24T23:32:31.091 INFO:teuthology.orchestra.run.smithi148.stdout: "recovery_started": 0, 2024-04-24T23:32:31.092 INFO:teuthology.orchestra.run.smithi148.stdout: "recovery_completed": 0, 2024-04-24T23:32:31.092 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_quiesce_path": 0, 2024-04-24T23:32:31.092 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_quiesce_inode": 0, 2024-04-24T23:32:31.092 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_enqueue_scrub": 0, 2024-04-24T23:32:31.092 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_exportdir": 0, 2024-04-24T23:32:31.092 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_flush": 0, 2024-04-24T23:32:31.092 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_fragmentdir": 0, 2024-04-24T23:32:31.093 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_fragstats": 0, 2024-04-24T23:32:31.093 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_inodestats": 0 2024-04-24T23:32:31.093 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:31.093 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_log": { 2024-04-24T23:32:31.093 INFO:teuthology.orchestra.run.smithi148.stdout: "evlrg": 0, 2024-04-24T23:32:31.093 INFO:teuthology.orchestra.run.smithi148.stdout: "evadd": 0, 2024-04-24T23:32:31.093 INFO:teuthology.orchestra.run.smithi148.stdout: "evex": 0, 2024-04-24T23:32:31.094 INFO:teuthology.orchestra.run.smithi148.stdout: "evtrm": 0, 2024-04-24T23:32:31.094 INFO:teuthology.orchestra.run.smithi148.stdout: "ev": 17975, 2024-04-24T23:32:31.094 INFO:teuthology.orchestra.run.smithi148.stdout: "evexg": 0, 2024-04-24T23:32:31.094 INFO:teuthology.orchestra.run.smithi148.stdout: "evexd": 0, 2024-04-24T23:32:31.094 INFO:teuthology.orchestra.run.smithi148.stdout: "segadd": 0, 2024-04-24T23:32:31.094 INFO:teuthology.orchestra.run.smithi148.stdout: "segex": 0, 2024-04-24T23:32:31.094 INFO:teuthology.orchestra.run.smithi148.stdout: "segtrm": 0, 2024-04-24T23:32:31.094 INFO:teuthology.orchestra.run.smithi148.stdout: "seg": 22, 2024-04-24T23:32:31.095 INFO:teuthology.orchestra.run.smithi148.stdout: "segmjr": 3, 2024-04-24T23:32:31.095 INFO:teuthology.orchestra.run.smithi148.stdout: "segexg": 0, 2024-04-24T23:32:31.095 INFO:teuthology.orchestra.run.smithi148.stdout: "segexd": 0, 2024-04-24T23:32:31.095 INFO:teuthology.orchestra.run.smithi148.stdout: "expos": 4194304, 2024-04-24T23:32:31.095 INFO:teuthology.orchestra.run.smithi148.stdout: "wrpos": 42554566, 2024-04-24T23:32:31.095 INFO:teuthology.orchestra.run.smithi148.stdout: "rdpos": 42552243, 2024-04-24T23:32:31.095 INFO:teuthology.orchestra.run.smithi148.stdout: "jlat": { 2024-04-24T23:32:31.095 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-24T23:32:31.095 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-24T23:32:31.096 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-24T23:32:31.096 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:31.096 INFO:teuthology.orchestra.run.smithi148.stdout: "replayed": 17975 2024-04-24T23:32:31.096 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:31.096 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_mem": { 2024-04-24T23:32:31.096 INFO:teuthology.orchestra.run.smithi148.stdout: "ino": 73, 2024-04-24T23:32:31.096 INFO:teuthology.orchestra.run.smithi148.stdout: "ino+": 6019, 2024-04-24T23:32:31.096 INFO:teuthology.orchestra.run.smithi148.stdout: "ino-": 5946, 2024-04-24T23:32:31.096 INFO:teuthology.orchestra.run.smithi148.stdout: "dir": 63, 2024-04-24T23:32:31.097 INFO:teuthology.orchestra.run.smithi148.stdout: "dir+": 294, 2024-04-24T23:32:31.097 INFO:teuthology.orchestra.run.smithi148.stdout: "dir-": 231, 2024-04-24T23:32:31.097 INFO:teuthology.orchestra.run.smithi148.stdout: "dn": 2578, 2024-04-24T23:32:31.097 INFO:teuthology.orchestra.run.smithi148.stdout: "dn+": 7175, 2024-04-24T23:32:31.097 INFO:teuthology.orchestra.run.smithi148.stdout: "dn-": 4597, 2024-04-24T23:32:31.097 INFO:teuthology.orchestra.run.smithi148.stdout: "cap": 0, 2024-04-24T23:32:31.097 INFO:teuthology.orchestra.run.smithi148.stdout: "cap+": 0, 2024-04-24T23:32:31.097 INFO:teuthology.orchestra.run.smithi148.stdout: "cap-": 0, 2024-04-24T23:32:31.097 INFO:teuthology.orchestra.run.smithi148.stdout: "rss": 67132, 2024-04-24T23:32:31.098 INFO:teuthology.orchestra.run.smithi148.stdout: "heap": 215420 2024-04-24T23:32:31.098 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:31.098 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_server": { 2024-04-24T23:32:31.098 INFO:teuthology.orchestra.run.smithi148.stdout: "dispatch_client_request": 0, 2024-04-24T23:32:31.098 INFO:teuthology.orchestra.run.smithi148.stdout: "dispatch_server_request": 0, 2024-04-24T23:32:31.098 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_request": 0, 2024-04-24T23:32:31.098 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_session": 0, 2024-04-24T23:32:31.098 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_peer_request": 0, 2024-04-24T23:32:31.099 INFO:teuthology.orchestra.run.smithi148.stdout: "req_create_latency": { 2024-04-24T23:32:31.099 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-24T23:32:31.099 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-24T23:32:31.099 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-24T23:32:31.099 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:31.099 INFO:teuthology.orchestra.run.smithi148.stdout: "req_getattr_latency": { 2024-04-24T23:32:31.099 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-24T23:32:31.099 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-24T23:32:31.100 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-24T23:32:31.100 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:31.100 INFO:teuthology.orchestra.run.smithi148.stdout: "req_getfilelock_latency": { 2024-04-24T23:32:31.100 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-24T23:32:31.100 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-24T23:32:31.100 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-24T23:32:31.100 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:31.100 INFO:teuthology.orchestra.run.smithi148.stdout: "req_link_latency": { 2024-04-24T23:32:31.100 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-24T23:32:31.101 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-24T23:32:31.101 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-24T23:32:31.101 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:31.101 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookup_latency": { 2024-04-24T23:32:31.101 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-24T23:32:31.101 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-24T23:32:31.101 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-24T23:32:31.101 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:31.102 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookuphash_latency": { 2024-04-24T23:32:31.102 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-24T23:32:31.102 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-24T23:32:31.102 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-24T23:32:31.102 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:31.102 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookupino_latency": { 2024-04-24T23:32:31.102 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-24T23:32:31.102 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-24T23:32:31.103 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-24T23:32:31.103 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:31.103 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookupname_latency": { 2024-04-24T23:32:31.103 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-24T23:32:31.103 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-24T23:32:31.103 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-24T23:32:31.103 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:31.103 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookupparent_latency": { 2024-04-24T23:32:31.104 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-24T23:32:31.104 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-24T23:32:31.104 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-24T23:32:31.104 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:31.104 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookupsnap_latency": { 2024-04-24T23:32:31.104 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-24T23:32:31.104 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-24T23:32:31.104 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-24T23:32:31.104 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:31.105 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lssnap_latency": { 2024-04-24T23:32:31.105 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-24T23:32:31.105 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-24T23:32:31.105 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-24T23:32:31.105 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:31.106 INFO:teuthology.orchestra.run.smithi148.stdout: "req_mkdir_latency": { 2024-04-24T23:32:31.106 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-24T23:32:31.106 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-24T23:32:31.106 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-24T23:32:31.106 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:31.106 INFO:teuthology.orchestra.run.smithi148.stdout: "req_mknod_latency": { 2024-04-24T23:32:31.107 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-24T23:32:31.107 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-24T23:32:31.107 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-24T23:32:31.107 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:31.107 INFO:teuthology.orchestra.run.smithi148.stdout: "req_mksnap_latency": { 2024-04-24T23:32:31.107 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-24T23:32:31.107 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-24T23:32:31.107 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-24T23:32:31.107 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:31.108 INFO:teuthology.orchestra.run.smithi148.stdout: "req_open_latency": { 2024-04-24T23:32:31.108 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-24T23:32:31.108 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-24T23:32:31.108 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-24T23:32:31.108 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:31.108 INFO:teuthology.orchestra.run.smithi148.stdout: "req_readdir_latency": { 2024-04-24T23:32:31.108 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-24T23:32:31.108 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-24T23:32:31.109 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-24T23:32:31.109 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:31.109 INFO:teuthology.orchestra.run.smithi148.stdout: "req_rename_latency": { 2024-04-24T23:32:31.109 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-24T23:32:31.109 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-24T23:32:31.109 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-24T23:32:31.109 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:31.109 INFO:teuthology.orchestra.run.smithi148.stdout: "req_renamesnap_latency": { 2024-04-24T23:32:31.109 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-24T23:32:31.110 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-24T23:32:31.110 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-24T23:32:31.110 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:31.110 INFO:teuthology.orchestra.run.smithi148.stdout: "req_snapdiff_latency": { 2024-04-24T23:32:31.110 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-24T23:32:31.110 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-24T23:32:31.110 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-24T23:32:31.110 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:31.111 INFO:teuthology.orchestra.run.smithi148.stdout: "req_rmdir_latency": { 2024-04-24T23:32:31.111 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-24T23:32:31.111 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-24T23:32:31.111 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-24T23:32:31.111 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:31.111 INFO:teuthology.orchestra.run.smithi148.stdout: "req_rmsnap_latency": { 2024-04-24T23:32:31.111 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-24T23:32:31.111 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-24T23:32:31.112 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-24T23:32:31.112 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:31.112 INFO:teuthology.orchestra.run.smithi148.stdout: "req_rmxattr_latency": { 2024-04-24T23:32:31.112 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-24T23:32:31.112 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-24T23:32:31.112 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-24T23:32:31.112 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:31.112 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setattr_latency": { 2024-04-24T23:32:31.113 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-24T23:32:31.113 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-24T23:32:31.113 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-24T23:32:31.113 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:31.113 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setdirlayout_latency": { 2024-04-24T23:32:31.113 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-24T23:32:31.113 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-24T23:32:31.113 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-24T23:32:31.114 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:31.114 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setfilelock_latency": { 2024-04-24T23:32:31.114 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-24T23:32:31.114 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-24T23:32:31.114 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-24T23:32:31.114 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:31.114 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setlayout_latency": { 2024-04-24T23:32:31.114 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-24T23:32:31.115 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-24T23:32:31.115 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-24T23:32:31.115 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:31.115 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setxattr_latency": { 2024-04-24T23:32:31.115 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-24T23:32:31.115 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-24T23:32:31.115 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-24T23:32:31.115 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:31.116 INFO:teuthology.orchestra.run.smithi148.stdout: "req_symlink_latency": { 2024-04-24T23:32:31.116 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-24T23:32:31.116 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-24T23:32:31.116 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-24T23:32:31.116 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:31.116 INFO:teuthology.orchestra.run.smithi148.stdout: "req_unlink_latency": { 2024-04-24T23:32:31.116 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-24T23:32:31.116 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-24T23:32:31.116 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-24T23:32:31.117 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:31.117 INFO:teuthology.orchestra.run.smithi148.stdout: "cap_revoke_eviction": 0, 2024-04-24T23:32:31.117 INFO:teuthology.orchestra.run.smithi148.stdout: "cap_acquisition_throttle": 0, 2024-04-24T23:32:31.117 INFO:teuthology.orchestra.run.smithi148.stdout: "req_getvxattr_latency": { 2024-04-24T23:32:31.117 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-24T23:32:31.117 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-24T23:32:31.117 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-24T23:32:31.118 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-24T23:32:31.118 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:31.118 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_sessions": { 2024-04-24T23:32:31.118 INFO:teuthology.orchestra.run.smithi148.stdout: "session_count": 1, 2024-04-24T23:32:31.118 INFO:teuthology.orchestra.run.smithi148.stdout: "session_add": 3, 2024-04-24T23:32:31.118 INFO:teuthology.orchestra.run.smithi148.stdout: "session_remove": 2, 2024-04-24T23:32:31.118 INFO:teuthology.orchestra.run.smithi148.stdout: "sessions_open": 1, 2024-04-24T23:32:31.118 INFO:teuthology.orchestra.run.smithi148.stdout: "sessions_stale": 0, 2024-04-24T23:32:31.118 INFO:teuthology.orchestra.run.smithi148.stdout: "total_load": 0, 2024-04-24T23:32:31.119 INFO:teuthology.orchestra.run.smithi148.stdout: "average_load": 0, 2024-04-24T23:32:31.119 INFO:teuthology.orchestra.run.smithi148.stdout: "avg_session_uptime": 1276, 2024-04-24T23:32:31.119 INFO:teuthology.orchestra.run.smithi148.stdout: "mdthresh_evicted": 0 2024-04-24T23:32:31.119 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:31.119 INFO:teuthology.orchestra.run.smithi148.stdout: "mempool": { 2024-04-24T23:32:31.119 INFO:teuthology.orchestra.run.smithi148.stdout: "bloom_filter_bytes": 0, 2024-04-24T23:32:31.119 INFO:teuthology.orchestra.run.smithi148.stdout: "bloom_filter_items": 0, 2024-04-24T23:32:31.119 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_alloc_bytes": 0, 2024-04-24T23:32:31.120 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_alloc_items": 0, 2024-04-24T23:32:31.120 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_data_bytes": 0, 2024-04-24T23:32:31.120 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_data_items": 0, 2024-04-24T23:32:31.120 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-24T23:32:31.120 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_onode_items": 0, 2024-04-24T23:32:31.120 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-24T23:32:31.120 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_meta_items": 0, 2024-04-24T23:32:31.120 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_other_bytes": 0, 2024-04-24T23:32:31.120 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_other_items": 0, 2024-04-24T23:32:31.121 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-24T23:32:31.121 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_buffer_items": 0, 2024-04-24T23:32:31.121 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_extent_bytes": 0, 2024-04-24T23:32:31.121 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_extent_items": 0, 2024-04-24T23:32:31.121 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_blob_bytes": 0, 2024-04-24T23:32:31.121 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_blob_items": 0, 2024-04-24T23:32:31.121 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-24T23:32:31.121 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_shared_blob_items": 0, 2024-04-24T23:32:31.121 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-24T23:32:31.122 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_inline_bl_items": 0, 2024-04-24T23:32:31.122 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_fsck_bytes": 0, 2024-04-24T23:32:31.122 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_fsck_items": 0, 2024-04-24T23:32:31.122 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_txc_bytes": 0, 2024-04-24T23:32:31.122 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_txc_items": 0, 2024-04-24T23:32:31.122 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-24T23:32:31.122 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_deferred_items": 0, 2024-04-24T23:32:31.122 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_bytes": 0, 2024-04-24T23:32:31.123 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_items": 0, 2024-04-24T23:32:31.123 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_bytes": 0, 2024-04-24T23:32:31.123 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_items": 0, 2024-04-24T23:32:31.123 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_reader_bytes": 0, 2024-04-24T23:32:31.123 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_reader_items": 0, 2024-04-24T23:32:31.123 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_writer_bytes": 0, 2024-04-24T23:32:31.123 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_writer_items": 0, 2024-04-24T23:32:31.123 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_anon_bytes": 215420, 2024-04-24T23:32:31.124 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_anon_items": 113, 2024-04-24T23:32:31.124 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_meta_bytes": 0, 2024-04-24T23:32:31.124 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_meta_items": 0, 2024-04-24T23:32:31.124 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_bytes": 0, 2024-04-24T23:32:31.124 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_items": 0, 2024-04-24T23:32:31.124 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_mapbl_bytes": 0, 2024-04-24T23:32:31.124 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_mapbl_items": 0, 2024-04-24T23:32:31.124 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_pglog_bytes": 0, 2024-04-24T23:32:31.125 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_pglog_items": 0, 2024-04-24T23:32:31.125 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_bytes": 9080, 2024-04-24T23:32:31.125 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_items": 174, 2024-04-24T23:32:31.125 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_mapping_bytes": 0, 2024-04-24T23:32:31.125 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_mapping_items": 0, 2024-04-24T23:32:31.125 INFO:teuthology.orchestra.run.smithi148.stdout: "pgmap_bytes": 0, 2024-04-24T23:32:31.125 INFO:teuthology.orchestra.run.smithi148.stdout: "pgmap_items": 0, 2024-04-24T23:32:31.125 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_co_bytes": 2068731, 2024-04-24T23:32:31.126 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_co_items": 12220, 2024-04-24T23:32:31.126 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_1_bytes": 0, 2024-04-24T23:32:31.126 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_1_items": 0, 2024-04-24T23:32:31.126 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_2_bytes": 0, 2024-04-24T23:32:31.126 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_2_items": 0 2024-04-24T23:32:31.126 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:31.126 INFO:teuthology.orchestra.run.smithi148.stdout: "objecter": { 2024-04-24T23:32:31.126 INFO:teuthology.orchestra.run.smithi148.stdout: "op_active": 0, 2024-04-24T23:32:31.127 INFO:teuthology.orchestra.run.smithi148.stdout: "op_laggy": 0, 2024-04-24T23:32:31.127 INFO:teuthology.orchestra.run.smithi148.stdout: "op_send": 964, 2024-04-24T23:32:31.127 INFO:teuthology.orchestra.run.smithi148.stdout: "op_send_bytes": 26, 2024-04-24T23:32:31.127 INFO:teuthology.orchestra.run.smithi148.stdout: "op_resend": 4, 2024-04-24T23:32:31.127 INFO:teuthology.orchestra.run.smithi148.stdout: "op_reply": 964, 2024-04-24T23:32:31.127 INFO:teuthology.orchestra.run.smithi148.stdout: "op_latency": { 2024-04-24T23:32:31.127 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 964, 2024-04-24T23:32:31.127 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.622985157, 2024-04-24T23:32:31.127 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000646250 2024-04-24T23:32:31.128 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:31.128 INFO:teuthology.orchestra.run.smithi148.stdout: "op_inflight": 0, 2024-04-24T23:32:31.128 INFO:teuthology.orchestra.run.smithi148.stdout: "oplen_avg": { 2024-04-24T23:32:31.128 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 964, 2024-04-24T23:32:31.128 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 965 2024-04-24T23:32:31.128 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:31.128 INFO:teuthology.orchestra.run.smithi148.stdout: "op": 964, 2024-04-24T23:32:31.128 INFO:teuthology.orchestra.run.smithi148.stdout: "op_r": 964, 2024-04-24T23:32:31.129 INFO:teuthology.orchestra.run.smithi148.stdout: "op_w": 0, 2024-04-24T23:32:31.129 INFO:teuthology.orchestra.run.smithi148.stdout: "op_rmw": 0, 2024-04-24T23:32:31.129 INFO:teuthology.orchestra.run.smithi148.stdout: "op_pg": 0, 2024-04-24T23:32:31.129 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_stat": 602, 2024-04-24T23:32:31.129 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_create": 0, 2024-04-24T23:32:31.129 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_read": 359, 2024-04-24T23:32:31.129 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_write": 0, 2024-04-24T23:32:31.129 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_writefull": 0, 2024-04-24T23:32:31.130 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_writesame": 0, 2024-04-24T23:32:31.130 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_append": 0, 2024-04-24T23:32:31.130 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_zero": 0, 2024-04-24T23:32:31.130 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_truncate": 0, 2024-04-24T23:32:31.130 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_delete": 0, 2024-04-24T23:32:31.130 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_mapext": 0, 2024-04-24T23:32:31.130 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_sparse_read": 0, 2024-04-24T23:32:31.130 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_clonerange": 0, 2024-04-24T23:32:31.130 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_getxattr": 2, 2024-04-24T23:32:31.131 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_setxattr": 0, 2024-04-24T23:32:31.131 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_cmpxattr": 0, 2024-04-24T23:32:31.131 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_rmxattr": 0, 2024-04-24T23:32:31.131 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_resetxattrs": 0, 2024-04-24T23:32:31.131 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_call": 0, 2024-04-24T23:32:31.131 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_watch": 0, 2024-04-24T23:32:31.131 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_notify": 0, 2024-04-24T23:32:31.131 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_src_cmpxattr": 0, 2024-04-24T23:32:31.132 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_pgls": 0, 2024-04-24T23:32:31.132 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_pgls_filter": 0, 2024-04-24T23:32:31.132 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_other": 0, 2024-04-24T23:32:31.132 INFO:teuthology.orchestra.run.smithi148.stdout: "linger_active": 0, 2024-04-24T23:32:31.132 INFO:teuthology.orchestra.run.smithi148.stdout: "linger_send": 0, 2024-04-24T23:32:31.132 INFO:teuthology.orchestra.run.smithi148.stdout: "linger_resend": 0, 2024-04-24T23:32:31.132 INFO:teuthology.orchestra.run.smithi148.stdout: "linger_ping": 0, 2024-04-24T23:32:31.132 INFO:teuthology.orchestra.run.smithi148.stdout: "poolop_active": 0, 2024-04-24T23:32:31.133 INFO:teuthology.orchestra.run.smithi148.stdout: "poolop_send": 0, 2024-04-24T23:32:31.133 INFO:teuthology.orchestra.run.smithi148.stdout: "poolop_resend": 0, 2024-04-24T23:32:31.133 INFO:teuthology.orchestra.run.smithi148.stdout: "poolstat_active": 0, 2024-04-24T23:32:31.133 INFO:teuthology.orchestra.run.smithi148.stdout: "poolstat_send": 0, 2024-04-24T23:32:31.133 INFO:teuthology.orchestra.run.smithi148.stdout: "poolstat_resend": 0, 2024-04-24T23:32:31.133 INFO:teuthology.orchestra.run.smithi148.stdout: "statfs_active": 0, 2024-04-24T23:32:31.133 INFO:teuthology.orchestra.run.smithi148.stdout: "statfs_send": 0, 2024-04-24T23:32:31.133 INFO:teuthology.orchestra.run.smithi148.stdout: "statfs_resend": 0, 2024-04-24T23:32:31.133 INFO:teuthology.orchestra.run.smithi148.stdout: "command_active": 0, 2024-04-24T23:32:31.134 INFO:teuthology.orchestra.run.smithi148.stdout: "command_send": 0, 2024-04-24T23:32:31.134 INFO:teuthology.orchestra.run.smithi148.stdout: "command_resend": 0, 2024-04-24T23:32:31.134 INFO:teuthology.orchestra.run.smithi148.stdout: "map_epoch": 94, 2024-04-24T23:32:31.134 INFO:teuthology.orchestra.run.smithi148.stdout: "map_full": 0, 2024-04-24T23:32:31.134 INFO:teuthology.orchestra.run.smithi148.stdout: "map_inc": 4, 2024-04-24T23:32:31.134 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_sessions": 8, 2024-04-24T23:32:31.134 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_session_open": 8, 2024-04-24T23:32:31.134 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_session_close": 0, 2024-04-24T23:32:31.135 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_laggy": 0, 2024-04-24T23:32:31.135 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_wr": 0, 2024-04-24T23:32:31.135 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_rd": 2, 2024-04-24T23:32:31.135 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_del": 0 2024-04-24T23:32:31.135 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:31.135 INFO:teuthology.orchestra.run.smithi148.stdout: "oft": { 2024-04-24T23:32:31.135 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_total_objs": 0, 2024-04-24T23:32:31.135 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_total_kv_pairs": 0, 2024-04-24T23:32:31.136 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_total_updates": 0, 2024-04-24T23:32:31.136 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_total_removes": 0 2024-04-24T23:32:31.136 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:31.136 INFO:teuthology.orchestra.run.smithi148.stdout: "osd-slow-ops": { 2024-04-24T23:32:31.136 INFO:teuthology.orchestra.run.smithi148.stdout: "slow_ops_count": 0 2024-04-24T23:32:31.136 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:31.136 INFO:teuthology.orchestra.run.smithi148.stdout: "purge_queue": { 2024-04-24T23:32:31.136 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executing_ops": 0, 2024-04-24T23:32:31.137 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executing_ops_high_water": 0, 2024-04-24T23:32:31.137 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executing": 0, 2024-04-24T23:32:31.137 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executing_high_water": 0, 2024-04-24T23:32:31.137 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executed_ops": 0, 2024-04-24T23:32:31.137 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executed": 0, 2024-04-24T23:32:31.137 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_item_in_journal": 0 2024-04-24T23:32:31.137 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:31.137 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-24T23:32:31.137 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-04-24T23:32:31.138 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 104857600, 2024-04-24T23:32:31.138 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-04-24T23:32:31.138 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 1074, 2024-04-24T23:32:31.138 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 39005949, 2024-04-24T23:32:31.138 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-04-24T23:32:31.138 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 1074, 2024-04-24T23:32:31.138 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 0, 2024-04-24T23:32:31.139 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 0, 2024-04-24T23:32:31.139 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 1074, 2024-04-24T23:32:31.139 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 39005949, 2024-04-24T23:32:31.139 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-04-24T23:32:31.139 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-24T23:32:31.139 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-24T23:32:31.139 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-24T23:32:31.139 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-24T23:32:31.139 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:31.140 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-objecter_bytes": { 2024-04-24T23:32:31.140 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-04-24T23:32:31.140 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 104857600, 2024-04-24T23:32:31.140 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-04-24T23:32:31.140 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 0, 2024-04-24T23:32:31.140 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 0, 2024-04-24T23:32:31.140 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-04-24T23:32:31.140 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 0, 2024-04-24T23:32:31.140 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 964, 2024-04-24T23:32:31.141 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 38366497, 2024-04-24T23:32:31.141 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 964, 2024-04-24T23:32:31.141 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 38366497, 2024-04-24T23:32:31.141 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-04-24T23:32:31.141 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-24T23:32:31.141 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-24T23:32:31.141 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-24T23:32:31.141 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-24T23:32:31.142 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:31.142 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-objecter_ops": { 2024-04-24T23:32:31.142 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-04-24T23:32:31.142 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 1024, 2024-04-24T23:32:31.142 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-04-24T23:32:31.142 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 0, 2024-04-24T23:32:31.142 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 0, 2024-04-24T23:32:31.142 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-04-24T23:32:31.142 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 0, 2024-04-24T23:32:31.143 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 964, 2024-04-24T23:32:31.143 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 964, 2024-04-24T23:32:31.143 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 964, 2024-04-24T23:32:31.143 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 964, 2024-04-24T23:32:31.143 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-04-24T23:32:31.143 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-24T23:32:31.143 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-24T23:32:31.143 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-24T23:32:31.143 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-24T23:32:31.144 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:31.144 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-write_buf_throttle": { 2024-04-24T23:32:31.144 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-04-24T23:32:31.144 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 3758096384, 2024-04-24T23:32:31.144 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-04-24T23:32:31.144 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 0, 2024-04-24T23:32:31.144 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 0, 2024-04-24T23:32:31.144 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-04-24T23:32:31.144 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 0, 2024-04-24T23:32:31.145 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 0, 2024-04-24T23:32:31.145 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 0, 2024-04-24T23:32:31.145 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 0, 2024-04-24T23:32:31.145 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 0, 2024-04-24T23:32:31.145 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-04-24T23:32:31.145 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-24T23:32:31.145 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-24T23:32:31.145 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-24T23:32:31.145 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-24T23:32:31.146 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:31.146 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-write_buf_throttle-0x558d5f4046e0": { 2024-04-24T23:32:31.146 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-04-24T23:32:31.146 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 3758096384, 2024-04-24T23:32:31.146 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-04-24T23:32:31.146 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 0, 2024-04-24T23:32:31.146 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 0, 2024-04-24T23:32:31.147 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-04-24T23:32:31.147 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 0, 2024-04-24T23:32:31.147 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 0, 2024-04-24T23:32:31.147 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 0, 2024-04-24T23:32:31.147 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 0, 2024-04-24T23:32:31.147 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 0, 2024-04-24T23:32:31.147 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-04-24T23:32:31.147 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-24T23:32:31.147 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-24T23:32:31.148 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-24T23:32:31.148 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-24T23:32:31.148 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-24T23:32:31.148 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-24T23:32:32.034 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-04-24T23:32:32.035 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-04-24T23:32:32.035 INFO:tasks.check_counter:Daemon mds.f mds.dir_split=0 2024-04-24T23:32:32.035 DEBUG:tasks.check_counter:Getting stats from i 2024-04-24T23:32:32.035 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid ec7885aa-0290-11ef-bc93-c7b262605968 -- ceph daemon mds.i perf dump 2024-04-24T23:32:32.275 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T23:32:32.398 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:32:32 smithi148 ceph-mon[32528]: pgmap v456: 129 pgs: 129 active+clean; 75 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 122 KiB/s wr, 7 op/s 2024-04-24T23:32:32.446 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:32:32 smithi134 ceph-mon[32619]: pgmap v456: 129 pgs: 129 active+clean; 75 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 122 KiB/s wr, 7 op/s 2024-04-24T23:32:32.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:32:32 smithi080 ceph-mon[29152]: pgmap v456: 129 pgs: 129 active+clean; 75 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 122 KiB/s wr, 7 op/s 2024-04-24T23:32:34.074 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-24T23:32:34.074 INFO:teuthology.orchestra.run.smithi148.stdout: "AsyncMessenger::Worker-0": { 2024-04-24T23:32:34.074 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_messages": 21011, 2024-04-24T23:32:34.074 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages": 38864, 2024-04-24T23:32:34.074 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_bytes": 7318900, 2024-04-24T23:32:34.074 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_bytes": 36006193, 2024-04-24T23:32:34.075 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_created_connections": 47, 2024-04-24T23:32:34.075 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_active_connections": 8, 2024-04-24T23:32:34.075 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_total_time": 1.508541984, 2024-04-24T23:32:34.075 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_send_time": 0.720711184, 2024-04-24T23:32:34.075 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_recv_time": 1.043429693, 2024-04-24T23:32:34.075 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_fast_dispatch_time": 0.092616672, 2024-04-24T23:32:34.075 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:32:34.076 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 38859, 2024-04-24T23:32:34.076 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 1.491353653, 2024-04-24T23:32:34.076 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000038378 2024-04-24T23:32:34.076 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:34.076 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:32:34.076 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 50878, 2024-04-24T23:32:34.076 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.006615711, 2024-04-24T23:32:34.076 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000130 2024-04-24T23:32:34.076 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:34.077 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_encrypted_bytes": 6617568, 2024-04-24T23:32:34.077 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_encrypted_bytes": 18723904 2024-04-24T23:32:34.077 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:34.077 INFO:teuthology.orchestra.run.smithi148.stdout: "AsyncMessenger::Worker-1": { 2024-04-24T23:32:34.077 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_messages": 5329, 2024-04-24T23:32:34.077 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages": 4300, 2024-04-24T23:32:34.077 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_bytes": 1718793, 2024-04-24T23:32:34.077 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_bytes": 11136030, 2024-04-24T23:32:34.077 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_created_connections": 8, 2024-04-24T23:32:34.078 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_active_connections": 5, 2024-04-24T23:32:34.078 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_total_time": 0.354236458, 2024-04-24T23:32:34.078 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_send_time": 0.141954076, 2024-04-24T23:32:34.078 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_recv_time": 0.168846994, 2024-04-24T23:32:34.078 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_fast_dispatch_time": 0.071395931, 2024-04-24T23:32:34.078 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:32:34.078 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 4302, 2024-04-24T23:32:34.078 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.408427739, 2024-04-24T23:32:34.079 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000094939 2024-04-24T23:32:34.079 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:34.079 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:32:34.079 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 4624, 2024-04-24T23:32:34.079 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000574474, 2024-04-24T23:32:34.079 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000124 2024-04-24T23:32:34.079 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:34.079 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_encrypted_bytes": 3152, 2024-04-24T23:32:34.079 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_encrypted_bytes": 200912 2024-04-24T23:32:34.080 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:34.080 INFO:teuthology.orchestra.run.smithi148.stdout: "AsyncMessenger::Worker-2": { 2024-04-24T23:32:34.080 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_messages": 7675, 2024-04-24T23:32:34.080 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages": 5128, 2024-04-24T23:32:34.080 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_bytes": 2546509, 2024-04-24T23:32:34.080 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_bytes": 15511183, 2024-04-24T23:32:34.080 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_created_connections": 10, 2024-04-24T23:32:34.080 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_active_connections": 6, 2024-04-24T23:32:34.081 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_total_time": 0.449993225, 2024-04-24T23:32:34.081 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_send_time": 0.174166250, 2024-04-24T23:32:34.081 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_recv_time": 0.198906285, 2024-04-24T23:32:34.081 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_fast_dispatch_time": 0.101357292, 2024-04-24T23:32:34.081 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:32:34.081 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 5127, 2024-04-24T23:32:34.081 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.558925679, 2024-04-24T23:32:34.081 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000109016 2024-04-24T23:32:34.081 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:34.082 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:32:34.082 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 5820, 2024-04-24T23:32:34.082 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000676318, 2024-04-24T23:32:34.082 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000116 2024-04-24T23:32:34.082 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:34.082 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_encrypted_bytes": 499456, 2024-04-24T23:32:34.082 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_encrypted_bytes": 68208 2024-04-24T23:32:34.082 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:34.083 INFO:teuthology.orchestra.run.smithi148.stdout: "cct": { 2024-04-24T23:32:34.083 INFO:teuthology.orchestra.run.smithi148.stdout: "total_workers": 1, 2024-04-24T23:32:34.083 INFO:teuthology.orchestra.run.smithi148.stdout: "unhealthy_workers": 0 2024-04-24T23:32:34.083 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:34.083 INFO:teuthology.orchestra.run.smithi148.stdout: "finisher-MDSRank": { 2024-04-24T23:32:34.083 INFO:teuthology.orchestra.run.smithi148.stdout: "queue_len": 0, 2024-04-24T23:32:34.083 INFO:teuthology.orchestra.run.smithi148.stdout: "complete_latency": { 2024-04-24T23:32:34.083 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 2710, 2024-04-24T23:32:34.083 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 10.997019934, 2024-04-24T23:32:34.084 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.004057940 2024-04-24T23:32:34.084 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-24T23:32:34.084 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:34.084 INFO:teuthology.orchestra.run.smithi148.stdout: "finisher-PurgeQueue": { 2024-04-24T23:32:34.084 INFO:teuthology.orchestra.run.smithi148.stdout: "queue_len": 0, 2024-04-24T23:32:34.084 INFO:teuthology.orchestra.run.smithi148.stdout: "complete_latency": { 2024-04-24T23:32:34.084 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 3940, 2024-04-24T23:32:34.084 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 1.111056149, 2024-04-24T23:32:34.085 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000281993 2024-04-24T23:32:34.085 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-24T23:32:34.085 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:34.085 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": { 2024-04-24T23:32:34.085 INFO:teuthology.orchestra.run.smithi148.stdout: "request": 10150, 2024-04-24T23:32:34.085 INFO:teuthology.orchestra.run.smithi148.stdout: "reply": 10030, 2024-04-24T23:32:34.085 INFO:teuthology.orchestra.run.smithi148.stdout: "reply_latency": { 2024-04-24T23:32:34.085 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 10030, 2024-04-24T23:32:34.086 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 28.931773034, 2024-04-24T23:32:34.086 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.002884523 2024-04-24T23:32:34.086 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:34.086 INFO:teuthology.orchestra.run.smithi148.stdout: "slow_reply": 0, 2024-04-24T23:32:34.086 INFO:teuthology.orchestra.run.smithi148.stdout: "forward": 120, 2024-04-24T23:32:34.086 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_fetch_complete": 0, 2024-04-24T23:32:34.086 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_fetch_keys": 0, 2024-04-24T23:32:34.086 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_commit": 218, 2024-04-24T23:32:34.087 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_split": 25, 2024-04-24T23:32:34.087 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_merge": 0, 2024-04-24T23:32:34.087 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes": 1491, 2024-04-24T23:32:34.087 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_top": 607, 2024-04-24T23:32:34.087 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_bottom": 261, 2024-04-24T23:32:34.087 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_pin_tail": 623, 2024-04-24T23:32:34.087 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_pinned": 623, 2024-04-24T23:32:34.087 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_expired": 4065, 2024-04-24T23:32:34.088 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_with_caps": 54, 2024-04-24T23:32:34.088 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": 54, 2024-04-24T23:32:34.088 INFO:teuthology.orchestra.run.smithi148.stdout: "subtrees": 83, 2024-04-24T23:32:34.088 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse": 11311, 2024-04-24T23:32:34.088 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_hit": 10629, 2024-04-24T23:32:34.088 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_forward": 117, 2024-04-24T23:32:34.088 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_discover": 0, 2024-04-24T23:32:34.088 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_dir_fetch": 0, 2024-04-24T23:32:34.088 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_remote_ino": 0, 2024-04-24T23:32:34.089 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_lock": 4, 2024-04-24T23:32:34.089 INFO:teuthology.orchestra.run.smithi148.stdout: "load_cent": 1098, 2024-04-24T23:32:34.089 INFO:teuthology.orchestra.run.smithi148.stdout: "q": 0, 2024-04-24T23:32:34.089 INFO:teuthology.orchestra.run.smithi148.stdout: "exported": 122, 2024-04-24T23:32:34.089 INFO:teuthology.orchestra.run.smithi148.stdout: "exported_inodes": 4833, 2024-04-24T23:32:34.089 INFO:teuthology.orchestra.run.smithi148.stdout: "imported": 0, 2024-04-24T23:32:34.089 INFO:teuthology.orchestra.run.smithi148.stdout: "imported_inodes": 0, 2024-04-24T23:32:34.089 INFO:teuthology.orchestra.run.smithi148.stdout: "openino_dir_fetch": 0, 2024-04-24T23:32:34.090 INFO:teuthology.orchestra.run.smithi148.stdout: "openino_backtrace_fetch": 2028, 2024-04-24T23:32:34.090 INFO:teuthology.orchestra.run.smithi148.stdout: "openino_peer_discover": 0, 2024-04-24T23:32:34.090 INFO:teuthology.orchestra.run.smithi148.stdout: "root_rfiles": 3454, 2024-04-24T23:32:34.090 INFO:teuthology.orchestra.run.smithi148.stdout: "root_rbytes": 26715083, 2024-04-24T23:32:34.090 INFO:teuthology.orchestra.run.smithi148.stdout: "root_rsnaps": 0, 2024-04-24T23:32:34.090 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_backtrace_fetch": 2028, 2024-04-24T23:32:34.090 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_set_tag": 0, 2024-04-24T23:32:34.090 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_backtrace_repaired": 0, 2024-04-24T23:32:34.091 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_inotable_repaired": 0, 2024-04-24T23:32:34.091 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_dir_inodes": 78, 2024-04-24T23:32:34.091 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_dir_base_inodes": 4, 2024-04-24T23:32:34.091 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_dirfrag_rstats": 74, 2024-04-24T23:32:34.091 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_file_inodes": 1950, 2024-04-24T23:32:34.091 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_inode_file_caps": 7, 2024-04-24T23:32:34.091 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_revoke": 116, 2024-04-24T23:32:34.091 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_grant": 9431, 2024-04-24T23:32:34.092 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_trunc": 8, 2024-04-24T23:32:34.092 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-24T23:32:34.092 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-24T23:32:34.092 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_caps": 6923, 2024-04-24T23:32:34.092 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_caps_dirty": 4426, 2024-04-24T23:32:34.092 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_cap_release": 884, 2024-04-24T23:32:34.092 INFO:teuthology.orchestra.run.smithi148.stdout: "process_request_cap_release": 3583 2024-04-24T23:32:34.092 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:34.092 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_cache": { 2024-04-24T23:32:34.093 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_update": 0, 2024-04-24T23:32:34.093 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_update_receipt": 0, 2024-04-24T23:32:34.093 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_try_discover": 0, 2024-04-24T23:32:34.093 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_send_discover": 0, 2024-04-24T23:32:34.093 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_handle_discover": 123, 2024-04-24T23:32:34.093 INFO:teuthology.orchestra.run.smithi148.stdout: "num_strays": 0, 2024-04-24T23:32:34.093 INFO:teuthology.orchestra.run.smithi148.stdout: "num_strays_delayed": 0, 2024-04-24T23:32:34.093 INFO:teuthology.orchestra.run.smithi148.stdout: "num_strays_enqueuing": 0, 2024-04-24T23:32:34.093 INFO:teuthology.orchestra.run.smithi148.stdout: "strays_created": 2472, 2024-04-24T23:32:34.094 INFO:teuthology.orchestra.run.smithi148.stdout: "strays_enqueued": 2469, 2024-04-24T23:32:34.094 INFO:teuthology.orchestra.run.smithi148.stdout: "strays_reintegrated": 3, 2024-04-24T23:32:34.094 INFO:teuthology.orchestra.run.smithi148.stdout: "strays_migrated": 0, 2024-04-24T23:32:34.094 INFO:teuthology.orchestra.run.smithi148.stdout: "num_recovering_processing": 0, 2024-04-24T23:32:34.095 INFO:teuthology.orchestra.run.smithi148.stdout: "num_recovering_enqueued": 0, 2024-04-24T23:32:34.095 INFO:teuthology.orchestra.run.smithi148.stdout: "num_recovering_prioritized": 0, 2024-04-24T23:32:34.095 INFO:teuthology.orchestra.run.smithi148.stdout: "recovery_started": 0, 2024-04-24T23:32:34.095 INFO:teuthology.orchestra.run.smithi148.stdout: "recovery_completed": 0, 2024-04-24T23:32:34.095 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_quiesce_path": 0, 2024-04-24T23:32:34.095 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_quiesce_inode": 0, 2024-04-24T23:32:34.095 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_enqueue_scrub": 4, 2024-04-24T23:32:34.096 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_exportdir": 122, 2024-04-24T23:32:34.096 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_flush": 0, 2024-04-24T23:32:34.096 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_fragmentdir": 25, 2024-04-24T23:32:34.096 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_fragstats": 0, 2024-04-24T23:32:34.096 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_inodestats": 0 2024-04-24T23:32:34.096 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:34.096 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_log": { 2024-04-24T23:32:34.096 INFO:teuthology.orchestra.run.smithi148.stdout: "evlrg": 0, 2024-04-24T23:32:34.097 INFO:teuthology.orchestra.run.smithi148.stdout: "evadd": 17975, 2024-04-24T23:32:34.097 INFO:teuthology.orchestra.run.smithi148.stdout: "evex": 0, 2024-04-24T23:32:34.097 INFO:teuthology.orchestra.run.smithi148.stdout: "evtrm": 0, 2024-04-24T23:32:34.097 INFO:teuthology.orchestra.run.smithi148.stdout: "ev": 17975, 2024-04-24T23:32:34.097 INFO:teuthology.orchestra.run.smithi148.stdout: "evexg": 0, 2024-04-24T23:32:34.097 INFO:teuthology.orchestra.run.smithi148.stdout: "evexd": 0, 2024-04-24T23:32:34.097 INFO:teuthology.orchestra.run.smithi148.stdout: "segadd": 22, 2024-04-24T23:32:34.097 INFO:teuthology.orchestra.run.smithi148.stdout: "segex": 0, 2024-04-24T23:32:34.098 INFO:teuthology.orchestra.run.smithi148.stdout: "segtrm": 0, 2024-04-24T23:32:34.098 INFO:teuthology.orchestra.run.smithi148.stdout: "seg": 22, 2024-04-24T23:32:34.098 INFO:teuthology.orchestra.run.smithi148.stdout: "segmjr": 3, 2024-04-24T23:32:34.098 INFO:teuthology.orchestra.run.smithi148.stdout: "segexg": 0, 2024-04-24T23:32:34.098 INFO:teuthology.orchestra.run.smithi148.stdout: "segexd": 0, 2024-04-24T23:32:34.098 INFO:teuthology.orchestra.run.smithi148.stdout: "expos": 4194304, 2024-04-24T23:32:34.098 INFO:teuthology.orchestra.run.smithi148.stdout: "wrpos": 42554566, 2024-04-24T23:32:34.099 INFO:teuthology.orchestra.run.smithi148.stdout: "rdpos": 0, 2024-04-24T23:32:34.099 INFO:teuthology.orchestra.run.smithi148.stdout: "jlat": { 2024-04-24T23:32:34.099 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 580, 2024-04-24T23:32:34.099 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 14.745337885, 2024-04-24T23:32:34.099 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.025422996 2024-04-24T23:32:34.099 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:34.099 INFO:teuthology.orchestra.run.smithi148.stdout: "replayed": 0 2024-04-24T23:32:34.100 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:34.100 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_mem": { 2024-04-24T23:32:34.100 INFO:teuthology.orchestra.run.smithi148.stdout: "ino": 947, 2024-04-24T23:32:34.100 INFO:teuthology.orchestra.run.smithi148.stdout: "ino+": 6025, 2024-04-24T23:32:34.100 INFO:teuthology.orchestra.run.smithi148.stdout: "ino-": 5078, 2024-04-24T23:32:34.100 INFO:teuthology.orchestra.run.smithi148.stdout: "dir": 196, 2024-04-24T23:32:34.100 INFO:teuthology.orchestra.run.smithi148.stdout: "dir+": 296, 2024-04-24T23:32:34.101 INFO:teuthology.orchestra.run.smithi148.stdout: "dir-": 100, 2024-04-24T23:32:34.101 INFO:teuthology.orchestra.run.smithi148.stdout: "dn": 1491, 2024-04-24T23:32:34.101 INFO:teuthology.orchestra.run.smithi148.stdout: "dn+": 7597, 2024-04-24T23:32:34.101 INFO:teuthology.orchestra.run.smithi148.stdout: "dn-": 6106, 2024-04-24T23:32:34.101 INFO:teuthology.orchestra.run.smithi148.stdout: "cap": 54, 2024-04-24T23:32:34.101 INFO:teuthology.orchestra.run.smithi148.stdout: "cap+": 6013, 2024-04-24T23:32:34.101 INFO:teuthology.orchestra.run.smithi148.stdout: "cap-": 5959, 2024-04-24T23:32:34.101 INFO:teuthology.orchestra.run.smithi148.stdout: "rss": 100136, 2024-04-24T23:32:34.102 INFO:teuthology.orchestra.run.smithi148.stdout: "heap": 231804 2024-04-24T23:32:34.102 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:34.102 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_server": { 2024-04-24T23:32:34.102 INFO:teuthology.orchestra.run.smithi148.stdout: "dispatch_client_request": 10420, 2024-04-24T23:32:34.102 INFO:teuthology.orchestra.run.smithi148.stdout: "dispatch_server_request": 1517, 2024-04-24T23:32:34.102 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_request": 10150, 2024-04-24T23:32:34.102 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_session": 137, 2024-04-24T23:32:34.103 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_peer_request": 3034, 2024-04-24T23:32:34.103 INFO:teuthology.orchestra.run.smithi148.stdout: "req_create_latency": { 2024-04-24T23:32:34.103 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 5917, 2024-04-24T23:32:34.103 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 20.238870008, 2024-04-24T23:32:34.103 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.003420461 2024-04-24T23:32:34.103 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:34.103 INFO:teuthology.orchestra.run.smithi148.stdout: "req_getattr_latency": { 2024-04-24T23:32:34.104 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 83, 2024-04-24T23:32:34.104 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.540456894, 2024-04-24T23:32:34.104 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.006511528 2024-04-24T23:32:34.104 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:34.104 INFO:teuthology.orchestra.run.smithi148.stdout: "req_getfilelock_latency": { 2024-04-24T23:32:34.104 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-24T23:32:34.104 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-24T23:32:34.105 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-24T23:32:34.105 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:34.105 INFO:teuthology.orchestra.run.smithi148.stdout: "req_link_latency": { 2024-04-24T23:32:34.105 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 3, 2024-04-24T23:32:34.105 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.005810480, 2024-04-24T23:32:34.105 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.001936826 2024-04-24T23:32:34.106 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:34.106 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookup_latency": { 2024-04-24T23:32:34.106 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 896, 2024-04-24T23:32:34.106 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.296809552, 2024-04-24T23:32:34.106 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000331260 2024-04-24T23:32:34.106 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:34.106 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookuphash_latency": { 2024-04-24T23:32:34.107 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-24T23:32:34.107 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-24T23:32:34.107 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-24T23:32:34.107 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:34.107 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookupino_latency": { 2024-04-24T23:32:34.107 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 1, 2024-04-24T23:32:34.107 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000263960, 2024-04-24T23:32:34.108 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000263960 2024-04-24T23:32:34.108 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:34.108 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookupname_latency": { 2024-04-24T23:32:34.108 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-24T23:32:34.108 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-24T23:32:34.108 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-24T23:32:34.108 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:34.109 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookupparent_latency": { 2024-04-24T23:32:34.109 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-24T23:32:34.109 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-24T23:32:34.109 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-24T23:32:34.109 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:34.109 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookupsnap_latency": { 2024-04-24T23:32:34.109 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-24T23:32:34.109 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-24T23:32:34.110 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-24T23:32:34.110 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:34.111 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lssnap_latency": { 2024-04-24T23:32:34.111 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-24T23:32:34.111 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-24T23:32:34.111 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-24T23:32:34.111 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:34.111 INFO:teuthology.orchestra.run.smithi148.stdout: "req_mkdir_latency": { 2024-04-24T23:32:34.111 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 84, 2024-04-24T23:32:34.112 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.601416896, 2024-04-24T23:32:34.112 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.007159724 2024-04-24T23:32:34.112 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:34.112 INFO:teuthology.orchestra.run.smithi148.stdout: "req_mknod_latency": { 2024-04-24T23:32:34.112 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-24T23:32:34.112 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-24T23:32:34.112 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-24T23:32:34.113 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:34.113 INFO:teuthology.orchestra.run.smithi148.stdout: "req_mksnap_latency": { 2024-04-24T23:32:34.113 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-24T23:32:34.113 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-24T23:32:34.113 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-24T23:32:34.113 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:34.113 INFO:teuthology.orchestra.run.smithi148.stdout: "req_open_latency": { 2024-04-24T23:32:34.114 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 409, 2024-04-24T23:32:34.114 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.101979713, 2024-04-24T23:32:34.114 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000249339 2024-04-24T23:32:34.114 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:34.114 INFO:teuthology.orchestra.run.smithi148.stdout: "req_readdir_latency": { 2024-04-24T23:32:34.114 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 88, 2024-04-24T23:32:34.114 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.571946063, 2024-04-24T23:32:34.115 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.006499387 2024-04-24T23:32:34.115 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:34.115 INFO:teuthology.orchestra.run.smithi148.stdout: "req_rename_latency": { 2024-04-24T23:32:34.115 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 79, 2024-04-24T23:32:34.115 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.185890912, 2024-04-24T23:32:34.115 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.002353049 2024-04-24T23:32:34.116 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:34.116 INFO:teuthology.orchestra.run.smithi148.stdout: "req_renamesnap_latency": { 2024-04-24T23:32:34.116 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-24T23:32:34.116 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-24T23:32:34.116 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-24T23:32:34.116 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:34.116 INFO:teuthology.orchestra.run.smithi148.stdout: "req_snapdiff_latency": { 2024-04-24T23:32:34.116 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-24T23:32:34.117 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-24T23:32:34.117 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-24T23:32:34.117 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:34.117 INFO:teuthology.orchestra.run.smithi148.stdout: "req_rmdir_latency": { 2024-04-24T23:32:34.117 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 21, 2024-04-24T23:32:34.117 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.772235225, 2024-04-24T23:32:34.117 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.036773105 2024-04-24T23:32:34.118 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:34.118 INFO:teuthology.orchestra.run.smithi148.stdout: "req_rmsnap_latency": { 2024-04-24T23:32:34.118 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-24T23:32:34.118 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-24T23:32:34.118 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-24T23:32:34.118 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:34.118 INFO:teuthology.orchestra.run.smithi148.stdout: "req_rmxattr_latency": { 2024-04-24T23:32:34.118 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-24T23:32:34.119 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-24T23:32:34.119 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-24T23:32:34.119 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:34.119 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setattr_latency": { 2024-04-24T23:32:34.119 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 10, 2024-04-24T23:32:34.119 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.063903345, 2024-04-24T23:32:34.119 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.006390334 2024-04-24T23:32:34.119 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:34.120 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setdirlayout_latency": { 2024-04-24T23:32:34.120 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-24T23:32:34.120 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-24T23:32:34.120 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-24T23:32:34.120 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:34.120 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setfilelock_latency": { 2024-04-24T23:32:34.120 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 8, 2024-04-24T23:32:34.121 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 1.000655838, 2024-04-24T23:32:34.121 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.125081979 2024-04-24T23:32:34.121 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:34.121 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setlayout_latency": { 2024-04-24T23:32:34.121 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-24T23:32:34.121 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-24T23:32:34.121 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-24T23:32:34.122 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:34.122 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setxattr_latency": { 2024-04-24T23:32:34.122 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 8, 2024-04-24T23:32:34.122 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.022458177, 2024-04-24T23:32:34.122 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.002807272 2024-04-24T23:32:34.122 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:34.122 INFO:teuthology.orchestra.run.smithi148.stdout: "req_symlink_latency": { 2024-04-24T23:32:34.123 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 7, 2024-04-24T23:32:34.123 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.006041595, 2024-04-24T23:32:34.123 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000863085 2024-04-24T23:32:34.123 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:34.123 INFO:teuthology.orchestra.run.smithi148.stdout: "req_unlink_latency": { 2024-04-24T23:32:34.123 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 2416, 2024-04-24T23:32:34.123 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 4.523034376, 2024-04-24T23:32:34.124 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.001872116 2024-04-24T23:32:34.124 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:34.124 INFO:teuthology.orchestra.run.smithi148.stdout: "cap_revoke_eviction": 0, 2024-04-24T23:32:34.124 INFO:teuthology.orchestra.run.smithi148.stdout: "cap_acquisition_throttle": 0, 2024-04-24T23:32:34.124 INFO:teuthology.orchestra.run.smithi148.stdout: "req_getvxattr_latency": { 2024-04-24T23:32:34.124 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-24T23:32:34.124 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-24T23:32:34.125 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-24T23:32:34.125 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-24T23:32:34.125 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:34.125 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_sessions": { 2024-04-24T23:32:34.125 INFO:teuthology.orchestra.run.smithi148.stdout: "session_count": 1, 2024-04-24T23:32:34.125 INFO:teuthology.orchestra.run.smithi148.stdout: "session_add": 3, 2024-04-24T23:32:34.125 INFO:teuthology.orchestra.run.smithi148.stdout: "session_remove": 2, 2024-04-24T23:32:34.125 INFO:teuthology.orchestra.run.smithi148.stdout: "sessions_open": 1, 2024-04-24T23:32:34.126 INFO:teuthology.orchestra.run.smithi148.stdout: "sessions_stale": 0, 2024-04-24T23:32:34.126 INFO:teuthology.orchestra.run.smithi148.stdout: "total_load": 7221, 2024-04-24T23:32:34.126 INFO:teuthology.orchestra.run.smithi148.stdout: "average_load": 7221, 2024-04-24T23:32:34.126 INFO:teuthology.orchestra.run.smithi148.stdout: "avg_session_uptime": 168, 2024-04-24T23:32:34.126 INFO:teuthology.orchestra.run.smithi148.stdout: "mdthresh_evicted": 0 2024-04-24T23:32:34.126 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:34.127 INFO:teuthology.orchestra.run.smithi148.stdout: "mempool": { 2024-04-24T23:32:34.127 INFO:teuthology.orchestra.run.smithi148.stdout: "bloom_filter_bytes": 0, 2024-04-24T23:32:34.127 INFO:teuthology.orchestra.run.smithi148.stdout: "bloom_filter_items": 0, 2024-04-24T23:32:34.127 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_alloc_bytes": 0, 2024-04-24T23:32:34.127 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_alloc_items": 0, 2024-04-24T23:32:34.127 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_data_bytes": 0, 2024-04-24T23:32:34.127 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_data_items": 0, 2024-04-24T23:32:34.128 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-24T23:32:34.128 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_onode_items": 0, 2024-04-24T23:32:34.128 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-24T23:32:34.128 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_meta_items": 0, 2024-04-24T23:32:34.128 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_other_bytes": 0, 2024-04-24T23:32:34.128 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_other_items": 0, 2024-04-24T23:32:34.128 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-24T23:32:34.128 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_buffer_items": 0, 2024-04-24T23:32:34.129 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_extent_bytes": 0, 2024-04-24T23:32:34.129 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_extent_items": 0, 2024-04-24T23:32:34.129 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_blob_bytes": 0, 2024-04-24T23:32:34.129 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_blob_items": 0, 2024-04-24T23:32:34.129 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-24T23:32:34.129 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_shared_blob_items": 0, 2024-04-24T23:32:34.129 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-24T23:32:34.130 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_inline_bl_items": 0, 2024-04-24T23:32:34.130 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_fsck_bytes": 0, 2024-04-24T23:32:34.130 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_fsck_items": 0, 2024-04-24T23:32:34.130 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_txc_bytes": 0, 2024-04-24T23:32:34.130 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_txc_items": 0, 2024-04-24T23:32:34.130 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-24T23:32:34.130 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_deferred_items": 0, 2024-04-24T23:32:34.131 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_bytes": 0, 2024-04-24T23:32:34.131 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_items": 0, 2024-04-24T23:32:34.131 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_bytes": 0, 2024-04-24T23:32:34.131 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_items": 0, 2024-04-24T23:32:34.131 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_reader_bytes": 0, 2024-04-24T23:32:34.131 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_reader_items": 0, 2024-04-24T23:32:34.131 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_writer_bytes": 0, 2024-04-24T23:32:34.132 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_writer_items": 0, 2024-04-24T23:32:34.132 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_anon_bytes": 332899, 2024-04-24T23:32:34.132 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_anon_items": 1055, 2024-04-24T23:32:34.132 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_meta_bytes": 1408, 2024-04-24T23:32:34.132 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_meta_items": 16, 2024-04-24T23:32:34.132 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_bytes": 0, 2024-04-24T23:32:34.132 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_items": 0, 2024-04-24T23:32:34.132 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_mapbl_bytes": 0, 2024-04-24T23:32:34.133 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_mapbl_items": 0, 2024-04-24T23:32:34.133 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_pglog_bytes": 0, 2024-04-24T23:32:34.133 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_pglog_items": 0, 2024-04-24T23:32:34.133 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_bytes": 9080, 2024-04-24T23:32:34.133 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_items": 174, 2024-04-24T23:32:34.133 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_mapping_bytes": 0, 2024-04-24T23:32:34.134 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_mapping_items": 0, 2024-04-24T23:32:34.134 INFO:teuthology.orchestra.run.smithi148.stdout: "pgmap_bytes": 0, 2024-04-24T23:32:34.134 INFO:teuthology.orchestra.run.smithi148.stdout: "pgmap_items": 0, 2024-04-24T23:32:34.134 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_co_bytes": 3680240, 2024-04-24T23:32:34.134 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_co_items": 49094, 2024-04-24T23:32:34.134 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_1_bytes": 0, 2024-04-24T23:32:34.134 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_1_items": 0, 2024-04-24T23:32:34.135 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_2_bytes": 0, 2024-04-24T23:32:34.135 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_2_items": 0 2024-04-24T23:32:34.135 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:34.135 INFO:teuthology.orchestra.run.smithi148.stdout: "objecter": { 2024-04-24T23:32:34.135 INFO:teuthology.orchestra.run.smithi148.stdout: "op_active": 0, 2024-04-24T23:32:34.135 INFO:teuthology.orchestra.run.smithi148.stdout: "op_laggy": 0, 2024-04-24T23:32:34.135 INFO:teuthology.orchestra.run.smithi148.stdout: "op_send": 8457, 2024-04-24T23:32:34.135 INFO:teuthology.orchestra.run.smithi148.stdout: "op_send_bytes": 42001832, 2024-04-24T23:32:34.136 INFO:teuthology.orchestra.run.smithi148.stdout: "op_resend": 37, 2024-04-24T23:32:34.136 INFO:teuthology.orchestra.run.smithi148.stdout: "op_reply": 8457, 2024-04-24T23:32:34.136 INFO:teuthology.orchestra.run.smithi148.stdout: "op_latency": { 2024-04-24T23:32:34.136 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 8457, 2024-04-24T23:32:34.136 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 43.047974260, 2024-04-24T23:32:34.136 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.005090218 2024-04-24T23:32:34.136 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:34.136 INFO:teuthology.orchestra.run.smithi148.stdout: "op_inflight": 0, 2024-04-24T23:32:34.137 INFO:teuthology.orchestra.run.smithi148.stdout: "oplen_avg": { 2024-04-24T23:32:34.137 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 8457, 2024-04-24T23:32:34.137 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 8746 2024-04-24T23:32:34.137 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:34.137 INFO:teuthology.orchestra.run.smithi148.stdout: "op": 8457, 2024-04-24T23:32:34.137 INFO:teuthology.orchestra.run.smithi148.stdout: "op_r": 2063, 2024-04-24T23:32:34.137 INFO:teuthology.orchestra.run.smithi148.stdout: "op_w": 6394, 2024-04-24T23:32:34.138 INFO:teuthology.orchestra.run.smithi148.stdout: "op_rmw": 0, 2024-04-24T23:32:34.138 INFO:teuthology.orchestra.run.smithi148.stdout: "op_pg": 0, 2024-04-24T23:32:34.138 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_stat": 0, 2024-04-24T23:32:34.138 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_create": 11, 2024-04-24T23:32:34.138 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_read": 31, 2024-04-24T23:32:34.138 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_write": 607, 2024-04-24T23:32:34.138 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_writefull": 30, 2024-04-24T23:32:34.138 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_writesame": 0, 2024-04-24T23:32:34.139 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_append": 0, 2024-04-24T23:32:34.139 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_zero": 0, 2024-04-24T23:32:34.139 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_truncate": 9, 2024-04-24T23:32:34.139 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_delete": 5461, 2024-04-24T23:32:34.139 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_mapext": 0, 2024-04-24T23:32:34.139 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_sparse_read": 0, 2024-04-24T23:32:34.139 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_clonerange": 0, 2024-04-24T23:32:34.139 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_getxattr": 2032, 2024-04-24T23:32:34.140 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_setxattr": 22, 2024-04-24T23:32:34.140 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_cmpxattr": 0, 2024-04-24T23:32:34.140 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_rmxattr": 0, 2024-04-24T23:32:34.140 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_resetxattrs": 0, 2024-04-24T23:32:34.140 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_call": 0, 2024-04-24T23:32:34.140 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_watch": 0, 2024-04-24T23:32:34.140 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_notify": 0, 2024-04-24T23:32:34.141 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_src_cmpxattr": 0, 2024-04-24T23:32:34.141 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_pgls": 0, 2024-04-24T23:32:34.141 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_pgls_filter": 0, 2024-04-24T23:32:34.141 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_other": 8, 2024-04-24T23:32:34.141 INFO:teuthology.orchestra.run.smithi148.stdout: "linger_active": 0, 2024-04-24T23:32:34.141 INFO:teuthology.orchestra.run.smithi148.stdout: "linger_send": 0, 2024-04-24T23:32:34.141 INFO:teuthology.orchestra.run.smithi148.stdout: "linger_resend": 0, 2024-04-24T23:32:34.142 INFO:teuthology.orchestra.run.smithi148.stdout: "linger_ping": 0, 2024-04-24T23:32:34.142 INFO:teuthology.orchestra.run.smithi148.stdout: "poolop_active": 0, 2024-04-24T23:32:34.142 INFO:teuthology.orchestra.run.smithi148.stdout: "poolop_send": 0, 2024-04-24T23:32:34.142 INFO:teuthology.orchestra.run.smithi148.stdout: "poolop_resend": 0, 2024-04-24T23:32:34.142 INFO:teuthology.orchestra.run.smithi148.stdout: "poolstat_active": 0, 2024-04-24T23:32:34.142 INFO:teuthology.orchestra.run.smithi148.stdout: "poolstat_send": 0, 2024-04-24T23:32:34.142 INFO:teuthology.orchestra.run.smithi148.stdout: "poolstat_resend": 0, 2024-04-24T23:32:34.143 INFO:teuthology.orchestra.run.smithi148.stdout: "statfs_active": 0, 2024-04-24T23:32:34.143 INFO:teuthology.orchestra.run.smithi148.stdout: "statfs_send": 0, 2024-04-24T23:32:34.143 INFO:teuthology.orchestra.run.smithi148.stdout: "statfs_resend": 0, 2024-04-24T23:32:34.143 INFO:teuthology.orchestra.run.smithi148.stdout: "command_active": 0, 2024-04-24T23:32:34.143 INFO:teuthology.orchestra.run.smithi148.stdout: "command_send": 0, 2024-04-24T23:32:34.143 INFO:teuthology.orchestra.run.smithi148.stdout: "command_resend": 0, 2024-04-24T23:32:34.143 INFO:teuthology.orchestra.run.smithi148.stdout: "map_epoch": 94, 2024-04-24T23:32:34.144 INFO:teuthology.orchestra.run.smithi148.stdout: "map_full": 0, 2024-04-24T23:32:34.144 INFO:teuthology.orchestra.run.smithi148.stdout: "map_inc": 10, 2024-04-24T23:32:34.144 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_sessions": 12, 2024-04-24T23:32:34.144 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_session_open": 12, 2024-04-24T23:32:34.144 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_session_close": 0, 2024-04-24T23:32:34.144 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_laggy": 0, 2024-04-24T23:32:34.144 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_wr": 505, 2024-04-24T23:32:34.144 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_rd": 0, 2024-04-24T23:32:34.145 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_del": 30 2024-04-24T23:32:34.145 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:34.145 INFO:teuthology.orchestra.run.smithi148.stdout: "oft": { 2024-04-24T23:32:34.145 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_total_objs": 1, 2024-04-24T23:32:34.145 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_total_kv_pairs": 0, 2024-04-24T23:32:34.145 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_total_updates": 0, 2024-04-24T23:32:34.145 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_total_removes": 0 2024-04-24T23:32:34.146 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:34.146 INFO:teuthology.orchestra.run.smithi148.stdout: "osd-slow-ops": { 2024-04-24T23:32:34.146 INFO:teuthology.orchestra.run.smithi148.stdout: "slow_ops_count": 1 2024-04-24T23:32:34.146 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:34.146 INFO:teuthology.orchestra.run.smithi148.stdout: "purge_queue": { 2024-04-24T23:32:34.146 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executing_ops": 0, 2024-04-24T23:32:34.146 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executing_ops_high_water": 17, 2024-04-24T23:32:34.147 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executing": 0, 2024-04-24T23:32:34.147 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executing_high_water": 9, 2024-04-24T23:32:34.147 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executed_ops": 4924, 2024-04-24T23:32:34.147 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executed": 2469, 2024-04-24T23:32:34.147 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_item_in_journal": 0 2024-04-24T23:32:34.147 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:34.147 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-24T23:32:34.148 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-04-24T23:32:34.148 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 104857600, 2024-04-24T23:32:34.148 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-04-24T23:32:34.148 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 34011, 2024-04-24T23:32:34.148 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 7643569, 2024-04-24T23:32:34.148 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-04-24T23:32:34.148 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 34011, 2024-04-24T23:32:34.148 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 0, 2024-04-24T23:32:34.149 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 0, 2024-04-24T23:32:34.149 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 34011, 2024-04-24T23:32:34.149 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 7643569, 2024-04-24T23:32:34.149 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-04-24T23:32:34.149 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-24T23:32:34.149 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-24T23:32:34.149 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-24T23:32:34.150 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-24T23:32:34.150 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:34.150 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-objecter_bytes": { 2024-04-24T23:32:34.150 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-04-24T23:32:34.150 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 104857600, 2024-04-24T23:32:34.150 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-04-24T23:32:34.150 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 0, 2024-04-24T23:32:34.150 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 0, 2024-04-24T23:32:34.151 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-04-24T23:32:34.151 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 0, 2024-04-24T23:32:34.151 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 8457, 2024-04-24T23:32:34.151 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 42270841, 2024-04-24T23:32:34.151 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 8457, 2024-04-24T23:32:34.151 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 42270841, 2024-04-24T23:32:34.151 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-04-24T23:32:34.152 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-24T23:32:34.152 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-24T23:32:34.152 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-24T23:32:34.152 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-24T23:32:34.152 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:34.152 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-objecter_ops": { 2024-04-24T23:32:34.152 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-04-24T23:32:34.152 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 1024, 2024-04-24T23:32:34.153 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-04-24T23:32:34.153 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 0, 2024-04-24T23:32:34.153 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 0, 2024-04-24T23:32:34.153 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-04-24T23:32:34.153 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 0, 2024-04-24T23:32:34.153 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 8457, 2024-04-24T23:32:34.153 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 8457, 2024-04-24T23:32:34.154 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 8457, 2024-04-24T23:32:34.154 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 8457, 2024-04-24T23:32:34.154 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-04-24T23:32:34.154 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-24T23:32:34.154 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-24T23:32:34.154 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-24T23:32:34.154 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-24T23:32:34.155 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:34.155 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-write_buf_throttle": { 2024-04-24T23:32:34.155 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-04-24T23:32:34.155 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 3758096384, 2024-04-24T23:32:34.155 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-04-24T23:32:34.155 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 2469, 2024-04-24T23:32:34.155 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 269009, 2024-04-24T23:32:34.155 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-04-24T23:32:34.156 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 2469, 2024-04-24T23:32:34.156 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 0, 2024-04-24T23:32:34.156 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 0, 2024-04-24T23:32:34.156 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 27, 2024-04-24T23:32:34.156 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 269009, 2024-04-24T23:32:34.156 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-04-24T23:32:34.156 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-24T23:32:34.157 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-24T23:32:34.157 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-24T23:32:34.157 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-24T23:32:34.157 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:34.157 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-write_buf_throttle-0x55bae553c500": { 2024-04-24T23:32:34.157 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-04-24T23:32:34.157 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 3758096384, 2024-04-24T23:32:34.158 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-04-24T23:32:34.158 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 17975, 2024-04-24T23:32:34.158 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 38360262, 2024-04-24T23:32:34.158 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-04-24T23:32:34.158 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 17975, 2024-04-24T23:32:34.158 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 0, 2024-04-24T23:32:34.158 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 0, 2024-04-24T23:32:34.159 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 580, 2024-04-24T23:32:34.159 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 38360262, 2024-04-24T23:32:34.159 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-04-24T23:32:34.159 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-24T23:32:34.159 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-24T23:32:34.159 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-24T23:32:34.159 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-24T23:32:34.159 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-24T23:32:34.160 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-24T23:32:34.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:32:34 smithi134 ceph-mon[32619]: pgmap v457: 129 pgs: 129 active+clean; 75 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 122 KiB/s wr, 6 op/s 2024-04-24T23:32:34.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:32:34 smithi148 ceph-mon[32528]: pgmap v457: 129 pgs: 129 active+clean; 75 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 122 KiB/s wr, 6 op/s 2024-04-24T23:32:34.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:32:34 smithi080 ceph-mon[29152]: pgmap v457: 129 pgs: 129 active+clean; 75 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 122 KiB/s wr, 6 op/s 2024-04-24T23:32:34.908 INFO:tasks.check_counter:Daemon mds.i mds.exported=122 2024-04-24T23:32:34.908 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-04-24T23:32:34.908 INFO:tasks.check_counter:Daemon mds.i mds.dir_split=25 2024-04-24T23:32:34.909 DEBUG:tasks.check_counter:Getting stats from l 2024-04-24T23:32:34.909 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid ec7885aa-0290-11ef-bc93-c7b262605968 -- ceph daemon mds.l perf dump 2024-04-24T23:32:35.146 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T23:32:36.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:32:36 smithi134 ceph-mon[32619]: pgmap v458: 129 pgs: 129 active+clean; 75 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 123 KiB/s wr, 9 op/s 2024-04-24T23:32:36.490 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:32:36 smithi148 ceph-mon[32528]: pgmap v458: 129 pgs: 129 active+clean; 75 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 123 KiB/s wr, 9 op/s 2024-04-24T23:32:36.538 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:32:36 smithi080 ceph-mon[29152]: pgmap v458: 129 pgs: 129 active+clean; 75 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 123 KiB/s wr, 9 op/s 2024-04-24T23:32:36.948 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-24T23:32:36.949 INFO:teuthology.orchestra.run.smithi148.stdout: "AsyncMessenger::Worker-0": { 2024-04-24T23:32:36.949 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_messages": 101, 2024-04-24T23:32:36.949 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages": 98, 2024-04-24T23:32:36.949 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_bytes": 444048, 2024-04-24T23:32:36.949 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_bytes": 59232, 2024-04-24T23:32:36.949 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_created_connections": 4, 2024-04-24T23:32:36.949 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_active_connections": 0, 2024-04-24T23:32:36.949 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_total_time": 0.018810608, 2024-04-24T23:32:36.950 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_send_time": 0.006262745, 2024-04-24T23:32:36.950 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_recv_time": 0.011130416, 2024-04-24T23:32:36.950 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_fast_dispatch_time": 0.002442666, 2024-04-24T23:32:36.950 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:32:36.950 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 97, 2024-04-24T23:32:36.950 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.006246032, 2024-04-24T23:32:36.950 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000064392 2024-04-24T23:32:36.951 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:36.951 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:32:36.951 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-24T23:32:36.951 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-24T23:32:36.951 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-24T23:32:36.951 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:36.951 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_encrypted_bytes": 444048, 2024-04-24T23:32:36.951 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_encrypted_bytes": 59232 2024-04-24T23:32:36.952 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:36.952 INFO:teuthology.orchestra.run.smithi148.stdout: "AsyncMessenger::Worker-1": { 2024-04-24T23:32:36.952 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_messages": 5, 2024-04-24T23:32:36.952 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages": 77, 2024-04-24T23:32:36.952 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_bytes": 3152, 2024-04-24T23:32:36.952 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_bytes": 91200, 2024-04-24T23:32:36.952 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_created_connections": 3, 2024-04-24T23:32:36.952 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_active_connections": 2, 2024-04-24T23:32:36.953 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_total_time": 0.005369470, 2024-04-24T23:32:36.953 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_send_time": 0.003497285, 2024-04-24T23:32:36.953 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_recv_time": 0.000970014, 2024-04-24T23:32:36.953 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-24T23:32:36.953 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:32:36.953 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 76, 2024-04-24T23:32:36.953 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.005340877, 2024-04-24T23:32:36.954 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000070274 2024-04-24T23:32:36.954 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:36.954 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:32:36.954 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-24T23:32:36.954 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-24T23:32:36.954 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-24T23:32:36.954 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:36.954 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_encrypted_bytes": 3152, 2024-04-24T23:32:36.955 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_encrypted_bytes": 91200 2024-04-24T23:32:36.955 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:36.955 INFO:teuthology.orchestra.run.smithi148.stdout: "AsyncMessenger::Worker-2": { 2024-04-24T23:32:36.955 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_messages": 0, 2024-04-24T23:32:36.955 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages": 2, 2024-04-24T23:32:36.955 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_bytes": 0, 2024-04-24T23:32:36.955 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_bytes": 0, 2024-04-24T23:32:36.955 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_created_connections": 2, 2024-04-24T23:32:36.956 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_active_connections": 1, 2024-04-24T23:32:36.956 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_total_time": 0.000382862, 2024-04-24T23:32:36.956 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_send_time": 0.000000000, 2024-04-24T23:32:36.956 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_recv_time": 0.000310615, 2024-04-24T23:32:36.956 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-24T23:32:36.956 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:32:36.956 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-24T23:32:36.956 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-24T23:32:36.957 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-24T23:32:36.957 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:36.957 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:32:36.957 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-24T23:32:36.957 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-24T23:32:36.957 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-24T23:32:36.957 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:36.957 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-24T23:32:36.958 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_encrypted_bytes": 0 2024-04-24T23:32:36.958 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:36.958 INFO:teuthology.orchestra.run.smithi148.stdout: "cct": { 2024-04-24T23:32:36.958 INFO:teuthology.orchestra.run.smithi148.stdout: "total_workers": 0, 2024-04-24T23:32:36.958 INFO:teuthology.orchestra.run.smithi148.stdout: "unhealthy_workers": 0 2024-04-24T23:32:36.958 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:36.958 INFO:teuthology.orchestra.run.smithi148.stdout: "mempool": { 2024-04-24T23:32:36.958 INFO:teuthology.orchestra.run.smithi148.stdout: "bloom_filter_bytes": 0, 2024-04-24T23:32:36.959 INFO:teuthology.orchestra.run.smithi148.stdout: "bloom_filter_items": 0, 2024-04-24T23:32:36.959 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_alloc_bytes": 0, 2024-04-24T23:32:36.959 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_alloc_items": 0, 2024-04-24T23:32:36.959 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_data_bytes": 0, 2024-04-24T23:32:36.959 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_data_items": 0, 2024-04-24T23:32:36.959 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-24T23:32:36.959 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_onode_items": 0, 2024-04-24T23:32:36.959 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-24T23:32:36.960 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_meta_items": 0, 2024-04-24T23:32:36.960 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_other_bytes": 0, 2024-04-24T23:32:36.960 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_other_items": 0, 2024-04-24T23:32:36.960 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-24T23:32:36.960 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_buffer_items": 0, 2024-04-24T23:32:36.960 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_extent_bytes": 0, 2024-04-24T23:32:36.960 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_extent_items": 0, 2024-04-24T23:32:36.960 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_blob_bytes": 0, 2024-04-24T23:32:36.961 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_blob_items": 0, 2024-04-24T23:32:36.961 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-24T23:32:36.961 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_shared_blob_items": 0, 2024-04-24T23:32:36.961 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-24T23:32:36.961 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_inline_bl_items": 0, 2024-04-24T23:32:36.961 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_fsck_bytes": 0, 2024-04-24T23:32:36.961 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_fsck_items": 0, 2024-04-24T23:32:36.961 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_txc_bytes": 0, 2024-04-24T23:32:36.962 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_txc_items": 0, 2024-04-24T23:32:36.962 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-24T23:32:36.962 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_deferred_items": 0, 2024-04-24T23:32:36.962 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_bytes": 0, 2024-04-24T23:32:36.962 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_items": 0, 2024-04-24T23:32:36.962 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_bytes": 0, 2024-04-24T23:32:36.962 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_items": 0, 2024-04-24T23:32:36.962 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_reader_bytes": 0, 2024-04-24T23:32:36.963 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_reader_items": 0, 2024-04-24T23:32:36.963 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_writer_bytes": 0, 2024-04-24T23:32:36.963 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_writer_items": 0, 2024-04-24T23:32:36.963 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_anon_bytes": 148880, 2024-04-24T23:32:36.963 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_anon_items": 29, 2024-04-24T23:32:36.963 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_meta_bytes": 0, 2024-04-24T23:32:36.963 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_meta_items": 0, 2024-04-24T23:32:36.963 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_bytes": 0, 2024-04-24T23:32:36.964 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_items": 0, 2024-04-24T23:32:36.964 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_mapbl_bytes": 0, 2024-04-24T23:32:36.964 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_mapbl_items": 0, 2024-04-24T23:32:36.964 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_pglog_bytes": 0, 2024-04-24T23:32:36.964 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_pglog_items": 0, 2024-04-24T23:32:36.964 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_bytes": 0, 2024-04-24T23:32:36.964 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_items": 0, 2024-04-24T23:32:36.964 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_mapping_bytes": 0, 2024-04-24T23:32:36.964 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_mapping_items": 0, 2024-04-24T23:32:36.965 INFO:teuthology.orchestra.run.smithi148.stdout: "pgmap_bytes": 0, 2024-04-24T23:32:36.965 INFO:teuthology.orchestra.run.smithi148.stdout: "pgmap_items": 0, 2024-04-24T23:32:36.965 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_co_bytes": 928, 2024-04-24T23:32:36.965 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_co_items": 2, 2024-04-24T23:32:36.965 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_1_bytes": 0, 2024-04-24T23:32:36.965 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_1_items": 0, 2024-04-24T23:32:36.965 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_2_bytes": 0, 2024-04-24T23:32:36.965 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_2_items": 0 2024-04-24T23:32:36.966 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-24T23:32:36.966 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-24T23:32:36.966 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-04-24T23:32:36.966 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 104857600, 2024-04-24T23:32:36.966 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-04-24T23:32:36.966 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 102, 2024-04-24T23:32:36.966 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 429806, 2024-04-24T23:32:36.966 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-04-24T23:32:36.967 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 102, 2024-04-24T23:32:36.967 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 0, 2024-04-24T23:32:36.967 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 0, 2024-04-24T23:32:36.967 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 102, 2024-04-24T23:32:36.967 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 429806, 2024-04-24T23:32:36.967 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-04-24T23:32:36.968 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-24T23:32:36.968 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-24T23:32:36.968 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-24T23:32:36.968 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-24T23:32:36.968 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-24T23:32:36.968 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-24T23:32:37.774 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-04-24T23:32:37.775 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-04-24T23:32:37.775 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.l 2024-04-24T23:32:37.775 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-04-24T23:32:37.788 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-24T23:32:37.789 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-24T23:32:37.789 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:32:37.813 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-24T23:32:37.813 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:32:37.869 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:32:38.031 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.028+0000 7fd611dce640 1 -- 172.21.15.80:0/1592611222 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd60c102260 msgr2=0x7fd60c104650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:38.031 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.028+0000 7fd611dce640 1 --2- 172.21.15.80:0/1592611222 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd60c102260 0x7fd60c104650 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fd5fc009900 tx=0x7fd5fc02a410 comp rx=0 tx=0).stop 2024-04-24T23:32:38.031 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.028+0000 7fd611dce640 1 -- 172.21.15.80:0/1592611222 shutdown_connections 2024-04-24T23:32:38.031 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.028+0000 7fd611dce640 1 --2- 172.21.15.80:0/1592611222 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd60c1074c0 0x7fd60c1098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:38.032 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.028+0000 7fd611dce640 1 --2- 172.21.15.80:0/1592611222 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd60c104b90 0x7fd60c106f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:38.032 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.028+0000 7fd611dce640 1 --2- 172.21.15.80:0/1592611222 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd60c102260 0x7fd60c104650 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:38.032 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.028+0000 7fd611dce640 1 -- 172.21.15.80:0/1592611222 >> 172.21.15.80:0/1592611222 conn(0x7fd60c0fbbf0 msgr2=0x7fd60c0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:32:38.032 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.028+0000 7fd611dce640 1 -- 172.21.15.80:0/1592611222 shutdown_connections 2024-04-24T23:32:38.032 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.028+0000 7fd611dce640 1 -- 172.21.15.80:0/1592611222 wait complete. 2024-04-24T23:32:38.032 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.028+0000 7fd611dce640 1 Processor -- start 2024-04-24T23:32:38.032 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.029+0000 7fd611dce640 1 -- start start 2024-04-24T23:32:38.032 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.029+0000 7fd611dce640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd60c102260 0x7fd60c197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:38.033 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.029+0000 7fd611dce640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd60c104b90 0x7fd60c198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:38.033 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.029+0000 7fd611dce640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd60c1074c0 0x7fd60c19f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:38.033 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.029+0000 7fd611dce640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fd60c10d420 con 0x7fd60c104b90 2024-04-24T23:32:38.033 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.029+0000 7fd611dce640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd60c10d2a0 con 0x7fd60c1074c0 2024-04-24T23:32:38.033 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.029+0000 7fd611dce640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd60c10d5a0 con 0x7fd60c102260 2024-04-24T23:32:38.033 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.029+0000 7fd60affd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd60c104b90 0x7fd60c198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:38.033 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.029+0000 7fd60affd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd60c104b90 0x7fd60c198030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:49124/0 (socket says 172.21.15.80:49124) 2024-04-24T23:32:38.033 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.029+0000 7fd60affd640 1 -- 172.21.15.80:0/2714246738 learned_addr learned my addr 172.21.15.80:0/2714246738 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:32:38.033 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.029+0000 7fd60b7fe640 1 --2- 172.21.15.80:0/2714246738 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd60c102260 0x7fd60c197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:38.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.029+0000 7fd60bfff640 1 --2- 172.21.15.80:0/2714246738 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd60c1074c0 0x7fd60c19f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:38.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.029+0000 7fd60affd640 1 -- 172.21.15.80:0/2714246738 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd60c102260 msgr2=0x7fd60c197af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:38.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.029+0000 7fd60affd640 1 --2- 172.21.15.80:0/2714246738 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd60c102260 0x7fd60c197af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:38.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.029+0000 7fd60affd640 1 -- 172.21.15.80:0/2714246738 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd60c1074c0 msgr2=0x7fd60c19f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:38.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.029+0000 7fd60affd640 1 --2- 172.21.15.80:0/2714246738 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd60c1074c0 0x7fd60c19f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:38.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.029+0000 7fd60affd640 1 -- 172.21.15.80:0/2714246738 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd60c19f710 con 0x7fd60c104b90 2024-04-24T23:32:38.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.029+0000 7fd60bfff640 1 --2- 172.21.15.80:0/2714246738 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd60c1074c0 0x7fd60c19f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:32:38.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.030+0000 7fd60affd640 1 --2- 172.21.15.80:0/2714246738 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd60c104b90 0x7fd60c198030 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fd60000d740 tx=0x7fd60000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:32:38.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.030+0000 7fd608ff9640 1 -- 172.21.15.80:0/2714246738 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd600017070 con 0x7fd60c104b90 2024-04-24T23:32:38.035 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.030+0000 7fd608ff9640 1 -- 172.21.15.80:0/2714246738 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd60000de50 con 0x7fd60c104b90 2024-04-24T23:32:38.035 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.030+0000 7fd611dce640 1 -- 172.21.15.80:0/2714246738 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd60c19fa00 con 0x7fd60c104b90 2024-04-24T23:32:38.035 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.030+0000 7fd608ff9640 1 -- 172.21.15.80:0/2714246738 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd600004e00 con 0x7fd60c104b90 2024-04-24T23:32:38.035 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.030+0000 7fd611dce640 1 -- 172.21.15.80:0/2714246738 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd60c19ff00 con 0x7fd60c104b90 2024-04-24T23:32:38.036 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.031+0000 7fd611dce640 1 -- 172.21.15.80:0/2714246738 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd60c0fd6e0 con 0x7fd60c104b90 2024-04-24T23:32:38.036 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.032+0000 7fd608ff9640 1 -- 172.21.15.80:0/2714246738 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd600004fa0 con 0x7fd60c104b90 2024-04-24T23:32:38.036 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.033+0000 7fd608ff9640 1 --2- 172.21.15.80:0/2714246738 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fd5f40a2b90 0x7fd5f40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:38.036 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.033+0000 7fd60b7fe640 1 --2- 172.21.15.80:0/2714246738 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fd5f40a2b90 0x7fd5f40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:38.036 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.033+0000 7fd608ff9640 1 -- 172.21.15.80:0/2714246738 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fd6000cf200 con 0x7fd60c104b90 2024-04-24T23:32:38.037 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.033+0000 7fd60b7fe640 1 --2- 172.21.15.80:0/2714246738 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fd5f40a2b90 0x7fd5f40a5050 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fd5fc0314e0 tx=0x7fd5fc000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:32:38.037 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.037+0000 7fd608ff9640 1 -- 172.21.15.80:0/2714246738 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fd60009bd30 con 0x7fd60c104b90 2024-04-24T23:32:38.216 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.216+0000 7fd611dce640 1 -- 172.21.15.80:0/2714246738 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd60c02d3c0 con 0x7fd60c104b90 2024-04-24T23:32:38.218 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.217+0000 7fd608ff9640 1 -- 172.21.15.80:0/2714246738 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24839 (secure 0 0 0) 0x7fd60009b630 con 0x7fd60c104b90 2024-04-24T23:32:38.218 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-24T23:32:38.218 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":94,"fsid":"ec7885aa-0290-11ef-bc93-c7b262605968","created":"2024-04-24T23:19:05.586407+0000","modified":"2024-04-24T23:29:08.209684+0000","last_up_change":"2024-04-24T23:26:25.938316+0000","last_in_change":"2024-04-24T23:26:04.786699+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:22:33.124428+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:27:17.575788+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:27:18.492762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T23:27:22.438503+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a525741d-b46c-4eef-aa64-3f057cac1497","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":2597718112},{"type":"v1","addr":"172.21.15.80:6803","nonce":2597718112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":2597718112},{"type":"v1","addr":"172.21.15.80:6805","nonce":2597718112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2597718112},{"type":"v1","addr":"172.21.15.80:6809","nonce":2597718112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":2597718112},{"type":"v1","addr":"172.21.15.80:6807","nonce":2597718112}]},"public_addr":"172.21.15.80:6803/2597718112","cluster_addr":"172.21.15.80:6805/2597718112","heartbeat_back_addr":"172.21.15.80:6809/2597718112","heartbeat_front_addr":"172.21.15.80:6807/2597718112","state":["exists","up"]},{"osd":1,"uuid":"2d0f0d67-6db4-40ac-91cc-58604ab6795a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":2395610201},{"type":"v1","addr":"172.21.15.134:6801","nonce":2395610201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":2395610201},{"type":"v1","addr":"172.21.15.134:6803","nonce":2395610201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":2395610201},{"type":"v1","addr":"172.21.15.134:6807","nonce":2395610201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":2395610201},{"type":"v1","addr":"172.21.15.134:6805","nonce":2395610201}]},"public_addr":"172.21.15.134:6801/2395610201","cluster_addr":"172.21.15.134:6803/2395610201","heartbeat_back_addr":"172.21.15.134:6807/2395610201","heartbeat_front_addr":"172.21.15.134:6805/2395610201","state":["exists","up"]},{"osd":2,"uuid":"781fb879-fedc-4c84-82fb-59e073bc4cf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1629152187},{"type":"v1","addr":"172.21.15.148:6801","nonce":1629152187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1629152187},{"type":"v1","addr":"172.21.15.148:6803","nonce":1629152187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1629152187},{"type":"v1","addr":"172.21.15.148:6807","nonce":1629152187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1629152187},{"type":"v1","addr":"172.21.15.148:6805","nonce":1629152187}]},"public_addr":"172.21.15.148:6801/1629152187","cluster_addr":"172.21.15.148:6803/1629152187","heartbeat_back_addr":"172.21.15.148:6807/1629152187","heartbeat_front_addr":"172.21.15.148:6805/1629152187","state":["exists","up"]},{"osd":3,"uuid":"cd982757-f2f6-4296-b3e7-4cd40fbd8d9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":3128298510},{"type":"v1","addr":"172.21.15.80:6811","nonce":3128298510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":3128298510},{"type":"v1","addr":"172.21.15.80:6813","nonce":3128298510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":3128298510},{"type":"v1","addr":"172.21.15.80:6817","nonce":3128298510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":3128298510},{"type":"v1","addr":"172.21.15.80:6815","nonce":3128298510}]},"public_addr":"172.21.15.80:6811/3128298510","cluster_addr":"172.21.15.80:6813/3128298510","heartbeat_back_addr":"172.21.15.80:6817/3128298510","heartbeat_front_addr":"172.21.15.80:6815/3128298510","state":["exists","up"]},{"osd":4,"uuid":"dfc55a79-a7e8-417a-b898-ec44171a54af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1917556060},{"type":"v1","addr":"172.21.15.134:6809","nonce":1917556060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1917556060},{"type":"v1","addr":"172.21.15.134:6811","nonce":1917556060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1917556060},{"type":"v1","addr":"172.21.15.134:6815","nonce":1917556060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1917556060},{"type":"v1","addr":"172.21.15.134:6813","nonce":1917556060}]},"public_addr":"172.21.15.134:6809/1917556060","cluster_addr":"172.21.15.134:6811/1917556060","heartbeat_back_addr":"172.21.15.134:6815/1917556060","heartbeat_front_addr":"172.21.15.134:6813/1917556060","state":["exists","up"]},{"osd":5,"uuid":"b5491337-0694-440c-b679-312f3d14cf65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":2715951281},{"type":"v1","addr":"172.21.15.148:6809","nonce":2715951281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":2715951281},{"type":"v1","addr":"172.21.15.148:6811","nonce":2715951281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":2715951281},{"type":"v1","addr":"172.21.15.148:6815","nonce":2715951281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":2715951281},{"type":"v1","addr":"172.21.15.148:6813","nonce":2715951281}]},"public_addr":"172.21.15.148:6809/2715951281","cluster_addr":"172.21.15.148:6811/2715951281","heartbeat_back_addr":"172.21.15.148:6815/2715951281","heartbeat_front_addr":"172.21.15.148:6813/2715951281","state":["exists","up"]},{"osd":6,"uuid":"ba4b5df7-d86e-47cd-ab67-cf763c4e4d6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":953261724},{"type":"v1","addr":"172.21.15.80:6819","nonce":953261724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":953261724},{"type":"v1","addr":"172.21.15.80:6821","nonce":953261724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":953261724},{"type":"v1","addr":"172.21.15.80:6825","nonce":953261724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":953261724},{"type":"v1","addr":"172.21.15.80:6823","nonce":953261724}]},"public_addr":"172.21.15.80:6819/953261724","cluster_addr":"172.21.15.80:6821/953261724","heartbeat_back_addr":"172.21.15.80:6825/953261724","heartbeat_front_addr":"172.21.15.80:6823/953261724","state":["exists","up"]},{"osd":7,"uuid":"464a503a-75ae-49e7-ab6e-a7f9f2a6a307","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":108354844},{"type":"v1","addr":"172.21.15.134:6817","nonce":108354844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":108354844},{"type":"v1","addr":"172.21.15.134:6819","nonce":108354844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":108354844},{"type":"v1","addr":"172.21.15.134:6823","nonce":108354844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":108354844},{"type":"v1","addr":"172.21.15.134:6821","nonce":108354844}]},"public_addr":"172.21.15.134:6817/108354844","cluster_addr":"172.21.15.134:6819/108354844","heartbeat_back_addr":"172.21.15.134:6823/108354844","heartbeat_front_addr":"172.21.15.134:6821/108354844","state":["exists","up"]},{"osd":8,"uuid":"ce544b2e-4540-47c0-9997-500c45351330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3559655329},{"type":"v1","addr":"172.21.15.148:6817","nonce":3559655329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":3559655329},{"type":"v1","addr":"172.21.15.148:6819","nonce":3559655329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":3559655329},{"type":"v1","addr":"172.21.15.148:6823","nonce":3559655329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3559655329},{"type":"v1","addr":"172.21.15.148:6821","nonce":3559655329}]},"public_addr":"172.21.15.148:6817/3559655329","cluster_addr":"172.21.15.148:6819/3559655329","heartbeat_back_addr":"172.21.15.148:6823/3559655329","heartbeat_front_addr":"172.21.15.148:6821/3559655329","state":["exists","up"]},{"osd":9,"uuid":"06de3732-19b9-496c-a464-55322efa6759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":4019913545},{"type":"v1","addr":"172.21.15.80:6827","nonce":4019913545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":4019913545},{"type":"v1","addr":"172.21.15.80:6829","nonce":4019913545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":4019913545},{"type":"v1","addr":"172.21.15.80:6833","nonce":4019913545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":4019913545},{"type":"v1","addr":"172.21.15.80:6831","nonce":4019913545}]},"public_addr":"172.21.15.80:6827/4019913545","cluster_addr":"172.21.15.80:6829/4019913545","heartbeat_back_addr":"172.21.15.80:6833/4019913545","heartbeat_front_addr":"172.21.15.80:6831/4019913545","state":["exists","up"]},{"osd":10,"uuid":"8d92f0d9-5c64-429a-b750-c33ae0e070c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":647343766},{"type":"v1","addr":"172.21.15.134:6825","nonce":647343766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":647343766},{"type":"v1","addr":"172.21.15.134:6827","nonce":647343766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":647343766},{"type":"v1","addr":"172.21.15.134:6831","nonce":647343766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":647343766},{"type":"v1","addr":"172.21.15.134:6829","nonce":647343766}]},"public_addr":"172.21.15.134:6825/647343766","cluster_addr":"172.21.15.134:6827/647343766","heartbeat_back_addr":"172.21.15.134:6831/647343766","heartbeat_front_addr":"172.21.15.134:6829/647343766","state":["exists","up"]},{"osd":11,"uuid":"4c003a41-f65b-479f-bc80-b27d680cf936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":4068024116},{"type":"v1","addr":"172.21.15.148:6825","nonce":4068024116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":4068024116},{"type":"v1","addr":"172.21.15.148:6827","nonce":4068024116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":4068024116},{"type":"v1","addr":"172.21.15.148:6831","nonce":4068024116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":4068024116},{"type":"v1","addr":"172.21.15.148:6829","nonce":4068024116}]},"public_addr":"172.21.15.148:6825/4068024116","cluster_addr":"172.21.15.148:6827/4068024116","heartbeat_back_addr":"172.21.15.148:6831/4068024116","heartbeat_front_addr":"172.21.15.148:6829/4068024116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:21:42.433631+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:22:06.145933+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:22:30.726143+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:22:55.729961+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:23:21.280913+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:23:46.609983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:24:12.585247+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:24:38.359287+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:25:04.491232+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:25:30.788089+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:25:56.891078+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:26:23.899113+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":7}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"4.a","mappings":[{"from":5,"to":8}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.80:0/2828378749":"2024-04-25T23:20:06.793609+0000","172.21.15.80:6801/233364790":"2024-04-25T23:20:06.793609+0000","172.21.15.80:6800/1614691410":"2024-04-25T23:19:40.490692+0000","172.21.15.80:0/1406187145":"2024-04-25T23:20:06.793609+0000","172.21.15.80:0/92953396":"2024-04-25T23:19:40.490692+0000","172.21.15.80:0/598129432":"2024-04-25T23:20:06.793609+0000","172.21.15.80:0/232193118":"2024-04-25T23:19:40.490692+0000","172.21.15.80:6800/233364790":"2024-04-25T23:20:06.793609+0000","172.21.15.80:6801/1614691410":"2024-04-25T23:19:40.490692+0000","172.21.15.80:0/2081562529":"2024-04-25T23:19:40.490692+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:32:38.220 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.219+0000 7fd611dce640 1 -- 172.21.15.80:0/2714246738 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fd5f40a2b90 msgr2=0x7fd5f40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:38.220 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.219+0000 7fd611dce640 1 --2- 172.21.15.80:0/2714246738 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fd5f40a2b90 0x7fd5f40a5050 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fd5fc0314e0 tx=0x7fd5fc000bc0 comp rx=0 tx=0).stop 2024-04-24T23:32:38.221 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.219+0000 7fd611dce640 1 -- 172.21.15.80:0/2714246738 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd60c104b90 msgr2=0x7fd60c198030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:38.221 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.219+0000 7fd611dce640 1 --2- 172.21.15.80:0/2714246738 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd60c104b90 0x7fd60c198030 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fd60000d740 tx=0x7fd60000dc00 comp rx=0 tx=0).stop 2024-04-24T23:32:38.221 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.219+0000 7fd611dce640 1 -- 172.21.15.80:0/2714246738 shutdown_connections 2024-04-24T23:32:38.221 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.219+0000 7fd611dce640 1 --2- 172.21.15.80:0/2714246738 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fd5f40a2b90 0x7fd5f40a5050 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:38.221 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.219+0000 7fd611dce640 1 --2- 172.21.15.80:0/2714246738 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd60c1074c0 0x7fd60c19f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:38.221 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.219+0000 7fd611dce640 1 --2- 172.21.15.80:0/2714246738 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd60c104b90 0x7fd60c198030 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:38.221 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.219+0000 7fd611dce640 1 --2- 172.21.15.80:0/2714246738 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd60c102260 0x7fd60c197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:38.221 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.219+0000 7fd611dce640 1 -- 172.21.15.80:0/2714246738 >> 172.21.15.80:0/2714246738 conn(0x7fd60c0fbbf0 msgr2=0x7fd60c102c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:32:38.222 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.220+0000 7fd611dce640 1 -- 172.21.15.80:0/2714246738 shutdown_connections 2024-04-24T23:32:38.222 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.220+0000 7fd611dce640 1 -- 172.21.15.80:0/2714246738 wait complete. 2024-04-24T23:32:38.230 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:32:38 smithi080 ceph-mon[29152]: pgmap v459: 129 pgs: 129 active+clean; 75 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 59 KiB/s wr, 6 op/s 2024-04-24T23:32:38.232 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:22:33.124428+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T23:27:17.575788+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T23:27:18.492762+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-24T23:27:22.438503+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-24T23:32:38.232 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:32:38.335 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.332+0000 7f4a9ddee640 1 -- 172.21.15.80:0/2972982421 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4a98102250 msgr2=0x7f4a98108ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:38.335 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.332+0000 7f4a9ddee640 1 --2- 172.21.15.80:0/2972982421 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4a98102250 0x7f4a98108ae0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f4a90009ca0 tx=0x7f4a9002a8c0 comp rx=0 tx=0).stop 2024-04-24T23:32:38.335 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.332+0000 7f4a9ddee640 1 -- 172.21.15.80:0/2972982421 shutdown_connections 2024-04-24T23:32:38.335 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.332+0000 7f4a9ddee640 1 --2- 172.21.15.80:0/2972982421 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4a98102250 0x7f4a98108ae0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:38.335 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.332+0000 7f4a9ddee640 1 --2- 172.21.15.80:0/2972982421 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4a98101890 0x7f4a98101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:38.336 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.332+0000 7f4a9ddee640 1 --2- 172.21.15.80:0/2972982421 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a981006a0 0x7f4a98100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:38.336 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.332+0000 7f4a9ddee640 1 -- 172.21.15.80:0/2972982421 >> 172.21.15.80:0/2972982421 conn(0x7f4a980fbc30 msgr2=0x7f4a980fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:32:38.336 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.332+0000 7f4a9ddee640 1 -- 172.21.15.80:0/2972982421 shutdown_connections 2024-04-24T23:32:38.336 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.332+0000 7f4a9ddee640 1 -- 172.21.15.80:0/2972982421 wait complete. 2024-04-24T23:32:38.336 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.333+0000 7f4a9ddee640 1 Processor -- start 2024-04-24T23:32:38.336 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.333+0000 7f4a9ddee640 1 -- start start 2024-04-24T23:32:38.336 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.333+0000 7f4a9ddee640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a981006a0 0x7f4a9806a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:38.336 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.333+0000 7f4a9ddee640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4a98101890 0x7f4a9819c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:38.336 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.333+0000 7f4a9ddee640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4a98102250 0x7f4a9819ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:38.337 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.333+0000 7f4a9ddee640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f4a9810c300 con 0x7f4a98101890 2024-04-24T23:32:38.337 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.333+0000 7f4a9ddee640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4a9810c180 con 0x7f4a98102250 2024-04-24T23:32:38.337 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.333+0000 7f4a9ddee640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4a9810c480 con 0x7f4a981006a0 2024-04-24T23:32:38.337 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.333+0000 7f4a96ffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4a98101890 0x7f4a9819c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:38.337 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.333+0000 7f4a96ffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4a98101890 0x7f4a9819c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:49144/0 (socket says 172.21.15.80:49144) 2024-04-24T23:32:38.337 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.333+0000 7f4a96ffd640 1 -- 172.21.15.80:0/1264146903 learned_addr learned my addr 172.21.15.80:0/1264146903 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:32:38.337 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.333+0000 7f4a977fe640 1 --2- 172.21.15.80:0/1264146903 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a981006a0 0x7f4a9806a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:38.337 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.333+0000 7f4a96ffd640 1 -- 172.21.15.80:0/1264146903 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a981006a0 msgr2=0x7f4a9806a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:38.337 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.333+0000 7f4a96ffd640 1 --2- 172.21.15.80:0/1264146903 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a981006a0 0x7f4a9806a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:38.338 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.333+0000 7f4a96ffd640 1 -- 172.21.15.80:0/1264146903 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4a98102250 msgr2=0x7f4a9819ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:38.338 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.333+0000 7f4a96ffd640 1 --2- 172.21.15.80:0/1264146903 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4a98102250 0x7f4a9819ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:38.338 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.333+0000 7f4a96ffd640 1 -- 172.21.15.80:0/1264146903 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a9819f5e0 con 0x7f4a98101890 2024-04-24T23:32:38.338 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.334+0000 7f4a96ffd640 1 --2- 172.21.15.80:0/1264146903 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4a98101890 0x7f4a9819c6e0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f4a8000d950 tx=0x7f4a8000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:32:38.338 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.334+0000 7f4a94ff9640 1 -- 172.21.15.80:0/1264146903 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a80017070 con 0x7f4a98101890 2024-04-24T23:32:38.338 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.334+0000 7f4a94ff9640 1 -- 172.21.15.80:0/1264146903 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a80009c30 con 0x7f4a98101890 2024-04-24T23:32:38.338 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.334+0000 7f4a9ddee640 1 -- 172.21.15.80:0/1264146903 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a9819f8d0 con 0x7f4a98101890 2024-04-24T23:32:38.339 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.334+0000 7f4a94ff9640 1 -- 172.21.15.80:0/1264146903 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a8000f6b0 con 0x7f4a98101890 2024-04-24T23:32:38.339 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.334+0000 7f4a9ddee640 1 -- 172.21.15.80:0/1264146903 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a9819fe20 con 0x7f4a98101890 2024-04-24T23:32:38.340 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.335+0000 7f4a9ddee640 1 -- 172.21.15.80:0/1264146903 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a98107960 con 0x7f4a98101890 2024-04-24T23:32:38.340 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.336+0000 7f4a94ff9640 1 -- 172.21.15.80:0/1264146903 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4a800109d0 con 0x7f4a98101890 2024-04-24T23:32:38.340 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.336+0000 7f4a94ff9640 1 --2- 172.21.15.80:0/1264146903 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f4a840a2b90 0x7f4a840a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:38.340 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.336+0000 7f4a977fe640 1 --2- 172.21.15.80:0/1264146903 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f4a840a2b90 0x7f4a840a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:38.340 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.336+0000 7f4a94ff9640 1 -- 172.21.15.80:0/1264146903 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f4a800cf110 con 0x7f4a98101890 2024-04-24T23:32:38.340 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.337+0000 7f4a977fe640 1 --2- 172.21.15.80:0/1264146903 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f4a840a2b90 0x7f4a840a5050 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f4a880058c0 tx=0x7f4a88005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:32:38.341 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.341+0000 7f4a94ff9640 1 -- 172.21.15.80:0/1264146903 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f4a80009de0 con 0x7f4a98101890 2024-04-24T23:32:38.446 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:32:38 smithi134 ceph-mon[32619]: pgmap v459: 129 pgs: 129 active+clean; 75 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 59 KiB/s wr, 6 op/s 2024-04-24T23:32:38.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:32:38 smithi148 ceph-mon[32528]: pgmap v459: 129 pgs: 129 active+clean; 75 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 59 KiB/s wr, 6 op/s 2024-04-24T23:32:38.518 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.517+0000 7f4a9ddee640 1 -- 172.21.15.80:0/1264146903 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4a98062340 con 0x7f4a98101890 2024-04-24T23:32:38.518 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.517+0000 7f4a94ff9640 1 -- 172.21.15.80:0/1264146903 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7f4a8009bca0 con 0x7f4a98101890 2024-04-24T23:32:38.518 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 1 2024-04-24T23:32:38.520 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.519+0000 7f4a9ddee640 1 -- 172.21.15.80:0/1264146903 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f4a840a2b90 msgr2=0x7f4a840a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:38.520 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.519+0000 7f4a9ddee640 1 --2- 172.21.15.80:0/1264146903 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f4a840a2b90 0x7f4a840a5050 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f4a880058c0 tx=0x7f4a88005850 comp rx=0 tx=0).stop 2024-04-24T23:32:38.520 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.519+0000 7f4a9ddee640 1 -- 172.21.15.80:0/1264146903 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4a98101890 msgr2=0x7f4a9819c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:38.520 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.519+0000 7f4a9ddee640 1 --2- 172.21.15.80:0/1264146903 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4a98101890 0x7f4a9819c6e0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f4a8000d950 tx=0x7f4a8000de10 comp rx=0 tx=0).stop 2024-04-24T23:32:38.521 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.519+0000 7f4a9ddee640 1 -- 172.21.15.80:0/1264146903 shutdown_connections 2024-04-24T23:32:38.521 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.519+0000 7f4a9ddee640 1 --2- 172.21.15.80:0/1264146903 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f4a840a2b90 0x7f4a840a5050 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:38.521 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.519+0000 7f4a9ddee640 1 --2- 172.21.15.80:0/1264146903 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4a98102250 0x7f4a9819ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:38.521 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.519+0000 7f4a9ddee640 1 --2- 172.21.15.80:0/1264146903 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4a98101890 0x7f4a9819c6e0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:38.521 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.519+0000 7f4a9ddee640 1 --2- 172.21.15.80:0/1264146903 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a981006a0 0x7f4a9806a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:38.521 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.519+0000 7f4a9ddee640 1 -- 172.21.15.80:0/1264146903 >> 172.21.15.80:0/1264146903 conn(0x7f4a980fbc30 msgr2=0x7f4a98104b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:32:38.521 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.520+0000 7f4a9ddee640 1 -- 172.21.15.80:0/1264146903 shutdown_connections 2024-04-24T23:32:38.521 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.520+0000 7f4a9ddee640 1 -- 172.21.15.80:0/1264146903 wait complete. 2024-04-24T23:32:38.531 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T23:32:38.633 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.631+0000 7fd37befc640 1 -- 172.21.15.80:0/2585613182 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd374103350 msgr2=0x7fd374109be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:38.634 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.631+0000 7fd37befc640 1 --2- 172.21.15.80:0/2585613182 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd374103350 0x7fd374109be0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fd36c009ca0 tx=0x7fd36c02a8c0 comp rx=0 tx=0).stop 2024-04-24T23:32:38.634 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.631+0000 7fd37befc640 1 -- 172.21.15.80:0/2585613182 shutdown_connections 2024-04-24T23:32:38.634 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.631+0000 7fd37befc640 1 --2- 172.21.15.80:0/2585613182 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd374103350 0x7fd374109be0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:38.634 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.631+0000 7fd37befc640 1 --2- 172.21.15.80:0/2585613182 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd374102990 0x7fd374102e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:38.634 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.631+0000 7fd37befc640 1 --2- 172.21.15.80:0/2585613182 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd3741017a0 0x7fd374101bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:38.634 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.631+0000 7fd37befc640 1 -- 172.21.15.80:0/2585613182 >> 172.21.15.80:0/2585613182 conn(0x7fd3740fcd10 msgr2=0x7fd3740ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:32:38.635 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.631+0000 7fd37befc640 1 -- 172.21.15.80:0/2585613182 shutdown_connections 2024-04-24T23:32:38.635 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.631+0000 7fd37befc640 1 -- 172.21.15.80:0/2585613182 wait complete. 2024-04-24T23:32:38.635 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.632+0000 7fd37befc640 1 Processor -- start 2024-04-24T23:32:38.635 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.632+0000 7fd37befc640 1 -- start start 2024-04-24T23:32:38.635 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.632+0000 7fd37befc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd3741017a0 0x7fd374197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:38.635 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.632+0000 7fd37befc640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd374102990 0x7fd374197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:38.635 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.632+0000 7fd37befc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd374103350 0x7fd37419efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:38.635 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.632+0000 7fd37befc640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fd37410d400 con 0x7fd374102990 2024-04-24T23:32:38.636 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.632+0000 7fd37befc640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd37410d280 con 0x7fd3741017a0 2024-04-24T23:32:38.636 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.632+0000 7fd37befc640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd37410d580 con 0x7fd374103350 2024-04-24T23:32:38.636 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.632+0000 7fd379497640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd374102990 0x7fd374197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:38.636 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.632+0000 7fd379497640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd374102990 0x7fd374197f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:49160/0 (socket says 172.21.15.80:49160) 2024-04-24T23:32:38.636 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.632+0000 7fd379497640 1 -- 172.21.15.80:0/2459247910 learned_addr learned my addr 172.21.15.80:0/2459247910 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:32:38.636 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.632+0000 7fd379c98640 1 --2- 172.21.15.80:0/2459247910 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd3741017a0 0x7fd374197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:38.636 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.632+0000 7fd379497640 1 -- 172.21.15.80:0/2459247910 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd374103350 msgr2=0x7fd37419efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:38.636 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.632+0000 7fd379497640 1 --2- 172.21.15.80:0/2459247910 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd374103350 0x7fd37419efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:38.636 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.632+0000 7fd379497640 1 -- 172.21.15.80:0/2459247910 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd3741017a0 msgr2=0x7fd374197a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:38.637 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.632+0000 7fd379497640 1 --2- 172.21.15.80:0/2459247910 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd3741017a0 0x7fd374197a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:38.637 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.632+0000 7fd379497640 1 -- 172.21.15.80:0/2459247910 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd37419f640 con 0x7fd374102990 2024-04-24T23:32:38.637 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.632+0000 7fd379c98640 1 --2- 172.21.15.80:0/2459247910 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd3741017a0 0x7fd374197a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:32:38.637 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.632+0000 7fd379497640 1 --2- 172.21.15.80:0/2459247910 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd374102990 0x7fd374197f60 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fd36400c920 tx=0x7fd36400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:32:38.637 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.632+0000 7fd36affd640 1 -- 172.21.15.80:0/2459247910 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd364007c70 con 0x7fd374102990 2024-04-24T23:32:38.637 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.632+0000 7fd37befc640 1 -- 172.21.15.80:0/2459247910 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd37419f930 con 0x7fd374102990 2024-04-24T23:32:38.637 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.633+0000 7fd36affd640 1 -- 172.21.15.80:0/2459247910 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd364007e10 con 0x7fd374102990 2024-04-24T23:32:38.637 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.633+0000 7fd36affd640 1 -- 172.21.15.80:0/2459247910 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd364018890 con 0x7fd374102990 2024-04-24T23:32:38.638 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.633+0000 7fd37befc640 1 -- 172.21.15.80:0/2459247910 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd37419fe80 con 0x7fd374102990 2024-04-24T23:32:38.639 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.633+0000 7fd37befc640 1 -- 172.21.15.80:0/2459247910 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd374108a60 con 0x7fd374102990 2024-04-24T23:32:38.639 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.635+0000 7fd36affd640 1 -- 172.21.15.80:0/2459247910 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd364018a50 con 0x7fd374102990 2024-04-24T23:32:38.639 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.635+0000 7fd36affd640 1 --2- 172.21.15.80:0/2459247910 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fd3600a2b90 0x7fd3600a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:38.639 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.635+0000 7fd379c98640 1 --2- 172.21.15.80:0/2459247910 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fd3600a2b90 0x7fd3600a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:38.639 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.635+0000 7fd36affd640 1 -- 172.21.15.80:0/2459247910 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fd3640cf830 con 0x7fd374102990 2024-04-24T23:32:38.639 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.636+0000 7fd379c98640 1 --2- 172.21.15.80:0/2459247910 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fd3600a2b90 0x7fd3600a5050 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fd370005d00 tx=0x7fd370005ef0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:32:38.640 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.639+0000 7fd36affd640 1 -- 172.21.15.80:0/2459247910 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fd364014030 con 0x7fd374102990 2024-04-24T23:32:38.816 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.816+0000 7fd37befc640 1 -- 172.21.15.80:0/2459247910 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd37402d300 con 0x7fd374102990 2024-04-24T23:32:38.817 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.816+0000 7fd36affd640 1 -- 172.21.15.80:0/2459247910 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) ==== 104+0+11 (secure 0 0 0) 0x7fd36400de20 con 0x7fd374102990 2024-04-24T23:32:38.817 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 32 2024-04-24T23:32:38.819 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.818+0000 7fd37befc640 1 -- 172.21.15.80:0/2459247910 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fd3600a2b90 msgr2=0x7fd3600a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:38.819 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.818+0000 7fd37befc640 1 --2- 172.21.15.80:0/2459247910 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fd3600a2b90 0x7fd3600a5050 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fd370005d00 tx=0x7fd370005ef0 comp rx=0 tx=0).stop 2024-04-24T23:32:38.819 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.818+0000 7fd37befc640 1 -- 172.21.15.80:0/2459247910 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd374102990 msgr2=0x7fd374197f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:38.819 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.818+0000 7fd37befc640 1 --2- 172.21.15.80:0/2459247910 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd374102990 0x7fd374197f60 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fd36400c920 tx=0x7fd36400cde0 comp rx=0 tx=0).stop 2024-04-24T23:32:38.819 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.818+0000 7fd37befc640 1 -- 172.21.15.80:0/2459247910 shutdown_connections 2024-04-24T23:32:38.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.818+0000 7fd37befc640 1 --2- 172.21.15.80:0/2459247910 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fd3600a2b90 0x7fd3600a5050 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:38.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.818+0000 7fd37befc640 1 --2- 172.21.15.80:0/2459247910 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd374103350 0x7fd37419efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:38.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.818+0000 7fd37befc640 1 --2- 172.21.15.80:0/2459247910 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd374102990 0x7fd374197f60 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:38.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.818+0000 7fd37befc640 1 --2- 172.21.15.80:0/2459247910 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd3741017a0 0x7fd374197a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:38.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.818+0000 7fd37befc640 1 -- 172.21.15.80:0/2459247910 >> 172.21.15.80:0/2459247910 conn(0x7fd3740fcd10 msgr2=0x7fd374105c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:32:38.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.818+0000 7fd37befc640 1 -- 172.21.15.80:0/2459247910 shutdown_connections 2024-04-24T23:32:38.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.818+0000 7fd37befc640 1 -- 172.21.15.80:0/2459247910 wait complete. 2024-04-24T23:32:38.830 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T23:32:38.932 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.930+0000 7f4a0b261640 1 -- 172.21.15.80:0/1164907183 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4a04101870 msgr2=0x7f4a04101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:38.933 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.930+0000 7f4a0b261640 1 --2- 172.21.15.80:0/1164907183 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4a04101870 0x7f4a04101cf0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f49f8009900 tx=0x7f49f802a410 comp rx=0 tx=0).stop 2024-04-24T23:32:38.933 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.930+0000 7f4a0b261640 1 -- 172.21.15.80:0/1164907183 shutdown_connections 2024-04-24T23:32:38.933 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.930+0000 7f4a0b261640 1 --2- 172.21.15.80:0/1164907183 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a04102230 0x7f4a04108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:38.933 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.930+0000 7f4a0b261640 1 --2- 172.21.15.80:0/1164907183 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4a04101870 0x7f4a04101cf0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:38.933 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.930+0000 7f4a0b261640 1 --2- 172.21.15.80:0/1164907183 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4a04100680 0x7f4a04100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:38.933 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.930+0000 7f4a0b261640 1 -- 172.21.15.80:0/1164907183 >> 172.21.15.80:0/1164907183 conn(0x7f4a040fbbf0 msgr2=0x7f4a040fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:32:38.934 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.930+0000 7f4a0b261640 1 -- 172.21.15.80:0/1164907183 shutdown_connections 2024-04-24T23:32:38.934 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.930+0000 7f4a0b261640 1 -- 172.21.15.80:0/1164907183 wait complete. 2024-04-24T23:32:38.934 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.930+0000 7f4a0b261640 1 Processor -- start 2024-04-24T23:32:38.934 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.930+0000 7f4a0b261640 1 -- start start 2024-04-24T23:32:38.934 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.930+0000 7f4a0b261640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4a04100680 0x7f4a04196980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:38.934 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.930+0000 7f4a0b261640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4a04101870 0x7f4a04196ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:38.934 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.930+0000 7f4a0b261640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a04102230 0x7f4a0419df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:38.934 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.930+0000 7f4a0b261640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f4a0410c2e0 con 0x7f4a04101870 2024-04-24T23:32:38.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.930+0000 7f4a0b261640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4a0410c160 con 0x7f4a04100680 2024-04-24T23:32:38.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.930+0000 7f4a0b261640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4a0410c460 con 0x7f4a04102230 2024-04-24T23:32:38.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.930+0000 7f4a03fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4a04101870 0x7f4a04196ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:38.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.930+0000 7f4a03fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4a04101870 0x7f4a04196ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:49192/0 (socket says 172.21.15.80:49192) 2024-04-24T23:32:38.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.930+0000 7f4a03fff640 1 -- 172.21.15.80:0/3923050808 learned_addr learned my addr 172.21.15.80:0/3923050808 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:32:38.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.931+0000 7f4a08ffd640 1 --2- 172.21.15.80:0/3923050808 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4a04100680 0x7f4a04196980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:38.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.931+0000 7f4a097fe640 1 --2- 172.21.15.80:0/3923050808 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a04102230 0x7f4a0419df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:38.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.931+0000 7f4a03fff640 1 -- 172.21.15.80:0/3923050808 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a04102230 msgr2=0x7f4a0419df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:38.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.931+0000 7f4a03fff640 1 --2- 172.21.15.80:0/3923050808 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a04102230 0x7f4a0419df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:38.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.931+0000 7f4a03fff640 1 -- 172.21.15.80:0/3923050808 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4a04100680 msgr2=0x7f4a04196980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:38.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.931+0000 7f4a03fff640 1 --2- 172.21.15.80:0/3923050808 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4a04100680 0x7f4a04196980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:38.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.931+0000 7f4a03fff640 1 -- 172.21.15.80:0/3923050808 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a0419e5a0 con 0x7f4a04101870 2024-04-24T23:32:38.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.931+0000 7f4a03fff640 1 --2- 172.21.15.80:0/3923050808 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4a04101870 0x7f4a04196ec0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f49f8009230 tx=0x7f49f8004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:32:38.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.931+0000 7f4a01ffb640 1 -- 172.21.15.80:0/3923050808 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49f803f030 con 0x7f4a04101870 2024-04-24T23:32:38.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.931+0000 7f4a01ffb640 1 -- 172.21.15.80:0/3923050808 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49f8003ce0 con 0x7f4a04101870 2024-04-24T23:32:38.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.931+0000 7f4a0b261640 1 -- 172.21.15.80:0/3923050808 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a0419e830 con 0x7f4a04101870 2024-04-24T23:32:38.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.931+0000 7f4a01ffb640 1 -- 172.21.15.80:0/3923050808 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49f8035070 con 0x7f4a04101870 2024-04-24T23:32:38.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.931+0000 7f4a0b261640 1 -- 172.21.15.80:0/3923050808 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a0419ed20 con 0x7f4a04101870 2024-04-24T23:32:38.937 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.932+0000 7f4a0b261640 1 -- 172.21.15.80:0/3923050808 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a041079c0 con 0x7f4a04101870 2024-04-24T23:32:38.937 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.933+0000 7f4a01ffb640 1 -- 172.21.15.80:0/3923050808 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f49f8042030 con 0x7f4a04101870 2024-04-24T23:32:38.938 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.934+0000 7f4a01ffb640 1 --2- 172.21.15.80:0/3923050808 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f49ec0a2b90 0x7f49ec0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:38.938 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.934+0000 7f4a08ffd640 1 --2- 172.21.15.80:0/3923050808 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f49ec0a2b90 0x7f49ec0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:38.938 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.934+0000 7f4a01ffb640 1 -- 172.21.15.80:0/3923050808 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f49f80efe00 con 0x7f4a04101870 2024-04-24T23:32:38.938 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.934+0000 7f4a08ffd640 1 --2- 172.21.15.80:0/3923050808 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f49ec0a2b90 0x7f49ec0a5050 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f4a041016d0 tx=0x7f49f400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:32:38.938 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:38.938+0000 7f4a01ffb640 1 -- 172.21.15.80:0/3923050808 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f49f80bc990 con 0x7f4a04101870 2024-04-24T23:32:39.119 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:39.118+0000 7f4a0b261640 1 -- 172.21.15.80:0/3923050808 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f4a04062340 con 0x7f4a04101870 2024-04-24T23:32:39.119 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:39.119+0000 7f4a01ffb640 1 -- 172.21.15.80:0/3923050808 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) ==== 100+0+11 (secure 0 0 0) 0x7f49f80bc290 con 0x7f4a04101870 2024-04-24T23:32:39.120 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 64 2024-04-24T23:32:39.121 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:39.121+0000 7f4a0b261640 1 -- 172.21.15.80:0/3923050808 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f49ec0a2b90 msgr2=0x7f49ec0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:39.121 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:39.121+0000 7f4a0b261640 1 --2- 172.21.15.80:0/3923050808 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f49ec0a2b90 0x7f49ec0a5050 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f4a041016d0 tx=0x7f49f400b390 comp rx=0 tx=0).stop 2024-04-24T23:32:39.122 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:39.121+0000 7f4a0b261640 1 -- 172.21.15.80:0/3923050808 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4a04101870 msgr2=0x7f4a04196ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:39.122 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:39.121+0000 7f4a0b261640 1 --2- 172.21.15.80:0/3923050808 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4a04101870 0x7f4a04196ec0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f49f8009230 tx=0x7f49f8004360 comp rx=0 tx=0).stop 2024-04-24T23:32:39.122 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:39.121+0000 7f4a0b261640 1 -- 172.21.15.80:0/3923050808 shutdown_connections 2024-04-24T23:32:39.122 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:39.121+0000 7f4a0b261640 1 --2- 172.21.15.80:0/3923050808 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f49ec0a2b90 0x7f49ec0a5050 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:39.122 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:39.121+0000 7f4a0b261640 1 --2- 172.21.15.80:0/3923050808 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a04102230 0x7f4a0419df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:39.122 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:39.121+0000 7f4a0b261640 1 --2- 172.21.15.80:0/3923050808 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4a04101870 0x7f4a04196ec0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:39.122 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:39.121+0000 7f4a0b261640 1 --2- 172.21.15.80:0/3923050808 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4a04100680 0x7f4a04196980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:39.123 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:39.121+0000 7f4a0b261640 1 -- 172.21.15.80:0/3923050808 >> 172.21.15.80:0/3923050808 conn(0x7f4a040fbbf0 msgr2=0x7f4a04104b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:32:39.123 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:39.121+0000 7f4a0b261640 1 -- 172.21.15.80:0/3923050808 shutdown_connections 2024-04-24T23:32:39.123 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:39.121+0000 7f4a0b261640 1 -- 172.21.15.80:0/3923050808 wait complete. 2024-04-24T23:32:39.132 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-24T23:32:39.237 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:39.233+0000 7fe46449e640 1 -- 172.21.15.80:0/1096922251 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe45c101890 msgr2=0x7fe45c101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:39.237 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:39.233+0000 7fe46449e640 1 --2- 172.21.15.80:0/1096922251 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe45c101890 0x7fe45c101d10 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fe44c009900 tx=0x7fe44c02a410 comp rx=0 tx=0).stop 2024-04-24T23:32:39.237 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:39.233+0000 7fe46449e640 1 -- 172.21.15.80:0/1096922251 shutdown_connections 2024-04-24T23:32:39.237 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:39.233+0000 7fe46449e640 1 --2- 172.21.15.80:0/1096922251 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe45c102250 0x7fe45c108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:39.237 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:39.233+0000 7fe46449e640 1 --2- 172.21.15.80:0/1096922251 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe45c101890 0x7fe45c101d10 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:39.237 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:39.233+0000 7fe46449e640 1 --2- 172.21.15.80:0/1096922251 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe45c1006a0 0x7fe45c100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:39.237 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:39.233+0000 7fe46449e640 1 -- 172.21.15.80:0/1096922251 >> 172.21.15.80:0/1096922251 conn(0x7fe45c0fbc30 msgr2=0x7fe45c0fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:32:39.238 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:39.233+0000 7fe46449e640 1 -- 172.21.15.80:0/1096922251 shutdown_connections 2024-04-24T23:32:39.238 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:39.233+0000 7fe46449e640 1 -- 172.21.15.80:0/1096922251 wait complete. 2024-04-24T23:32:39.238 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:39.234+0000 7fe46449e640 1 Processor -- start 2024-04-24T23:32:39.238 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:39.234+0000 7fe46449e640 1 -- start start 2024-04-24T23:32:39.238 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:39.234+0000 7fe46449e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe45c1006a0 0x7fe45c197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:39.238 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:39.234+0000 7fe46449e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe45c101890 0x7fe45c1980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:39.239 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:39.234+0000 7fe46449e640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe45c102250 0x7fe45c19f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:39.239 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:39.234+0000 7fe46449e640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fe45c10c390 con 0x7fe45c102250 2024-04-24T23:32:39.239 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:39.234+0000 7fe46449e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe45c10c210 con 0x7fe45c101890 2024-04-24T23:32:39.239 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:39.234+0000 7fe46449e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe45c10c510 con 0x7fe45c1006a0 2024-04-24T23:32:39.239 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:39.234+0000 7fe462a3b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe45c102250 0x7fe45c19f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:39.239 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:39.234+0000 7fe462a3b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe45c102250 0x7fe45c19f0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:49202/0 (socket says 172.21.15.80:49202) 2024-04-24T23:32:39.239 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:39.234+0000 7fe462a3b640 1 -- 172.21.15.80:0/668472154 learned_addr learned my addr 172.21.15.80:0/668472154 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:32:39.239 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:39.234+0000 7fe46223a640 1 --2- 172.21.15.80:0/668472154 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe45c1006a0 0x7fe45c197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:39.241 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:39.234+0000 7fe461a39640 1 --2- 172.21.15.80:0/668472154 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe45c101890 0x7fe45c1980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:39.241 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:39.234+0000 7fe462a3b640 1 -- 172.21.15.80:0/668472154 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe45c1006a0 msgr2=0x7fe45c197b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:39.241 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:39.234+0000 7fe462a3b640 1 --2- 172.21.15.80:0/668472154 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe45c1006a0 0x7fe45c197b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:39.241 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:39.234+0000 7fe462a3b640 1 -- 172.21.15.80:0/668472154 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe45c101890 msgr2=0x7fe45c1980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:39.241 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:39.234+0000 7fe462a3b640 1 --2- 172.21.15.80:0/668472154 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe45c101890 0x7fe45c1980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:39.241 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:39.234+0000 7fe462a3b640 1 -- 172.21.15.80:0/668472154 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe45c19f750 con 0x7fe45c102250 2024-04-24T23:32:39.242 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:39.234+0000 7fe462a3b640 1 --2- 172.21.15.80:0/668472154 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe45c102250 0x7fe45c19f0e0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fe45400cce0 tx=0x7fe45400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:32:39.242 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:39.234+0000 7fe4537fe640 1 -- 172.21.15.80:0/668472154 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe45400f450 con 0x7fe45c102250 2024-04-24T23:32:39.242 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:39.234+0000 7fe4537fe640 1 -- 172.21.15.80:0/668472154 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe45400fad0 con 0x7fe45c102250 2024-04-24T23:32:39.242 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:39.234+0000 7fe46449e640 1 -- 172.21.15.80:0/668472154 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe45c19fa40 con 0x7fe45c102250 2024-04-24T23:32:39.242 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:39.234+0000 7fe4537fe640 1 -- 172.21.15.80:0/668472154 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe454018c10 con 0x7fe45c102250 2024-04-24T23:32:39.242 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:39.235+0000 7fe46449e640 1 -- 172.21.15.80:0/668472154 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe45c19ff90 con 0x7fe45c102250 2024-04-24T23:32:39.242 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:39.235+0000 7fe46449e640 1 -- 172.21.15.80:0/668472154 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe45c1a0220 con 0x7fe45c102250 2024-04-24T23:32:39.243 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:39.236+0000 7fe4537fe640 1 -- 172.21.15.80:0/668472154 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe45400f5f0 con 0x7fe45c102250 2024-04-24T23:32:39.243 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:39.237+0000 7fe4537fe640 1 --2- 172.21.15.80:0/668472154 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fe4480a2b90 0x7fe4480a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:39.244 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:39.237+0000 7fe46223a640 1 --2- 172.21.15.80:0/668472154 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fe4480a2b90 0x7fe4480a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:39.244 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:39.237+0000 7fe4537fe640 1 -- 172.21.15.80:0/668472154 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fe4540cfb80 con 0x7fe45c102250 2024-04-24T23:32:39.244 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:39.237+0000 7fe46223a640 1 --2- 172.21.15.80:0/668472154 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fe4480a2b90 0x7fe4480a5050 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fe45c1016f0 tx=0x7fe458008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:32:39.244 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:39.241+0000 7fe4537fe640 1 -- 172.21.15.80:0/668472154 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fe454014030 con 0x7fe45c102250 2024-04-24T23:32:39.255 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:32:39 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/2714246738' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:32:39.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:32:39 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/1264146903' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:32:39.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:32:39 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/2459247910' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T23:32:39.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:32:39 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/3923050808' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T23:32:39.415 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:39.415+0000 7fe46449e640 1 -- 172.21.15.80:0/668472154 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fe45c1a03f0 con 0x7fe45c102250 2024-04-24T23:32:39.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:39.415+0000 7fe4537fe640 1 -- 172.21.15.80:0/668472154 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) ==== 103+0+11 (secure 0 0 0) 0x7fe45409c710 con 0x7fe45c102250 2024-04-24T23:32:39.416 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 32 2024-04-24T23:32:39.418 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:39.417+0000 7fe46449e640 1 -- 172.21.15.80:0/668472154 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fe4480a2b90 msgr2=0x7fe4480a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:39.418 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:39.417+0000 7fe46449e640 1 --2- 172.21.15.80:0/668472154 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fe4480a2b90 0x7fe4480a5050 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fe45c1016f0 tx=0x7fe458008040 comp rx=0 tx=0).stop 2024-04-24T23:32:39.418 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:39.417+0000 7fe46449e640 1 -- 172.21.15.80:0/668472154 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe45c102250 msgr2=0x7fe45c19f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:39.418 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:39.417+0000 7fe46449e640 1 --2- 172.21.15.80:0/668472154 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe45c102250 0x7fe45c19f0e0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fe45400cce0 tx=0x7fe45400a620 comp rx=0 tx=0).stop 2024-04-24T23:32:39.418 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:39.417+0000 7fe46449e640 1 -- 172.21.15.80:0/668472154 shutdown_connections 2024-04-24T23:32:39.418 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:39.417+0000 7fe46449e640 1 --2- 172.21.15.80:0/668472154 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fe4480a2b90 0x7fe4480a5050 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:39.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:39.417+0000 7fe46449e640 1 --2- 172.21.15.80:0/668472154 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe45c102250 0x7fe45c19f0e0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:39.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:39.417+0000 7fe46449e640 1 --2- 172.21.15.80:0/668472154 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe45c101890 0x7fe45c1980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:39.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:39.417+0000 7fe46449e640 1 --2- 172.21.15.80:0/668472154 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe45c1006a0 0x7fe45c197b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:39.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:39.417+0000 7fe46449e640 1 -- 172.21.15.80:0/668472154 >> 172.21.15.80:0/668472154 conn(0x7fe45c0fbc30 msgr2=0x7fe45c069d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:32:39.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:39.417+0000 7fe46449e640 1 -- 172.21.15.80:0/668472154 shutdown_connections 2024-04-24T23:32:39.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:39.417+0000 7fe46449e640 1 -- 172.21.15.80:0/668472154 wait complete. 2024-04-24T23:32:39.428 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:32:39 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/2714246738' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:32:39.428 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:32:39 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/1264146903' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:32:39.428 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:32:39 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/2459247910' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T23:32:39.428 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:32:39 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/3923050808' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T23:32:39.430 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-24T23:32:39.446 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:32:39 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/2714246738' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:32:39.450 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:32:39 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/1264146903' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:32:39.450 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:32:39 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/2459247910' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T23:32:39.450 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:32:39 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/3923050808' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T23:32:39.532 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:39.528+0000 7faeb8f90640 1 -- 172.21.15.80:0/2250406654 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faeb4100680 msgr2=0x7faeb4100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:39.532 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:39.528+0000 7faeb8f90640 1 --2- 172.21.15.80:0/2250406654 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faeb4100680 0x7faeb4100aa0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7faeac009ca0 tx=0x7faeac02a8c0 comp rx=0 tx=0).stop 2024-04-24T23:32:39.532 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:39.529+0000 7faeb8f90640 1 -- 172.21.15.80:0/2250406654 shutdown_connections 2024-04-24T23:32:39.532 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:39.529+0000 7faeb8f90640 1 --2- 172.21.15.80:0/2250406654 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faeb4102230 0x7faeb4108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:39.532 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:39.529+0000 7faeb8f90640 1 --2- 172.21.15.80:0/2250406654 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faeb4101870 0x7faeb4101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:39.532 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:39.529+0000 7faeb8f90640 1 --2- 172.21.15.80:0/2250406654 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faeb4100680 0x7faeb4100aa0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:39.532 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:39.529+0000 7faeb8f90640 1 -- 172.21.15.80:0/2250406654 >> 172.21.15.80:0/2250406654 conn(0x7faeb40fbbf0 msgr2=0x7faeb40fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:32:39.533 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:39.529+0000 7faeb8f90640 1 -- 172.21.15.80:0/2250406654 shutdown_connections 2024-04-24T23:32:39.533 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:39.529+0000 7faeb8f90640 1 -- 172.21.15.80:0/2250406654 wait complete. 2024-04-24T23:32:39.533 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:39.529+0000 7faeb8f90640 1 Processor -- start 2024-04-24T23:32:39.533 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:39.529+0000 7faeb8f90640 1 -- start start 2024-04-24T23:32:39.533 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:39.529+0000 7faeb8f90640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faeb4100680 0x7faeb4197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:39.533 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:39.529+0000 7faeb8f90640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faeb4101870 0x7faeb4197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:39.533 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:39.529+0000 7faeb8f90640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faeb4102230 0x7faeb419f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:39.533 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:39.529+0000 7faeb8f90640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7faeb410d420 con 0x7faeb4101870 2024-04-24T23:32:39.534 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:39.529+0000 7faeb8f90640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7faeb410d2a0 con 0x7faeb4102230 2024-04-24T23:32:39.534 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:39.529+0000 7faeb8f90640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7faeb410d5a0 con 0x7faeb4100680 2024-04-24T23:32:39.534 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:39.530+0000 7faeb1d9b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faeb4101870 0x7faeb4197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:39.534 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:39.530+0000 7faeb1d9b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faeb4101870 0x7faeb4197fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:49216/0 (socket says 172.21.15.80:49216) 2024-04-24T23:32:39.534 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:39.530+0000 7faeb1d9b640 1 -- 172.21.15.80:0/3385034251 learned_addr learned my addr 172.21.15.80:0/3385034251 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:32:39.534 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:39.530+0000 7faeb259c640 1 --2- 172.21.15.80:0/3385034251 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faeb4100680 0x7faeb4197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:39.534 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:39.530+0000 7faeb2d9d640 1 --2- 172.21.15.80:0/3385034251 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faeb4102230 0x7faeb419f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:39.534 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:39.530+0000 7faeb1d9b640 1 -- 172.21.15.80:0/3385034251 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faeb4100680 msgr2=0x7faeb4197aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:39.535 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:39.530+0000 7faeb1d9b640 1 --2- 172.21.15.80:0/3385034251 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faeb4100680 0x7faeb4197aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:39.535 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:39.530+0000 7faeb1d9b640 1 -- 172.21.15.80:0/3385034251 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faeb4102230 msgr2=0x7faeb419f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:39.535 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:39.530+0000 7faeb1d9b640 1 --2- 172.21.15.80:0/3385034251 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faeb4102230 0x7faeb419f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:39.535 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:39.530+0000 7faeb1d9b640 1 -- 172.21.15.80:0/3385034251 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faeb419f6c0 con 0x7faeb4101870 2024-04-24T23:32:39.535 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:39.530+0000 7faeb259c640 1 --2- 172.21.15.80:0/3385034251 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faeb4100680 0x7faeb4197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:32:39.535 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:39.530+0000 7faeb1d9b640 1 --2- 172.21.15.80:0/3385034251 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faeb4101870 0x7faeb4197fe0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7faea800c970 tx=0x7faea800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:32:39.535 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:39.530+0000 7faea37fe640 1 -- 172.21.15.80:0/3385034251 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faea8007bb0 con 0x7faeb4101870 2024-04-24T23:32:39.535 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:39.530+0000 7faeb8f90640 1 -- 172.21.15.80:0/3385034251 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faeb419f9b0 con 0x7faeb4101870 2024-04-24T23:32:39.535 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:39.530+0000 7faea37fe640 1 -- 172.21.15.80:0/3385034251 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faea8007d50 con 0x7faeb4101870 2024-04-24T23:32:39.536 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:39.530+0000 7faea37fe640 1 -- 172.21.15.80:0/3385034251 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faea800f450 con 0x7faeb4101870 2024-04-24T23:32:39.536 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:39.530+0000 7faeb8f90640 1 -- 172.21.15.80:0/3385034251 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faeb419ff00 con 0x7faeb4101870 2024-04-24T23:32:39.536 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:39.531+0000 7faeb8f90640 1 -- 172.21.15.80:0/3385034251 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faeb41079a0 con 0x7faeb4101870 2024-04-24T23:32:39.537 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:39.532+0000 7faea37fe640 1 -- 172.21.15.80:0/3385034251 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7faea8005020 con 0x7faeb4101870 2024-04-24T23:32:39.537 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:39.533+0000 7faea37fe640 1 --2- 172.21.15.80:0/3385034251 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fae9c0a2b90 0x7fae9c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:39.537 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:39.533+0000 7faeb259c640 1 --2- 172.21.15.80:0/3385034251 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fae9c0a2b90 0x7fae9c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:39.537 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:39.533+0000 7faea37fe640 1 -- 172.21.15.80:0/3385034251 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7faea80cf2a0 con 0x7faeb4101870 2024-04-24T23:32:39.538 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:39.533+0000 7faeb259c640 1 --2- 172.21.15.80:0/3385034251 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fae9c0a2b90 0x7fae9c0a5050 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7faeac006010 tx=0x7faeac005fa0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:32:39.538 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:39.537+0000 7faea37fe640 1 -- 172.21.15.80:0/3385034251 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7faea8014030 con 0x7faeb4101870 2024-04-24T23:32:39.733 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:39.732+0000 7faeb8f90640 1 -- 172.21.15.80:0/3385034251 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7faeb4062340 con 0x7faeb4101870 2024-04-24T23:32:39.733 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:39.733+0000 7faea37fe640 1 -- 172.21.15.80:0/3385034251 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v94) ==== 81+0+586 (secure 0 0 0) 0x7faea809be30 con 0x7faeb4101870 2024-04-24T23:32:39.734 INFO:teuthology.orchestra.run.smithi080.stdout:172.21.15.80:0/2828378749 2024-04-25T23:20:06.793609+0000 2024-04-24T23:32:39.734 INFO:teuthology.orchestra.run.smithi080.stdout:172.21.15.80:6801/233364790 2024-04-25T23:20:06.793609+0000 2024-04-24T23:32:39.734 INFO:teuthology.orchestra.run.smithi080.stdout:172.21.15.80:6800/1614691410 2024-04-25T23:19:40.490692+0000 2024-04-24T23:32:39.734 INFO:teuthology.orchestra.run.smithi080.stdout:172.21.15.80:0/1406187145 2024-04-25T23:20:06.793609+0000 2024-04-24T23:32:39.734 INFO:teuthology.orchestra.run.smithi080.stdout:172.21.15.80:0/92953396 2024-04-25T23:19:40.490692+0000 2024-04-24T23:32:39.734 INFO:teuthology.orchestra.run.smithi080.stdout:172.21.15.80:0/598129432 2024-04-25T23:20:06.793609+0000 2024-04-24T23:32:39.735 INFO:teuthology.orchestra.run.smithi080.stdout:172.21.15.80:0/232193118 2024-04-25T23:19:40.490692+0000 2024-04-24T23:32:39.735 INFO:teuthology.orchestra.run.smithi080.stdout:172.21.15.80:6800/233364790 2024-04-25T23:20:06.793609+0000 2024-04-24T23:32:39.735 INFO:teuthology.orchestra.run.smithi080.stdout:172.21.15.80:6801/1614691410 2024-04-25T23:19:40.490692+0000 2024-04-24T23:32:39.735 INFO:teuthology.orchestra.run.smithi080.stdout:172.21.15.80:0/2081562529 2024-04-25T23:19:40.490692+0000 2024-04-24T23:32:39.736 INFO:teuthology.orchestra.run.smithi080.stderr:listed 10 entries 2024-04-24T23:32:39.736 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:39.735+0000 7faeb8f90640 1 -- 172.21.15.80:0/3385034251 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fae9c0a2b90 msgr2=0x7fae9c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:39.736 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:39.735+0000 7faeb8f90640 1 --2- 172.21.15.80:0/3385034251 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fae9c0a2b90 0x7fae9c0a5050 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7faeac006010 tx=0x7faeac005fa0 comp rx=0 tx=0).stop 2024-04-24T23:32:39.736 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:39.735+0000 7faeb8f90640 1 -- 172.21.15.80:0/3385034251 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faeb4101870 msgr2=0x7faeb4197fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:39.736 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:39.735+0000 7faeb8f90640 1 --2- 172.21.15.80:0/3385034251 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faeb4101870 0x7faeb4197fe0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7faea800c970 tx=0x7faea800ce30 comp rx=0 tx=0).stop 2024-04-24T23:32:39.737 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:39.735+0000 7faeb8f90640 1 -- 172.21.15.80:0/3385034251 shutdown_connections 2024-04-24T23:32:39.737 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:39.735+0000 7faeb8f90640 1 --2- 172.21.15.80:0/3385034251 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fae9c0a2b90 0x7fae9c0a5050 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:39.737 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:39.735+0000 7faeb8f90640 1 --2- 172.21.15.80:0/3385034251 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faeb4102230 0x7faeb419f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:39.737 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:39.735+0000 7faeb8f90640 1 --2- 172.21.15.80:0/3385034251 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faeb4101870 0x7faeb4197fe0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:39.737 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:39.735+0000 7faeb8f90640 1 --2- 172.21.15.80:0/3385034251 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faeb4100680 0x7faeb4197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:39.737 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:39.735+0000 7faeb8f90640 1 -- 172.21.15.80:0/3385034251 >> 172.21.15.80:0/3385034251 conn(0x7faeb40fbbf0 msgr2=0x7faeb4104b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:32:39.737 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:39.735+0000 7faeb8f90640 1 -- 172.21.15.80:0/3385034251 shutdown_connections 2024-04-24T23:32:39.737 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:39.735+0000 7faeb8f90640 1 -- 172.21.15.80:0/3385034251 wait complete. 2024-04-24T23:32:39.747 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-24T23:32:39.747 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:32:39.747 DEBUG:teuthology.orchestra.run.smithi080:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-24T23:32:39.839 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi080.front.sepia.ceph.com 2024-04-24T23:32:39.839 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:32:39.839 DEBUG:teuthology.orchestra.run.smithi080:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-24T23:32:39.865 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:32:39.865 DEBUG:teuthology.orchestra.run.smithi080:> ip netns list 2024-04-24T23:32:39.920 INFO:teuthology.orchestra.run.smithi080.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-24T23:32:39.920 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:32:39.921 DEBUG:teuthology.orchestra.run.smithi080:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T23:32:39.985 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:32:39.985 DEBUG:teuthology.orchestra.run.smithi080:> sudo ip link delete ceph-brx 2024-04-24T23:32:40.079 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-04-24T23:32:40.090 INFO:tasks.cephadm:Teardown begin 2024-04-24T23:32:40.090 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-24T23:32:40.091 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:32:40.192 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.189+0000 7fe260293640 1 -- 172.21.15.80:0/1183036294 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe258103340 msgr2=0x7fe258109bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:40.192 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.189+0000 7fe260293640 1 --2- 172.21.15.80:0/1183036294 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe258103340 0x7fe258109bd0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fe254009900 tx=0x7fe25402a3f0 comp rx=0 tx=0).stop 2024-04-24T23:32:40.192 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.190+0000 7fe260293640 1 -- 172.21.15.80:0/1183036294 shutdown_connections 2024-04-24T23:32:40.193 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.190+0000 7fe260293640 1 --2- 172.21.15.80:0/1183036294 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe258103340 0x7fe258109bd0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:40.193 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.190+0000 7fe260293640 1 --2- 172.21.15.80:0/1183036294 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe258102980 0x7fe258102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:40.193 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.190+0000 7fe260293640 1 --2- 172.21.15.80:0/1183036294 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe258101790 0x7fe258101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:40.193 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.190+0000 7fe260293640 1 -- 172.21.15.80:0/1183036294 >> 172.21.15.80:0/1183036294 conn(0x7fe2580fcd20 msgr2=0x7fe2580ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:32:40.193 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.190+0000 7fe260293640 1 -- 172.21.15.80:0/1183036294 shutdown_connections 2024-04-24T23:32:40.193 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.190+0000 7fe260293640 1 -- 172.21.15.80:0/1183036294 wait complete. 2024-04-24T23:32:40.193 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.190+0000 7fe260293640 1 Processor -- start 2024-04-24T23:32:40.193 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.190+0000 7fe260293640 1 -- start start 2024-04-24T23:32:40.194 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.190+0000 7fe260293640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe258101790 0x7fe258197a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:40.194 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.190+0000 7fe260293640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe258102980 0x7fe258197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:40.194 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.190+0000 7fe260293640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe258103340 0x7fe25819efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:40.194 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.190+0000 7fe260293640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fe25810d3f0 con 0x7fe258102980 2024-04-24T23:32:40.194 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.190+0000 7fe260293640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe25810d270 con 0x7fe258101790 2024-04-24T23:32:40.194 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.190+0000 7fe260293640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe25810d570 con 0x7fe258103340 2024-04-24T23:32:40.194 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.190+0000 7fe25d82e640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe258102980 0x7fe258197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:40.194 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.190+0000 7fe25d82e640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe258102980 0x7fe258197f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:49246/0 (socket says 172.21.15.80:49246) 2024-04-24T23:32:40.195 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.190+0000 7fe25d82e640 1 -- 172.21.15.80:0/3307629127 learned_addr learned my addr 172.21.15.80:0/3307629127 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:32:40.195 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.191+0000 7fe25e02f640 1 --2- 172.21.15.80:0/3307629127 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe258101790 0x7fe258197a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:40.195 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.191+0000 7fe25e830640 1 --2- 172.21.15.80:0/3307629127 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe258103340 0x7fe25819efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:40.195 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.191+0000 7fe25d82e640 1 -- 172.21.15.80:0/3307629127 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe258103340 msgr2=0x7fe25819efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:40.195 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.191+0000 7fe25d82e640 1 --2- 172.21.15.80:0/3307629127 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe258103340 0x7fe25819efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:40.195 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.191+0000 7fe25d82e640 1 -- 172.21.15.80:0/3307629127 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe258101790 msgr2=0x7fe258197a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:40.195 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.191+0000 7fe25d82e640 1 --2- 172.21.15.80:0/3307629127 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe258101790 0x7fe258197a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:40.195 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.191+0000 7fe25d82e640 1 -- 172.21.15.80:0/3307629127 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe25819f630 con 0x7fe258102980 2024-04-24T23:32:40.195 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.191+0000 7fe25d82e640 1 --2- 172.21.15.80:0/3307629127 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe258102980 0x7fe258197f50 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fe24800c920 tx=0x7fe24800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:32:40.196 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.191+0000 7fe24f7fe640 1 -- 172.21.15.80:0/3307629127 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe248007c70 con 0x7fe258102980 2024-04-24T23:32:40.196 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.191+0000 7fe260293640 1 -- 172.21.15.80:0/3307629127 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe25819f920 con 0x7fe258102980 2024-04-24T23:32:40.196 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.191+0000 7fe24f7fe640 1 -- 172.21.15.80:0/3307629127 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe248007e10 con 0x7fe258102980 2024-04-24T23:32:40.196 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.191+0000 7fe24f7fe640 1 -- 172.21.15.80:0/3307629127 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe248018890 con 0x7fe258102980 2024-04-24T23:32:40.196 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.191+0000 7fe260293640 1 -- 172.21.15.80:0/3307629127 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe25819fe70 con 0x7fe258102980 2024-04-24T23:32:40.197 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.192+0000 7fe260293640 1 -- 172.21.15.80:0/3307629127 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe258108ab0 con 0x7fe258102980 2024-04-24T23:32:40.197 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.193+0000 7fe24f7fe640 1 -- 172.21.15.80:0/3307629127 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe248018a70 con 0x7fe258102980 2024-04-24T23:32:40.197 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.194+0000 7fe24f7fe640 1 --2- 172.21.15.80:0/3307629127 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fe2440a2b90 0x7fe2440a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:40.198 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.194+0000 7fe25e02f640 1 --2- 172.21.15.80:0/3307629127 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fe2440a2b90 0x7fe2440a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:40.198 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.194+0000 7fe24f7fe640 1 -- 172.21.15.80:0/3307629127 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fe2480cf510 con 0x7fe258102980 2024-04-24T23:32:40.198 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.194+0000 7fe25e02f640 1 --2- 172.21.15.80:0/3307629127 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fe2440a2b90 0x7fe2440a5050 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fe2581027e0 tx=0x7fe250009720 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:32:40.198 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.198+0000 7fe24f7fe640 1 -- 172.21.15.80:0/3307629127 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fe248014030 con 0x7fe258102980 2024-04-24T23:32:40.410 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.409+0000 7fe260293640 1 -- 172.21.15.80:0/3307629127 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe258062340 con 0x7fe258102980 2024-04-24T23:32:40.441 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.410+0000 7fe24f7fe640 1 -- 172.21.15.80:0/3307629127 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11875 (secure 0 0 0) 0x7fe24809c0a0 con 0x7fe258102980 2024-04-24T23:32:40.441 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-24T23:32:40.441 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/891591085","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":891591085},{"type":"v1","addr":"172.21.15.80:6835","nonce":891591085}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6837/259529026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6836","nonce":259529026},{"type":"v1","addr":"172.21.15.80:6837","nonce":259529026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14646,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6839/4194284854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6838","nonce":4194284854},{"type":"v1","addr":"172.21.15.80:6839","nonce":4194284854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14658,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6841/3145730958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6840","nonce":3145730958},{"type":"v1","addr":"172.21.15.80:6841","nonce":3145730958}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24395,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/3736784385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3736784385},{"type":"v1","addr":"172.21.15.134:6833","nonce":3736784385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/124777462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":124777462},{"type":"v1","addr":"172.21.15.148:6833","nonce":124777462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T23:27:19.300566+0000","modified":"2024-04-24T23:31:19.604599+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24496,"mds_1":24488,"mds_2":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/1399801120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1399801120},{"type":"v1","addr":"172.21.15.134:6835","nonce":1399801120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/430512704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":430512704},{"type":"v1","addr":"172.21.15.134:6837","nonce":430512704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/3002319310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3002319310},{"type":"v1","addr":"172.21.15.148:6835","nonce":3002319310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.148:6837/856528403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":856528403},{"type":"v1","addr":"172.21.15.148:6837","nonce":856528403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6839/1743815021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":1743815021},{"type":"v1","addr":"172.21.15.134:6839","nonce":1743815021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.148:6839/2952757594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":2952757594},{"type":"v1","addr":"172.21.15.148:6839","nonce":2952757594}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24496,"qdb_cluster":[24472,24488,24496]},"id":1}]} 2024-04-24T23:32:40.442 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 27 2024-04-24T23:32:40.443 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.412+0000 7fe260293640 1 -- 172.21.15.80:0/3307629127 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fe2440a2b90 msgr2=0x7fe2440a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:40.443 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.412+0000 7fe260293640 1 --2- 172.21.15.80:0/3307629127 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fe2440a2b90 0x7fe2440a5050 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fe2581027e0 tx=0x7fe250009720 comp rx=0 tx=0).stop 2024-04-24T23:32:40.443 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.412+0000 7fe260293640 1 -- 172.21.15.80:0/3307629127 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe258102980 msgr2=0x7fe258197f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:40.443 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.412+0000 7fe260293640 1 --2- 172.21.15.80:0/3307629127 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe258102980 0x7fe258197f50 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fe24800c920 tx=0x7fe24800cde0 comp rx=0 tx=0).stop 2024-04-24T23:32:40.443 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.412+0000 7fe260293640 1 -- 172.21.15.80:0/3307629127 shutdown_connections 2024-04-24T23:32:40.443 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.412+0000 7fe260293640 1 --2- 172.21.15.80:0/3307629127 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fe2440a2b90 0x7fe2440a5050 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:40.443 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.412+0000 7fe260293640 1 --2- 172.21.15.80:0/3307629127 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe258103340 0x7fe25819efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:40.443 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.412+0000 7fe260293640 1 --2- 172.21.15.80:0/3307629127 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe258102980 0x7fe258197f50 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:40.444 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.412+0000 7fe260293640 1 --2- 172.21.15.80:0/3307629127 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe258101790 0x7fe258197a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:40.444 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.412+0000 7fe260293640 1 -- 172.21.15.80:0/3307629127 >> 172.21.15.80:0/3307629127 conn(0x7fe2580fcd20 msgr2=0x7fe258105c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:32:40.444 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.412+0000 7fe260293640 1 -- 172.21.15.80:0/3307629127 shutdown_connections 2024-04-24T23:32:40.444 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.412+0000 7fe260293640 1 -- 172.21.15.80:0/3307629127 wait complete. 2024-04-24T23:32:40.444 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:32:40 smithi080 ceph-mon[29152]: pgmap v460: 129 pgs: 129 active+clean; 75 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 64 KiB/s wr, 8 op/s 2024-04-24T23:32:40.444 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:32:40 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/668472154' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T23:32:40.444 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:32:40 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/3385034251' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-24T23:32:40.445 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:32:40.446 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:32:40 smithi134 ceph-mon[32619]: pgmap v460: 129 pgs: 129 active+clean; 75 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 64 KiB/s wr, 8 op/s 2024-04-24T23:32:40.446 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:32:40 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/668472154' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T23:32:40.446 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:32:40 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/3385034251' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-24T23:32:40.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:32:40 smithi148 ceph-mon[32528]: pgmap v460: 129 pgs: 129 active+clean; 75 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 64 KiB/s wr, 8 op/s 2024-04-24T23:32:40.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:32:40 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/668472154' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T23:32:40.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:32:40 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/3385034251' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-24T23:32:40.546 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.543+0000 7f5cca989640 1 -- 172.21.15.80:0/4973435 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5cc4100690 msgr2=0x7f5cc4100ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:40.547 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.543+0000 7f5cca989640 1 --2- 172.21.15.80:0/4973435 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5cc4100690 0x7f5cc4100ab0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f5cb4009900 tx=0x7f5cb402a410 comp rx=0 tx=0).stop 2024-04-24T23:32:40.547 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.543+0000 7f5cca989640 1 -- 172.21.15.80:0/4973435 shutdown_connections 2024-04-24T23:32:40.547 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.543+0000 7f5cca989640 1 --2- 172.21.15.80:0/4973435 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5cc4102240 0x7f5cc4108ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:40.547 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.543+0000 7f5cca989640 1 --2- 172.21.15.80:0/4973435 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5cc4101880 0x7f5cc4101d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:40.547 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.543+0000 7f5cca989640 1 --2- 172.21.15.80:0/4973435 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5cc4100690 0x7f5cc4100ab0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:40.547 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.543+0000 7f5cca989640 1 -- 172.21.15.80:0/4973435 >> 172.21.15.80:0/4973435 conn(0x7f5cc40fbc20 msgr2=0x7f5cc40fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:32:40.547 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.543+0000 7f5cca989640 1 -- 172.21.15.80:0/4973435 shutdown_connections 2024-04-24T23:32:40.547 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.543+0000 7f5cca989640 1 -- 172.21.15.80:0/4973435 wait complete. 2024-04-24T23:32:40.548 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.544+0000 7f5cca989640 1 Processor -- start 2024-04-24T23:32:40.548 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.544+0000 7f5cca989640 1 -- start start 2024-04-24T23:32:40.548 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.544+0000 7f5cca989640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5cc4100690 0x7f5cc410b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:40.548 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.544+0000 7f5cca989640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5cc4101880 0x7f5cc4109650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:40.548 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.544+0000 7f5cca989640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5cc4102240 0x7f5cc4109b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:40.548 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.544+0000 7f5cca989640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc410d430 con 0x7f5cc4100690 2024-04-24T23:32:40.548 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.544+0000 7f5cca989640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc410d2b0 con 0x7f5cc4101880 2024-04-24T23:32:40.548 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.544+0000 7f5cca989640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc410d5b0 con 0x7f5cc4102240 2024-04-24T23:32:40.549 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.544+0000 7f5cc3fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5cc4100690 0x7f5cc410b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:40.549 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.544+0000 7f5cc3fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5cc4100690 0x7f5cc410b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:49256/0 (socket says 172.21.15.80:49256) 2024-04-24T23:32:40.549 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.544+0000 7f5cc3fff640 1 -- 172.21.15.80:0/2576949753 learned_addr learned my addr 172.21.15.80:0/2576949753 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:32:40.549 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.544+0000 7f5cc37fe640 1 --2- 172.21.15.80:0/2576949753 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5cc4101880 0x7f5cc4109650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:40.549 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.544+0000 7f5cc8f26640 1 --2- 172.21.15.80:0/2576949753 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5cc4102240 0x7f5cc4109b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:40.550 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.545+0000 7f5cc3fff640 1 -- 172.21.15.80:0/2576949753 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5cc4102240 msgr2=0x7f5cc4109b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:40.550 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.545+0000 7f5cc3fff640 1 --2- 172.21.15.80:0/2576949753 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5cc4102240 0x7f5cc4109b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:40.550 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.545+0000 7f5cc3fff640 1 -- 172.21.15.80:0/2576949753 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5cc4101880 msgr2=0x7f5cc4109650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:40.550 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.545+0000 7f5cc3fff640 1 --2- 172.21.15.80:0/2576949753 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5cc4101880 0x7f5cc4109650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:40.550 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.545+0000 7f5cc3fff640 1 -- 172.21.15.80:0/2576949753 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cc410a420 con 0x7f5cc4100690 2024-04-24T23:32:40.550 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.545+0000 7f5cc37fe640 1 --2- 172.21.15.80:0/2576949753 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5cc4101880 0x7f5cc4109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:32:40.550 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.545+0000 7f5cc3fff640 1 --2- 172.21.15.80:0/2576949753 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5cc4100690 0x7f5cc410b010 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f5cb4031610 tx=0x7f5cb4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:32:40.550 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.545+0000 7f5cc17fa640 1 -- 172.21.15.80:0/2576949753 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb403f030 con 0x7f5cc4100690 2024-04-24T23:32:40.551 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.545+0000 7f5cc17fa640 1 -- 172.21.15.80:0/2576949753 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5cb4003ad0 con 0x7f5cc4100690 2024-04-24T23:32:40.551 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.545+0000 7f5cca989640 1 -- 172.21.15.80:0/2576949753 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cc41a1b50 con 0x7f5cc4100690 2024-04-24T23:32:40.551 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.545+0000 7f5cc17fa640 1 -- 172.21.15.80:0/2576949753 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb4035070 con 0x7f5cc4100690 2024-04-24T23:32:40.551 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.545+0000 7f5cca989640 1 -- 172.21.15.80:0/2576949753 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cc41a20a0 con 0x7f5cc4100690 2024-04-24T23:32:40.551 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.546+0000 7f5cca989640 1 -- 172.21.15.80:0/2576949753 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cc4107a30 con 0x7f5cc4100690 2024-04-24T23:32:40.552 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.547+0000 7f5cc17fa640 1 -- 172.21.15.80:0/2576949753 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5cb4042030 con 0x7f5cc4100690 2024-04-24T23:32:40.552 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.547+0000 7f5cc17fa640 1 --2- 172.21.15.80:0/2576949753 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f5cac0a2b90 0x7f5cac0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:40.553 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.548+0000 7f5cc17fa640 1 -- 172.21.15.80:0/2576949753 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f5cb40ef9a0 con 0x7f5cc4100690 2024-04-24T23:32:40.553 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.548+0000 7f5cc37fe640 1 --2- 172.21.15.80:0/2576949753 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f5cac0a2b90 0x7f5cac0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:40.553 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.548+0000 7f5cc37fe640 1 --2- 172.21.15.80:0/2576949753 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f5cac0a2b90 0x7f5cac0a5050 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f5cc410ad80 tx=0x7f5cb8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:32:40.553 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.552+0000 7f5cc17fa640 1 -- 172.21.15.80:0/2576949753 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f5cb40bbee0 con 0x7f5cc4100690 2024-04-24T23:32:40.763 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.762+0000 7f5cca989640 1 -- 172.21.15.80:0/2576949753 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5cc402d390 con 0x7f5cc4100690 2024-04-24T23:32:40.763 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.763+0000 7f5cc17fa640 1 -- 172.21.15.80:0/2576949753 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11875 (secure 0 0 0) 0x7f5cb40c01e0 con 0x7f5cc4100690 2024-04-24T23:32:40.764 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-24T23:32:40.764 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/891591085","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":891591085},{"type":"v1","addr":"172.21.15.80:6835","nonce":891591085}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6837/259529026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6836","nonce":259529026},{"type":"v1","addr":"172.21.15.80:6837","nonce":259529026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14646,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6839/4194284854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6838","nonce":4194284854},{"type":"v1","addr":"172.21.15.80:6839","nonce":4194284854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14658,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6841/3145730958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6840","nonce":3145730958},{"type":"v1","addr":"172.21.15.80:6841","nonce":3145730958}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24395,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/3736784385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3736784385},{"type":"v1","addr":"172.21.15.134:6833","nonce":3736784385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/124777462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":124777462},{"type":"v1","addr":"172.21.15.148:6833","nonce":124777462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T23:27:19.300566+0000","modified":"2024-04-24T23:31:19.604599+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24496,"mds_1":24488,"mds_2":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/1399801120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1399801120},{"type":"v1","addr":"172.21.15.134:6835","nonce":1399801120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/430512704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":430512704},{"type":"v1","addr":"172.21.15.134:6837","nonce":430512704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/3002319310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3002319310},{"type":"v1","addr":"172.21.15.148:6835","nonce":3002319310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.148:6837/856528403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":856528403},{"type":"v1","addr":"172.21.15.148:6837","nonce":856528403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6839/1743815021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":1743815021},{"type":"v1","addr":"172.21.15.134:6839","nonce":1743815021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.148:6839/2952757594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":2952757594},{"type":"v1","addr":"172.21.15.148:6839","nonce":2952757594}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24496,"qdb_cluster":[24472,24488,24496]},"id":1}]} 2024-04-24T23:32:40.764 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 27 2024-04-24T23:32:40.766 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.765+0000 7f5cca989640 1 -- 172.21.15.80:0/2576949753 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f5cac0a2b90 msgr2=0x7f5cac0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:40.766 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.765+0000 7f5cca989640 1 --2- 172.21.15.80:0/2576949753 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f5cac0a2b90 0x7f5cac0a5050 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f5cc410ad80 tx=0x7f5cb8005f90 comp rx=0 tx=0).stop 2024-04-24T23:32:40.766 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.765+0000 7f5cca989640 1 -- 172.21.15.80:0/2576949753 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5cc4100690 msgr2=0x7f5cc410b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:40.766 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.765+0000 7f5cca989640 1 --2- 172.21.15.80:0/2576949753 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5cc4100690 0x7f5cc410b010 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f5cb4031610 tx=0x7f5cb4004270 comp rx=0 tx=0).stop 2024-04-24T23:32:40.766 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.765+0000 7f5cca989640 1 -- 172.21.15.80:0/2576949753 shutdown_connections 2024-04-24T23:32:40.767 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.765+0000 7f5cca989640 1 --2- 172.21.15.80:0/2576949753 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f5cac0a2b90 0x7f5cac0a5050 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:40.767 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.765+0000 7f5cca989640 1 --2- 172.21.15.80:0/2576949753 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5cc4102240 0x7f5cc4109b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:40.767 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.765+0000 7f5cca989640 1 --2- 172.21.15.80:0/2576949753 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5cc4101880 0x7f5cc4109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:40.767 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.765+0000 7f5cca989640 1 --2- 172.21.15.80:0/2576949753 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5cc4100690 0x7f5cc410b010 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:40.767 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.765+0000 7f5cca989640 1 -- 172.21.15.80:0/2576949753 >> 172.21.15.80:0/2576949753 conn(0x7f5cc40fbc20 msgr2=0x7f5cc4104c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:32:40.767 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.765+0000 7f5cca989640 1 -- 172.21.15.80:0/2576949753 shutdown_connections 2024-04-24T23:32:40.767 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.765+0000 7f5cca989640 1 -- 172.21.15.80:0/2576949753 wait complete. 2024-04-24T23:32:40.777 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:32:40.878 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.873+0000 7f85adea5640 1 -- 172.21.15.80:0/4131103753 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f85a8101880 msgr2=0x7f85a8101d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:40.879 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.873+0000 7f85adea5640 1 --2- 172.21.15.80:0/4131103753 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f85a8101880 0x7f85a8101d00 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f859c009900 tx=0x7f859c02a3f0 comp rx=0 tx=0).stop 2024-04-24T23:32:40.879 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.873+0000 7f85adea5640 1 -- 172.21.15.80:0/4131103753 shutdown_connections 2024-04-24T23:32:40.879 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.873+0000 7f85adea5640 1 --2- 172.21.15.80:0/4131103753 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85a8102240 0x7f85a8108ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:40.879 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.873+0000 7f85adea5640 1 --2- 172.21.15.80:0/4131103753 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f85a8101880 0x7f85a8101d00 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:40.879 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.873+0000 7f85adea5640 1 --2- 172.21.15.80:0/4131103753 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f85a8100690 0x7f85a8100ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:40.879 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.873+0000 7f85adea5640 1 -- 172.21.15.80:0/4131103753 >> 172.21.15.80:0/4131103753 conn(0x7f85a80fbc40 msgr2=0x7f85a80fe060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:32:40.880 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.873+0000 7f85adea5640 1 -- 172.21.15.80:0/4131103753 shutdown_connections 2024-04-24T23:32:40.880 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.873+0000 7f85adea5640 1 -- 172.21.15.80:0/4131103753 wait complete. 2024-04-24T23:32:40.880 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.874+0000 7f85adea5640 1 Processor -- start 2024-04-24T23:32:40.880 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.874+0000 7f85adea5640 1 -- start start 2024-04-24T23:32:40.880 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.874+0000 7f85adea5640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f85a8100690 0x7f85a8197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:40.880 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.874+0000 7f85adea5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f85a8101880 0x7f85a8197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:40.880 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.874+0000 7f85adea5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85a8102240 0x7f85a819f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:40.880 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.874+0000 7f85adea5640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f85a810d430 con 0x7f85a8100690 2024-04-24T23:32:40.880 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.874+0000 7f85adea5640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f85a810d2b0 con 0x7f85a8102240 2024-04-24T23:32:40.881 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.874+0000 7f85adea5640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f85a810d5b0 con 0x7f85a8101880 2024-04-24T23:32:40.881 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.874+0000 7f85a77fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f85a8100690 0x7f85a8197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:40.881 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.874+0000 7f85a77fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f85a8100690 0x7f85a8197a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:49272/0 (socket says 172.21.15.80:49272) 2024-04-24T23:32:40.881 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.874+0000 7f85a77fe640 1 -- 172.21.15.80:0/85598703 learned_addr learned my addr 172.21.15.80:0/85598703 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:32:40.881 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.874+0000 7f85a6ffd640 1 --2- 172.21.15.80:0/85598703 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f85a8101880 0x7f85a8197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:40.881 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.874+0000 7f85a7fff640 1 --2- 172.21.15.80:0/85598703 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85a8102240 0x7f85a819f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:40.881 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.874+0000 7f85a77fe640 1 -- 172.21.15.80:0/85598703 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f85a8101880 msgr2=0x7f85a8197fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:40.881 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.874+0000 7f85a77fe640 1 --2- 172.21.15.80:0/85598703 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f85a8101880 0x7f85a8197fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:40.882 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.874+0000 7f85a77fe640 1 -- 172.21.15.80:0/85598703 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85a8102240 msgr2=0x7f85a819f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:40.882 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.874+0000 7f85a77fe640 1 --2- 172.21.15.80:0/85598703 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85a8102240 0x7f85a819f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:40.882 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.874+0000 7f85a77fe640 1 -- 172.21.15.80:0/85598703 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85a819f690 con 0x7f85a8100690 2024-04-24T23:32:40.882 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.874+0000 7f85a7fff640 1 --2- 172.21.15.80:0/85598703 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85a8102240 0x7f85a819f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:32:40.882 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.875+0000 7f85a77fe640 1 --2- 172.21.15.80:0/85598703 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f85a8100690 0x7f85a8197a70 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f859800c970 tx=0x7f859800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:32:40.882 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.875+0000 7f85a4ff9640 1 -- 172.21.15.80:0/85598703 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8598004220 con 0x7f85a8100690 2024-04-24T23:32:40.882 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.875+0000 7f85adea5640 1 -- 172.21.15.80:0/85598703 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85a819f980 con 0x7f85a8100690 2024-04-24T23:32:40.883 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.875+0000 7f85a4ff9640 1 -- 172.21.15.80:0/85598703 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85980043c0 con 0x7f85a8100690 2024-04-24T23:32:40.883 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.875+0000 7f85a4ff9640 1 -- 172.21.15.80:0/85598703 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8598004a00 con 0x7f85a8100690 2024-04-24T23:32:40.883 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.875+0000 7f85adea5640 1 -- 172.21.15.80:0/85598703 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85a819fe50 con 0x7f85a8100690 2024-04-24T23:32:40.883 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.876+0000 7f85adea5640 1 -- 172.21.15.80:0/85598703 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85a8107950 con 0x7f85a8100690 2024-04-24T23:32:40.883 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.877+0000 7f85a4ff9640 1 -- 172.21.15.80:0/85598703 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f85980079b0 con 0x7f85a8100690 2024-04-24T23:32:40.883 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.877+0000 7f85a4ff9640 1 --2- 172.21.15.80:0/85598703 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f85900a2b90 0x7f85900a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:40.883 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.877+0000 7f85a6ffd640 1 --2- 172.21.15.80:0/85598703 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f85900a2b90 0x7f85900a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:40.884 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.877+0000 7f85a4ff9640 1 -- 172.21.15.80:0/85598703 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f85980cf260 con 0x7f85a8100690 2024-04-24T23:32:40.884 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.878+0000 7f85a6ffd640 1 --2- 172.21.15.80:0/85598703 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f85900a2b90 0x7f85900a5050 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f859c000c00 tx=0x7f859c031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:32:40.884 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:40.882+0000 7f85a4ff9640 1 -- 172.21.15.80:0/85598703 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f8598014030 con 0x7f85a8100690 2024-04-24T23:32:41.062 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:41.061+0000 7f85adea5640 1 -- 172.21.15.80:0/85598703 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f85a8062370 con 0x7f85a8100690 2024-04-24T23:32:41.063 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:41.063+0000 7f85a4ff9640 1 -- 172.21.15.80:0/85598703 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24839 (secure 0 0 0) 0x7f859809bdf0 con 0x7f85a8100690 2024-04-24T23:32:41.064 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-24T23:32:41.064 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":94,"fsid":"ec7885aa-0290-11ef-bc93-c7b262605968","created":"2024-04-24T23:19:05.586407+0000","modified":"2024-04-24T23:29:08.209684+0000","last_up_change":"2024-04-24T23:26:25.938316+0000","last_in_change":"2024-04-24T23:26:04.786699+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:22:33.124428+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:27:17.575788+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:27:18.492762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T23:27:22.438503+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a525741d-b46c-4eef-aa64-3f057cac1497","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":2597718112},{"type":"v1","addr":"172.21.15.80:6803","nonce":2597718112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":2597718112},{"type":"v1","addr":"172.21.15.80:6805","nonce":2597718112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2597718112},{"type":"v1","addr":"172.21.15.80:6809","nonce":2597718112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":2597718112},{"type":"v1","addr":"172.21.15.80:6807","nonce":2597718112}]},"public_addr":"172.21.15.80:6803/2597718112","cluster_addr":"172.21.15.80:6805/2597718112","heartbeat_back_addr":"172.21.15.80:6809/2597718112","heartbeat_front_addr":"172.21.15.80:6807/2597718112","state":["exists","up"]},{"osd":1,"uuid":"2d0f0d67-6db4-40ac-91cc-58604ab6795a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":2395610201},{"type":"v1","addr":"172.21.15.134:6801","nonce":2395610201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":2395610201},{"type":"v1","addr":"172.21.15.134:6803","nonce":2395610201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":2395610201},{"type":"v1","addr":"172.21.15.134:6807","nonce":2395610201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":2395610201},{"type":"v1","addr":"172.21.15.134:6805","nonce":2395610201}]},"public_addr":"172.21.15.134:6801/2395610201","cluster_addr":"172.21.15.134:6803/2395610201","heartbeat_back_addr":"172.21.15.134:6807/2395610201","heartbeat_front_addr":"172.21.15.134:6805/2395610201","state":["exists","up"]},{"osd":2,"uuid":"781fb879-fedc-4c84-82fb-59e073bc4cf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1629152187},{"type":"v1","addr":"172.21.15.148:6801","nonce":1629152187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1629152187},{"type":"v1","addr":"172.21.15.148:6803","nonce":1629152187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1629152187},{"type":"v1","addr":"172.21.15.148:6807","nonce":1629152187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1629152187},{"type":"v1","addr":"172.21.15.148:6805","nonce":1629152187}]},"public_addr":"172.21.15.148:6801/1629152187","cluster_addr":"172.21.15.148:6803/1629152187","heartbeat_back_addr":"172.21.15.148:6807/1629152187","heartbeat_front_addr":"172.21.15.148:6805/1629152187","state":["exists","up"]},{"osd":3,"uuid":"cd982757-f2f6-4296-b3e7-4cd40fbd8d9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":3128298510},{"type":"v1","addr":"172.21.15.80:6811","nonce":3128298510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":3128298510},{"type":"v1","addr":"172.21.15.80:6813","nonce":3128298510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":3128298510},{"type":"v1","addr":"172.21.15.80:6817","nonce":3128298510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":3128298510},{"type":"v1","addr":"172.21.15.80:6815","nonce":3128298510}]},"public_addr":"172.21.15.80:6811/3128298510","cluster_addr":"172.21.15.80:6813/3128298510","heartbeat_back_addr":"172.21.15.80:6817/3128298510","heartbeat_front_addr":"172.21.15.80:6815/3128298510","state":["exists","up"]},{"osd":4,"uuid":"dfc55a79-a7e8-417a-b898-ec44171a54af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1917556060},{"type":"v1","addr":"172.21.15.134:6809","nonce":1917556060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1917556060},{"type":"v1","addr":"172.21.15.134:6811","nonce":1917556060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1917556060},{"type":"v1","addr":"172.21.15.134:6815","nonce":1917556060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1917556060},{"type":"v1","addr":"172.21.15.134:6813","nonce":1917556060}]},"public_addr":"172.21.15.134:6809/1917556060","cluster_addr":"172.21.15.134:6811/1917556060","heartbeat_back_addr":"172.21.15.134:6815/1917556060","heartbeat_front_addr":"172.21.15.134:6813/1917556060","state":["exists","up"]},{"osd":5,"uuid":"b5491337-0694-440c-b679-312f3d14cf65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":2715951281},{"type":"v1","addr":"172.21.15.148:6809","nonce":2715951281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":2715951281},{"type":"v1","addr":"172.21.15.148:6811","nonce":2715951281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":2715951281},{"type":"v1","addr":"172.21.15.148:6815","nonce":2715951281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":2715951281},{"type":"v1","addr":"172.21.15.148:6813","nonce":2715951281}]},"public_addr":"172.21.15.148:6809/2715951281","cluster_addr":"172.21.15.148:6811/2715951281","heartbeat_back_addr":"172.21.15.148:6815/2715951281","heartbeat_front_addr":"172.21.15.148:6813/2715951281","state":["exists","up"]},{"osd":6,"uuid":"ba4b5df7-d86e-47cd-ab67-cf763c4e4d6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":953261724},{"type":"v1","addr":"172.21.15.80:6819","nonce":953261724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":953261724},{"type":"v1","addr":"172.21.15.80:6821","nonce":953261724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":953261724},{"type":"v1","addr":"172.21.15.80:6825","nonce":953261724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":953261724},{"type":"v1","addr":"172.21.15.80:6823","nonce":953261724}]},"public_addr":"172.21.15.80:6819/953261724","cluster_addr":"172.21.15.80:6821/953261724","heartbeat_back_addr":"172.21.15.80:6825/953261724","heartbeat_front_addr":"172.21.15.80:6823/953261724","state":["exists","up"]},{"osd":7,"uuid":"464a503a-75ae-49e7-ab6e-a7f9f2a6a307","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":108354844},{"type":"v1","addr":"172.21.15.134:6817","nonce":108354844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":108354844},{"type":"v1","addr":"172.21.15.134:6819","nonce":108354844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":108354844},{"type":"v1","addr":"172.21.15.134:6823","nonce":108354844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":108354844},{"type":"v1","addr":"172.21.15.134:6821","nonce":108354844}]},"public_addr":"172.21.15.134:6817/108354844","cluster_addr":"172.21.15.134:6819/108354844","heartbeat_back_addr":"172.21.15.134:6823/108354844","heartbeat_front_addr":"172.21.15.134:6821/108354844","state":["exists","up"]},{"osd":8,"uuid":"ce544b2e-4540-47c0-9997-500c45351330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3559655329},{"type":"v1","addr":"172.21.15.148:6817","nonce":3559655329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":3559655329},{"type":"v1","addr":"172.21.15.148:6819","nonce":3559655329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":3559655329},{"type":"v1","addr":"172.21.15.148:6823","nonce":3559655329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3559655329},{"type":"v1","addr":"172.21.15.148:6821","nonce":3559655329}]},"public_addr":"172.21.15.148:6817/3559655329","cluster_addr":"172.21.15.148:6819/3559655329","heartbeat_back_addr":"172.21.15.148:6823/3559655329","heartbeat_front_addr":"172.21.15.148:6821/3559655329","state":["exists","up"]},{"osd":9,"uuid":"06de3732-19b9-496c-a464-55322efa6759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":4019913545},{"type":"v1","addr":"172.21.15.80:6827","nonce":4019913545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":4019913545},{"type":"v1","addr":"172.21.15.80:6829","nonce":4019913545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":4019913545},{"type":"v1","addr":"172.21.15.80:6833","nonce":4019913545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":4019913545},{"type":"v1","addr":"172.21.15.80:6831","nonce":4019913545}]},"public_addr":"172.21.15.80:6827/4019913545","cluster_addr":"172.21.15.80:6829/4019913545","heartbeat_back_addr":"172.21.15.80:6833/4019913545","heartbeat_front_addr":"172.21.15.80:6831/4019913545","state":["exists","up"]},{"osd":10,"uuid":"8d92f0d9-5c64-429a-b750-c33ae0e070c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":647343766},{"type":"v1","addr":"172.21.15.134:6825","nonce":647343766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":647343766},{"type":"v1","addr":"172.21.15.134:6827","nonce":647343766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":647343766},{"type":"v1","addr":"172.21.15.134:6831","nonce":647343766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":647343766},{"type":"v1","addr":"172.21.15.134:6829","nonce":647343766}]},"public_addr":"172.21.15.134:6825/647343766","cluster_addr":"172.21.15.134:6827/647343766","heartbeat_back_addr":"172.21.15.134:6831/647343766","heartbeat_front_addr":"172.21.15.134:6829/647343766","state":["exists","up"]},{"osd":11,"uuid":"4c003a41-f65b-479f-bc80-b27d680cf936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":4068024116},{"type":"v1","addr":"172.21.15.148:6825","nonce":4068024116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":4068024116},{"type":"v1","addr":"172.21.15.148:6827","nonce":4068024116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":4068024116},{"type":"v1","addr":"172.21.15.148:6831","nonce":4068024116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":4068024116},{"type":"v1","addr":"172.21.15.148:6829","nonce":4068024116}]},"public_addr":"172.21.15.148:6825/4068024116","cluster_addr":"172.21.15.148:6827/4068024116","heartbeat_back_addr":"172.21.15.148:6831/4068024116","heartbeat_front_addr":"172.21.15.148:6829/4068024116","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:21:42.433631+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:22:06.145933+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:22:30.726143+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:22:55.729961+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:23:21.280913+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:23:46.609983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:24:12.585247+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:24:38.359287+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:25:04.491232+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:25:30.788089+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:25:56.891078+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:26:23.899113+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":7}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"4.a","mappings":[{"from":5,"to":8}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.80:0/2828378749":"2024-04-25T23:20:06.793609+0000","172.21.15.80:6801/233364790":"2024-04-25T23:20:06.793609+0000","172.21.15.80:6800/1614691410":"2024-04-25T23:19:40.490692+0000","172.21.15.80:0/1406187145":"2024-04-25T23:20:06.793609+0000","172.21.15.80:0/92953396":"2024-04-25T23:19:40.490692+0000","172.21.15.80:0/598129432":"2024-04-25T23:20:06.793609+0000","172.21.15.80:0/232193118":"2024-04-25T23:19:40.490692+0000","172.21.15.80:6800/233364790":"2024-04-25T23:20:06.793609+0000","172.21.15.80:6801/1614691410":"2024-04-25T23:19:40.490692+0000","172.21.15.80:0/2081562529":"2024-04-25T23:19:40.490692+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:32:41.065 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:41.065+0000 7f85adea5640 1 -- 172.21.15.80:0/85598703 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f85900a2b90 msgr2=0x7f85900a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:41.066 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:41.065+0000 7f85adea5640 1 --2- 172.21.15.80:0/85598703 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f85900a2b90 0x7f85900a5050 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f859c000c00 tx=0x7f859c031400 comp rx=0 tx=0).stop 2024-04-24T23:32:41.066 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:41.065+0000 7f85adea5640 1 -- 172.21.15.80:0/85598703 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f85a8100690 msgr2=0x7f85a8197a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:41.066 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:41.065+0000 7f85adea5640 1 --2- 172.21.15.80:0/85598703 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f85a8100690 0x7f85a8197a70 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f859800c970 tx=0x7f859800ce30 comp rx=0 tx=0).stop 2024-04-24T23:32:41.066 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:41.065+0000 7f85adea5640 1 -- 172.21.15.80:0/85598703 shutdown_connections 2024-04-24T23:32:41.066 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:41.065+0000 7f85adea5640 1 --2- 172.21.15.80:0/85598703 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f85900a2b90 0x7f85900a5050 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:41.067 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:41.065+0000 7f85adea5640 1 --2- 172.21.15.80:0/85598703 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85a8102240 0x7f85a819f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:41.067 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:41.065+0000 7f85adea5640 1 --2- 172.21.15.80:0/85598703 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f85a8101880 0x7f85a8197fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:41.067 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:41.065+0000 7f85adea5640 1 --2- 172.21.15.80:0/85598703 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f85a8100690 0x7f85a8197a70 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:41.067 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:41.065+0000 7f85adea5640 1 -- 172.21.15.80:0/85598703 >> 172.21.15.80:0/85598703 conn(0x7f85a80fbc40 msgr2=0x7f85a8104b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:32:41.067 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:41.065+0000 7f85adea5640 1 -- 172.21.15.80:0/85598703 shutdown_connections 2024-04-24T23:32:41.067 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:41.065+0000 7f85adea5640 1 -- 172.21.15.80:0/85598703 wait complete. 2024-04-24T23:32:41.077 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-24T23:32:41.182 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:41.179+0000 7f1671f90640 1 -- 172.21.15.80:0/3279685030 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f166c101870 msgr2=0x7f166c101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:41.182 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:41.179+0000 7f1671f90640 1 --2- 172.21.15.80:0/3279685030 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f166c101870 0x7f166c101cf0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f1660009900 tx=0x7f166002a410 comp rx=0 tx=0).stop 2024-04-24T23:32:41.182 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:41.179+0000 7f1671f90640 1 -- 172.21.15.80:0/3279685030 shutdown_connections 2024-04-24T23:32:41.183 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:41.179+0000 7f1671f90640 1 --2- 172.21.15.80:0/3279685030 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f166c102230 0x7f166c108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:41.183 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:41.179+0000 7f1671f90640 1 --2- 172.21.15.80:0/3279685030 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f166c101870 0x7f166c101cf0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:41.183 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:41.179+0000 7f1671f90640 1 --2- 172.21.15.80:0/3279685030 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f166c100680 0x7f166c100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:41.183 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:41.179+0000 7f1671f90640 1 -- 172.21.15.80:0/3279685030 >> 172.21.15.80:0/3279685030 conn(0x7f166c0fbbf0 msgr2=0x7f166c0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:32:41.183 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:41.179+0000 7f1671f90640 1 -- 172.21.15.80:0/3279685030 shutdown_connections 2024-04-24T23:32:41.183 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:41.179+0000 7f1671f90640 1 -- 172.21.15.80:0/3279685030 wait complete. 2024-04-24T23:32:41.183 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:41.180+0000 7f1671f90640 1 Processor -- start 2024-04-24T23:32:41.183 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:41.180+0000 7f1671f90640 1 -- start start 2024-04-24T23:32:41.184 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:41.180+0000 7f1671f90640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f166c100680 0x7f166c197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:41.184 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:41.180+0000 7f1671f90640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f166c101870 0x7f166c197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:41.184 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:41.180+0000 7f1671f90640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f166c102230 0x7f166c19f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:41.184 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:41.180+0000 7f1671f90640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f166c10c2e0 con 0x7f166c102230 2024-04-24T23:32:41.184 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:41.180+0000 7f1671f90640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f166c10c160 con 0x7f166c100680 2024-04-24T23:32:41.184 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:41.180+0000 7f1671f90640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f166c10c460 con 0x7f166c101870 2024-04-24T23:32:41.184 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:41.180+0000 7f166bfff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f166c102230 0x7f166c19f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:41.184 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:41.180+0000 7f166bfff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f166c102230 0x7f166c19f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:49284/0 (socket says 172.21.15.80:49284) 2024-04-24T23:32:41.185 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:41.180+0000 7f166bfff640 1 -- 172.21.15.80:0/2749136792 learned_addr learned my addr 172.21.15.80:0/2749136792 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:32:41.185 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:41.180+0000 7f166affd640 1 --2- 172.21.15.80:0/2749136792 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f166c101870 0x7f166c197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:41.185 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:41.180+0000 7f166bfff640 1 -- 172.21.15.80:0/2749136792 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f166c101870 msgr2=0x7f166c197ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:41.185 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:41.180+0000 7f166bfff640 1 --2- 172.21.15.80:0/2749136792 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f166c101870 0x7f166c197ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:41.185 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:41.180+0000 7f166bfff640 1 -- 172.21.15.80:0/2749136792 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f166c100680 msgr2=0x7f166c197ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:41.185 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:41.180+0000 7f166b7fe640 1 --2- 172.21.15.80:0/2749136792 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f166c100680 0x7f166c197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:41.185 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:41.181+0000 7f166bfff640 1 --2- 172.21.15.80:0/2749136792 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f166c100680 0x7f166c197ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:41.185 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:41.181+0000 7f166bfff640 1 -- 172.21.15.80:0/2749136792 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f166c19f6d0 con 0x7f166c102230 2024-04-24T23:32:41.185 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:41.181+0000 7f166bfff640 1 --2- 172.21.15.80:0/2749136792 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f166c102230 0x7f166c19f060 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f166400eaa0 tx=0x7f166400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:32:41.186 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:41.181+0000 7f1668ff9640 1 -- 172.21.15.80:0/2749136792 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f166400cdb0 con 0x7f166c102230 2024-04-24T23:32:41.186 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:41.181+0000 7f1671f90640 1 -- 172.21.15.80:0/2749136792 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f166c19f9c0 con 0x7f166c102230 2024-04-24T23:32:41.186 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:41.181+0000 7f1668ff9640 1 -- 172.21.15.80:0/2749136792 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1664005020 con 0x7f166c102230 2024-04-24T23:32:41.186 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:41.181+0000 7f1668ff9640 1 -- 172.21.15.80:0/2749136792 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16640116b0 con 0x7f166c102230 2024-04-24T23:32:41.186 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:41.181+0000 7f1671f90640 1 -- 172.21.15.80:0/2749136792 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f166c19fee0 con 0x7f166c102230 2024-04-24T23:32:41.186 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:41.182+0000 7f1671f90640 1 -- 172.21.15.80:0/2749136792 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f166c068560 con 0x7f166c102230 2024-04-24T23:32:41.187 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:41.184+0000 7f1668ff9640 1 -- 172.21.15.80:0/2749136792 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1664005560 con 0x7f166c102230 2024-04-24T23:32:41.187 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:41.184+0000 7f1668ff9640 1 --2- 172.21.15.80:0/2749136792 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f16540a2b90 0x7f16540a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:41.187 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:41.185+0000 7f166b7fe640 1 --2- 172.21.15.80:0/2749136792 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f16540a2b90 0x7f16540a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:41.187 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:41.185+0000 7f1668ff9640 1 -- 172.21.15.80:0/2749136792 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f16640cfb30 con 0x7f166c102230 2024-04-24T23:32:41.188 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:41.185+0000 7f166b7fe640 1 --2- 172.21.15.80:0/2749136792 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f16540a2b90 0x7f16540a5050 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f165c005fd0 tx=0x7f165c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:32:41.188 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:41.187+0000 7f1668ff9640 1 -- 172.21.15.80:0/2749136792 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f1664016030 con 0x7f166c102230 2024-04-24T23:32:41.348 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:32:41 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/3307629127' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:32:41.348 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:32:41 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/2576949753' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:32:41.348 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:32:41 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/85598703' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:32:41.410 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:41.409+0000 7f1671f90640 1 -- 172.21.15.80:0/2749136792 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f166c1a0360 con 0x7f166c102230 2024-04-24T23:32:41.412 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:41.412+0000 7f1668ff9640 1 -- 172.21.15.80:0/2749136792 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v28) ==== 161+0+0 (secure 0 0 0) 0x7f166409c6c0 con 0x7f166c102230 2024-04-24T23:32:41.412 INFO:teuthology.orchestra.run.smithi080.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-24T23:32:41.415 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:41.415+0000 7f1671f90640 1 -- 172.21.15.80:0/2749136792 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f16540a2b90 msgr2=0x7f16540a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:41.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:41.415+0000 7f1671f90640 1 --2- 172.21.15.80:0/2749136792 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f16540a2b90 0x7f16540a5050 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f165c005fd0 tx=0x7f165c005850 comp rx=0 tx=0).stop 2024-04-24T23:32:41.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:41.415+0000 7f1671f90640 1 -- 172.21.15.80:0/2749136792 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f166c102230 msgr2=0x7f166c19f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:41.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:41.415+0000 7f1671f90640 1 --2- 172.21.15.80:0/2749136792 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f166c102230 0x7f166c19f060 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f166400eaa0 tx=0x7f166400ef60 comp rx=0 tx=0).stop 2024-04-24T23:32:41.417 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:41.416+0000 7f1671f90640 1 -- 172.21.15.80:0/2749136792 shutdown_connections 2024-04-24T23:32:41.417 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:41.416+0000 7f1671f90640 1 --2- 172.21.15.80:0/2749136792 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f16540a2b90 0x7f16540a5050 secure :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f165c005fd0 tx=0x7f165c005850 comp rx=0 tx=0).stop 2024-04-24T23:32:41.417 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:41.416+0000 7f1671f90640 1 --2- 172.21.15.80:0/2749136792 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f166c102230 0x7f166c19f060 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:41.417 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:41.416+0000 7f1671f90640 1 --2- 172.21.15.80:0/2749136792 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f166c101870 0x7f166c197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:41.417 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:41.416+0000 7f1671f90640 1 --2- 172.21.15.80:0/2749136792 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f166c100680 0x7f166c197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:41.418 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:41.416+0000 7f1671f90640 1 -- 172.21.15.80:0/2749136792 >> 172.21.15.80:0/2749136792 conn(0x7f166c0fbbf0 msgr2=0x7f166c0fd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:32:41.418 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:41.416+0000 7f1671f90640 1 -- 172.21.15.80:0/2749136792 shutdown_connections 2024-04-24T23:32:41.418 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:41.416+0000 7f1671f90640 1 -- 172.21.15.80:0/2749136792 wait complete. 2024-04-24T23:32:41.428 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-24T23:32:41.446 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:32:41 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/3307629127' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:32:41.446 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:32:41 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/2576949753' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:32:41.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:32:41 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/85598703' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:32:41.457 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:32:41 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/3307629127' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:32:41.457 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:32:41 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/2576949753' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:32:41.457 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:32:41 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/85598703' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:32:41.545 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:41.543+0000 7fe583853640 1 -- 172.21.15.80:0/2398758143 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe57c107460 msgr2=0x7fe57c1098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:41.546 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:41.543+0000 7fe583853640 1 --2- 172.21.15.80:0/2398758143 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe57c107460 0x7fe57c1098b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe574009be0 tx=0x7fe57402a6c0 comp rx=0 tx=0).stop 2024-04-24T23:32:41.546 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:41.543+0000 7fe583853640 1 -- 172.21.15.80:0/2398758143 shutdown_connections 2024-04-24T23:32:41.546 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:41.543+0000 7fe583853640 1 --2- 172.21.15.80:0/2398758143 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe57c107460 0x7fe57c1098b0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:41.546 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:41.543+0000 7fe583853640 1 --2- 172.21.15.80:0/2398758143 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe57c104b30 0x7fe57c106f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:41.546 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:41.543+0000 7fe583853640 1 --2- 172.21.15.80:0/2398758143 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe57c102200 0x7fe57c1045f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:41.547 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:41.543+0000 7fe583853640 1 -- 172.21.15.80:0/2398758143 >> 172.21.15.80:0/2398758143 conn(0x7fe57c0fbbf0 msgr2=0x7fe57c0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:32:41.547 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:41.543+0000 7fe583853640 1 -- 172.21.15.80:0/2398758143 shutdown_connections 2024-04-24T23:32:41.547 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:41.543+0000 7fe583853640 1 -- 172.21.15.80:0/2398758143 wait complete. 2024-04-24T23:32:41.547 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:41.544+0000 7fe583853640 1 Processor -- start 2024-04-24T23:32:41.547 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:41.544+0000 7fe583853640 1 -- start start 2024-04-24T23:32:41.547 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:41.544+0000 7fe583853640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe57c102200 0x7fe57c197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:41.547 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:41.544+0000 7fe583853640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe57c104b30 0x7fe57c198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:41.547 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:41.544+0000 7fe583853640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe57c107460 0x7fe57c19f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:41.548 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:41.544+0000 7fe583853640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fe57c10d420 con 0x7fe57c102200 2024-04-24T23:32:41.548 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:41.544+0000 7fe583853640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe57c10d2a0 con 0x7fe57c104b30 2024-04-24T23:32:41.548 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:41.544+0000 7fe583853640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe57c10d5a0 con 0x7fe57c107460 2024-04-24T23:32:41.548 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:41.544+0000 7fe5815ef640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe57c102200 0x7fe57c197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:41.548 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:41.544+0000 7fe5815ef640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe57c102200 0x7fe57c197ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:49300/0 (socket says 172.21.15.80:49300) 2024-04-24T23:32:41.548 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:41.544+0000 7fe5815ef640 1 -- 172.21.15.80:0/376657377 learned_addr learned my addr 172.21.15.80:0/376657377 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:32:41.548 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:41.544+0000 7fe580dee640 1 --2- 172.21.15.80:0/376657377 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe57c104b30 0x7fe57c198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:41.548 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:41.544+0000 7fe581df0640 1 --2- 172.21.15.80:0/376657377 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe57c107460 0x7fe57c19f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:41.549 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:41.544+0000 7fe5815ef640 1 -- 172.21.15.80:0/376657377 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe57c107460 msgr2=0x7fe57c19f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:41.549 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:41.544+0000 7fe5815ef640 1 --2- 172.21.15.80:0/376657377 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe57c107460 0x7fe57c19f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:41.549 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:41.544+0000 7fe5815ef640 1 -- 172.21.15.80:0/376657377 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe57c104b30 msgr2=0x7fe57c198000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:41.549 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:41.544+0000 7fe5815ef640 1 --2- 172.21.15.80:0/376657377 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe57c104b30 0x7fe57c198000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:41.549 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:41.544+0000 7fe5815ef640 1 -- 172.21.15.80:0/376657377 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe57c19f690 con 0x7fe57c102200 2024-04-24T23:32:41.549 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:41.544+0000 7fe581df0640 1 --2- 172.21.15.80:0/376657377 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe57c107460 0x7fe57c19f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:32:41.549 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:41.544+0000 7fe580dee640 1 --2- 172.21.15.80:0/376657377 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe57c104b30 0x7fe57c198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:32:41.549 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:41.544+0000 7fe5815ef640 1 --2- 172.21.15.80:0/376657377 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe57c102200 0x7fe57c197ac0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fe56c00e9c0 tx=0x7fe56c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:32:41.550 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:41.545+0000 7fe5727fc640 1 -- 172.21.15.80:0/376657377 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe56c00cde0 con 0x7fe57c102200 2024-04-24T23:32:41.550 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:41.545+0000 7fe583853640 1 -- 172.21.15.80:0/376657377 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe57c19f980 con 0x7fe57c102200 2024-04-24T23:32:41.550 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:41.545+0000 7fe5727fc640 1 -- 172.21.15.80:0/376657377 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe56c0035f0 con 0x7fe57c102200 2024-04-24T23:32:41.550 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:41.545+0000 7fe5727fc640 1 -- 172.21.15.80:0/376657377 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe56c010650 con 0x7fe57c102200 2024-04-24T23:32:41.550 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:41.545+0000 7fe583853640 1 -- 172.21.15.80:0/376657377 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe57c19ff20 con 0x7fe57c102200 2024-04-24T23:32:41.551 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:41.546+0000 7fe583853640 1 -- 172.21.15.80:0/376657377 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe57c0fd6e0 con 0x7fe57c102200 2024-04-24T23:32:41.551 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:41.547+0000 7fe5727fc640 1 -- 172.21.15.80:0/376657377 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe56c0037a0 con 0x7fe57c102200 2024-04-24T23:32:41.552 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:41.547+0000 7fe5727fc640 1 --2- 172.21.15.80:0/376657377 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fe5680a2b90 0x7fe5680a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:41.552 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:41.547+0000 7fe580dee640 1 --2- 172.21.15.80:0/376657377 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fe5680a2b90 0x7fe5680a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:41.552 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:41.547+0000 7fe5727fc640 1 -- 172.21.15.80:0/376657377 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9466+0+0 (secure 0 0 0) 0x7fe56c0cfa60 con 0x7fe57c102200 2024-04-24T23:32:41.552 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:41.548+0000 7fe580dee640 1 --2- 172.21.15.80:0/376657377 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fe5680a2b90 0x7fe5680a5050 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fe57c198f80 tx=0x7fe578005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:32:41.552 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:41.551+0000 7fe5727fc640 1 -- 172.21.15.80:0/376657377 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fe56c09c580 con 0x7fe57c102200 2024-04-24T23:32:41.756 INFO:journalctl@ceph.mds.c.smithi148.stdout:Apr 24 23:32:41 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-c[63396]: ignoring --setuser ceph since I am not root 2024-04-24T23:32:41.756 INFO:journalctl@ceph.mds.c.smithi148.stdout:Apr 24 23:32:41 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-c[63396]: ignoring --setgroup ceph since I am not root 2024-04-24T23:32:41.757 INFO:journalctl@ceph.mds.c.smithi148.stdout:Apr 24 23:32:41 smithi148 ceph-mds[63400]: -- 172.21.15.148:0/485465000 <== mon.1 v2:172.21.15.148:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5572707cba40 con 0x55727079ec00 2024-04-24T23:32:41.757 INFO:journalctl@ceph.mds.c.smithi148.stdout:Apr 24 23:32:41 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-c[63396]: starting mds.c at 2024-04-24T23:32:41.757 INFO:journalctl@ceph.mds.f.smithi148.stdout:Apr 24 23:32:41 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-f[63087]: ignoring --setuser ceph since I am not root 2024-04-24T23:32:41.757 INFO:journalctl@ceph.mds.f.smithi148.stdout:Apr 24 23:32:41 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-f[63087]: ignoring --setgroup ceph since I am not root 2024-04-24T23:32:41.757 INFO:journalctl@ceph.mds.f.smithi148.stdout:Apr 24 23:32:41 smithi148 ceph-mds[63091]: -- 172.21.15.148:0/331724575 <== mon.2 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55679b1c5a40 con 0x55679b198800 2024-04-24T23:32:41.757 INFO:journalctl@ceph.mds.f.smithi148.stdout:Apr 24 23:32:41 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-f[63087]: starting mds.f at 2024-04-24T23:32:41.758 INFO:journalctl@ceph.mds.i.smithi148.stdout:Apr 24 23:32:41 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-i[63704]: ignoring --setuser ceph since I am not root 2024-04-24T23:32:41.758 INFO:journalctl@ceph.mds.i.smithi148.stdout:Apr 24 23:32:41 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-i[63704]: ignoring --setgroup ceph since I am not root 2024-04-24T23:32:41.758 INFO:journalctl@ceph.mds.i.smithi148.stdout:Apr 24 23:32:41 smithi148 ceph-mds[63708]: -- 172.21.15.148:0/1955198230 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55f722cc64e0 con 0x55f722c9c800 2024-04-24T23:32:41.758 INFO:journalctl@ceph.mds.i.smithi148.stdout:Apr 24 23:32:41 smithi148 ceph-mds[63708]: -- 172.21.15.148:0/1955198230 <== mon.1 v2:172.21.15.148:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f722cc9a40 con 0x55f722c9c800 2024-04-24T23:32:41.758 INFO:journalctl@ceph.mds.i.smithi148.stdout:Apr 24 23:32:41 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-i[63704]: starting mds.i at 2024-04-24T23:32:41.774 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:41.774+0000 7fe583853640 1 -- 172.21.15.80:0/376657377 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fe57c02d170 con 0x7fe57c102200 2024-04-24T23:32:41.787 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:32:41 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mon-a[29148]: 2024-04-24T23:32:41.410+0000 7f1265e23700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-24T23:32:41.947 INFO:journalctl@ceph.mds.e.smithi134.stdout:Apr 24 23:32:41 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-e[62785]: ignoring --setuser ceph since I am not root 2024-04-24T23:32:41.947 INFO:journalctl@ceph.mds.e.smithi134.stdout:Apr 24 23:32:41 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-e[62785]: ignoring --setgroup ceph since I am not root 2024-04-24T23:32:41.947 INFO:journalctl@ceph.mds.e.smithi134.stdout:Apr 24 23:32:41 smithi134 ceph-mds[62789]: -- 172.21.15.134:0/2127626635 <== mon.2 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a97a04fa40 con 0x55a97a023000 2024-04-24T23:32:41.948 INFO:journalctl@ceph.mds.e.smithi134.stdout:Apr 24 23:32:41 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-e[62785]: starting mds.e at 2024-04-24T23:32:41.948 INFO:journalctl@ceph.mds.b.smithi134.stdout:Apr 24 23:32:41 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-b[63405]: ignoring --setuser ceph since I am not root 2024-04-24T23:32:41.948 INFO:journalctl@ceph.mds.b.smithi134.stdout:Apr 24 23:32:41 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-b[63405]: ignoring --setgroup ceph since I am not root 2024-04-24T23:32:41.948 INFO:journalctl@ceph.mds.b.smithi134.stdout:Apr 24 23:32:41 smithi134 ceph-mds[63409]: -- 172.21.15.134:0/1271266658 <== mon.2 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55efecf89a40 con 0x55efecf5d000 2024-04-24T23:32:41.949 INFO:journalctl@ceph.mds.b.smithi134.stdout:Apr 24 23:32:41 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-b[63405]: starting mds.b at 2024-04-24T23:32:41.949 INFO:journalctl@ceph.mds.h.smithi134.stdout:Apr 24 23:32:41 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-h[63097]: ignoring --setuser ceph since I am not root 2024-04-24T23:32:41.949 INFO:journalctl@ceph.mds.h.smithi134.stdout:Apr 24 23:32:41 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-h[63097]: ignoring --setgroup ceph since I am not root 2024-04-24T23:32:41.949 INFO:journalctl@ceph.mds.h.smithi134.stdout:Apr 24 23:32:41 smithi134 ceph-mds[63101]: -- 172.21.15.134:0/1737588929 <== mon.2 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ccb844fa40 con 0x55ccb8422800 2024-04-24T23:32:41.949 INFO:journalctl@ceph.mds.h.smithi134.stdout:Apr 24 23:32:41 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-h[63097]: starting mds.h at 2024-04-24T23:32:42.414 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:42.413+0000 7fe5727fc640 1 -- 172.21.15.80:0/376657377 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v29) ==== 104+0+0 (secure 0 0 0) 0x7fe56c09be80 con 0x7fe57c102200 2024-04-24T23:32:42.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:42.415+0000 7fe583853640 1 -- 172.21.15.80:0/376657377 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fe5680a2b90 msgr2=0x7fe5680a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:42.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:42.415+0000 7fe583853640 1 --2- 172.21.15.80:0/376657377 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fe5680a2b90 0x7fe5680a5050 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fe57c198f80 tx=0x7fe578005f90 comp rx=0 tx=0).stop 2024-04-24T23:32:42.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:42.415+0000 7fe583853640 1 -- 172.21.15.80:0/376657377 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe57c102200 msgr2=0x7fe57c197ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:42.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:42.415+0000 7fe583853640 1 --2- 172.21.15.80:0/376657377 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe57c102200 0x7fe57c197ac0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fe56c00e9c0 tx=0x7fe56c00ee80 comp rx=0 tx=0).stop 2024-04-24T23:32:42.417 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:42.415+0000 7fe583853640 1 -- 172.21.15.80:0/376657377 shutdown_connections 2024-04-24T23:32:42.417 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:42.415+0000 7fe583853640 1 --2- 172.21.15.80:0/376657377 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7fe5680a2b90 0x7fe5680a5050 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:42.417 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:42.415+0000 7fe583853640 1 --2- 172.21.15.80:0/376657377 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe57c107460 0x7fe57c19f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:42.417 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:42.415+0000 7fe583853640 1 --2- 172.21.15.80:0/376657377 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe57c104b30 0x7fe57c198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:42.417 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:42.415+0000 7fe583853640 1 --2- 172.21.15.80:0/376657377 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe57c102200 0x7fe57c197ac0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:42.417 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:42.415+0000 7fe583853640 1 -- 172.21.15.80:0/376657377 >> 172.21.15.80:0/376657377 conn(0x7fe57c0fbbf0 msgr2=0x7fe57c102b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:32:42.417 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:42.415+0000 7fe583853640 1 -- 172.21.15.80:0/376657377 shutdown_connections 2024-04-24T23:32:42.417 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:42.415+0000 7fe583853640 1 -- 172.21.15.80:0/376657377 wait complete. 2024-04-24T23:32:42.427 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:32:42 smithi080 ceph-mon[29152]: pgmap v461: 129 pgs: 129 active+clean; 75 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.2 KiB/s wr, 7 op/s 2024-04-24T23:32:42.427 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:32:42 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/2749136792' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-24T23:32:42.428 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:32:42 smithi080 ceph-mon[29152]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-24T23:32:42.428 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:32:42 smithi080 ceph-mon[29152]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-24T23:32:42.428 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:32:42 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/2749136792' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-24T23:32:42.428 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:32:42 smithi080 ceph-mon[29152]: osdmap e95: 12 total, 12 up, 12 in 2024-04-24T23:32:42.428 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:32:42 smithi080 ceph-mon[29152]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-04-24T23:32:42.428 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:32:42 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/376657377' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-24T23:32:42.429 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-24T23:32:42.446 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:32:42 smithi134 ceph-mon[32619]: pgmap v461: 129 pgs: 129 active+clean; 75 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.2 KiB/s wr, 7 op/s 2024-04-24T23:32:42.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:32:42 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/2749136792' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-24T23:32:42.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:32:42 smithi134 ceph-mon[32619]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-24T23:32:42.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:32:42 smithi134 ceph-mon[32619]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-24T23:32:42.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:32:42 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/2749136792' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-24T23:32:42.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:32:42 smithi134 ceph-mon[32619]: osdmap e95: 12 total, 12 up, 12 in 2024-04-24T23:32:42.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:32:42 smithi134 ceph-mon[32619]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-04-24T23:32:42.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:32:42 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/376657377' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-24T23:32:42.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:32:42 smithi148 ceph-mon[32528]: pgmap v461: 129 pgs: 129 active+clean; 75 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.2 KiB/s wr, 7 op/s 2024-04-24T23:32:42.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:32:42 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/2749136792' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-24T23:32:42.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:32:42 smithi148 ceph-mon[32528]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-24T23:32:42.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:32:42 smithi148 ceph-mon[32528]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-24T23:32:42.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:32:42 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/2749136792' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-24T23:32:42.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:32:42 smithi148 ceph-mon[32528]: osdmap e95: 12 total, 12 up, 12 in 2024-04-24T23:32:42.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:32:42 smithi148 ceph-mon[32528]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-04-24T23:32:42.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:32:42 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/376657377' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-24T23:32:42.532 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:42.531+0000 7f0c510fc640 1 -- 172.21.15.80:0/1397189584 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0c4c102200 msgr2=0x7f0c4c1045f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:42.532 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:42.531+0000 7f0c510fc640 1 --2- 172.21.15.80:0/1397189584 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0c4c102200 0x7f0c4c1045f0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c009900 tx=0x7f0c3c02a410 comp rx=0 tx=0).stop 2024-04-24T23:32:42.532 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:42.531+0000 7f0c510fc640 1 -- 172.21.15.80:0/1397189584 shutdown_connections 2024-04-24T23:32:42.532 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:42.531+0000 7f0c510fc640 1 --2- 172.21.15.80:0/1397189584 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c4c107460 0x7f0c4c1098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:42.532 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:42.531+0000 7f0c510fc640 1 --2- 172.21.15.80:0/1397189584 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c4c104b30 0x7f0c4c106f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:42.532 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:42.531+0000 7f0c510fc640 1 --2- 172.21.15.80:0/1397189584 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0c4c102200 0x7f0c4c1045f0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:42.533 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:42.531+0000 7f0c510fc640 1 -- 172.21.15.80:0/1397189584 >> 172.21.15.80:0/1397189584 conn(0x7f0c4c0fbc30 msgr2=0x7f0c4c0fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:32:42.533 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:42.531+0000 7f0c510fc640 1 -- 172.21.15.80:0/1397189584 shutdown_connections 2024-04-24T23:32:42.533 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:42.531+0000 7f0c510fc640 1 -- 172.21.15.80:0/1397189584 wait complete. 2024-04-24T23:32:42.535 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:42.532+0000 7f0c510fc640 1 Processor -- start 2024-04-24T23:32:42.535 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:42.532+0000 7f0c510fc640 1 -- start start 2024-04-24T23:32:42.536 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:42.532+0000 7f0c510fc640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0c4c102200 0x7f0c4c197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:42.536 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:42.532+0000 7f0c510fc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c4c104b30 0x7f0c4c198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:42.536 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:42.532+0000 7f0c510fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c4c107460 0x7f0c4c19f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:42.536 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:42.532+0000 7f0c510fc640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f0c4c10d4f0 con 0x7f0c4c102200 2024-04-24T23:32:42.536 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:42.532+0000 7f0c510fc640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0c4c10d370 con 0x7f0c4c104b30 2024-04-24T23:32:42.536 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:42.532+0000 7f0c510fc640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f0c4c10d670 con 0x7f0c4c107460 2024-04-24T23:32:42.536 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:42.532+0000 7f0c4ad9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0c4c102200 0x7f0c4c197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:42.536 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:42.532+0000 7f0c4ad9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0c4c102200 0x7f0c4c197b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:33536/0 (socket says 172.21.15.80:33536) 2024-04-24T23:32:42.536 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:42.532+0000 7f0c4ad9d640 1 -- 172.21.15.80:0/970200061 learned_addr learned my addr 172.21.15.80:0/970200061 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:32:42.537 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:42.532+0000 7f0c4b59e640 1 --2- 172.21.15.80:0/970200061 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c4c107460 0x7f0c4c19f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:42.537 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:42.532+0000 7f0c43fff640 1 --2- 172.21.15.80:0/970200061 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c4c104b30 0x7f0c4c198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:42.537 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:42.532+0000 7f0c4ad9d640 1 -- 172.21.15.80:0/970200061 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c4c107460 msgr2=0x7f0c4c19f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:42.537 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:42.532+0000 7f0c4ad9d640 1 --2- 172.21.15.80:0/970200061 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c4c107460 0x7f0c4c19f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:42.537 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:42.532+0000 7f0c4ad9d640 1 -- 172.21.15.80:0/970200061 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c4c104b30 msgr2=0x7f0c4c198080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:42.537 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:42.532+0000 7f0c4ad9d640 1 --2- 172.21.15.80:0/970200061 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c4c104b30 0x7f0c4c198080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:42.537 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:42.532+0000 7f0c4ad9d640 1 -- 172.21.15.80:0/970200061 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c4c19f760 con 0x7f0c4c102200 2024-04-24T23:32:42.537 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:42.532+0000 7f0c4b59e640 1 --2- 172.21.15.80:0/970200061 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c4c107460 0x7f0c4c19f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:32:42.537 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:42.532+0000 7f0c43fff640 1 --2- 172.21.15.80:0/970200061 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c4c104b30 0x7f0c4c198080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:32:42.538 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:42.532+0000 7f0c4ad9d640 1 --2- 172.21.15.80:0/970200061 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0c4c102200 0x7f0c4c197b40 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c031440 tx=0x7f0c3c032e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:32:42.538 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:42.533+0000 7f0c48d99640 1 -- 172.21.15.80:0/970200061 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c3c03f030 con 0x7f0c4c102200 2024-04-24T23:32:42.538 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:42.533+0000 7f0c48d99640 1 -- 172.21.15.80:0/970200061 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c3c003ce0 con 0x7f0c4c102200 2024-04-24T23:32:42.538 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:42.533+0000 7f0c48d99640 1 -- 172.21.15.80:0/970200061 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c3c035070 con 0x7f0c4c102200 2024-04-24T23:32:42.538 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:42.533+0000 7f0c510fc640 1 -- 172.21.15.80:0/970200061 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c4c19f9f0 con 0x7f0c4c102200 2024-04-24T23:32:42.538 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:42.533+0000 7f0c510fc640 1 -- 172.21.15.80:0/970200061 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c4c19ff60 con 0x7f0c4c102200 2024-04-24T23:32:42.538 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:42.534+0000 7f0c510fc640 1 -- 172.21.15.80:0/970200061 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c4c0fd720 con 0x7f0c4c102200 2024-04-24T23:32:42.538 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:42.535+0000 7f0c48d99640 1 -- 172.21.15.80:0/970200061 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0c3c042030 con 0x7f0c4c102200 2024-04-24T23:32:42.539 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:42.535+0000 7f0c48d99640 1 --2- 172.21.15.80:0/970200061 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f0c380a2b90 0x7f0c380a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:42.539 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:42.535+0000 7f0c43fff640 1 --2- 172.21.15.80:0/970200061 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f0c380a2b90 0x7f0c380a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:42.539 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:42.535+0000 7f0c48d99640 1 -- 172.21.15.80:0/970200061 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9466+0+0 (secure 0 0 0) 0x7f0c3c0f0070 con 0x7f0c4c102200 2024-04-24T23:32:42.540 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:42.536+0000 7f0c43fff640 1 --2- 172.21.15.80:0/970200061 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f0c380a2b90 0x7f0c380a5050 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f0c34006fd0 tx=0x7f0c34008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:32:42.540 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:42.539+0000 7f0c48d99640 1 -- 172.21.15.80:0/970200061 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f0c3c0bcb00 con 0x7f0c4c102200 2024-04-24T23:32:42.716 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:42.715+0000 7f0c510fc640 1 -- 172.21.15.80:0/970200061 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f0c4c02d170 con 0x7f0c4c102200 2024-04-24T23:32:43.415 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:43.415+0000 7f0c48d99640 1 -- 172.21.15.80:0/970200061 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v96) ==== 181+0+0 (secure 0 0 0) 0x7f0c3c0bc400 con 0x7f0c4c102200 2024-04-24T23:32:43.416 INFO:teuthology.orchestra.run.smithi080.stderr:pool 'cephfs_metadata' removed 2024-04-24T23:32:43.420 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:43.418+0000 7f0c510fc640 1 -- 172.21.15.80:0/970200061 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f0c380a2b90 msgr2=0x7f0c380a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:43.420 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:43.418+0000 7f0c510fc640 1 --2- 172.21.15.80:0/970200061 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f0c380a2b90 0x7f0c380a5050 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f0c34006fd0 tx=0x7f0c34008040 comp rx=0 tx=0).stop 2024-04-24T23:32:43.420 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:43.418+0000 7f0c510fc640 1 -- 172.21.15.80:0/970200061 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0c4c102200 msgr2=0x7f0c4c197b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:43.420 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:43.418+0000 7f0c510fc640 1 --2- 172.21.15.80:0/970200061 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0c4c102200 0x7f0c4c197b40 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c031440 tx=0x7f0c3c032e20 comp rx=0 tx=0).stop 2024-04-24T23:32:43.420 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:43.418+0000 7f0c510fc640 1 -- 172.21.15.80:0/970200061 shutdown_connections 2024-04-24T23:32:43.421 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:43.419+0000 7f0c510fc640 1 --2- 172.21.15.80:0/970200061 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f0c380a2b90 0x7f0c380a5050 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:43.421 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:43.419+0000 7f0c510fc640 1 --2- 172.21.15.80:0/970200061 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c4c107460 0x7f0c4c19f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:43.421 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:43.419+0000 7f0c510fc640 1 --2- 172.21.15.80:0/970200061 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c4c104b30 0x7f0c4c198080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:43.421 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:43.419+0000 7f0c510fc640 1 --2- 172.21.15.80:0/970200061 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0c4c102200 0x7f0c4c197b40 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:43.421 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:43.419+0000 7f0c510fc640 1 -- 172.21.15.80:0/970200061 >> 172.21.15.80:0/970200061 conn(0x7f0c4c0fbc30 msgr2=0x7f0c4c102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:32:43.421 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:43.419+0000 7f0c510fc640 1 -- 172.21.15.80:0/970200061 shutdown_connections 2024-04-24T23:32:43.421 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:43.419+0000 7f0c510fc640 1 -- 172.21.15.80:0/970200061 wait complete. 2024-04-24T23:32:43.433 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:32:43 smithi080 ceph-mon[29152]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-24T23:32:43.433 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:32:43 smithi080 ceph-mon[29152]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-24T23:32:43.434 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:32:43 smithi080 ceph-mon[29152]: Cluster is now healthy 2024-04-24T23:32:43.434 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:32:43 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/376657377' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-24T23:32:43.434 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:32:43 smithi080 ceph-mon[29152]: mds.? [v2:172.21.15.134:6834/3304330342,v1:172.21.15.134:6835/3304330342] up:boot 2024-04-24T23:32:43.434 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:32:43 smithi080 ceph-mon[29152]: mds.? [v2:172.21.15.148:6835/1453317,v1:172.21.15.148:6838/1453317] up:boot 2024-04-24T23:32:43.434 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:32:43 smithi080 ceph-mon[29152]: mds.? [v2:172.21.15.148:6834/3155580468,v1:172.21.15.148:6837/3155580468] up:boot 2024-04-24T23:32:43.434 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:32:43 smithi080 ceph-mon[29152]: mds.? [v2:172.21.15.148:6836/1802283245,v1:172.21.15.148:6839/1802283245] up:boot 2024-04-24T23:32:43.434 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:32:43 smithi080 ceph-mon[29152]: mds.? [v2:172.21.15.134:6837/863792166,v1:172.21.15.134:6838/863792166] up:boot 2024-04-24T23:32:43.435 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:32:43 smithi080 ceph-mon[29152]: mds.? [v2:172.21.15.134:6836/2112081304,v1:172.21.15.134:6839/2112081304] up:boot 2024-04-24T23:32:43.435 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:32:43 smithi080 ceph-mon[29152]: fsmap 12 up:standby 2024-04-24T23:32:43.435 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:32:43 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-24T23:32:43.435 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:32:43 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-24T23:32:43.435 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:32:43 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-24T23:32:43.435 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:32:43 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-24T23:32:43.436 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:32:43 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-24T23:32:43.436 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:32:43 smithi080 ceph-mon[29152]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-24T23:32:43.436 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:32:43 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/970200061' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-24T23:32:43.437 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-24T23:32:43.446 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:32:43 smithi134 ceph-mon[32619]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-24T23:32:43.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:32:43 smithi134 ceph-mon[32619]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-24T23:32:43.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:32:43 smithi134 ceph-mon[32619]: Cluster is now healthy 2024-04-24T23:32:43.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:32:43 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/376657377' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-24T23:32:43.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:32:43 smithi134 ceph-mon[32619]: mds.? [v2:172.21.15.134:6834/3304330342,v1:172.21.15.134:6835/3304330342] up:boot 2024-04-24T23:32:43.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:32:43 smithi134 ceph-mon[32619]: mds.? [v2:172.21.15.148:6835/1453317,v1:172.21.15.148:6838/1453317] up:boot 2024-04-24T23:32:43.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:32:43 smithi134 ceph-mon[32619]: mds.? [v2:172.21.15.148:6834/3155580468,v1:172.21.15.148:6837/3155580468] up:boot 2024-04-24T23:32:43.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:32:43 smithi134 ceph-mon[32619]: mds.? [v2:172.21.15.148:6836/1802283245,v1:172.21.15.148:6839/1802283245] up:boot 2024-04-24T23:32:43.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:32:43 smithi134 ceph-mon[32619]: mds.? [v2:172.21.15.134:6837/863792166,v1:172.21.15.134:6838/863792166] up:boot 2024-04-24T23:32:43.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:32:43 smithi134 ceph-mon[32619]: mds.? [v2:172.21.15.134:6836/2112081304,v1:172.21.15.134:6839/2112081304] up:boot 2024-04-24T23:32:43.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:32:43 smithi134 ceph-mon[32619]: fsmap 12 up:standby 2024-04-24T23:32:43.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:32:43 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-24T23:32:43.448 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:32:43 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-24T23:32:43.449 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:32:43 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-24T23:32:43.449 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:32:43 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-24T23:32:43.449 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:32:43 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-24T23:32:43.449 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:32:43 smithi134 ceph-mon[32619]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-24T23:32:43.449 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:32:43 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/970200061' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-24T23:32:43.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:32:43 smithi148 ceph-mon[32528]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-24T23:32:43.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:32:43 smithi148 ceph-mon[32528]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-24T23:32:43.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:32:43 smithi148 ceph-mon[32528]: Cluster is now healthy 2024-04-24T23:32:43.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:32:43 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/376657377' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-24T23:32:43.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:32:43 smithi148 ceph-mon[32528]: mds.? [v2:172.21.15.134:6834/3304330342,v1:172.21.15.134:6835/3304330342] up:boot 2024-04-24T23:32:43.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:32:43 smithi148 ceph-mon[32528]: mds.? [v2:172.21.15.148:6835/1453317,v1:172.21.15.148:6838/1453317] up:boot 2024-04-24T23:32:43.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:32:43 smithi148 ceph-mon[32528]: mds.? [v2:172.21.15.148:6834/3155580468,v1:172.21.15.148:6837/3155580468] up:boot 2024-04-24T23:32:43.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:32:43 smithi148 ceph-mon[32528]: mds.? [v2:172.21.15.148:6836/1802283245,v1:172.21.15.148:6839/1802283245] up:boot 2024-04-24T23:32:43.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:32:43 smithi148 ceph-mon[32528]: mds.? [v2:172.21.15.134:6837/863792166,v1:172.21.15.134:6838/863792166] up:boot 2024-04-24T23:32:43.507 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:32:43 smithi148 ceph-mon[32528]: mds.? [v2:172.21.15.134:6836/2112081304,v1:172.21.15.134:6839/2112081304] up:boot 2024-04-24T23:32:43.508 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:32:43 smithi148 ceph-mon[32528]: fsmap 12 up:standby 2024-04-24T23:32:43.508 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:32:43 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-24T23:32:43.508 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:32:43 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-24T23:32:43.508 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:32:43 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-24T23:32:43.508 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:32:43 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-24T23:32:43.508 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:32:43 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-24T23:32:43.508 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:32:43 smithi148 ceph-mon[32528]: from='mgr.14152 172.21.15.80:0/573784130' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-24T23:32:43.508 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:32:43 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/970200061' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-24T23:32:43.547 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:43.544+0000 7f32d1cbd640 1 -- 172.21.15.80:0/1864188205 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f32cc103340 msgr2=0x7f32cc109bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:43.548 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:43.544+0000 7f32d1cbd640 1 --2- 172.21.15.80:0/1864188205 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f32cc103340 0x7f32cc109bd0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f32c4009ca0 tx=0x7f32c402a8c0 comp rx=0 tx=0).stop 2024-04-24T23:32:43.548 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:43.545+0000 7f32d1cbd640 1 -- 172.21.15.80:0/1864188205 shutdown_connections 2024-04-24T23:32:43.548 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:43.545+0000 7f32d1cbd640 1 --2- 172.21.15.80:0/1864188205 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f32cc103340 0x7f32cc109bd0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:43.548 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:43.545+0000 7f32d1cbd640 1 --2- 172.21.15.80:0/1864188205 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32cc102980 0x7f32cc102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:43.548 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:43.545+0000 7f32d1cbd640 1 --2- 172.21.15.80:0/1864188205 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f32cc101790 0x7f32cc101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:43.548 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:43.545+0000 7f32d1cbd640 1 -- 172.21.15.80:0/1864188205 >> 172.21.15.80:0/1864188205 conn(0x7f32cc0fcce0 msgr2=0x7f32cc0ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:32:43.549 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:43.545+0000 7f32d1cbd640 1 -- 172.21.15.80:0/1864188205 shutdown_connections 2024-04-24T23:32:43.549 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:43.545+0000 7f32d1cbd640 1 -- 172.21.15.80:0/1864188205 wait complete. 2024-04-24T23:32:43.549 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:43.545+0000 7f32d1cbd640 1 Processor -- start 2024-04-24T23:32:43.549 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:43.545+0000 7f32d1cbd640 1 -- start start 2024-04-24T23:32:43.549 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:43.545+0000 7f32d1cbd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f32cc101790 0x7f32cc0754f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:43.549 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:43.545+0000 7f32d1cbd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f32cc102980 0x7f32cc073b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:43.549 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:43.545+0000 7f32d1cbd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32cc103340 0x7f32cc074070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:43.550 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:43.545+0000 7f32d1cbd640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f32cc10d410 con 0x7f32cc102980 2024-04-24T23:32:43.550 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:43.545+0000 7f32d1cbd640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f32cc10d290 con 0x7f32cc101790 2024-04-24T23:32:43.550 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:43.545+0000 7f32d1cbd640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f32cc10d590 con 0x7f32cc103340 2024-04-24T23:32:43.550 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:43.545+0000 7f32c3fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f32cc102980 0x7f32cc073b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:43.550 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:43.545+0000 7f32c3fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f32cc102980 0x7f32cc073b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:33560/0 (socket says 172.21.15.80:33560) 2024-04-24T23:32:43.550 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:43.545+0000 7f32c3fff640 1 -- 172.21.15.80:0/3635112448 learned_addr learned my addr 172.21.15.80:0/3635112448 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:32:43.550 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:43.545+0000 7f32cb7fe640 1 --2- 172.21.15.80:0/3635112448 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f32cc101790 0x7f32cc0754f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:43.550 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:43.546+0000 7f32cbfff640 1 --2- 172.21.15.80:0/3635112448 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32cc103340 0x7f32cc074070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:43.551 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:43.546+0000 7f32c3fff640 1 -- 172.21.15.80:0/3635112448 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32cc103340 msgr2=0x7f32cc074070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:43.551 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:43.546+0000 7f32c3fff640 1 --2- 172.21.15.80:0/3635112448 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32cc103340 0x7f32cc074070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:43.551 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:43.546+0000 7f32c3fff640 1 -- 172.21.15.80:0/3635112448 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f32cc101790 msgr2=0x7f32cc0754f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:43.551 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:43.546+0000 7f32c3fff640 1 --2- 172.21.15.80:0/3635112448 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f32cc101790 0x7f32cc0754f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:43.551 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:43.546+0000 7f32c3fff640 1 -- 172.21.15.80:0/3635112448 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32cc074900 con 0x7f32cc102980 2024-04-24T23:32:43.551 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:43.546+0000 7f32c3fff640 1 --2- 172.21.15.80:0/3635112448 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f32cc102980 0x7f32cc073b30 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f32b400e9f0 tx=0x7f32b400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:32:43.551 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:43.546+0000 7f32c97fa640 1 -- 172.21.15.80:0/3635112448 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32b400cd80 con 0x7f32cc102980 2024-04-24T23:32:43.551 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:43.546+0000 7f32c97fa640 1 -- 172.21.15.80:0/3635112448 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32b4005500 con 0x7f32cc102980 2024-04-24T23:32:43.552 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:43.546+0000 7f32d1cbd640 1 -- 172.21.15.80:0/3635112448 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32cc1a1b70 con 0x7f32cc102980 2024-04-24T23:32:43.552 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:43.546+0000 7f32c97fa640 1 -- 172.21.15.80:0/3635112448 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32b4010690 con 0x7f32cc102980 2024-04-24T23:32:43.552 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:43.546+0000 7f32d1cbd640 1 -- 172.21.15.80:0/3635112448 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32cc1a2070 con 0x7f32cc102980 2024-04-24T23:32:43.552 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:43.547+0000 7f32d1cbd640 1 -- 172.21.15.80:0/3635112448 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32cc108ab0 con 0x7f32cc102980 2024-04-24T23:32:43.553 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:43.548+0000 7f32c97fa640 1 -- 172.21.15.80:0/3635112448 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f32b40046c0 con 0x7f32cc102980 2024-04-24T23:32:43.553 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:43.549+0000 7f32c97fa640 1 --2- 172.21.15.80:0/3635112448 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f32b80a2b90 0x7f32b80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:43.553 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:43.549+0000 7f32cb7fe640 1 --2- 172.21.15.80:0/3635112448 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f32b80a2b90 0x7f32b80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:43.553 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:43.549+0000 7f32c97fa640 1 -- 172.21.15.80:0/3635112448 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8912+0+0 (secure 0 0 0) 0x7f32b40cf760 con 0x7f32cc102980 2024-04-24T23:32:43.554 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:43.549+0000 7f32cb7fe640 1 --2- 172.21.15.80:0/3635112448 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f32b80a2b90 0x7f32b80a5050 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f32cc1027e0 tx=0x7f32bc00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:32:43.554 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:43.554+0000 7f32c97fa640 1 -- 172.21.15.80:0/3635112448 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f32b409c420 con 0x7f32cc102980 2024-04-24T23:32:43.730 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:43.729+0000 7f32d1cbd640 1 -- 172.21.15.80:0/3635112448 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f32cc02d170 con 0x7f32cc102980 2024-04-24T23:32:44.423 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:44.422+0000 7f32c97fa640 1 -- 172.21.15.80:0/3635112448 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v97) ==== 169+0+0 (secure 0 0 0) 0x7f32b409bd20 con 0x7f32cc102980 2024-04-24T23:32:44.423 INFO:teuthology.orchestra.run.smithi080.stderr:pool 'cephfs_data' removed 2024-04-24T23:32:44.427 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:44.425+0000 7f32d1cbd640 1 -- 172.21.15.80:0/3635112448 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f32b80a2b90 msgr2=0x7f32b80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:44.427 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:44.425+0000 7f32d1cbd640 1 --2- 172.21.15.80:0/3635112448 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f32b80a2b90 0x7f32b80a5050 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f32cc1027e0 tx=0x7f32bc00b390 comp rx=0 tx=0).stop 2024-04-24T23:32:44.427 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:44.425+0000 7f32d1cbd640 1 -- 172.21.15.80:0/3635112448 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f32cc102980 msgr2=0x7f32cc073b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:44.428 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:44.425+0000 7f32d1cbd640 1 --2- 172.21.15.80:0/3635112448 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f32cc102980 0x7f32cc073b30 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f32b400e9f0 tx=0x7f32b400edf0 comp rx=0 tx=0).stop 2024-04-24T23:32:44.428 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:44.425+0000 7f32d1cbd640 1 -- 172.21.15.80:0/3635112448 shutdown_connections 2024-04-24T23:32:44.428 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:44.425+0000 7f32d1cbd640 1 --2- 172.21.15.80:0/3635112448 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7f32b80a2b90 0x7f32b80a5050 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:44.428 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:44.425+0000 7f32d1cbd640 1 --2- 172.21.15.80:0/3635112448 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32cc103340 0x7f32cc074070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:44.428 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:44.425+0000 7f32d1cbd640 1 --2- 172.21.15.80:0/3635112448 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f32cc102980 0x7f32cc073b30 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:44.428 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:44.425+0000 7f32d1cbd640 1 --2- 172.21.15.80:0/3635112448 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f32cc101790 0x7f32cc0754f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:44.429 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:44.425+0000 7f32d1cbd640 1 -- 172.21.15.80:0/3635112448 >> 172.21.15.80:0/3635112448 conn(0x7f32cc0fcce0 msgr2=0x7f32cc105c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:32:44.429 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:44.425+0000 7f32d1cbd640 1 -- 172.21.15.80:0/3635112448 shutdown_connections 2024-04-24T23:32:44.429 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:44.426+0000 7f32d1cbd640 1 -- 172.21.15.80:0/3635112448 wait complete. 2024-04-24T23:32:44.439 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:32:44 smithi080 ceph-mon[29152]: pgmap v463: 129 pgs: 129 active+clean; 75 MiB data, 544 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.8 KiB/s wr, 7 op/s 2024-04-24T23:32:44.440 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:32:44 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/970200061' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-24T23:32:44.440 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:32:44 smithi080 ceph-mon[29152]: osdmap e96: 12 total, 12 up, 12 in 2024-04-24T23:32:44.440 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:32:44 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/3635112448' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-24T23:32:44.441 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-04-24T23:32:44.446 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:32:44 smithi134 ceph-mon[32619]: pgmap v463: 129 pgs: 129 active+clean; 75 MiB data, 544 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.8 KiB/s wr, 7 op/s 2024-04-24T23:32:44.446 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:32:44 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/970200061' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-24T23:32:44.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:32:44 smithi134 ceph-mon[32619]: osdmap e96: 12 total, 12 up, 12 in 2024-04-24T23:32:44.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:32:44 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/3635112448' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-24T23:32:44.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:32:44 smithi148 ceph-mon[32528]: pgmap v463: 129 pgs: 129 active+clean; 75 MiB data, 544 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.8 KiB/s wr, 7 op/s 2024-04-24T23:32:44.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:32:44 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/970200061' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-24T23:32:44.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:32:44 smithi148 ceph-mon[32528]: osdmap e96: 12 total, 12 up, 12 in 2024-04-24T23:32:44.506 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:32:44 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/3635112448' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-24T23:32:44.547 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:44.544+0000 7ff2ddb77640 1 -- 172.21.15.80:0/2848961422 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff2d8100bb0 msgr2=0x7ff2d8100fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:44.547 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:44.544+0000 7ff2ddb77640 1 --2- 172.21.15.80:0/2848961422 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff2d8100bb0 0x7ff2d8100fd0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7ff2c8009900 tx=0x7ff2c802a410 comp rx=0 tx=0).stop 2024-04-24T23:32:44.547 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:44.544+0000 7ff2ddb77640 1 -- 172.21.15.80:0/2848961422 shutdown_connections 2024-04-24T23:32:44.547 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:44.544+0000 7ff2ddb77640 1 --2- 172.21.15.80:0/2848961422 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff2d8102280 0x7ff2d8108b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:44.547 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:44.544+0000 7ff2ddb77640 1 --2- 172.21.15.80:0/2848961422 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff2d81018c0 0x7ff2d8101d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:44.548 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:44.544+0000 7ff2ddb77640 1 --2- 172.21.15.80:0/2848961422 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff2d8100bb0 0x7ff2d8100fd0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:44.548 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:44.544+0000 7ff2ddb77640 1 -- 172.21.15.80:0/2848961422 >> 172.21.15.80:0/2848961422 conn(0x7ff2d80fb380 msgr2=0x7ff2d80fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:32:44.548 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:44.544+0000 7ff2ddb77640 1 -- 172.21.15.80:0/2848961422 shutdown_connections 2024-04-24T23:32:44.548 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:44.544+0000 7ff2ddb77640 1 -- 172.21.15.80:0/2848961422 wait complete. 2024-04-24T23:32:44.548 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:44.544+0000 7ff2ddb77640 1 Processor -- start 2024-04-24T23:32:44.548 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:44.545+0000 7ff2ddb77640 1 -- start start 2024-04-24T23:32:44.548 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:44.545+0000 7ff2ddb77640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff2d8100bb0 0x7ff2d8104660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:44.548 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:44.545+0000 7ff2ddb77640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff2d81018c0 0x7ff2d8102ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:44.549 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:44.545+0000 7ff2ddb77640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff2d8102280 0x7ff2d81031e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:44.549 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:44.545+0000 7ff2ddb77640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7ff2d810c370 con 0x7ff2d8100bb0 2024-04-24T23:32:44.549 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:44.545+0000 7ff2ddb77640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff2d810c1f0 con 0x7ff2d81018c0 2024-04-24T23:32:44.549 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:44.545+0000 7ff2ddb77640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff2d810c4f0 con 0x7ff2d8102280 2024-04-24T23:32:44.549 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:44.545+0000 7ff2d77fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff2d8100bb0 0x7ff2d8104660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:44.549 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:44.545+0000 7ff2d77fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff2d8100bb0 0x7ff2d8104660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:33578/0 (socket says 172.21.15.80:33578) 2024-04-24T23:32:44.549 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:44.545+0000 7ff2d77fe640 1 -- 172.21.15.80:0/1364676042 learned_addr learned my addr 172.21.15.80:0/1364676042 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-24T23:32:44.549 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:44.545+0000 7ff2d7fff640 1 --2- 172.21.15.80:0/1364676042 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff2d8102280 0x7ff2d81031e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:44.549 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:44.545+0000 7ff2d6ffd640 1 --2- 172.21.15.80:0/1364676042 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff2d81018c0 0x7ff2d8102ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:44.550 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:44.545+0000 7ff2d77fe640 1 -- 172.21.15.80:0/1364676042 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff2d8102280 msgr2=0x7ff2d81031e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:44.550 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:44.545+0000 7ff2d77fe640 1 --2- 172.21.15.80:0/1364676042 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff2d8102280 0x7ff2d81031e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:44.550 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:44.545+0000 7ff2d77fe640 1 -- 172.21.15.80:0/1364676042 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff2d81018c0 msgr2=0x7ff2d8102ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:44.550 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:44.545+0000 7ff2d77fe640 1 --2- 172.21.15.80:0/1364676042 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff2d81018c0 0x7ff2d8102ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:44.550 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:44.545+0000 7ff2d77fe640 1 -- 172.21.15.80:0/1364676042 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2d8103a70 con 0x7ff2d8100bb0 2024-04-24T23:32:44.550 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:44.545+0000 7ff2d77fe640 1 --2- 172.21.15.80:0/1364676042 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff2d8100bb0 0x7ff2d8104660 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7ff2c8031610 tx=0x7ff2c80043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:32:44.550 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:44.545+0000 7ff2d4ff9640 1 -- 172.21.15.80:0/1364676042 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2c803f030 con 0x7ff2d8100bb0 2024-04-24T23:32:44.550 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:44.546+0000 7ff2d4ff9640 1 -- 172.21.15.80:0/1364676042 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2c8003ce0 con 0x7ff2d8100bb0 2024-04-24T23:32:44.551 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:44.546+0000 7ff2ddb77640 1 -- 172.21.15.80:0/1364676042 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2d81a1c00 con 0x7ff2d8100bb0 2024-04-24T23:32:44.551 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:44.546+0000 7ff2d4ff9640 1 -- 172.21.15.80:0/1364676042 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2c8035070 con 0x7ff2d8100bb0 2024-04-24T23:32:44.551 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:44.546+0000 7ff2ddb77640 1 -- 172.21.15.80:0/1364676042 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2d81a2150 con 0x7ff2d8100bb0 2024-04-24T23:32:44.551 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:44.546+0000 7ff2ddb77640 1 -- 172.21.15.80:0/1364676042 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2d81a2320 con 0x7ff2d8100bb0 2024-04-24T23:32:44.553 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:44.547+0000 7ff2d4ff9640 1 -- 172.21.15.80:0/1364676042 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff2c8042030 con 0x7ff2d8100bb0 2024-04-24T23:32:44.554 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:44.548+0000 7ff2d4ff9640 1 --2- 172.21.15.80:0/1364676042 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7ff2c00a2b90 0x7ff2c00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:32:44.554 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:44.548+0000 7ff2d6ffd640 1 --2- 172.21.15.80:0/1364676042 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7ff2c00a2b90 0x7ff2c00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:32:44.554 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:44.548+0000 7ff2d4ff9640 1 -- 172.21.15.80:0/1364676042 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8324+0+0 (secure 0 0 0) 0x7ff2c80ef640 con 0x7ff2d8100bb0 2024-04-24T23:32:44.554 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:44.548+0000 7ff2d6ffd640 1 --2- 172.21.15.80:0/1364676042 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7ff2c00a2b90 0x7ff2c00a5050 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc005fd0 tx=0x7ff2cc005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:32:44.554 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:44.553+0000 7ff2d4ff9640 1 -- 172.21.15.80:0/1364676042 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff2c80bc540 con 0x7ff2d8100bb0 2024-04-24T23:32:44.735 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:44.735+0000 7ff2ddb77640 1 -- 172.21.15.80:0/1364676042 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7ff2d81a24f0 con 0x7ff2d8100bb0 2024-04-24T23:32:45.429 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:45.428+0000 7ff2d4ff9640 1 -- 172.21.15.80:0/1364676042 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v98) ==== 178+0+0 (secure 0 0 0) 0x7ff2c80bbe40 con 0x7ff2d8100bb0 2024-04-24T23:32:45.429 INFO:teuthology.orchestra.run.smithi080.stderr:pool 'cephfs_data_ec' removed 2024-04-24T23:32:45.432 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:45.431+0000 7ff2ddb77640 1 -- 172.21.15.80:0/1364676042 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7ff2c00a2b90 msgr2=0x7ff2c00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:45.432 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:45.431+0000 7ff2ddb77640 1 --2- 172.21.15.80:0/1364676042 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7ff2c00a2b90 0x7ff2c00a5050 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc005fd0 tx=0x7ff2cc005850 comp rx=0 tx=0).stop 2024-04-24T23:32:45.432 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:45.431+0000 7ff2ddb77640 1 -- 172.21.15.80:0/1364676042 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff2d8100bb0 msgr2=0x7ff2d8104660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:32:45.432 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:45.431+0000 7ff2ddb77640 1 --2- 172.21.15.80:0/1364676042 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff2d8100bb0 0x7ff2d8104660 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7ff2c8031610 tx=0x7ff2c80043b0 comp rx=0 tx=0).stop 2024-04-24T23:32:45.432 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:45.431+0000 7ff2ddb77640 1 -- 172.21.15.80:0/1364676042 shutdown_connections 2024-04-24T23:32:45.432 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:45.431+0000 7ff2ddb77640 1 --2- 172.21.15.80:0/1364676042 >> [v2:172.21.15.80:6800/3830596922,v1:172.21.15.80:6801/3830596922] conn(0x7ff2c00a2b90 0x7ff2c00a5050 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:45.432 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:45.431+0000 7ff2ddb77640 1 --2- 172.21.15.80:0/1364676042 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff2d8102280 0x7ff2d81031e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:45.433 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:45.431+0000 7ff2ddb77640 1 --2- 172.21.15.80:0/1364676042 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff2d81018c0 0x7ff2d8102ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:45.433 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:45.431+0000 7ff2ddb77640 1 --2- 172.21.15.80:0/1364676042 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff2d8100bb0 0x7ff2d8104660 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:32:45.433 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:45.431+0000 7ff2ddb77640 1 -- 172.21.15.80:0/1364676042 >> 172.21.15.80:0/1364676042 conn(0x7ff2d80fb380 msgr2=0x7ff2d80fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:32:45.433 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:45.431+0000 7ff2ddb77640 1 -- 172.21.15.80:0/1364676042 shutdown_connections 2024-04-24T23:32:45.433 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-24T23:32:45.431+0000 7ff2ddb77640 1 -- 172.21.15.80:0/1364676042 wait complete. 2024-04-24T23:32:45.448 DEBUG:teuthology.orchestra.run.smithi080:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-24T23:32:45.477 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-24T23:32:45.508 DEBUG:teuthology.orchestra.run.smithi148:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-24T23:32:45.539 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-04-24T23:32:45.539 DEBUG:teuthology.orchestra.run.smithi080:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-24T23:32:45.556 DEBUG:teuthology.orchestra.run.smithi134:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-24T23:32:45.573 DEBUG:teuthology.orchestra.run.smithi148:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-24T23:32:45.596 INFO:tasks.cephadm:Stopping all daemons... 2024-04-24T23:32:45.597 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-04-24T23:32:45.597 DEBUG:teuthology.orchestra.run.smithi080:> sudo systemctl stop ceph-ec7885aa-0290-11ef-bc93-c7b262605968@mon.a 2024-04-24T23:32:45.697 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:32:45 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/3635112448' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-24T23:32:45.697 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:32:45 smithi134 ceph-mon[32619]: osdmap e97: 12 total, 12 up, 12 in 2024-04-24T23:32:45.697 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:32:45 smithi134 ceph-mon[32619]: from='client.? 172.21.15.80:0/1364676042' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-24T23:32:45.697 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:32:45 smithi134 ceph-mon[32619]: pgmap v466: 33 pgs: 33 active+clean; 26 MiB data, 544 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:32:45.756 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:32:45 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/3635112448' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-24T23:32:45.756 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:32:45 smithi148 ceph-mon[32528]: osdmap e97: 12 total, 12 up, 12 in 2024-04-24T23:32:45.756 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:32:45 smithi148 ceph-mon[32528]: from='client.? 172.21.15.80:0/1364676042' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-24T23:32:45.756 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:32:45 smithi148 ceph-mon[32528]: pgmap v466: 33 pgs: 33 active+clean; 26 MiB data, 544 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:32:45.789 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:32:45 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/3635112448' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-24T23:32:45.789 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:32:45 smithi080 ceph-mon[29152]: osdmap e97: 12 total, 12 up, 12 in 2024-04-24T23:32:45.789 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:32:45 smithi080 ceph-mon[29152]: from='client.? 172.21.15.80:0/1364676042' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-24T23:32:45.789 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:32:45 smithi080 ceph-mon[29152]: pgmap v466: 33 pgs: 33 active+clean; 26 MiB data, 544 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:32:45.790 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:32:45 smithi080 systemd[1]: Stopping Ceph mon.a for ec7885aa-0290-11ef-bc93-c7b262605968... 2024-04-24T23:32:46.183 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:32:45 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mon-a[29148]: 2024-04-24T23:32:45.917+0000 7f126b62e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:32:46.183 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:32:45 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mon-a[29148]: 2024-04-24T23:32:45.917+0000 7f126b62e700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-04-24T23:32:46.477 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:32:46 smithi080 podman[94102]: 2024-04-24 23:32:46.183287017 +0000 UTC m=+0.400897838 container died 789fd293dfe79fb3cb1d886b44b1ee6d45add80bc3ec75b005da5e8f3b3fefb6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mon-a, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.vendor=CentOS) 2024-04-24T23:32:46.788 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:32:46 smithi080 podman[94102]: 2024-04-24 23:32:46.685897434 +0000 UTC m=+0.903508305 container cleanup 789fd293dfe79fb3cb1d886b44b1ee6d45add80bc3ec75b005da5e8f3b3fefb6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mon-a, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS) 2024-04-24T23:32:46.788 INFO:journalctl@ceph.mon.a.smithi080.stdout:Apr 24 23:32:46 smithi080 bash[94102]: ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mon-a 2024-04-24T23:32:47.105 DEBUG:teuthology.orchestra.run.smithi080:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ec7885aa-0290-11ef-bc93-c7b262605968@mon.a.service' 2024-04-24T23:32:47.153 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:32:47.154 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-04-24T23:32:47.154 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-04-24T23:32:47.154 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-ec7885aa-0290-11ef-bc93-c7b262605968@mon.b 2024-04-24T23:32:47.446 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:32:47 smithi134 systemd[1]: Stopping Ceph mon.b for ec7885aa-0290-11ef-bc93-c7b262605968... 2024-04-24T23:32:47.934 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:32:47 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mon-b[32614]: 2024-04-24T23:32:47.470+0000 7efc7afe6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:32:47.934 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:32:47 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mon-b[32614]: 2024-04-24T23:32:47.470+0000 7efc7afe6700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-04-24T23:32:47.934 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:32:47 smithi134 podman[66345]: 2024-04-24 23:32:47.634203224 +0000 UTC m=+0.288334122 container died a4ce1fe4d7fd256abfad81381f2bb84ffc10f22fd4c23ae717736001ca0c4eef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mon-b, GIT_CLEAN=True, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-04-24T23:32:48.196 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:32:48 smithi134 podman[66345]: 2024-04-24 23:32:48.032798619 +0000 UTC m=+0.686929527 container cleanup a4ce1fe4d7fd256abfad81381f2bb84ffc10f22fd4c23ae717736001ca0c4eef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mon-b, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-04-24T23:32:48.197 INFO:journalctl@ceph.mon.b.smithi134.stdout:Apr 24 23:32:48 smithi134 bash[66345]: ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mon-b 2024-04-24T23:32:48.446 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ec7885aa-0290-11ef-bc93-c7b262605968@mon.b.service' 2024-04-24T23:32:48.490 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:32:48.490 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-04-24T23:32:48.491 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-04-24T23:32:48.491 DEBUG:teuthology.orchestra.run.smithi148:> sudo systemctl stop ceph-ec7885aa-0290-11ef-bc93-c7b262605968@mon.c 2024-04-24T23:32:48.966 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:32:48 smithi148 systemd[1]: Stopping Ceph mon.c for ec7885aa-0290-11ef-bc93-c7b262605968... 2024-04-24T23:32:49.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:32:48 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mon-c[32524]: 2024-04-24T23:32:48.965+0000 7fe87ca82700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:32:49.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:32:48 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mon-c[32524]: 2024-04-24T23:32:48.965+0000 7fe87ca82700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-04-24T23:32:49.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:32:49 smithi148 podman[66667]: 2024-04-24 23:32:49.119222032 +0000 UTC m=+0.425523387 container died 24ebdbf62b15b1033b329f642fc39a8fd54a71232db0bd11bc5db0e6184d3ca1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mon-c, GIT_CLEAN=True, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424) 2024-04-24T23:32:49.756 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:32:49 smithi148 podman[66667]: 2024-04-24 23:32:49.541048736 +0000 UTC m=+0.847350088 container cleanup 24ebdbf62b15b1033b329f642fc39a8fd54a71232db0bd11bc5db0e6184d3ca1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mon-c, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-04-24T23:32:49.756 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:32:49 smithi148 bash[66667]: ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mon-c 2024-04-24T23:32:50.034 DEBUG:teuthology.orchestra.run.smithi148:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ec7885aa-0290-11ef-bc93-c7b262605968@mon.c.service' 2024-04-24T23:32:50.055 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:32:49 smithi148 podman[66681]: 2024-04-24 23:32:49.801094274 +0000 UTC m=+0.678462433 container remove 24ebdbf62b15b1033b329f642fc39a8fd54a71232db0bd11bc5db0e6184d3ca1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mon-c, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T23:32:50.055 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:32:50 smithi148 systemd[1]: ceph-ec7885aa-0290-11ef-bc93-c7b262605968@mon.c.service: Deactivated successfully. 2024-04-24T23:32:50.056 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:32:50 smithi148 systemd[1]: Stopped Ceph mon.c for ec7885aa-0290-11ef-bc93-c7b262605968. 2024-04-24T23:32:50.056 INFO:journalctl@ceph.mon.c.smithi148.stdout:Apr 24 23:32:50 smithi148 systemd[1]: ceph-ec7885aa-0290-11ef-bc93-c7b262605968@mon.c.service: Consumed 8.569s CPU time. 2024-04-24T23:32:50.091 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:32:50.091 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-04-24T23:32:50.091 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-04-24T23:32:50.091 DEBUG:teuthology.orchestra.run.smithi080:> sudo systemctl stop ceph-ec7885aa-0290-11ef-bc93-c7b262605968@mgr.x 2024-04-24T23:32:50.538 INFO:journalctl@ceph.mgr.x.smithi080.stdout:Apr 24 23:32:50 smithi080 systemd[1]: Stopping Ceph mgr.x for ec7885aa-0290-11ef-bc93-c7b262605968... 2024-04-24T23:32:51.034 INFO:journalctl@ceph.mgr.x.smithi080.stdout:Apr 24 23:32:50 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-x[29381]: 2024-04-24T23:32:50.565+0000 7f5da8aba700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:32:51.034 INFO:journalctl@ceph.mgr.x.smithi080.stdout:Apr 24 23:32:50 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-x[29381]: 2024-04-24T23:32:50.565+0000 7f5da8aba700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-04-24T23:32:51.034 INFO:journalctl@ceph.mgr.x.smithi080.stdout:Apr 24 23:32:50 smithi080 podman[94223]: 2024-04-24 23:32:50.596959835 +0000 UTC m=+0.318729548 container died 3dde3e4c4b2190d55d7dcb85dcd421b9d724caa4f64659459f63e1a8ae953b65 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-04-24T23:32:51.288 INFO:journalctl@ceph.mgr.x.smithi080.stdout:Apr 24 23:32:51 smithi080 podman[94223]: 2024-04-24 23:32:51.192529819 +0000 UTC m=+0.914299536 container cleanup 3dde3e4c4b2190d55d7dcb85dcd421b9d724caa4f64659459f63e1a8ae953b65 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-x, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True) 2024-04-24T23:32:51.288 INFO:journalctl@ceph.mgr.x.smithi080.stdout:Apr 24 23:32:51 smithi080 bash[94223]: ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-x 2024-04-24T23:32:51.618 DEBUG:teuthology.orchestra.run.smithi080:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ec7885aa-0290-11ef-bc93-c7b262605968@mgr.x.service' 2024-04-24T23:32:51.654 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:32:51.654 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-04-24T23:32:51.655 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-04-24T23:32:51.655 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-ec7885aa-0290-11ef-bc93-c7b262605968@mgr.y 2024-04-24T23:32:51.947 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Apr 24 23:32:51 smithi134 systemd[1]: Stopping Ceph mgr.y for ec7885aa-0290-11ef-bc93-c7b262605968... 2024-04-24T23:32:52.286 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Apr 24 23:32:51 smithi134 podman[66469]: 2024-04-24 23:32:51.9841011 +0000 UTC m=+0.140630711 container died bb25bcb9fea2a95e0c824e75824ffd3729f5175c8e9d2485d8763fc06384cc7b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-y, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9) 2024-04-24T23:32:52.648 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Apr 24 23:32:52 smithi134 podman[66469]: 2024-04-24 23:32:52.397422853 +0000 UTC m=+0.553952454 container cleanup bb25bcb9fea2a95e0c824e75824ffd3729f5175c8e9d2485d8763fc06384cc7b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-y, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T23:32:52.648 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Apr 24 23:32:52 smithi134 bash[66469]: ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-y 2024-04-24T23:32:52.813 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ec7885aa-0290-11ef-bc93-c7b262605968@mgr.y.service' 2024-04-24T23:32:52.846 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:32:52.846 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-04-24T23:32:52.846 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-04-24T23:32:52.846 DEBUG:teuthology.orchestra.run.smithi148:> sudo systemctl stop ceph-ec7885aa-0290-11ef-bc93-c7b262605968@mgr.z 2024-04-24T23:32:53.183 INFO:journalctl@ceph.mgr.z.smithi148.stdout:Apr 24 23:32:52 smithi148 systemd[1]: Stopping Ceph mgr.z for ec7885aa-0290-11ef-bc93-c7b262605968... 2024-04-24T23:32:53.489 INFO:journalctl@ceph.mgr.z.smithi148.stdout:Apr 24 23:32:53 smithi148 podman[66789]: 2024-04-24 23:32:53.183526246 +0000 UTC m=+0.145470411 container died fd286875e6f7f93993588cf75e70c73f17f7ef907104f4344627932dd722baa4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-z, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-04-24T23:32:53.755 INFO:journalctl@ceph.mgr.z.smithi148.stdout:Apr 24 23:32:53 smithi148 podman[66789]: 2024-04-24 23:32:53.588836258 +0000 UTC m=+0.550780422 container cleanup fd286875e6f7f93993588cf75e70c73f17f7ef907104f4344627932dd722baa4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-z, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-24T23:32:53.756 INFO:journalctl@ceph.mgr.z.smithi148.stdout:Apr 24 23:32:53 smithi148 bash[66789]: ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-z 2024-04-24T23:32:54.044 INFO:journalctl@ceph.mgr.z.smithi148.stdout:Apr 24 23:32:54 smithi148 podman[66803]: 2024-04-24 23:32:54.040586296 +0000 UTC m=+0.850279287 container remove fd286875e6f7f93993588cf75e70c73f17f7ef907104f4344627932dd722baa4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mgr-z, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T23:32:54.165 DEBUG:teuthology.orchestra.run.smithi148:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ec7885aa-0290-11ef-bc93-c7b262605968@mgr.z.service' 2024-04-24T23:32:54.202 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:32:54.202 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-04-24T23:32:54.202 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-04-24T23:32:54.202 DEBUG:teuthology.orchestra.run.smithi080:> sudo systemctl stop ceph-ec7885aa-0290-11ef-bc93-c7b262605968@osd.0 2024-04-24T23:32:54.538 INFO:journalctl@ceph.osd.0.smithi080.stdout:Apr 24 23:32:54 smithi080 systemd[1]: Stopping Ceph osd.0 for ec7885aa-0290-11ef-bc93-c7b262605968... 2024-04-24T23:32:55.038 INFO:journalctl@ceph.osd.0.smithi080.stdout:Apr 24 23:32:54 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-0[43667]: 2024-04-24T23:32:54.679+0000 7fb3bb064700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:32:55.038 INFO:journalctl@ceph.osd.0.smithi080.stdout:Apr 24 23:32:54 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-0[43667]: 2024-04-24T23:32:54.679+0000 7fb3bb064700 -1 osd.0 98 *** Got signal Terminated *** 2024-04-24T23:32:55.038 INFO:journalctl@ceph.osd.0.smithi080.stdout:Apr 24 23:32:54 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-0[43667]: 2024-04-24T23:32:54.679+0000 7fb3bb064700 -1 osd.0 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T23:33:00.281 INFO:journalctl@ceph.osd.0.smithi080.stdout:Apr 24 23:33:00 smithi080 podman[94343]: 2024-04-24 23:33:00.003591432 +0000 UTC m=+5.600299995 container died 1328aa490536aefbca81513e77648388c23124a88f221f8d038e2a4b0c4ffaa9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424) 2024-04-24T23:33:00.538 INFO:journalctl@ceph.osd.0.smithi080.stdout:Apr 24 23:33:00 smithi080 podman[94343]: 2024-04-24 23:33:00.422801065 +0000 UTC m=+6.019509611 container cleanup 1328aa490536aefbca81513e77648388c23124a88f221f8d038e2a4b0c4ffaa9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-0, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424) 2024-04-24T23:33:00.538 INFO:journalctl@ceph.osd.0.smithi080.stdout:Apr 24 23:33:00 smithi080 bash[94343]: ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-0 2024-04-24T23:33:01.038 INFO:journalctl@ceph.osd.0.smithi080.stdout:Apr 24 23:33:00 smithi080 podman[94361]: 2024-04-24 23:33:00.691337187 +0000 UTC m=+0.681188222 container remove 1328aa490536aefbca81513e77648388c23124a88f221f8d038e2a4b0c4ffaa9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T23:33:01.436 INFO:journalctl@ceph.osd.0.smithi080.stdout:Apr 24 23:33:01 smithi080 podman[94453]: 2024-04-24 23:33:01.003107778 +0000 UTC m=+0.018253041 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:33:01.437 INFO:journalctl@ceph.osd.0.smithi080.stdout:Apr 24 23:33:01 smithi080 podman[94453]: 2024-04-24 23:33:01.109092015 +0000 UTC m=+0.124237255 container create 11713f18ca77557b31db3bfb895c05b938441fefd55760834cc96e6cc5526912 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-0-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, ceph=True, GIT_CLEAN=True, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T23:33:01.437 INFO:journalctl@ceph.osd.0.smithi080.stdout:Apr 24 23:33:01 smithi080 podman[94453]: 2024-04-24 23:33:01.337143648 +0000 UTC m=+0.352288898 container init 11713f18ca77557b31db3bfb895c05b938441fefd55760834cc96e6cc5526912 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-04-24T23:33:01.437 INFO:journalctl@ceph.osd.0.smithi080.stdout:Apr 24 23:33:01 smithi080 podman[94453]: 2024-04-24 23:33:01.340519761 +0000 UTC m=+0.355665008 container start 11713f18ca77557b31db3bfb895c05b938441fefd55760834cc96e6cc5526912 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-04-24T23:33:01.437 INFO:journalctl@ceph.osd.0.smithi080.stdout:Apr 24 23:33:01 smithi080 podman[94453]: 2024-04-24 23:33:01.43680228 +0000 UTC m=+0.451947530 container attach 11713f18ca77557b31db3bfb895c05b938441fefd55760834cc96e6cc5526912 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True) 2024-04-24T23:33:01.788 INFO:journalctl@ceph.osd.0.smithi080.stdout:Apr 24 23:33:01 smithi080 podman[94453]: 2024-04-24 23:33:01.638372483 +0000 UTC m=+0.653517722 container died 11713f18ca77557b31db3bfb895c05b938441fefd55760834cc96e6cc5526912 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-0-deactivate, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T23:33:02.313 DEBUG:teuthology.orchestra.run.smithi080:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ec7885aa-0290-11ef-bc93-c7b262605968@osd.0.service' 2024-04-24T23:33:02.353 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:33:02.353 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-04-24T23:33:02.353 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-04-24T23:33:02.353 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-ec7885aa-0290-11ef-bc93-c7b262605968@osd.1 2024-04-24T23:33:02.660 INFO:journalctl@ceph.osd.1.smithi134.stdout:Apr 24 23:33:02 smithi134 systemd[1]: Stopping Ceph osd.1 for ec7885aa-0290-11ef-bc93-c7b262605968... 2024-04-24T23:33:02.946 INFO:journalctl@ceph.osd.1.smithi134.stdout:Apr 24 23:33:02 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-1[37250]: 2024-04-24T23:33:02.659+0000 7f9bf4ac5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:33:02.947 INFO:journalctl@ceph.osd.1.smithi134.stdout:Apr 24 23:33:02 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-1[37250]: 2024-04-24T23:33:02.659+0000 7f9bf4ac5700 -1 osd.1 98 *** Got signal Terminated *** 2024-04-24T23:33:02.947 INFO:journalctl@ceph.osd.1.smithi134.stdout:Apr 24 23:33:02 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-1[37250]: 2024-04-24T23:33:02.659+0000 7f9bf4ac5700 -1 osd.1 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T23:33:08.303 INFO:journalctl@ceph.osd.1.smithi134.stdout:Apr 24 23:33:08 smithi134 podman[66606]: 2024-04-24 23:33:08.014382431 +0000 UTC m=+5.473727699 container died d344c11510ccf6b8fa434e8008ade1dabdcc1bb18ca394bc0a181bad15362e91 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-1, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-04-24T23:33:08.686 INFO:journalctl@ceph.osd.1.smithi134.stdout:Apr 24 23:33:08 smithi134 podman[66606]: 2024-04-24 23:33:08.401993529 +0000 UTC m=+5.861338838 container cleanup d344c11510ccf6b8fa434e8008ade1dabdcc1bb18ca394bc0a181bad15362e91 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-1, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-04-24T23:33:08.686 INFO:journalctl@ceph.osd.1.smithi134.stdout:Apr 24 23:33:08 smithi134 bash[66606]: ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-1 2024-04-24T23:33:08.947 INFO:journalctl@ceph.osd.1.smithi134.stdout:Apr 24 23:33:08 smithi134 podman[66623]: 2024-04-24 23:33:08.686788679 +0000 UTC m=+0.668893071 container remove d344c11510ccf6b8fa434e8008ade1dabdcc1bb18ca394bc0a181bad15362e91 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-1, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9) 2024-04-24T23:33:09.324 INFO:journalctl@ceph.osd.1.smithi134.stdout:Apr 24 23:33:09 smithi134 podman[66713]: 2024-04-24 23:33:08.920036964 +0000 UTC m=+0.017966201 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:33:09.324 INFO:journalctl@ceph.osd.1.smithi134.stdout:Apr 24 23:33:09 smithi134 podman[66713]: 2024-04-24 23:33:09.062783831 +0000 UTC m=+0.160713051 container create 6b037694453ff221e857a6ff5c69927bdb4d9d091af971851f7ebd0828c2a0c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-1-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-04-24T23:33:09.622 INFO:journalctl@ceph.osd.1.smithi134.stdout:Apr 24 23:33:09 smithi134 podman[66713]: 2024-04-24 23:33:09.324583201 +0000 UTC m=+0.422512423 container init 6b037694453ff221e857a6ff5c69927bdb4d9d091af971851f7ebd0828c2a0c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-1-deactivate, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T23:33:09.622 INFO:journalctl@ceph.osd.1.smithi134.stdout:Apr 24 23:33:09 smithi134 podman[66713]: 2024-04-24 23:33:09.328221041 +0000 UTC m=+0.426150265 container start 6b037694453ff221e857a6ff5c69927bdb4d9d091af971851f7ebd0828c2a0c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T23:33:09.623 INFO:journalctl@ceph.osd.1.smithi134.stdout:Apr 24 23:33:09 smithi134 podman[66713]: 2024-04-24 23:33:09.424770582 +0000 UTC m=+0.522699814 container attach 6b037694453ff221e857a6ff5c69927bdb4d9d091af971851f7ebd0828c2a0c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, RELEASE=squid-d2c45d9, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T23:33:09.944 INFO:journalctl@ceph.osd.1.smithi134.stdout:Apr 24 23:33:09 smithi134 podman[66713]: 2024-04-24 23:33:09.623298997 +0000 UTC m=+0.721228229 container died 6b037694453ff221e857a6ff5c69927bdb4d9d091af971851f7ebd0828c2a0c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=) 2024-04-24T23:33:10.286 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ec7885aa-0290-11ef-bc93-c7b262605968@osd.1.service' 2024-04-24T23:33:10.328 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:33:10.328 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-04-24T23:33:10.328 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-04-24T23:33:10.329 DEBUG:teuthology.orchestra.run.smithi148:> sudo systemctl stop ceph-ec7885aa-0290-11ef-bc93-c7b262605968@osd.2 2024-04-24T23:33:10.643 INFO:journalctl@ceph.osd.2.smithi148.stdout:Apr 24 23:33:10 smithi148 systemd[1]: Stopping Ceph osd.2 for ec7885aa-0290-11ef-bc93-c7b262605968... 2024-04-24T23:33:11.006 INFO:journalctl@ceph.osd.2.smithi148.stdout:Apr 24 23:33:10 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-2[37716]: 2024-04-24T23:33:10.642+0000 7f58270d6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:33:11.006 INFO:journalctl@ceph.osd.2.smithi148.stdout:Apr 24 23:33:10 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-2[37716]: 2024-04-24T23:33:10.642+0000 7f58270d6700 -1 osd.2 98 *** Got signal Terminated *** 2024-04-24T23:33:11.006 INFO:journalctl@ceph.osd.2.smithi148.stdout:Apr 24 23:33:10 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-2[37716]: 2024-04-24T23:33:10.642+0000 7f58270d6700 -1 osd.2 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T23:33:16.256 INFO:journalctl@ceph.osd.2.smithi148.stdout:Apr 24 23:33:15 smithi148 podman[66913]: 2024-04-24 23:33:15.973640321 +0000 UTC m=+5.448764447 container died 61e90c42b2965ba67774777dd203cada07acbe3ec431271d7c4a9cc568a58af3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-2, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-04-24T23:33:16.628 INFO:journalctl@ceph.osd.2.smithi148.stdout:Apr 24 23:33:16 smithi148 podman[66913]: 2024-04-24 23:33:16.37651369 +0000 UTC m=+5.851637815 container cleanup 61e90c42b2965ba67774777dd203cada07acbe3ec431271d7c4a9cc568a58af3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-2, GIT_CLEAN=True, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424) 2024-04-24T23:33:16.629 INFO:journalctl@ceph.osd.2.smithi148.stdout:Apr 24 23:33:16 smithi148 bash[66913]: ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-2 2024-04-24T23:33:16.945 INFO:journalctl@ceph.osd.2.smithi148.stdout:Apr 24 23:33:16 smithi148 podman[66930]: 2024-04-24 23:33:16.629163683 +0000 UTC m=+0.651402181 container remove 61e90c42b2965ba67774777dd203cada07acbe3ec431271d7c4a9cc568a58af3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-2, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux ) 2024-04-24T23:33:17.242 INFO:journalctl@ceph.osd.2.smithi148.stdout:Apr 24 23:33:16 smithi148 podman[67021]: 2024-04-24 23:33:16.848318083 +0000 UTC m=+0.018009831 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:33:17.242 INFO:journalctl@ceph.osd.2.smithi148.stdout:Apr 24 23:33:16 smithi148 podman[67021]: 2024-04-24 23:33:16.980615573 +0000 UTC m=+0.150307389 container create f5f0b37130c96888990751344757de87bd10b56a279d7e0b761ff63b4ed2120b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T23:33:17.242 INFO:journalctl@ceph.osd.2.smithi148.stdout:Apr 24 23:33:17 smithi148 podman[67021]: 2024-04-24 23:33:17.242369991 +0000 UTC m=+0.412061748 container init f5f0b37130c96888990751344757de87bd10b56a279d7e0b761ff63b4ed2120b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True) 2024-04-24T23:33:17.506 INFO:journalctl@ceph.osd.2.smithi148.stdout:Apr 24 23:33:17 smithi148 podman[67021]: 2024-04-24 23:33:17.245981752 +0000 UTC m=+0.415673502 container start f5f0b37130c96888990751344757de87bd10b56a279d7e0b761ff63b4ed2120b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-04-24T23:33:17.506 INFO:journalctl@ceph.osd.2.smithi148.stdout:Apr 24 23:33:17 smithi148 podman[67021]: 2024-04-24 23:33:17.342117991 +0000 UTC m=+0.511809752 container attach f5f0b37130c96888990751344757de87bd10b56a279d7e0b761ff63b4ed2120b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-04-24T23:33:17.929 INFO:journalctl@ceph.osd.2.smithi148.stdout:Apr 24 23:33:17 smithi148 podman[67021]: 2024-04-24 23:33:17.543244882 +0000 UTC m=+0.712936626 container died f5f0b37130c96888990751344757de87bd10b56a279d7e0b761ff63b4ed2120b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-04-24T23:33:18.328 DEBUG:teuthology.orchestra.run.smithi148:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ec7885aa-0290-11ef-bc93-c7b262605968@osd.2.service' 2024-04-24T23:33:18.366 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:33:18.366 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-04-24T23:33:18.366 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-04-24T23:33:18.367 DEBUG:teuthology.orchestra.run.smithi080:> sudo systemctl stop ceph-ec7885aa-0290-11ef-bc93-c7b262605968@osd.3 2024-04-24T23:33:18.684 INFO:journalctl@ceph.osd.3.smithi080.stdout:Apr 24 23:33:18 smithi080 systemd[1]: Stopping Ceph osd.3 for ec7885aa-0290-11ef-bc93-c7b262605968... 2024-04-24T23:33:19.038 INFO:journalctl@ceph.osd.3.smithi080.stdout:Apr 24 23:33:18 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-3[50507]: 2024-04-24T23:33:18.683+0000 7f269da83700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:33:19.038 INFO:journalctl@ceph.osd.3.smithi080.stdout:Apr 24 23:33:18 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-3[50507]: 2024-04-24T23:33:18.683+0000 7f269da83700 -1 osd.3 98 *** Got signal Terminated *** 2024-04-24T23:33:19.038 INFO:journalctl@ceph.osd.3.smithi080.stdout:Apr 24 23:33:18 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-3[50507]: 2024-04-24T23:33:18.683+0000 7f269da83700 -1 osd.3 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T23:33:21.506 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:33:21 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:33:21.252+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.848330+0000 front 2024-04-24T23:32:55.848387+0000 (oldest deadline 2024-04-24T23:33:20.548352+0000) 2024-04-24T23:33:21.697 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:33:21 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:33:21.337+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:56.929716+0000 front 2024-04-24T23:32:56.929774+0000 (oldest deadline 2024-04-24T23:33:20.429679+0000) 2024-04-24T23:33:21.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:33:21 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:33:21.478+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.738540+0000 front 2024-04-24T23:32:55.738561+0000 (oldest deadline 2024-04-24T23:33:21.038786+0000) 2024-04-24T23:33:22.006 INFO:journalctl@ceph.osd.5.smithi148.stdout:Apr 24 23:33:21 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-5[44694]: 2024-04-24T23:33:21.606+0000 7f6fcf9cd700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.322016+0000 front 2024-04-24T23:32:55.321839+0000 (oldest deadline 2024-04-24T23:33:21.221915+0000) 2024-04-24T23:33:22.559 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:33:22 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:33:22.257+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.848330+0000 front 2024-04-24T23:32:55.848387+0000 (oldest deadline 2024-04-24T23:33:20.548352+0000) 2024-04-24T23:33:22.697 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:33:22 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:33:22.356+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:56.929716+0000 front 2024-04-24T23:32:56.929774+0000 (oldest deadline 2024-04-24T23:33:20.429679+0000) 2024-04-24T23:33:22.787 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:33:22 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:33:22.462+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.738540+0000 front 2024-04-24T23:32:55.738561+0000 (oldest deadline 2024-04-24T23:33:21.038786+0000) 2024-04-24T23:33:23.005 INFO:journalctl@ceph.osd.5.smithi148.stdout:Apr 24 23:33:22 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-5[44694]: 2024-04-24T23:33:22.558+0000 7f6fcf9cd700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.322016+0000 front 2024-04-24T23:32:55.321839+0000 (oldest deadline 2024-04-24T23:33:21.221915+0000) 2024-04-24T23:33:23.287 INFO:journalctl@ceph.osd.3.smithi080.stdout:Apr 24 23:33:22 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-3[50507]: 2024-04-24T23:33:22.845+0000 7f269606f700 -1 osd.3 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:57.407358+0000 front 2024-04-24T23:32:57.407382+0000 (oldest deadline 2024-04-24T23:33:22.107453+0000) 2024-04-24T23:33:23.505 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:33:23 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:33:23.252+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.848330+0000 front 2024-04-24T23:32:55.848387+0000 (oldest deadline 2024-04-24T23:33:20.548352+0000) 2024-04-24T23:33:23.696 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:33:23 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:33:23.376+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:56.929716+0000 front 2024-04-24T23:32:56.929774+0000 (oldest deadline 2024-04-24T23:33:20.429679+0000) 2024-04-24T23:33:23.787 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:33:23 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:33:23.488+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.738540+0000 front 2024-04-24T23:32:55.738561+0000 (oldest deadline 2024-04-24T23:33:21.038786+0000) 2024-04-24T23:33:24.006 INFO:journalctl@ceph.osd.5.smithi148.stdout:Apr 24 23:33:23 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-5[44694]: 2024-04-24T23:33:23.521+0000 7f6fcf9cd700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.322016+0000 front 2024-04-24T23:32:55.321839+0000 (oldest deadline 2024-04-24T23:33:21.221915+0000) 2024-04-24T23:33:24.385 INFO:journalctl@ceph.osd.3.smithi080.stdout:Apr 24 23:33:24 smithi080 podman[94569]: 2024-04-24 23:33:24.106333018 +0000 UTC m=+5.542602632 container died 88c0764dff1624d7786a969d9656fc8587695a8a9760f801baa8ddacbe82b5cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-3, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T23:33:24.567 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:33:24 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:33:24.298+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.848330+0000 front 2024-04-24T23:32:55.848387+0000 (oldest deadline 2024-04-24T23:33:20.548352+0000) 2024-04-24T23:33:24.696 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:33:24 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:33:24.413+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:56.929716+0000 front 2024-04-24T23:32:56.929774+0000 (oldest deadline 2024-04-24T23:33:20.429679+0000) 2024-04-24T23:33:24.787 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:33:24 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:33:24.441+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.738540+0000 front 2024-04-24T23:32:55.738561+0000 (oldest deadline 2024-04-24T23:33:21.038786+0000) 2024-04-24T23:33:24.788 INFO:journalctl@ceph.osd.3.smithi080.stdout:Apr 24 23:33:24 smithi080 podman[94569]: 2024-04-24 23:33:24.59318522 +0000 UTC m=+6.029454878 container cleanup 88c0764dff1624d7786a969d9656fc8587695a8a9760f801baa8ddacbe82b5cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-3, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-04-24T23:33:24.788 INFO:journalctl@ceph.osd.3.smithi080.stdout:Apr 24 23:33:24 smithi080 bash[94569]: ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-3 2024-04-24T23:33:25.005 INFO:journalctl@ceph.osd.5.smithi148.stdout:Apr 24 23:33:24 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-5[44694]: 2024-04-24T23:33:24.567+0000 7f6fcf9cd700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.322016+0000 front 2024-04-24T23:32:55.321839+0000 (oldest deadline 2024-04-24T23:33:21.221915+0000) 2024-04-24T23:33:25.145 INFO:journalctl@ceph.osd.3.smithi080.stdout:Apr 24 23:33:24 smithi080 podman[94587]: 2024-04-24 23:33:24.811212626 +0000 UTC m=+0.705364957 container remove 88c0764dff1624d7786a969d9656fc8587695a8a9760f801baa8ddacbe82b5cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424) 2024-04-24T23:33:25.414 INFO:journalctl@ceph.osd.3.smithi080.stdout:Apr 24 23:33:25 smithi080 podman[94676]: 2024-04-24 23:33:25.049061788 +0000 UTC m=+0.018233528 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:33:25.415 INFO:journalctl@ceph.osd.3.smithi080.stdout:Apr 24 23:33:25 smithi080 podman[94676]: 2024-04-24 23:33:25.178860951 +0000 UTC m=+0.148032682 container create f6691515faf8fce07578dba2fa8138d602dc92c120e4918f56304c3505c29d08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS) 2024-04-24T23:33:25.505 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:33:25 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:33:25.254+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.848330+0000 front 2024-04-24T23:32:55.848387+0000 (oldest deadline 2024-04-24T23:33:20.548352+0000) 2024-04-24T23:33:25.696 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:33:25 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:33:25.375+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:56.929716+0000 front 2024-04-24T23:32:56.929774+0000 (oldest deadline 2024-04-24T23:33:20.429679+0000) 2024-04-24T23:33:25.708 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:33:25 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:33:25.426+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.738540+0000 front 2024-04-24T23:32:55.738561+0000 (oldest deadline 2024-04-24T23:33:21.038786+0000) 2024-04-24T23:33:25.708 INFO:journalctl@ceph.osd.3.smithi080.stdout:Apr 24 23:33:25 smithi080 podman[94676]: 2024-04-24 23:33:25.414760148 +0000 UTC m=+0.383931882 container init f6691515faf8fce07578dba2fa8138d602dc92c120e4918f56304c3505c29d08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-04-24T23:33:25.708 INFO:journalctl@ceph.osd.3.smithi080.stdout:Apr 24 23:33:25 smithi080 podman[94676]: 2024-04-24 23:33:25.418492712 +0000 UTC m=+0.387664444 container start f6691515faf8fce07578dba2fa8138d602dc92c120e4918f56304c3505c29d08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5) 2024-04-24T23:33:25.708 INFO:journalctl@ceph.osd.3.smithi080.stdout:Apr 24 23:33:25 smithi080 podman[94676]: 2024-04-24 23:33:25.51488307 +0000 UTC m=+0.484054813 container attach f6691515faf8fce07578dba2fa8138d602dc92c120e4918f56304c3505c29d08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-04-24T23:33:26.005 INFO:journalctl@ceph.osd.5.smithi148.stdout:Apr 24 23:33:25 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-5[44694]: 2024-04-24T23:33:25.566+0000 7f6fcf9cd700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.322016+0000 front 2024-04-24T23:32:55.321839+0000 (oldest deadline 2024-04-24T23:33:21.221915+0000) 2024-04-24T23:33:26.038 INFO:journalctl@ceph.osd.3.smithi080.stdout:Apr 24 23:33:25 smithi080 podman[94676]: 2024-04-24 23:33:25.708807658 +0000 UTC m=+0.677979395 container died f6691515faf8fce07578dba2fa8138d602dc92c120e4918f56304c3505c29d08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-04-24T23:33:26.505 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:33:26 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:33:26.222+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.848330+0000 front 2024-04-24T23:32:55.848387+0000 (oldest deadline 2024-04-24T23:33:20.548352+0000) 2024-04-24T23:33:26.510 DEBUG:teuthology.orchestra.run.smithi080:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ec7885aa-0290-11ef-bc93-c7b262605968@osd.3.service' 2024-04-24T23:33:26.548 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:33:26.548 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-04-24T23:33:26.548 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-04-24T23:33:26.549 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-ec7885aa-0290-11ef-bc93-c7b262605968@osd.4 2024-04-24T23:33:26.598 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:33:26 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:33:26.343+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:56.929716+0000 front 2024-04-24T23:32:56.929774+0000 (oldest deadline 2024-04-24T23:33:20.429679+0000) 2024-04-24T23:33:26.760 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:33:26 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:33:26.448+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.738540+0000 front 2024-04-24T23:32:55.738561+0000 (oldest deadline 2024-04-24T23:33:21.038786+0000) 2024-04-24T23:33:26.873 INFO:journalctl@ceph.osd.4.smithi134.stdout:Apr 24 23:33:26 smithi134 systemd[1]: Stopping Ceph osd.4 for ec7885aa-0290-11ef-bc93-c7b262605968... 2024-04-24T23:33:27.005 INFO:journalctl@ceph.osd.5.smithi148.stdout:Apr 24 23:33:26 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-5[44694]: 2024-04-24T23:33:26.604+0000 7f6fcf9cd700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.322016+0000 front 2024-04-24T23:32:55.321839+0000 (oldest deadline 2024-04-24T23:33:21.221915+0000) 2024-04-24T23:33:27.197 INFO:journalctl@ceph.osd.4.smithi134.stdout:Apr 24 23:33:26 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-4[44187]: 2024-04-24T23:33:26.872+0000 7f91f563e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:33:27.197 INFO:journalctl@ceph.osd.4.smithi134.stdout:Apr 24 23:33:26 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-4[44187]: 2024-04-24T23:33:26.872+0000 7f91f563e700 -1 osd.4 98 *** Got signal Terminated *** 2024-04-24T23:33:27.197 INFO:journalctl@ceph.osd.4.smithi134.stdout:Apr 24 23:33:26 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-4[44187]: 2024-04-24T23:33:26.872+0000 7f91f563e700 -1 osd.4 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T23:33:27.505 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:33:27 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:33:27.186+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.848330+0000 front 2024-04-24T23:32:55.848387+0000 (oldest deadline 2024-04-24T23:33:20.548352+0000) 2024-04-24T23:33:27.696 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:33:27 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:33:27.417+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.738540+0000 front 2024-04-24T23:32:55.738561+0000 (oldest deadline 2024-04-24T23:33:21.038786+0000) 2024-04-24T23:33:27.697 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:33:27 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:33:27.301+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:56.929716+0000 front 2024-04-24T23:32:56.929774+0000 (oldest deadline 2024-04-24T23:33:20.429679+0000) 2024-04-24T23:33:28.006 INFO:journalctl@ceph.osd.5.smithi148.stdout:Apr 24 23:33:27 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-5[44694]: 2024-04-24T23:33:27.628+0000 7f6fcf9cd700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.322016+0000 front 2024-04-24T23:32:55.321839+0000 (oldest deadline 2024-04-24T23:33:21.221915+0000) 2024-04-24T23:33:28.505 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:33:28 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:33:28.179+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.848330+0000 front 2024-04-24T23:32:55.848387+0000 (oldest deadline 2024-04-24T23:33:20.548352+0000) 2024-04-24T23:33:28.506 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:33:28 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:33:28.179+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:02.150412+0000 front 2024-04-24T23:33:02.150294+0000 (oldest deadline 2024-04-24T23:33:28.050409+0000) 2024-04-24T23:33:28.696 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:33:28 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:33:28.281+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:56.929716+0000 front 2024-04-24T23:32:56.929774+0000 (oldest deadline 2024-04-24T23:33:20.429679+0000) 2024-04-24T23:33:28.787 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:33:28 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:33:28.395+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.738540+0000 front 2024-04-24T23:32:55.738561+0000 (oldest deadline 2024-04-24T23:33:21.038786+0000) 2024-04-24T23:33:29.005 INFO:journalctl@ceph.osd.5.smithi148.stdout:Apr 24 23:33:28 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-5[44694]: 2024-04-24T23:33:28.627+0000 7f6fcf9cd700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.322016+0000 front 2024-04-24T23:32:55.321839+0000 (oldest deadline 2024-04-24T23:33:21.221915+0000) 2024-04-24T23:33:29.506 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:33:29 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:33:29.220+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.848330+0000 front 2024-04-24T23:32:55.848387+0000 (oldest deadline 2024-04-24T23:33:20.548352+0000) 2024-04-24T23:33:29.506 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:33:29 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:33:29.220+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:02.150412+0000 front 2024-04-24T23:33:02.150294+0000 (oldest deadline 2024-04-24T23:33:28.050409+0000) 2024-04-24T23:33:29.696 INFO:journalctl@ceph.osd.7.smithi134.stdout:Apr 24 23:33:29 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T23:33:29.373+0000 7f676da99700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:05.625943+0000 front 2024-04-24T23:33:05.625903+0000 (oldest deadline 2024-04-24T23:33:29.126101+0000) 2024-04-24T23:33:29.697 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:33:29 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:33:29.308+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:56.929716+0000 front 2024-04-24T23:32:56.929774+0000 (oldest deadline 2024-04-24T23:33:20.429679+0000) 2024-04-24T23:33:29.697 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:33:29 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:33:29.308+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:04.530694+0000 front 2024-04-24T23:33:04.530743+0000 (oldest deadline 2024-04-24T23:33:29.230721+0000) 2024-04-24T23:33:29.787 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:33:29 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:33:29.427+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.738540+0000 front 2024-04-24T23:32:55.738561+0000 (oldest deadline 2024-04-24T23:33:21.038786+0000) 2024-04-24T23:33:30.006 INFO:journalctl@ceph.osd.5.smithi148.stdout:Apr 24 23:33:29 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-5[44694]: 2024-04-24T23:33:29.582+0000 7f6fcf9cd700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.322016+0000 front 2024-04-24T23:32:55.321839+0000 (oldest deadline 2024-04-24T23:33:21.221915+0000) 2024-04-24T23:33:30.006 INFO:journalctl@ceph.osd.8.smithi148.stdout:Apr 24 23:33:29 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-8[51367]: 2024-04-24T23:33:29.603+0000 7f119e765700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:05.559825+0000 front 2024-04-24T23:33:05.559812+0000 (oldest deadline 2024-04-24T23:33:29.560381+0000) 2024-04-24T23:33:30.505 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:33:30 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:33:30.222+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.848330+0000 front 2024-04-24T23:32:55.848387+0000 (oldest deadline 2024-04-24T23:33:20.548352+0000) 2024-04-24T23:33:30.506 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:33:30 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:33:30.222+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:02.150412+0000 front 2024-04-24T23:33:02.150294+0000 (oldest deadline 2024-04-24T23:33:28.050409+0000) 2024-04-24T23:33:30.689 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:33:30 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:33:30.387+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.738540+0000 front 2024-04-24T23:32:55.738561+0000 (oldest deadline 2024-04-24T23:33:21.038786+0000) 2024-04-24T23:33:30.689 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:33:30 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:33:30.387+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:05.740014+0000 front 2024-04-24T23:33:05.739865+0000 (oldest deadline 2024-04-24T23:33:29.839897+0000) 2024-04-24T23:33:30.696 INFO:journalctl@ceph.osd.7.smithi134.stdout:Apr 24 23:33:30 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T23:33:30.374+0000 7f676da99700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:05.625943+0000 front 2024-04-24T23:33:05.625903+0000 (oldest deadline 2024-04-24T23:33:29.126101+0000) 2024-04-24T23:33:30.697 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:33:30 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:33:30.330+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:56.929716+0000 front 2024-04-24T23:32:56.929774+0000 (oldest deadline 2024-04-24T23:33:20.429679+0000) 2024-04-24T23:33:30.697 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:33:30 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:33:30.330+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:04.530694+0000 front 2024-04-24T23:33:04.530743+0000 (oldest deadline 2024-04-24T23:33:29.230721+0000) 2024-04-24T23:33:31.005 INFO:journalctl@ceph.osd.5.smithi148.stdout:Apr 24 23:33:30 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-5[44694]: 2024-04-24T23:33:30.547+0000 7f6fcf9cd700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.322016+0000 front 2024-04-24T23:32:55.321839+0000 (oldest deadline 2024-04-24T23:33:21.221915+0000) 2024-04-24T23:33:31.006 INFO:journalctl@ceph.osd.8.smithi148.stdout:Apr 24 23:33:30 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-8[51367]: 2024-04-24T23:33:30.571+0000 7f119e765700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:05.559825+0000 front 2024-04-24T23:33:05.559812+0000 (oldest deadline 2024-04-24T23:33:29.560381+0000) 2024-04-24T23:33:31.505 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:33:31 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:33:31.182+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.848330+0000 front 2024-04-24T23:32:55.848387+0000 (oldest deadline 2024-04-24T23:33:20.548352+0000) 2024-04-24T23:33:31.506 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:33:31 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:33:31.182+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:02.150412+0000 front 2024-04-24T23:33:02.150294+0000 (oldest deadline 2024-04-24T23:33:28.050409+0000) 2024-04-24T23:33:31.697 INFO:journalctl@ceph.osd.7.smithi134.stdout:Apr 24 23:33:31 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T23:33:31.381+0000 7f676da99700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:05.625943+0000 front 2024-04-24T23:33:05.625903+0000 (oldest deadline 2024-04-24T23:33:29.126101+0000) 2024-04-24T23:33:31.697 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:33:31 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:33:31.308+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:56.929716+0000 front 2024-04-24T23:32:56.929774+0000 (oldest deadline 2024-04-24T23:33:20.429679+0000) 2024-04-24T23:33:31.697 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:33:31 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:33:31.308+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:04.530694+0000 front 2024-04-24T23:33:04.530743+0000 (oldest deadline 2024-04-24T23:33:29.230721+0000) 2024-04-24T23:33:31.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:33:31 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:33:31.360+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.738540+0000 front 2024-04-24T23:32:55.738561+0000 (oldest deadline 2024-04-24T23:33:21.038786+0000) 2024-04-24T23:33:31.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:33:31 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:33:31.360+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:05.740014+0000 front 2024-04-24T23:33:05.739865+0000 (oldest deadline 2024-04-24T23:33:29.839897+0000) 2024-04-24T23:33:32.006 INFO:journalctl@ceph.osd.5.smithi148.stdout:Apr 24 23:33:31 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-5[44694]: 2024-04-24T23:33:31.526+0000 7f6fcf9cd700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.322016+0000 front 2024-04-24T23:32:55.321839+0000 (oldest deadline 2024-04-24T23:33:21.221915+0000) 2024-04-24T23:33:32.006 INFO:journalctl@ceph.osd.8.smithi148.stdout:Apr 24 23:33:31 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-8[51367]: 2024-04-24T23:33:31.596+0000 7f119e765700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:05.559825+0000 front 2024-04-24T23:33:05.559812+0000 (oldest deadline 2024-04-24T23:33:29.560381+0000) 2024-04-24T23:33:32.196 INFO:journalctl@ceph.osd.4.smithi134.stdout:Apr 24 23:33:31 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-4[44187]: 2024-04-24T23:33:31.832+0000 7f91f1c32700 -1 osd.4 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:06.787634+0000 front 2024-04-24T23:33:06.787661+0000 (oldest deadline 2024-04-24T23:33:30.887923+0000) 2024-04-24T23:33:32.506 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:33:32 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:33:32.171+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.848330+0000 front 2024-04-24T23:32:55.848387+0000 (oldest deadline 2024-04-24T23:33:20.548352+0000) 2024-04-24T23:33:32.506 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:33:32 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:33:32.171+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:02.150412+0000 front 2024-04-24T23:33:02.150294+0000 (oldest deadline 2024-04-24T23:33:28.050409+0000) 2024-04-24T23:33:32.537 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:33:32 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:33:32.337+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.738540+0000 front 2024-04-24T23:32:55.738561+0000 (oldest deadline 2024-04-24T23:33:21.038786+0000) 2024-04-24T23:33:32.538 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:33:32 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:33:32.337+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:05.740014+0000 front 2024-04-24T23:33:05.739865+0000 (oldest deadline 2024-04-24T23:33:29.839897+0000) 2024-04-24T23:33:32.538 INFO:journalctl@ceph.osd.6.smithi080.stdout:Apr 24 23:33:32 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-6[57095]: 2024-04-24T23:33:32.282+0000 7fee6c0e2700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:06.138406+0000 front 2024-04-24T23:33:06.138353+0000 (oldest deadline 2024-04-24T23:33:32.038442+0000) 2024-04-24T23:33:32.674 INFO:journalctl@ceph.osd.7.smithi134.stdout:Apr 24 23:33:32 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T23:33:32.361+0000 7f676da99700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:05.625943+0000 front 2024-04-24T23:33:05.625903+0000 (oldest deadline 2024-04-24T23:33:29.126101+0000) 2024-04-24T23:33:32.674 INFO:journalctl@ceph.osd.4.smithi134.stdout:Apr 24 23:33:32 smithi134 podman[66842]: 2024-04-24 23:33:32.40857428 +0000 UTC m=+5.663246295 container died 3ddf7c1b2c4f73d29c625fcb0b7cd0e64b971be88f35f73e8c113073948e40d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-4, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD) 2024-04-24T23:33:32.674 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:33:32 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:33:32.276+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:56.929716+0000 front 2024-04-24T23:32:56.929774+0000 (oldest deadline 2024-04-24T23:33:20.429679+0000) 2024-04-24T23:33:32.675 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:33:32 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:33:32.276+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:04.530694+0000 front 2024-04-24T23:33:04.530743+0000 (oldest deadline 2024-04-24T23:33:29.230721+0000) 2024-04-24T23:33:32.946 INFO:journalctl@ceph.osd.4.smithi134.stdout:Apr 24 23:33:32 smithi134 podman[66842]: 2024-04-24 23:33:32.765436479 +0000 UTC m=+6.020108488 container cleanup 3ddf7c1b2c4f73d29c625fcb0b7cd0e64b971be88f35f73e8c113073948e40d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-4, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-04-24T23:33:32.947 INFO:journalctl@ceph.osd.4.smithi134.stdout:Apr 24 23:33:32 smithi134 bash[66842]: ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-4 2024-04-24T23:33:33.006 INFO:journalctl@ceph.osd.8.smithi148.stdout:Apr 24 23:33:32 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-8[51367]: 2024-04-24T23:33:32.643+0000 7f119e765700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:05.559825+0000 front 2024-04-24T23:33:05.559812+0000 (oldest deadline 2024-04-24T23:33:29.560381+0000) 2024-04-24T23:33:33.006 INFO:journalctl@ceph.osd.5.smithi148.stdout:Apr 24 23:33:32 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-5[44694]: 2024-04-24T23:33:32.567+0000 7f6fcf9cd700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.322016+0000 front 2024-04-24T23:32:55.321839+0000 (oldest deadline 2024-04-24T23:33:21.221915+0000) 2024-04-24T23:33:33.006 INFO:journalctl@ceph.osd.5.smithi148.stdout:Apr 24 23:33:32 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-5[44694]: 2024-04-24T23:33:32.567+0000 7f6fcf9cd700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:07.623595+0000 front 2024-04-24T23:33:07.623580+0000 (oldest deadline 2024-04-24T23:33:31.723487+0000) 2024-04-24T23:33:33.373 INFO:journalctl@ceph.osd.4.smithi134.stdout:Apr 24 23:33:33 smithi134 podman[66857]: 2024-04-24 23:33:33.058636129 +0000 UTC m=+0.651980121 container remove 3ddf7c1b2c4f73d29c625fcb0b7cd0e64b971be88f35f73e8c113073948e40d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-4, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9) 2024-04-24T23:33:33.373 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:33:33 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:33:33.256+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:56.929716+0000 front 2024-04-24T23:32:56.929774+0000 (oldest deadline 2024-04-24T23:33:20.429679+0000) 2024-04-24T23:33:33.374 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:33:33 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:33:33.256+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:04.530694+0000 front 2024-04-24T23:33:04.530743+0000 (oldest deadline 2024-04-24T23:33:29.230721+0000) 2024-04-24T23:33:33.506 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:33:33 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:33:33.189+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.848330+0000 front 2024-04-24T23:32:55.848387+0000 (oldest deadline 2024-04-24T23:33:20.548352+0000) 2024-04-24T23:33:33.506 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:33:33 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:33:33.189+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:02.150412+0000 front 2024-04-24T23:33:02.150294+0000 (oldest deadline 2024-04-24T23:33:28.050409+0000) 2024-04-24T23:33:33.628 INFO:journalctl@ceph.osd.7.smithi134.stdout:Apr 24 23:33:33 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T23:33:33.373+0000 7f676da99700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:05.625943+0000 front 2024-04-24T23:33:05.625903+0000 (oldest deadline 2024-04-24T23:33:29.126101+0000) 2024-04-24T23:33:33.629 INFO:journalctl@ceph.osd.4.smithi134.stdout:Apr 24 23:33:33 smithi134 podman[66951]: 2024-04-24 23:33:33.284688152 +0000 UTC m=+0.017692087 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:33:33.629 INFO:journalctl@ceph.osd.4.smithi134.stdout:Apr 24 23:33:33 smithi134 podman[66951]: 2024-04-24 23:33:33.401168963 +0000 UTC m=+0.134172876 container create 65b797e545fdeb1a153ceb17e7544b13597bd91bdcb21bc80d6e9097822efdc9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424) 2024-04-24T23:33:33.629 INFO:journalctl@ceph.osd.4.smithi134.stdout:Apr 24 23:33:33 smithi134 podman[66951]: 2024-04-24 23:33:33.629222448 +0000 UTC m=+0.362226375 container init 65b797e545fdeb1a153ceb17e7544b13597bd91bdcb21bc80d6e9097822efdc9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-04-24T23:33:33.788 INFO:journalctl@ceph.osd.6.smithi080.stdout:Apr 24 23:33:33 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-6[57095]: 2024-04-24T23:33:33.306+0000 7fee6c0e2700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:06.138406+0000 front 2024-04-24T23:33:06.138353+0000 (oldest deadline 2024-04-24T23:33:32.038442+0000) 2024-04-24T23:33:33.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:33:33 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:33:33.312+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.738540+0000 front 2024-04-24T23:32:55.738561+0000 (oldest deadline 2024-04-24T23:33:21.038786+0000) 2024-04-24T23:33:33.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:33:33 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:33:33.312+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:05.740014+0000 front 2024-04-24T23:33:05.739865+0000 (oldest deadline 2024-04-24T23:33:29.839897+0000) 2024-04-24T23:33:33.914 INFO:journalctl@ceph.osd.4.smithi134.stdout:Apr 24 23:33:33 smithi134 podman[66951]: 2024-04-24 23:33:33.632470057 +0000 UTC m=+0.365473974 container start 65b797e545fdeb1a153ceb17e7544b13597bd91bdcb21bc80d6e9097822efdc9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-4-deactivate, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-04-24T23:33:33.914 INFO:journalctl@ceph.osd.4.smithi134.stdout:Apr 24 23:33:33 smithi134 podman[66951]: 2024-04-24 23:33:33.729232731 +0000 UTC m=+0.462236650 container attach 65b797e545fdeb1a153ceb17e7544b13597bd91bdcb21bc80d6e9097822efdc9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T23:33:34.005 INFO:journalctl@ceph.osd.8.smithi148.stdout:Apr 24 23:33:33 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-8[51367]: 2024-04-24T23:33:33.625+0000 7f119e765700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:05.559825+0000 front 2024-04-24T23:33:05.559812+0000 (oldest deadline 2024-04-24T23:33:29.560381+0000) 2024-04-24T23:33:34.006 INFO:journalctl@ceph.osd.5.smithi148.stdout:Apr 24 23:33:33 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-5[44694]: 2024-04-24T23:33:33.530+0000 7f6fcf9cd700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.322016+0000 front 2024-04-24T23:32:55.321839+0000 (oldest deadline 2024-04-24T23:33:21.221915+0000) 2024-04-24T23:33:34.006 INFO:journalctl@ceph.osd.5.smithi148.stdout:Apr 24 23:33:33 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-5[44694]: 2024-04-24T23:33:33.530+0000 7f6fcf9cd700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:07.623595+0000 front 2024-04-24T23:33:07.623580+0000 (oldest deadline 2024-04-24T23:33:31.723487+0000) 2024-04-24T23:33:34.196 INFO:journalctl@ceph.osd.4.smithi134.stdout:Apr 24 23:33:33 smithi134 podman[66951]: 2024-04-24 23:33:33.914849213 +0000 UTC m=+0.647853140 container died 65b797e545fdeb1a153ceb17e7544b13597bd91bdcb21bc80d6e9097822efdc9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD) 2024-04-24T23:33:34.506 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:33:34 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:33:34.154+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.848330+0000 front 2024-04-24T23:32:55.848387+0000 (oldest deadline 2024-04-24T23:33:20.548352+0000) 2024-04-24T23:33:34.506 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:33:34 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:33:34.154+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:02.150412+0000 front 2024-04-24T23:33:02.150294+0000 (oldest deadline 2024-04-24T23:33:28.050409+0000) 2024-04-24T23:33:34.540 INFO:journalctl@ceph.osd.7.smithi134.stdout:Apr 24 23:33:34 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T23:33:34.417+0000 7f676da99700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:05.625943+0000 front 2024-04-24T23:33:05.625903+0000 (oldest deadline 2024-04-24T23:33:29.126101+0000) 2024-04-24T23:33:34.541 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:33:34 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:33:34.243+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:56.929716+0000 front 2024-04-24T23:32:56.929774+0000 (oldest deadline 2024-04-24T23:33:20.429679+0000) 2024-04-24T23:33:34.541 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:33:34 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:33:34.243+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:04.530694+0000 front 2024-04-24T23:33:04.530743+0000 (oldest deadline 2024-04-24T23:33:29.230721+0000) 2024-04-24T23:33:34.568 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ec7885aa-0290-11ef-bc93-c7b262605968@osd.4.service' 2024-04-24T23:33:34.605 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:33:34.606 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-04-24T23:33:34.606 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-04-24T23:33:34.606 DEBUG:teuthology.orchestra.run.smithi148:> sudo systemctl stop ceph-ec7885aa-0290-11ef-bc93-c7b262605968@osd.5 2024-04-24T23:33:34.788 INFO:journalctl@ceph.osd.6.smithi080.stdout:Apr 24 23:33:34 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-6[57095]: 2024-04-24T23:33:34.344+0000 7fee6c0e2700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:06.138406+0000 front 2024-04-24T23:33:06.138353+0000 (oldest deadline 2024-04-24T23:33:32.038442+0000) 2024-04-24T23:33:34.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:33:34 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:33:34.343+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.738540+0000 front 2024-04-24T23:32:55.738561+0000 (oldest deadline 2024-04-24T23:33:21.038786+0000) 2024-04-24T23:33:34.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:33:34 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:33:34.343+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:05.740014+0000 front 2024-04-24T23:33:05.739865+0000 (oldest deadline 2024-04-24T23:33:29.839897+0000) 2024-04-24T23:33:34.941 INFO:journalctl@ceph.osd.8.smithi148.stdout:Apr 24 23:33:34 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-8[51367]: 2024-04-24T23:33:34.620+0000 7f119e765700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:05.559825+0000 front 2024-04-24T23:33:05.559812+0000 (oldest deadline 2024-04-24T23:33:29.560381+0000) 2024-04-24T23:33:34.941 INFO:journalctl@ceph.osd.5.smithi148.stdout:Apr 24 23:33:34 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-5[44694]: 2024-04-24T23:33:34.516+0000 7f6fcf9cd700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.322016+0000 front 2024-04-24T23:32:55.321839+0000 (oldest deadline 2024-04-24T23:33:21.221915+0000) 2024-04-24T23:33:34.942 INFO:journalctl@ceph.osd.5.smithi148.stdout:Apr 24 23:33:34 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-5[44694]: 2024-04-24T23:33:34.516+0000 7f6fcf9cd700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:07.623595+0000 front 2024-04-24T23:33:07.623580+0000 (oldest deadline 2024-04-24T23:33:31.723487+0000) 2024-04-24T23:33:34.942 INFO:journalctl@ceph.osd.5.smithi148.stdout:Apr 24 23:33:34 smithi148 systemd[1]: Stopping Ceph osd.5 for ec7885aa-0290-11ef-bc93-c7b262605968... 2024-04-24T23:33:34.942 INFO:journalctl@ceph.osd.5.smithi148.stdout:Apr 24 23:33:34 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-5[44694]: 2024-04-24T23:33:34.941+0000 7f6fd33d9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:33:34.942 INFO:journalctl@ceph.osd.5.smithi148.stdout:Apr 24 23:33:34 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-5[44694]: 2024-04-24T23:33:34.941+0000 7f6fd33d9700 -1 osd.5 98 *** Got signal Terminated *** 2024-04-24T23:33:35.255 INFO:journalctl@ceph.osd.5.smithi148.stdout:Apr 24 23:33:34 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-5[44694]: 2024-04-24T23:33:34.941+0000 7f6fd33d9700 -1 osd.5 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T23:33:35.256 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:33:35 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:33:35.112+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.848330+0000 front 2024-04-24T23:32:55.848387+0000 (oldest deadline 2024-04-24T23:33:20.548352+0000) 2024-04-24T23:33:35.256 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:33:35 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:33:35.112+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:02.150412+0000 front 2024-04-24T23:33:02.150294+0000 (oldest deadline 2024-04-24T23:33:28.050409+0000) 2024-04-24T23:33:35.697 INFO:journalctl@ceph.osd.7.smithi134.stdout:Apr 24 23:33:35 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T23:33:35.417+0000 7f676da99700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:05.625943+0000 front 2024-04-24T23:33:05.625903+0000 (oldest deadline 2024-04-24T23:33:29.126101+0000) 2024-04-24T23:33:35.697 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:33:35 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:33:35.231+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:56.929716+0000 front 2024-04-24T23:32:56.929774+0000 (oldest deadline 2024-04-24T23:33:20.429679+0000) 2024-04-24T23:33:35.697 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:33:35 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:33:35.231+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:04.530694+0000 front 2024-04-24T23:33:04.530743+0000 (oldest deadline 2024-04-24T23:33:29.230721+0000) 2024-04-24T23:33:35.788 INFO:journalctl@ceph.osd.6.smithi080.stdout:Apr 24 23:33:35 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-6[57095]: 2024-04-24T23:33:35.322+0000 7fee6c0e2700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:06.138406+0000 front 2024-04-24T23:33:06.138353+0000 (oldest deadline 2024-04-24T23:33:32.038442+0000) 2024-04-24T23:33:35.789 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:33:35 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:33:35.340+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.738540+0000 front 2024-04-24T23:32:55.738561+0000 (oldest deadline 2024-04-24T23:33:21.038786+0000) 2024-04-24T23:33:35.789 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:33:35 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:33:35.340+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:05.740014+0000 front 2024-04-24T23:33:05.739865+0000 (oldest deadline 2024-04-24T23:33:29.839897+0000) 2024-04-24T23:33:36.006 INFO:journalctl@ceph.osd.8.smithi148.stdout:Apr 24 23:33:35 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-8[51367]: 2024-04-24T23:33:35.656+0000 7f119e765700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:05.559825+0000 front 2024-04-24T23:33:05.559812+0000 (oldest deadline 2024-04-24T23:33:29.560381+0000) 2024-04-24T23:33:36.006 INFO:journalctl@ceph.osd.5.smithi148.stdout:Apr 24 23:33:35 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-5[44694]: 2024-04-24T23:33:35.518+0000 7f6fcf9cd700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.322016+0000 front 2024-04-24T23:32:55.321839+0000 (oldest deadline 2024-04-24T23:33:21.221915+0000) 2024-04-24T23:33:36.007 INFO:journalctl@ceph.osd.5.smithi148.stdout:Apr 24 23:33:35 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-5[44694]: 2024-04-24T23:33:35.518+0000 7f6fcf9cd700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:07.623595+0000 front 2024-04-24T23:33:07.623580+0000 (oldest deadline 2024-04-24T23:33:31.723487+0000) 2024-04-24T23:33:36.505 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:33:36 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:33:36.114+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.848330+0000 front 2024-04-24T23:32:55.848387+0000 (oldest deadline 2024-04-24T23:33:20.548352+0000) 2024-04-24T23:33:36.506 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:33:36 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:33:36.114+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:02.150412+0000 front 2024-04-24T23:33:02.150294+0000 (oldest deadline 2024-04-24T23:33:28.050409+0000) 2024-04-24T23:33:36.696 INFO:journalctl@ceph.osd.7.smithi134.stdout:Apr 24 23:33:36 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T23:33:36.462+0000 7f676da99700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:05.625943+0000 front 2024-04-24T23:33:05.625903+0000 (oldest deadline 2024-04-24T23:33:29.126101+0000) 2024-04-24T23:33:36.697 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:33:36 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:33:36.212+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:56.929716+0000 front 2024-04-24T23:32:56.929774+0000 (oldest deadline 2024-04-24T23:33:20.429679+0000) 2024-04-24T23:33:36.697 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:33:36 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:33:36.212+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:04.530694+0000 front 2024-04-24T23:33:04.530743+0000 (oldest deadline 2024-04-24T23:33:29.230721+0000) 2024-04-24T23:33:36.788 INFO:journalctl@ceph.osd.6.smithi080.stdout:Apr 24 23:33:36 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-6[57095]: 2024-04-24T23:33:36.362+0000 7fee6c0e2700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:06.138406+0000 front 2024-04-24T23:33:06.138353+0000 (oldest deadline 2024-04-24T23:33:32.038442+0000) 2024-04-24T23:33:36.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:33:36 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:33:36.327+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.738540+0000 front 2024-04-24T23:32:55.738561+0000 (oldest deadline 2024-04-24T23:33:21.038786+0000) 2024-04-24T23:33:36.789 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:33:36 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:33:36.327+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:05.740014+0000 front 2024-04-24T23:33:05.739865+0000 (oldest deadline 2024-04-24T23:33:29.839897+0000) 2024-04-24T23:33:37.006 INFO:journalctl@ceph.osd.8.smithi148.stdout:Apr 24 23:33:36 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-8[51367]: 2024-04-24T23:33:36.680+0000 7f119e765700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:05.559825+0000 front 2024-04-24T23:33:05.559812+0000 (oldest deadline 2024-04-24T23:33:29.560381+0000) 2024-04-24T23:33:37.006 INFO:journalctl@ceph.osd.8.smithi148.stdout:Apr 24 23:33:36 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-8[51367]: 2024-04-24T23:33:36.680+0000 7f119e765700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:13.061482+0000 front 2024-04-24T23:33:13.061509+0000 (oldest deadline 2024-04-24T23:33:36.561622+0000) 2024-04-24T23:33:37.007 INFO:journalctl@ceph.osd.5.smithi148.stdout:Apr 24 23:33:36 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-5[44694]: 2024-04-24T23:33:36.508+0000 7f6fcf9cd700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.322016+0000 front 2024-04-24T23:32:55.321839+0000 (oldest deadline 2024-04-24T23:33:21.221915+0000) 2024-04-24T23:33:37.007 INFO:journalctl@ceph.osd.5.smithi148.stdout:Apr 24 23:33:36 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-5[44694]: 2024-04-24T23:33:36.508+0000 7f6fcf9cd700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:07.623595+0000 front 2024-04-24T23:33:07.623580+0000 (oldest deadline 2024-04-24T23:33:31.723487+0000) 2024-04-24T23:33:37.471 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:33:37 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:33:37.136+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.848330+0000 front 2024-04-24T23:32:55.848387+0000 (oldest deadline 2024-04-24T23:33:20.548352+0000) 2024-04-24T23:33:37.471 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:33:37 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:33:37.136+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:02.150412+0000 front 2024-04-24T23:33:02.150294+0000 (oldest deadline 2024-04-24T23:33:28.050409+0000) 2024-04-24T23:33:37.696 INFO:journalctl@ceph.osd.7.smithi134.stdout:Apr 24 23:33:37 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T23:33:37.450+0000 7f676da99700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:05.625943+0000 front 2024-04-24T23:33:05.625903+0000 (oldest deadline 2024-04-24T23:33:29.126101+0000) 2024-04-24T23:33:37.697 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:33:37 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:33:37.218+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:56.929716+0000 front 2024-04-24T23:32:56.929774+0000 (oldest deadline 2024-04-24T23:33:20.429679+0000) 2024-04-24T23:33:37.697 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:33:37 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:33:37.218+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:04.530694+0000 front 2024-04-24T23:33:04.530743+0000 (oldest deadline 2024-04-24T23:33:29.230721+0000) 2024-04-24T23:33:37.756 INFO:journalctl@ceph.osd.5.smithi148.stdout:Apr 24 23:33:37 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-5[44694]: 2024-04-24T23:33:37.470+0000 7f6fcf9cd700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.322016+0000 front 2024-04-24T23:32:55.321839+0000 (oldest deadline 2024-04-24T23:33:21.221915+0000) 2024-04-24T23:33:37.756 INFO:journalctl@ceph.osd.5.smithi148.stdout:Apr 24 23:33:37 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-5[44694]: 2024-04-24T23:33:37.470+0000 7f6fcf9cd700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:07.623595+0000 front 2024-04-24T23:33:07.623580+0000 (oldest deadline 2024-04-24T23:33:31.723487+0000) 2024-04-24T23:33:37.756 INFO:journalctl@ceph.osd.8.smithi148.stdout:Apr 24 23:33:37 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-8[51367]: 2024-04-24T23:33:37.705+0000 7f119e765700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:05.559825+0000 front 2024-04-24T23:33:05.559812+0000 (oldest deadline 2024-04-24T23:33:29.560381+0000) 2024-04-24T23:33:37.756 INFO:journalctl@ceph.osd.8.smithi148.stdout:Apr 24 23:33:37 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-8[51367]: 2024-04-24T23:33:37.705+0000 7f119e765700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:13.061482+0000 front 2024-04-24T23:33:13.061509+0000 (oldest deadline 2024-04-24T23:33:36.561622+0000) 2024-04-24T23:33:37.787 INFO:journalctl@ceph.osd.6.smithi080.stdout:Apr 24 23:33:37 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-6[57095]: 2024-04-24T23:33:37.395+0000 7fee6c0e2700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:06.138406+0000 front 2024-04-24T23:33:06.138353+0000 (oldest deadline 2024-04-24T23:33:32.038442+0000) 2024-04-24T23:33:37.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:33:37 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:33:37.335+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.738540+0000 front 2024-04-24T23:32:55.738561+0000 (oldest deadline 2024-04-24T23:33:21.038786+0000) 2024-04-24T23:33:37.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:33:37 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:33:37.335+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:05.740014+0000 front 2024-04-24T23:33:05.739865+0000 (oldest deadline 2024-04-24T23:33:29.839897+0000) 2024-04-24T23:33:38.505 INFO:journalctl@ceph.osd.5.smithi148.stdout:Apr 24 23:33:38 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-5[44694]: 2024-04-24T23:33:38.423+0000 7f6fcf9cd700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.322016+0000 front 2024-04-24T23:32:55.321839+0000 (oldest deadline 2024-04-24T23:33:21.221915+0000) 2024-04-24T23:33:38.506 INFO:journalctl@ceph.osd.5.smithi148.stdout:Apr 24 23:33:38 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-5[44694]: 2024-04-24T23:33:38.423+0000 7f6fcf9cd700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:07.623595+0000 front 2024-04-24T23:33:07.623580+0000 (oldest deadline 2024-04-24T23:33:31.723487+0000) 2024-04-24T23:33:38.506 INFO:journalctl@ceph.osd.5.smithi148.stdout:Apr 24 23:33:38 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-5[44694]: 2024-04-24T23:33:38.423+0000 7f6fcf9cd700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:12.224329+0000 front 2024-04-24T23:33:12.224253+0000 (oldest deadline 2024-04-24T23:33:38.124270+0000) 2024-04-24T23:33:38.506 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:33:38 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:33:38.185+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.848330+0000 front 2024-04-24T23:32:55.848387+0000 (oldest deadline 2024-04-24T23:33:20.548352+0000) 2024-04-24T23:33:38.506 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:33:38 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:33:38.185+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:02.150412+0000 front 2024-04-24T23:33:02.150294+0000 (oldest deadline 2024-04-24T23:33:28.050409+0000) 2024-04-24T23:33:38.697 INFO:journalctl@ceph.osd.7.smithi134.stdout:Apr 24 23:33:38 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T23:33:38.413+0000 7f676da99700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:05.625943+0000 front 2024-04-24T23:33:05.625903+0000 (oldest deadline 2024-04-24T23:33:29.126101+0000) 2024-04-24T23:33:38.697 INFO:journalctl@ceph.osd.7.smithi134.stdout:Apr 24 23:33:38 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T23:33:38.413+0000 7f676da99700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:14.927842+0000 front 2024-04-24T23:33:14.927944+0000 (oldest deadline 2024-04-24T23:33:37.827908+0000) 2024-04-24T23:33:38.697 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:33:38 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:33:38.225+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:56.929716+0000 front 2024-04-24T23:32:56.929774+0000 (oldest deadline 2024-04-24T23:33:20.429679+0000) 2024-04-24T23:33:38.698 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:33:38 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:33:38.225+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:04.530694+0000 front 2024-04-24T23:33:04.530743+0000 (oldest deadline 2024-04-24T23:33:29.230721+0000) 2024-04-24T23:33:38.698 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:33:38 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:33:38.225+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:15.632228+0000 front 2024-04-24T23:33:15.631978+0000 (oldest deadline 2024-04-24T23:33:37.331905+0000) 2024-04-24T23:33:38.788 INFO:journalctl@ceph.osd.6.smithi080.stdout:Apr 24 23:33:38 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-6[57095]: 2024-04-24T23:33:38.395+0000 7fee6c0e2700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:06.138406+0000 front 2024-04-24T23:33:06.138353+0000 (oldest deadline 2024-04-24T23:33:32.038442+0000) 2024-04-24T23:33:38.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:33:38 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:33:38.298+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.738540+0000 front 2024-04-24T23:32:55.738561+0000 (oldest deadline 2024-04-24T23:33:21.038786+0000) 2024-04-24T23:33:38.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:33:38 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:33:38.298+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:05.740014+0000 front 2024-04-24T23:33:05.739865+0000 (oldest deadline 2024-04-24T23:33:29.839897+0000) 2024-04-24T23:33:38.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:33:38 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:33:38.298+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:12.140601+0000 front 2024-04-24T23:33:12.140627+0000 (oldest deadline 2024-04-24T23:33:38.040825+0000) 2024-04-24T23:33:39.006 INFO:journalctl@ceph.osd.8.smithi148.stdout:Apr 24 23:33:38 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-8[51367]: 2024-04-24T23:33:38.655+0000 7f119e765700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:05.559825+0000 front 2024-04-24T23:33:05.559812+0000 (oldest deadline 2024-04-24T23:33:29.560381+0000) 2024-04-24T23:33:39.006 INFO:journalctl@ceph.osd.8.smithi148.stdout:Apr 24 23:33:38 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-8[51367]: 2024-04-24T23:33:38.655+0000 7f119e765700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:13.061482+0000 front 2024-04-24T23:33:13.061509+0000 (oldest deadline 2024-04-24T23:33:36.561622+0000) 2024-04-24T23:33:39.506 INFO:journalctl@ceph.osd.5.smithi148.stdout:Apr 24 23:33:39 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-5[44694]: 2024-04-24T23:33:39.413+0000 7f6fcf9cd700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.322016+0000 front 2024-04-24T23:32:55.321839+0000 (oldest deadline 2024-04-24T23:33:21.221915+0000) 2024-04-24T23:33:39.506 INFO:journalctl@ceph.osd.5.smithi148.stdout:Apr 24 23:33:39 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-5[44694]: 2024-04-24T23:33:39.413+0000 7f6fcf9cd700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:07.623595+0000 front 2024-04-24T23:33:07.623580+0000 (oldest deadline 2024-04-24T23:33:31.723487+0000) 2024-04-24T23:33:39.506 INFO:journalctl@ceph.osd.5.smithi148.stdout:Apr 24 23:33:39 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-5[44694]: 2024-04-24T23:33:39.413+0000 7f6fcf9cd700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:12.224329+0000 front 2024-04-24T23:33:12.224253+0000 (oldest deadline 2024-04-24T23:33:38.124270+0000) 2024-04-24T23:33:39.506 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:33:39 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:33:39.166+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.848330+0000 front 2024-04-24T23:32:55.848387+0000 (oldest deadline 2024-04-24T23:33:20.548352+0000) 2024-04-24T23:33:39.507 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:33:39 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:33:39.166+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:02.150412+0000 front 2024-04-24T23:33:02.150294+0000 (oldest deadline 2024-04-24T23:33:28.050409+0000) 2024-04-24T23:33:39.537 INFO:journalctl@ceph.osd.6.smithi080.stdout:Apr 24 23:33:39 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-6[57095]: 2024-04-24T23:33:39.369+0000 7fee6c0e2700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:06.138406+0000 front 2024-04-24T23:33:06.138353+0000 (oldest deadline 2024-04-24T23:33:32.038442+0000) 2024-04-24T23:33:39.538 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:33:39 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:33:39.270+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.738540+0000 front 2024-04-24T23:32:55.738561+0000 (oldest deadline 2024-04-24T23:33:21.038786+0000) 2024-04-24T23:33:39.538 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:33:39 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:33:39.270+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:05.740014+0000 front 2024-04-24T23:33:05.739865+0000 (oldest deadline 2024-04-24T23:33:29.839897+0000) 2024-04-24T23:33:39.538 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:33:39 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:33:39.270+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:12.140601+0000 front 2024-04-24T23:33:12.140627+0000 (oldest deadline 2024-04-24T23:33:38.040825+0000) 2024-04-24T23:33:39.697 INFO:journalctl@ceph.osd.7.smithi134.stdout:Apr 24 23:33:39 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T23:33:39.451+0000 7f676da99700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:05.625943+0000 front 2024-04-24T23:33:05.625903+0000 (oldest deadline 2024-04-24T23:33:29.126101+0000) 2024-04-24T23:33:39.697 INFO:journalctl@ceph.osd.7.smithi134.stdout:Apr 24 23:33:39 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T23:33:39.451+0000 7f676da99700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:14.927842+0000 front 2024-04-24T23:33:14.927944+0000 (oldest deadline 2024-04-24T23:33:37.827908+0000) 2024-04-24T23:33:39.697 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:33:39 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:33:39.263+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:56.929716+0000 front 2024-04-24T23:32:56.929774+0000 (oldest deadline 2024-04-24T23:33:20.429679+0000) 2024-04-24T23:33:39.697 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:33:39 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:33:39.263+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:04.530694+0000 front 2024-04-24T23:33:04.530743+0000 (oldest deadline 2024-04-24T23:33:29.230721+0000) 2024-04-24T23:33:39.697 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:33:39 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:33:39.263+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:15.632228+0000 front 2024-04-24T23:33:15.631978+0000 (oldest deadline 2024-04-24T23:33:37.331905+0000) 2024-04-24T23:33:40.006 INFO:journalctl@ceph.osd.8.smithi148.stdout:Apr 24 23:33:39 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-8[51367]: 2024-04-24T23:33:39.676+0000 7f119e765700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:05.559825+0000 front 2024-04-24T23:33:05.559812+0000 (oldest deadline 2024-04-24T23:33:29.560381+0000) 2024-04-24T23:33:40.006 INFO:journalctl@ceph.osd.8.smithi148.stdout:Apr 24 23:33:39 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-8[51367]: 2024-04-24T23:33:39.676+0000 7f119e765700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:13.061482+0000 front 2024-04-24T23:33:13.061509+0000 (oldest deadline 2024-04-24T23:33:36.561622+0000) 2024-04-24T23:33:40.506 INFO:journalctl@ceph.osd.5.smithi148.stdout:Apr 24 23:33:40 smithi148 podman[67139]: 2024-04-24 23:33:40.379754726 +0000 UTC m=+5.574764942 container died c2aa36f25152d45fec914f505bf9106b14a67890a8b6a565b9cf3c90fc95da66 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9) 2024-04-24T23:33:40.506 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:33:40 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:33:40.149+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.848330+0000 front 2024-04-24T23:32:55.848387+0000 (oldest deadline 2024-04-24T23:33:20.548352+0000) 2024-04-24T23:33:40.506 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:33:40 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:33:40.149+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:02.150412+0000 front 2024-04-24T23:33:02.150294+0000 (oldest deadline 2024-04-24T23:33:28.050409+0000) 2024-04-24T23:33:40.696 INFO:journalctl@ceph.osd.7.smithi134.stdout:Apr 24 23:33:40 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T23:33:40.421+0000 7f676da99700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:05.625943+0000 front 2024-04-24T23:33:05.625903+0000 (oldest deadline 2024-04-24T23:33:29.126101+0000) 2024-04-24T23:33:40.696 INFO:journalctl@ceph.osd.7.smithi134.stdout:Apr 24 23:33:40 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T23:33:40.421+0000 7f676da99700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:14.927842+0000 front 2024-04-24T23:33:14.927944+0000 (oldest deadline 2024-04-24T23:33:37.827908+0000) 2024-04-24T23:33:40.697 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:33:40 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:33:40.231+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:56.929716+0000 front 2024-04-24T23:32:56.929774+0000 (oldest deadline 2024-04-24T23:33:20.429679+0000) 2024-04-24T23:33:40.697 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:33:40 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:33:40.231+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:04.530694+0000 front 2024-04-24T23:33:04.530743+0000 (oldest deadline 2024-04-24T23:33:29.230721+0000) 2024-04-24T23:33:40.697 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:33:40 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:33:40.231+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:15.632228+0000 front 2024-04-24T23:33:15.631978+0000 (oldest deadline 2024-04-24T23:33:37.331905+0000) 2024-04-24T23:33:40.742 INFO:journalctl@ceph.osd.6.smithi080.stdout:Apr 24 23:33:40 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-6[57095]: 2024-04-24T23:33:40.380+0000 7fee6c0e2700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:06.138406+0000 front 2024-04-24T23:33:06.138353+0000 (oldest deadline 2024-04-24T23:33:32.038442+0000) 2024-04-24T23:33:40.742 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:33:40 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:33:40.313+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.738540+0000 front 2024-04-24T23:32:55.738561+0000 (oldest deadline 2024-04-24T23:33:21.038786+0000) 2024-04-24T23:33:40.742 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:33:40 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:33:40.313+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:05.740014+0000 front 2024-04-24T23:33:05.739865+0000 (oldest deadline 2024-04-24T23:33:29.839897+0000) 2024-04-24T23:33:40.742 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:33:40 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:33:40.313+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:12.140601+0000 front 2024-04-24T23:33:12.140627+0000 (oldest deadline 2024-04-24T23:33:38.040825+0000) 2024-04-24T23:33:41.006 INFO:journalctl@ceph.osd.5.smithi148.stdout:Apr 24 23:33:40 smithi148 podman[67139]: 2024-04-24 23:33:40.733198508 +0000 UTC m=+5.928208726 container cleanup c2aa36f25152d45fec914f505bf9106b14a67890a8b6a565b9cf3c90fc95da66 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-5, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-04-24T23:33:41.006 INFO:journalctl@ceph.osd.5.smithi148.stdout:Apr 24 23:33:40 smithi148 bash[67139]: ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-5 2024-04-24T23:33:41.007 INFO:journalctl@ceph.osd.8.smithi148.stdout:Apr 24 23:33:40 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-8[51367]: 2024-04-24T23:33:40.711+0000 7f119e765700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:05.559825+0000 front 2024-04-24T23:33:05.559812+0000 (oldest deadline 2024-04-24T23:33:29.560381+0000) 2024-04-24T23:33:41.007 INFO:journalctl@ceph.osd.8.smithi148.stdout:Apr 24 23:33:40 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-8[51367]: 2024-04-24T23:33:40.711+0000 7f119e765700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:13.061482+0000 front 2024-04-24T23:33:13.061509+0000 (oldest deadline 2024-04-24T23:33:36.561622+0000) 2024-04-24T23:33:41.361 INFO:journalctl@ceph.osd.5.smithi148.stdout:Apr 24 23:33:41 smithi148 podman[67156]: 2024-04-24 23:33:41.043037442 +0000 UTC m=+0.660012379 container remove c2aa36f25152d45fec914f505bf9106b14a67890a8b6a565b9cf3c90fc95da66 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, GIT_CLEAN=True) 2024-04-24T23:33:41.362 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:33:41 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:33:41.175+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.848330+0000 front 2024-04-24T23:32:55.848387+0000 (oldest deadline 2024-04-24T23:33:20.548352+0000) 2024-04-24T23:33:41.362 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:33:41 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:33:41.175+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:02.150412+0000 front 2024-04-24T23:33:02.150294+0000 (oldest deadline 2024-04-24T23:33:28.050409+0000) 2024-04-24T23:33:41.362 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:33:41 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:33:41.175+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:14.952538+0000 front 2024-04-24T23:33:14.952635+0000 (oldest deadline 2024-04-24T23:33:40.852722+0000) 2024-04-24T23:33:41.672 INFO:journalctl@ceph.osd.5.smithi148.stdout:Apr 24 23:33:41 smithi148 podman[67248]: 2024-04-24 23:33:41.265059536 +0000 UTC m=+0.017926518 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:33:41.672 INFO:journalctl@ceph.osd.5.smithi148.stdout:Apr 24 23:33:41 smithi148 podman[67248]: 2024-04-24 23:33:41.410623388 +0000 UTC m=+0.163490420 container create 2e65540bea6ab305f3be81060692ed3e7346d4f5bdd76dc3d82a35e8fddfc506 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-04-24T23:33:41.672 INFO:journalctl@ceph.osd.5.smithi148.stdout:Apr 24 23:33:41 smithi148 podman[67248]: 2024-04-24 23:33:41.672747891 +0000 UTC m=+0.425614860 container init 2e65540bea6ab305f3be81060692ed3e7346d4f5bdd76dc3d82a35e8fddfc506 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-04-24T23:33:41.697 INFO:journalctl@ceph.osd.7.smithi134.stdout:Apr 24 23:33:41 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T23:33:41.401+0000 7f676da99700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:05.625943+0000 front 2024-04-24T23:33:05.625903+0000 (oldest deadline 2024-04-24T23:33:29.126101+0000) 2024-04-24T23:33:41.697 INFO:journalctl@ceph.osd.7.smithi134.stdout:Apr 24 23:33:41 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T23:33:41.401+0000 7f676da99700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:14.927842+0000 front 2024-04-24T23:33:14.927944+0000 (oldest deadline 2024-04-24T23:33:37.827908+0000) 2024-04-24T23:33:41.697 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:33:41 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:33:41.257+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:56.929716+0000 front 2024-04-24T23:32:56.929774+0000 (oldest deadline 2024-04-24T23:33:20.429679+0000) 2024-04-24T23:33:41.697 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:33:41 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:33:41.257+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:04.530694+0000 front 2024-04-24T23:33:04.530743+0000 (oldest deadline 2024-04-24T23:33:29.230721+0000) 2024-04-24T23:33:41.697 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:33:41 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:33:41.257+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:15.632228+0000 front 2024-04-24T23:33:15.631978+0000 (oldest deadline 2024-04-24T23:33:37.331905+0000) 2024-04-24T23:33:41.788 INFO:journalctl@ceph.osd.6.smithi080.stdout:Apr 24 23:33:41 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-6[57095]: 2024-04-24T23:33:41.358+0000 7fee6c0e2700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:06.138406+0000 front 2024-04-24T23:33:06.138353+0000 (oldest deadline 2024-04-24T23:33:32.038442+0000) 2024-04-24T23:33:41.788 INFO:journalctl@ceph.osd.6.smithi080.stdout:Apr 24 23:33:41 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-6[57095]: 2024-04-24T23:33:41.358+0000 7fee6c0e2700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:15.539424+0000 front 2024-04-24T23:33:15.539472+0000 (oldest deadline 2024-04-24T23:33:40.839483+0000) 2024-04-24T23:33:41.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:33:41 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:33:41.347+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.738540+0000 front 2024-04-24T23:32:55.738561+0000 (oldest deadline 2024-04-24T23:33:21.038786+0000) 2024-04-24T23:33:41.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:33:41 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:33:41.347+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:05.740014+0000 front 2024-04-24T23:33:05.739865+0000 (oldest deadline 2024-04-24T23:33:29.839897+0000) 2024-04-24T23:33:41.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:33:41 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:33:41.347+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:12.140601+0000 front 2024-04-24T23:33:12.140627+0000 (oldest deadline 2024-04-24T23:33:38.040825+0000) 2024-04-24T23:33:41.962 INFO:journalctl@ceph.osd.8.smithi148.stdout:Apr 24 23:33:41 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-8[51367]: 2024-04-24T23:33:41.725+0000 7f119e765700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:05.559825+0000 front 2024-04-24T23:33:05.559812+0000 (oldest deadline 2024-04-24T23:33:29.560381+0000) 2024-04-24T23:33:41.962 INFO:journalctl@ceph.osd.8.smithi148.stdout:Apr 24 23:33:41 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-8[51367]: 2024-04-24T23:33:41.725+0000 7f119e765700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:13.061482+0000 front 2024-04-24T23:33:13.061509+0000 (oldest deadline 2024-04-24T23:33:36.561622+0000) 2024-04-24T23:33:41.963 INFO:journalctl@ceph.osd.5.smithi148.stdout:Apr 24 23:33:41 smithi148 podman[67248]: 2024-04-24 23:33:41.67582252 +0000 UTC m=+0.428689481 container start 2e65540bea6ab305f3be81060692ed3e7346d4f5bdd76dc3d82a35e8fddfc506 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-5-deactivate, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, io.buildah.version=1.33.5) 2024-04-24T23:33:41.963 INFO:journalctl@ceph.osd.5.smithi148.stdout:Apr 24 23:33:41 smithi148 podman[67248]: 2024-04-24 23:33:41.772991448 +0000 UTC m=+0.525858418 container attach 2e65540bea6ab305f3be81060692ed3e7346d4f5bdd76dc3d82a35e8fddfc506 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240424, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-04-24T23:33:42.256 INFO:journalctl@ceph.osd.5.smithi148.stdout:Apr 24 23:33:41 smithi148 podman[67248]: 2024-04-24 23:33:41.963078933 +0000 UTC m=+0.715945911 container died 2e65540bea6ab305f3be81060692ed3e7346d4f5bdd76dc3d82a35e8fddfc506 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-5-deactivate, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-04-24T23:33:42.256 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:33:42 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:33:42.157+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.848330+0000 front 2024-04-24T23:32:55.848387+0000 (oldest deadline 2024-04-24T23:33:20.548352+0000) 2024-04-24T23:33:42.257 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:33:42 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:33:42.157+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:02.150412+0000 front 2024-04-24T23:33:02.150294+0000 (oldest deadline 2024-04-24T23:33:28.050409+0000) 2024-04-24T23:33:42.257 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:33:42 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:33:42.157+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:14.952538+0000 front 2024-04-24T23:33:14.952635+0000 (oldest deadline 2024-04-24T23:33:40.852722+0000) 2024-04-24T23:33:42.623 DEBUG:teuthology.orchestra.run.smithi148:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ec7885aa-0290-11ef-bc93-c7b262605968@osd.5.service' 2024-04-24T23:33:42.660 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:33:42.660 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-04-24T23:33:42.661 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-04-24T23:33:42.661 DEBUG:teuthology.orchestra.run.smithi080:> sudo systemctl stop ceph-ec7885aa-0290-11ef-bc93-c7b262605968@osd.6 2024-04-24T23:33:42.689 INFO:journalctl@ceph.osd.6.smithi080.stdout:Apr 24 23:33:42 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-6[57095]: 2024-04-24T23:33:42.345+0000 7fee6c0e2700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:06.138406+0000 front 2024-04-24T23:33:06.138353+0000 (oldest deadline 2024-04-24T23:33:32.038442+0000) 2024-04-24T23:33:42.689 INFO:journalctl@ceph.osd.6.smithi080.stdout:Apr 24 23:33:42 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-6[57095]: 2024-04-24T23:33:42.345+0000 7fee6c0e2700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:15.539424+0000 front 2024-04-24T23:33:15.539472+0000 (oldest deadline 2024-04-24T23:33:40.839483+0000) 2024-04-24T23:33:42.690 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:33:42 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:33:42.373+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.738540+0000 front 2024-04-24T23:32:55.738561+0000 (oldest deadline 2024-04-24T23:33:21.038786+0000) 2024-04-24T23:33:42.690 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:33:42 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:33:42.373+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:05.740014+0000 front 2024-04-24T23:33:05.739865+0000 (oldest deadline 2024-04-24T23:33:29.839897+0000) 2024-04-24T23:33:42.690 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:33:42 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:33:42.373+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:12.140601+0000 front 2024-04-24T23:33:12.140627+0000 (oldest deadline 2024-04-24T23:33:38.040825+0000) 2024-04-24T23:33:42.696 INFO:journalctl@ceph.osd.7.smithi134.stdout:Apr 24 23:33:42 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T23:33:42.410+0000 7f676da99700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:05.625943+0000 front 2024-04-24T23:33:05.625903+0000 (oldest deadline 2024-04-24T23:33:29.126101+0000) 2024-04-24T23:33:42.696 INFO:journalctl@ceph.osd.7.smithi134.stdout:Apr 24 23:33:42 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T23:33:42.410+0000 7f676da99700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:14.927842+0000 front 2024-04-24T23:33:14.927944+0000 (oldest deadline 2024-04-24T23:33:37.827908+0000) 2024-04-24T23:33:42.697 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:33:42 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:33:42.291+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:56.929716+0000 front 2024-04-24T23:32:56.929774+0000 (oldest deadline 2024-04-24T23:33:20.429679+0000) 2024-04-24T23:33:42.697 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:33:42 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:33:42.291+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:04.530694+0000 front 2024-04-24T23:33:04.530743+0000 (oldest deadline 2024-04-24T23:33:29.230721+0000) 2024-04-24T23:33:42.697 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:33:42 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:33:42.291+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:15.632228+0000 front 2024-04-24T23:33:15.631978+0000 (oldest deadline 2024-04-24T23:33:37.331905+0000) 2024-04-24T23:33:42.965 INFO:journalctl@ceph.osd.6.smithi080.stdout:Apr 24 23:33:42 smithi080 systemd[1]: Stopping Ceph osd.6 for ec7885aa-0290-11ef-bc93-c7b262605968... 2024-04-24T23:33:43.005 INFO:journalctl@ceph.osd.8.smithi148.stdout:Apr 24 23:33:42 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-8[51367]: 2024-04-24T23:33:42.772+0000 7f119e765700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:05.559825+0000 front 2024-04-24T23:33:05.559812+0000 (oldest deadline 2024-04-24T23:33:29.560381+0000) 2024-04-24T23:33:43.005 INFO:journalctl@ceph.osd.8.smithi148.stdout:Apr 24 23:33:42 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-8[51367]: 2024-04-24T23:33:42.772+0000 7f119e765700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:13.061482+0000 front 2024-04-24T23:33:13.061509+0000 (oldest deadline 2024-04-24T23:33:36.561622+0000) 2024-04-24T23:33:43.288 INFO:journalctl@ceph.osd.6.smithi080.stdout:Apr 24 23:33:42 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-6[57095]: 2024-04-24T23:33:42.964+0000 7fee73af6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:33:43.288 INFO:journalctl@ceph.osd.6.smithi080.stdout:Apr 24 23:33:42 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-6[57095]: 2024-04-24T23:33:42.964+0000 7fee73af6700 -1 osd.6 98 *** Got signal Terminated *** 2024-04-24T23:33:43.288 INFO:journalctl@ceph.osd.6.smithi080.stdout:Apr 24 23:33:42 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-6[57095]: 2024-04-24T23:33:42.964+0000 7fee73af6700 -1 osd.6 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T23:33:43.506 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:33:43 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:33:43.161+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.848330+0000 front 2024-04-24T23:32:55.848387+0000 (oldest deadline 2024-04-24T23:33:20.548352+0000) 2024-04-24T23:33:43.507 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:33:43 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:33:43.161+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:02.150412+0000 front 2024-04-24T23:33:02.150294+0000 (oldest deadline 2024-04-24T23:33:28.050409+0000) 2024-04-24T23:33:43.507 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:33:43 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:33:43.161+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:14.952538+0000 front 2024-04-24T23:33:14.952635+0000 (oldest deadline 2024-04-24T23:33:40.852722+0000) 2024-04-24T23:33:43.697 INFO:journalctl@ceph.osd.7.smithi134.stdout:Apr 24 23:33:43 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T23:33:43.458+0000 7f676da99700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:05.625943+0000 front 2024-04-24T23:33:05.625903+0000 (oldest deadline 2024-04-24T23:33:29.126101+0000) 2024-04-24T23:33:43.697 INFO:journalctl@ceph.osd.7.smithi134.stdout:Apr 24 23:33:43 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T23:33:43.458+0000 7f676da99700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:14.927842+0000 front 2024-04-24T23:33:14.927944+0000 (oldest deadline 2024-04-24T23:33:37.827908+0000) 2024-04-24T23:33:43.697 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:33:43 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:33:43.244+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:56.929716+0000 front 2024-04-24T23:32:56.929774+0000 (oldest deadline 2024-04-24T23:33:20.429679+0000) 2024-04-24T23:33:43.697 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:33:43 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:33:43.244+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:04.530694+0000 front 2024-04-24T23:33:04.530743+0000 (oldest deadline 2024-04-24T23:33:29.230721+0000) 2024-04-24T23:33:43.697 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:33:43 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:33:43.244+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:15.632228+0000 front 2024-04-24T23:33:15.631978+0000 (oldest deadline 2024-04-24T23:33:37.331905+0000) 2024-04-24T23:33:43.788 INFO:journalctl@ceph.osd.6.smithi080.stdout:Apr 24 23:33:43 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-6[57095]: 2024-04-24T23:33:43.326+0000 7fee6c0e2700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:06.138406+0000 front 2024-04-24T23:33:06.138353+0000 (oldest deadline 2024-04-24T23:33:32.038442+0000) 2024-04-24T23:33:43.788 INFO:journalctl@ceph.osd.6.smithi080.stdout:Apr 24 23:33:43 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-6[57095]: 2024-04-24T23:33:43.326+0000 7fee6c0e2700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:15.539424+0000 front 2024-04-24T23:33:15.539472+0000 (oldest deadline 2024-04-24T23:33:40.839483+0000) 2024-04-24T23:33:43.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:33:43 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:33:43.384+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.738540+0000 front 2024-04-24T23:32:55.738561+0000 (oldest deadline 2024-04-24T23:33:21.038786+0000) 2024-04-24T23:33:43.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:33:43 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:33:43.384+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:05.740014+0000 front 2024-04-24T23:33:05.739865+0000 (oldest deadline 2024-04-24T23:33:29.839897+0000) 2024-04-24T23:33:43.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:33:43 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:33:43.384+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:12.140601+0000 front 2024-04-24T23:33:12.140627+0000 (oldest deadline 2024-04-24T23:33:38.040825+0000) 2024-04-24T23:33:44.159 INFO:journalctl@ceph.osd.8.smithi148.stdout:Apr 24 23:33:43 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-8[51367]: 2024-04-24T23:33:43.793+0000 7f119e765700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:05.559825+0000 front 2024-04-24T23:33:05.559812+0000 (oldest deadline 2024-04-24T23:33:29.560381+0000) 2024-04-24T23:33:44.159 INFO:journalctl@ceph.osd.8.smithi148.stdout:Apr 24 23:33:43 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-8[51367]: 2024-04-24T23:33:43.793+0000 7f119e765700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:13.061482+0000 front 2024-04-24T23:33:13.061509+0000 (oldest deadline 2024-04-24T23:33:36.561622+0000) 2024-04-24T23:33:44.482 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:33:44 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:33:44.202+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:56.929716+0000 front 2024-04-24T23:32:56.929774+0000 (oldest deadline 2024-04-24T23:33:20.429679+0000) 2024-04-24T23:33:44.483 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:33:44 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:33:44.202+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:04.530694+0000 front 2024-04-24T23:33:04.530743+0000 (oldest deadline 2024-04-24T23:33:29.230721+0000) 2024-04-24T23:33:44.483 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:33:44 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:33:44.202+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:15.632228+0000 front 2024-04-24T23:33:15.631978+0000 (oldest deadline 2024-04-24T23:33:37.331905+0000) 2024-04-24T23:33:44.505 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:33:44 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:33:44.158+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.848330+0000 front 2024-04-24T23:32:55.848387+0000 (oldest deadline 2024-04-24T23:33:20.548352+0000) 2024-04-24T23:33:44.506 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:33:44 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:33:44.158+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:02.150412+0000 front 2024-04-24T23:33:02.150294+0000 (oldest deadline 2024-04-24T23:33:28.050409+0000) 2024-04-24T23:33:44.506 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:33:44 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:33:44.158+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:14.952538+0000 front 2024-04-24T23:33:14.952635+0000 (oldest deadline 2024-04-24T23:33:40.852722+0000) 2024-04-24T23:33:44.537 INFO:journalctl@ceph.osd.6.smithi080.stdout:Apr 24 23:33:44 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-6[57095]: 2024-04-24T23:33:44.285+0000 7fee6c0e2700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:06.138406+0000 front 2024-04-24T23:33:06.138353+0000 (oldest deadline 2024-04-24T23:33:32.038442+0000) 2024-04-24T23:33:44.538 INFO:journalctl@ceph.osd.6.smithi080.stdout:Apr 24 23:33:44 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-6[57095]: 2024-04-24T23:33:44.285+0000 7fee6c0e2700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:15.539424+0000 front 2024-04-24T23:33:15.539472+0000 (oldest deadline 2024-04-24T23:33:40.839483+0000) 2024-04-24T23:33:44.538 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:33:44 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:33:44.434+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.738540+0000 front 2024-04-24T23:32:55.738561+0000 (oldest deadline 2024-04-24T23:33:21.038786+0000) 2024-04-24T23:33:44.538 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:33:44 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:33:44.434+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:05.740014+0000 front 2024-04-24T23:33:05.739865+0000 (oldest deadline 2024-04-24T23:33:29.839897+0000) 2024-04-24T23:33:44.538 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:33:44 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:33:44.434+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:12.140601+0000 front 2024-04-24T23:33:12.140627+0000 (oldest deadline 2024-04-24T23:33:38.040825+0000) 2024-04-24T23:33:44.946 INFO:journalctl@ceph.osd.7.smithi134.stdout:Apr 24 23:33:44 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T23:33:44.482+0000 7f676da99700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:05.625943+0000 front 2024-04-24T23:33:05.625903+0000 (oldest deadline 2024-04-24T23:33:29.126101+0000) 2024-04-24T23:33:44.947 INFO:journalctl@ceph.osd.7.smithi134.stdout:Apr 24 23:33:44 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T23:33:44.482+0000 7f676da99700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:14.927842+0000 front 2024-04-24T23:33:14.927944+0000 (oldest deadline 2024-04-24T23:33:37.827908+0000) 2024-04-24T23:33:45.005 INFO:journalctl@ceph.osd.8.smithi148.stdout:Apr 24 23:33:44 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-8[51367]: 2024-04-24T23:33:44.752+0000 7f119e765700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:05.559825+0000 front 2024-04-24T23:33:05.559812+0000 (oldest deadline 2024-04-24T23:33:29.560381+0000) 2024-04-24T23:33:45.006 INFO:journalctl@ceph.osd.8.smithi148.stdout:Apr 24 23:33:44 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-8[51367]: 2024-04-24T23:33:44.752+0000 7f119e765700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:13.061482+0000 front 2024-04-24T23:33:13.061509+0000 (oldest deadline 2024-04-24T23:33:36.561622+0000) 2024-04-24T23:33:45.006 INFO:journalctl@ceph.osd.8.smithi148.stdout:Apr 24 23:33:44 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-8[51367]: 2024-04-24T23:33:44.752+0000 7f119e765700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.80:6814 osd.3 since back 2024-04-24T23:33:22.863404+0000 front 2024-04-24T23:33:22.863339+0000 (oldest deadline 2024-04-24T23:33:44.563342+0000) 2024-04-24T23:33:45.506 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:33:45 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:33:45.183+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.848330+0000 front 2024-04-24T23:32:55.848387+0000 (oldest deadline 2024-04-24T23:33:20.548352+0000) 2024-04-24T23:33:45.506 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:33:45 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:33:45.183+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:02.150412+0000 front 2024-04-24T23:33:02.150294+0000 (oldest deadline 2024-04-24T23:33:28.050409+0000) 2024-04-24T23:33:45.506 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:33:45 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:33:45.183+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:14.952538+0000 front 2024-04-24T23:33:14.952635+0000 (oldest deadline 2024-04-24T23:33:40.852722+0000) 2024-04-24T23:33:45.508 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:33:45 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:33:45.233+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:56.929716+0000 front 2024-04-24T23:32:56.929774+0000 (oldest deadline 2024-04-24T23:33:20.429679+0000) 2024-04-24T23:33:45.508 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:33:45 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:33:45.233+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:04.530694+0000 front 2024-04-24T23:33:04.530743+0000 (oldest deadline 2024-04-24T23:33:29.230721+0000) 2024-04-24T23:33:45.509 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:33:45 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:33:45.233+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:15.632228+0000 front 2024-04-24T23:33:15.631978+0000 (oldest deadline 2024-04-24T23:33:37.331905+0000) 2024-04-24T23:33:45.509 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:33:45 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:33:45.233+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.80:6814 osd.3 since back 2024-04-24T23:33:22.632991+0000 front 2024-04-24T23:33:22.633039+0000 (oldest deadline 2024-04-24T23:33:44.932941+0000) 2024-04-24T23:33:45.538 INFO:journalctl@ceph.osd.6.smithi080.stdout:Apr 24 23:33:45 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-6[57095]: 2024-04-24T23:33:45.282+0000 7fee6c0e2700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:06.138406+0000 front 2024-04-24T23:33:06.138353+0000 (oldest deadline 2024-04-24T23:33:32.038442+0000) 2024-04-24T23:33:45.538 INFO:journalctl@ceph.osd.6.smithi080.stdout:Apr 24 23:33:45 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-6[57095]: 2024-04-24T23:33:45.282+0000 7fee6c0e2700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:15.539424+0000 front 2024-04-24T23:33:15.539472+0000 (oldest deadline 2024-04-24T23:33:40.839483+0000) 2024-04-24T23:33:45.538 INFO:journalctl@ceph.osd.6.smithi080.stdout:Apr 24 23:33:45 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-6[57095]: 2024-04-24T23:33:45.282+0000 7fee6c0e2700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.80:6814 osd.3 since back 2024-04-24T23:33:20.839793+0000 front 2024-04-24T23:33:20.839857+0000 (oldest deadline 2024-04-24T23:33:44.939901+0000) 2024-04-24T23:33:45.538 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:33:45 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:33:45.458+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.738540+0000 front 2024-04-24T23:32:55.738561+0000 (oldest deadline 2024-04-24T23:33:21.038786+0000) 2024-04-24T23:33:45.539 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:33:45 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:33:45.458+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:05.740014+0000 front 2024-04-24T23:33:05.739865+0000 (oldest deadline 2024-04-24T23:33:29.839897+0000) 2024-04-24T23:33:45.539 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:33:45 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:33:45.458+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:12.140601+0000 front 2024-04-24T23:33:12.140627+0000 (oldest deadline 2024-04-24T23:33:38.040825+0000) 2024-04-24T23:33:45.946 INFO:journalctl@ceph.osd.7.smithi134.stdout:Apr 24 23:33:45 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T23:33:45.508+0000 7f676da99700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:05.625943+0000 front 2024-04-24T23:33:05.625903+0000 (oldest deadline 2024-04-24T23:33:29.126101+0000) 2024-04-24T23:33:45.947 INFO:journalctl@ceph.osd.7.smithi134.stdout:Apr 24 23:33:45 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T23:33:45.508+0000 7f676da99700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:14.927842+0000 front 2024-04-24T23:33:14.927944+0000 (oldest deadline 2024-04-24T23:33:37.827908+0000) 2024-04-24T23:33:45.947 INFO:journalctl@ceph.osd.7.smithi134.stdout:Apr 24 23:33:45 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T23:33:45.508+0000 7f676da99700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.80:6814 osd.3 since back 2024-04-24T23:33:22.929618+0000 front 2024-04-24T23:33:22.929735+0000 (oldest deadline 2024-04-24T23:33:45.229659+0000) 2024-04-24T23:33:46.005 INFO:journalctl@ceph.osd.8.smithi148.stdout:Apr 24 23:33:45 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-8[51367]: 2024-04-24T23:33:45.706+0000 7f119e765700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:05.559825+0000 front 2024-04-24T23:33:05.559812+0000 (oldest deadline 2024-04-24T23:33:29.560381+0000) 2024-04-24T23:33:46.006 INFO:journalctl@ceph.osd.8.smithi148.stdout:Apr 24 23:33:45 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-8[51367]: 2024-04-24T23:33:45.706+0000 7f119e765700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:13.061482+0000 front 2024-04-24T23:33:13.061509+0000 (oldest deadline 2024-04-24T23:33:36.561622+0000) 2024-04-24T23:33:46.006 INFO:journalctl@ceph.osd.8.smithi148.stdout:Apr 24 23:33:45 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-8[51367]: 2024-04-24T23:33:45.706+0000 7f119e765700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.80:6814 osd.3 since back 2024-04-24T23:33:22.863404+0000 front 2024-04-24T23:33:22.863339+0000 (oldest deadline 2024-04-24T23:33:44.563342+0000) 2024-04-24T23:33:46.280 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:33:46 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:33:46.162+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.848330+0000 front 2024-04-24T23:32:55.848387+0000 (oldest deadline 2024-04-24T23:33:20.548352+0000) 2024-04-24T23:33:46.280 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:33:46 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:33:46.162+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:02.150412+0000 front 2024-04-24T23:33:02.150294+0000 (oldest deadline 2024-04-24T23:33:28.050409+0000) 2024-04-24T23:33:46.280 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:33:46 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:33:46.162+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:14.952538+0000 front 2024-04-24T23:33:14.952635+0000 (oldest deadline 2024-04-24T23:33:40.852722+0000) 2024-04-24T23:33:46.447 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:33:46 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:33:46.185+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:56.929716+0000 front 2024-04-24T23:32:56.929774+0000 (oldest deadline 2024-04-24T23:33:20.429679+0000) 2024-04-24T23:33:46.447 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:33:46 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:33:46.185+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:04.530694+0000 front 2024-04-24T23:33:04.530743+0000 (oldest deadline 2024-04-24T23:33:29.230721+0000) 2024-04-24T23:33:46.447 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:33:46 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:33:46.185+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:15.632228+0000 front 2024-04-24T23:33:15.631978+0000 (oldest deadline 2024-04-24T23:33:37.331905+0000) 2024-04-24T23:33:46.447 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:33:46 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:33:46.185+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.80:6814 osd.3 since back 2024-04-24T23:33:22.632991+0000 front 2024-04-24T23:33:22.633039+0000 (oldest deadline 2024-04-24T23:33:44.932941+0000) 2024-04-24T23:33:46.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:33:46 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:33:46.465+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.738540+0000 front 2024-04-24T23:32:55.738561+0000 (oldest deadline 2024-04-24T23:33:21.038786+0000) 2024-04-24T23:33:46.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:33:46 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:33:46.465+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:05.740014+0000 front 2024-04-24T23:33:05.739865+0000 (oldest deadline 2024-04-24T23:33:29.839897+0000) 2024-04-24T23:33:46.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:33:46 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:33:46.465+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:12.140601+0000 front 2024-04-24T23:33:12.140627+0000 (oldest deadline 2024-04-24T23:33:38.040825+0000) 2024-04-24T23:33:46.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:33:46 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:33:46.465+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.80:6814 osd.3 since back 2024-04-24T23:33:22.042138+0000 front 2024-04-24T23:33:22.042191+0000 (oldest deadline 2024-04-24T23:33:45.542231+0000) 2024-04-24T23:33:46.789 INFO:journalctl@ceph.osd.6.smithi080.stdout:Apr 24 23:33:46 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-6[57095]: 2024-04-24T23:33:46.303+0000 7fee6c0e2700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:06.138406+0000 front 2024-04-24T23:33:06.138353+0000 (oldest deadline 2024-04-24T23:33:32.038442+0000) 2024-04-24T23:33:46.789 INFO:journalctl@ceph.osd.6.smithi080.stdout:Apr 24 23:33:46 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-6[57095]: 2024-04-24T23:33:46.303+0000 7fee6c0e2700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:15.539424+0000 front 2024-04-24T23:33:15.539472+0000 (oldest deadline 2024-04-24T23:33:40.839483+0000) 2024-04-24T23:33:46.789 INFO:journalctl@ceph.osd.6.smithi080.stdout:Apr 24 23:33:46 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-6[57095]: 2024-04-24T23:33:46.303+0000 7fee6c0e2700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.80:6814 osd.3 since back 2024-04-24T23:33:20.839793+0000 front 2024-04-24T23:33:20.839857+0000 (oldest deadline 2024-04-24T23:33:44.939901+0000) 2024-04-24T23:33:46.946 INFO:journalctl@ceph.osd.7.smithi134.stdout:Apr 24 23:33:46 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T23:33:46.537+0000 7f676da99700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:05.625943+0000 front 2024-04-24T23:33:05.625903+0000 (oldest deadline 2024-04-24T23:33:29.126101+0000) 2024-04-24T23:33:46.982 INFO:journalctl@ceph.osd.7.smithi134.stdout:Apr 24 23:33:46 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T23:33:46.537+0000 7f676da99700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:14.927842+0000 front 2024-04-24T23:33:14.927944+0000 (oldest deadline 2024-04-24T23:33:37.827908+0000) 2024-04-24T23:33:46.982 INFO:journalctl@ceph.osd.7.smithi134.stdout:Apr 24 23:33:46 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T23:33:46.537+0000 7f676da99700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.80:6814 osd.3 since back 2024-04-24T23:33:22.929618+0000 front 2024-04-24T23:33:22.929735+0000 (oldest deadline 2024-04-24T23:33:45.229659+0000) 2024-04-24T23:33:47.005 INFO:journalctl@ceph.osd.8.smithi148.stdout:Apr 24 23:33:46 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-8[51367]: 2024-04-24T23:33:46.709+0000 7f119e765700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:05.559825+0000 front 2024-04-24T23:33:05.559812+0000 (oldest deadline 2024-04-24T23:33:29.560381+0000) 2024-04-24T23:33:47.005 INFO:journalctl@ceph.osd.8.smithi148.stdout:Apr 24 23:33:46 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-8[51367]: 2024-04-24T23:33:46.709+0000 7f119e765700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:13.061482+0000 front 2024-04-24T23:33:13.061509+0000 (oldest deadline 2024-04-24T23:33:36.561622+0000) 2024-04-24T23:33:47.006 INFO:journalctl@ceph.osd.8.smithi148.stdout:Apr 24 23:33:46 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-8[51367]: 2024-04-24T23:33:46.709+0000 7f119e765700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.80:6814 osd.3 since back 2024-04-24T23:33:22.863404+0000 front 2024-04-24T23:33:22.863339+0000 (oldest deadline 2024-04-24T23:33:44.563342+0000) 2024-04-24T23:33:47.446 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:33:47 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:33:47.136+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:56.929716+0000 front 2024-04-24T23:32:56.929774+0000 (oldest deadline 2024-04-24T23:33:20.429679+0000) 2024-04-24T23:33:47.447 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:33:47 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:33:47.136+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:04.530694+0000 front 2024-04-24T23:33:04.530743+0000 (oldest deadline 2024-04-24T23:33:29.230721+0000) 2024-04-24T23:33:47.447 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:33:47 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:33:47.136+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:15.632228+0000 front 2024-04-24T23:33:15.631978+0000 (oldest deadline 2024-04-24T23:33:37.331905+0000) 2024-04-24T23:33:47.447 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:33:47 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:33:47.136+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.80:6814 osd.3 since back 2024-04-24T23:33:22.632991+0000 front 2024-04-24T23:33:22.633039+0000 (oldest deadline 2024-04-24T23:33:44.932941+0000) 2024-04-24T23:33:47.506 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:33:47 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:33:47.113+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.848330+0000 front 2024-04-24T23:32:55.848387+0000 (oldest deadline 2024-04-24T23:33:20.548352+0000) 2024-04-24T23:33:47.506 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:33:47 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:33:47.113+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:02.150412+0000 front 2024-04-24T23:33:02.150294+0000 (oldest deadline 2024-04-24T23:33:28.050409+0000) 2024-04-24T23:33:47.506 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:33:47 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:33:47.113+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:14.952538+0000 front 2024-04-24T23:33:14.952635+0000 (oldest deadline 2024-04-24T23:33:40.852722+0000) 2024-04-24T23:33:47.538 INFO:journalctl@ceph.osd.6.smithi080.stdout:Apr 24 23:33:47 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-6[57095]: 2024-04-24T23:33:47.270+0000 7fee6c0e2700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:06.138406+0000 front 2024-04-24T23:33:06.138353+0000 (oldest deadline 2024-04-24T23:33:32.038442+0000) 2024-04-24T23:33:47.538 INFO:journalctl@ceph.osd.6.smithi080.stdout:Apr 24 23:33:47 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-6[57095]: 2024-04-24T23:33:47.270+0000 7fee6c0e2700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:15.539424+0000 front 2024-04-24T23:33:15.539472+0000 (oldest deadline 2024-04-24T23:33:40.839483+0000) 2024-04-24T23:33:47.538 INFO:journalctl@ceph.osd.6.smithi080.stdout:Apr 24 23:33:47 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-6[57095]: 2024-04-24T23:33:47.270+0000 7fee6c0e2700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.80:6814 osd.3 since back 2024-04-24T23:33:20.839793+0000 front 2024-04-24T23:33:20.839857+0000 (oldest deadline 2024-04-24T23:33:44.939901+0000) 2024-04-24T23:33:47.538 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:33:47 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:33:47.446+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.738540+0000 front 2024-04-24T23:32:55.738561+0000 (oldest deadline 2024-04-24T23:33:21.038786+0000) 2024-04-24T23:33:47.538 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:33:47 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:33:47.446+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:05.740014+0000 front 2024-04-24T23:33:05.739865+0000 (oldest deadline 2024-04-24T23:33:29.839897+0000) 2024-04-24T23:33:47.539 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:33:47 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:33:47.446+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:12.140601+0000 front 2024-04-24T23:33:12.140627+0000 (oldest deadline 2024-04-24T23:33:38.040825+0000) 2024-04-24T23:33:47.539 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:33:47 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:33:47.446+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.80:6814 osd.3 since back 2024-04-24T23:33:22.042138+0000 front 2024-04-24T23:33:22.042191+0000 (oldest deadline 2024-04-24T23:33:45.542231+0000) 2024-04-24T23:33:47.947 INFO:journalctl@ceph.osd.7.smithi134.stdout:Apr 24 23:33:47 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T23:33:47.489+0000 7f676da99700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:05.625943+0000 front 2024-04-24T23:33:05.625903+0000 (oldest deadline 2024-04-24T23:33:29.126101+0000) 2024-04-24T23:33:47.947 INFO:journalctl@ceph.osd.7.smithi134.stdout:Apr 24 23:33:47 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T23:33:47.489+0000 7f676da99700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:14.927842+0000 front 2024-04-24T23:33:14.927944+0000 (oldest deadline 2024-04-24T23:33:37.827908+0000) 2024-04-24T23:33:47.947 INFO:journalctl@ceph.osd.7.smithi134.stdout:Apr 24 23:33:47 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T23:33:47.489+0000 7f676da99700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.80:6814 osd.3 since back 2024-04-24T23:33:22.929618+0000 front 2024-04-24T23:33:22.929735+0000 (oldest deadline 2024-04-24T23:33:45.229659+0000) 2024-04-24T23:33:48.006 INFO:journalctl@ceph.osd.8.smithi148.stdout:Apr 24 23:33:47 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-8[51367]: 2024-04-24T23:33:47.712+0000 7f119e765700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:05.559825+0000 front 2024-04-24T23:33:05.559812+0000 (oldest deadline 2024-04-24T23:33:29.560381+0000) 2024-04-24T23:33:48.006 INFO:journalctl@ceph.osd.8.smithi148.stdout:Apr 24 23:33:47 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-8[51367]: 2024-04-24T23:33:47.712+0000 7f119e765700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:13.061482+0000 front 2024-04-24T23:33:13.061509+0000 (oldest deadline 2024-04-24T23:33:36.561622+0000) 2024-04-24T23:33:48.009 INFO:journalctl@ceph.osd.8.smithi148.stdout:Apr 24 23:33:47 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-8[51367]: 2024-04-24T23:33:47.712+0000 7f119e765700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.80:6814 osd.3 since back 2024-04-24T23:33:22.863404+0000 front 2024-04-24T23:33:22.863339+0000 (oldest deadline 2024-04-24T23:33:44.563342+0000) 2024-04-24T23:33:48.446 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:33:48 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:33:48.118+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:56.929716+0000 front 2024-04-24T23:32:56.929774+0000 (oldest deadline 2024-04-24T23:33:20.429679+0000) 2024-04-24T23:33:48.447 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:33:48 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:33:48.118+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:04.530694+0000 front 2024-04-24T23:33:04.530743+0000 (oldest deadline 2024-04-24T23:33:29.230721+0000) 2024-04-24T23:33:48.447 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:33:48 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:33:48.118+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:15.632228+0000 front 2024-04-24T23:33:15.631978+0000 (oldest deadline 2024-04-24T23:33:37.331905+0000) 2024-04-24T23:33:48.448 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:33:48 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:33:48.118+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.80:6814 osd.3 since back 2024-04-24T23:33:22.632991+0000 front 2024-04-24T23:33:22.633039+0000 (oldest deadline 2024-04-24T23:33:44.932941+0000) 2024-04-24T23:33:48.506 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:33:48 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:33:48.123+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.848330+0000 front 2024-04-24T23:32:55.848387+0000 (oldest deadline 2024-04-24T23:33:20.548352+0000) 2024-04-24T23:33:48.506 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:33:48 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:33:48.123+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:02.150412+0000 front 2024-04-24T23:33:02.150294+0000 (oldest deadline 2024-04-24T23:33:28.050409+0000) 2024-04-24T23:33:48.506 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:33:48 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:33:48.123+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:14.952538+0000 front 2024-04-24T23:33:14.952635+0000 (oldest deadline 2024-04-24T23:33:40.852722+0000) 2024-04-24T23:33:48.723 INFO:journalctl@ceph.osd.6.smithi080.stdout:Apr 24 23:33:48 smithi080 podman[94794]: 2024-04-24 23:33:48.460003972 +0000 UTC m=+5.605661635 container died c6137981ab8f7a27ca440f1c0ef3036146eae75ebb730bf2cc553bdadeb3c205 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-6, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T23:33:48.724 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:33:48 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:33:48.442+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.738540+0000 front 2024-04-24T23:32:55.738561+0000 (oldest deadline 2024-04-24T23:33:21.038786+0000) 2024-04-24T23:33:48.724 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:33:48 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:33:48.442+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:05.740014+0000 front 2024-04-24T23:33:05.739865+0000 (oldest deadline 2024-04-24T23:33:29.839897+0000) 2024-04-24T23:33:48.724 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:33:48 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:33:48.442+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:12.140601+0000 front 2024-04-24T23:33:12.140627+0000 (oldest deadline 2024-04-24T23:33:38.040825+0000) 2024-04-24T23:33:48.724 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:33:48 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:33:48.442+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.80:6814 osd.3 since back 2024-04-24T23:33:22.042138+0000 front 2024-04-24T23:33:22.042191+0000 (oldest deadline 2024-04-24T23:33:45.542231+0000) 2024-04-24T23:33:48.946 INFO:journalctl@ceph.osd.7.smithi134.stdout:Apr 24 23:33:48 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T23:33:48.467+0000 7f676da99700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:05.625943+0000 front 2024-04-24T23:33:05.625903+0000 (oldest deadline 2024-04-24T23:33:29.126101+0000) 2024-04-24T23:33:48.947 INFO:journalctl@ceph.osd.7.smithi134.stdout:Apr 24 23:33:48 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T23:33:48.467+0000 7f676da99700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:14.927842+0000 front 2024-04-24T23:33:14.927944+0000 (oldest deadline 2024-04-24T23:33:37.827908+0000) 2024-04-24T23:33:48.947 INFO:journalctl@ceph.osd.7.smithi134.stdout:Apr 24 23:33:48 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T23:33:48.467+0000 7f676da99700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.80:6814 osd.3 since back 2024-04-24T23:33:22.929618+0000 front 2024-04-24T23:33:22.929735+0000 (oldest deadline 2024-04-24T23:33:45.229659+0000) 2024-04-24T23:33:49.006 INFO:journalctl@ceph.osd.8.smithi148.stdout:Apr 24 23:33:48 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-8[51367]: 2024-04-24T23:33:48.675+0000 7f119e765700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:05.559825+0000 front 2024-04-24T23:33:05.559812+0000 (oldest deadline 2024-04-24T23:33:29.560381+0000) 2024-04-24T23:33:49.006 INFO:journalctl@ceph.osd.8.smithi148.stdout:Apr 24 23:33:48 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-8[51367]: 2024-04-24T23:33:48.675+0000 7f119e765700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:13.061482+0000 front 2024-04-24T23:33:13.061509+0000 (oldest deadline 2024-04-24T23:33:36.561622+0000) 2024-04-24T23:33:49.006 INFO:journalctl@ceph.osd.8.smithi148.stdout:Apr 24 23:33:48 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-8[51367]: 2024-04-24T23:33:48.675+0000 7f119e765700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.80:6814 osd.3 since back 2024-04-24T23:33:22.863404+0000 front 2024-04-24T23:33:22.863339+0000 (oldest deadline 2024-04-24T23:33:44.563342+0000) 2024-04-24T23:33:49.037 INFO:journalctl@ceph.osd.6.smithi080.stdout:Apr 24 23:33:48 smithi080 podman[94794]: 2024-04-24 23:33:48.931915526 +0000 UTC m=+6.077573231 container cleanup c6137981ab8f7a27ca440f1c0ef3036146eae75ebb730bf2cc553bdadeb3c205 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-6, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux ) 2024-04-24T23:33:49.038 INFO:journalctl@ceph.osd.6.smithi080.stdout:Apr 24 23:33:48 smithi080 bash[94794]: ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-6 2024-04-24T23:33:49.446 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:33:49 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:33:49.154+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:56.929716+0000 front 2024-04-24T23:32:56.929774+0000 (oldest deadline 2024-04-24T23:33:20.429679+0000) 2024-04-24T23:33:49.447 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:33:49 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:33:49.154+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:04.530694+0000 front 2024-04-24T23:33:04.530743+0000 (oldest deadline 2024-04-24T23:33:29.230721+0000) 2024-04-24T23:33:49.447 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:33:49 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:33:49.154+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:15.632228+0000 front 2024-04-24T23:33:15.631978+0000 (oldest deadline 2024-04-24T23:33:37.331905+0000) 2024-04-24T23:33:49.447 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:33:49 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:33:49.154+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.80:6814 osd.3 since back 2024-04-24T23:33:22.632991+0000 front 2024-04-24T23:33:22.633039+0000 (oldest deadline 2024-04-24T23:33:44.932941+0000) 2024-04-24T23:33:49.506 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:33:49 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:33:49.109+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.848330+0000 front 2024-04-24T23:32:55.848387+0000 (oldest deadline 2024-04-24T23:33:20.548352+0000) 2024-04-24T23:33:49.506 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:33:49 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:33:49.109+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:02.150412+0000 front 2024-04-24T23:33:02.150294+0000 (oldest deadline 2024-04-24T23:33:28.050409+0000) 2024-04-24T23:33:49.506 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:33:49 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:33:49.110+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:14.952538+0000 front 2024-04-24T23:33:14.952635+0000 (oldest deadline 2024-04-24T23:33:40.852722+0000) 2024-04-24T23:33:49.538 INFO:journalctl@ceph.osd.6.smithi080.stdout:Apr 24 23:33:49 smithi080 podman[94809]: 2024-04-24 23:33:49.241896519 +0000 UTC m=+0.782553549 container remove c6137981ab8f7a27ca440f1c0ef3036146eae75ebb730bf2cc553bdadeb3c205 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-6, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-04-24T23:33:49.538 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:33:49 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:33:49.416+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.738540+0000 front 2024-04-24T23:32:55.738561+0000 (oldest deadline 2024-04-24T23:33:21.038786+0000) 2024-04-24T23:33:49.538 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:33:49 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:33:49.416+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:05.740014+0000 front 2024-04-24T23:33:05.739865+0000 (oldest deadline 2024-04-24T23:33:29.839897+0000) 2024-04-24T23:33:49.539 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:33:49 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:33:49.416+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:12.140601+0000 front 2024-04-24T23:33:12.140627+0000 (oldest deadline 2024-04-24T23:33:38.040825+0000) 2024-04-24T23:33:49.539 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:33:49 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:33:49.416+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.80:6814 osd.3 since back 2024-04-24T23:33:22.042138+0000 front 2024-04-24T23:33:22.042191+0000 (oldest deadline 2024-04-24T23:33:45.542231+0000) 2024-04-24T23:33:49.869 INFO:journalctl@ceph.osd.6.smithi080.stdout:Apr 24 23:33:49 smithi080 podman[94903]: 2024-04-24 23:33:49.510865832 +0000 UTC m=+0.017888494 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:33:49.870 INFO:journalctl@ceph.osd.6.smithi080.stdout:Apr 24 23:33:49 smithi080 podman[94903]: 2024-04-24 23:33:49.626042077 +0000 UTC m=+0.133064707 container create 69d1ec0a8e0ed46388b829ce99e45dc9f5fb574d2fcf292cfff60bf6fdbf9e17 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-6-deactivate, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9) 2024-04-24T23:33:49.870 INFO:journalctl@ceph.osd.6.smithi080.stdout:Apr 24 23:33:49 smithi080 podman[94903]: 2024-04-24 23:33:49.870338253 +0000 UTC m=+0.377360891 container init 69d1ec0a8e0ed46388b829ce99e45dc9f5fb574d2fcf292cfff60bf6fdbf9e17 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-6-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-04-24T23:33:49.946 INFO:journalctl@ceph.osd.7.smithi134.stdout:Apr 24 23:33:49 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T23:33:49.516+0000 7f676da99700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:05.625943+0000 front 2024-04-24T23:33:05.625903+0000 (oldest deadline 2024-04-24T23:33:29.126101+0000) 2024-04-24T23:33:49.947 INFO:journalctl@ceph.osd.7.smithi134.stdout:Apr 24 23:33:49 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T23:33:49.516+0000 7f676da99700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:14.927842+0000 front 2024-04-24T23:33:14.927944+0000 (oldest deadline 2024-04-24T23:33:37.827908+0000) 2024-04-24T23:33:49.947 INFO:journalctl@ceph.osd.7.smithi134.stdout:Apr 24 23:33:49 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T23:33:49.516+0000 7f676da99700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.80:6814 osd.3 since back 2024-04-24T23:33:22.929618+0000 front 2024-04-24T23:33:22.929735+0000 (oldest deadline 2024-04-24T23:33:45.229659+0000) 2024-04-24T23:33:50.005 INFO:journalctl@ceph.osd.8.smithi148.stdout:Apr 24 23:33:49 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-8[51367]: 2024-04-24T23:33:49.656+0000 7f119e765700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:05.559825+0000 front 2024-04-24T23:33:05.559812+0000 (oldest deadline 2024-04-24T23:33:29.560381+0000) 2024-04-24T23:33:50.006 INFO:journalctl@ceph.osd.8.smithi148.stdout:Apr 24 23:33:49 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-8[51367]: 2024-04-24T23:33:49.656+0000 7f119e765700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:13.061482+0000 front 2024-04-24T23:33:13.061509+0000 (oldest deadline 2024-04-24T23:33:36.561622+0000) 2024-04-24T23:33:50.006 INFO:journalctl@ceph.osd.8.smithi148.stdout:Apr 24 23:33:49 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-8[51367]: 2024-04-24T23:33:49.656+0000 7f119e765700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.80:6814 osd.3 since back 2024-04-24T23:33:22.863404+0000 front 2024-04-24T23:33:22.863339+0000 (oldest deadline 2024-04-24T23:33:44.563342+0000) 2024-04-24T23:33:50.157 INFO:journalctl@ceph.osd.6.smithi080.stdout:Apr 24 23:33:49 smithi080 podman[94903]: 2024-04-24 23:33:49.873358383 +0000 UTC m=+0.380381021 container start 69d1ec0a8e0ed46388b829ce99e45dc9f5fb574d2fcf292cfff60bf6fdbf9e17 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-6-deactivate, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True) 2024-04-24T23:33:50.158 INFO:journalctl@ceph.osd.6.smithi080.stdout:Apr 24 23:33:49 smithi080 podman[94903]: 2024-04-24 23:33:49.945339401 +0000 UTC m=+0.452362043 container attach 69d1ec0a8e0ed46388b829ce99e45dc9f5fb574d2fcf292cfff60bf6fdbf9e17 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-6-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-04-24T23:33:50.421 INFO:journalctl@ceph.osd.6.smithi080.stdout:Apr 24 23:33:50 smithi080 podman[94903]: 2024-04-24 23:33:50.158313293 +0000 UTC m=+0.665335939 container died 69d1ec0a8e0ed46388b829ce99e45dc9f5fb574d2fcf292cfff60bf6fdbf9e17 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-6-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T23:33:50.446 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:33:50 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:33:50.175+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:56.929716+0000 front 2024-04-24T23:32:56.929774+0000 (oldest deadline 2024-04-24T23:33:20.429679+0000) 2024-04-24T23:33:50.447 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:33:50 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:33:50.175+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:04.530694+0000 front 2024-04-24T23:33:04.530743+0000 (oldest deadline 2024-04-24T23:33:29.230721+0000) 2024-04-24T23:33:50.447 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:33:50 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:33:50.175+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:15.632228+0000 front 2024-04-24T23:33:15.631978+0000 (oldest deadline 2024-04-24T23:33:37.331905+0000) 2024-04-24T23:33:50.447 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:33:50 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:33:50.175+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.80:6814 osd.3 since back 2024-04-24T23:33:22.632991+0000 front 2024-04-24T23:33:22.633039+0000 (oldest deadline 2024-04-24T23:33:44.932941+0000) 2024-04-24T23:33:50.506 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:33:50 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:33:50.062+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.848330+0000 front 2024-04-24T23:32:55.848387+0000 (oldest deadline 2024-04-24T23:33:20.548352+0000) 2024-04-24T23:33:50.506 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:33:50 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:33:50.062+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:02.150412+0000 front 2024-04-24T23:33:02.150294+0000 (oldest deadline 2024-04-24T23:33:28.050409+0000) 2024-04-24T23:33:50.506 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:33:50 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:33:50.062+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:14.952538+0000 front 2024-04-24T23:33:14.952635+0000 (oldest deadline 2024-04-24T23:33:40.852722+0000) 2024-04-24T23:33:50.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:33:50 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:33:50.420+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.738540+0000 front 2024-04-24T23:32:55.738561+0000 (oldest deadline 2024-04-24T23:33:21.038786+0000) 2024-04-24T23:33:50.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:33:50 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:33:50.420+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:05.740014+0000 front 2024-04-24T23:33:05.739865+0000 (oldest deadline 2024-04-24T23:33:29.839897+0000) 2024-04-24T23:33:50.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:33:50 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:33:50.420+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:12.140601+0000 front 2024-04-24T23:33:12.140627+0000 (oldest deadline 2024-04-24T23:33:38.040825+0000) 2024-04-24T23:33:50.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:33:50 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:33:50.420+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.80:6814 osd.3 since back 2024-04-24T23:33:22.042138+0000 front 2024-04-24T23:33:22.042191+0000 (oldest deadline 2024-04-24T23:33:45.542231+0000) 2024-04-24T23:33:50.932 DEBUG:teuthology.orchestra.run.smithi080:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ec7885aa-0290-11ef-bc93-c7b262605968@osd.6.service' 2024-04-24T23:33:50.946 INFO:journalctl@ceph.osd.7.smithi134.stdout:Apr 24 23:33:50 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T23:33:50.479+0000 7f676da99700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:05.625943+0000 front 2024-04-24T23:33:05.625903+0000 (oldest deadline 2024-04-24T23:33:29.126101+0000) 2024-04-24T23:33:50.946 INFO:journalctl@ceph.osd.7.smithi134.stdout:Apr 24 23:33:50 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T23:33:50.479+0000 7f676da99700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:14.927842+0000 front 2024-04-24T23:33:14.927944+0000 (oldest deadline 2024-04-24T23:33:37.827908+0000) 2024-04-24T23:33:50.947 INFO:journalctl@ceph.osd.7.smithi134.stdout:Apr 24 23:33:50 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T23:33:50.479+0000 7f676da99700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.80:6814 osd.3 since back 2024-04-24T23:33:22.929618+0000 front 2024-04-24T23:33:22.929735+0000 (oldest deadline 2024-04-24T23:33:45.229659+0000) 2024-04-24T23:33:51.005 INFO:journalctl@ceph.osd.8.smithi148.stdout:Apr 24 23:33:50 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-8[51367]: 2024-04-24T23:33:50.656+0000 7f119e765700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:05.559825+0000 front 2024-04-24T23:33:05.559812+0000 (oldest deadline 2024-04-24T23:33:29.560381+0000) 2024-04-24T23:33:51.006 INFO:journalctl@ceph.osd.8.smithi148.stdout:Apr 24 23:33:50 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-8[51367]: 2024-04-24T23:33:50.656+0000 7f119e765700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:13.061482+0000 front 2024-04-24T23:33:13.061509+0000 (oldest deadline 2024-04-24T23:33:36.561622+0000) 2024-04-24T23:33:51.006 INFO:journalctl@ceph.osd.8.smithi148.stdout:Apr 24 23:33:50 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-8[51367]: 2024-04-24T23:33:50.656+0000 7f119e765700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.80:6814 osd.3 since back 2024-04-24T23:33:22.863404+0000 front 2024-04-24T23:33:22.863339+0000 (oldest deadline 2024-04-24T23:33:44.563342+0000) 2024-04-24T23:33:51.012 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:33:51.012 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-04-24T23:33:51.012 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-04-24T23:33:51.012 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-ec7885aa-0290-11ef-bc93-c7b262605968@osd.7 2024-04-24T23:33:51.327 INFO:journalctl@ceph.osd.7.smithi134.stdout:Apr 24 23:33:51 smithi134 systemd[1]: Stopping Ceph osd.7 for ec7885aa-0290-11ef-bc93-c7b262605968... 2024-04-24T23:33:51.328 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:33:51 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:33:51.126+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:56.929716+0000 front 2024-04-24T23:32:56.929774+0000 (oldest deadline 2024-04-24T23:33:20.429679+0000) 2024-04-24T23:33:51.328 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:33:51 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:33:51.126+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:04.530694+0000 front 2024-04-24T23:33:04.530743+0000 (oldest deadline 2024-04-24T23:33:29.230721+0000) 2024-04-24T23:33:51.328 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:33:51 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:33:51.126+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:15.632228+0000 front 2024-04-24T23:33:15.631978+0000 (oldest deadline 2024-04-24T23:33:37.331905+0000) 2024-04-24T23:33:51.328 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:33:51 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:33:51.126+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.80:6814 osd.3 since back 2024-04-24T23:33:22.632991+0000 front 2024-04-24T23:33:22.633039+0000 (oldest deadline 2024-04-24T23:33:44.932941+0000) 2024-04-24T23:33:51.506 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:33:51 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:33:51.095+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.848330+0000 front 2024-04-24T23:32:55.848387+0000 (oldest deadline 2024-04-24T23:33:20.548352+0000) 2024-04-24T23:33:51.506 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:33:51 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:33:51.095+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:02.150412+0000 front 2024-04-24T23:33:02.150294+0000 (oldest deadline 2024-04-24T23:33:28.050409+0000) 2024-04-24T23:33:51.507 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:33:51 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:33:51.095+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:14.952538+0000 front 2024-04-24T23:33:14.952635+0000 (oldest deadline 2024-04-24T23:33:40.852722+0000) 2024-04-24T23:33:51.696 INFO:journalctl@ceph.osd.7.smithi134.stdout:Apr 24 23:33:51 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T23:33:51.327+0000 7f67714a5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:33:51.697 INFO:journalctl@ceph.osd.7.smithi134.stdout:Apr 24 23:33:51 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T23:33:51.327+0000 7f67714a5700 -1 osd.7 98 *** Got signal Terminated *** 2024-04-24T23:33:51.697 INFO:journalctl@ceph.osd.7.smithi134.stdout:Apr 24 23:33:51 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T23:33:51.327+0000 7f67714a5700 -1 osd.7 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T23:33:51.697 INFO:journalctl@ceph.osd.7.smithi134.stdout:Apr 24 23:33:51 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T23:33:51.469+0000 7f676da99700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:05.625943+0000 front 2024-04-24T23:33:05.625903+0000 (oldest deadline 2024-04-24T23:33:29.126101+0000) 2024-04-24T23:33:51.697 INFO:journalctl@ceph.osd.7.smithi134.stdout:Apr 24 23:33:51 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T23:33:51.469+0000 7f676da99700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:14.927842+0000 front 2024-04-24T23:33:14.927944+0000 (oldest deadline 2024-04-24T23:33:37.827908+0000) 2024-04-24T23:33:51.697 INFO:journalctl@ceph.osd.7.smithi134.stdout:Apr 24 23:33:51 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T23:33:51.469+0000 7f676da99700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.80:6814 osd.3 since back 2024-04-24T23:33:22.929618+0000 front 2024-04-24T23:33:22.929735+0000 (oldest deadline 2024-04-24T23:33:45.229659+0000) 2024-04-24T23:33:51.787 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:33:51 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:33:51.432+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.738540+0000 front 2024-04-24T23:32:55.738561+0000 (oldest deadline 2024-04-24T23:33:21.038786+0000) 2024-04-24T23:33:51.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:33:51 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:33:51.432+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:05.740014+0000 front 2024-04-24T23:33:05.739865+0000 (oldest deadline 2024-04-24T23:33:29.839897+0000) 2024-04-24T23:33:51.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:33:51 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:33:51.432+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:12.140601+0000 front 2024-04-24T23:33:12.140627+0000 (oldest deadline 2024-04-24T23:33:38.040825+0000) 2024-04-24T23:33:51.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:33:51 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:33:51.432+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.80:6814 osd.3 since back 2024-04-24T23:33:22.042138+0000 front 2024-04-24T23:33:22.042191+0000 (oldest deadline 2024-04-24T23:33:45.542231+0000) 2024-04-24T23:33:52.006 INFO:journalctl@ceph.osd.8.smithi148.stdout:Apr 24 23:33:51 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-8[51367]: 2024-04-24T23:33:51.657+0000 7f119e765700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:05.559825+0000 front 2024-04-24T23:33:05.559812+0000 (oldest deadline 2024-04-24T23:33:29.560381+0000) 2024-04-24T23:33:52.006 INFO:journalctl@ceph.osd.8.smithi148.stdout:Apr 24 23:33:51 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-8[51367]: 2024-04-24T23:33:51.657+0000 7f119e765700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:13.061482+0000 front 2024-04-24T23:33:13.061509+0000 (oldest deadline 2024-04-24T23:33:36.561622+0000) 2024-04-24T23:33:52.006 INFO:journalctl@ceph.osd.8.smithi148.stdout:Apr 24 23:33:51 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-8[51367]: 2024-04-24T23:33:51.657+0000 7f119e765700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.80:6814 osd.3 since back 2024-04-24T23:33:22.863404+0000 front 2024-04-24T23:33:22.863339+0000 (oldest deadline 2024-04-24T23:33:44.563342+0000) 2024-04-24T23:33:52.446 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:33:52 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:33:52.124+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:56.929716+0000 front 2024-04-24T23:32:56.929774+0000 (oldest deadline 2024-04-24T23:33:20.429679+0000) 2024-04-24T23:33:52.447 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:33:52 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:33:52.124+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:04.530694+0000 front 2024-04-24T23:33:04.530743+0000 (oldest deadline 2024-04-24T23:33:29.230721+0000) 2024-04-24T23:33:52.447 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:33:52 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:33:52.124+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:15.632228+0000 front 2024-04-24T23:33:15.631978+0000 (oldest deadline 2024-04-24T23:33:37.331905+0000) 2024-04-24T23:33:52.447 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:33:52 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:33:52.124+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.80:6814 osd.3 since back 2024-04-24T23:33:22.632991+0000 front 2024-04-24T23:33:22.633039+0000 (oldest deadline 2024-04-24T23:33:44.932941+0000) 2024-04-24T23:33:52.506 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:33:52 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:33:52.081+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.848330+0000 front 2024-04-24T23:32:55.848387+0000 (oldest deadline 2024-04-24T23:33:20.548352+0000) 2024-04-24T23:33:52.506 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:33:52 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:33:52.081+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:02.150412+0000 front 2024-04-24T23:33:02.150294+0000 (oldest deadline 2024-04-24T23:33:28.050409+0000) 2024-04-24T23:33:52.506 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:33:52 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:33:52.081+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:14.952538+0000 front 2024-04-24T23:33:14.952635+0000 (oldest deadline 2024-04-24T23:33:40.852722+0000) 2024-04-24T23:33:52.787 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:33:52 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:33:52.454+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.738540+0000 front 2024-04-24T23:32:55.738561+0000 (oldest deadline 2024-04-24T23:33:21.038786+0000) 2024-04-24T23:33:52.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:33:52 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:33:52.454+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:05.740014+0000 front 2024-04-24T23:33:05.739865+0000 (oldest deadline 2024-04-24T23:33:29.839897+0000) 2024-04-24T23:33:52.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:33:52 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:33:52.454+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:12.140601+0000 front 2024-04-24T23:33:12.140627+0000 (oldest deadline 2024-04-24T23:33:38.040825+0000) 2024-04-24T23:33:52.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:33:52 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:33:52.454+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.80:6814 osd.3 since back 2024-04-24T23:33:22.042138+0000 front 2024-04-24T23:33:22.042191+0000 (oldest deadline 2024-04-24T23:33:45.542231+0000) 2024-04-24T23:33:52.946 INFO:journalctl@ceph.osd.7.smithi134.stdout:Apr 24 23:33:52 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T23:33:52.454+0000 7f676da99700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:05.625943+0000 front 2024-04-24T23:33:05.625903+0000 (oldest deadline 2024-04-24T23:33:29.126101+0000) 2024-04-24T23:33:52.947 INFO:journalctl@ceph.osd.7.smithi134.stdout:Apr 24 23:33:52 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T23:33:52.454+0000 7f676da99700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:14.927842+0000 front 2024-04-24T23:33:14.927944+0000 (oldest deadline 2024-04-24T23:33:37.827908+0000) 2024-04-24T23:33:52.947 INFO:journalctl@ceph.osd.7.smithi134.stdout:Apr 24 23:33:52 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T23:33:52.454+0000 7f676da99700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.80:6814 osd.3 since back 2024-04-24T23:33:22.929618+0000 front 2024-04-24T23:33:22.929735+0000 (oldest deadline 2024-04-24T23:33:45.229659+0000) 2024-04-24T23:33:53.005 INFO:journalctl@ceph.osd.8.smithi148.stdout:Apr 24 23:33:52 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-8[51367]: 2024-04-24T23:33:52.623+0000 7f119e765700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:05.559825+0000 front 2024-04-24T23:33:05.559812+0000 (oldest deadline 2024-04-24T23:33:29.560381+0000) 2024-04-24T23:33:53.006 INFO:journalctl@ceph.osd.8.smithi148.stdout:Apr 24 23:33:52 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-8[51367]: 2024-04-24T23:33:52.623+0000 7f119e765700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:13.061482+0000 front 2024-04-24T23:33:13.061509+0000 (oldest deadline 2024-04-24T23:33:36.561622+0000) 2024-04-24T23:33:53.006 INFO:journalctl@ceph.osd.8.smithi148.stdout:Apr 24 23:33:52 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-8[51367]: 2024-04-24T23:33:52.623+0000 7f119e765700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.80:6814 osd.3 since back 2024-04-24T23:33:22.863404+0000 front 2024-04-24T23:33:22.863339+0000 (oldest deadline 2024-04-24T23:33:44.563342+0000) 2024-04-24T23:33:53.447 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:33:53 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:33:53.154+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:56.929716+0000 front 2024-04-24T23:32:56.929774+0000 (oldest deadline 2024-04-24T23:33:20.429679+0000) 2024-04-24T23:33:53.447 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:33:53 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:33:53.154+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:04.530694+0000 front 2024-04-24T23:33:04.530743+0000 (oldest deadline 2024-04-24T23:33:29.230721+0000) 2024-04-24T23:33:53.447 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:33:53 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:33:53.154+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:15.632228+0000 front 2024-04-24T23:33:15.631978+0000 (oldest deadline 2024-04-24T23:33:37.331905+0000) 2024-04-24T23:33:53.447 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:33:53 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:33:53.154+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.80:6814 osd.3 since back 2024-04-24T23:33:22.632991+0000 front 2024-04-24T23:33:22.633039+0000 (oldest deadline 2024-04-24T23:33:44.932941+0000) 2024-04-24T23:33:53.447 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:33:53 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:33:53.154+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-04-24T23:33:30.134179+0000 front 2024-04-24T23:33:30.134163+0000 (oldest deadline 2024-04-24T23:33:52.434250+0000) 2024-04-24T23:33:53.506 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:33:53 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:33:53.107+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.848330+0000 front 2024-04-24T23:32:55.848387+0000 (oldest deadline 2024-04-24T23:33:20.548352+0000) 2024-04-24T23:33:53.506 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:33:53 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:33:53.107+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:02.150412+0000 front 2024-04-24T23:33:02.150294+0000 (oldest deadline 2024-04-24T23:33:28.050409+0000) 2024-04-24T23:33:53.506 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:33:53 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:33:53.107+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:14.952538+0000 front 2024-04-24T23:33:14.952635+0000 (oldest deadline 2024-04-24T23:33:40.852722+0000) 2024-04-24T23:33:53.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:33:53 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:33:53.455+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.738540+0000 front 2024-04-24T23:32:55.738561+0000 (oldest deadline 2024-04-24T23:33:21.038786+0000) 2024-04-24T23:33:53.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:33:53 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:33:53.455+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:05.740014+0000 front 2024-04-24T23:33:05.739865+0000 (oldest deadline 2024-04-24T23:33:29.839897+0000) 2024-04-24T23:33:53.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:33:53 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:33:53.455+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:12.140601+0000 front 2024-04-24T23:33:12.140627+0000 (oldest deadline 2024-04-24T23:33:38.040825+0000) 2024-04-24T23:33:53.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:33:53 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:33:53.455+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.80:6814 osd.3 since back 2024-04-24T23:33:22.042138+0000 front 2024-04-24T23:33:22.042191+0000 (oldest deadline 2024-04-24T23:33:45.542231+0000) 2024-04-24T23:33:53.946 INFO:journalctl@ceph.osd.7.smithi134.stdout:Apr 24 23:33:53 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T23:33:53.460+0000 7f676da99700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:05.625943+0000 front 2024-04-24T23:33:05.625903+0000 (oldest deadline 2024-04-24T23:33:29.126101+0000) 2024-04-24T23:33:53.947 INFO:journalctl@ceph.osd.7.smithi134.stdout:Apr 24 23:33:53 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T23:33:53.460+0000 7f676da99700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:14.927842+0000 front 2024-04-24T23:33:14.927944+0000 (oldest deadline 2024-04-24T23:33:37.827908+0000) 2024-04-24T23:33:53.947 INFO:journalctl@ceph.osd.7.smithi134.stdout:Apr 24 23:33:53 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T23:33:53.460+0000 7f676da99700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.80:6814 osd.3 since back 2024-04-24T23:33:22.929618+0000 front 2024-04-24T23:33:22.929735+0000 (oldest deadline 2024-04-24T23:33:45.229659+0000) 2024-04-24T23:33:54.005 INFO:journalctl@ceph.osd.8.smithi148.stdout:Apr 24 23:33:53 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-8[51367]: 2024-04-24T23:33:53.623+0000 7f119e765700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:05.559825+0000 front 2024-04-24T23:33:05.559812+0000 (oldest deadline 2024-04-24T23:33:29.560381+0000) 2024-04-24T23:33:54.006 INFO:journalctl@ceph.osd.8.smithi148.stdout:Apr 24 23:33:53 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-8[51367]: 2024-04-24T23:33:53.623+0000 7f119e765700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:13.061482+0000 front 2024-04-24T23:33:13.061509+0000 (oldest deadline 2024-04-24T23:33:36.561622+0000) 2024-04-24T23:33:54.006 INFO:journalctl@ceph.osd.8.smithi148.stdout:Apr 24 23:33:53 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-8[51367]: 2024-04-24T23:33:53.623+0000 7f119e765700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.80:6814 osd.3 since back 2024-04-24T23:33:22.863404+0000 front 2024-04-24T23:33:22.863339+0000 (oldest deadline 2024-04-24T23:33:44.563342+0000) 2024-04-24T23:33:54.006 INFO:journalctl@ceph.osd.8.smithi148.stdout:Apr 24 23:33:53 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-8[51367]: 2024-04-24T23:33:53.623+0000 7f119e765700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-04-24T23:33:29.864584+0000 front 2024-04-24T23:33:29.864717+0000 (oldest deadline 2024-04-24T23:33:53.364437+0000) 2024-04-24T23:33:54.442 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:33:54 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:33:54.137+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:56.929716+0000 front 2024-04-24T23:32:56.929774+0000 (oldest deadline 2024-04-24T23:33:20.429679+0000) 2024-04-24T23:33:54.443 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:33:54 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:33:54.137+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:04.530694+0000 front 2024-04-24T23:33:04.530743+0000 (oldest deadline 2024-04-24T23:33:29.230721+0000) 2024-04-24T23:33:54.443 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:33:54 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:33:54.137+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:15.632228+0000 front 2024-04-24T23:33:15.631978+0000 (oldest deadline 2024-04-24T23:33:37.331905+0000) 2024-04-24T23:33:54.443 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:33:54 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:33:54.137+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.80:6814 osd.3 since back 2024-04-24T23:33:22.632991+0000 front 2024-04-24T23:33:22.633039+0000 (oldest deadline 2024-04-24T23:33:44.932941+0000) 2024-04-24T23:33:54.443 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:33:54 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:33:54.137+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-04-24T23:33:30.134179+0000 front 2024-04-24T23:33:30.134163+0000 (oldest deadline 2024-04-24T23:33:52.434250+0000) 2024-04-24T23:33:54.505 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:33:54 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:33:54.107+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.848330+0000 front 2024-04-24T23:32:55.848387+0000 (oldest deadline 2024-04-24T23:33:20.548352+0000) 2024-04-24T23:33:54.506 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:33:54 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:33:54.107+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:02.150412+0000 front 2024-04-24T23:33:02.150294+0000 (oldest deadline 2024-04-24T23:33:28.050409+0000) 2024-04-24T23:33:54.506 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:33:54 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:33:54.107+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:14.952538+0000 front 2024-04-24T23:33:14.952635+0000 (oldest deadline 2024-04-24T23:33:40.852722+0000) 2024-04-24T23:33:54.696 INFO:journalctl@ceph.osd.7.smithi134.stdout:Apr 24 23:33:54 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T23:33:54.442+0000 7f676da99700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:05.625943+0000 front 2024-04-24T23:33:05.625903+0000 (oldest deadline 2024-04-24T23:33:29.126101+0000) 2024-04-24T23:33:54.697 INFO:journalctl@ceph.osd.7.smithi134.stdout:Apr 24 23:33:54 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T23:33:54.442+0000 7f676da99700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:14.927842+0000 front 2024-04-24T23:33:14.927944+0000 (oldest deadline 2024-04-24T23:33:37.827908+0000) 2024-04-24T23:33:54.697 INFO:journalctl@ceph.osd.7.smithi134.stdout:Apr 24 23:33:54 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T23:33:54.442+0000 7f676da99700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.80:6814 osd.3 since back 2024-04-24T23:33:22.929618+0000 front 2024-04-24T23:33:22.929735+0000 (oldest deadline 2024-04-24T23:33:45.229659+0000) 2024-04-24T23:33:54.697 INFO:journalctl@ceph.osd.7.smithi134.stdout:Apr 24 23:33:54 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T23:33:54.442+0000 7f676da99700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-04-24T23:33:26.830942+0000 front 2024-04-24T23:33:26.830916+0000 (oldest deadline 2024-04-24T23:33:53.831595+0000) 2024-04-24T23:33:54.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:33:54 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:33:54.482+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.738540+0000 front 2024-04-24T23:32:55.738561+0000 (oldest deadline 2024-04-24T23:33:21.038786+0000) 2024-04-24T23:33:54.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:33:54 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:33:54.482+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:05.740014+0000 front 2024-04-24T23:33:05.739865+0000 (oldest deadline 2024-04-24T23:33:29.839897+0000) 2024-04-24T23:33:54.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:33:54 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:33:54.482+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:12.140601+0000 front 2024-04-24T23:33:12.140627+0000 (oldest deadline 2024-04-24T23:33:38.040825+0000) 2024-04-24T23:33:54.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:33:54 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:33:54.482+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.80:6814 osd.3 since back 2024-04-24T23:33:22.042138+0000 front 2024-04-24T23:33:22.042191+0000 (oldest deadline 2024-04-24T23:33:45.542231+0000) 2024-04-24T23:33:55.006 INFO:journalctl@ceph.osd.8.smithi148.stdout:Apr 24 23:33:54 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-8[51367]: 2024-04-24T23:33:54.574+0000 7f119e765700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:05.559825+0000 front 2024-04-24T23:33:05.559812+0000 (oldest deadline 2024-04-24T23:33:29.560381+0000) 2024-04-24T23:33:55.006 INFO:journalctl@ceph.osd.8.smithi148.stdout:Apr 24 23:33:54 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-8[51367]: 2024-04-24T23:33:54.574+0000 7f119e765700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:13.061482+0000 front 2024-04-24T23:33:13.061509+0000 (oldest deadline 2024-04-24T23:33:36.561622+0000) 2024-04-24T23:33:55.006 INFO:journalctl@ceph.osd.8.smithi148.stdout:Apr 24 23:33:54 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-8[51367]: 2024-04-24T23:33:54.574+0000 7f119e765700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.80:6814 osd.3 since back 2024-04-24T23:33:22.863404+0000 front 2024-04-24T23:33:22.863339+0000 (oldest deadline 2024-04-24T23:33:44.563342+0000) 2024-04-24T23:33:55.006 INFO:journalctl@ceph.osd.8.smithi148.stdout:Apr 24 23:33:54 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-8[51367]: 2024-04-24T23:33:54.574+0000 7f119e765700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-04-24T23:33:29.864584+0000 front 2024-04-24T23:33:29.864717+0000 (oldest deadline 2024-04-24T23:33:53.364437+0000) 2024-04-24T23:33:55.447 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:33:55 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:33:55.126+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:56.929716+0000 front 2024-04-24T23:32:56.929774+0000 (oldest deadline 2024-04-24T23:33:20.429679+0000) 2024-04-24T23:33:55.447 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:33:55 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:33:55.126+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:04.530694+0000 front 2024-04-24T23:33:04.530743+0000 (oldest deadline 2024-04-24T23:33:29.230721+0000) 2024-04-24T23:33:55.447 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:33:55 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:33:55.126+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:15.632228+0000 front 2024-04-24T23:33:15.631978+0000 (oldest deadline 2024-04-24T23:33:37.331905+0000) 2024-04-24T23:33:55.448 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:33:55 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:33:55.126+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.80:6814 osd.3 since back 2024-04-24T23:33:22.632991+0000 front 2024-04-24T23:33:22.633039+0000 (oldest deadline 2024-04-24T23:33:44.932941+0000) 2024-04-24T23:33:55.448 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:33:55 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:33:55.126+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-04-24T23:33:30.134179+0000 front 2024-04-24T23:33:30.134163+0000 (oldest deadline 2024-04-24T23:33:52.434250+0000) 2024-04-24T23:33:55.506 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:33:55 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:33:55.148+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.848330+0000 front 2024-04-24T23:32:55.848387+0000 (oldest deadline 2024-04-24T23:33:20.548352+0000) 2024-04-24T23:33:55.506 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:33:55 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:33:55.148+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:02.150412+0000 front 2024-04-24T23:33:02.150294+0000 (oldest deadline 2024-04-24T23:33:28.050409+0000) 2024-04-24T23:33:55.506 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:33:55 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:33:55.148+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:14.952538+0000 front 2024-04-24T23:33:14.952635+0000 (oldest deadline 2024-04-24T23:33:40.852722+0000) 2024-04-24T23:33:55.506 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:33:55 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:33:55.148+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-04-24T23:33:26.054443+0000 front 2024-04-24T23:33:26.054377+0000 (oldest deadline 2024-04-24T23:33:54.255147+0000) 2024-04-24T23:33:55.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:33:55 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:33:55.473+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.738540+0000 front 2024-04-24T23:32:55.738561+0000 (oldest deadline 2024-04-24T23:33:21.038786+0000) 2024-04-24T23:33:55.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:33:55 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:33:55.473+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:05.740014+0000 front 2024-04-24T23:33:05.739865+0000 (oldest deadline 2024-04-24T23:33:29.839897+0000) 2024-04-24T23:33:55.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:33:55 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:33:55.473+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:12.140601+0000 front 2024-04-24T23:33:12.140627+0000 (oldest deadline 2024-04-24T23:33:38.040825+0000) 2024-04-24T23:33:55.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:33:55 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:33:55.473+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.80:6814 osd.3 since back 2024-04-24T23:33:22.042138+0000 front 2024-04-24T23:33:22.042191+0000 (oldest deadline 2024-04-24T23:33:45.542231+0000) 2024-04-24T23:33:55.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:33:55 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:33:55.473+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-04-24T23:33:31.443174+0000 front 2024-04-24T23:33:31.443143+0000 (oldest deadline 2024-04-24T23:33:54.943017+0000) 2024-04-24T23:33:55.947 INFO:journalctl@ceph.osd.7.smithi134.stdout:Apr 24 23:33:55 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T23:33:55.480+0000 7f676da99700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:05.625943+0000 front 2024-04-24T23:33:05.625903+0000 (oldest deadline 2024-04-24T23:33:29.126101+0000) 2024-04-24T23:33:55.947 INFO:journalctl@ceph.osd.7.smithi134.stdout:Apr 24 23:33:55 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T23:33:55.480+0000 7f676da99700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:14.927842+0000 front 2024-04-24T23:33:14.927944+0000 (oldest deadline 2024-04-24T23:33:37.827908+0000) 2024-04-24T23:33:55.947 INFO:journalctl@ceph.osd.7.smithi134.stdout:Apr 24 23:33:55 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T23:33:55.480+0000 7f676da99700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.80:6814 osd.3 since back 2024-04-24T23:33:22.929618+0000 front 2024-04-24T23:33:22.929735+0000 (oldest deadline 2024-04-24T23:33:45.229659+0000) 2024-04-24T23:33:55.947 INFO:journalctl@ceph.osd.7.smithi134.stdout:Apr 24 23:33:55 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T23:33:55.480+0000 7f676da99700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-04-24T23:33:26.830942+0000 front 2024-04-24T23:33:26.830916+0000 (oldest deadline 2024-04-24T23:33:53.831595+0000) 2024-04-24T23:33:56.006 INFO:journalctl@ceph.osd.8.smithi148.stdout:Apr 24 23:33:55 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-8[51367]: 2024-04-24T23:33:55.609+0000 7f119e765700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:05.559825+0000 front 2024-04-24T23:33:05.559812+0000 (oldest deadline 2024-04-24T23:33:29.560381+0000) 2024-04-24T23:33:56.006 INFO:journalctl@ceph.osd.8.smithi148.stdout:Apr 24 23:33:55 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-8[51367]: 2024-04-24T23:33:55.609+0000 7f119e765700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:13.061482+0000 front 2024-04-24T23:33:13.061509+0000 (oldest deadline 2024-04-24T23:33:36.561622+0000) 2024-04-24T23:33:56.006 INFO:journalctl@ceph.osd.8.smithi148.stdout:Apr 24 23:33:55 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-8[51367]: 2024-04-24T23:33:55.609+0000 7f119e765700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.80:6814 osd.3 since back 2024-04-24T23:33:22.863404+0000 front 2024-04-24T23:33:22.863339+0000 (oldest deadline 2024-04-24T23:33:44.563342+0000) 2024-04-24T23:33:56.006 INFO:journalctl@ceph.osd.8.smithi148.stdout:Apr 24 23:33:55 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-8[51367]: 2024-04-24T23:33:55.609+0000 7f119e765700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-04-24T23:33:29.864584+0000 front 2024-04-24T23:33:29.864717+0000 (oldest deadline 2024-04-24T23:33:53.364437+0000) 2024-04-24T23:33:56.446 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:33:56 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:33:56.165+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:56.929716+0000 front 2024-04-24T23:32:56.929774+0000 (oldest deadline 2024-04-24T23:33:20.429679+0000) 2024-04-24T23:33:56.447 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:33:56 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:33:56.165+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:04.530694+0000 front 2024-04-24T23:33:04.530743+0000 (oldest deadline 2024-04-24T23:33:29.230721+0000) 2024-04-24T23:33:56.447 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:33:56 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:33:56.165+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:15.632228+0000 front 2024-04-24T23:33:15.631978+0000 (oldest deadline 2024-04-24T23:33:37.331905+0000) 2024-04-24T23:33:56.447 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:33:56 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:33:56.165+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.80:6814 osd.3 since back 2024-04-24T23:33:22.632991+0000 front 2024-04-24T23:33:22.633039+0000 (oldest deadline 2024-04-24T23:33:44.932941+0000) 2024-04-24T23:33:56.447 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:33:56 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:33:56.165+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-04-24T23:33:30.134179+0000 front 2024-04-24T23:33:30.134163+0000 (oldest deadline 2024-04-24T23:33:52.434250+0000) 2024-04-24T23:33:56.506 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:33:56 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:33:56.132+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.848330+0000 front 2024-04-24T23:32:55.848387+0000 (oldest deadline 2024-04-24T23:33:20.548352+0000) 2024-04-24T23:33:56.506 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:33:56 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:33:56.132+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:02.150412+0000 front 2024-04-24T23:33:02.150294+0000 (oldest deadline 2024-04-24T23:33:28.050409+0000) 2024-04-24T23:33:56.506 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:33:56 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:33:56.132+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:14.952538+0000 front 2024-04-24T23:33:14.952635+0000 (oldest deadline 2024-04-24T23:33:40.852722+0000) 2024-04-24T23:33:56.507 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:33:56 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:33:56.132+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-04-24T23:33:26.054443+0000 front 2024-04-24T23:33:26.054377+0000 (oldest deadline 2024-04-24T23:33:54.255147+0000) 2024-04-24T23:33:56.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:33:56 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:33:56.454+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.738540+0000 front 2024-04-24T23:32:55.738561+0000 (oldest deadline 2024-04-24T23:33:21.038786+0000) 2024-04-24T23:33:56.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:33:56 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:33:56.454+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:05.740014+0000 front 2024-04-24T23:33:05.739865+0000 (oldest deadline 2024-04-24T23:33:29.839897+0000) 2024-04-24T23:33:56.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:33:56 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:33:56.454+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:12.140601+0000 front 2024-04-24T23:33:12.140627+0000 (oldest deadline 2024-04-24T23:33:38.040825+0000) 2024-04-24T23:33:56.789 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:33:56 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:33:56.454+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.80:6814 osd.3 since back 2024-04-24T23:33:22.042138+0000 front 2024-04-24T23:33:22.042191+0000 (oldest deadline 2024-04-24T23:33:45.542231+0000) 2024-04-24T23:33:56.789 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:33:56 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:33:56.454+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-04-24T23:33:31.443174+0000 front 2024-04-24T23:33:31.443143+0000 (oldest deadline 2024-04-24T23:33:54.943017+0000) 2024-04-24T23:33:57.006 INFO:journalctl@ceph.osd.8.smithi148.stdout:Apr 24 23:33:56 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-8[51367]: 2024-04-24T23:33:56.643+0000 7f119e765700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:05.559825+0000 front 2024-04-24T23:33:05.559812+0000 (oldest deadline 2024-04-24T23:33:29.560381+0000) 2024-04-24T23:33:57.006 INFO:journalctl@ceph.osd.8.smithi148.stdout:Apr 24 23:33:56 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-8[51367]: 2024-04-24T23:33:56.643+0000 7f119e765700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:13.061482+0000 front 2024-04-24T23:33:13.061509+0000 (oldest deadline 2024-04-24T23:33:36.561622+0000) 2024-04-24T23:33:57.006 INFO:journalctl@ceph.osd.8.smithi148.stdout:Apr 24 23:33:56 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-8[51367]: 2024-04-24T23:33:56.643+0000 7f119e765700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.80:6814 osd.3 since back 2024-04-24T23:33:22.863404+0000 front 2024-04-24T23:33:22.863339+0000 (oldest deadline 2024-04-24T23:33:44.563342+0000) 2024-04-24T23:33:57.006 INFO:journalctl@ceph.osd.8.smithi148.stdout:Apr 24 23:33:56 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-8[51367]: 2024-04-24T23:33:56.643+0000 7f119e765700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-04-24T23:33:29.864584+0000 front 2024-04-24T23:33:29.864717+0000 (oldest deadline 2024-04-24T23:33:53.364437+0000) 2024-04-24T23:33:57.044 INFO:journalctl@ceph.osd.7.smithi134.stdout:Apr 24 23:33:56 smithi134 podman[67056]: 2024-04-24 23:33:56.741678563 +0000 UTC m=+5.535355363 container died 569b518b7d185500d801ff01775e0f1eb3937653adfc506529bcb0249997a52b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-7, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-04-24T23:33:57.395 INFO:journalctl@ceph.osd.7.smithi134.stdout:Apr 24 23:33:57 smithi134 podman[67056]: 2024-04-24 23:33:57.153045082 +0000 UTC m=+5.946721930 container cleanup 569b518b7d185500d801ff01775e0f1eb3937653adfc506529bcb0249997a52b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-7, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-04-24T23:33:57.395 INFO:journalctl@ceph.osd.7.smithi134.stdout:Apr 24 23:33:57 smithi134 bash[67056]: ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-7 2024-04-24T23:33:57.396 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:33:57 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:33:57.124+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:56.929716+0000 front 2024-04-24T23:32:56.929774+0000 (oldest deadline 2024-04-24T23:33:20.429679+0000) 2024-04-24T23:33:57.396 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:33:57 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:33:57.124+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:04.530694+0000 front 2024-04-24T23:33:04.530743+0000 (oldest deadline 2024-04-24T23:33:29.230721+0000) 2024-04-24T23:33:57.396 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:33:57 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:33:57.124+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:15.632228+0000 front 2024-04-24T23:33:15.631978+0000 (oldest deadline 2024-04-24T23:33:37.331905+0000) 2024-04-24T23:33:57.396 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:33:57 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:33:57.124+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.80:6814 osd.3 since back 2024-04-24T23:33:22.632991+0000 front 2024-04-24T23:33:22.633039+0000 (oldest deadline 2024-04-24T23:33:44.932941+0000) 2024-04-24T23:33:57.396 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:33:57 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:33:57.124+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-04-24T23:33:30.134179+0000 front 2024-04-24T23:33:30.134163+0000 (oldest deadline 2024-04-24T23:33:52.434250+0000) 2024-04-24T23:33:57.505 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:33:57 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:33:57.118+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.848330+0000 front 2024-04-24T23:32:55.848387+0000 (oldest deadline 2024-04-24T23:33:20.548352+0000) 2024-04-24T23:33:57.506 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:33:57 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:33:57.118+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:02.150412+0000 front 2024-04-24T23:33:02.150294+0000 (oldest deadline 2024-04-24T23:33:28.050409+0000) 2024-04-24T23:33:57.506 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:33:57 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:33:57.118+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:14.952538+0000 front 2024-04-24T23:33:14.952635+0000 (oldest deadline 2024-04-24T23:33:40.852722+0000) 2024-04-24T23:33:57.506 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:33:57 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:33:57.118+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-04-24T23:33:26.054443+0000 front 2024-04-24T23:33:26.054377+0000 (oldest deadline 2024-04-24T23:33:54.255147+0000) 2024-04-24T23:33:57.696 INFO:journalctl@ceph.osd.7.smithi134.stdout:Apr 24 23:33:57 smithi134 podman[67073]: 2024-04-24 23:33:57.395649707 +0000 UTC m=+0.649064962 container remove 569b518b7d185500d801ff01775e0f1eb3937653adfc506529bcb0249997a52b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-7, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=) 2024-04-24T23:33:57.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:33:57 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:33:57.428+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.738540+0000 front 2024-04-24T23:32:55.738561+0000 (oldest deadline 2024-04-24T23:33:21.038786+0000) 2024-04-24T23:33:57.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:33:57 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:33:57.428+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:05.740014+0000 front 2024-04-24T23:33:05.739865+0000 (oldest deadline 2024-04-24T23:33:29.839897+0000) 2024-04-24T23:33:57.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:33:57 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:33:57.428+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:12.140601+0000 front 2024-04-24T23:33:12.140627+0000 (oldest deadline 2024-04-24T23:33:38.040825+0000) 2024-04-24T23:33:57.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:33:57 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:33:57.428+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.80:6814 osd.3 since back 2024-04-24T23:33:22.042138+0000 front 2024-04-24T23:33:22.042191+0000 (oldest deadline 2024-04-24T23:33:45.542231+0000) 2024-04-24T23:33:57.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:33:57 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:33:57.428+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-04-24T23:33:31.443174+0000 front 2024-04-24T23:33:31.443143+0000 (oldest deadline 2024-04-24T23:33:54.943017+0000) 2024-04-24T23:33:58.006 INFO:journalctl@ceph.osd.8.smithi148.stdout:Apr 24 23:33:57 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-8[51367]: 2024-04-24T23:33:57.669+0000 7f119e765700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:05.559825+0000 front 2024-04-24T23:33:05.559812+0000 (oldest deadline 2024-04-24T23:33:29.560381+0000) 2024-04-24T23:33:58.006 INFO:journalctl@ceph.osd.8.smithi148.stdout:Apr 24 23:33:57 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-8[51367]: 2024-04-24T23:33:57.669+0000 7f119e765700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:13.061482+0000 front 2024-04-24T23:33:13.061509+0000 (oldest deadline 2024-04-24T23:33:36.561622+0000) 2024-04-24T23:33:58.006 INFO:journalctl@ceph.osd.8.smithi148.stdout:Apr 24 23:33:57 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-8[51367]: 2024-04-24T23:33:57.669+0000 7f119e765700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.80:6814 osd.3 since back 2024-04-24T23:33:22.863404+0000 front 2024-04-24T23:33:22.863339+0000 (oldest deadline 2024-04-24T23:33:44.563342+0000) 2024-04-24T23:33:58.006 INFO:journalctl@ceph.osd.8.smithi148.stdout:Apr 24 23:33:57 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-8[51367]: 2024-04-24T23:33:57.669+0000 7f119e765700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-04-24T23:33:29.864584+0000 front 2024-04-24T23:33:29.864717+0000 (oldest deadline 2024-04-24T23:33:53.364437+0000) 2024-04-24T23:33:58.007 INFO:journalctl@ceph.osd.7.smithi134.stdout:Apr 24 23:33:57 smithi134 podman[67163]: 2024-04-24 23:33:57.624040153 +0000 UTC m=+0.017955041 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:33:58.007 INFO:journalctl@ceph.osd.7.smithi134.stdout:Apr 24 23:33:57 smithi134 podman[67163]: 2024-04-24 23:33:57.738330245 +0000 UTC m=+0.132245135 container create 11aa2598f4f578e1744b03aeb1603b0afc76f469884e9c67b8e389add05cf89c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9) 2024-04-24T23:33:58.297 INFO:journalctl@ceph.osd.7.smithi134.stdout:Apr 24 23:33:58 smithi134 podman[67163]: 2024-04-24 23:33:58.007969881 +0000 UTC m=+0.401884774 container init 11aa2598f4f578e1744b03aeb1603b0afc76f469884e9c67b8e389add05cf89c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-7-deactivate, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD) 2024-04-24T23:33:58.297 INFO:journalctl@ceph.osd.7.smithi134.stdout:Apr 24 23:33:58 smithi134 podman[67163]: 2024-04-24 23:33:58.011453534 +0000 UTC m=+0.405368422 container start 11aa2598f4f578e1744b03aeb1603b0afc76f469884e9c67b8e389add05cf89c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-04-24T23:33:58.297 INFO:journalctl@ceph.osd.7.smithi134.stdout:Apr 24 23:33:58 smithi134 podman[67163]: 2024-04-24 23:33:58.107457501 +0000 UTC m=+0.501372389 container attach 11aa2598f4f578e1744b03aeb1603b0afc76f469884e9c67b8e389add05cf89c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-7-deactivate, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-24T23:33:58.297 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:33:58 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:33:58.121+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:56.929716+0000 front 2024-04-24T23:32:56.929774+0000 (oldest deadline 2024-04-24T23:33:20.429679+0000) 2024-04-24T23:33:58.297 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:33:58 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:33:58.121+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:04.530694+0000 front 2024-04-24T23:33:04.530743+0000 (oldest deadline 2024-04-24T23:33:29.230721+0000) 2024-04-24T23:33:58.298 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:33:58 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:33:58.121+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:15.632228+0000 front 2024-04-24T23:33:15.631978+0000 (oldest deadline 2024-04-24T23:33:37.331905+0000) 2024-04-24T23:33:58.298 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:33:58 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:33:58.121+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.80:6814 osd.3 since back 2024-04-24T23:33:22.632991+0000 front 2024-04-24T23:33:22.633039+0000 (oldest deadline 2024-04-24T23:33:44.932941+0000) 2024-04-24T23:33:58.298 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:33:58 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:33:58.121+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-04-24T23:33:30.134179+0000 front 2024-04-24T23:33:30.134163+0000 (oldest deadline 2024-04-24T23:33:52.434250+0000) 2024-04-24T23:33:58.506 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:33:58 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:33:58.164+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.848330+0000 front 2024-04-24T23:32:55.848387+0000 (oldest deadline 2024-04-24T23:33:20.548352+0000) 2024-04-24T23:33:58.506 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:33:58 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:33:58.164+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:02.150412+0000 front 2024-04-24T23:33:02.150294+0000 (oldest deadline 2024-04-24T23:33:28.050409+0000) 2024-04-24T23:33:58.506 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:33:58 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:33:58.164+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:14.952538+0000 front 2024-04-24T23:33:14.952635+0000 (oldest deadline 2024-04-24T23:33:40.852722+0000) 2024-04-24T23:33:58.506 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:33:58 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:33:58.164+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-04-24T23:33:26.054443+0000 front 2024-04-24T23:33:26.054377+0000 (oldest deadline 2024-04-24T23:33:54.255147+0000) 2024-04-24T23:33:58.577 INFO:journalctl@ceph.osd.7.smithi134.stdout:Apr 24 23:33:58 smithi134 podman[67163]: 2024-04-24 23:33:58.297602881 +0000 UTC m=+0.691517767 container died 11aa2598f4f578e1744b03aeb1603b0afc76f469884e9c67b8e389add05cf89c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-7-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2) 2024-04-24T23:33:58.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:33:58 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:33:58.449+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.738540+0000 front 2024-04-24T23:32:55.738561+0000 (oldest deadline 2024-04-24T23:33:21.038786+0000) 2024-04-24T23:33:58.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:33:58 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:33:58.449+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:05.740014+0000 front 2024-04-24T23:33:05.739865+0000 (oldest deadline 2024-04-24T23:33:29.839897+0000) 2024-04-24T23:33:58.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:33:58 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:33:58.449+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:12.140601+0000 front 2024-04-24T23:33:12.140627+0000 (oldest deadline 2024-04-24T23:33:38.040825+0000) 2024-04-24T23:33:58.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:33:58 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:33:58.449+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.80:6814 osd.3 since back 2024-04-24T23:33:22.042138+0000 front 2024-04-24T23:33:22.042191+0000 (oldest deadline 2024-04-24T23:33:45.542231+0000) 2024-04-24T23:33:58.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:33:58 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:33:58.449+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-04-24T23:33:31.443174+0000 front 2024-04-24T23:33:31.443143+0000 (oldest deadline 2024-04-24T23:33:54.943017+0000) 2024-04-24T23:33:58.925 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ec7885aa-0290-11ef-bc93-c7b262605968@osd.7.service' 2024-04-24T23:33:58.962 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:33:58.962 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-04-24T23:33:58.963 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-04-24T23:33:58.963 DEBUG:teuthology.orchestra.run.smithi148:> sudo systemctl stop ceph-ec7885aa-0290-11ef-bc93-c7b262605968@osd.8 2024-04-24T23:33:58.991 INFO:journalctl@ceph.osd.8.smithi148.stdout:Apr 24 23:33:58 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-8[51367]: 2024-04-24T23:33:58.671+0000 7f119e765700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:05.559825+0000 front 2024-04-24T23:33:05.559812+0000 (oldest deadline 2024-04-24T23:33:29.560381+0000) 2024-04-24T23:33:58.991 INFO:journalctl@ceph.osd.8.smithi148.stdout:Apr 24 23:33:58 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-8[51367]: 2024-04-24T23:33:58.671+0000 7f119e765700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:13.061482+0000 front 2024-04-24T23:33:13.061509+0000 (oldest deadline 2024-04-24T23:33:36.561622+0000) 2024-04-24T23:33:58.991 INFO:journalctl@ceph.osd.8.smithi148.stdout:Apr 24 23:33:58 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-8[51367]: 2024-04-24T23:33:58.671+0000 7f119e765700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.80:6814 osd.3 since back 2024-04-24T23:33:22.863404+0000 front 2024-04-24T23:33:22.863339+0000 (oldest deadline 2024-04-24T23:33:44.563342+0000) 2024-04-24T23:33:58.991 INFO:journalctl@ceph.osd.8.smithi148.stdout:Apr 24 23:33:58 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-8[51367]: 2024-04-24T23:33:58.671+0000 7f119e765700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-04-24T23:33:29.864584+0000 front 2024-04-24T23:33:29.864717+0000 (oldest deadline 2024-04-24T23:33:53.364437+0000) 2024-04-24T23:33:59.197 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:33:59 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:33:59.129+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:56.929716+0000 front 2024-04-24T23:32:56.929774+0000 (oldest deadline 2024-04-24T23:33:20.429679+0000) 2024-04-24T23:33:59.197 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:33:59 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:33:59.129+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:04.530694+0000 front 2024-04-24T23:33:04.530743+0000 (oldest deadline 2024-04-24T23:33:29.230721+0000) 2024-04-24T23:33:59.197 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:33:59 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:33:59.129+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:15.632228+0000 front 2024-04-24T23:33:15.631978+0000 (oldest deadline 2024-04-24T23:33:37.331905+0000) 2024-04-24T23:33:59.197 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:33:59 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:33:59.129+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.80:6814 osd.3 since back 2024-04-24T23:33:22.632991+0000 front 2024-04-24T23:33:22.633039+0000 (oldest deadline 2024-04-24T23:33:44.932941+0000) 2024-04-24T23:33:59.197 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:33:59 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:33:59.129+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-04-24T23:33:30.134179+0000 front 2024-04-24T23:33:30.134163+0000 (oldest deadline 2024-04-24T23:33:52.434250+0000) 2024-04-24T23:33:59.257 INFO:journalctl@ceph.osd.8.smithi148.stdout:Apr 24 23:33:59 smithi148 systemd[1]: Stopping Ceph osd.8 for ec7885aa-0290-11ef-bc93-c7b262605968... 2024-04-24T23:33:59.257 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:33:59 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:33:59.185+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.848330+0000 front 2024-04-24T23:32:55.848387+0000 (oldest deadline 2024-04-24T23:33:20.548352+0000) 2024-04-24T23:33:59.257 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:33:59 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:33:59.185+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:02.150412+0000 front 2024-04-24T23:33:02.150294+0000 (oldest deadline 2024-04-24T23:33:28.050409+0000) 2024-04-24T23:33:59.257 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:33:59 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:33:59.185+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:14.952538+0000 front 2024-04-24T23:33:14.952635+0000 (oldest deadline 2024-04-24T23:33:40.852722+0000) 2024-04-24T23:33:59.258 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:33:59 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:33:59.185+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-04-24T23:33:26.054443+0000 front 2024-04-24T23:33:26.054377+0000 (oldest deadline 2024-04-24T23:33:54.255147+0000) 2024-04-24T23:33:59.629 INFO:journalctl@ceph.osd.8.smithi148.stdout:Apr 24 23:33:59 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-8[51367]: 2024-04-24T23:33:59.288+0000 7f11a2171700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:33:59.629 INFO:journalctl@ceph.osd.8.smithi148.stdout:Apr 24 23:33:59 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-8[51367]: 2024-04-24T23:33:59.288+0000 7f11a2171700 -1 osd.8 98 *** Got signal Terminated *** 2024-04-24T23:33:59.629 INFO:journalctl@ceph.osd.8.smithi148.stdout:Apr 24 23:33:59 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-8[51367]: 2024-04-24T23:33:59.288+0000 7f11a2171700 -1 osd.8 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T23:33:59.629 INFO:journalctl@ceph.osd.8.smithi148.stdout:Apr 24 23:33:59 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-8[51367]: 2024-04-24T23:33:59.628+0000 7f119e765700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:05.559825+0000 front 2024-04-24T23:33:05.559812+0000 (oldest deadline 2024-04-24T23:33:29.560381+0000) 2024-04-24T23:33:59.787 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:33:59 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:33:59.452+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.738540+0000 front 2024-04-24T23:32:55.738561+0000 (oldest deadline 2024-04-24T23:33:21.038786+0000) 2024-04-24T23:33:59.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:33:59 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:33:59.452+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:05.740014+0000 front 2024-04-24T23:33:05.739865+0000 (oldest deadline 2024-04-24T23:33:29.839897+0000) 2024-04-24T23:33:59.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:33:59 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:33:59.452+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:12.140601+0000 front 2024-04-24T23:33:12.140627+0000 (oldest deadline 2024-04-24T23:33:38.040825+0000) 2024-04-24T23:33:59.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:33:59 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:33:59.452+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.80:6814 osd.3 since back 2024-04-24T23:33:22.042138+0000 front 2024-04-24T23:33:22.042191+0000 (oldest deadline 2024-04-24T23:33:45.542231+0000) 2024-04-24T23:33:59.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:33:59 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:33:59.452+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-04-24T23:33:31.443174+0000 front 2024-04-24T23:33:31.443143+0000 (oldest deadline 2024-04-24T23:33:54.943017+0000) 2024-04-24T23:34:00.006 INFO:journalctl@ceph.osd.8.smithi148.stdout:Apr 24 23:33:59 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-8[51367]: 2024-04-24T23:33:59.628+0000 7f119e765700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:13.061482+0000 front 2024-04-24T23:33:13.061509+0000 (oldest deadline 2024-04-24T23:33:36.561622+0000) 2024-04-24T23:34:00.006 INFO:journalctl@ceph.osd.8.smithi148.stdout:Apr 24 23:33:59 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-8[51367]: 2024-04-24T23:33:59.628+0000 7f119e765700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.80:6814 osd.3 since back 2024-04-24T23:33:22.863404+0000 front 2024-04-24T23:33:22.863339+0000 (oldest deadline 2024-04-24T23:33:44.563342+0000) 2024-04-24T23:34:00.006 INFO:journalctl@ceph.osd.8.smithi148.stdout:Apr 24 23:33:59 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-8[51367]: 2024-04-24T23:33:59.628+0000 7f119e765700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-04-24T23:33:29.864584+0000 front 2024-04-24T23:33:29.864717+0000 (oldest deadline 2024-04-24T23:33:53.364437+0000) 2024-04-24T23:34:00.446 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:00 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:00.162+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:56.929716+0000 front 2024-04-24T23:32:56.929774+0000 (oldest deadline 2024-04-24T23:33:20.429679+0000) 2024-04-24T23:34:00.447 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:00 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:00.162+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:04.530694+0000 front 2024-04-24T23:33:04.530743+0000 (oldest deadline 2024-04-24T23:33:29.230721+0000) 2024-04-24T23:34:00.447 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:00 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:00.162+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:15.632228+0000 front 2024-04-24T23:33:15.631978+0000 (oldest deadline 2024-04-24T23:33:37.331905+0000) 2024-04-24T23:34:00.447 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:00 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:00.162+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.80:6814 osd.3 since back 2024-04-24T23:33:22.632991+0000 front 2024-04-24T23:33:22.633039+0000 (oldest deadline 2024-04-24T23:33:44.932941+0000) 2024-04-24T23:34:00.447 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:00 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:00.162+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-04-24T23:33:30.134179+0000 front 2024-04-24T23:33:30.134163+0000 (oldest deadline 2024-04-24T23:33:52.434250+0000) 2024-04-24T23:34:00.506 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:00 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:00.211+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.848330+0000 front 2024-04-24T23:32:55.848387+0000 (oldest deadline 2024-04-24T23:33:20.548352+0000) 2024-04-24T23:34:00.506 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:00 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:00.211+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:02.150412+0000 front 2024-04-24T23:33:02.150294+0000 (oldest deadline 2024-04-24T23:33:28.050409+0000) 2024-04-24T23:34:00.506 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:00 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:00.211+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:14.952538+0000 front 2024-04-24T23:33:14.952635+0000 (oldest deadline 2024-04-24T23:33:40.852722+0000) 2024-04-24T23:34:00.506 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:00 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:00.211+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-04-24T23:33:26.054443+0000 front 2024-04-24T23:33:26.054377+0000 (oldest deadline 2024-04-24T23:33:54.255147+0000) 2024-04-24T23:34:00.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:34:00 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:34:00.463+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.738540+0000 front 2024-04-24T23:32:55.738561+0000 (oldest deadline 2024-04-24T23:33:21.038786+0000) 2024-04-24T23:34:00.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:34:00 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:34:00.463+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:05.740014+0000 front 2024-04-24T23:33:05.739865+0000 (oldest deadline 2024-04-24T23:33:29.839897+0000) 2024-04-24T23:34:00.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:34:00 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:34:00.463+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:12.140601+0000 front 2024-04-24T23:33:12.140627+0000 (oldest deadline 2024-04-24T23:33:38.040825+0000) 2024-04-24T23:34:00.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:34:00 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:34:00.463+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.80:6814 osd.3 since back 2024-04-24T23:33:22.042138+0000 front 2024-04-24T23:33:22.042191+0000 (oldest deadline 2024-04-24T23:33:45.542231+0000) 2024-04-24T23:34:00.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:34:00 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:34:00.463+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-04-24T23:33:31.443174+0000 front 2024-04-24T23:33:31.443143+0000 (oldest deadline 2024-04-24T23:33:54.943017+0000) 2024-04-24T23:34:01.006 INFO:journalctl@ceph.osd.8.smithi148.stdout:Apr 24 23:34:00 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-8[51367]: 2024-04-24T23:34:00.644+0000 7f119e765700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:05.559825+0000 front 2024-04-24T23:33:05.559812+0000 (oldest deadline 2024-04-24T23:33:29.560381+0000) 2024-04-24T23:34:01.006 INFO:journalctl@ceph.osd.8.smithi148.stdout:Apr 24 23:34:00 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-8[51367]: 2024-04-24T23:34:00.644+0000 7f119e765700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:13.061482+0000 front 2024-04-24T23:33:13.061509+0000 (oldest deadline 2024-04-24T23:33:36.561622+0000) 2024-04-24T23:34:01.006 INFO:journalctl@ceph.osd.8.smithi148.stdout:Apr 24 23:34:00 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-8[51367]: 2024-04-24T23:34:00.644+0000 7f119e765700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.80:6814 osd.3 since back 2024-04-24T23:33:22.863404+0000 front 2024-04-24T23:33:22.863339+0000 (oldest deadline 2024-04-24T23:33:44.563342+0000) 2024-04-24T23:34:01.006 INFO:journalctl@ceph.osd.8.smithi148.stdout:Apr 24 23:34:00 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-8[51367]: 2024-04-24T23:34:00.644+0000 7f119e765700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-04-24T23:33:29.864584+0000 front 2024-04-24T23:33:29.864717+0000 (oldest deadline 2024-04-24T23:33:53.364437+0000) 2024-04-24T23:34:01.446 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:01 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:01.128+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:56.929716+0000 front 2024-04-24T23:32:56.929774+0000 (oldest deadline 2024-04-24T23:33:20.429679+0000) 2024-04-24T23:34:01.447 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:01 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:01.128+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:04.530694+0000 front 2024-04-24T23:33:04.530743+0000 (oldest deadline 2024-04-24T23:33:29.230721+0000) 2024-04-24T23:34:01.447 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:01 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:01.128+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:15.632228+0000 front 2024-04-24T23:33:15.631978+0000 (oldest deadline 2024-04-24T23:33:37.331905+0000) 2024-04-24T23:34:01.447 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:01 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:01.128+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.80:6814 osd.3 since back 2024-04-24T23:33:22.632991+0000 front 2024-04-24T23:33:22.633039+0000 (oldest deadline 2024-04-24T23:33:44.932941+0000) 2024-04-24T23:34:01.447 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:01 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:01.128+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-04-24T23:33:30.134179+0000 front 2024-04-24T23:33:30.134163+0000 (oldest deadline 2024-04-24T23:33:52.434250+0000) 2024-04-24T23:34:01.506 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:01 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:01.246+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.848330+0000 front 2024-04-24T23:32:55.848387+0000 (oldest deadline 2024-04-24T23:33:20.548352+0000) 2024-04-24T23:34:01.506 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:01 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:01.246+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:02.150412+0000 front 2024-04-24T23:33:02.150294+0000 (oldest deadline 2024-04-24T23:33:28.050409+0000) 2024-04-24T23:34:01.506 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:01 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:01.246+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:14.952538+0000 front 2024-04-24T23:33:14.952635+0000 (oldest deadline 2024-04-24T23:33:40.852722+0000) 2024-04-24T23:34:01.506 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:01 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:01.246+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-04-24T23:33:26.054443+0000 front 2024-04-24T23:33:26.054377+0000 (oldest deadline 2024-04-24T23:33:54.255147+0000) 2024-04-24T23:34:01.506 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:01 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:01.246+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.148:6812 osd.5 since back 2024-04-24T23:33:36.556162+0000 front 2024-04-24T23:33:36.556180+0000 (oldest deadline 2024-04-24T23:34:00.656332+0000) 2024-04-24T23:34:01.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:34:01 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:34:01.513+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.738540+0000 front 2024-04-24T23:32:55.738561+0000 (oldest deadline 2024-04-24T23:33:21.038786+0000) 2024-04-24T23:34:01.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:34:01 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:34:01.513+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:05.740014+0000 front 2024-04-24T23:33:05.739865+0000 (oldest deadline 2024-04-24T23:33:29.839897+0000) 2024-04-24T23:34:01.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:34:01 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:34:01.513+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:12.140601+0000 front 2024-04-24T23:33:12.140627+0000 (oldest deadline 2024-04-24T23:33:38.040825+0000) 2024-04-24T23:34:01.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:34:01 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:34:01.513+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.80:6814 osd.3 since back 2024-04-24T23:33:22.042138+0000 front 2024-04-24T23:33:22.042191+0000 (oldest deadline 2024-04-24T23:33:45.542231+0000) 2024-04-24T23:34:01.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:34:01 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:34:01.513+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-04-24T23:33:31.443174+0000 front 2024-04-24T23:33:31.443143+0000 (oldest deadline 2024-04-24T23:33:54.943017+0000) 2024-04-24T23:34:02.006 INFO:journalctl@ceph.osd.8.smithi148.stdout:Apr 24 23:34:01 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-8[51367]: 2024-04-24T23:34:01.660+0000 7f119e765700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:05.559825+0000 front 2024-04-24T23:33:05.559812+0000 (oldest deadline 2024-04-24T23:33:29.560381+0000) 2024-04-24T23:34:02.006 INFO:journalctl@ceph.osd.8.smithi148.stdout:Apr 24 23:34:01 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-8[51367]: 2024-04-24T23:34:01.660+0000 7f119e765700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:13.061482+0000 front 2024-04-24T23:33:13.061509+0000 (oldest deadline 2024-04-24T23:33:36.561622+0000) 2024-04-24T23:34:02.006 INFO:journalctl@ceph.osd.8.smithi148.stdout:Apr 24 23:34:01 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-8[51367]: 2024-04-24T23:34:01.660+0000 7f119e765700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.80:6814 osd.3 since back 2024-04-24T23:33:22.863404+0000 front 2024-04-24T23:33:22.863339+0000 (oldest deadline 2024-04-24T23:33:44.563342+0000) 2024-04-24T23:34:02.006 INFO:journalctl@ceph.osd.8.smithi148.stdout:Apr 24 23:34:01 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-8[51367]: 2024-04-24T23:34:01.660+0000 7f119e765700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-04-24T23:33:29.864584+0000 front 2024-04-24T23:33:29.864717+0000 (oldest deadline 2024-04-24T23:33:53.364437+0000) 2024-04-24T23:34:02.446 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:02 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:02.105+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:56.929716+0000 front 2024-04-24T23:32:56.929774+0000 (oldest deadline 2024-04-24T23:33:20.429679+0000) 2024-04-24T23:34:02.447 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:02 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:02.105+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:04.530694+0000 front 2024-04-24T23:33:04.530743+0000 (oldest deadline 2024-04-24T23:33:29.230721+0000) 2024-04-24T23:34:02.447 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:02 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:02.105+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:15.632228+0000 front 2024-04-24T23:33:15.631978+0000 (oldest deadline 2024-04-24T23:33:37.331905+0000) 2024-04-24T23:34:02.447 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:02 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:02.105+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.80:6814 osd.3 since back 2024-04-24T23:33:22.632991+0000 front 2024-04-24T23:33:22.633039+0000 (oldest deadline 2024-04-24T23:33:44.932941+0000) 2024-04-24T23:34:02.447 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:02 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:02.105+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-04-24T23:33:30.134179+0000 front 2024-04-24T23:33:30.134163+0000 (oldest deadline 2024-04-24T23:33:52.434250+0000) 2024-04-24T23:34:02.506 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:02 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:02.229+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.848330+0000 front 2024-04-24T23:32:55.848387+0000 (oldest deadline 2024-04-24T23:33:20.548352+0000) 2024-04-24T23:34:02.506 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:02 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:02.229+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:02.150412+0000 front 2024-04-24T23:33:02.150294+0000 (oldest deadline 2024-04-24T23:33:28.050409+0000) 2024-04-24T23:34:02.506 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:02 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:02.229+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:14.952538+0000 front 2024-04-24T23:33:14.952635+0000 (oldest deadline 2024-04-24T23:33:40.852722+0000) 2024-04-24T23:34:02.506 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:02 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:02.229+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-04-24T23:33:26.054443+0000 front 2024-04-24T23:33:26.054377+0000 (oldest deadline 2024-04-24T23:33:54.255147+0000) 2024-04-24T23:34:02.506 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:02 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:02.229+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.148:6812 osd.5 since back 2024-04-24T23:33:36.556162+0000 front 2024-04-24T23:33:36.556180+0000 (oldest deadline 2024-04-24T23:34:00.656332+0000) 2024-04-24T23:34:02.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:34:02 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:34:02.477+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.738540+0000 front 2024-04-24T23:32:55.738561+0000 (oldest deadline 2024-04-24T23:33:21.038786+0000) 2024-04-24T23:34:02.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:34:02 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:34:02.477+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:05.740014+0000 front 2024-04-24T23:33:05.739865+0000 (oldest deadline 2024-04-24T23:33:29.839897+0000) 2024-04-24T23:34:02.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:34:02 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:34:02.477+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:12.140601+0000 front 2024-04-24T23:33:12.140627+0000 (oldest deadline 2024-04-24T23:33:38.040825+0000) 2024-04-24T23:34:02.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:34:02 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:34:02.477+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.80:6814 osd.3 since back 2024-04-24T23:33:22.042138+0000 front 2024-04-24T23:33:22.042191+0000 (oldest deadline 2024-04-24T23:33:45.542231+0000) 2024-04-24T23:34:02.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:34:02 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:34:02.477+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-04-24T23:33:31.443174+0000 front 2024-04-24T23:33:31.443143+0000 (oldest deadline 2024-04-24T23:33:54.943017+0000) 2024-04-24T23:34:03.006 INFO:journalctl@ceph.osd.8.smithi148.stdout:Apr 24 23:34:02 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-8[51367]: 2024-04-24T23:34:02.624+0000 7f119e765700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:05.559825+0000 front 2024-04-24T23:33:05.559812+0000 (oldest deadline 2024-04-24T23:33:29.560381+0000) 2024-04-24T23:34:03.006 INFO:journalctl@ceph.osd.8.smithi148.stdout:Apr 24 23:34:02 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-8[51367]: 2024-04-24T23:34:02.624+0000 7f119e765700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:13.061482+0000 front 2024-04-24T23:33:13.061509+0000 (oldest deadline 2024-04-24T23:33:36.561622+0000) 2024-04-24T23:34:03.006 INFO:journalctl@ceph.osd.8.smithi148.stdout:Apr 24 23:34:02 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-8[51367]: 2024-04-24T23:34:02.624+0000 7f119e765700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.80:6814 osd.3 since back 2024-04-24T23:33:22.863404+0000 front 2024-04-24T23:33:22.863339+0000 (oldest deadline 2024-04-24T23:33:44.563342+0000) 2024-04-24T23:34:03.006 INFO:journalctl@ceph.osd.8.smithi148.stdout:Apr 24 23:34:02 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-8[51367]: 2024-04-24T23:34:02.624+0000 7f119e765700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-04-24T23:33:29.864584+0000 front 2024-04-24T23:33:29.864717+0000 (oldest deadline 2024-04-24T23:33:53.364437+0000) 2024-04-24T23:34:03.446 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:03 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:03.151+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:56.929716+0000 front 2024-04-24T23:32:56.929774+0000 (oldest deadline 2024-04-24T23:33:20.429679+0000) 2024-04-24T23:34:03.447 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:03 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:03.151+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:04.530694+0000 front 2024-04-24T23:33:04.530743+0000 (oldest deadline 2024-04-24T23:33:29.230721+0000) 2024-04-24T23:34:03.447 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:03 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:03.151+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:15.632228+0000 front 2024-04-24T23:33:15.631978+0000 (oldest deadline 2024-04-24T23:33:37.331905+0000) 2024-04-24T23:34:03.447 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:03 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:03.151+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.80:6814 osd.3 since back 2024-04-24T23:33:22.632991+0000 front 2024-04-24T23:33:22.633039+0000 (oldest deadline 2024-04-24T23:33:44.932941+0000) 2024-04-24T23:34:03.447 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:03 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:03.151+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-04-24T23:33:30.134179+0000 front 2024-04-24T23:33:30.134163+0000 (oldest deadline 2024-04-24T23:33:52.434250+0000) 2024-04-24T23:34:03.652 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:03 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:03.261+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.848330+0000 front 2024-04-24T23:32:55.848387+0000 (oldest deadline 2024-04-24T23:33:20.548352+0000) 2024-04-24T23:34:03.652 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:03 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:03.261+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:02.150412+0000 front 2024-04-24T23:33:02.150294+0000 (oldest deadline 2024-04-24T23:33:28.050409+0000) 2024-04-24T23:34:03.652 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:03 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:03.261+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:14.952538+0000 front 2024-04-24T23:33:14.952635+0000 (oldest deadline 2024-04-24T23:33:40.852722+0000) 2024-04-24T23:34:03.652 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:03 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:03.261+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-04-24T23:33:26.054443+0000 front 2024-04-24T23:33:26.054377+0000 (oldest deadline 2024-04-24T23:33:54.255147+0000) 2024-04-24T23:34:03.653 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:03 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:03.261+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.148:6812 osd.5 since back 2024-04-24T23:33:36.556162+0000 front 2024-04-24T23:33:36.556180+0000 (oldest deadline 2024-04-24T23:34:00.656332+0000) 2024-04-24T23:34:03.787 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:34:03 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:34:03.477+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.738540+0000 front 2024-04-24T23:32:55.738561+0000 (oldest deadline 2024-04-24T23:33:21.038786+0000) 2024-04-24T23:34:03.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:34:03 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:34:03.477+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:05.740014+0000 front 2024-04-24T23:33:05.739865+0000 (oldest deadline 2024-04-24T23:33:29.839897+0000) 2024-04-24T23:34:03.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:34:03 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:34:03.477+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:12.140601+0000 front 2024-04-24T23:33:12.140627+0000 (oldest deadline 2024-04-24T23:33:38.040825+0000) 2024-04-24T23:34:03.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:34:03 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:34:03.477+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.80:6814 osd.3 since back 2024-04-24T23:33:22.042138+0000 front 2024-04-24T23:33:22.042191+0000 (oldest deadline 2024-04-24T23:33:45.542231+0000) 2024-04-24T23:34:03.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:34:03 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:34:03.477+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-04-24T23:33:31.443174+0000 front 2024-04-24T23:33:31.443143+0000 (oldest deadline 2024-04-24T23:33:54.943017+0000) 2024-04-24T23:34:04.006 INFO:journalctl@ceph.osd.8.smithi148.stdout:Apr 24 23:34:03 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-8[51367]: 2024-04-24T23:34:03.651+0000 7f119e765700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:05.559825+0000 front 2024-04-24T23:33:05.559812+0000 (oldest deadline 2024-04-24T23:33:29.560381+0000) 2024-04-24T23:34:04.006 INFO:journalctl@ceph.osd.8.smithi148.stdout:Apr 24 23:34:03 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-8[51367]: 2024-04-24T23:34:03.651+0000 7f119e765700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:13.061482+0000 front 2024-04-24T23:33:13.061509+0000 (oldest deadline 2024-04-24T23:33:36.561622+0000) 2024-04-24T23:34:04.006 INFO:journalctl@ceph.osd.8.smithi148.stdout:Apr 24 23:34:03 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-8[51367]: 2024-04-24T23:34:03.651+0000 7f119e765700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.80:6814 osd.3 since back 2024-04-24T23:33:22.863404+0000 front 2024-04-24T23:33:22.863339+0000 (oldest deadline 2024-04-24T23:33:44.563342+0000) 2024-04-24T23:34:04.006 INFO:journalctl@ceph.osd.8.smithi148.stdout:Apr 24 23:34:03 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-8[51367]: 2024-04-24T23:34:03.651+0000 7f119e765700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-04-24T23:33:29.864584+0000 front 2024-04-24T23:33:29.864717+0000 (oldest deadline 2024-04-24T23:33:53.364437+0000) 2024-04-24T23:34:04.447 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:04 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:04.161+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:56.929716+0000 front 2024-04-24T23:32:56.929774+0000 (oldest deadline 2024-04-24T23:33:20.429679+0000) 2024-04-24T23:34:04.447 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:04 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:04.161+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:04.530694+0000 front 2024-04-24T23:33:04.530743+0000 (oldest deadline 2024-04-24T23:33:29.230721+0000) 2024-04-24T23:34:04.447 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:04 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:04.161+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:15.632228+0000 front 2024-04-24T23:33:15.631978+0000 (oldest deadline 2024-04-24T23:33:37.331905+0000) 2024-04-24T23:34:04.447 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:04 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:04.161+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.80:6814 osd.3 since back 2024-04-24T23:33:22.632991+0000 front 2024-04-24T23:33:22.633039+0000 (oldest deadline 2024-04-24T23:33:44.932941+0000) 2024-04-24T23:34:04.447 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:04 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:04.161+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-04-24T23:33:30.134179+0000 front 2024-04-24T23:33:30.134163+0000 (oldest deadline 2024-04-24T23:33:52.434250+0000) 2024-04-24T23:34:04.448 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:04 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:04.161+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.148:6812 osd.5 since back 2024-04-24T23:33:39.336230+0000 front 2024-04-24T23:33:39.336245+0000 (oldest deadline 2024-04-24T23:34:04.036102+0000) 2024-04-24T23:34:04.756 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:04 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:04.285+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.848330+0000 front 2024-04-24T23:32:55.848387+0000 (oldest deadline 2024-04-24T23:33:20.548352+0000) 2024-04-24T23:34:04.756 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:04 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:04.285+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:02.150412+0000 front 2024-04-24T23:33:02.150294+0000 (oldest deadline 2024-04-24T23:33:28.050409+0000) 2024-04-24T23:34:04.756 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:04 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:04.285+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:14.952538+0000 front 2024-04-24T23:33:14.952635+0000 (oldest deadline 2024-04-24T23:33:40.852722+0000) 2024-04-24T23:34:04.756 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:04 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:04.285+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-04-24T23:33:26.054443+0000 front 2024-04-24T23:33:26.054377+0000 (oldest deadline 2024-04-24T23:33:54.255147+0000) 2024-04-24T23:34:04.757 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:04 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:04.285+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.148:6812 osd.5 since back 2024-04-24T23:33:36.556162+0000 front 2024-04-24T23:33:36.556180+0000 (oldest deadline 2024-04-24T23:34:00.656332+0000) 2024-04-24T23:34:04.787 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:34:04 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:34:04.481+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.738540+0000 front 2024-04-24T23:32:55.738561+0000 (oldest deadline 2024-04-24T23:33:21.038786+0000) 2024-04-24T23:34:04.787 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:34:04 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:34:04.481+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:05.740014+0000 front 2024-04-24T23:33:05.739865+0000 (oldest deadline 2024-04-24T23:33:29.839897+0000) 2024-04-24T23:34:04.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:34:04 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:34:04.481+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:12.140601+0000 front 2024-04-24T23:33:12.140627+0000 (oldest deadline 2024-04-24T23:33:38.040825+0000) 2024-04-24T23:34:04.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:34:04 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:34:04.481+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.80:6814 osd.3 since back 2024-04-24T23:33:22.042138+0000 front 2024-04-24T23:33:22.042191+0000 (oldest deadline 2024-04-24T23:33:45.542231+0000) 2024-04-24T23:34:04.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:34:04 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:34:04.481+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-04-24T23:33:31.443174+0000 front 2024-04-24T23:33:31.443143+0000 (oldest deadline 2024-04-24T23:33:54.943017+0000) 2024-04-24T23:34:04.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:34:04 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:34:04.481+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.148:6812 osd.5 since back 2024-04-24T23:33:38.444142+0000 front 2024-04-24T23:33:38.443932+0000 (oldest deadline 2024-04-24T23:34:03.644171+0000) 2024-04-24T23:34:05.079 INFO:journalctl@ceph.osd.8.smithi148.stdout:Apr 24 23:34:04 smithi148 podman[67366]: 2024-04-24 23:34:04.791021578 +0000 UTC m=+5.634455451 container died 21714e30a426cc4fc893125373a5f61028bacd91bc7cdca189cacbdf80aeb607 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-8, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9) 2024-04-24T23:34:05.430 INFO:journalctl@ceph.osd.8.smithi148.stdout:Apr 24 23:34:05 smithi148 podman[67366]: 2024-04-24 23:34:05.179051124 +0000 UTC m=+6.022484999 container cleanup 21714e30a426cc4fc893125373a5f61028bacd91bc7cdca189cacbdf80aeb607 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-8, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T23:34:05.430 INFO:journalctl@ceph.osd.8.smithi148.stdout:Apr 24 23:34:05 smithi148 bash[67366]: ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-8 2024-04-24T23:34:05.430 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:05 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:05.304+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.848330+0000 front 2024-04-24T23:32:55.848387+0000 (oldest deadline 2024-04-24T23:33:20.548352+0000) 2024-04-24T23:34:05.431 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:05 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:05.304+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:02.150412+0000 front 2024-04-24T23:33:02.150294+0000 (oldest deadline 2024-04-24T23:33:28.050409+0000) 2024-04-24T23:34:05.431 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:05 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:05.304+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:14.952538+0000 front 2024-04-24T23:33:14.952635+0000 (oldest deadline 2024-04-24T23:33:40.852722+0000) 2024-04-24T23:34:05.431 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:05 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:05.304+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-04-24T23:33:26.054443+0000 front 2024-04-24T23:33:26.054377+0000 (oldest deadline 2024-04-24T23:33:54.255147+0000) 2024-04-24T23:34:05.431 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:05 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:05.304+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.148:6812 osd.5 since back 2024-04-24T23:33:36.556162+0000 front 2024-04-24T23:33:36.556180+0000 (oldest deadline 2024-04-24T23:34:00.656332+0000) 2024-04-24T23:34:05.446 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:05 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:05.146+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:56.929716+0000 front 2024-04-24T23:32:56.929774+0000 (oldest deadline 2024-04-24T23:33:20.429679+0000) 2024-04-24T23:34:05.447 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:05 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:05.146+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:04.530694+0000 front 2024-04-24T23:33:04.530743+0000 (oldest deadline 2024-04-24T23:33:29.230721+0000) 2024-04-24T23:34:05.447 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:05 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:05.146+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:15.632228+0000 front 2024-04-24T23:33:15.631978+0000 (oldest deadline 2024-04-24T23:33:37.331905+0000) 2024-04-24T23:34:05.447 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:05 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:05.146+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.80:6814 osd.3 since back 2024-04-24T23:33:22.632991+0000 front 2024-04-24T23:33:22.633039+0000 (oldest deadline 2024-04-24T23:33:44.932941+0000) 2024-04-24T23:34:05.447 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:05 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:05.146+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-04-24T23:33:30.134179+0000 front 2024-04-24T23:33:30.134163+0000 (oldest deadline 2024-04-24T23:33:52.434250+0000) 2024-04-24T23:34:05.447 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:05 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:05.146+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.148:6812 osd.5 since back 2024-04-24T23:33:39.336230+0000 front 2024-04-24T23:33:39.336245+0000 (oldest deadline 2024-04-24T23:34:04.036102+0000) 2024-04-24T23:34:05.745 INFO:journalctl@ceph.osd.8.smithi148.stdout:Apr 24 23:34:05 smithi148 podman[67382]: 2024-04-24 23:34:05.430674948 +0000 UTC m=+0.637924974 container remove 21714e30a426cc4fc893125373a5f61028bacd91bc7cdca189cacbdf80aeb607 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-8, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T23:34:05.787 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:34:05 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:34:05.486+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.738540+0000 front 2024-04-24T23:32:55.738561+0000 (oldest deadline 2024-04-24T23:33:21.038786+0000) 2024-04-24T23:34:05.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:34:05 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:34:05.486+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:05.740014+0000 front 2024-04-24T23:33:05.739865+0000 (oldest deadline 2024-04-24T23:33:29.839897+0000) 2024-04-24T23:34:05.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:34:05 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:34:05.486+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:12.140601+0000 front 2024-04-24T23:33:12.140627+0000 (oldest deadline 2024-04-24T23:33:38.040825+0000) 2024-04-24T23:34:05.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:34:05 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:34:05.486+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.80:6814 osd.3 since back 2024-04-24T23:33:22.042138+0000 front 2024-04-24T23:33:22.042191+0000 (oldest deadline 2024-04-24T23:33:45.542231+0000) 2024-04-24T23:34:05.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:34:05 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:34:05.486+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-04-24T23:33:31.443174+0000 front 2024-04-24T23:33:31.443143+0000 (oldest deadline 2024-04-24T23:33:54.943017+0000) 2024-04-24T23:34:05.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:34:05 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:34:05.486+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.148:6812 osd.5 since back 2024-04-24T23:33:38.444142+0000 front 2024-04-24T23:33:38.443932+0000 (oldest deadline 2024-04-24T23:34:03.644171+0000) 2024-04-24T23:34:06.006 INFO:journalctl@ceph.osd.8.smithi148.stdout:Apr 24 23:34:05 smithi148 podman[67473]: 2024-04-24 23:34:05.649262426 +0000 UTC m=+0.018402510 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:34:06.006 INFO:journalctl@ceph.osd.8.smithi148.stdout:Apr 24 23:34:05 smithi148 podman[67473]: 2024-04-24 23:34:05.790013079 +0000 UTC m=+0.159153169 container create ca3b437435d8a4569230d06759ac1bf1fdd487d4cd6f9a7dde0d0c98f7c07b99 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, ceph=True) 2024-04-24T23:34:06.319 INFO:journalctl@ceph.osd.8.smithi148.stdout:Apr 24 23:34:06 smithi148 podman[67473]: 2024-04-24 23:34:06.04862076 +0000 UTC m=+0.417760845 container init ca3b437435d8a4569230d06759ac1bf1fdd487d4cd6f9a7dde0d0c98f7c07b99 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-24T23:34:06.319 INFO:journalctl@ceph.osd.8.smithi148.stdout:Apr 24 23:34:06 smithi148 podman[67473]: 2024-04-24 23:34:06.052248699 +0000 UTC m=+0.421388795 container start ca3b437435d8a4569230d06759ac1bf1fdd487d4cd6f9a7dde0d0c98f7c07b99 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-8-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9) 2024-04-24T23:34:06.319 INFO:journalctl@ceph.osd.8.smithi148.stdout:Apr 24 23:34:06 smithi148 podman[67473]: 2024-04-24 23:34:06.134907662 +0000 UTC m=+0.504047761 container attach ca3b437435d8a4569230d06759ac1bf1fdd487d4cd6f9a7dde0d0c98f7c07b99 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, GIT_CLEAN=True) 2024-04-24T23:34:06.446 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:06 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:06.163+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:56.929716+0000 front 2024-04-24T23:32:56.929774+0000 (oldest deadline 2024-04-24T23:33:20.429679+0000) 2024-04-24T23:34:06.447 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:06 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:06.163+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:04.530694+0000 front 2024-04-24T23:33:04.530743+0000 (oldest deadline 2024-04-24T23:33:29.230721+0000) 2024-04-24T23:34:06.447 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:06 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:06.163+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:15.632228+0000 front 2024-04-24T23:33:15.631978+0000 (oldest deadline 2024-04-24T23:33:37.331905+0000) 2024-04-24T23:34:06.447 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:06 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:06.163+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.80:6814 osd.3 since back 2024-04-24T23:33:22.632991+0000 front 2024-04-24T23:33:22.633039+0000 (oldest deadline 2024-04-24T23:33:44.932941+0000) 2024-04-24T23:34:06.447 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:06 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:06.163+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-04-24T23:33:30.134179+0000 front 2024-04-24T23:33:30.134163+0000 (oldest deadline 2024-04-24T23:33:52.434250+0000) 2024-04-24T23:34:06.447 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:06 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:06.163+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.148:6812 osd.5 since back 2024-04-24T23:33:39.336230+0000 front 2024-04-24T23:33:39.336245+0000 (oldest deadline 2024-04-24T23:34:04.036102+0000) 2024-04-24T23:34:06.625 INFO:journalctl@ceph.osd.8.smithi148.stdout:Apr 24 23:34:06 smithi148 podman[67473]: 2024-04-24 23:34:06.349707665 +0000 UTC m=+0.718847759 container died ca3b437435d8a4569230d06759ac1bf1fdd487d4cd6f9a7dde0d0c98f7c07b99 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-8-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-24T23:34:06.625 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:06 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:06.318+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.848330+0000 front 2024-04-24T23:32:55.848387+0000 (oldest deadline 2024-04-24T23:33:20.548352+0000) 2024-04-24T23:34:06.625 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:06 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:06.318+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:02.150412+0000 front 2024-04-24T23:33:02.150294+0000 (oldest deadline 2024-04-24T23:33:28.050409+0000) 2024-04-24T23:34:06.625 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:06 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:06.318+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:14.952538+0000 front 2024-04-24T23:33:14.952635+0000 (oldest deadline 2024-04-24T23:33:40.852722+0000) 2024-04-24T23:34:06.626 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:06 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:06.318+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-04-24T23:33:26.054443+0000 front 2024-04-24T23:33:26.054377+0000 (oldest deadline 2024-04-24T23:33:54.255147+0000) 2024-04-24T23:34:06.626 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:06 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:06.318+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.148:6812 osd.5 since back 2024-04-24T23:33:36.556162+0000 front 2024-04-24T23:33:36.556180+0000 (oldest deadline 2024-04-24T23:34:00.656332+0000) 2024-04-24T23:34:06.787 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:34:06 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:34:06.521+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.738540+0000 front 2024-04-24T23:32:55.738561+0000 (oldest deadline 2024-04-24T23:33:21.038786+0000) 2024-04-24T23:34:06.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:34:06 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:34:06.521+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:05.740014+0000 front 2024-04-24T23:33:05.739865+0000 (oldest deadline 2024-04-24T23:33:29.839897+0000) 2024-04-24T23:34:06.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:34:06 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:34:06.521+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:12.140601+0000 front 2024-04-24T23:33:12.140627+0000 (oldest deadline 2024-04-24T23:33:38.040825+0000) 2024-04-24T23:34:06.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:34:06 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:34:06.521+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.80:6814 osd.3 since back 2024-04-24T23:33:22.042138+0000 front 2024-04-24T23:33:22.042191+0000 (oldest deadline 2024-04-24T23:33:45.542231+0000) 2024-04-24T23:34:06.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:34:06 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:34:06.521+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-04-24T23:33:31.443174+0000 front 2024-04-24T23:33:31.443143+0000 (oldest deadline 2024-04-24T23:33:54.943017+0000) 2024-04-24T23:34:06.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:34:06 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:34:06.521+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.148:6812 osd.5 since back 2024-04-24T23:33:38.444142+0000 front 2024-04-24T23:33:38.443932+0000 (oldest deadline 2024-04-24T23:34:03.644171+0000) 2024-04-24T23:34:06.962 DEBUG:teuthology.orchestra.run.smithi148:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ec7885aa-0290-11ef-bc93-c7b262605968@osd.8.service' 2024-04-24T23:34:07.040 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:34:07.040 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-04-24T23:34:07.041 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-04-24T23:34:07.041 DEBUG:teuthology.orchestra.run.smithi080:> sudo systemctl stop ceph-ec7885aa-0290-11ef-bc93-c7b262605968@osd.9 2024-04-24T23:34:07.351 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:34:07 smithi080 systemd[1]: Stopping Ceph osd.9 for ec7885aa-0290-11ef-bc93-c7b262605968... 2024-04-24T23:34:07.447 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:07 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:07.127+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:56.929716+0000 front 2024-04-24T23:32:56.929774+0000 (oldest deadline 2024-04-24T23:33:20.429679+0000) 2024-04-24T23:34:07.447 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:07 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:07.127+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:04.530694+0000 front 2024-04-24T23:33:04.530743+0000 (oldest deadline 2024-04-24T23:33:29.230721+0000) 2024-04-24T23:34:07.447 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:07 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:07.127+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:15.632228+0000 front 2024-04-24T23:33:15.631978+0000 (oldest deadline 2024-04-24T23:33:37.331905+0000) 2024-04-24T23:34:07.447 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:07 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:07.127+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.80:6814 osd.3 since back 2024-04-24T23:33:22.632991+0000 front 2024-04-24T23:33:22.633039+0000 (oldest deadline 2024-04-24T23:33:44.932941+0000) 2024-04-24T23:34:07.447 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:07 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:07.127+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-04-24T23:33:30.134179+0000 front 2024-04-24T23:33:30.134163+0000 (oldest deadline 2024-04-24T23:33:52.434250+0000) 2024-04-24T23:34:07.447 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:07 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:07.127+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.148:6812 osd.5 since back 2024-04-24T23:33:39.336230+0000 front 2024-04-24T23:33:39.336245+0000 (oldest deadline 2024-04-24T23:34:04.036102+0000) 2024-04-24T23:34:07.561 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:07 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:07.310+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.848330+0000 front 2024-04-24T23:32:55.848387+0000 (oldest deadline 2024-04-24T23:33:20.548352+0000) 2024-04-24T23:34:07.561 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:07 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:07.310+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:02.150412+0000 front 2024-04-24T23:33:02.150294+0000 (oldest deadline 2024-04-24T23:33:28.050409+0000) 2024-04-24T23:34:07.561 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:07 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:07.310+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:14.952538+0000 front 2024-04-24T23:33:14.952635+0000 (oldest deadline 2024-04-24T23:33:40.852722+0000) 2024-04-24T23:34:07.561 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:07 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:07.310+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-04-24T23:33:26.054443+0000 front 2024-04-24T23:33:26.054377+0000 (oldest deadline 2024-04-24T23:33:54.255147+0000) 2024-04-24T23:34:07.562 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:07 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:07.310+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.148:6812 osd.5 since back 2024-04-24T23:33:36.556162+0000 front 2024-04-24T23:33:36.556180+0000 (oldest deadline 2024-04-24T23:34:00.656332+0000) 2024-04-24T23:34:07.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:34:07 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:34:07.350+0000 7f8b753cc700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:34:07.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:34:07 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:34:07.350+0000 7f8b753cc700 -1 osd.9 98 *** Got signal Terminated *** 2024-04-24T23:34:07.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:34:07 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:34:07.350+0000 7f8b753cc700 -1 osd.9 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T23:34:07.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:34:07 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:34:07.480+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.738540+0000 front 2024-04-24T23:32:55.738561+0000 (oldest deadline 2024-04-24T23:33:21.038786+0000) 2024-04-24T23:34:07.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:34:07 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:34:07.480+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:05.740014+0000 front 2024-04-24T23:33:05.739865+0000 (oldest deadline 2024-04-24T23:33:29.839897+0000) 2024-04-24T23:34:07.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:34:07 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:34:07.480+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:12.140601+0000 front 2024-04-24T23:33:12.140627+0000 (oldest deadline 2024-04-24T23:33:38.040825+0000) 2024-04-24T23:34:07.789 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:34:07 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:34:07.480+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.80:6814 osd.3 since back 2024-04-24T23:33:22.042138+0000 front 2024-04-24T23:33:22.042191+0000 (oldest deadline 2024-04-24T23:33:45.542231+0000) 2024-04-24T23:34:07.789 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:34:07 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:34:07.480+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-04-24T23:33:31.443174+0000 front 2024-04-24T23:33:31.443143+0000 (oldest deadline 2024-04-24T23:33:54.943017+0000) 2024-04-24T23:34:07.789 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:34:07 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:34:07.480+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.148:6812 osd.5 since back 2024-04-24T23:33:38.444142+0000 front 2024-04-24T23:33:38.443932+0000 (oldest deadline 2024-04-24T23:34:03.644171+0000) 2024-04-24T23:34:08.447 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:08 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:08.078+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:56.929716+0000 front 2024-04-24T23:32:56.929774+0000 (oldest deadline 2024-04-24T23:33:20.429679+0000) 2024-04-24T23:34:08.447 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:08 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:08.078+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:04.530694+0000 front 2024-04-24T23:33:04.530743+0000 (oldest deadline 2024-04-24T23:33:29.230721+0000) 2024-04-24T23:34:08.447 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:08 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:08.078+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:15.632228+0000 front 2024-04-24T23:33:15.631978+0000 (oldest deadline 2024-04-24T23:33:37.331905+0000) 2024-04-24T23:34:08.447 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:08 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:08.078+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.80:6814 osd.3 since back 2024-04-24T23:33:22.632991+0000 front 2024-04-24T23:33:22.633039+0000 (oldest deadline 2024-04-24T23:33:44.932941+0000) 2024-04-24T23:34:08.447 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:08 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:08.078+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-04-24T23:33:30.134179+0000 front 2024-04-24T23:33:30.134163+0000 (oldest deadline 2024-04-24T23:33:52.434250+0000) 2024-04-24T23:34:08.447 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:08 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:08.078+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.148:6812 osd.5 since back 2024-04-24T23:33:39.336230+0000 front 2024-04-24T23:33:39.336245+0000 (oldest deadline 2024-04-24T23:34:04.036102+0000) 2024-04-24T23:34:08.756 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:08 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:08.290+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.848330+0000 front 2024-04-24T23:32:55.848387+0000 (oldest deadline 2024-04-24T23:33:20.548352+0000) 2024-04-24T23:34:08.756 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:08 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:08.290+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:02.150412+0000 front 2024-04-24T23:33:02.150294+0000 (oldest deadline 2024-04-24T23:33:28.050409+0000) 2024-04-24T23:34:08.756 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:08 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:08.290+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:14.952538+0000 front 2024-04-24T23:33:14.952635+0000 (oldest deadline 2024-04-24T23:33:40.852722+0000) 2024-04-24T23:34:08.756 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:08 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:08.290+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-04-24T23:33:26.054443+0000 front 2024-04-24T23:33:26.054377+0000 (oldest deadline 2024-04-24T23:33:54.255147+0000) 2024-04-24T23:34:08.756 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:08 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:08.290+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.148:6812 osd.5 since back 2024-04-24T23:33:36.556162+0000 front 2024-04-24T23:33:36.556180+0000 (oldest deadline 2024-04-24T23:34:00.656332+0000) 2024-04-24T23:34:08.787 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:34:08 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:34:08.452+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.738540+0000 front 2024-04-24T23:32:55.738561+0000 (oldest deadline 2024-04-24T23:33:21.038786+0000) 2024-04-24T23:34:08.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:34:08 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:34:08.452+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:05.740014+0000 front 2024-04-24T23:33:05.739865+0000 (oldest deadline 2024-04-24T23:33:29.839897+0000) 2024-04-24T23:34:08.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:34:08 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:34:08.452+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:12.140601+0000 front 2024-04-24T23:33:12.140627+0000 (oldest deadline 2024-04-24T23:33:38.040825+0000) 2024-04-24T23:34:08.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:34:08 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:34:08.452+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.80:6814 osd.3 since back 2024-04-24T23:33:22.042138+0000 front 2024-04-24T23:33:22.042191+0000 (oldest deadline 2024-04-24T23:33:45.542231+0000) 2024-04-24T23:34:08.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:34:08 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:34:08.452+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-04-24T23:33:31.443174+0000 front 2024-04-24T23:33:31.443143+0000 (oldest deadline 2024-04-24T23:33:54.943017+0000) 2024-04-24T23:34:08.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:34:08 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:34:08.452+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.148:6812 osd.5 since back 2024-04-24T23:33:38.444142+0000 front 2024-04-24T23:33:38.443932+0000 (oldest deadline 2024-04-24T23:34:03.644171+0000) 2024-04-24T23:34:09.447 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:09 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:09.117+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:56.929716+0000 front 2024-04-24T23:32:56.929774+0000 (oldest deadline 2024-04-24T23:33:20.429679+0000) 2024-04-24T23:34:09.447 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:09 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:09.117+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:04.530694+0000 front 2024-04-24T23:33:04.530743+0000 (oldest deadline 2024-04-24T23:33:29.230721+0000) 2024-04-24T23:34:09.447 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:09 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:09.117+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:15.632228+0000 front 2024-04-24T23:33:15.631978+0000 (oldest deadline 2024-04-24T23:33:37.331905+0000) 2024-04-24T23:34:09.447 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:09 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:09.117+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.80:6814 osd.3 since back 2024-04-24T23:33:22.632991+0000 front 2024-04-24T23:33:22.633039+0000 (oldest deadline 2024-04-24T23:33:44.932941+0000) 2024-04-24T23:34:09.447 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:09 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:09.117+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-04-24T23:33:30.134179+0000 front 2024-04-24T23:33:30.134163+0000 (oldest deadline 2024-04-24T23:33:52.434250+0000) 2024-04-24T23:34:09.448 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:09 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:09.117+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.148:6812 osd.5 since back 2024-04-24T23:33:39.336230+0000 front 2024-04-24T23:33:39.336245+0000 (oldest deadline 2024-04-24T23:34:04.036102+0000) 2024-04-24T23:34:09.756 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:09 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:09.291+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.848330+0000 front 2024-04-24T23:32:55.848387+0000 (oldest deadline 2024-04-24T23:33:20.548352+0000) 2024-04-24T23:34:09.756 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:09 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:09.291+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:02.150412+0000 front 2024-04-24T23:33:02.150294+0000 (oldest deadline 2024-04-24T23:33:28.050409+0000) 2024-04-24T23:34:09.756 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:09 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:09.291+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:14.952538+0000 front 2024-04-24T23:33:14.952635+0000 (oldest deadline 2024-04-24T23:33:40.852722+0000) 2024-04-24T23:34:09.756 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:09 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:09.291+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-04-24T23:33:26.054443+0000 front 2024-04-24T23:33:26.054377+0000 (oldest deadline 2024-04-24T23:33:54.255147+0000) 2024-04-24T23:34:09.756 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:09 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:09.291+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.148:6812 osd.5 since back 2024-04-24T23:33:36.556162+0000 front 2024-04-24T23:33:36.556180+0000 (oldest deadline 2024-04-24T23:34:00.656332+0000) 2024-04-24T23:34:09.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:34:09 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:34:09.416+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.738540+0000 front 2024-04-24T23:32:55.738561+0000 (oldest deadline 2024-04-24T23:33:21.038786+0000) 2024-04-24T23:34:09.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:34:09 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:34:09.416+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:05.740014+0000 front 2024-04-24T23:33:05.739865+0000 (oldest deadline 2024-04-24T23:33:29.839897+0000) 2024-04-24T23:34:09.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:34:09 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:34:09.416+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:12.140601+0000 front 2024-04-24T23:33:12.140627+0000 (oldest deadline 2024-04-24T23:33:38.040825+0000) 2024-04-24T23:34:09.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:34:09 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:34:09.416+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.80:6814 osd.3 since back 2024-04-24T23:33:22.042138+0000 front 2024-04-24T23:33:22.042191+0000 (oldest deadline 2024-04-24T23:33:45.542231+0000) 2024-04-24T23:34:09.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:34:09 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:34:09.416+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-04-24T23:33:31.443174+0000 front 2024-04-24T23:33:31.443143+0000 (oldest deadline 2024-04-24T23:33:54.943017+0000) 2024-04-24T23:34:09.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:34:09 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:34:09.416+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.148:6812 osd.5 since back 2024-04-24T23:33:38.444142+0000 front 2024-04-24T23:33:38.443932+0000 (oldest deadline 2024-04-24T23:34:03.644171+0000) 2024-04-24T23:34:10.447 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:10 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:10.161+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:56.929716+0000 front 2024-04-24T23:32:56.929774+0000 (oldest deadline 2024-04-24T23:33:20.429679+0000) 2024-04-24T23:34:10.447 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:10 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:10.161+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:04.530694+0000 front 2024-04-24T23:33:04.530743+0000 (oldest deadline 2024-04-24T23:33:29.230721+0000) 2024-04-24T23:34:10.447 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:10 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:10.161+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:15.632228+0000 front 2024-04-24T23:33:15.631978+0000 (oldest deadline 2024-04-24T23:33:37.331905+0000) 2024-04-24T23:34:10.447 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:10 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:10.161+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.80:6814 osd.3 since back 2024-04-24T23:33:22.632991+0000 front 2024-04-24T23:33:22.633039+0000 (oldest deadline 2024-04-24T23:33:44.932941+0000) 2024-04-24T23:34:10.447 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:10 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:10.161+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-04-24T23:33:30.134179+0000 front 2024-04-24T23:33:30.134163+0000 (oldest deadline 2024-04-24T23:33:52.434250+0000) 2024-04-24T23:34:10.447 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:10 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:10.161+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.148:6812 osd.5 since back 2024-04-24T23:33:39.336230+0000 front 2024-04-24T23:33:39.336245+0000 (oldest deadline 2024-04-24T23:34:04.036102+0000) 2024-04-24T23:34:10.755 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:10 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:10.302+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.848330+0000 front 2024-04-24T23:32:55.848387+0000 (oldest deadline 2024-04-24T23:33:20.548352+0000) 2024-04-24T23:34:10.756 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:10 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:10.302+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:02.150412+0000 front 2024-04-24T23:33:02.150294+0000 (oldest deadline 2024-04-24T23:33:28.050409+0000) 2024-04-24T23:34:10.756 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:10 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:10.302+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:14.952538+0000 front 2024-04-24T23:33:14.952635+0000 (oldest deadline 2024-04-24T23:33:40.852722+0000) 2024-04-24T23:34:10.756 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:10 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:10.302+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-04-24T23:33:26.054443+0000 front 2024-04-24T23:33:26.054377+0000 (oldest deadline 2024-04-24T23:33:54.255147+0000) 2024-04-24T23:34:10.756 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:10 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:10.302+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.148:6812 osd.5 since back 2024-04-24T23:33:36.556162+0000 front 2024-04-24T23:33:36.556180+0000 (oldest deadline 2024-04-24T23:34:00.656332+0000) 2024-04-24T23:34:10.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:34:10 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:34:10.398+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.738540+0000 front 2024-04-24T23:32:55.738561+0000 (oldest deadline 2024-04-24T23:33:21.038786+0000) 2024-04-24T23:34:10.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:34:10 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:34:10.398+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:05.740014+0000 front 2024-04-24T23:33:05.739865+0000 (oldest deadline 2024-04-24T23:33:29.839897+0000) 2024-04-24T23:34:10.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:34:10 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:34:10.398+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:12.140601+0000 front 2024-04-24T23:33:12.140627+0000 (oldest deadline 2024-04-24T23:33:38.040825+0000) 2024-04-24T23:34:10.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:34:10 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:34:10.398+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.80:6814 osd.3 since back 2024-04-24T23:33:22.042138+0000 front 2024-04-24T23:33:22.042191+0000 (oldest deadline 2024-04-24T23:33:45.542231+0000) 2024-04-24T23:34:10.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:34:10 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:34:10.398+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-04-24T23:33:31.443174+0000 front 2024-04-24T23:33:31.443143+0000 (oldest deadline 2024-04-24T23:33:54.943017+0000) 2024-04-24T23:34:10.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:34:10 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:34:10.398+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.148:6812 osd.5 since back 2024-04-24T23:33:38.444142+0000 front 2024-04-24T23:33:38.443932+0000 (oldest deadline 2024-04-24T23:34:03.644171+0000) 2024-04-24T23:34:11.447 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:11 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:11.174+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:56.929716+0000 front 2024-04-24T23:32:56.929774+0000 (oldest deadline 2024-04-24T23:33:20.429679+0000) 2024-04-24T23:34:11.447 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:11 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:11.174+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:04.530694+0000 front 2024-04-24T23:33:04.530743+0000 (oldest deadline 2024-04-24T23:33:29.230721+0000) 2024-04-24T23:34:11.447 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:11 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:11.174+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:15.632228+0000 front 2024-04-24T23:33:15.631978+0000 (oldest deadline 2024-04-24T23:33:37.331905+0000) 2024-04-24T23:34:11.447 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:11 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:11.174+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.80:6814 osd.3 since back 2024-04-24T23:33:22.632991+0000 front 2024-04-24T23:33:22.633039+0000 (oldest deadline 2024-04-24T23:33:44.932941+0000) 2024-04-24T23:34:11.447 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:11 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:11.174+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-04-24T23:33:30.134179+0000 front 2024-04-24T23:33:30.134163+0000 (oldest deadline 2024-04-24T23:33:52.434250+0000) 2024-04-24T23:34:11.448 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:11 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:11.174+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.148:6812 osd.5 since back 2024-04-24T23:33:39.336230+0000 front 2024-04-24T23:33:39.336245+0000 (oldest deadline 2024-04-24T23:34:04.036102+0000) 2024-04-24T23:34:11.755 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:11 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:11.299+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.848330+0000 front 2024-04-24T23:32:55.848387+0000 (oldest deadline 2024-04-24T23:33:20.548352+0000) 2024-04-24T23:34:11.756 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:11 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:11.299+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:02.150412+0000 front 2024-04-24T23:33:02.150294+0000 (oldest deadline 2024-04-24T23:33:28.050409+0000) 2024-04-24T23:34:11.756 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:11 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:11.299+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:14.952538+0000 front 2024-04-24T23:33:14.952635+0000 (oldest deadline 2024-04-24T23:33:40.852722+0000) 2024-04-24T23:34:11.756 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:11 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:11.299+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-04-24T23:33:26.054443+0000 front 2024-04-24T23:33:26.054377+0000 (oldest deadline 2024-04-24T23:33:54.255147+0000) 2024-04-24T23:34:11.756 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:11 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:11.299+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.148:6812 osd.5 since back 2024-04-24T23:33:36.556162+0000 front 2024-04-24T23:33:36.556180+0000 (oldest deadline 2024-04-24T23:34:00.656332+0000) 2024-04-24T23:34:11.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:34:11 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:34:11.349+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.738540+0000 front 2024-04-24T23:32:55.738561+0000 (oldest deadline 2024-04-24T23:33:21.038786+0000) 2024-04-24T23:34:11.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:34:11 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:34:11.349+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:05.740014+0000 front 2024-04-24T23:33:05.739865+0000 (oldest deadline 2024-04-24T23:33:29.839897+0000) 2024-04-24T23:34:11.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:34:11 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:34:11.349+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:12.140601+0000 front 2024-04-24T23:33:12.140627+0000 (oldest deadline 2024-04-24T23:33:38.040825+0000) 2024-04-24T23:34:11.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:34:11 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:34:11.349+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.80:6814 osd.3 since back 2024-04-24T23:33:22.042138+0000 front 2024-04-24T23:33:22.042191+0000 (oldest deadline 2024-04-24T23:33:45.542231+0000) 2024-04-24T23:34:11.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:34:11 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:34:11.349+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-04-24T23:33:31.443174+0000 front 2024-04-24T23:33:31.443143+0000 (oldest deadline 2024-04-24T23:33:54.943017+0000) 2024-04-24T23:34:11.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:34:11 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9[63834]: 2024-04-24T23:34:11.349+0000 7f8b6d9b8700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.148:6812 osd.5 since back 2024-04-24T23:33:38.444142+0000 front 2024-04-24T23:33:38.443932+0000 (oldest deadline 2024-04-24T23:34:03.644171+0000) 2024-04-24T23:34:12.446 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:12 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:12.131+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:56.929716+0000 front 2024-04-24T23:32:56.929774+0000 (oldest deadline 2024-04-24T23:33:20.429679+0000) 2024-04-24T23:34:12.447 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:12 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:12.131+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:04.530694+0000 front 2024-04-24T23:33:04.530743+0000 (oldest deadline 2024-04-24T23:33:29.230721+0000) 2024-04-24T23:34:12.447 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:12 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:12.131+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:15.632228+0000 front 2024-04-24T23:33:15.631978+0000 (oldest deadline 2024-04-24T23:33:37.331905+0000) 2024-04-24T23:34:12.447 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:12 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:12.131+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.80:6814 osd.3 since back 2024-04-24T23:33:22.632991+0000 front 2024-04-24T23:33:22.633039+0000 (oldest deadline 2024-04-24T23:33:44.932941+0000) 2024-04-24T23:34:12.447 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:12 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:12.131+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-04-24T23:33:30.134179+0000 front 2024-04-24T23:33:30.134163+0000 (oldest deadline 2024-04-24T23:33:52.434250+0000) 2024-04-24T23:34:12.447 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:12 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:12.131+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.148:6812 osd.5 since back 2024-04-24T23:33:39.336230+0000 front 2024-04-24T23:33:39.336245+0000 (oldest deadline 2024-04-24T23:34:04.036102+0000) 2024-04-24T23:34:12.756 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:12 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:12.263+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.848330+0000 front 2024-04-24T23:32:55.848387+0000 (oldest deadline 2024-04-24T23:33:20.548352+0000) 2024-04-24T23:34:12.756 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:12 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:12.263+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:02.150412+0000 front 2024-04-24T23:33:02.150294+0000 (oldest deadline 2024-04-24T23:33:28.050409+0000) 2024-04-24T23:34:12.756 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:12 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:12.263+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:14.952538+0000 front 2024-04-24T23:33:14.952635+0000 (oldest deadline 2024-04-24T23:33:40.852722+0000) 2024-04-24T23:34:12.756 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:12 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:12.263+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-04-24T23:33:26.054443+0000 front 2024-04-24T23:33:26.054377+0000 (oldest deadline 2024-04-24T23:33:54.255147+0000) 2024-04-24T23:34:12.756 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:12 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:12.263+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.148:6812 osd.5 since back 2024-04-24T23:33:36.556162+0000 front 2024-04-24T23:33:36.556180+0000 (oldest deadline 2024-04-24T23:34:00.656332+0000) 2024-04-24T23:34:12.756 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:12 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:12.263+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.80:6822 osd.6 since back 2024-04-24T23:33:47.558297+0000 front 2024-04-24T23:33:47.558371+0000 (oldest deadline 2024-04-24T23:34:11.658305+0000) 2024-04-24T23:34:12.984 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:34:12 smithi080 podman[95020]: 2024-04-24 23:34:12.713852977 +0000 UTC m=+5.473500370 container died 5d879543b86f10add588a0df5d46964dd20a9a74126a473ff8ac2fedc6a1ec1d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-24T23:34:13.287 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:34:13 smithi080 podman[95020]: 2024-04-24 23:34:13.176121097 +0000 UTC m=+5.935768498 container cleanup 5d879543b86f10add588a0df5d46964dd20a9a74126a473ff8ac2fedc6a1ec1d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2) 2024-04-24T23:34:13.288 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:34:13 smithi080 bash[95020]: ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9 2024-04-24T23:34:13.446 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:13 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:13.175+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:56.929716+0000 front 2024-04-24T23:32:56.929774+0000 (oldest deadline 2024-04-24T23:33:20.429679+0000) 2024-04-24T23:34:13.447 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:13 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:13.175+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:04.530694+0000 front 2024-04-24T23:33:04.530743+0000 (oldest deadline 2024-04-24T23:33:29.230721+0000) 2024-04-24T23:34:13.447 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:13 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:13.175+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:15.632228+0000 front 2024-04-24T23:33:15.631978+0000 (oldest deadline 2024-04-24T23:33:37.331905+0000) 2024-04-24T23:34:13.447 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:13 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:13.175+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.80:6814 osd.3 since back 2024-04-24T23:33:22.632991+0000 front 2024-04-24T23:33:22.633039+0000 (oldest deadline 2024-04-24T23:33:44.932941+0000) 2024-04-24T23:34:13.447 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:13 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:13.175+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-04-24T23:33:30.134179+0000 front 2024-04-24T23:33:30.134163+0000 (oldest deadline 2024-04-24T23:33:52.434250+0000) 2024-04-24T23:34:13.447 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:13 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:13.175+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.148:6812 osd.5 since back 2024-04-24T23:33:39.336230+0000 front 2024-04-24T23:33:39.336245+0000 (oldest deadline 2024-04-24T23:34:04.036102+0000) 2024-04-24T23:34:13.447 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:13 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:13.175+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.80:6822 osd.6 since back 2024-04-24T23:33:47.536934+0000 front 2024-04-24T23:33:47.536873+0000 (oldest deadline 2024-04-24T23:34:12.737307+0000) 2024-04-24T23:34:13.506 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:13 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:13.229+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.848330+0000 front 2024-04-24T23:32:55.848387+0000 (oldest deadline 2024-04-24T23:33:20.548352+0000) 2024-04-24T23:34:13.507 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:13 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:13.229+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:02.150412+0000 front 2024-04-24T23:33:02.150294+0000 (oldest deadline 2024-04-24T23:33:28.050409+0000) 2024-04-24T23:34:13.507 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:13 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:13.229+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:14.952538+0000 front 2024-04-24T23:33:14.952635+0000 (oldest deadline 2024-04-24T23:33:40.852722+0000) 2024-04-24T23:34:13.507 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:13 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:13.229+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-04-24T23:33:26.054443+0000 front 2024-04-24T23:33:26.054377+0000 (oldest deadline 2024-04-24T23:33:54.255147+0000) 2024-04-24T23:34:13.507 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:13 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:13.229+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.148:6812 osd.5 since back 2024-04-24T23:33:36.556162+0000 front 2024-04-24T23:33:36.556180+0000 (oldest deadline 2024-04-24T23:34:00.656332+0000) 2024-04-24T23:34:13.507 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:13 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:13.229+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.80:6822 osd.6 since back 2024-04-24T23:33:47.558297+0000 front 2024-04-24T23:33:47.558371+0000 (oldest deadline 2024-04-24T23:34:11.658305+0000) 2024-04-24T23:34:13.788 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:34:13 smithi080 podman[95039]: 2024-04-24 23:34:13.477835053 +0000 UTC m=+0.754990404 container remove 5d879543b86f10add588a0df5d46964dd20a9a74126a473ff8ac2fedc6a1ec1d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-04-24T23:34:14.114 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:34:13 smithi080 podman[95129]: 2024-04-24 23:34:13.739211973 +0000 UTC m=+0.017800201 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:34:14.114 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:34:13 smithi080 podman[95129]: 2024-04-24 23:34:13.845295848 +0000 UTC m=+0.123884046 container create f92aff2617208374e2f20fcf8ed7348af0827db44c30a9cf0b16bfb3ba112652 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-04-24T23:34:14.115 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:34:14 smithi080 podman[95129]: 2024-04-24 23:34:14.039411293 +0000 UTC m=+0.317999490 container init f92aff2617208374e2f20fcf8ed7348af0827db44c30a9cf0b16bfb3ba112652 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, ceph=True) 2024-04-24T23:34:14.115 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:34:14 smithi080 podman[95129]: 2024-04-24 23:34:14.042497998 +0000 UTC m=+0.321086203 container start f92aff2617208374e2f20fcf8ed7348af0827db44c30a9cf0b16bfb3ba112652 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, ceph=True, GIT_CLEAN=True) 2024-04-24T23:34:14.115 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:34:14 smithi080 podman[95129]: 2024-04-24 23:34:14.114310148 +0000 UTC m=+0.392898348 container attach f92aff2617208374e2f20fcf8ed7348af0827db44c30a9cf0b16bfb3ba112652 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-04-24T23:34:14.446 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:14 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:14.195+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:56.929716+0000 front 2024-04-24T23:32:56.929774+0000 (oldest deadline 2024-04-24T23:33:20.429679+0000) 2024-04-24T23:34:14.447 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:14 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:14.195+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:04.530694+0000 front 2024-04-24T23:33:04.530743+0000 (oldest deadline 2024-04-24T23:33:29.230721+0000) 2024-04-24T23:34:14.447 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:14 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:14.195+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:15.632228+0000 front 2024-04-24T23:33:15.631978+0000 (oldest deadline 2024-04-24T23:33:37.331905+0000) 2024-04-24T23:34:14.447 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:14 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:14.195+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.80:6814 osd.3 since back 2024-04-24T23:33:22.632991+0000 front 2024-04-24T23:33:22.633039+0000 (oldest deadline 2024-04-24T23:33:44.932941+0000) 2024-04-24T23:34:14.447 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:14 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:14.195+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-04-24T23:33:30.134179+0000 front 2024-04-24T23:33:30.134163+0000 (oldest deadline 2024-04-24T23:33:52.434250+0000) 2024-04-24T23:34:14.447 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:14 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:14.195+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.148:6812 osd.5 since back 2024-04-24T23:33:39.336230+0000 front 2024-04-24T23:33:39.336245+0000 (oldest deadline 2024-04-24T23:34:04.036102+0000) 2024-04-24T23:34:14.448 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:14 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:14.195+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.80:6822 osd.6 since back 2024-04-24T23:33:47.536934+0000 front 2024-04-24T23:33:47.536873+0000 (oldest deadline 2024-04-24T23:34:12.737307+0000) 2024-04-24T23:34:14.506 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:14 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:14.246+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.848330+0000 front 2024-04-24T23:32:55.848387+0000 (oldest deadline 2024-04-24T23:33:20.548352+0000) 2024-04-24T23:34:14.506 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:14 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:14.246+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:02.150412+0000 front 2024-04-24T23:33:02.150294+0000 (oldest deadline 2024-04-24T23:33:28.050409+0000) 2024-04-24T23:34:14.506 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:14 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:14.246+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:14.952538+0000 front 2024-04-24T23:33:14.952635+0000 (oldest deadline 2024-04-24T23:33:40.852722+0000) 2024-04-24T23:34:14.506 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:14 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:14.246+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-04-24T23:33:26.054443+0000 front 2024-04-24T23:33:26.054377+0000 (oldest deadline 2024-04-24T23:33:54.255147+0000) 2024-04-24T23:34:14.507 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:14 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:14.246+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.148:6812 osd.5 since back 2024-04-24T23:33:36.556162+0000 front 2024-04-24T23:33:36.556180+0000 (oldest deadline 2024-04-24T23:34:00.656332+0000) 2024-04-24T23:34:14.507 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:14 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:14.246+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.80:6822 osd.6 since back 2024-04-24T23:33:47.558297+0000 front 2024-04-24T23:33:47.558371+0000 (oldest deadline 2024-04-24T23:34:11.658305+0000) 2024-04-24T23:34:14.538 INFO:journalctl@ceph.osd.9.smithi080.stdout:Apr 24 23:34:14 smithi080 podman[95129]: 2024-04-24 23:34:14.333917868 +0000 UTC m=+0.612506078 container died f92aff2617208374e2f20fcf8ed7348af0827db44c30a9cf0b16bfb3ba112652 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T23:34:15.011 DEBUG:teuthology.orchestra.run.smithi080:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ec7885aa-0290-11ef-bc93-c7b262605968@osd.9.service' 2024-04-24T23:34:15.047 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:34:15.048 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-04-24T23:34:15.048 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-04-24T23:34:15.048 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-ec7885aa-0290-11ef-bc93-c7b262605968@osd.10 2024-04-24T23:34:15.447 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:15 smithi134 systemd[1]: Stopping Ceph osd.10 for ec7885aa-0290-11ef-bc93-c7b262605968... 2024-04-24T23:34:15.447 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:15 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:15.184+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:56.929716+0000 front 2024-04-24T23:32:56.929774+0000 (oldest deadline 2024-04-24T23:33:20.429679+0000) 2024-04-24T23:34:15.447 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:15 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:15.184+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:04.530694+0000 front 2024-04-24T23:33:04.530743+0000 (oldest deadline 2024-04-24T23:33:29.230721+0000) 2024-04-24T23:34:15.448 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:15 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:15.184+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:15.632228+0000 front 2024-04-24T23:33:15.631978+0000 (oldest deadline 2024-04-24T23:33:37.331905+0000) 2024-04-24T23:34:15.448 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:15 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:15.184+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.80:6814 osd.3 since back 2024-04-24T23:33:22.632991+0000 front 2024-04-24T23:33:22.633039+0000 (oldest deadline 2024-04-24T23:33:44.932941+0000) 2024-04-24T23:34:15.448 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:15 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:15.184+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-04-24T23:33:30.134179+0000 front 2024-04-24T23:33:30.134163+0000 (oldest deadline 2024-04-24T23:33:52.434250+0000) 2024-04-24T23:34:15.448 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:15 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:15.184+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.148:6812 osd.5 since back 2024-04-24T23:33:39.336230+0000 front 2024-04-24T23:33:39.336245+0000 (oldest deadline 2024-04-24T23:34:04.036102+0000) 2024-04-24T23:34:15.448 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:15 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:15.184+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.80:6822 osd.6 since back 2024-04-24T23:33:47.536934+0000 front 2024-04-24T23:33:47.536873+0000 (oldest deadline 2024-04-24T23:34:12.737307+0000) 2024-04-24T23:34:15.756 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:15 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:15.282+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.848330+0000 front 2024-04-24T23:32:55.848387+0000 (oldest deadline 2024-04-24T23:33:20.548352+0000) 2024-04-24T23:34:15.756 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:15 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:15.282+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:02.150412+0000 front 2024-04-24T23:33:02.150294+0000 (oldest deadline 2024-04-24T23:33:28.050409+0000) 2024-04-24T23:34:15.756 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:15 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:15.282+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:14.952538+0000 front 2024-04-24T23:33:14.952635+0000 (oldest deadline 2024-04-24T23:33:40.852722+0000) 2024-04-24T23:34:15.756 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:15 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:15.282+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-04-24T23:33:26.054443+0000 front 2024-04-24T23:33:26.054377+0000 (oldest deadline 2024-04-24T23:33:54.255147+0000) 2024-04-24T23:34:15.756 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:15 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:15.282+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.148:6812 osd.5 since back 2024-04-24T23:33:36.556162+0000 front 2024-04-24T23:33:36.556180+0000 (oldest deadline 2024-04-24T23:34:00.656332+0000) 2024-04-24T23:34:15.757 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:15 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:15.282+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.80:6822 osd.6 since back 2024-04-24T23:33:47.558297+0000 front 2024-04-24T23:33:47.558371+0000 (oldest deadline 2024-04-24T23:34:11.658305+0000) 2024-04-24T23:34:15.947 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:15 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:15.521+0000 7f02a2d0a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:34:15.947 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:15 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:15.521+0000 7f02a2d0a700 -1 osd.10 98 *** Got signal Terminated *** 2024-04-24T23:34:15.947 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:15 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:15.521+0000 7f02a2d0a700 -1 osd.10 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T23:34:16.447 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:16 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:16.188+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:56.929716+0000 front 2024-04-24T23:32:56.929774+0000 (oldest deadline 2024-04-24T23:33:20.429679+0000) 2024-04-24T23:34:16.447 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:16 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:16.188+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:04.530694+0000 front 2024-04-24T23:33:04.530743+0000 (oldest deadline 2024-04-24T23:33:29.230721+0000) 2024-04-24T23:34:16.447 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:16 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:16.188+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:15.632228+0000 front 2024-04-24T23:33:15.631978+0000 (oldest deadline 2024-04-24T23:33:37.331905+0000) 2024-04-24T23:34:16.447 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:16 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:16.188+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.80:6814 osd.3 since back 2024-04-24T23:33:22.632991+0000 front 2024-04-24T23:33:22.633039+0000 (oldest deadline 2024-04-24T23:33:44.932941+0000) 2024-04-24T23:34:16.448 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:16 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:16.188+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-04-24T23:33:30.134179+0000 front 2024-04-24T23:33:30.134163+0000 (oldest deadline 2024-04-24T23:33:52.434250+0000) 2024-04-24T23:34:16.448 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:16 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:16.188+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.148:6812 osd.5 since back 2024-04-24T23:33:39.336230+0000 front 2024-04-24T23:33:39.336245+0000 (oldest deadline 2024-04-24T23:34:04.036102+0000) 2024-04-24T23:34:16.448 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:16 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:16.188+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.80:6822 osd.6 since back 2024-04-24T23:33:47.536934+0000 front 2024-04-24T23:33:47.536873+0000 (oldest deadline 2024-04-24T23:34:12.737307+0000) 2024-04-24T23:34:16.657 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:16 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:16.277+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.848330+0000 front 2024-04-24T23:32:55.848387+0000 (oldest deadline 2024-04-24T23:33:20.548352+0000) 2024-04-24T23:34:16.657 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:16 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:16.277+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:02.150412+0000 front 2024-04-24T23:33:02.150294+0000 (oldest deadline 2024-04-24T23:33:28.050409+0000) 2024-04-24T23:34:16.657 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:16 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:16.277+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:14.952538+0000 front 2024-04-24T23:33:14.952635+0000 (oldest deadline 2024-04-24T23:33:40.852722+0000) 2024-04-24T23:34:16.658 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:16 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:16.277+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-04-24T23:33:26.054443+0000 front 2024-04-24T23:33:26.054377+0000 (oldest deadline 2024-04-24T23:33:54.255147+0000) 2024-04-24T23:34:16.658 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:16 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:16.277+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.148:6812 osd.5 since back 2024-04-24T23:33:36.556162+0000 front 2024-04-24T23:33:36.556180+0000 (oldest deadline 2024-04-24T23:34:00.656332+0000) 2024-04-24T23:34:16.658 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:16 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:16.277+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.80:6822 osd.6 since back 2024-04-24T23:33:47.558297+0000 front 2024-04-24T23:33:47.558371+0000 (oldest deadline 2024-04-24T23:34:11.658305+0000) 2024-04-24T23:34:17.449 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:17 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:17.188+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:56.929716+0000 front 2024-04-24T23:32:56.929774+0000 (oldest deadline 2024-04-24T23:33:20.429679+0000) 2024-04-24T23:34:17.450 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:17 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:17.188+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:04.530694+0000 front 2024-04-24T23:33:04.530743+0000 (oldest deadline 2024-04-24T23:33:29.230721+0000) 2024-04-24T23:34:17.450 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:17 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:17.188+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:15.632228+0000 front 2024-04-24T23:33:15.631978+0000 (oldest deadline 2024-04-24T23:33:37.331905+0000) 2024-04-24T23:34:17.450 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:17 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:17.188+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.80:6814 osd.3 since back 2024-04-24T23:33:22.632991+0000 front 2024-04-24T23:33:22.633039+0000 (oldest deadline 2024-04-24T23:33:44.932941+0000) 2024-04-24T23:34:17.450 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:17 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:17.188+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-04-24T23:33:30.134179+0000 front 2024-04-24T23:33:30.134163+0000 (oldest deadline 2024-04-24T23:33:52.434250+0000) 2024-04-24T23:34:17.450 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:17 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:17.188+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.148:6812 osd.5 since back 2024-04-24T23:33:39.336230+0000 front 2024-04-24T23:33:39.336245+0000 (oldest deadline 2024-04-24T23:34:04.036102+0000) 2024-04-24T23:34:17.450 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:17 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:17.188+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.80:6822 osd.6 since back 2024-04-24T23:33:47.536934+0000 front 2024-04-24T23:33:47.536873+0000 (oldest deadline 2024-04-24T23:34:12.737307+0000) 2024-04-24T23:34:17.756 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:17 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:17.324+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.848330+0000 front 2024-04-24T23:32:55.848387+0000 (oldest deadline 2024-04-24T23:33:20.548352+0000) 2024-04-24T23:34:17.756 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:17 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:17.324+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:02.150412+0000 front 2024-04-24T23:33:02.150294+0000 (oldest deadline 2024-04-24T23:33:28.050409+0000) 2024-04-24T23:34:17.756 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:17 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:17.324+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:14.952538+0000 front 2024-04-24T23:33:14.952635+0000 (oldest deadline 2024-04-24T23:33:40.852722+0000) 2024-04-24T23:34:17.756 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:17 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:17.324+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-04-24T23:33:26.054443+0000 front 2024-04-24T23:33:26.054377+0000 (oldest deadline 2024-04-24T23:33:54.255147+0000) 2024-04-24T23:34:17.756 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:17 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:17.324+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.148:6812 osd.5 since back 2024-04-24T23:33:36.556162+0000 front 2024-04-24T23:33:36.556180+0000 (oldest deadline 2024-04-24T23:34:00.656332+0000) 2024-04-24T23:34:17.757 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:17 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:17.324+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.80:6822 osd.6 since back 2024-04-24T23:33:47.558297+0000 front 2024-04-24T23:33:47.558371+0000 (oldest deadline 2024-04-24T23:34:11.658305+0000) 2024-04-24T23:34:18.696 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:18 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:18.213+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:56.929716+0000 front 2024-04-24T23:32:56.929774+0000 (oldest deadline 2024-04-24T23:33:20.429679+0000) 2024-04-24T23:34:18.697 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:18 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:18.213+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:04.530694+0000 front 2024-04-24T23:33:04.530743+0000 (oldest deadline 2024-04-24T23:33:29.230721+0000) 2024-04-24T23:34:18.697 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:18 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:18.213+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:15.632228+0000 front 2024-04-24T23:33:15.631978+0000 (oldest deadline 2024-04-24T23:33:37.331905+0000) 2024-04-24T23:34:18.697 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:18 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:18.213+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.80:6814 osd.3 since back 2024-04-24T23:33:22.632991+0000 front 2024-04-24T23:33:22.633039+0000 (oldest deadline 2024-04-24T23:33:44.932941+0000) 2024-04-24T23:34:18.697 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:18 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:18.213+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-04-24T23:33:30.134179+0000 front 2024-04-24T23:33:30.134163+0000 (oldest deadline 2024-04-24T23:33:52.434250+0000) 2024-04-24T23:34:18.697 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:18 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:18.213+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.148:6812 osd.5 since back 2024-04-24T23:33:39.336230+0000 front 2024-04-24T23:33:39.336245+0000 (oldest deadline 2024-04-24T23:34:04.036102+0000) 2024-04-24T23:34:18.698 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:18 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:18.213+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.80:6822 osd.6 since back 2024-04-24T23:33:47.536934+0000 front 2024-04-24T23:33:47.536873+0000 (oldest deadline 2024-04-24T23:34:12.737307+0000) 2024-04-24T23:34:18.755 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:18 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:18.288+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.848330+0000 front 2024-04-24T23:32:55.848387+0000 (oldest deadline 2024-04-24T23:33:20.548352+0000) 2024-04-24T23:34:18.756 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:18 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:18.288+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:02.150412+0000 front 2024-04-24T23:33:02.150294+0000 (oldest deadline 2024-04-24T23:33:28.050409+0000) 2024-04-24T23:34:18.756 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:18 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:18.288+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:14.952538+0000 front 2024-04-24T23:33:14.952635+0000 (oldest deadline 2024-04-24T23:33:40.852722+0000) 2024-04-24T23:34:18.756 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:18 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:18.288+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-04-24T23:33:26.054443+0000 front 2024-04-24T23:33:26.054377+0000 (oldest deadline 2024-04-24T23:33:54.255147+0000) 2024-04-24T23:34:18.756 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:18 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:18.288+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.148:6812 osd.5 since back 2024-04-24T23:33:36.556162+0000 front 2024-04-24T23:33:36.556180+0000 (oldest deadline 2024-04-24T23:34:00.656332+0000) 2024-04-24T23:34:18.756 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:18 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:18.288+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.80:6822 osd.6 since back 2024-04-24T23:33:47.558297+0000 front 2024-04-24T23:33:47.558371+0000 (oldest deadline 2024-04-24T23:34:11.658305+0000) 2024-04-24T23:34:19.697 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:19 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:19.210+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:56.929716+0000 front 2024-04-24T23:32:56.929774+0000 (oldest deadline 2024-04-24T23:33:20.429679+0000) 2024-04-24T23:34:19.698 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:19 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:19.210+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:04.530694+0000 front 2024-04-24T23:33:04.530743+0000 (oldest deadline 2024-04-24T23:33:29.230721+0000) 2024-04-24T23:34:19.698 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:19 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:19.210+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:15.632228+0000 front 2024-04-24T23:33:15.631978+0000 (oldest deadline 2024-04-24T23:33:37.331905+0000) 2024-04-24T23:34:19.698 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:19 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:19.210+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.80:6814 osd.3 since back 2024-04-24T23:33:22.632991+0000 front 2024-04-24T23:33:22.633039+0000 (oldest deadline 2024-04-24T23:33:44.932941+0000) 2024-04-24T23:34:19.698 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:19 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:19.210+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-04-24T23:33:30.134179+0000 front 2024-04-24T23:33:30.134163+0000 (oldest deadline 2024-04-24T23:33:52.434250+0000) 2024-04-24T23:34:19.698 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:19 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:19.210+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.148:6812 osd.5 since back 2024-04-24T23:33:39.336230+0000 front 2024-04-24T23:33:39.336245+0000 (oldest deadline 2024-04-24T23:34:04.036102+0000) 2024-04-24T23:34:19.698 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:19 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:19.210+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.80:6822 osd.6 since back 2024-04-24T23:33:47.536934+0000 front 2024-04-24T23:33:47.536873+0000 (oldest deadline 2024-04-24T23:34:12.737307+0000) 2024-04-24T23:34:19.756 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:19 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:19.309+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.848330+0000 front 2024-04-24T23:32:55.848387+0000 (oldest deadline 2024-04-24T23:33:20.548352+0000) 2024-04-24T23:34:19.756 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:19 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:19.309+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:02.150412+0000 front 2024-04-24T23:33:02.150294+0000 (oldest deadline 2024-04-24T23:33:28.050409+0000) 2024-04-24T23:34:19.756 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:19 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:19.309+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:14.952538+0000 front 2024-04-24T23:33:14.952635+0000 (oldest deadline 2024-04-24T23:33:40.852722+0000) 2024-04-24T23:34:19.756 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:19 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:19.309+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-04-24T23:33:26.054443+0000 front 2024-04-24T23:33:26.054377+0000 (oldest deadline 2024-04-24T23:33:54.255147+0000) 2024-04-24T23:34:19.756 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:19 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:19.309+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.148:6812 osd.5 since back 2024-04-24T23:33:36.556162+0000 front 2024-04-24T23:33:36.556180+0000 (oldest deadline 2024-04-24T23:34:00.656332+0000) 2024-04-24T23:34:19.756 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:19 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:19.309+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.80:6822 osd.6 since back 2024-04-24T23:33:47.558297+0000 front 2024-04-24T23:33:47.558371+0000 (oldest deadline 2024-04-24T23:34:11.658305+0000) 2024-04-24T23:34:19.757 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:19 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:19.309+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.134:6820 osd.7 since back 2024-04-24T23:33:55.759582+0000 front 2024-04-24T23:33:55.759622+0000 (oldest deadline 2024-04-24T23:34:19.259456+0000) 2024-04-24T23:34:20.447 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:20 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:20.188+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:56.929716+0000 front 2024-04-24T23:32:56.929774+0000 (oldest deadline 2024-04-24T23:33:20.429679+0000) 2024-04-24T23:34:20.447 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:20 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:20.188+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:04.530694+0000 front 2024-04-24T23:33:04.530743+0000 (oldest deadline 2024-04-24T23:33:29.230721+0000) 2024-04-24T23:34:20.447 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:20 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:20.188+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:15.632228+0000 front 2024-04-24T23:33:15.631978+0000 (oldest deadline 2024-04-24T23:33:37.331905+0000) 2024-04-24T23:34:20.447 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:20 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:20.188+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.80:6814 osd.3 since back 2024-04-24T23:33:22.632991+0000 front 2024-04-24T23:33:22.633039+0000 (oldest deadline 2024-04-24T23:33:44.932941+0000) 2024-04-24T23:34:20.447 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:20 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:20.188+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-04-24T23:33:30.134179+0000 front 2024-04-24T23:33:30.134163+0000 (oldest deadline 2024-04-24T23:33:52.434250+0000) 2024-04-24T23:34:20.447 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:20 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:20.188+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.148:6812 osd.5 since back 2024-04-24T23:33:39.336230+0000 front 2024-04-24T23:33:39.336245+0000 (oldest deadline 2024-04-24T23:34:04.036102+0000) 2024-04-24T23:34:20.447 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:20 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10[57752]: 2024-04-24T23:34:20.188+0000 7f029f2fe700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.80:6822 osd.6 since back 2024-04-24T23:33:47.536934+0000 front 2024-04-24T23:33:47.536873+0000 (oldest deadline 2024-04-24T23:34:12.737307+0000) 2024-04-24T23:34:20.756 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:20 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:20.347+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.848330+0000 front 2024-04-24T23:32:55.848387+0000 (oldest deadline 2024-04-24T23:33:20.548352+0000) 2024-04-24T23:34:20.756 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:20 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:20.347+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:02.150412+0000 front 2024-04-24T23:33:02.150294+0000 (oldest deadline 2024-04-24T23:33:28.050409+0000) 2024-04-24T23:34:20.756 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:20 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:20.347+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:14.952538+0000 front 2024-04-24T23:33:14.952635+0000 (oldest deadline 2024-04-24T23:33:40.852722+0000) 2024-04-24T23:34:20.756 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:20 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:20.347+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-04-24T23:33:26.054443+0000 front 2024-04-24T23:33:26.054377+0000 (oldest deadline 2024-04-24T23:33:54.255147+0000) 2024-04-24T23:34:20.756 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:20 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:20.347+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.148:6812 osd.5 since back 2024-04-24T23:33:36.556162+0000 front 2024-04-24T23:33:36.556180+0000 (oldest deadline 2024-04-24T23:34:00.656332+0000) 2024-04-24T23:34:20.756 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:20 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:20.347+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.80:6822 osd.6 since back 2024-04-24T23:33:47.558297+0000 front 2024-04-24T23:33:47.558371+0000 (oldest deadline 2024-04-24T23:34:11.658305+0000) 2024-04-24T23:34:20.756 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:20 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:20.347+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.134:6820 osd.7 since back 2024-04-24T23:33:55.759582+0000 front 2024-04-24T23:33:55.759622+0000 (oldest deadline 2024-04-24T23:34:19.259456+0000) 2024-04-24T23:34:21.337 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:21 smithi134 podman[67268]: 2024-04-24 23:34:21.019574059 +0000 UTC m=+5.779714782 container died 7049d8168b25e4eba9d80535ba00b371ea441a62968abed5c0886ad3f37e3a26 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-04-24T23:34:21.337 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:21 smithi134 podman[67268]: 2024-04-24 23:34:21.337608883 +0000 UTC m=+6.097749606 container cleanup 7049d8168b25e4eba9d80535ba00b371ea441a62968abed5c0886ad3f37e3a26 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-04-24T23:34:21.696 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:21 smithi134 bash[67268]: ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10 2024-04-24T23:34:21.697 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:21 smithi134 podman[67285]: 2024-04-24 23:34:21.580889056 +0000 UTC m=+0.554080450 container remove 7049d8168b25e4eba9d80535ba00b371ea441a62968abed5c0886ad3f37e3a26 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-24T23:34:21.756 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:21 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:21.324+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.848330+0000 front 2024-04-24T23:32:55.848387+0000 (oldest deadline 2024-04-24T23:33:20.548352+0000) 2024-04-24T23:34:21.756 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:21 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:21.324+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:02.150412+0000 front 2024-04-24T23:33:02.150294+0000 (oldest deadline 2024-04-24T23:33:28.050409+0000) 2024-04-24T23:34:21.756 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:21 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:21.324+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:14.952538+0000 front 2024-04-24T23:33:14.952635+0000 (oldest deadline 2024-04-24T23:33:40.852722+0000) 2024-04-24T23:34:21.756 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:21 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:21.324+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-04-24T23:33:26.054443+0000 front 2024-04-24T23:33:26.054377+0000 (oldest deadline 2024-04-24T23:33:54.255147+0000) 2024-04-24T23:34:21.756 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:21 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:21.324+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.148:6812 osd.5 since back 2024-04-24T23:33:36.556162+0000 front 2024-04-24T23:33:36.556180+0000 (oldest deadline 2024-04-24T23:34:00.656332+0000) 2024-04-24T23:34:21.756 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:21 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:21.324+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.80:6822 osd.6 since back 2024-04-24T23:33:47.558297+0000 front 2024-04-24T23:33:47.558371+0000 (oldest deadline 2024-04-24T23:34:11.658305+0000) 2024-04-24T23:34:21.756 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:21 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:21.324+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.134:6820 osd.7 since back 2024-04-24T23:33:55.759582+0000 front 2024-04-24T23:33:55.759622+0000 (oldest deadline 2024-04-24T23:34:19.259456+0000) 2024-04-24T23:34:22.300 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:22 smithi134 podman[67374]: 2024-04-24 23:34:21.914635958 +0000 UTC m=+0.017958520 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:34:22.300 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:22 smithi134 podman[67374]: 2024-04-24 23:34:22.015016573 +0000 UTC m=+0.118339118 container create 349a7f79dc0da22d968d1d2fb25b7add0c20fb0c35d00d4661879b80a02c02fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux ) 2024-04-24T23:34:22.300 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:22 smithi134 podman[67374]: 2024-04-24 23:34:22.234035517 +0000 UTC m=+0.337358062 container init 349a7f79dc0da22d968d1d2fb25b7add0c20fb0c35d00d4661879b80a02c02fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-04-24T23:34:22.300 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:22 smithi134 podman[67374]: 2024-04-24 23:34:22.236986751 +0000 UTC m=+0.340309297 container start 349a7f79dc0da22d968d1d2fb25b7add0c20fb0c35d00d4661879b80a02c02fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, GIT_CLEAN=True) 2024-04-24T23:34:22.301 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:22 smithi134 podman[67374]: 2024-04-24 23:34:22.300620554 +0000 UTC m=+0.403943114 container attach 349a7f79dc0da22d968d1d2fb25b7add0c20fb0c35d00d4661879b80a02c02fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10-deactivate, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux ) 2024-04-24T23:34:22.696 INFO:journalctl@ceph.osd.10.smithi134.stdout:Apr 24 23:34:22 smithi134 podman[67374]: 2024-04-24 23:34:22.534699489 +0000 UTC m=+0.638022026 container died 349a7f79dc0da22d968d1d2fb25b7add0c20fb0c35d00d4661879b80a02c02fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-10-deactivate, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T23:34:22.756 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:22 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:22.304+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.848330+0000 front 2024-04-24T23:32:55.848387+0000 (oldest deadline 2024-04-24T23:33:20.548352+0000) 2024-04-24T23:34:22.756 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:22 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:22.304+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:02.150412+0000 front 2024-04-24T23:33:02.150294+0000 (oldest deadline 2024-04-24T23:33:28.050409+0000) 2024-04-24T23:34:22.756 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:22 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:22.304+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:14.952538+0000 front 2024-04-24T23:33:14.952635+0000 (oldest deadline 2024-04-24T23:33:40.852722+0000) 2024-04-24T23:34:22.757 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:22 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:22.304+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-04-24T23:33:26.054443+0000 front 2024-04-24T23:33:26.054377+0000 (oldest deadline 2024-04-24T23:33:54.255147+0000) 2024-04-24T23:34:22.757 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:22 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:22.304+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.148:6812 osd.5 since back 2024-04-24T23:33:36.556162+0000 front 2024-04-24T23:33:36.556180+0000 (oldest deadline 2024-04-24T23:34:00.656332+0000) 2024-04-24T23:34:22.757 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:22 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:22.304+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.80:6822 osd.6 since back 2024-04-24T23:33:47.558297+0000 front 2024-04-24T23:33:47.558371+0000 (oldest deadline 2024-04-24T23:34:11.658305+0000) 2024-04-24T23:34:22.758 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:22 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:22.304+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.134:6820 osd.7 since back 2024-04-24T23:33:55.759582+0000 front 2024-04-24T23:33:55.759622+0000 (oldest deadline 2024-04-24T23:34:19.259456+0000) 2024-04-24T23:34:23.114 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ec7885aa-0290-11ef-bc93-c7b262605968@osd.10.service' 2024-04-24T23:34:23.151 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:34:23.152 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-04-24T23:34:23.152 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-04-24T23:34:23.152 DEBUG:teuthology.orchestra.run.smithi148:> sudo systemctl stop ceph-ec7885aa-0290-11ef-bc93-c7b262605968@osd.11 2024-04-24T23:34:23.483 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:23 smithi148 systemd[1]: Stopping Ceph osd.11 for ec7885aa-0290-11ef-bc93-c7b262605968... 2024-04-24T23:34:23.483 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:23 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:23.304+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.848330+0000 front 2024-04-24T23:32:55.848387+0000 (oldest deadline 2024-04-24T23:33:20.548352+0000) 2024-04-24T23:34:23.483 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:23 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:23.304+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:02.150412+0000 front 2024-04-24T23:33:02.150294+0000 (oldest deadline 2024-04-24T23:33:28.050409+0000) 2024-04-24T23:34:23.484 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:23 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:23.304+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:14.952538+0000 front 2024-04-24T23:33:14.952635+0000 (oldest deadline 2024-04-24T23:33:40.852722+0000) 2024-04-24T23:34:23.484 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:23 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:23.304+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-04-24T23:33:26.054443+0000 front 2024-04-24T23:33:26.054377+0000 (oldest deadline 2024-04-24T23:33:54.255147+0000) 2024-04-24T23:34:23.484 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:23 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:23.304+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.148:6812 osd.5 since back 2024-04-24T23:33:36.556162+0000 front 2024-04-24T23:33:36.556180+0000 (oldest deadline 2024-04-24T23:34:00.656332+0000) 2024-04-24T23:34:23.484 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:23 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:23.304+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.80:6822 osd.6 since back 2024-04-24T23:33:47.558297+0000 front 2024-04-24T23:33:47.558371+0000 (oldest deadline 2024-04-24T23:34:11.658305+0000) 2024-04-24T23:34:23.484 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:23 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:23.304+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.134:6820 osd.7 since back 2024-04-24T23:33:55.759582+0000 front 2024-04-24T23:33:55.759622+0000 (oldest deadline 2024-04-24T23:34:19.259456+0000) 2024-04-24T23:34:23.484 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:23 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:23.482+0000 7f3d967a2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:34:23.484 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:23 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:23.482+0000 7f3d967a2700 -1 osd.11 98 *** Got signal Terminated *** 2024-04-24T23:34:23.485 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:23 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:23.482+0000 7f3d967a2700 -1 osd.11 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T23:34:24.756 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:24 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:24.256+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.848330+0000 front 2024-04-24T23:32:55.848387+0000 (oldest deadline 2024-04-24T23:33:20.548352+0000) 2024-04-24T23:34:24.756 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:24 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:24.256+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:02.150412+0000 front 2024-04-24T23:33:02.150294+0000 (oldest deadline 2024-04-24T23:33:28.050409+0000) 2024-04-24T23:34:24.756 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:24 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:24.256+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:14.952538+0000 front 2024-04-24T23:33:14.952635+0000 (oldest deadline 2024-04-24T23:33:40.852722+0000) 2024-04-24T23:34:24.757 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:24 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:24.256+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-04-24T23:33:26.054443+0000 front 2024-04-24T23:33:26.054377+0000 (oldest deadline 2024-04-24T23:33:54.255147+0000) 2024-04-24T23:34:24.757 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:24 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:24.256+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.148:6812 osd.5 since back 2024-04-24T23:33:36.556162+0000 front 2024-04-24T23:33:36.556180+0000 (oldest deadline 2024-04-24T23:34:00.656332+0000) 2024-04-24T23:34:24.757 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:24 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:24.256+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.80:6822 osd.6 since back 2024-04-24T23:33:47.558297+0000 front 2024-04-24T23:33:47.558371+0000 (oldest deadline 2024-04-24T23:34:11.658305+0000) 2024-04-24T23:34:24.757 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:24 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:24.256+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.134:6820 osd.7 since back 2024-04-24T23:33:55.759582+0000 front 2024-04-24T23:33:55.759622+0000 (oldest deadline 2024-04-24T23:34:19.259456+0000) 2024-04-24T23:34:25.756 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:25 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:25.261+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.848330+0000 front 2024-04-24T23:32:55.848387+0000 (oldest deadline 2024-04-24T23:33:20.548352+0000) 2024-04-24T23:34:25.756 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:25 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:25.261+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:02.150412+0000 front 2024-04-24T23:33:02.150294+0000 (oldest deadline 2024-04-24T23:33:28.050409+0000) 2024-04-24T23:34:25.757 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:25 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:25.261+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:14.952538+0000 front 2024-04-24T23:33:14.952635+0000 (oldest deadline 2024-04-24T23:33:40.852722+0000) 2024-04-24T23:34:25.757 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:25 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:25.261+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-04-24T23:33:26.054443+0000 front 2024-04-24T23:33:26.054377+0000 (oldest deadline 2024-04-24T23:33:54.255147+0000) 2024-04-24T23:34:25.757 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:25 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:25.261+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.148:6812 osd.5 since back 2024-04-24T23:33:36.556162+0000 front 2024-04-24T23:33:36.556180+0000 (oldest deadline 2024-04-24T23:34:00.656332+0000) 2024-04-24T23:34:25.757 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:25 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:25.261+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.80:6822 osd.6 since back 2024-04-24T23:33:47.558297+0000 front 2024-04-24T23:33:47.558371+0000 (oldest deadline 2024-04-24T23:34:11.658305+0000) 2024-04-24T23:34:25.757 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:25 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:25.261+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.134:6820 osd.7 since back 2024-04-24T23:33:55.759582+0000 front 2024-04-24T23:33:55.759622+0000 (oldest deadline 2024-04-24T23:34:19.259456+0000) 2024-04-24T23:34:26.506 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:26 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:26.223+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.848330+0000 front 2024-04-24T23:32:55.848387+0000 (oldest deadline 2024-04-24T23:33:20.548352+0000) 2024-04-24T23:34:26.507 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:26 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:26.223+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:02.150412+0000 front 2024-04-24T23:33:02.150294+0000 (oldest deadline 2024-04-24T23:33:28.050409+0000) 2024-04-24T23:34:26.507 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:26 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:26.223+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:14.952538+0000 front 2024-04-24T23:33:14.952635+0000 (oldest deadline 2024-04-24T23:33:40.852722+0000) 2024-04-24T23:34:26.507 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:26 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:26.223+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-04-24T23:33:26.054443+0000 front 2024-04-24T23:33:26.054377+0000 (oldest deadline 2024-04-24T23:33:54.255147+0000) 2024-04-24T23:34:26.507 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:26 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:26.223+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.148:6812 osd.5 since back 2024-04-24T23:33:36.556162+0000 front 2024-04-24T23:33:36.556180+0000 (oldest deadline 2024-04-24T23:34:00.656332+0000) 2024-04-24T23:34:26.507 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:26 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:26.223+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.80:6822 osd.6 since back 2024-04-24T23:33:47.558297+0000 front 2024-04-24T23:33:47.558371+0000 (oldest deadline 2024-04-24T23:34:11.658305+0000) 2024-04-24T23:34:26.508 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:26 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:26.223+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.134:6820 osd.7 since back 2024-04-24T23:33:55.759582+0000 front 2024-04-24T23:33:55.759622+0000 (oldest deadline 2024-04-24T23:34:19.259456+0000) 2024-04-24T23:34:27.756 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:27 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:27.267+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.848330+0000 front 2024-04-24T23:32:55.848387+0000 (oldest deadline 2024-04-24T23:33:20.548352+0000) 2024-04-24T23:34:27.756 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:27 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:27.267+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:02.150412+0000 front 2024-04-24T23:33:02.150294+0000 (oldest deadline 2024-04-24T23:33:28.050409+0000) 2024-04-24T23:34:27.756 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:27 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:27.267+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:14.952538+0000 front 2024-04-24T23:33:14.952635+0000 (oldest deadline 2024-04-24T23:33:40.852722+0000) 2024-04-24T23:34:27.756 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:27 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:27.267+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-04-24T23:33:26.054443+0000 front 2024-04-24T23:33:26.054377+0000 (oldest deadline 2024-04-24T23:33:54.255147+0000) 2024-04-24T23:34:27.756 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:27 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:27.267+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.148:6812 osd.5 since back 2024-04-24T23:33:36.556162+0000 front 2024-04-24T23:33:36.556180+0000 (oldest deadline 2024-04-24T23:34:00.656332+0000) 2024-04-24T23:34:27.757 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:27 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:27.267+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.80:6822 osd.6 since back 2024-04-24T23:33:47.558297+0000 front 2024-04-24T23:33:47.558371+0000 (oldest deadline 2024-04-24T23:34:11.658305+0000) 2024-04-24T23:34:27.757 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:27 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:27.267+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.134:6820 osd.7 since back 2024-04-24T23:33:55.759582+0000 front 2024-04-24T23:33:55.759622+0000 (oldest deadline 2024-04-24T23:34:19.259456+0000) 2024-04-24T23:34:27.757 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:27 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:27.267+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.148:6820 osd.8 since back 2024-04-24T23:33:59.259740+0000 front 2024-04-24T23:33:59.259757+0000 (oldest deadline 2024-04-24T23:34:26.260283+0000) 2024-04-24T23:34:28.756 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:28 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:28.306+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.80:6806 osd.0 since back 2024-04-24T23:32:55.848330+0000 front 2024-04-24T23:32:55.848387+0000 (oldest deadline 2024-04-24T23:33:20.548352+0000) 2024-04-24T23:34:28.756 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:28 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:28.306+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-04-24T23:33:02.150412+0000 front 2024-04-24T23:33:02.150294+0000 (oldest deadline 2024-04-24T23:33:28.050409+0000) 2024-04-24T23:34:28.756 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:28 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:28.306+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-04-24T23:33:14.952538+0000 front 2024-04-24T23:33:14.952635+0000 (oldest deadline 2024-04-24T23:33:40.852722+0000) 2024-04-24T23:34:28.757 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:28 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:28.306+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-04-24T23:33:26.054443+0000 front 2024-04-24T23:33:26.054377+0000 (oldest deadline 2024-04-24T23:33:54.255147+0000) 2024-04-24T23:34:28.757 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:28 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:28.306+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.148:6812 osd.5 since back 2024-04-24T23:33:36.556162+0000 front 2024-04-24T23:33:36.556180+0000 (oldest deadline 2024-04-24T23:34:00.656332+0000) 2024-04-24T23:34:28.757 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:28 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:28.306+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.80:6822 osd.6 since back 2024-04-24T23:33:47.558297+0000 front 2024-04-24T23:33:47.558371+0000 (oldest deadline 2024-04-24T23:34:11.658305+0000) 2024-04-24T23:34:28.757 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:28 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:28.306+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.134:6820 osd.7 since back 2024-04-24T23:33:55.759582+0000 front 2024-04-24T23:33:55.759622+0000 (oldest deadline 2024-04-24T23:34:19.259456+0000) 2024-04-24T23:34:28.757 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:28 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11[58410]: 2024-04-24T23:34:28.306+0000 7f3d8ed8e700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.148:6820 osd.8 since back 2024-04-24T23:33:59.259740+0000 front 2024-04-24T23:33:59.259757+0000 (oldest deadline 2024-04-24T23:34:26.260283+0000) 2024-04-24T23:34:29.068 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:28 smithi148 podman[67592]: 2024-04-24 23:34:28.793502723 +0000 UTC m=+5.452216854 container died 8912709582e1d978c4a673ba18410dc31f78055363e83936d0e0fd9ff4723968 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-04-24T23:34:29.408 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:29 smithi148 podman[67592]: 2024-04-24 23:34:29.174180548 +0000 UTC m=+5.832894678 container cleanup 8912709582e1d978c4a673ba18410dc31f78055363e83936d0e0fd9ff4723968 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11, GIT_CLEAN=True, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-04-24T23:34:29.408 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:29 smithi148 bash[67592]: ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11 2024-04-24T23:34:29.718 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:29 smithi148 podman[67609]: 2024-04-24 23:34:29.408667628 +0000 UTC m=+0.611610548 container remove 8912709582e1d978c4a673ba18410dc31f78055363e83936d0e0fd9ff4723968 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T23:34:30.006 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:29 smithi148 podman[67700]: 2024-04-24 23:34:29.619521934 +0000 UTC m=+0.018096523 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:34:30.006 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:29 smithi148 podman[67700]: 2024-04-24 23:34:29.725980976 +0000 UTC m=+0.124555565 container create 39e0b1b16fd4d550ff2c2a355db57be8e8dbd5874b36d2855bd247251c091d24 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11-deactivate, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, RELEASE=squid-d2c45d9, GIT_CLEAN=True) 2024-04-24T23:34:30.006 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:29 smithi148 podman[67700]: 2024-04-24 23:34:29.960442062 +0000 UTC m=+0.359016651 container init 39e0b1b16fd4d550ff2c2a355db57be8e8dbd5874b36d2855bd247251c091d24 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11-deactivate, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-04-24T23:34:30.006 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:29 smithi148 podman[67700]: 2024-04-24 23:34:29.963622395 +0000 UTC m=+0.362196987 container start 39e0b1b16fd4d550ff2c2a355db57be8e8dbd5874b36d2855bd247251c091d24 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11-deactivate, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9) 2024-04-24T23:34:30.506 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:30 smithi148 podman[67700]: 2024-04-24 23:34:30.053436986 +0000 UTC m=+0.452011581 container attach 39e0b1b16fd4d550ff2c2a355db57be8e8dbd5874b36d2855bd247251c091d24 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-04-24T23:34:30.506 INFO:journalctl@ceph.osd.11.smithi148.stdout:Apr 24 23:34:30 smithi148 podman[67700]: 2024-04-24 23:34:30.255176172 +0000 UTC m=+0.653750754 container died 39e0b1b16fd4d550ff2c2a355db57be8e8dbd5874b36d2855bd247251c091d24 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-osd-11-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T23:34:30.834 DEBUG:teuthology.orchestra.run.smithi148:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ec7885aa-0290-11ef-bc93-c7b262605968@osd.11.service' 2024-04-24T23:34:30.871 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:34:30.872 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-04-24T23:34:30.872 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-04-24T23:34:30.872 DEBUG:teuthology.orchestra.run.smithi080:> sudo systemctl stop ceph-ec7885aa-0290-11ef-bc93-c7b262605968@mds.a 2024-04-24T23:34:31.194 INFO:journalctl@ceph.mds.a.smithi080.stdout:Apr 24 23:34:30 smithi080 systemd[1]: Stopping Ceph mds.a for ec7885aa-0290-11ef-bc93-c7b262605968... 2024-04-24T23:34:31.538 INFO:journalctl@ceph.mds.a.smithi080.stdout:Apr 24 23:34:31 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-a[70379]: 2024-04-24T23:34:31.193+0000 7fbc70408700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:34:31.538 INFO:journalctl@ceph.mds.a.smithi080.stdout:Apr 24 23:34:31 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-a[70379]: 2024-04-24T23:34:31.193+0000 7fbc70408700 -1 mds.a *** got signal Terminated *** 2024-04-24T23:34:34.620 INFO:journalctl@ceph.mds.a.smithi080.stdout:Apr 24 23:34:34 smithi080 podman[95245]: 2024-04-24 23:34:34.356718551 +0000 UTC m=+3.286027885 container died 0e7f74f1b192bdd92c318ee9dab63d52d291c133a313e64233ac99c6ea9a503f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240424) 2024-04-24T23:34:34.987 INFO:journalctl@ceph.mds.a.smithi080.stdout:Apr 24 23:34:34 smithi080 podman[95245]: 2024-04-24 23:34:34.777989948 +0000 UTC m=+3.707299286 container cleanup 0e7f74f1b192bdd92c318ee9dab63d52d291c133a313e64233ac99c6ea9a503f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-a, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, ceph=True) 2024-04-24T23:34:34.988 INFO:journalctl@ceph.mds.a.smithi080.stdout:Apr 24 23:34:34 smithi080 bash[95245]: ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-a 2024-04-24T23:34:35.205 DEBUG:teuthology.orchestra.run.smithi080:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ec7885aa-0290-11ef-bc93-c7b262605968@mds.a.service' 2024-04-24T23:34:35.240 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:34:35.240 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-04-24T23:34:35.241 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-04-24T23:34:35.241 DEBUG:teuthology.orchestra.run.smithi080:> sudo systemctl stop ceph-ec7885aa-0290-11ef-bc93-c7b262605968@mds.d 2024-04-24T23:34:35.537 INFO:journalctl@ceph.mds.d.smithi080.stdout:Apr 24 23:34:35 smithi080 systemd[1]: Stopping Ceph mds.d for ec7885aa-0290-11ef-bc93-c7b262605968... 2024-04-24T23:34:36.037 INFO:journalctl@ceph.mds.d.smithi080.stdout:Apr 24 23:34:35 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-d[69417]: 2024-04-24T23:34:35.556+0000 7fc8ae666700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:34:36.038 INFO:journalctl@ceph.mds.d.smithi080.stdout:Apr 24 23:34:35 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-d[69417]: 2024-04-24T23:34:35.556+0000 7fc8ae666700 -1 mds.d *** got signal Terminated *** 2024-04-24T23:34:38.538 INFO:journalctl@ceph.mds.d.smithi080.stdout:Apr 24 23:34:38 smithi080 podman[95381]: 2024-04-24 23:34:38.28317511 +0000 UTC m=+2.827843269 container died bc0ce64524d4e912a17ee98fde0712324f610f6b53b487c3ea7d8c7500a0c465 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-d, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-04-24T23:34:39.008 INFO:journalctl@ceph.mds.d.smithi080.stdout:Apr 24 23:34:38 smithi080 podman[95381]: 2024-04-24 23:34:38.815821467 +0000 UTC m=+3.360489687 container cleanup bc0ce64524d4e912a17ee98fde0712324f610f6b53b487c3ea7d8c7500a0c465 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-d, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux ) 2024-04-24T23:34:39.008 INFO:journalctl@ceph.mds.d.smithi080.stdout:Apr 24 23:34:38 smithi080 bash[95381]: ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-d 2024-04-24T23:34:39.250 DEBUG:teuthology.orchestra.run.smithi080:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ec7885aa-0290-11ef-bc93-c7b262605968@mds.d.service' 2024-04-24T23:34:39.270 INFO:journalctl@ceph.mds.d.smithi080.stdout:Apr 24 23:34:39 smithi080 podman[95395]: 2024-04-24 23:34:39.008894054 +0000 UTC m=+0.722796136 container remove bc0ce64524d4e912a17ee98fde0712324f610f6b53b487c3ea7d8c7500a0c465 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-d, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-04-24T23:34:39.270 INFO:journalctl@ceph.mds.d.smithi080.stdout:Apr 24 23:34:39 smithi080 systemd[1]: ceph-ec7885aa-0290-11ef-bc93-c7b262605968@mds.d.service: Deactivated successfully. 2024-04-24T23:34:39.270 INFO:journalctl@ceph.mds.d.smithi080.stdout:Apr 24 23:34:39 smithi080 systemd[1]: Stopped Ceph mds.d for ec7885aa-0290-11ef-bc93-c7b262605968. 2024-04-24T23:34:39.281 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:34:39.282 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-04-24T23:34:39.282 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-04-24T23:34:39.282 DEBUG:teuthology.orchestra.run.smithi080:> sudo systemctl stop ceph-ec7885aa-0290-11ef-bc93-c7b262605968@mds.g 2024-04-24T23:34:39.537 INFO:journalctl@ceph.mds.g.smithi080.stdout:Apr 24 23:34:39 smithi080 systemd[1]: Stopping Ceph mds.g for ec7885aa-0290-11ef-bc93-c7b262605968... 2024-04-24T23:34:40.038 INFO:journalctl@ceph.mds.g.smithi080.stdout:Apr 24 23:34:39 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-g[70030]: 2024-04-24T23:34:39.620+0000 7fb1a1295700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:34:40.038 INFO:journalctl@ceph.mds.g.smithi080.stdout:Apr 24 23:34:39 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-g[70030]: 2024-04-24T23:34:39.620+0000 7fb1a1295700 -1 mds.g *** got signal Terminated *** 2024-04-24T23:34:43.787 INFO:journalctl@ceph.mds.g.smithi080.stdout:Apr 24 23:34:43 smithi080 podman[95502]: 2024-04-24 23:34:43.564143196 +0000 UTC m=+4.064983303 container died 5472b1bfb0cbf719a3895efa88f3d2d52747a7d439c7f2ffd1b88ed563e42f67 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-g, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T23:34:44.221 INFO:journalctl@ceph.mds.g.smithi080.stdout:Apr 24 23:34:44 smithi080 podman[95502]: 2024-04-24 23:34:44.028291519 +0000 UTC m=+4.529131673 container cleanup 5472b1bfb0cbf719a3895efa88f3d2d52747a7d439c7f2ffd1b88ed563e42f67 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-g, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T23:34:44.221 INFO:journalctl@ceph.mds.g.smithi080.stdout:Apr 24 23:34:44 smithi080 bash[95502]: ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-g 2024-04-24T23:34:44.449 DEBUG:teuthology.orchestra.run.smithi080:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ec7885aa-0290-11ef-bc93-c7b262605968@mds.g.service' 2024-04-24T23:34:44.479 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:34:44.480 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-04-24T23:34:44.480 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-04-24T23:34:44.480 DEBUG:teuthology.orchestra.run.smithi080:> sudo systemctl stop ceph-ec7885aa-0290-11ef-bc93-c7b262605968@mds.j 2024-04-24T23:34:44.788 INFO:journalctl@ceph.mds.j.smithi080.stdout:Apr 24 23:34:44 smithi080 systemd[1]: Stopping Ceph mds.j for ec7885aa-0290-11ef-bc93-c7b262605968... 2024-04-24T23:34:45.287 INFO:journalctl@ceph.mds.j.smithi080.stdout:Apr 24 23:34:44 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-j[69030]: 2024-04-24T23:34:44.799+0000 7fe03142c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:34:45.288 INFO:journalctl@ceph.mds.j.smithi080.stdout:Apr 24 23:34:44 smithi080 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-j[69030]: 2024-04-24T23:34:44.799+0000 7fe03142c700 -1 mds.j *** got signal Terminated *** 2024-04-24T23:34:47.782 INFO:journalctl@ceph.mds.j.smithi080.stdout:Apr 24 23:34:47 smithi080 podman[95626]: 2024-04-24 23:34:47.473709271 +0000 UTC m=+2.777759108 container died 9bb75b8333b673a5689732ca07f816a933ca12c3a4ceb02eb77e2f665ed4725d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-j, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-04-24T23:34:48.037 INFO:journalctl@ceph.mds.j.smithi080.stdout:Apr 24 23:34:47 smithi080 podman[95626]: 2024-04-24 23:34:47.865504335 +0000 UTC m=+3.169554175 container cleanup 9bb75b8333b673a5689732ca07f816a933ca12c3a4ceb02eb77e2f665ed4725d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-j, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-24T23:34:48.038 INFO:journalctl@ceph.mds.j.smithi080.stdout:Apr 24 23:34:47 smithi080 bash[95626]: ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-j 2024-04-24T23:34:48.243 DEBUG:teuthology.orchestra.run.smithi080:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ec7885aa-0290-11ef-bc93-c7b262605968@mds.j.service' 2024-04-24T23:34:48.274 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:34:48.274 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-04-24T23:34:48.274 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-04-24T23:34:48.274 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-ec7885aa-0290-11ef-bc93-c7b262605968@mds.b 2024-04-24T23:34:48.580 INFO:journalctl@ceph.mds.b.smithi134.stdout:Apr 24 23:34:48 smithi134 systemd[1]: Stopping Ceph mds.b for ec7885aa-0290-11ef-bc93-c7b262605968... 2024-04-24T23:34:48.946 INFO:journalctl@ceph.mds.b.smithi134.stdout:Apr 24 23:34:48 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-b[63405]: 2024-04-24T23:34:48.579+0000 7f9e99105700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:34:48.947 INFO:journalctl@ceph.mds.b.smithi134.stdout:Apr 24 23:34:48 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-b[63405]: 2024-04-24T23:34:48.579+0000 7f9e99105700 -1 mds.b *** got signal Terminated *** 2024-04-24T23:34:52.920 INFO:journalctl@ceph.mds.b.smithi134.stdout:Apr 24 23:34:52 smithi134 podman[67491]: 2024-04-24 23:34:52.591034788 +0000 UTC m=+4.126774190 container died 0e52ed6fdb94e1ebc4d6c475e166a124b9ac698b2b862fa91998711c1957cda7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-b, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T23:34:53.114 INFO:journalctl@ceph.mds.b.smithi134.stdout:Apr 24 23:34:52 smithi134 podman[67491]: 2024-04-24 23:34:52.920873033 +0000 UTC m=+4.456612436 container cleanup 0e52ed6fdb94e1ebc4d6c475e166a124b9ac698b2b862fa91998711c1957cda7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-04-24T23:34:53.114 INFO:journalctl@ceph.mds.b.smithi134.stdout:Apr 24 23:34:52 smithi134 bash[67491]: ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-b 2024-04-24T23:34:53.115 INFO:journalctl@ceph.mds.b.smithi134.stdout:Apr 24 23:34:53 smithi134 podman[67505]: 2024-04-24 23:34:53.113665266 +0000 UTC m=+0.518207925 container remove 0e52ed6fdb94e1ebc4d6c475e166a124b9ac698b2b862fa91998711c1957cda7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-b, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-24T23:34:53.350 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ec7885aa-0290-11ef-bc93-c7b262605968@mds.b.service' 2024-04-24T23:34:53.384 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:34:53.385 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-04-24T23:34:53.385 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-04-24T23:34:53.385 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-ec7885aa-0290-11ef-bc93-c7b262605968@mds.e 2024-04-24T23:34:53.696 INFO:journalctl@ceph.mds.e.smithi134.stdout:Apr 24 23:34:53 smithi134 systemd[1]: Stopping Ceph mds.e for ec7885aa-0290-11ef-bc93-c7b262605968... 2024-04-24T23:34:54.196 INFO:journalctl@ceph.mds.e.smithi134.stdout:Apr 24 23:34:53 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-e[62785]: 2024-04-24T23:34:53.698+0000 7fb150b8a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:34:54.196 INFO:journalctl@ceph.mds.e.smithi134.stdout:Apr 24 23:34:53 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-e[62785]: 2024-04-24T23:34:53.698+0000 7fb150b8a700 -1 mds.e *** got signal Terminated *** 2024-04-24T23:34:57.834 INFO:journalctl@ceph.mds.e.smithi134.stdout:Apr 24 23:34:57 smithi134 podman[67611]: 2024-04-24 23:34:57.480863766 +0000 UTC m=+3.876814763 container died 366560a918ca496bbf848611a0f5b9abc47ad79b25d39a2f4bba7db063753e2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-e, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-24T23:34:58.196 INFO:journalctl@ceph.mds.e.smithi134.stdout:Apr 24 23:34:58 smithi134 podman[67611]: 2024-04-24 23:34:58.016270552 +0000 UTC m=+4.412221564 container cleanup 366560a918ca496bbf848611a0f5b9abc47ad79b25d39a2f4bba7db063753e2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-e, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-24T23:34:58.197 INFO:journalctl@ceph.mds.e.smithi134.stdout:Apr 24 23:34:58 smithi134 bash[67611]: ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-e 2024-04-24T23:34:58.449 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ec7885aa-0290-11ef-bc93-c7b262605968@mds.e.service' 2024-04-24T23:34:58.471 INFO:journalctl@ceph.mds.e.smithi134.stdout:Apr 24 23:34:58 smithi134 podman[67637]: 2024-04-24 23:34:58.20900864 +0000 UTC m=+0.724584069 container remove 366560a918ca496bbf848611a0f5b9abc47ad79b25d39a2f4bba7db063753e2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-e, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-04-24T23:34:58.471 INFO:journalctl@ceph.mds.e.smithi134.stdout:Apr 24 23:34:58 smithi134 systemd[1]: ceph-ec7885aa-0290-11ef-bc93-c7b262605968@mds.e.service: Deactivated successfully. 2024-04-24T23:34:58.471 INFO:journalctl@ceph.mds.e.smithi134.stdout:Apr 24 23:34:58 smithi134 systemd[1]: Stopped Ceph mds.e for ec7885aa-0290-11ef-bc93-c7b262605968. 2024-04-24T23:34:58.471 INFO:journalctl@ceph.mds.e.smithi134.stdout:Apr 24 23:34:58 smithi134 systemd[1]: ceph-ec7885aa-0290-11ef-bc93-c7b262605968@mds.e.service: Consumed 2.048s CPU time. 2024-04-24T23:34:58.483 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:34:58.483 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-04-24T23:34:58.483 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-04-24T23:34:58.483 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-ec7885aa-0290-11ef-bc93-c7b262605968@mds.h 2024-04-24T23:34:58.828 INFO:journalctl@ceph.mds.h.smithi134.stdout:Apr 24 23:34:58 smithi134 systemd[1]: Stopping Ceph mds.h for ec7885aa-0290-11ef-bc93-c7b262605968... 2024-04-24T23:34:59.196 INFO:journalctl@ceph.mds.h.smithi134.stdout:Apr 24 23:34:58 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-h[63097]: 2024-04-24T23:34:58.828+0000 7fcdc66f0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:34:59.197 INFO:journalctl@ceph.mds.h.smithi134.stdout:Apr 24 23:34:58 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-h[63097]: 2024-04-24T23:34:58.828+0000 7fcdc66f0700 -1 mds.h *** got signal Terminated *** 2024-04-24T23:35:01.753 INFO:journalctl@ceph.mds.h.smithi134.stdout:Apr 24 23:35:01 smithi134 podman[67742]: 2024-04-24 23:35:01.492021968 +0000 UTC m=+2.794579684 container died d8e5842d437bad8b290cb0569bde221c5226d5c7e83b263ee2813388e3a36316 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-h, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-04-24T23:35:02.120 INFO:journalctl@ceph.mds.h.smithi134.stdout:Apr 24 23:35:01 smithi134 podman[67742]: 2024-04-24 23:35:01.853151046 +0000 UTC m=+3.155708779 container cleanup d8e5842d437bad8b290cb0569bde221c5226d5c7e83b263ee2813388e3a36316 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-h, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, ceph=True, GIT_CLEAN=True) 2024-04-24T23:35:02.120 INFO:journalctl@ceph.mds.h.smithi134.stdout:Apr 24 23:35:01 smithi134 bash[67742]: ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-h 2024-04-24T23:35:02.282 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ec7885aa-0290-11ef-bc93-c7b262605968@mds.h.service' 2024-04-24T23:35:02.315 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:35:02.315 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-04-24T23:35:02.315 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-04-24T23:35:02.315 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-ec7885aa-0290-11ef-bc93-c7b262605968@mds.k 2024-04-24T23:35:02.647 INFO:journalctl@ceph.mds.k.smithi134.stdout:Apr 24 23:35:02 smithi134 systemd[1]: Stopping Ceph mds.k for ec7885aa-0290-11ef-bc93-c7b262605968... 2024-04-24T23:35:02.946 INFO:journalctl@ceph.mds.k.smithi134.stdout:Apr 24 23:35:02 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-k[62463]: 2024-04-24T23:35:02.648+0000 7f657967b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:35:02.947 INFO:journalctl@ceph.mds.k.smithi134.stdout:Apr 24 23:35:02 smithi134 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-k[62463]: 2024-04-24T23:35:02.648+0000 7f657967b700 -1 mds.k *** got signal Terminated *** 2024-04-24T23:35:04.946 INFO:journalctl@ceph.mds.k.smithi134.stdout:Apr 24 23:35:04 smithi134 podman[67862]: 2024-04-24 23:35:04.696147678 +0000 UTC m=+2.159330160 container died def966a79765fee2677b3704200bdb30c8850c6cbdeca97a4f7b692d6609b915 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-k, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-04-24T23:35:05.440 INFO:journalctl@ceph.mds.k.smithi134.stdout:Apr 24 23:35:05 smithi134 podman[67862]: 2024-04-24 23:35:05.148033536 +0000 UTC m=+2.611216007 container cleanup def966a79765fee2677b3704200bdb30c8850c6cbdeca97a4f7b692d6609b915 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-k, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, GIT_CLEAN=True, org.label-schema.build-date=20240424, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-04-24T23:35:05.440 INFO:journalctl@ceph.mds.k.smithi134.stdout:Apr 24 23:35:05 smithi134 bash[67862]: ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-k 2024-04-24T23:35:05.569 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ec7885aa-0290-11ef-bc93-c7b262605968@mds.k.service' 2024-04-24T23:35:05.600 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:35:05.600 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-04-24T23:35:05.600 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-04-24T23:35:05.600 DEBUG:teuthology.orchestra.run.smithi148:> sudo systemctl stop ceph-ec7885aa-0290-11ef-bc93-c7b262605968@mds.c 2024-04-24T23:35:05.939 INFO:journalctl@ceph.mds.c.smithi148.stdout:Apr 24 23:35:05 smithi148 systemd[1]: Stopping Ceph mds.c for ec7885aa-0290-11ef-bc93-c7b262605968... 2024-04-24T23:35:06.205 INFO:journalctl@ceph.mds.c.smithi148.stdout:Apr 24 23:35:05 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-c[63396]: 2024-04-24T23:35:05.939+0000 7f2c40240700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:35:06.206 INFO:journalctl@ceph.mds.c.smithi148.stdout:Apr 24 23:35:05 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-c[63396]: 2024-04-24T23:35:05.939+0000 7f2c40240700 -1 mds.c *** got signal Terminated *** 2024-04-24T23:35:09.755 INFO:journalctl@ceph.mds.c.smithi148.stdout:Apr 24 23:35:09 smithi148 podman[67806]: 2024-04-24 23:35:09.483127875 +0000 UTC m=+3.686646875 container died 85fdccac975f816f9193eff752196fbe07ddf2176ba81b212dafcdcff2062b7d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-c, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-04-24T23:35:10.106 INFO:journalctl@ceph.mds.c.smithi148.stdout:Apr 24 23:35:09 smithi148 podman[67806]: 2024-04-24 23:35:09.847625063 +0000 UTC m=+4.051144052 container cleanup 85fdccac975f816f9193eff752196fbe07ddf2176ba81b212dafcdcff2062b7d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-c, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-04-24T23:35:10.107 INFO:journalctl@ceph.mds.c.smithi148.stdout:Apr 24 23:35:09 smithi148 bash[67806]: ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-c 2024-04-24T23:35:10.233 DEBUG:teuthology.orchestra.run.smithi148:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ec7885aa-0290-11ef-bc93-c7b262605968@mds.c.service' 2024-04-24T23:35:10.268 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:35:10.268 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-04-24T23:35:10.268 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-04-24T23:35:10.269 DEBUG:teuthology.orchestra.run.smithi148:> sudo systemctl stop ceph-ec7885aa-0290-11ef-bc93-c7b262605968@mds.f 2024-04-24T23:35:10.505 INFO:journalctl@ceph.mds.f.smithi148.stdout:Apr 24 23:35:10 smithi148 systemd[1]: Stopping Ceph mds.f for ec7885aa-0290-11ef-bc93-c7b262605968... 2024-04-24T23:35:11.006 INFO:journalctl@ceph.mds.f.smithi148.stdout:Apr 24 23:35:10 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-f[63087]: 2024-04-24T23:35:10.626+0000 7f6ad815b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:35:11.006 INFO:journalctl@ceph.mds.f.smithi148.stdout:Apr 24 23:35:10 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-f[63087]: 2024-04-24T23:35:10.626+0000 7f6ad815b700 -1 mds.f *** got signal Terminated *** 2024-04-24T23:35:13.753 INFO:journalctl@ceph.mds.f.smithi148.stdout:Apr 24 23:35:13 smithi148 podman[67938]: 2024-04-24 23:35:13.484314611 +0000 UTC m=+3.000652717 container died 9f0a8e178d1a527dbaf09fc8d5a3c430f989596aa16b17c0e7ba06f158555eaa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-f, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0) 2024-04-24T23:35:14.005 INFO:journalctl@ceph.mds.f.smithi148.stdout:Apr 24 23:35:13 smithi148 podman[67938]: 2024-04-24 23:35:13.85164116 +0000 UTC m=+3.367979263 container cleanup 9f0a8e178d1a527dbaf09fc8d5a3c430f989596aa16b17c0e7ba06f158555eaa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-f, org.label-schema.build-date=20240424, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-04-24T23:35:14.006 INFO:journalctl@ceph.mds.f.smithi148.stdout:Apr 24 23:35:13 smithi148 bash[67938]: ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-f 2024-04-24T23:35:14.231 DEBUG:teuthology.orchestra.run.smithi148:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ec7885aa-0290-11ef-bc93-c7b262605968@mds.f.service' 2024-04-24T23:35:14.263 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:35:14.263 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-04-24T23:35:14.263 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-04-24T23:35:14.264 DEBUG:teuthology.orchestra.run.smithi148:> sudo systemctl stop ceph-ec7885aa-0290-11ef-bc93-c7b262605968@mds.i 2024-04-24T23:35:14.506 INFO:journalctl@ceph.mds.i.smithi148.stdout:Apr 24 23:35:14 smithi148 systemd[1]: Stopping Ceph mds.i for ec7885aa-0290-11ef-bc93-c7b262605968... 2024-04-24T23:35:15.005 INFO:journalctl@ceph.mds.i.smithi148.stdout:Apr 24 23:35:14 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-i[63704]: 2024-04-24T23:35:14.604+0000 7f1ed011b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:35:15.006 INFO:journalctl@ceph.mds.i.smithi148.stdout:Apr 24 23:35:14 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-i[63704]: 2024-04-24T23:35:14.604+0000 7f1ed011b700 -1 mds.i *** got signal Terminated *** 2024-04-24T23:35:17.747 INFO:journalctl@ceph.mds.i.smithi148.stdout:Apr 24 23:35:17 smithi148 podman[68057]: 2024-04-24 23:35:17.480756575 +0000 UTC m=+3.004488135 container died b42a579c659da5e4e47221006d17636bc576710c114efe4561f342011b020f33 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-i, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, GIT_CLEAN=True) 2024-04-24T23:35:18.005 INFO:journalctl@ceph.mds.i.smithi148.stdout:Apr 24 23:35:17 smithi148 podman[68057]: 2024-04-24 23:35:17.84687995 +0000 UTC m=+3.370611501 container cleanup b42a579c659da5e4e47221006d17636bc576710c114efe4561f342011b020f33 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-i, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-04-24T23:35:18.006 INFO:journalctl@ceph.mds.i.smithi148.stdout:Apr 24 23:35:17 smithi148 bash[68057]: ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-i 2024-04-24T23:35:18.284 DEBUG:teuthology.orchestra.run.smithi148:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ec7885aa-0290-11ef-bc93-c7b262605968@mds.i.service' 2024-04-24T23:35:18.315 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:35:18.315 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-04-24T23:35:18.316 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-04-24T23:35:18.316 DEBUG:teuthology.orchestra.run.smithi148:> sudo systemctl stop ceph-ec7885aa-0290-11ef-bc93-c7b262605968@mds.l 2024-04-24T23:35:18.625 INFO:journalctl@ceph.mds.l.smithi148.stdout:Apr 24 23:35:18 smithi148 systemd[1]: Stopping Ceph mds.l for ec7885aa-0290-11ef-bc93-c7b262605968... 2024-04-24T23:35:19.005 INFO:journalctl@ceph.mds.l.smithi148.stdout:Apr 24 23:35:18 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-l[62779]: 2024-04-24T23:35:18.624+0000 7f8d5990a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:35:19.006 INFO:journalctl@ceph.mds.l.smithi148.stdout:Apr 24 23:35:18 smithi148 ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-l[62779]: 2024-04-24T23:35:18.624+0000 7f8d5990a700 -1 mds.l *** got signal Terminated *** 2024-04-24T23:35:22.100 INFO:journalctl@ceph.mds.l.smithi148.stdout:Apr 24 23:35:21 smithi148 podman[68181]: 2024-04-24 23:35:21.78047416 +0000 UTC m=+3.245692446 container died 8b2dcc3e6dc8b84363cd18d1443305f1a00700f0361d77019c08d73d1e8eb4f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-l, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, ceph=True) 2024-04-24T23:35:22.400 INFO:journalctl@ceph.mds.l.smithi148.stdout:Apr 24 23:35:22 smithi148 podman[68181]: 2024-04-24 23:35:22.217063102 +0000 UTC m=+3.682281385 container cleanup 8b2dcc3e6dc8b84363cd18d1443305f1a00700f0361d77019c08d73d1e8eb4f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-l, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T23:35:22.401 INFO:journalctl@ceph.mds.l.smithi148.stdout:Apr 24 23:35:22 smithi148 bash[68181]: ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-l 2024-04-24T23:35:22.642 DEBUG:teuthology.orchestra.run.smithi148:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ec7885aa-0290-11ef-bc93-c7b262605968@mds.l.service' 2024-04-24T23:35:22.662 INFO:journalctl@ceph.mds.l.smithi148.stdout:Apr 24 23:35:22 smithi148 podman[68195]: 2024-04-24 23:35:22.401313131 +0000 UTC m=+0.620911995 container remove 8b2dcc3e6dc8b84363cd18d1443305f1a00700f0361d77019c08d73d1e8eb4f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ec7885aa-0290-11ef-bc93-c7b262605968-mds-l, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T23:35:22.662 INFO:journalctl@ceph.mds.l.smithi148.stdout:Apr 24 23:35:22 smithi148 systemd[1]: ceph-ec7885aa-0290-11ef-bc93-c7b262605968@mds.l.service: Deactivated successfully. 2024-04-24T23:35:22.662 INFO:journalctl@ceph.mds.l.smithi148.stdout:Apr 24 23:35:22 smithi148 systemd[1]: Stopped Ceph mds.l for ec7885aa-0290-11ef-bc93-c7b262605968. 2024-04-24T23:35:22.672 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:35:22.672 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-04-24T23:35:22.672 DEBUG:teuthology.orchestra.run.smithi080:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid ec7885aa-0290-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-24T23:35:22.858 INFO:teuthology.orchestra.run.smithi080.stdout:Deleting cluster with fsid: ec7885aa-0290-11ef-bc93-c7b262605968 2024-04-24T23:35:25.290 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid ec7885aa-0290-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-24T23:35:25.480 INFO:teuthology.orchestra.run.smithi134.stdout:Deleting cluster with fsid: ec7885aa-0290-11ef-bc93-c7b262605968 2024-04-24T23:35:27.901 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid ec7885aa-0290-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-24T23:35:28.092 INFO:teuthology.orchestra.run.smithi148.stdout:Deleting cluster with fsid: ec7885aa-0290-11ef-bc93-c7b262605968 2024-04-24T23:35:30.403 DEBUG:teuthology.orchestra.run.smithi080:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-24T23:35:30.436 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-24T23:35:30.468 DEBUG:teuthology.orchestra.run.smithi148:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-24T23:35:30.494 INFO:tasks.cephadm:Archiving crash dumps... 2024-04-24T23:35:30.496 DEBUG:teuthology.misc:Transferring archived files from smithi080:/var/lib/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672189/remote/smithi080/crash 2024-04-24T23:35:30.497 DEBUG:teuthology.orchestra.run.smithi080:> sudo tar c -f - -C /var/lib/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/crash -- . 2024-04-24T23:35:30.535 INFO:teuthology.orchestra.run.smithi080.stderr:tar: /var/lib/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-24T23:35:30.535 INFO:teuthology.orchestra.run.smithi080.stderr:tar: Error is not recoverable: exiting now 2024-04-24T23:35:30.537 DEBUG:teuthology.misc:Transferring archived files from smithi134:/var/lib/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672189/remote/smithi134/crash 2024-04-24T23:35:30.538 DEBUG:teuthology.orchestra.run.smithi134:> sudo tar c -f - -C /var/lib/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/crash -- . 2024-04-24T23:35:30.581 INFO:teuthology.orchestra.run.smithi134.stderr:tar: /var/lib/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-24T23:35:30.582 INFO:teuthology.orchestra.run.smithi134.stderr:tar: Error is not recoverable: exiting now 2024-04-24T23:35:30.583 DEBUG:teuthology.misc:Transferring archived files from smithi148:/var/lib/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672189/remote/smithi148/crash 2024-04-24T23:35:30.584 DEBUG:teuthology.orchestra.run.smithi148:> sudo tar c -f - -C /var/lib/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/crash -- . 2024-04-24T23:35:30.631 INFO:teuthology.orchestra.run.smithi148.stderr:tar: /var/lib/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-24T23:35:30.631 INFO:teuthology.orchestra.run.smithi148.stderr:tar: Error is not recoverable: exiting now 2024-04-24T23:35:30.632 INFO:tasks.cephadm:Checking cluster log for badness... 2024-04-24T23:35:30.632 DEBUG:teuthology.orchestra.run.smithi080:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-04-24T23:35:30.671 INFO:tasks.cephadm:Compressing logs... 2024-04-24T23:35:30.672 DEBUG:teuthology.orchestra.run.smithi080:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-24T23:35:30.714 DEBUG:teuthology.orchestra.run.smithi134:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-24T23:35:30.716 DEBUG:teuthology.orchestra.run.smithi148:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-24T23:35:30.737 INFO:teuthology.orchestra.run.smithi080.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-24T23:35:30.738 INFO:teuthology.orchestra.run.smithi080.stderr:: No such file or directory 2024-04-24T23:35:30.738 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph.log 2024-04-24T23:35:30.738 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-24T23:35:30.740 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph-mds.g.log 2024-04-24T23:35:30.741 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph-mds.j.log 2024-04-24T23:35:30.741 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose --/var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph-mds.g.log: /var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph-osd.3.log 2024-04-24T23:35:30.741 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph-osd.0.log 2024-04-24T23:35:30.741 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph-mgr.x.log 2024-04-24T23:35:30.741 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph-osd.0.log: 86.2% -- replaced with /var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-24T23:35:30.741 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph-mon.a.log 2024-04-24T23:35:30.757 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph-mgr.x.log: 89.9% -- replaced with /var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-24T23:35:30.757 INFO:teuthology.orchestra.run.smithi080.stderr: 94.7% -- replaced with /var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph-mds.g.log.gz 2024-04-24T23:35:30.758 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph-osd.6.log 2024-04-24T23:35:30.758 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph-mon.a.log: gzip 94.7% -- replaced with /var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph-mds.j.log.gz 2024-04-24T23:35:30.758 INFO:teuthology.orchestra.run.smithi080.stderr: -5 --verbose -- /var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-24T23:35:30.758 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph-mds.a.log 2024-04-24T23:35:30.758 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-24T23:35:30.758 INFO:teuthology.orchestra.run.smithi080.stderr: 82.9% -- replaced with /var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-24T23:35:30.758 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph-mds.d.log 2024-04-24T23:35:30.759 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph-volume.log: 93.3% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-24T23:35:30.759 INFO:teuthology.orchestra.run.smithi080.stderr: 94.7% -- replaced with /var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph-mds.a.log.gz 2024-04-24T23:35:30.759 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph-osd.9.log 2024-04-24T23:35:30.759 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-04-24T23:35:30.759 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph-osd.9.log: 94.5% -- replaced with /var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph-mds.d.log.gz 2024-04-24T23:35:30.761 INFO:teuthology.orchestra.run.smithi080.stderr: 91.1% -- replaced with /var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-24T23:35:30.762 INFO:teuthology.orchestra.run.smithi134.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-24T23:35:30.762 INFO:teuthology.orchestra.run.smithi134.stderr:: No such file or directory 2024-04-24T23:35:30.762 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph.log 2024-04-24T23:35:30.762 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-24T23:35:30.762 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph-osd.7.log 2024-04-24T23:35:30.762 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph-mds.b.log 2024-04-24T23:35:30.762 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph-osd.4.log 2024-04-24T23:35:30.763 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph-osd.7.log: /var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph-mds.b.log: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-24T23:35:30.763 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph-osd.1.log 2024-04-24T23:35:30.763 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph-mgr.y.log 2024-04-24T23:35:30.763 INFO:teuthology.orchestra.run.smithi134.stderr: 86.1% -- replaced with /var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-24T23:35:30.763 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph-mds.h.log 2024-04-24T23:35:30.763 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph-osd.10.log 2024-04-24T23:35:30.763 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-24T23:35:30.763 INFO:teuthology.orchestra.run.smithi134.stderr: 90.1% -- replaced with /var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-24T23:35:30.764 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph-mds.e.log 2024-04-24T23:35:30.764 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph.cephadm.log: 82.8% -- replaced with /var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-24T23:35:30.764 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-24T23:35:30.764 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph-mds.e.log: 92.3% -- replaced with /var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph-mgr.y.log.gz 2024-04-24T23:35:30.764 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph-mon.b.log 2024-04-24T23:35:30.764 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph-mds.k.log 2024-04-24T23:35:30.764 INFO:teuthology.orchestra.run.smithi148.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-24T23:35:30.765 INFO:teuthology.orchestra.run.smithi148.stderr:: No such file or directory 2024-04-24T23:35:30.765 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph.log 2024-04-24T23:35:30.765 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph-osd.2.log 2024-04-24T23:35:30.765 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/cephadm.log: /var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph-mds.f.log 2024-04-24T23:35:30.765 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-24T23:35:30.765 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph-osd.5.log 2024-04-24T23:35:30.765 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph-mds.f.log: /var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph-mds.c.log 2024-04-24T23:35:30.766 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- 92.5% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-24T23:35:30.766 INFO:teuthology.orchestra.run.smithi148.stderr: /var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph-mon.c.log 2024-04-24T23:35:30.766 INFO:teuthology.orchestra.run.smithi148.stderr: 86.1%/var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph-mds.c.log: -- replaced with /var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-24T23:35:30.766 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph-mgr.z.log 2024-04-24T23:35:30.766 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph-mds.i.log 2024-04-24T23:35:30.766 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-24T23:35:30.766 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph-mds.i.log: 90.1% -- replaced with /var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-24T23:35:30.766 INFO:teuthology.orchestra.run.smithi148.stderr: 92.4% -- replaced with /var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph-mgr.z.log.gz 2024-04-24T23:35:30.767 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph-osd.11.log 2024-04-24T23:35:30.767 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph-osd.8.log 82.7% -- replaced with /var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-24T23:35:30.767 INFO:teuthology.orchestra.run.smithi148.stderr: 2024-04-24T23:35:30.767 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-24T23:35:30.768 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.log: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-04-24T23:35:30.769 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph-mon.b.log: 91.1% -- replaced with /var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-24T23:35:30.769 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph-mds.k.log: 94.7% -- replaced with /var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph-mds.k.log.gz 2024-04-24T23:35:30.770 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph-osd.8.log: /var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph-mds.l.log 2024-04-24T23:35:30.785 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph-volume.log: /var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph-mds.l.log: 95.0% -- replaced with /var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph-mds.l.log.gz 2024-04-24T23:35:30.786 INFO:teuthology.orchestra.run.smithi148.stderr: 91.0% -- replaced with /var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-24T23:35:30.946 INFO:teuthology.orchestra.run.smithi080.stderr: 90.0% -- replaced with /var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph-mgr.x.log.gz 2024-04-24T23:35:31.006 INFO:teuthology.orchestra.run.smithi148.stderr: 91.9% -- replaced with /var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph-mon.c.log.gz 2024-04-24T23:35:31.020 INFO:teuthology.orchestra.run.smithi134.stderr: 91.9% -- replaced with /var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph-mon.b.log.gz 2024-04-24T23:35:31.030 INFO:teuthology.orchestra.run.smithi134.stderr: 94.7% -- replaced with /var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph-mds.e.log.gz 2024-04-24T23:35:31.035 INFO:teuthology.orchestra.run.smithi134.stderr: 95.0% -- replaced with /var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph-mds.h.log.gz 2024-04-24T23:35:31.547 INFO:teuthology.orchestra.run.smithi080.stderr: 90.6% -- replaced with /var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph-mon.a.log.gz 2024-04-24T23:35:31.742 INFO:teuthology.orchestra.run.smithi148.stderr: 95.2% -- replaced with /var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph-mds.f.log.gz 2024-04-24T23:35:32.392 INFO:teuthology.orchestra.run.smithi080.stderr: 91.9% -- replaced with /var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph-osd.0.log.gz 2024-04-24T23:35:32.488 INFO:teuthology.orchestra.run.smithi134.stderr: 92.4% -- replaced with /var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph-osd.10.log.gz 2024-04-24T23:35:32.612 INFO:teuthology.orchestra.run.smithi134.stderr: 91.9% -- replaced with /var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph-osd.7.log.gz 2024-04-24T23:35:32.751 INFO:teuthology.orchestra.run.smithi148.stderr: 92.2% -- replaced with /var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph-osd.8.log.gz 2024-04-24T23:35:32.756 INFO:teuthology.orchestra.run.smithi134.stderr: 92.5% -- replaced with /var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph-osd.1.log.gz 2024-04-24T23:35:33.009 INFO:teuthology.orchestra.run.smithi134.stderr: 94.3% -- replaced with /var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph-mds.b.log.gz 2024-04-24T23:35:33.011 INFO:teuthology.orchestra.run.smithi080.stderr: 92.3% -- replaced with /var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph-osd.3.log.gz 2024-04-24T23:35:33.044 INFO:teuthology.orchestra.run.smithi148.stderr: 92.6% -- replaced with /var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph-osd.11.log.gz 2024-04-24T23:35:33.111 INFO:teuthology.orchestra.run.smithi134.stderr: 91.6% -- replaced with /var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph-osd.4.log.gz 2024-04-24T23:35:33.113 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-04-24T23:35:33.113 INFO:teuthology.orchestra.run.smithi134.stderr:real 0m2.384s 2024-04-24T23:35:33.113 INFO:teuthology.orchestra.run.smithi134.stderr:user 0m10.762s 2024-04-24T23:35:33.113 INFO:teuthology.orchestra.run.smithi134.stderr:sys 0m0.373s 2024-04-24T23:35:33.197 INFO:teuthology.orchestra.run.smithi080.stderr: 92.3% -- replaced with /var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph-osd.6.log.gz 2024-04-24T23:35:33.217 INFO:teuthology.orchestra.run.smithi148.stderr: 93.8% -- replaced with /var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph-mds.c.log.gz 2024-04-24T23:35:33.314 INFO:teuthology.orchestra.run.smithi080.stderr: 92.3% -- replaced with /var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph-osd.9.log.gz 2024-04-24T23:35:33.316 INFO:teuthology.orchestra.run.smithi080.stderr: 2024-04-24T23:35:33.316 INFO:teuthology.orchestra.run.smithi080.stderr:real 0m2.589s 2024-04-24T23:35:33.317 INFO:teuthology.orchestra.run.smithi080.stderr:user 0m9.666s 2024-04-24T23:35:33.317 INFO:teuthology.orchestra.run.smithi080.stderr:sys 0m0.347s 2024-04-24T23:35:33.325 INFO:teuthology.orchestra.run.smithi148.stderr: 91.9% -- replaced with /var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph-osd.5.log.gz 2024-04-24T23:35:33.876 INFO:teuthology.orchestra.run.smithi148.stderr: 92.2% -- replaced with /var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph-osd.2.log.gz 2024-04-24T23:35:43.313 INFO:teuthology.orchestra.run.smithi148.stderr: 93.5% -- replaced with /var/log/ceph/ec7885aa-0290-11ef-bc93-c7b262605968/ceph-mds.i.log.gz 2024-04-24T23:35:43.315 INFO:teuthology.orchestra.run.smithi148.stderr: 2024-04-24T23:35:43.315 INFO:teuthology.orchestra.run.smithi148.stderr:real 0m12.585s 2024-04-24T23:35:43.315 INFO:teuthology.orchestra.run.smithi148.stderr:user 0m25.531s 2024-04-24T23:35:43.315 INFO:teuthology.orchestra.run.smithi148.stderr:sys 0m0.785s 2024-04-24T23:35:43.316 INFO:tasks.cephadm:Archiving logs... 2024-04-24T23:35:43.316 DEBUG:teuthology.misc:Transferring archived files from smithi080:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672189/remote/smithi080/log 2024-04-24T23:35:43.317 DEBUG:teuthology.orchestra.run.smithi080:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-24T23:35:44.902 DEBUG:teuthology.misc:Transferring archived files from smithi134:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672189/remote/smithi134/log 2024-04-24T23:35:44.903 DEBUG:teuthology.orchestra.run.smithi134:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-24T23:35:46.427 DEBUG:teuthology.misc:Transferring archived files from smithi148:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672189/remote/smithi148/log 2024-04-24T23:35:46.428 DEBUG:teuthology.orchestra.run.smithi148:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-24T23:35:49.894 INFO:tasks.cephadm:Removing cluster... 2024-04-24T23:35:49.894 DEBUG:teuthology.orchestra.run.smithi080:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid ec7885aa-0290-11ef-bc93-c7b262605968 --force 2024-04-24T23:35:50.082 INFO:teuthology.orchestra.run.smithi080.stdout:Deleting cluster with fsid: ec7885aa-0290-11ef-bc93-c7b262605968 2024-04-24T23:35:50.411 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid ec7885aa-0290-11ef-bc93-c7b262605968 --force 2024-04-24T23:35:50.600 INFO:teuthology.orchestra.run.smithi134.stdout:Deleting cluster with fsid: ec7885aa-0290-11ef-bc93-c7b262605968 2024-04-24T23:35:50.864 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid ec7885aa-0290-11ef-bc93-c7b262605968 --force 2024-04-24T23:35:51.055 INFO:teuthology.orchestra.run.smithi148.stdout:Deleting cluster with fsid: ec7885aa-0290-11ef-bc93-c7b262605968 2024-04-24T23:35:51.332 INFO:tasks.cephadm:Removing cephadm ... 2024-04-24T23:35:51.332 DEBUG:teuthology.orchestra.run.smithi080:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-24T23:35:51.350 DEBUG:teuthology.orchestra.run.smithi134:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-24T23:35:51.367 DEBUG:teuthology.orchestra.run.smithi148:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-24T23:35:51.382 INFO:tasks.cephadm:Teardown complete 2024-04-24T23:35:51.383 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-24T23:35:51.393 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-24T23:35:51.394 DEBUG:teuthology.orchestra.run.smithi080:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-24T23:35:51.396 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-24T23:35:51.409 DEBUG:teuthology.orchestra.run.smithi148:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-24T23:35:51.521 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-24T23:35:51.532 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-24T23:35:51.532 DEBUG:teuthology.orchestra.run.smithi080:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-24T23:35:51.535 DEBUG:teuthology.orchestra.run.smithi134:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-24T23:35:51.537 DEBUG:teuthology.orchestra.run.smithi148:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-24T23:35:51.548 INFO:teuthology.orchestra.run.smithi080.stderr:bash: line 1: ntpq: command not found 2024-04-24T23:35:51.549 INFO:teuthology.orchestra.run.smithi134.stderr:bash: line 1: ntpq: command not found 2024-04-24T23:35:51.551 INFO:teuthology.orchestra.run.smithi148.stderr:bash: line 1: ntpq: command not found 2024-04-24T23:35:51.553 INFO:teuthology.orchestra.run.smithi080.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-24T23:35:51.553 INFO:teuthology.orchestra.run.smithi080.stdout:=============================================================================== 2024-04-24T23:35:51.553 INFO:teuthology.orchestra.run.smithi080.stdout:^- hv01.front.sepia.ceph.com 2 6 377 55 +983us[ +981us] +/- 573ms 2024-04-24T23:35:51.553 INFO:teuthology.orchestra.run.smithi080.stdout:^* hv02.front.sepia.ceph.com 3 6 377 53 +134us[ +132us] +/- 75ms 2024-04-24T23:35:51.553 INFO:teuthology.orchestra.run.smithi080.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 55 -149us[ -151us] +/- 82ms 2024-04-24T23:35:51.553 INFO:teuthology.orchestra.run.smithi080.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T23:35:51.555 INFO:teuthology.orchestra.run.smithi134.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-24T23:35:51.555 INFO:teuthology.orchestra.run.smithi134.stdout:=============================================================================== 2024-04-24T23:35:51.555 INFO:teuthology.orchestra.run.smithi134.stdout:^- hv01.front.sepia.ceph.com 2 6 377 55 +1008us[+1012us] +/- 573ms 2024-04-24T23:35:51.555 INFO:teuthology.orchestra.run.smithi134.stdout:^* hv02.front.sepia.ceph.com 3 6 377 53 +133us[ +137us] +/- 75ms 2024-04-24T23:35:51.555 INFO:teuthology.orchestra.run.smithi134.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 51 -146us[ -146us] +/- 83ms 2024-04-24T23:35:51.556 INFO:teuthology.orchestra.run.smithi134.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T23:35:51.557 INFO:teuthology.orchestra.run.smithi148.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-24T23:35:51.557 INFO:teuthology.orchestra.run.smithi148.stdout:=============================================================================== 2024-04-24T23:35:51.557 INFO:teuthology.orchestra.run.smithi148.stdout:^- hv01.front.sepia.ceph.com 2 6 377 53 +1054us[+1049us] +/- 573ms 2024-04-24T23:35:51.557 INFO:teuthology.orchestra.run.smithi148.stdout:^* hv02.front.sepia.ceph.com 3 6 377 53 +127us[ +121us] +/- 75ms 2024-04-24T23:35:51.557 INFO:teuthology.orchestra.run.smithi148.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 55 -157us[ -162us] +/- 82ms 2024-04-24T23:35:51.557 INFO:teuthology.orchestra.run.smithi148.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T23:35:51.558 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-24T23:35:51.568 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-24T23:35:51.569 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-24T23:35:51.579 DEBUG:teuthology.orchestra.run.smithi080:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-24T23:35:51.672 DEBUG:teuthology.orchestra.run.smithi134:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-24T23:35:51.736 DEBUG:teuthology.orchestra.run.smithi148:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-24T23:35:51.798 DEBUG:teuthology.orchestra.run.smithi080:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-24T23:35:51.826 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T23:35:51.827 DEBUG:teuthology.orchestra.run.smithi134:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-24T23:35:51.854 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T23:35:51.855 DEBUG:teuthology.orchestra.run.smithi148:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-24T23:35:51.881 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T23:35:51.881 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-24T23:35:51.893 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-24T23:35:51.953 INFO:teuthology.task.internal:Duration was 1399.623567 seconds 2024-04-24T23:35:51.954 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-24T23:35:52.089 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-24T23:35:52.089 DEBUG:teuthology.orchestra.run.smithi080:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-24T23:35:52.092 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-24T23:35:52.094 DEBUG:teuthology.orchestra.run.smithi148:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-24T23:35:52.131 INFO:teuthology.orchestra.run.smithi134.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-24T23:35:52.131 INFO:teuthology.orchestra.run.smithi148.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-24T23:35:52.133 INFO:teuthology.orchestra.run.smithi080.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-24T23:35:52.545 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-24T23:35:52.546 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi080.front.sepia.ceph.com 2024-04-24T23:35:52.546 DEBUG:teuthology.orchestra.run.smithi080:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-24T23:35:52.565 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi134.front.sepia.ceph.com 2024-04-24T23:35:52.565 DEBUG:teuthology.orchestra.run.smithi134:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-24T23:35:52.586 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi148.front.sepia.ceph.com 2024-04-24T23:35:52.586 DEBUG:teuthology.orchestra.run.smithi148:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-24T23:35:52.615 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-24T23:35:52.615 DEBUG:teuthology.orchestra.run.smithi080:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-24T23:35:52.617 DEBUG:teuthology.orchestra.run.smithi134:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-24T23:35:52.629 DEBUG:teuthology.orchestra.run.smithi148:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-24T23:35:52.679 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-24T23:35:52.680 DEBUG:teuthology.orchestra.run.smithi080:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-24T23:35:52.682 DEBUG:teuthology.orchestra.run.smithi134:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-24T23:35:52.696 DEBUG:teuthology.orchestra.run.smithi148:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-24T23:35:53.063 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-24T23:35:53.075 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-24T23:35:53.075 DEBUG:teuthology.orchestra.run.smithi080:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-24T23:35:53.098 DEBUG:teuthology.orchestra.run.smithi134:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-24T23:35:53.122 DEBUG:teuthology.orchestra.run.smithi148:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-24T23:35:53.147 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-24T23:35:53.158 DEBUG:teuthology.orchestra.run.smithi080:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-24T23:35:53.160 DEBUG:teuthology.orchestra.run.smithi134:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-24T23:35:53.165 DEBUG:teuthology.orchestra.run.smithi148:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-24T23:35:53.180 INFO:teuthology.orchestra.run.smithi080.stdout:kernel.core_pattern = core 2024-04-24T23:35:53.184 INFO:teuthology.orchestra.run.smithi134.stdout:kernel.core_pattern = core 2024-04-24T23:35:53.208 INFO:teuthology.orchestra.run.smithi148.stdout:kernel.core_pattern = core 2024-04-24T23:35:53.227 DEBUG:teuthology.orchestra.run.smithi080:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-24T23:35:53.245 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T23:35:53.245 DEBUG:teuthology.orchestra.run.smithi134:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-24T23:35:53.259 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T23:35:53.260 DEBUG:teuthology.orchestra.run.smithi148:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-24T23:35:53.281 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T23:35:53.282 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-24T23:35:53.293 INFO:teuthology.task.internal:Transferring archived files... 2024-04-24T23:35:53.294 DEBUG:teuthology.misc:Transferring archived files from smithi080:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672189/remote/smithi080 2024-04-24T23:35:53.294 DEBUG:teuthology.orchestra.run.smithi080:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-24T23:35:53.335 DEBUG:teuthology.misc:Transferring archived files from smithi134:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672189/remote/smithi134 2024-04-24T23:35:53.335 DEBUG:teuthology.orchestra.run.smithi134:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-24T23:35:53.369 DEBUG:teuthology.misc:Transferring archived files from smithi148:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672189/remote/smithi148 2024-04-24T23:35:53.370 DEBUG:teuthology.orchestra.run.smithi148:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-24T23:35:53.404 INFO:teuthology.task.internal:Removing archive directory... 2024-04-24T23:35:53.404 DEBUG:teuthology.orchestra.run.smithi080:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-24T23:35:53.407 DEBUG:teuthology.orchestra.run.smithi134:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-24T23:35:53.409 DEBUG:teuthology.orchestra.run.smithi148:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-24T23:35:53.453 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-24T23:35:53.467 INFO:teuthology.task.internal:Not uploading archives. 2024-04-24T23:35:53.468 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-24T23:35:53.478 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-24T23:35:53.478 DEBUG:teuthology.orchestra.run.smithi080:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-24T23:35:53.481 DEBUG:teuthology.orchestra.run.smithi134:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-24T23:35:53.483 DEBUG:teuthology.orchestra.run.smithi148:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-24T23:35:53.493 INFO:teuthology.orchestra.run.smithi080.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 24 23:35 /home/ubuntu/cephtest 2024-04-24T23:35:53.495 INFO:teuthology.orchestra.run.smithi134.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 24 23:35 /home/ubuntu/cephtest 2024-04-24T23:35:53.508 INFO:teuthology.orchestra.run.smithi148.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 24 23:35 /home/ubuntu/cephtest 2024-04-24T23:35:53.509 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-24T23:35:53.521 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-24T23:35:53.547 INFO:teuthology.nuke:Checking targets against current locks 2024-04-24T23:35:53.572 DEBUG:teuthology.nuke:shortname: smithi080 2024-04-24T23:35:53.573 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-24T23:35:53.597 DEBUG:teuthology.nuke:shortname: smithi134 2024-04-24T23:35:53.597 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-24T23:35:53.603 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi080.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672189', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-24 23:07:00.395184', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMfkpGuLodaIMGxyO3ew6OGCQMhmGce6YeQgbc/L0GoJALnJd+ieQDvMBf1nP//vUIkWuhFkJTH9nim4SZwAsrA='} 2024-04-24T23:35:53.618 DEBUG:teuthology.nuke:shortname: smithi148 2024-04-24T23:35:53.619 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-24T23:35:53.625 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi134.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672189', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-24 23:07:00.391732', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJLfJ5mhzX16mpPe9YV6p4eHLv7USo0CBUIrFqOF1DlNHnlFM66BDtOrHNeJbtSJ7H06Zt4NupSbjNvrGsCCzRM='} 2024-04-24T23:35:53.648 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi148.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672189', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-24 23:07:00.393666', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMZ+s72X4mo4KKu4DWptXk2E4sLii0Ouj8PIOMAf2YPxQssgQIYWOSPJx0vZoey7l7mg0a9DFv5El/rIb4fdkvg='} 2024-04-24T23:35:53.658 INFO:teuthology.orchestra.console.smithi080:Power off 2024-04-24T23:35:53.659 DEBUG:teuthology.orchestra.console.smithi080:pexpect command: ipmitool -H smithi080.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-24T23:35:53.681 DEBUG:teuthology.orchestra.console.smithi080:power off output: Chassis Power Control: Down/Off 2024-04-24T23:35:53.681 DEBUG:teuthology.orchestra.console.smithi080:pexpect command: ipmitool -H smithi080.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-24T23:35:53.703 INFO:teuthology.orchestra.console.smithi134:Power off 2024-04-24T23:35:53.704 DEBUG:teuthology.orchestra.console.smithi134:pexpect command: ipmitool -H smithi134.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-24T23:35:53.714 DEBUG:teuthology.orchestra.console.smithi080:check power output: Chassis Power is on 2024-04-24T23:35:53.728 DEBUG:teuthology.orchestra.console.smithi134:power off output: Chassis Power Control: Down/Off 2024-04-24T23:35:53.728 DEBUG:teuthology.orchestra.console.smithi134:pexpect command: ipmitool -H smithi134.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-24T23:35:53.744 INFO:teuthology.orchestra.console.smithi148:Power off 2024-04-24T23:35:53.745 DEBUG:teuthology.orchestra.console.smithi148:pexpect command: ipmitool -H smithi148.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-24T23:35:53.754 DEBUG:teuthology.orchestra.console.smithi134:check power output: Chassis Power is on 2024-04-24T23:35:53.767 DEBUG:teuthology.orchestra.console.smithi148:power off output: Chassis Power Control: Down/Off 2024-04-24T23:35:53.767 DEBUG:teuthology.orchestra.console.smithi148:pexpect command: ipmitool -H smithi148.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-24T23:35:53.793 DEBUG:teuthology.orchestra.console.smithi148:check power output: Chassis Power is on 2024-04-24T23:35:57.715 DEBUG:teuthology.orchestra.console.smithi080:pexpect command: ipmitool -H smithi080.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-24T23:35:57.755 DEBUG:teuthology.orchestra.console.smithi134:pexpect command: ipmitool -H smithi134.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-24T23:35:57.795 DEBUG:teuthology.orchestra.console.smithi148:pexpect command: ipmitool -H smithi148.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-24T23:35:57.826 DEBUG:teuthology.orchestra.console.smithi080:check power output: Chassis Power is on 2024-04-24T23:35:57.866 DEBUG:teuthology.orchestra.console.smithi134:check power output: Chassis Power is on 2024-04-24T23:35:57.909 DEBUG:teuthology.orchestra.console.smithi148:check power output: Chassis Power is on 2024-04-24T23:36:01.830 DEBUG:teuthology.orchestra.console.smithi080:pexpect command: ipmitool -H smithi080.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-24T23:36:01.866 DEBUG:teuthology.orchestra.console.smithi134:pexpect command: ipmitool -H smithi134.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-24T23:36:01.911 DEBUG:teuthology.orchestra.console.smithi148:pexpect command: ipmitool -H smithi148.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-24T23:36:01.941 DEBUG:teuthology.orchestra.console.smithi080:check power output: Chassis Power is off 2024-04-24T23:36:01.977 DEBUG:teuthology.orchestra.console.smithi134:check power output: Chassis Power is off 2024-04-24T23:36:02.021 DEBUG:teuthology.orchestra.console.smithi148:check power output: Chassis Power is off 2024-04-24T23:36:02.043 INFO:teuthology.orchestra.console.smithi080:Power off completed 2024-04-24T23:36:02.078 INFO:teuthology.orchestra.console.smithi134:Power off completed 2024-04-24T23:36:02.122 INFO:teuthology.orchestra.console.smithi148:Power off completed 2024-04-24T23:36:02.229 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-workunit/suites/fsx}} duration: 1399.6235671043396 failure_reason: 'Command failed (workunit test suites/fsx.sh) on smithi080 with status 2: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=d2c45d977142e024a59b68a8f1dce8f0b9840551 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsx.sh''' flavor: default owner: scheduled_teuthology@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=0d5992db68f441259d405d210fc7c8f7 status: fail success: false 2024-04-24T23:36:02.229 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-24T23:36:02.334 INFO:teuthology.run:FAIL